-1

Problem Overview:

Create an application that can identify the objects in a single photo, and as well as identify the correct arrangement of said objects.

I'm relatively new to image processing and image recognition concepts and I've tried looking for solutions or studies but couldn't find anything related to object arrangements (maybe or most probably my google-fu is just weak).

Overall, I would just like to know whether this is achievable or not and maybe someone can help me with regards to what terms to search for specifically in order to help me look for the solution to my problem.

What I'm trying to achieve is the ff:

  1. Take a picture (contains different objects in a certain order)

Sample picture - correct order

  1. Analyze picture and determine the objects in it.

Sample picture - analyzed objects

  1. Rearrange objects.

  2. Take photo again of the rearranged objects and identify which objects are re-arranged or not in order based on the first picture (See Step 1: Sample picture - correct order) . Ideally, results should return "Correct/Incorrect" and at the same time, the names of the objects that aren't in the right order.

Sample picture - Rearranged and Identified

I would greatly appreciate any feedback.

Gioza
  • 11
  • 3

3 Answers3

1

If the objects are pictured with a camera, the problem is more difficult than what you show on the diagram (these hand-drawn objects are very easy to locate).

If you can ensure that there is no rotation nor scaling between the two pictures, you can use template matching.

In a first step (training), frame all the objects with a rectangular ROI. This must be done manually.

In a second step (matching), use template matching to find the new locations of the objects. If all goes well, it suffices to compare the coordinates (with a tolerance) to know if the objects have been modes.

  • Thanks @Yves. Most likely there will be rotation and scaling between the two pictures. However, thank you for introducing me to "template matching" and "Rectangular ROI". I'll have a look and see if it is applicable. – Gioza Jul 16 '18 at 08:01
  • @Gioza: you will need more powerful template matching techniques then, such as with interest points. –  Jul 16 '18 at 08:58
1

You would have to do alot of R&D to make this happen because image processing is a tough task but fortunately Google provides an API for this kind of task known as TensorFlow

please follow this Link:

https://github.com/EdjeElectronics/TensorFlow-Object-Detection-API-Tutorial-Train-Multiple-Objects-Windows-10

This is basically an object detection API. You will get to know the right way to start, Go and dive into image processing :)

You can also refer to OpenCvLibrary.

Mohan Radhakrishnan
  • 3,002
  • 5
  • 28
  • 42
Ahsan bhatti
  • 7
  • 5
  • you can also browse this link :https://medium.com/mindorks/detection-on-android-using-tensorflow-a3f6fe423349 this is complete tutorial for TensorFlow beginners – Ahsan bhatti Jul 16 '18 at 06:51
  • Thank you for your feedback @Ahsan bhatti. I'm currently on that path right now. So far I've tried simple image recognition using TensorFlow Classifier (referred to a simple TF implementation using this link: [link](https://proandroiddev.com/using-tensorflow-in-android-step-by-step-code-explanation-fee36c281f32). I got the gist on how to implement the recognition part however, still trying to look for solutions regarding the arrangement. Anyway, thank you for the link! It is very informative and much more in-depth. – Gioza Jul 16 '18 at 07:45
0

Use any Object Detection Methods like SSD model, Which gives the location as well as the classification of the objects. By determining the location, we can see if any object is rearranged.

Ajai
  • 1,049
  • 1
  • 11
  • 23