-1

So, i'm looking for the best way to go about this, I want to take in a 2d image of something like a room with random object spread around it, and be able to detect and separate each individual object from one another.

*None of the objects are know to the program, this is not object recognition, and there is no recognition code involved. the program would not know what any of these objects are or how to identify them, only that they are not the same objects (even if there 2 of the same objects near each other)

*Another thing is that I am trying to distinguish EVERY components from one another even if its a sub part of another object; ie: not trying to detect that a whole lamp is separate from a desk, but also the light bulb, lamp pole, and the electrical cords are separate from the lamp. id like to detect as many separate objects as possible

I am hoping between SURF , edge detection, RGB matching, this is possible. but may need to also use the infrared sensors from the kinect for a 3d reconstruction and depth finding to help separate the objects

*something I believe that may help very much at distinguishing objects from each other is texture detection, if that exists.

1) is this possible?
2) what tools/programs would be needed?
3) strategies on going about this?
4) does a solution already exist?

NOTES: Im using vs2010 with opencv and kinect

Cœur
  • 37,241
  • 25
  • 195
  • 267
user3064588
  • 13
  • 2
  • THis problem is ill-constrained. If you don't know anything about the objets are, how can you know whether particular feature detectors will be useful? – Bull Dec 04 '13 at 09:02
  • Not sure I understand the comment. im not looking to identify objects based on if they are one solid piece of mass. IE: a picture with a frame is one piece, but i would like the frame to be detected as being different than the painting inside of it. Im also trying to use feature detectors to tell me major differences in materials, like texture, color, and shape, in hopes of building a confidence index of how likely the 2 perceived objects are different. I wouldnt want a image of a white wall to be seen as 2 objects because theres a shadow on half of it, but it still work forme – user3064588 Dec 05 '13 at 02:10

2 Answers2

1

1) I would say it is not possible. From computer vision perspective (without 3D depth information) you might want to have a look at the "Gestalt psychology" problems (http://en.wikipedia.org/wiki/Gestalt_psychology), where you can see to find a lot of ambiguity already in very simple images. Analyzing a real world scenario without (and often even with) any background knowledge is much worse. Sometimes even a human isn't able to distinguish between seeing one or two objects and/or it depends on the interpretation and you have to consider that a human has many years of experience and a lot of background knowledge.

3) what you want to do is image segmentation. Without any background knowledge I would suggest some edge detection (e.g. Canny edge detection) combined on both, depth image and color-/grayimage. You will have to group detected edges (have a look at Gestalt psychology again) and/or extract contours, but at the end you'll still be far away from detecting all objects and subobjects.

Micka
  • 19,585
  • 4
  • 56
  • 74
  • Thanks for input. I expected depth / 3d reconstruction would be needed, along with canny edge detection. Another question is if a comparing a stream of images (video) of the objects around the environment would be helpful – user3064588 Dec 05 '13 at 01:48
  • Also, for my purposes, errors merging 2 separate object into one is not acceptable, but errors that split what should be one object into multiple pieces will still work, to an extent. ( in the hundreds total per image ) – user3064588 Dec 05 '13 at 02:17
  • as an example: there will always be errors merging multiple objects. for example if you have a pile of papersheets, is that a single object or multiple ones? what about a book (consisting of a 'pile' of papersheets)? You can try out a 3D reconstruction from a videostream (have a look at kinectFusion e.g.) and split objects by some heuristics (since objects don't fly in the air they're nearly ALL connected), but chances of getting what you seem to want are quite small I guess. – Micka Dec 05 '13 at 09:02
  • kinectfusion is what i am using. In case your wondering, my objective was to outline components that make up an object and manually label them, as well as giving multiple components that make up an object a single label. Since i would be manually labeling the objects/components, i can select the ones that are best represented, so a level of inaccuracy would be acceptable – user3064588 Dec 06 '13 at 00:19
  • ok, that's a quite different task from what I read in your initial posting. You will still have to split from some kind of heuristic, and later manually fit components together to an object. Heuristics might be to separate components from cuboids or from spheres. Example: http://s9.postimg.org/sd3ep63an/Object_Splitting.png Left is a typical situation after KinFu reconstruction (viewed from the side). Detect the plane/centroid and split whatever sticks out and interpret it as an object/component (black plane,blue "object"). Should be a very open topic though(guess) maybe worth a dissertation – Micka Dec 06 '13 at 11:01
  • thanks, think this is about as close to an answer im going to get. – user3064588 Dec 06 '13 at 17:36
0

Try to run active contours on each bounding box of an object.

GilLevi
  • 2,117
  • 5
  • 22
  • 38