How to annotate the ground truth for image segmentation?

Question

I'm trying to train a CNN model that perform image segmentation, but I'm confused how to create the ground truth if I have several image samples?

Image segmentation can classify each pixel in input image to a pre-defined class, such as cars, buildings, people, or any else.

Is there any tools or some good idea to create the ground truth for image segmentation?

Thanks!

Answer 1

One tool that pops to mind is MIT's LabelMe toolbox: this toolbox is mainly for browsing the existing labeled images of the dataset, but it has an option to annotated new images as well.

There's alos this github repository for COCO UI you might find useful.

Answer 2

For semantic segmentation every pixel of an image should be labeled. There are three following ways to address the task:

1. Vector based - polygons, polylines

2. Pixel based - brush, eraser

3. AI-powered tools

In Supervisely, tools to perform 1,2,3 are available.

Below are two videos that compare polygon vs AI-powered tools: cars segmentation and food segmentation.

More details about annotation features of Supervisely can be found here.

Answer 3

Try out https://www.labelbox.io/. Here is what their image segmentation template looks like...

A lot of the code is open source and they have a hosted service to manage labeling end to end.

Answer 4

I created an open source tool called COCO Annotator: It provide any features where other tools fall short:

• Directly export to COCO format
• Segmentation of objects
• Useful API endpoints to analyze data
• Import datasets already annotated in COCO format
• Annotated disconnected objects as a single instance
• Labeling image segments with any number of labels simultaneously
• Allow custom metadata for each instance or object
• Magic wand/select tool
• Generate datasets using google images
• User authentication system

Answer 5

SageMaker Ground Truth has dataset management and UIs to enable semantic segmentation. https://aws.amazon.com/sagemaker/groundtruth/

We recently released an enhancement to the UI which speeds up annotations considerably, by automatically finding region boundaries. Read about it here: https://aws.amazon.com/blogs/machine-learning/auto-segmenting-objects-when-performing-semantic-segmentation-labeling-with-amazon-sagemaker-ground-truth/

Answer 6

The overall process is:

1. Load the data into a tool
2. Draw a shape. E.G. Using a polygon tool or other shape tools
3. Export the points and use them for training directly, or convert them into a dense pixel mask.

Annotation Process Detail

Generally speaking, vector based tools are preferred because they are more accurate. For example on the left is 4 polygon points and on the right is a brush.

Counter intuitively it can even be faster to trace out the points. For example here, I simply moved my mouse over the outline which records this:

But it's actually stored as points:

Other shapes are used

For example, you can use an ellipse or a circle to create segmentation masks. There's no requirement to actually manually create a polygon.

Autobordering

One of the main concepts behind wanting to get to 100% coverage of pixels is a concept called autobordering. Basically it just means that the edges of objects intersect well. In Diffgram for example you can select two points and it will create a matching border.

Here's an example:

Finally there are many other concepts like pre-labeling, copying to next, tracking etc.

Tooling

Diffgram's Segmentation Labeling Software is a great tool option for Image and Video Annotation. It's open-core, you can run it on your kubernetes cluster.

Answer 7

ITK-SNAP is designed for medical image segmentation. It is easy to learn and powerful still. You can try it.

Answer 8

you can use averaging and normalization techniques all over your dataset so create your ground truth and then label different structures. for this purpose, you can think about creating a so called |template|. make sure all your data are co registered initially.