Smart Segment Tool: Segment Faster on Ango Hub

In data labeling, efficiency matters. It’s important that quality data gets delivered as fast as possible so that model training, validation and deployment may follow. However, labeled data often becomes the biggest bottleneck in deploying ML models. Here’s where Smart Segment comes into play.

Image Segmentation is one of the areas where this bottleneck is particularly acute due to the pixel level accuracy required. At Ango AI our goal is to ensure that the time it takes to label a data set is kept at its lowest possible, while maintaining accuracy.

To achieve this we have an arsenal of AI assistance tools that greatly speed up the annotation workflow, learn more about them here. Smart Segment is another addition to our suite of AI assistance tools.

The tool takes a rough mask (polygon or segmentation) and processes it to create highly accurate tightly bound masks for the object of interest contained within the original mask. 

Advantages

While segmenting a moderately complex object by its boundaries up to a pixel level can take multiple minutes, Smart Segment takes a rough mask drawn in 2-3 seconds and refines it to capture the boundaries of this complex object, effectively reducing time per label.

Based on our practical experiments on a variety of data we have observed the following:

• For complex objects reduction of segmentation time from 60-100 seconds to 5-10 seconds.
• For simpler objects reduction of segmentation time from 20-30 seconds to 5-10 seconds.

Reduction of annotation time per object consequently results in the following benefits:

• Faster preparation of dataset, thus faster time to model training
• Lower mental load for annotators
• Lower cost per label
• Tighter segmentations, thus more accurately segmented (higher quality) dataset.
  

How Smart Segment Works

Smart Segment works by employing a deep learning model trained on a very wide variety of images coming from multiple open source datasets with more that 1000 classes for segmentation. Using the training data the model is able to accurately predict the pixels that contain the object vs the pixels that don’t.

The model has a ResNet 101 backbone along with an FPN to retrieve features within images of different scales. The model performs well class agnostically as it was trained to learn “objectness” (i.e. Object class only) rather than classification of pixels into multiple classes.

How to use Smart Segment

The tool is extremely easy to use:

1. Open an asset on Ango Hub
2. Select the Smart Segment tool
3. Draw a rough boundary around the object 
4. Let Smart Segment refine this boundary for higher accuracy

Use Cases

Smart Segment works well with common world objects and can be used for a variety of segmentation tasks:

• Agriculture
  • Plants, fruits and crops often have complex boundaries and labeling each object can take multiple minutes with Smart Segment this can be done in a matter of seconds.
• Autonomous Vehicles
  • Common objects encountered in AV datasets such as cars, pedestrians, traffic lights, signs e.t.c. have a moderate complexity in terms of segmentation, using Smart Segment, this complexity can be highly reduced. Allowing us to create AV datasets faster.
• Medical
  • While the tool is trained on understanding the definition of objects in real world context (common objects) for medical domain, where objects are clearly discernible such as skin lesions, microscopy the refiner works well in getting tight boundaries

The quality assurance use-cases of this tool are: 

• It can also be used on already labeled dataset to take images in batch and simply refine the labeled masks, the performance of the model is such that the resultant IOU (compared to the groundtruth) of masks is almost always higher, meaning even if the mask is already good the tool further increases tightness and accuracy. 
• It can be used on bounding box projects(object detection rather than segmentation) as well to obtain tighter bounding boxes, also increasing the overall IOU.

The Smart Segment tool will be available soon on our all-in-one data labeling platform Ango Hub, used every month by industry leaders to annotate millions of data points in mission-critical environments.

Try Ango Hub now, free for small projects with no credit card required. Book a demo with us to talk about how to solve your data labeling needs.

---

Author: Balaj Saleem
Technical Proofreader: Onur Aydın