Label and train

1. Label your first images

Make sure to follow the guidelines (right toolbar), which has the most detailed instructions to walk you through creating your first model.

Your labeling tools and how to use them

Select your first class on the left toolbar. Once you click, you will have three different tools you can toggle between by clicking them: AI Detect, AI Select, and Pen. You'll also now be in drawing mode, so you will now need to hold Space to be able to pan across your image.

AI Detect is the fastest, and you should use it unless it isn't working well for you, filling in the gaps with AI Select. The Pen is the slowest, and should be used if AI Select is not working for you.

It is expected and normal to have object cut off by the tiles. When the model is used for analysis, it will automatically stitch across tile boundaries. Do not adjust your tile size to avoid this.

AI Detect might not work well for harder images - in these cases, you may need to use AI Select heavily. We'll be improving AI Detect over time.

To use AI Detect on a tile, drag a tight box around one example of your object, using the crosshairs to help you. AI will locate the position of similar-looking objects, and also segment them.

Using the filters

After you use AI detect, you'll need to filter the results, using the sliders. The confidence score slider controls the model score threshold at which objects are filtered out. Drag higher to get fewer labels, and lower to get more labels.

The size range slider has a minimum and maximum upper bound. You can filter out or include larger or smaller objects using these ranges.

Labeling guidelines

  1. Focus on one tile at a time. Label every object in that tile, including objects that are cut off by the edge of the tile. Once every example in that tile has been labeled, click Fully Labeled on top of that tile (in the gray box).

  2. Track your progress in the right sidebar tutorial as you mark more tiles as Fully labeled, until you fill up the bar.

  3. Make sure you are being careful and consistent with labeling. Don't miss objects, and label the border of objects precisely, moving to the Pen when necessary. It is far better to label precisely than it is to label more data less precisely.

Other settings you might use for labeling

You might need to switch images, add images, configure classes or size groups class, or adjust your brightness/contrast settings. Find the instructions for those below.

Changing class color

You can also choose from different colors to represent the class. Simply click the colored circle to open up a color picker.

Changing the name or deleting a class

Hover over the class to reveal the 3 dots button, which you can click. From there, use the dropdown to rename or delete a class. If you delete a class, every object created for that class will be deleted as well.

Add a new class

Click the + button next to your size group on the left sidebar to add a new class. You might need to add a new size group if your new class represents an object that is of significantly different size or a different segmentation type.

2. Train your first AI model

Once you've filled up the Tutorial progress bar, you're ready to train your first AI model! Click on the Train tab, and click Train a new model. You'll see the following screen. You can add an optional description and customize transforms if needed. Then, click Start training.

Now, you can monitor your training progress. It will have a progress bar and an approximate ETA.

3. Test and improve your model

Once fully trained, we'll see how well your first model does on tiles it hasn't seen.

Navigate to Project and model details, and click on the AI model version, and click on the Statistics tab. You'll see "Quantitative metrics," These metrics indicate how this AI model performed on the images selected for training and test. If there is a particular group of data your analysis didn't do well on, it might mean you need more data in that area.

"Training Set" refers to tiles that you labeled that were used to train the AI model. "Test Set" refers to tiles that were not included in training the AI model. The statistics given are a measure of how close the AI model was to replicating your labels, that is how accurate it is.

Intersection-over-union (IoU) indicates how closely the AI's predictions are with your training labels you put in. It's calculated by dividing the area of overlap between the predictions and your training labels by the union, that is the area covered by at least one prediction. That gives a score between 0 and 100, where higher scores indicate better accuracy.

The bbox column under statistics means a bounding box around the predicted and labeled objects while the mask means the outline of the detected object itself.

If the score is 100, that indicates the predicted labels were exactly overlapping with the labels you put in, whereas a score of zero means no predicted labels overlapped with your training labels.

This can be helpful in quantifying how accurate a model is, but qualitatively determining if a models is making the right calls or not is often better.

Navigate to Label on the top bar, or click Evaluate on your trained pipeline to be taken there.

Test using AI Prelabel

As prompted by the right toolbar, use the new AI Prelabel button next to your size group to test your new model out. Click the tool, and then click unlabeled tiles on your images to prelabel them - they should be labeled in less than 10 seconds. If you have multiple size groups, you'll need to do this for each size group.

Your first model will almost always have mistakes after your first training run, due to the very small number of objects it was trained on. Choose Improve model.

Improving your model

Now that you've trained a first model, you should probably stop using AI Detect. Using AI Prelabel, followed by AI Select for corrections is more efficient after training your first model.

Once you've trained and evaluated your first AI model, you will now be able to label and improve assisted by AI. Your workflow is now:

  1. Prelabel tiles with AI - Use the new AI prelabel tool on the left toolbar to test your model, and generate labels with your last model version.

  2. Correct prelabeled tiles - Prelabeled tiles are shown in purple. Correct these by adding, removing, and editing objects, and mark them as corrected. Make sure to focus on one tile at a time, only clicking Corrected once that tile is completely correct. Verify that every object has been labeled, even ones on the edge, and incorrect labels have been deleted.

  3. Train and test - Once you have a good amount of newly fully labeled and fully corrected tiles, train a new version like before to get a better model. The tutorial will suggest how many tiles you should correct to get a big boost in performance, but you can do this at any time.

You'll want to repeat these 3 steps until you have perfect or close to perfect performance! In most cases, it's best to improve your model at least two times before moving on, but you can always come back to improve it more.

Last updated