Advanced Annotation Tools

Master interpolation, track mode, attribute annotation, object propagation, grouping, merging, and splitting

CVAT provides powerful advanced tools that significantly accelerate video annotation and enable sophisticated annotation workflows. These tools leverage temporal continuity, object relationships, and intelligent algorithms.

Track Mode and Interpolation

Understanding Tracks

Tracks are temporal annotations that represent the same object across multiple video frames:

• Single identity: One track = one object throughout its appearance
• Keyframe-based: You define object position at keyframes
• Automatic interpolation: CVAT calculates positions between keyframes
• Efficient: Annotate once every N frames instead of every frame

Interpolation can reduce annotation time by 5-10x for videos compared to frame-by-frame annotation.

Creating Tracks

1. Select any drawing tool (rectangle, polygon, etc.)
2. Choose a label
3. Click Track (instead of Shape)
4. Draw the object on the current frame
5. Navigate forward several frames
6. Adjust the object position if needed (creates a new keyframe)
7. Repeat until the object disappears or leaves the frame

Track Navigation Shortcuts

K        - Toggle keyframe on current frame
F        - Previous frame
D        - Next frame
V        - Next frame (while drawing)
C        - Previous frame (while drawing)

Keyframes

Keyframes are frames where you explicitly set the object’s position and shape:

• Manual keyframes: Created when you adjust the object
• Automatic keyframes: First and last frame of track
• Visual indicator: Diamond icon in the objects sidebar
• Toggle keyframe: Select object and press K

When to add keyframes:

• Object changes direction
• Speed changes significantly
• Shape deforms
• Object enters/exits occlusion
• Camera motion changes
• At least every 30-60 frames

Rule of thumb: Add a keyframe whenever you notice the interpolated position is inaccurate. The more keyframes, the more accurate the interpolation, but more work required.

Interpolation Behavior

CVAT interpolates different properties:

For all shapes:

• Position (x, y coordinates)
• Size (width, height)
• Rotation angle

For polygons and polylines:

• Each vertex position independently
• Shape can morph smoothly

For skeletons:

• Each keypoint position independently
• Natural pose transitions

Not interpolated:

• Masks (must set on each frame)
• Attributes marked as “immutable”
• Discrete properties (label, color)

Interpolation Types

CVAT uses linear interpolation by default:

• Smooth, constant velocity between keyframes
• Works well for most objects
• Simple and predictable

For complex motion:

• Add more keyframes at motion changes
• Consider using auto-tracking tools

Managing Track Visibility

Outside Property (Q):

Mark a track as “outside” when the object leaves the frame:

1. Navigate to the frame where object exits
2. Select the track
3. Press Q or toggle “Outside” in sidebar
4. Object disappears from this frame onward
5. Toggle off when object re-enters

“Outside” is frame-specific in tracks. An object can be outside on frames 50-100 and visible again on frame 101.

Occluded Property (O):

Mark frames where the object is hidden but still in the scene:

1. Select the track
2. Press O on frames where object is occluded
3. Visual indicator appears
4. Useful for datasets that distinguish visibility states

---

Object Propagation

Propagate copies an object to subsequent frames:

When to Use

• Object is stationary or moves slowly
• Faster than creating a track for short durations
• Initial annotation before refinement

Using Propagate

1. Select an object (shape or track)
2. Press Ctrl+B or right-click → “Propagate”
3. In the dialog:
   • Up to frame: End frame number
   • Propagate as: Shape or track
4. Click OK
5. Object is copied to all frames in range

Propagate as shape to create independent copies on each frame. Propagate as track to create a single interpolated track.

Propagate vs Track

Feature	Propagate (shapes)	Track
Independence	Each frame separate	Single object
Editing	Edit each copy individually	Edit keyframes, interpolates between
Use case	Static objects	Moving objects
Efficiency	Good for few frames	Excellent for many frames

---

Grouping Objects

Purpose

Group multiple objects that logically belong together:

• Parts of a single entity (car + wheels)
• Related objects (person + accessories)
• Maintain relationships across frames
• Export as single composite object

Creating Groups

Method 1: Group Mode

1. Press G to enter group mode
2. Click objects to add to group (or select first)
3. Click “Group” button or press G again
4. Selected objects are now grouped

Method 2: Shift-Click Selection

1. Select first object
2. Shift+Click additional objects
3. Right-click → “Group”

Grouping Shortcuts

G          - Toggle group mode
Shift+G    - Reset group (ungroup selected objects)
Shift+Click - Multi-select objects

Group Properties

• Group ID: All grouped objects share a group ID
• Color: Can set uniform color for group
• Selection: Clicking one object highlights the group
• Operations: Some operations affect entire group

Ungrouping

1. Select any object in the group
2. Press Shift+G or right-click → “Ungroup”
3. Objects become independent

Best Practices

• Group related objects that move together
• Use consistent grouping across all frames (for tracks)
• Document grouping rules in annotation guidelines
• Consider if hierarchical labels might be better than grouping

---

Merging Objects

Purpose

Combine multiple separate annotations into a single object:

• Merge over-segmented regions
• Combine parts annotated separately
• Fix duplicate annotations
• Combine tracks that represent the same object

Using Merge Mode

1. Press M to activate merge mode
2. Click objects to merge (minimum 2)
3. Selected objects are highlighted
4. Click “Merge” or press M again to complete
5. Objects combine into one

Merged objects take the label of the first selected object. Attributes may need manual reconciliation.

Merge Behavior by Type

Rectangles:

• Creates bounding box that encompasses all input rectangles

Polygons:

• Combines into a single polygon (may have complex boundary)
• Handles overlapping regions

Masks:

• Performs pixel-wise union
• Perfect for combining segmentation regions

Tracks:

• Merges track histories
• Useful for fixing tracking errors where same object has multiple IDs

Merge Shortcuts

M  - Toggle merge mode

Limitations

• Cannot merge different shape types (e.g., rectangle + polygon)
• Merging tracks requires temporal overlap consideration
• Attributes from only one object are retained

---

Splitting Tracks

Purpose

Divide a track into two separate tracks at the current frame:

• Fix tracking errors (wrong ID continuation)
• Split tracks when objects separate
• Correct annotation mistakes

Using Split Mode

1. Navigate to the frame where you want to split
2. Press Alt+M to activate split mode
3. Click the track to split
4. Track divides:
   • Before current frame: Original track
   • Current frame onward: New track with new ID

Splitting is irreversible without undo. Make sure the split point is correct before confirming.

Common Scenarios

ID Switch:

• Two objects cross paths
• Tracker assigns wrong ID
• Split both tracks at crossing, then join correctly

Object Separation:

• Single object splits into multiple (e.g., flock disperses)
• Split track, annotate each resulting object separately

Split Shortcuts

Alt+M  - Toggle split mode

---

Joining Tracks

Purpose

Combine two separate tracks into a single continuous track:

• Fix fragmented annotation of same object
• Connect tracks across occlusion
• Correct tracking gaps

Using Join

1. Click the Join tool in controls sidebar
2. Click the first track (earlier in time)
3. Click the second track (later in time)
4. Tracks merge into one continuous track
5. Object maintains single identity across both track segments

Requirements

• Tracks must be same shape type
• Tracks should be temporally sequential (or overlapping)
• Both tracks must have same label (or you’ll be prompted to choose)

Join vs Merge

Feature	Join	Merge
Purpose	Combine sequential track segments	Combine simultaneous objects
Input	2 tracks	2+ objects
Result	Single continuous track	Single object
Use case	Fix fragmented tracking	Combine over-segmentation

Workflow for fixing tracking gaps:

1. Object is tracked incorrectly, creating multiple track IDs
2. Split incorrect tracks at error points
3. Join correct segments to form proper continuous tracks

---

Slicing Objects

Purpose

Cut an object into multiple parts using a line:

• Divide over-segmented regions
• Split connected components
• Create separate annotations from one

Using Slice Mode

1. Click the Slice tool in controls sidebar
2. Select the object to slice
3. Draw a line across the object:
   • Click to place line vertices
   • Press N or double-click to finish line
4. Object splits into multiple objects along the cut line

Slice Behavior

For polygons:

• Creates 2+ separate polygons based on where line intersects
• Each new polygon becomes an independent object

For masks:

• Divides mask into connected components
• Each component becomes a separate mask

Applications

• Separate touching objects annotated as one
• Divide regions after approximate annotation
• Fix over-segmentation from auto-annotation
• Create part-based annotations

---

Attribute Annotation

Understanding Attributes

Attributes are properties associated with objects:

• Defined per label: Each label has its own attributes
• Types: Checkbox, radio, select, number, text
• Mutable vs Immutable:
  • Immutable: Same value across all frames in track
  • Mutable: Can change per frame

Setting Attributes

1. Select an object
2. Expand DETAILS section in objects sidebar
3. Set attribute values
4. For mutable attributes in tracks:
   • Navigate to frames where value changes
   • Update attribute value
   • CVAT tracks changes per frame

Attribute Annotation Workspace

For efficient attribute annotation:

1. Switch to Attribute Annotation workspace
2. Interface shows:
   • Large object view
   • Attribute controls
   • Object and frame navigation
3. Keyboard shortcuts for attributes
4. Rapid annotation workflow

Workflow:

1. Navigate to object (Tab or sidebar)
2. Set attributes with keyboard or clicks
3. Press Ctrl+Enter to save and move to next object
4. Repeat

Keyboard shortcuts for attributes are automatically assigned:

• Number keys (1-9) for radio/select options
• Enter to confirm
• Tab to next object

Best Practices

Immutable Attributes

Use for properties that don’t change (object color, vehicle type, etc.)

Mutable Attributes

Use for time-varying properties (door open/closed, lights on/off)

Consistent Values

Maintain consistency across similar objects

Documentation

Document attribute definitions clearly in project guidelines

---

Copy, Paste, and Duplicate

Copying Objects

Within same frame:

1. Select object
2. Ctrl+C to copy
3. Ctrl+V to paste
4. Adjust position of pasted object

Across frames:

1. Copy object on one frame
2. Navigate to target frame
3. Paste with Ctrl+V
4. Useful for objects that appear multiple times

Pasted objects inherit all properties from the source: label, attributes, size, shape. Only position may need adjustment.

Copy Shortcuts

Ctrl+C  - Copy selected object
Ctrl+V  - Paste object
Ctrl+X  - Cut object (copy and delete)

Use Cases

• Repeated objects: Copy-paste-adjust faster than drawing from scratch
• Similar objects: Copy, paste, change label
• Template annotation: Copy object with attributes set, paste to new instances

---

Object Relationships

Z-Order (Depth Ordering)

Control which objects appear in front when overlapping:

Change Z-Order:

• + or =: Bring selected object forward
• -: Send selected object backward
• Ctrl++: Bring to front
• Ctrl+-: Send to back

Z-Order Layers:

• Objects sidebar can display objects grouped by Z-order layer
• Higher Z-order = rendered on top
• Useful for organizing complex scenes

Parent-Child Relationships

Some label configurations support hierarchical relationships:

• Skeleton sublabels: Keypoints belong to skeleton
• Part annotations: Wheels belong to vehicle
• Nested structures: Building parts, room elements

Parent-child relationships are defined in label configuration.

---

Batch Operations

Lock/Unlock All Objects

• T+L: Toggle lock for all objects on current frame
• Prevents accidental edits
• Unlock to edit again

Hide/Show All Objects

• T+H: Toggle hide for all objects on current frame
• Clear canvas for easier viewing
• Objects still exist, just hidden

Delete Multiple Objects

1. Select multiple objects (Shift+Click)
2. Press Del or Backspace
3. All selected objects are removed

Batch delete is immediate and can only be undone with Ctrl+Z. Be careful when deleting multiple objects.

---

Advanced Keyboard Shortcuts

Track and Interpolation

K            - Toggle keyframe
O            - Toggle occluded
Q            - Toggle outside
P            - Toggle pinned
L            - Toggle lock
Ctrl+B       - Propagate object

Object Operations

G            - Group mode
Shift+G      - Reset group
M            - Merge mode
Alt+M        - Split mode
+/-          - Adjust Z-order
Ctrl+C/V/X   - Copy/Paste/Cut

Batch Operations

T+L          - Lock all
T+H          - Hide all
Shift+Del    - Delete all on frame

Navigation During Annotation

V            - Next frame (while drawing)
C            - Previous frame (while drawing)
Tab          - Cycle through overlapping objects

---

Best Practices for Video Annotation

Efficient Workflow

1. First pass: Create tracks for all objects with keyframes every 30-60 frames
2. Review pass: Navigate through interpolated frames, add keyframes where needed
3. Refinement pass: Adjust boundaries, set attributes
4. Quality check: Review entire video for consistency

Keyframe Strategy

1. Start and End

   Always annotate first and last frame where object is visible

2. Motion Changes

   Add keyframe whenever object changes direction or speed significantly

3. Occlusions

   Keyframe before entering and after exiting occlusion

4. Shape Changes

   For polygons/skeletons, keyframe when shape deforms

5. Regular Intervals

   Even for smooth motion, add keyframes every 30-60 frames to prevent drift

Tracking Difficult Objects

Fast motion:

• More frequent keyframes
• Consider auto-tracking tools

Occlusions:

• Mark “outside” during full occlusion
• Mark “occluded” during partial occlusion
• Keyframe before and after

Camera motion:

• Background motion affects interpolation
• More keyframes needed
• Use stabilization if available

Deformable objects:

• Use polygons or skeletons
• Frequent keyframes
• Each vertex interpolates independently

---

Troubleshooting

Interpolation looks wrong

• Add more keyframes at motion changes
• Check that keyframes are properly set (press K to toggle)
• Ensure “outside” flag is not incorrectly set
• Consider if object motion is too complex for linear interpolation

Can't merge objects

• Objects must be same type (all rectangles, all polygons, etc.)
• Both objects must be on the same frame
• Check that merge mode is active (M key)

Track split in wrong place

• Use Undo (Ctrl+Z) immediately
• Navigate to correct frame before splitting
• Consider joining tracks instead

Grouped objects behave unexpectedly

• Ungroup (Shift+G) and regroup
• Check that all intended objects are selected
• Some operations may not support groups

Propagate created too many objects

• Delete propagated objects and try again with shorter range
• Consider using track mode instead
• Use Ctrl+Z to undo if just completed

---

Next Steps

Manual Annotation

Basic annotation workflows and techniques

Auto-Annotation

Use AI tracking to automate interpolation

Quality Control

Review and validate annotations

Export Data

Export your completed annotations