User Manual

Initialize raster annotation session

Create a raster annotation set (ART)

1. Go to the field of view (FOV) of interest, use Right Mouse Key to access menu, and select RACT - ART (Raster Annotation Correction Tool - Automated Region Segmentation of Tumor).

FOV/ROI

For simplicity, we will refer to FOV (Field of View) throughout this document. Note that RACT can also be used for annotation of specific parts of the image specified by types 'FOV' or 'ROI'.


2. This will ask you to lock the current FOV. (Not applicable if the FOV is already locked)
3. Then RACT page will open.

4. Toggle off Brush Auto-Fill in the Tools panel (2). This fills areas that become completely encircled by a class with the encircling class, e.g. when drawing region outlines.
5. Select Region in the Layer Selection (3), and select a class in a dropdown.
6. Adjust Opacity as needed in the Display Options (1), and Brush Size in the Tools panel.
7. Outline your areas with Shift + Left Mouse Key to draw a new object in the annotation view panel (5).
8. This will open pop-up Start a new session. Confirm the raster annotation set name. The vector annotation set name is pre-filled by default, but can be modified.

9. Always wait until the saving is completed (i.e., wait until the "Saved" message appears in the top panel (4)).

10. After the saving is completed, you will see the raster annotation set name displayed under User annotations in the Download panel on the right (7).
11. Ensure the area outlines are closed before using the Reclassify tool to color all pixels within the area with the correct class (Left Mouse Key).
12. Finish session Ctrl + S to save your latest changes. You can start a new session on the next edit.
13. In the worklog section in the right panel (8) you can see the log of the relevant actions.
14. Navigate through the FOVs by either directly selecting it in the image in the slide overview panel (6), or using the arrow buttons. Alternatively, you can return to the slide view by clicking Return To Full Slide.

Create a raster annotation set (SSTC)

1. Go to the field of view (FOV) of interest, use Right Mouse Key to access menu, and select RACT - SSTC (Raster Annotation Correction Tool - Subcellular Segmentation of Tumor Cell).

2. This will ask you to lock the current FOV. (Not applicable if the FOV is already locked)
3. Then RACT page will open.

4. Load Proposed annotations in the Download panel (7).
5. Apply Recalculate in the Recalculate panel (9).
6. If annotation errors are detected, Confirm error and manually correct the errors in the annotations.
7. This will open pop-up Start a new session. Confirm the raster annotation set name. The vector annotation set name is pre-filled by default, but can be modified.
8. After the saving is completed, you will see the raster annotation set name displayed under User annotations in the Download panel on the right (7).
9. Repeat step(-s) 5, 6 until annotations are correct.
10. SHOW ALL layers and confirm they are correct.
11. Finish session Ctrl + S to save your latest changes. You can start a new session on the next edit.
12. Hide all layers in the Layer Selection panel (3).
13. Show region layer (toggle visibility ON).
14. Correct region outlines as needed with Left Mouse Key.
15. Show cell center layer.
    • Toggle Recenter ON only if you want to enable repositioning of the cell centers during Recalculate.
16. Place missing cell centers (Left Mouse Key) or remove cell centers (Right Mouse Key).
17. Correct other annotation types (i.e., cell membranes, nuclei membranes, exclusions) as needed.
18. SHOW ALL layers and confirm they are correct.
19. Finish session Ctrl + S to save your latest changes. You can start a new session on the next edit.
20. In the worklog section in the right panel (8) you can see the log of the relevant actions.

Annotation input types

There are two input types available for initializing annotations in RACT: Proposed (DRC Deliveries) and User (existing raster annotations).

• Proposed:
  1. In the Input Data Type dropdown menu, select Proposed annotations.
  2. In the second dropdown menu, choose a DRC delivery name. Only DRC deliveries that have the correct level and classes are offered for selection. This will load annotations for the current FOV based on the respective model predictions.
  

• User:
  1. In the Input Data Type dropdown menu, select User annotations.
  2. In the second dropdown menu, choose the name of a previously created raster annotation set. This will load the annotations for the current FOV from the given set.
  

Edit raster annotation

1. Activate layer by clicking on the radio button (1) that belongs to the layer.
2. Hide and show individual layer by enabling on and off the visibility toggle next to the layer name (2).
3. Hide all layers by clicking the HIDE ALL button (3).
4. Show all layers by clicking the SHOW ALL button (4).

Editing options

• Display options: In the Display Options panel, you can adjust the overall appearance of the annotations (1), as well as the thickness (2) and opacity (3) of the annotations.

• Tools: In the Tools panel, you can:
  • Toggle the brush auto-fill (1). This fills areas that become completely encircled by a class with the encircling class, e.g. when drawing region outlines.
  • Adjust the brush size (2).
  • Toggle pan & zoom mode (3).
  • Undo (4) or redo (5) the last performed annotation.
  • Reclassify (6) (change the annotation class of an already drawn annotation or clear the selected region by Right Mouse click).
  • Apply SAM (Segment Anything Model) .

Introduction to SAM (Segment Anything Model)

When using the SAM, you can easily view proposed annotations by hovering over the image, scrolling through different SAM segmentation proposals with Mouse Wheel, and segmenting the area with a Left mouse click. To remove any annotations, simply use the Right mouse click.

Important Note: The first use of SAM includes a disclaimer warning about occasionally wrong annotations. SAM embeddings are computed at the full image resolution, hence it is expected to perform equally well at different zoom levels.

Tip: Utilize X to temporarily enable SAM, and then switch to the classic brush tool for final corrections. Use the F shortcut to switch to Reclassify and remove non-accurate regions.

Raster annotation classes

• Classes used for vector region annotations can also be valid raster annotation classes. This applies to both class names and class colors.
• Note that only a subset of classes is available by default when creating new raster annotations.
• Additional classes will be displayed in RACT if available, for example, as a result of rasterization of legacy classes present in the vector annotation set.
• The class label map shall be dense without the latter being necessarily enforced. For additional documentation, please refer to the raster annotation image format.
• Available classes and colours can be obtained using Automation API Gateway (AAGW) GET /v1/annotationClasses endpoint.

ART Regions classes

Below are current annotation classes for Regions layer for the Automated Region Segmentation of Tumor (ART) annotation mode.

ART Exclusions classes

Below are current annotation classes for Exclusions layer for the Automated Region Segmentation of Tumor (ART) annotation mode.

Raster annotation review cycle

Restore raster annotation

1. Click the Show version selector button. This prompts a new window.
2. Use the dropdown to choose the annotation version.
3. Note that the latest version cannot be restored.
4. After selecting a previous version, click Restore. This will make the restored past version the new head of the versioning history.

View work log

After the annotation session has been started, the user can see the key actions logged in the work log panel (8). Each log entry includes: date, time, user name, and action.

Annotate as part of workflow

In addition to standalone RACT usage, starting from version 4.2.0, RACT has been integrated as part of several Workflow types. This integration streamlines the annotation process by embedding RACT functionality directly into workflow tasks.

When working within a workflow, RACT provides the same powerful annotation capabilities while being guided by the workflow structure.

• Testset Annotation Workflow - for creating test set annotations with ART and SSTC
• Training Annotation Workflow - for creating training annotations based on model performance
• Validation Set Annotation (Morphology based) Workflow - for morphology-based validation annotations
• Validation Set Annotation (Model-performance based) Workflow - for performance-based validation annotations

Key differences for Test Set Annotation Workflow

The Test Set Annotation Workflow represents a more complex workflow implementation with specific user interface restrictions, particularly regarding ART and SSTC integration. When RACT operates as part of this workflow, several key differences apply compared to standalone usage:

Access and Mode Indication

• Role-based access: Only users with the ProjectWorkflow role can access and annotate within the workflow. Users without an assigned task will be automatically redirected to the task list if they attempt to access a task.
• Mode visibility: The system clearly indicates when operating in ART annotation mode for Test set (1), ensuring users are aware of the workflow context.

Annotation Set and Class Restrictions

• Pre-existing annotation sets: Users work with annotation sets automatically created and named by system (2) rather than creating new sets from scratch.
• Predefined Regions classes: Annotation can be restricted to a finite set of predefined Regions layer classes (3), as specified in the workflow configuration. Users cannot add custom classes.

Disallowed Features and Tools

To ensure consistency and adherence to the Test Set Annotation Workflow requirements, several RACT capabilities are restricted or disallowed:

• Tools: Segment Anything Model (SAM) is not available (4).
• Layer editing: Field of View (FOV) / Region of Interest (ROI) and Exclusions layers cannot be edited (5).
• Navigation: FOV/ROI backward/forward navigation and direct FOV/ROI selection are disallowed (6).
• Download options: Input data type is fixed to 'User' and annotation set is locked to the existing workflow annotation set (2).
• Lifecycle transitions: Users cannot transition annotation status from "To Review" to "Under Review" status (7) as Review step is absent in this workflow.

Note: When a feature is restricted, hovering over the disallowed control will display a tooltip explaining the reason for the restriction.

Navigation and Task Completion

• Restricted navigation: Users can only navigate to other tasks through the task list, preventing unintended workflow interruptions.
• Status requirement for completion: Annotation tasks can only be marked as finished when the annotation review lifecycle status is set to "To Review".
• Work instructions: Click the information icon (i) in the Utilities panel to access detailed work instructions in a new tab.
• Automatic data transfer: Upon Finalize ART task completion, the system automatically transfers the Regions layer Epithelium-Tumor class to the SSTC model annotation set for use in subsequent Annotate SSTC workflow step, and redirects the user to the task list.
• Sign off: Task completion is signed off automatically. Users will see the notification "Signing off on this step is not supported with a 21 CFR Part 11 compliant electronic signature" until the eSignature functionality is implemented.

Troubleshooting guide