Training Vision Language Action (VLA) models and other modern computer vision systems requires large, high-quality annotated datasets. Meta's Segment Anything Model 3 (SAM 3) can segment objects in images with remarkable speed and accuracy — but the original annotation tool needed significant work to be usable in production workflows.
I set out to refactor the existing SAM 3 Annotation Tool into a professional-grade labeling platform with a stable UI, reliable export pipelines, and optimized performance for day-to-day dataset creation.
Starting Point
The original SAM3_Annotation_Tool by software-ai-life provided the core SAM 3 integration — click-to-segment, mask generation, and basic annotation saving. It was a solid proof of concept, but had limitations for production use:
- UI library compatibility issues causing instability during long annotation sessions
- No standardized export format for downstream training pipelines
- Limited error handling when SAM 3 inference failed on edge cases
Key Enhancements
1. Complete UI Overhaul
I replaced the original UI framework with a more robust, production-grade setup. The new interface provides clear visual feedback during segmentation, intuitive mask editing controls, and a layout optimized for annotators spending hours on dataset work.
2. COCO-Format Export
Most ML training pipelines expect annotations in COCO format — JSON with image metadata, bounding boxes, segmentation polygons, and category labels. I built a reliable export pipeline that converts SAM 3 masks into COCO-compatible annotations, ready for direct ingestion into training frameworks like Detectron2, MMDetection, or custom PyTorch loaders.
3. Performance Optimization
Annotation speed directly impacts dataset throughput. Optimizations included:
- Lazy loading of images in large datasets to reduce memory pressure
- Cached model weights to avoid reload between sessions
- Batch export operations for multi-image annotation sessions
Workflow for VLA Dataset Creation
Vision Language Action models need pixel-precise segmentation paired with semantic labels. The typical workflow with this tool looks like:
- Load a batch of images into the annotation session
- Use SAM 3's click-to-segment to generate initial masks
- Refine masks with manual adjustments where SAM 3 under-segments
- Assign category labels to each segmented region
- Export the session as COCO JSON for training pipeline ingestion
Licensing & Compliance
This project is a professional refactor licensed under Apache 2.0, incorporating elements from the original MIT-licensed work. Both licenses are maintained with proper attribution in the repository's NOTICE file — important when building on open-source foundations for commercial or research use.
Lessons Learned
- Foundation models need production wrappers. SAM 3's segmentation quality is excellent, but raw model output is not a product — the UI, export, and error handling layer is what makes it usable.
- Export format is a first-class feature. Annotators will abandon a tool if they cannot get data out in the format their training pipeline expects.
- Refactoring beats rewriting. Building on the original tool's SAM 3 integration saved weeks of model wiring work.
Conclusion
The SAM-3 Professional Annotation Tool demonstrates that taking a research-grade model and wrapping it in production tooling can dramatically accelerate dataset creation for computer vision projects. Check out the source on GitHub.