Real-time CV for your robots, production-grade, ROS-integrated, edge-deployed

The Problem

Robotics teams that need vision keep hitting one recurring problem: off-the-shelf computer-vision libraries work in clean demos, then fail at the edge cases that matter. Take a pick-and-place robot that grasps 95% of items cleanly. It drops every 20th one because of a glare or an unfamiliar shape. That is not a 95%-useful robot. It is a constant operator-attention problem. The vision module that works in the lab is the one that stops the line in production.

What teams actually need is three things together. First, a vision module that handles real-world conditions: lighting, occlusion, partial views. Second, an integration with ROS that does not fight the rest of the stack. Third, a continuous-improvement loop, so the system gets better with deployment. Most systems get shipped and forgotten.

Most computer-vision vendors give you the first piece. Robotics integrators give you the second. Almost nobody runs the third. That is why so many robot deployments look great at handover and then degrade over the first year of production.

What the Solution Does

We develop computer vision modules that integrate cleanly with ROS-based (or other) robotic control systems, with built-in operator fallback and a retraining pipeline.

1. Vision module, object detection, classification, segmentation tuned for the robot’s specific task.
2. ROS integration, vision data flows through sensor_msgs / vision_msgs, synchronized with the rest of the robot’s data streams.
3. Edge deployment, models optimized for Jetson Orin Nano / NVIDIA Xavier / Intel Movidius. TensorRT and ONNX quantization for sub-30ms inference.
4. Human-in-the-loop fallback, when confidence is low, the system asks an operator rather than guess. Operator decisions get logged.
5. Continuous retraining, operator corrections become training data. Datapipe handles incremental retraining without rebuilding the pipeline.

Where It Fits

This makes sense if you…

• Are building or operating ROS-based industrial / logistics robots
• Need real-time vision at the edge, around 30 FPS, beyond batch image processing
• Have safety or quality requirements that prohibit “guess and hope”
• Can support an operator workflow for the cases the model isn’t sure about
• Will operate the robot long enough that ongoing model improvement matters

This is probably not the right time if you…

• Need only one-off image classification with no real-time perception (cheaper off-the-shelf models will do)
• Are running a non-ROS robotic stack with no plan to migrate (we work primarily in ROS / ROS2)
• Cannot put operators in the loop. Fully autonomous deployment with zero human oversight is a different risk profile
• Have no edge hardware budget. Jetson-class is required for the real-time performance

Business Value

Real-time, edge-deployable. 30 FPS at 25ms frame latency on Jetson Xavier, that’s the standard target for robotic perception pipelines we ship. Models are optimized via TensorRT / ONNX / quantization to fit the hardware envelope without sacrificing accuracy.

Production stability through human-in-the-loop. The most common failure mode of robot vision in production is the silent confidence collapse: the model encounters an edge case, returns a wrong answer with high confidence, the robot acts on it, something breaks. We architect around this by deliberately surfacing uncertainty: when confidence is below threshold, the system routes the decision to an operator. Operator decisions feed back into retraining.

Continuous improvement after handover. Most robot deployments degrade over their first year as the production environment drifts from the training conditions. Our deployments improve over their first year, because accumulated operator corrections feed Datapipe retraining cycles.

ROS-native integration. We don’t bolt a vision API on top of ROS via a JSON shim. The vision modules ship as ROS nodes, with proper synchronization on sensor_msgs::Image, output via vision_msgs::Detection2DArray, and full compatibility with ROS 1 and ROS 2.

How It Works

The pipeline below is what we shipped for the ECOBOT recycling robot (detailed in automated-waste-sorting) and what we adapt for other industrial robotic deployments.

1. Camera and capture

Industrial cameras (we work with ZED2i and similar, RGB and depth in one device) feed video into ROS via standard drivers. For systems that need 3D environmental perception, depth and color are time-synchronized.

2. Detection / classification / segmentation

YOLOv8 / YOLOv11 for detection. ResNet50 for classification when the detector outputs need refinement. Custom segmentation models for shape analysis, robotic grasping, and volume estimation. For 3D-aware tasks, we combine 2D detection with depth fusion and ArUco-marker calibration for absolute coordinate mapping.

3. ROS integration

CV outputs become ROS messages. Synchronized with motion commands, gripper feedback, and other sensors. Compatible with ROS 1 (legacy systems) and ROS 2 (modern stack).

4. Edge optimization

Models compiled to TensorRT (Jetson family) or ONNX (broader compatibility). Quantization where it doesn’t degrade accuracy. Operator fusion to minimize latency. Output: 25ms frame processing time on Jetson Xavier, leaving budget for the rest of the robotic control loop.

5. Human-in-the-loop fallback

When detection / classification confidence falls below a configurable threshold, the system pauses, surfaces the image to an operator interface, and waits for a decision. The operator confirms, corrects, or escalates. Operator response time is the rate limit on these cases; for high-throughput operations we recommend setting thresholds conservatively to keep operator interrupt rate low.

6. Continuous retraining

Every operator correction gets logged with the input image. Datapipe handles the data pipeline: deduplication, annotation routing, retraining triggers, model registry. Updated models deploy back to the edge automatically on a schedule (typically weekly).

Stack

YOLO family (v8-v11) for detection. ResNet50 for classification. ZED2i and industrial RGBD cameras. ROS 1 and ROS 2 (OpenCV and PCL integration). TensorRT and ONNX for edge deployment. Datapipe for the retraining pipeline. Label Studio for annotation. Jetson Orin Nano, Xavier, or Intel Movidius for inference. Cloud or on-prem for training and the operator dashboard.

What You Need to Make This Work

Data. Example images per object class, typically 100-500 to start, more for harder classes. We collect during the test phase if you don’t have a corpus.

Integrations. ROS environment (we work in ROS 1 and ROS 2). Camera hardware (we spec). Edge compute (Jetson family preferred). Operator interface, we ship a default web-based one; alternatively integrate with whatever your operators already use.

Hardware. Industrial cameras and edge compute (Jetson Orin Nano, Xavier, or comparable). The robot itself, of course, but that’s your existing capex.

Team. A robotics engineer who owns the ROS integration on your side (~40-60 hours during pilot). An operator pool for the human-in-the-loop step. A QA / ML lead who’ll track accuracy metrics and approve retraining rollouts.

Implementation Roadmap

1. Test (4-8 weeks)

Pick one task (object class set, one workflow). Collect data. Train baseline model. Integrate into ROS. Measure end-to-end performance, accuracy, latency, operator-interrupt rate. Output: a written report with measured numbers, a working ROS node, recommendations for hardware and operator workflow.

2. Pilot (2-4 months)

Production deployment to one robotic cell. Tune confidence thresholds. Wire up the operator workflow. Set up retraining pipeline with Datapipe. Output: working production deployment, dashboards for accuracy and operator-interrupt rate, go/no-go on multi-cell rollout.

3. Production (4-9 months)

Scale to additional cells / additional tasks. Quarterly retraining review. Continuous accuracy improvement via the retraining loop. By the end your robotics team owns day-to-day; we stay on for retraining oversight and new task onboarding.

Keep in Mind

The limits worth knowing:

• ROS integration is real work. ROS message synchronization, lifecycle management, error propagation across nodes: this is engineering, well beyond plugging in a model. A pilot runs 8 weeks rather than 2 because of this.
• Operator interrupt rate matters more than peak accuracy. A model with 99% peak accuracy that interrupts the operator 1% of the time can be worse than a model with 95% peak accuracy that interrupts 0.1% of the time. We tune thresholds with you, against your real workload.
• 3D perception requires camera calibration. ArUco markers, depth-camera calibration, time synchronization across multiple sensors: all real maintenance overhead. Build it into your operations plan.
• Edge hardware has memory budgets. Multi-task models on Jetson Nano hit memory walls fast. Plan for the Xavier or Orin AGX class if you need multiple concurrent CV tasks per robot.
• Continuous retraining is not autonomous improvement. A retraining pipeline still needs someone to approve new models before they deploy. We typically gate deployment on automated metric tests with human approval.

FAQ

ROS 1 or ROS 2?

We support both. ROS 1 is still widespread in deployed industrial robotics; ROS 2 is the path forward for new builds. We can advise on migration if that’s part of your roadmap.

Which edge hardware do you recommend?

Depends on task complexity and budget. Jetson Orin Nano is the sweet spot for single-task perception; Xavier or Orin AGX when you need multiple concurrent vision tasks or large models. We size during the test phase.

Can you integrate with non-ROS systems?

Yes, but with more integration overhead. ROS is our home; non-ROS systems get adapter layers but it’s not the fastest path.

How do you handle the safety case?

The human-in-the-loop fallback is the core of the safety story: the system surfaces uncertainty rather than guess. For hard safety-critical applications (collaborative robots near humans, etc.) we additionally recommend safety-rated hardware interlocks that don’t depend on the CV layer.

Can the system handle entirely new objects without retraining?

It depends on the architecture. A frozen classifier cannot. A two-stage detection-and-classification pipeline can flag “unknown class” and route to an operator rather than guess. We typically build the two-stage version for production deployments.

Ready to Discuss?

If you’re building or operating industrial robotics that need real-time vision, and the off-the-shelf CV options have disappointed in production, this is the right conversation. We’ll talk through your robotic stack and your task definition. We’ll walk your operator workflow. Then we’ll tell you what a pilot would cost and what to expect.

Discuss your project