ECOBOT, a recycling robot that sees

Manual waste sorting is slow, unsafe, and capped on throughput. The purity of each fraction drives its price. ECOBOT automates the pick. The system takes in a bag of mixed trash, opens it, and spreads the contents across a rotating table. A camera mounted above streams the surface to the onboard system over ROS, and the arm waits for coordinates before it moves. Our scope was the vision module: recognize and classify each item in real time, then turn that into precise commands for the arm.

The training data came from the lab itself: frame grabs from video of waste moving on the prototype, plus stills captured through ROS, labeled in Label Studio. The detector was trained and measured on four material classes, plastic, paper, metal, and glass. Two things about the data shaped the whole project. The classes were uneven, with paper and plastic common and glass and metal scarce, and the camera height over the table varied, yet about 90% of the labeled photos were shot with the camera close and only 10% from far away.

We first tried to skip the data problem with transfer learning from TACO, a public dataset of trash photographed in the wild. We mapped 34 of its categories onto our four classes and trained a CenterNet detector with a ResNet50 FPN backbone at 512x512. It did not generalize well to the prototype: precision and recall came out at 0.67 and 0.43. It missed small pieces, objects that blended into sand or water, and items piled close together. It also leaned on the dominant plastic class, tagging paper as plastic and drawing extra plastic boxes around things like a stray stick.

So we ran a proper bake-off on lab data instead. We compared Faster R-CNN, YOLOv3, SSD, CenterNet, ATSS, and FCOS, all started from COCO-pretrained weights, with both full fine-tuning and frozen backbones. ATSS led the single-stage models and CenterNet and FCOS were close behind. We chose CenterNet with a ResNet50 FPN backbone, because it had to run on a Jetson Nano at the table, and that constraint matters as much as the score.

The detector had to give the right boxes and classes no matter how high the camera sat above the table. That was a problem, because the labeled set was about 90% close-camera shots. Trained as-is, the model fell apart on far shots: on a held-out far-camera angle, recall dropped to 0.14.

Rather than shoot and label thousands of new far photos, we synthesized them. Scale augmentations, random scale crop and pad to square and random image scale at ratios from 0.3 to 0.8, shrank objects in the frame so they looked like they were seen from further away. After retraining with these, far-camera recall climbed from 0.14 to 0.64, and the model held up across heights without new data collection.

The final CenterNet ResNet50 FPN detector runs on the Jetson Nano at the table, on-device with no cloud call, at around 4 FPS. Overall precision and recall on the test set are 0.81 and 0.76. It holds across camera height: 0.82 / 0.78 close and 0.75 / 0.64 far. By class, precision and recall are 0.88 / 0.73 for plastic, 0.80 / 0.83 for paper, 0.71 / 0.60 for metal, and 0.60 / 0.89 for glass.

The gaps are known. Transparent plastic and light paper that blend into the background are still the hardest, more so on far shots, and the model sometimes draws two boxes on one object, which a manual non-maximum-suppression step is meant to clean up. Glass and metal would improve with more examples in the training set.

Recognizing the trash is only the first of three steps. The hard engineering is turning a box on a 2D frame into a move the hardware can make.

At time t, the camera snapshot goes through ROS to the detector, which returns boxes and classes. For each object the system computes the angle between it and a fixed zero angle, plus the distance from the object to the arm's parking position in millimeters. Those numbers go over serial to both the table and the arm: the table rotates by the angle, the arm travels the distance, grips the item, and lifts it off the table. The cycle runs without a person in the loop.

The recognition engine that runs the arm also works as a lower-risk first step for a line that is not ready for robotics: stream monitoring that measures the waste flow on the belt without touching it. The same approach carries over to other settings where a camera has to watch and sort a large number of objects in real time.