Why 95% accuracy isn't enough in-store

Every recognition vendor promises accuracy near 100%. Then the system meets a real store, and the result rarely matches the demo. A model that’s flawless in training doesn’t always deliver in practice. Knowing why is the difference between buying a demo and buying a system that works.

Why high-scoring models still disappoint

Two things explain most of the gap.

Different conditions. Training runs on clean images under good light, and models really do score extremely well on that data. Those polished datasets are what vendors use to demo their systems. They are not photos from an average neighborhood store, so the score tells you little about your shelves.

Curated examples. Image quality is only part of it. A demo shows edge cases the system has already learned. Your store throws new ones at it: packaging variants, layouts, lighting it has never seen. It makes mistakes until it learns those. So judge a system by how it performs in your environment. A demo proves little.

Why the system makes mistakes

Even a strong system meets the unpredictability of a real store. Four causes account for most errors.

Too many similar products. The system works from pixels alone. Two yogurt flavors or two bottle sizes are just similar color patterns to it, so it confuses them. Without context about what usually sits in that spot, it guesses.

Occlusion. Shelves are in constant motion: a shopper blocks a product, one package hides another, a new box appears. The camera doesn’t see the hidden item, so the system honestly logs it as out of stock. A person understands the product is merely blocked; the system takes the scene literally.

No tracking between frames. In reality, products don’t blink in and out of existence every second. If the system never compares the current frame against earlier ones, its reports start to flicker: present, then gone, then swapped for something else. Nothing on the shelf actually changed.

Outdated context. Errors creep in even on a familiar layout. If the system ignores how current the shelf labeling is, or where a product was last seen, it slips. Mistakes then show up in spots that usually work fine.

Errors are inevitable. They shrink with tracking, context about the shelf, regular data updates, and configuration tuned to your store.

Read the metrics that matter

Here’s the trap. A system can show 95% accuracy on paper while operators still correct every second item. The technical number looks impressive, and the business outcome is still poor. Accuracy alone can hide a broken deployment.

The metrics that actually describe business performance:

• Manual correction rate. Out of 100 cases, how often does an operator have to fix the result? This is the real labor cost of the system.
• Missed out-of-stock rate. How often does it fail to catch an empty shelf? This is the lost-sales cost.
• False alert rate. How many alerts turn out to be noise? Too many, and people start ignoring all of them. This is the cost in staff trust.

A model optimized for headline accuracy can score well and still fail all three. We report these numbers during a shelf-monitoring pilot. They are the ones tied to money, and to whether your team keeps using the system.

Keeping quality up

Quality isn’t a launch-day number; it’s a process:

• Review edge cases and disputed situations regularly.
• Collect operator feedback and retrain on it.
• Don’t paper over imperfect metrics. Naming them honestly is the only way to improve them.

The same theme runs through how we triage recognition errors. A system you can trust is one that’s honest about where it’s wrong.