Barcodie

The problem

Bakeries sell dozens of items that have no barcode. Cashiers memorise the menu, staff turns over fast, and every new product means retraining someone. Existing vision pilots stall at the same point: adding a new product needs an engineer and a training run.

The idea: no training, just nearest neighbours

Barcodie never trains a classifier. Each product is enrolled with three to five photos, turned into DINOv2 embeddings and stored in Qdrant. Recognition is a cosine nearest-neighbour search with a calibrated threshold. A new product takes seconds.

1. Capture — one frame of the tray.
2. Detect — YOLO-World (open vocabulary) boxes every object, so a mixed tray is read in one shot.
3. Match — each crop is embedded and matched against the shop’s catalog.
4. Confirm — a two-threshold trust rule: above match goes straight into the basket, between reject and match is a one-tap confirmation, below reject never touches the basket.
5. Correct by voice — “add two simit” goes through Google STT v2 and a small intent parser.
6. Pay — via Moka.

It never charges a wrong price. It either asks or refuses.

Built to survive a live demo

Components talk only over HTTP against one OpenAPI contract: an Expo thin client, a .NET API that owns the catalog and recalculates every total, and a Python AI service. The demo had three fallback levels — fully live, canned photo + live AI, fully offline mock — switchable with a flag and pixel-identical in the UI.

Benchmarks, not adjectives

After the hackathon I benchmarked six vision models on the same data with the service’s own decision rule. From an overhead counter camera, with three photos per product, first-guess accuracy is 92.5–100%. The service auto-approves 38.8% of predictions without asking the cashier, and 99.2% of those are correct. Embedding takes 47 ms per frame on CPU.

What it became

After the hackathon the direction changed: POS vendors don’t want another POS, they want recognition inside theirs. Barcodie is now a product-recognition API: a fixed camera at the till sends a frame and gets back the POS’s own product code; price stays in the POS, weight comes from the scale.