Computer Vision for Assembly Line Quality Control

The problem

Foreign objects on an assembly line are rare, unpredictable and expensive. Rare, because the process is good. Unpredictable, because by definition you cannot enumerate what should not be there. Expensive, because a part that gets through carries the defect downstream, where finding it costs several times more.

That combination rules out the obvious approach. A supervised classifier needs examples of every class it will be asked to recognise, and here the interesting class is “anything that should not be in this picture”.

Data

Good parts are abundant, so I built the dataset around normality rather than around defects.

• Images captured directly on the line, in the real lighting and vibration conditions, not in a lab
• Fixed fixture and illumination so the part presents itself the same way every cycle
• Natural process variation deliberately kept in: surface finish, residue, tool wear, shift-to-shift differences
• A small, carefully curated set of known anomalies held back purely for validation — never for training

The single largest accuracy gain in this project came from the imaging setup, not from the model. Once acquisition was repeatable, everything downstream got easier and stayed stable.

Approach

I used the semi-supervised anomaly detection family implemented in Anomalib on top of PyTorch: a pretrained backbone extracts local features from normal parts, those features describe what “normal” looks like, and at inference each region is scored by how far it sits from that description.

   image  →  backbone  →  patch features
                               ↓
               distance to the normal feature distribution
                               ↓
            anomaly heatmap  +  scalar anomaly score

Two decisions mattered more than the architecture choice:

1. The operating point came from cost, not from accuracy. A missed foreign object and a false alarm have very different prices, and production had a clear opinion on both. The threshold was chosen on the precision–recall curve accordingly, and it is a parameter the plant can revisit.
2. The heatmap is part of the product. Operators do not trust a number. They trust a picture that shows them where the system is looking. That feedback loop is what turned the model from “the AI thing” into a tool people actually use.

Deployment

• Packaged as a containerised inference service, integrated with the line via API, running within the cycle time
• Every decision persisted with image, score, heatmap and model version, so any call can be reconstructed later
• Input drift monitored on image statistics — a shifted camera or a dirty lens is caught before the model degrades
• Explicit fallback path: if the service is unavailable, parts route to manual inspection instead of silently passing

What I would do differently

I would invest in the labelling and review interface earlier. The model was ready well before the process around it was, and the bottleneck in industrial ML is almost never the model — it is who looks at the output, how quickly, and what they are allowed to do about it.