Computer vision is the discipline that teaches machines to extract meaningful information from images and video. What for a person is an instant glance — recognising a number plate, reading a label, counting parts on a conveyor belt — is for a computer a problem of processing numerical matrices. In this technical guide we walk through how object detection, OCR and facial recognition work, which architectures underpin them and, above all, what legal and ethical framework governs their use in Europe.
From pixels to representations: how a machine sees
A digital image is a tensor: width × height × channels (3 for RGB). The great leap of modern vision was convolutional neural networks (CNNs), which apply learned filters to detect edges, textures and, in deeper layers, full shapes and objects. The historic milestone was AlexNet (2012), which dramatically cut the error rate in the ImageNet challenge. Since then, architectures such as ResNet introduced residual connections to train very deep networks, and more recently Vision Transformers (ViT) brought the attention mechanism — splitting the image into patches — over to vision, matching or surpassing CNNs on many benchmarks.
A technique that is decisive in practice is transfer learning: you start from a model pre-trained on millions of images and fine-tune it with a small set from the specific domain (weld defects, dermatological lesions, products in a catalogue). This cuts the need for labelled data from months to days.
Even before the neural network, performance depends on preprocessing: normalising pixel values, resizing, correcting illumination and, above all, data augmentation. The latter generates synthetic variations of the training set — rotations, crops, brightness changes, controlled noise — so that the model learns robust features rather than memorising specific images. Well-designed augmentation is often the difference between a model that generalises and one that collapses at the first real condition it never saw in the lab.
Object detection: locate and classify
Whereas classification answers "what is in the image", object detection answers "what is there and where", drawing bounding boxes. Two families dominate the field:
- Single-stage (YOLO, SSD): predict boxes and classes in a single pass. Fast, ideal for real time (surveillance, driver assistance, robotics).
- Two-stage (Faster R-CNN): first propose candidate regions and then classify them. More accurate, somewhat slower.
The benchmark metric is mAP (mean Average Precision), which combines precision and recall at various overlap thresholds (IoU, Intersection over Union) between the predicted box and the ground truth. For tasks that demand exact outlines — not just boxes — you turn to semantic segmentation (classifying every pixel) or instance segmentation (separating objects of the same type), with models such as Mask R-CNN or SAM.
OCR: turning images of text into data
Optical character recognition (OCR) transforms printed or handwritten text into editable strings. A modern pipeline chains together: detection of text regions, character recognition using recurrent networks or transformers with a CTC loss function, and linguistic post-processing that corrects errors with language models. Its business applications are immediate: digitising invoices and delivery notes, automatic number-plate recognition (ANPR), extracting data from identity documents or automating data entry into an ERP. Quality depends crudely on the resolution, contrast and perspective correction of the input document.
Facial recognition: power and a strict legal framework
Facial recognition encodes a face into a numerical vector (embedding) and measures distances to verify identity (1:1) or to identify someone within a database (1:N). Its technical accuracy is high, but its legal treatment is the most sensitive in all of computer vision. The facial features used to uniquely identify a person are biometric data and therefore a special category under Article 9 of the GDPR: their processing is prohibited except in narrowly defined cases (explicit consent, substantial public interest with a legal basis, and so on).
In addition, the European Artificial Intelligence Regulation (AI Act) classifies uses by risk. Real-time remote biometric identification in public spaces by authorities is, with very limited exceptions, prohibited, and biometric categorisation based on sensitive data is also restricted. Any biometric identification system is considered high-risk and requires a conformity assessment, risk management, registration and human oversight. Deploying facial recognition without a prior compliance analysis is not an option.
Edge versus cloud deployment
Where the model runs shapes the entire architecture. Cloud processing offers practically unlimited compute power and makes it easy to update models, but it introduces network latency, a recurring transmission cost and a privacy issue: sending images — which may contain faces or number plates — to an external server has GDPR implications. Edge processing, by contrast, runs inference on the camera itself or on a local device, which cuts latency to milliseconds, works without a connection and keeps the images inside the perimeter, mitigating privacy risks.
The price of the edge is resource constraint: an industrial camera or an embedded module does not have a data centre's GPU. That is why model optimisation techniques are used: quantisation reduces the numerical precision of the weights (from 32 to 8 bits), speeding up inference with minimal loss of accuracy; pruning removes low-relevance connections; and knowledge distillation trains a small model to imitate a large one. Formats such as ONNX and engines such as TensorRT make it easier to bring a trained model to specific inference hardware. The cloud-versus-edge decision is not ideological: it is taken by measuring target latency, cost, available connectivity and the sensitivity of the data captured.
Comparison table of vision tasks
| Task | Output | Typical models | Business use |
|---|---|---|---|
| Classification | Global label | ResNet, ViT | Pass/fail quality control |
| Detection | Boxes + class | YOLO, Faster R-CNN | Counting, surveillance, retail |
| Segmentation | Per-pixel mask | Mask R-CNN, SAM | Medical imaging, agriculture |
| OCR | Text | CRNN, transformers | Document digitisation |
| Facial recognition | Identity | Embedding networks | Secure access (high legal risk) |
Implementation: from prototype to production
- Define the task precisely: classify, detect, segment or read? The choice changes the entire pipeline.
- Build a representative dataset: include real conditions of illumination, angles and difficult cases; label with consistent criteria.
- Start from transfer learning and retrain on your domain before considering training from scratch.
- Measure with the right metric (mAP, IoU, F1) on a test set the model has never seen.
- Optimise for the target: quantisation and pruning if you are going to deploy at the edge (cameras, mobiles) with limited resources.
- Monitor for drift: performance drops when conditions change; retrain periodically.
Common mistakes
- Training with "lab" images and deploying in real conditions that are radically different.
- Trusting global accuracy while ignoring bias across subgroups (a detector that fails more with certain skin tones or angles).
- Processing facial data without a GDPR legal basis or an AI Act assessment.
- Forgetting capture quality: the best network does not compensate for a poorly focused or poorly lit camera.
- Not versioning datasets or models, making results impossible to reproduce.
Frequently asked questions
What is the difference between classification and object detection?
Classification assigns a label to the whole image; detection locates each object with a bounding box and assigns it a class. Detection is more costly but necessary when there are several objects or their position matters.
Is it legal to use facial recognition in my company?
Only under strict conditions. Facial data is a special category under the GDPR, and biometric identification systems are high-risk under the AI Act. A legal basis and an impact assessment are required and, in many public uses, it is directly prohibited. Always seek advice before deploying.
How many images do I need to train a model?
With transfer learning, a few hundred to a few thousand well-labelled examples per class are usually enough for a solid prototype. Quality and representativeness matter more than raw quantity.
What is segmentation and when do I need it?
It is pixel-by-pixel classification. It is used when a box is not enough and you need the exact outline: measuring the surface of a lesion, separating crops in an aerial image or isolating a product from the background.
Conclusion
Computer vision has moved beyond being an experimental field to become an operational layer that inspects parts, digitises documents and automates quality control with superhuman precision on well-defined tasks. But its technical maturity comes with a growing responsibility: the easier it is to identify and track people from an image, the stricter the framework that regulates it. The success of a vision project is measured not only by its mAP, but by its robustness under real conditions, the absence of harmful bias and scrupulous compliance with the GDPR and the AI Act. At Summum Artificial Intelligence we design vision systems that perform in production and are born compliant from the very first prototype, because a model the law does not allow you to deploy is not a model: it is a risk.