Every time a platform suggests the next article, the next series or the product you "might be interested in", there is a recommender system behind it: an engine that predicts the affinity between a person and an item based on their behaviour and that of millions of users. Far from being magic, these are well-defined machine learning architectures. In this technical guide we break down the three classic families — collaborative filtering, content-based and hybrid — how the results are ordered (ranking), how they are served in real time and what the European regulatory framework demands in terms of transparency.
Collaborative filtering: the wisdom of the crowd
Collaborative filtering starts from a simple idea: someone who liked the same things as you will probably like something you do not yet know. It does not need to understand the content of the item, only the pattern of interactions. It has two approaches:
- Memory-based: computes direct similarities between users (user-based) or between items (item-based) using measures such as cosine similarity or Pearson correlation over the interaction matrix.
- Model-based: applies matrix factorisation (SVD, ALS) to decompose the sparse user-item matrix into low-dimensional latent factors that capture abstract tastes. This was the technique popularised by the Netflix Prize.
Its great limitation is the cold start problem: a new user or item, with no history, has nothing to be compared against. And it struggles with sparsity: in huge catalogues, each user interacts with a tiny fraction of the items.
A relevant technical distinction is the nature of the signal that feeds the model. Explicit feedback — star ratings, "likes", reviews — is precise but scarce: most users rate nothing. Implicit feedback — clicks, watch time, purchases, searches — is abundant but ambiguous: watching a video to the end suggests interest, but abandoning it after ten seconds does not always mean rejection. Modern systems rely above all on implicit signals, which makes it necessary to model the confidence of each interaction and to handle missing data carefully, since it is not equivalent to a negative rating.
Content-based recommendation
The content-based approach recommends items similar to those already liked, using their attributes: genre, author, tags, textual description or, in editorial content, the topics covered. A user profile is built as an aggregate of the features of what was consumed, and similarity with the catalogue is measured. Today these attributes are often represented as embeddings — dense vectors generated from text or metadata — that capture semantic similarity far better than the old bag-of-words approaches. Its advantage: it partially solves the cold start for new items (it is enough to know their attributes). Its weakness: it tends towards over-specialisation, locking the user into a bubble of what they already know.
Hybrid systems: the best of each technique
In production almost no one uses a pure approach. Hybrid systems combine collaborative and content-based methods to offset their mutual weaknesses, through strategies such as weighting several signals, switching according to data availability or cascading (one model refines the output of another). The dominant architecture in large platforms is two-stage:
- Candidate generation (retrieval): from a catalogue of millions, the field is narrowed to a few hundred plausible candidates using fast, approximate models (for example, nearest-neighbour search over embeddings).
- Ranking: a heavier model (deep neural networks, gradient boosting) precisely orders those candidates according to the probability of interaction, incorporating context, time of day, device and recent signals.
Ranking and metrics: what is a good recommendation?
Ordering well matters as much as getting it right. Ranking is evaluated with metrics that penalise placing the relevant items low down:
- Precision@K and Recall@K: hits among the top K results.
- NDCG (Normalized Discounted Cumulative Gain): rewards placing the most relevant items in the top positions.
- MAP (Mean Average Precision): averages precision across the list.
But offline metrics are not enough. The measure that decides things in production is the A/B test: two variants of the recommender serving different cohorts and comparing real business objectives (consumption time, conversion, retention). And here a key nuance comes in: optimising only immediate relevance degrades the experience. You have to balance it with diversity, novelty and serendipity to avoid the filter bubble and keep the catalogue alive.
Comparison table of approaches
| Approach | Needs | Strength | Weakness |
|---|---|---|---|
| Collaborative | Interaction history | Discovers non-obvious tastes | Cold start and sparsity |
| Content-based | Item attributes | Works with new items | Over-specialisation |
| Hybrid | Both | Balances weaknesses | Greater complexity |
Real-time personalisation: the architecture
"Real time" means the recommendation reflects what the user did seconds ago, not yesterday's profile. This demands a specific architecture: event ingestion via streaming (Kafka, Flink), a feature store that serves up-to-date features with low latency, approximate nearest-neighbour indexes (ANN, such as HNSW) to retrieve candidates in milliseconds, and an inference service that returns the ranking within the page's latency budget (typically tens of milliseconds). The engineering challenge is not the model, but serving it at scale without the user noticing any delay.
Filter bubble and feedback loops
A recommender system is not a neutral observer: it shapes the very behaviour it later measures. If the model only shows what it considers most aligned, the user can only interact with that, which reinforces the model's belief and progressively narrows what they see. This is the filter bubble, and its catalogue-scale version is the feedback loop: popular items are recommended more, gain more interactions, rise in the ranking and bury the long tail — the majority of the catalogue that never gets shown. This impoverishes the experience, wastes inventory and can have undesirable social effects when applied to informational content.
The countermeasures are both technical and a matter of product design: introducing deliberate exploration (multi-armed bandit strategies that reserve a fraction of the recommendations to test items with little exposure), penalising redundancy with re-ranking techniques that reward diversity, and incorporating novelty signals to favour discovery. The aim is not to give up relevance, but to balance it with the long-term health of the ecosystem, measuring not only today's click but tomorrow's retention and satisfaction.
Regulatory framework: mandatory transparency
Recommending is not legally neutral. The Digital Services Act (DSA, Regulation (EU) 2022/2065) obliges platforms to explain in their terms the main parameters of their recommender systems and to offer, on very large platforms, at least one option not based on profiling. The GDPR governs the processing of the behavioural data that feeds these systems: it requires a legal basis, minimisation and rights of access and objection. And the AI Act imposes transparency obligations when the user interacts with an AI system. Designing a recommender without considering these three frameworks is building on regulatory risk.
Common mistakes
- Optimising solely for the immediate click and impoverishing the medium-term experience with filter bubbles.
- Relying only on offline metrics and skipping the A/B test, which is where the real impact is revealed.
- Ignoring the cold start and offering poor experiences to new users, precisely when retaining them matters most.
- Creating feedback loops: the model shows only what is popular, which reinforces its popularity and buries the long-tail catalogue.
- Processing behavioural data without a legal basis or transparency, breaching the GDPR and the DSA.
Frequently asked questions
What is the cold start problem?
It is the difficulty of recommending to a new user or item that lacks an interaction history. It is mitigated with content-based approaches, recommendations by initial popularity or by asking for preferences at sign-up.
Collaborative filtering or content-based?
It depends on the available data. Collaborative filtering shines with a lot of interaction history; content-based methods work when you have good item attributes or fast-changing catalogues. In practice, production systems combine both.
Why is diversity important in recommendations?
Because a recommender that only shows the most aligned items locks the user into a bubble, exhausts their interest and leaves most of the catalogue without visibility. Introducing diversity and novelty improves satisfaction and long-term discovery.
What legal obligations does a recommender system have in Europe?
It must respect the GDPR in the processing of behavioural data, comply with the Digital Services Act's transparency obligations regarding recommendation parameters and meet the AI Act's transparency requirements. Opacity is no longer legally sustainable.
Conclusion
A good recommender system is not the one that predicts the next click most accurately, but the one that sustains a valuable, lasting relationship between the person and the catalogue: one that gets it right, yes, but also surprises, diversifies and respects. The technique is mature — from collaborative filtering to two-stage hybrid architectures served in milliseconds — but the challenge has shifted towards balance: relevance versus diversity, popularity versus the long tail, and personalisation versus the transparency requirements of the DSA, the GDPR and the AI Act. At Summum Artificial Intelligence we design recommendation engines that combine technical precision, metrics validated with A/B tests and compliance by design, because recommending well in 2026 is as much a problem of engineering as of responsibility.