Ethics in artificial intelligence has stopped being an academic debate and become a legal and operational requirement. When a model decides who is granted a loan, which CV reaches the interview stage or which patient is prioritised, its errors are not anecdotal: they reproduce and amplify inequalities on an industrial scale. This guide addresses the three pillars that hold up responsible AI—fairness against bias, data privacy and algorithmic transparency—together with the regulatory framework applicable in Europe and the concrete steps to audit a system before putting it into production.
Where algorithmic bias comes from
Bias is not a flaw in the code but almost always a reflection of the data. Several sources can be distinguished. Historical bias appears when the training data encodes past discriminatory human decisions: if a bank historically rejected more applications from a particular postcode, the model will learn that pattern even if the postcode is not included as a variable, inferring it from proxies such as income level. Representation bias arises when a group is underrepresented in the dataset, so the model performs worse for it. And measurement bias occurs when the label we predict is an imperfect surrogate for what really matters.
The paradigmatic case is that of the criminal recidivism models studied in the United States, where the algorithm assigned higher risk scores to defendants from minorities with the same actual recidivism rate. The problem was not the developer's intent, but that the system optimised an overall accuracy metric that masked the unequal treatment between subgroups.
Fairness metrics: there is no single correct definition
Measuring fairness forces you to choose between definitions that are mathematically incompatible with one another. Demographic parity requires the rate of positive outcomes to be equal across groups. Equality of opportunity requires an equal true-positive rate. Calibration requires that, for a given score, the actual probability be equal across groups. A well-known impossibility theorem proves that, except in trivial cases, you cannot satisfy all three at once.
The practical consequence is that the choice of metric is an ethical and business decision, not a technical one, and it must be documented and justified. In a recruitment process, for example, equality of opportunity is usually prioritised so as not to penalise valid candidates from protected groups. Tools such as AIF360 or Fairlearn make it possible to measure these metrics by subgroup and to apply mitigations in pre-processing, during training or in post-processing.
The legal framework: AI Act and GDPR
Since 2024, Europe has had the Artificial Intelligence Regulation (AI Act, Regulation (EU) 2024/1689), the world's first horizontal framework. It classifies systems by risk: prohibited practices (social scoring, subliminal manipulation), high-risk systems (recruitment, credit scoring, biometrics, critical infrastructure) subject to strict obligations, and limited-risk systems with transparency duties. High-risk systems must have documented risk management, data governance, activity logs, effective human oversight and a conformity assessment. It is worth consulting the text of the AI Act for the phased application timetable.
In parallel, the GDPR already governs the processing of personal data. Its Article 22 grants the data subject the right not to be subject to decisions based solely on automated processing that produce legal effects, and it imposes a duty to provide information about the logic applied. The Spanish Data Protection Agency (AEPD) has published specific guidance on auditing processing operations that incorporate AI, which is worth keeping as a reference before any deployment.
Privacy: minimisation and privacy-preserving techniques
The conflict between AI's need for data and the right to privacy is resolved with concrete techniques, not good intentions. The GDPR's data minimisation obliges you to use only what is necessary for the purpose. Beyond that, differential privacy adds calibrated noise to queries or to training so that the presence or absence of a specific individual is not distinguishable, offering a measurable mathematical guarantee through a privacy-budget parameter. Federated learning trains the model without centralising the data: each node trains locally and only shares weight updates. And anonymisation—not mere pseudonymisation—takes the data out of the scope of the GDPR, although demonstrating irreversible anonymisation is harder than usually assumed because of the risk of re-identification.
Transparency and explainability
An opaque model that gets it right is not acceptable when it affects rights. Explainability (XAI) aims to make decisions comprehensible. Post-hoc techniques such as SHAP and LIME attribute the decision to the contribution of each input variable, which makes it possible to tell an affected person why something was denied to them. But there are nuances: a post-hoc explanation approximates the model's behaviour; it is not its exact internal logic, and it can induce false confidence.
That is why structured documentation is gaining ground: model cards describe the model's purpose, data, per-subgroup metrics and limitations; datasheets for datasets document how each dataset was collected and processed. The ISO/IEC 42001 standard, the first management-system standard for AI, formalises these practices in an auditable and certifiable framework.
Governance of the model life cycle
Algorithmic accountability is not resolved at the moment of training, but throughout the model's entire life cycle, from conception to retirement. Governance requires clear roles: who owns the model, who validates data quality, who approves the move to production and who answers to an affected person who lodges a complaint. Without those roles, accountability dilutes and, when harm occurs, there is no one to point to.
A critical element is drift monitoring. A deployed model degrades its performance over time because reality changes: the distribution of the input data shifts (data drift) or the relationship between inputs and outputs alters (concept drift). A scoring model trained before an economic crisis can become unfair or imprecise afterwards. Continuous monitoring of fairness metrics by subgroup, not just overall accuracy, is what detects that bias has reappeared. ISO/IEC 42001 formalises this oversight as part of the management system, requiring records, periodic reviews and continual improvement.
Full traceability—versioning of data, code, hyperparameters and model weights—makes it possible to reproduce any decision months later. If a regulator or a court questions why the system denied something on a specific date, the organisation must be able to reconstruct exactly which version of the model decided, with what data and under what thresholds. Without that traceability, defending against the GDPR's duty of explanation is unfeasible.
Meaningful human oversight
The AI Act requires human oversight in high-risk systems, but the rule insists that it be effective, not nominal. There is a well-documented phenomenon, automation bias, whereby people tend to accept uncritically whatever a machine proposes, especially under time pressure. Human oversight that amounts to clicking "approve" on every recommendation the system makes is not oversight: it is rubber-stamping.
For human intervention to add value, the supervisor needs three things: enough information to understand the recommendation (hence the explainability), real capability and authority to override it, and time to do so. Designing the process so that borderline or high-impact cases are escalated to reinforced human review, while routine cases flow through, balances efficiency and control. Oversight must also be able to halt the entire system if it detects anomalous behaviour, a safeguard that the AI Act explicitly includes.
Table: fairness criteria and when to apply them
| Criterion | What it equalises | Typical use case | Tension |
|---|---|---|---|
| Demographic parity | Positive rate per group | Advertising, reach | Ignores actual merit |
| Equality of opportunity | True-positive rate | Recruitment, admissions | Incompatible with calibration |
| Calibration | Reliability of the score | Risk scoring | May yield disparate rates |
| Predictive parity | Positive predictive value | Clinical diagnosis | Clashes with equality of opportunity |
Common mistakes when building responsible AI
The first is treating fairness as a box to tick at the end, rather than a requirement from the design of the data onwards. The second is removing the sensitive variable (gender, ethnicity) in the belief that this eliminates bias, ignoring that the model reconstructs it from proxies. The third is measuring only overall accuracy, which hides unequal performance across subgroups. The fourth is entrusting human oversight to a person who merely approves whatever the system proposes, with neither real capability nor time to review. And the fifth is versioning neither data nor models, which makes it impossible to reproduce a contested decision months later.
Frequently asked questions
Does removing the gender variable avoid gender discrimination? No. The model infers gender from correlated variables (occupation, hobbies, purchasing patterns). To mitigate bias you have to measure it explicitly by subgroup and apply correction techniques, not hide the variable.
Does the AI Act apply to every company? It applies to providers and users of AI systems according to the level of risk. An SME that uses a high-risk system (for example, to recruit staff) takes on oversight, logging and transparency obligations, even if it did not develop the model.
Is explainability mandatory? The GDPR requires informing people of the logic of automated decisions with legal effect, and the AI Act reinforces transparency for high-risk systems. Technical explainability (SHAP, LIME) is the usual means of meeting that duty.
Does anonymising the data take me out of the GDPR? Only if the anonymisation is irreversible and does not allow re-identification. Pseudonymisation (replacing the identifier) is not enough: it remains personal data.
Conclusion
Ethical AI is not achieved with a declaration of principles, but with traceable technical decisions: choosing and justifying a specific fairness metric, measuring performance by subgroup, applying differential privacy or federated learning when the data is sensitive, documenting the model with model cards and subjecting the entire life cycle to an auditable management system such as ISO/IEC 42001. The AI Act and the GDPR no longer leave room for opacity in decisions that affect people. Algorithmic accountability is, ultimately, a matter of governance: knowing what the model decides, why, with what data and who answers when it gets it wrong. At Summum Marketing we support organisations through that audit, from reviewing the dataset to producing the conformity documentation.