Risks & safety
Bias (in AI)
Also known as: algorithmic bias
When an AI system produces systematically unfair or skewed results, usually because it learned patterns from biased data.
Bias in AI means the system treats some people or topics unfairly, for example working less well for certain groups, or repeating stereotypes. It usually is not deliberate: the AI learned from data that already contained human biases, and it copied them. Spotting and reducing bias matters because AI is increasingly used in decisions that affect real people.
Bias is systematic, unfair skew in an AI system's outputs. Because machine learning models learn patterns from data, they absorb the biases present in that data, and can amplify them.
Where it comes from
- Data bias: the training data over- or under-represents certain groups or viewpoints.
- Labelling bias: the human judgements used to train the model carry their own assumptions.
- Deployment bias: a model used in a context it was not designed for.
Why it matters and what helps
Biased AI can produce discriminatory or stereotyped results in hiring, lending, healthcare and more. A well-known study found some facial-analysis systems were far less accurate for darker-skinned women than for lighter-skinned men. Mitigations include more representative data, testing across subgroups, and human oversight, part of the broader AI safety and alignment effort.
Bias denotes systematic, unjustified disparities in model behaviour across groups, contexts or content, arising from training data distribution, label generation, objective specification, or deployment mismatch, and often amplified relative to the source data. Auditing revealed stark intersectional performance gaps in commercial vision systems (Buolamwini & Gebru 2018), catalysing subgroup evaluation as standard practice.
Rigorous treatment distinguishes multiple, sometimes mutually incompatible, fairness criteria (demographic parity, equalised odds, calibration), so "unbiased" requires specifying which notion and accepting trade-offs. Sources compound across the pipeline (sampling, annotation, feature/proxy variables, feedback loops), and mitigation spans data curation and reweighting, constrained or adversarial training, evaluation on disaggregated benchmarks, and post-hoc adjustment, none a complete fix. In generative models bias manifests as representational harms (stereotyping, erasure) alongside allocative ones, and interacts with alignment choices; it is a core concern of AI safety and of emerging regulation.