Why AI can be unfair, and what you can do about it
AI learns from things people made, so it can repeat old unfairness and stereotypes. Here’s how that happens, where it matters most, how to spot it and what to do.
Video transcript
AI has no opinions about people. So how can it treat them unfairly? It comes down to what it learns from, and there’s plenty you can do about it.
AI learns from things people made: photos, writing and records of past decisions. Those carry our history, unfairness included, and the AI can learn those patterns as if they were the rule.
When OpenAI tested an early version of its image tool, nurses mostly came out as women, builders too often as men, and lawyers too often as white men. OpenAI traced it to the training data and how the model was trained.
A stereotyped picture is annoying. An unfair decision can change a life. So take most care when AI helps decide about jobs, loans, housing or health.
To spot bias, look at who’s missing. Ask the same question with a different name, age or gender, and compare the answers. And watch for defaults, like every doctor being a man.
If you find it, challenge it and ask for another go. Report it with the feedback button. And if AI was used in a decision about you, ask for a person to review it.
In England, Scotland and Wales, the Equality Act 2010 still applies when organisations use AI. Using a computer doesn’t excuse unfair treatment.
Read the full guide below for prompts that help you get a fairer, wider view.
In 30 seconds
- AI learns from data made by people, so it can pick up past unfairness and stereotypes.
- It matters most when AI helps make decisions about jobs, loans, housing or health.
- Challenge it and report it. Equality law still applies when organisations use AI.
AI has no opinions about people. But it learns from things people made, such as photos, articles and records of past decisions, and those carry our history, unfairness included. So AI can repeat old stereotypes, or treat some people worse than others, without anyone meaning it to.
How unfairness gets in
AI learns by finding patterns in a huge number of examples, a stage called Jargon busterTraining: The stage where AI learns, by finding patterns in a huge number of examples.. If most of the examples show men as engineers, or past decisions favoured some groups, the AI learns those patterns as if they were the rule. When that makes its answers unfair to some people, it’s called Jargon busterBias: When an AI system’s answers or decisions are unfair to some groups of people, often because of patterns in the data it learnt from..
- Human-made dataPhotos, writing and past decisions
- TrainingThe AI finds the patterns
- PatternsOld stereotypes look like the rule
- AnswersWhich can repeat them
What it looks like
What two AI companies found in their own tools
- Stereotyped picturesIn 2022, OpenAI reported that its DALL·E 2 image tool tended to show women for “nurse” and over-represented white men for “lawyer”.
- Uneven decisionsIn 2023, Anthropic tested its Claude 2.0 model on 70 kinds of decision, such as loans and housing, and found discrimination in some.
Both companies published these results alongside work to reduce the problem. Anthropic found that carefully worded instructions cut discrimination significantly. But bias is hard to remove completely, so it’s worth knowing where it matters and how to spot it.
Where it matters most
A stereotyped picture is annoying. An unfair decision can change someone’s life. Take most care when AI helps decide who gets a job interview, a loan, a home or medical care. Anthropic’s own usage policy requires businesses using its AI for decisions like these to have a qualified professional review them before they’re final.
How to spot it, and widen the view
- Who’s missing? Look at who appears in the pictures and examples, and who never does.
- Swap a detail. Ask the same question with a different name, age or gender, and compare the answers. They should match unless there’s a good reason.
- Watch the defaults. Is every doctor in its stories “he” and every nurse “she”?
Give me [examples / a picture / a short story] of [topic] that shows a realistic mix of people: different ages, genders, ethnic backgrounds, body types and abilities. Avoid stereotypes, and tell me about any assumptions you made.
Check this [job advert / letter / description] for words or assumptions that could be unfair to some groups of people, or put them off, such as older people, women or disabled people. Explain each one and suggest fairer wording. [paste your text]
What you can do about it
- Challenge itTell the AI what looks wrong and ask it to try again. It may well correct itself.
- Report itUse the feedback button, often a thumbs-down, so the company can see the problem.
- Ask for a personIf an organisation used AI in a decision about you, ask how the decision was made, and ask for a person to review it.
- Get adviceIf you think you’ve been treated unfairly, Citizens Advice can explain your options.
Check yourself
3 quick questions nothing is savedSources (3)
- DALL·E 2 Preview: Risks and LimitationsOpenAI, April 2022
- Evaluating and Mitigating Discrimination in Language Model DecisionsAnthropic, December 2023
- Usage PolicyAnthropic, September 2025
Spotted a mistake? Tell us and an editor will check it.
