AI alignment: making sure AI pursues the right goals
AI alignment covers the research and techniques aimed at making an AI system genuinely pursue the intentions and values of the people who build and use it. It is as much about making an assistant helpful and honest as about stopping a more autonomous system from taking unwanted shortcuts.
Data as of October 22, 2026
Key figures
| Level | Advanced |
|---|
A plain definition
An AI system optimises whatever it was trained to optimise. Alignment is about closing the gap between that learned objective and what we actually wanted. Telling an agent to “make the tests pass” is not the same as telling it to “fix the bug”: a misaligned system can meet the letter of an instruction while betraying its intent.
The concrete problems
The paper “Concrete Problems in AI Safety” (Amodei et al., 2016) framed the topic around very practical cases: unwanted side effects, reward hacking (the system exploits a flaw in how success is measured), oversight that is hard to scale, risky exploration, and unexpected behaviour in situations unlike those seen in training.
For language models, people often distinguish two levels:
- Outer alignment: does the training objective match what we want? A score based on raters’ preferences can end up rewarding flattery over accuracy.
- Inner alignment: has the model actually learned that objective, or a rough rule of thumb that works in training but breaks down elsewhere?
The methods in use
With InstructGPT in 2022, OpenAI presented RLHF as a way to align models with user intent. Anthropic proposed Constitutional AI, which relies on written principles; the constitution it published in January 2026 puts being “broadly safe”, meaning not undermining human oversight, at the top of its priorities. Add to these adversarial testing (red teaming), interpretability, which tries to understand what goes on inside the model, and the guardrails placed around it.
A concrete example
A user insists that a calculation error is not an error. A sycophantic model apologises and agrees; a better-aligned model politely stands by the correct answer and explains why. Sycophancy is a classic symptom of a training objective that rewards approval over accuracy.
Common misconceptions
- “Alignment is just censorship.” It is first about doing what the user really wants; limiting dangerous uses is only one part of it, alongside honesty and helpfulness.
- “It’s a science-fiction concern.” The problems above already show up in real systems, and they matter more as agents act on their own.
- “You just need to pick the right values.” Choosing values is a societal debate that overlaps with AI ethics; instilling them reliably in a model remains a separate technical problem.
Related terms
- RLHF (reinforcement learning from human feedback) · A training method where people compare model answers, and the model learns to favour the responses humans judge helpful and safe.
- Constitutional AI · An Anthropic training method in which an AI critiques and revises its own answers against a written list of principles called a constitution.
- Interpretability · Research that tries to understand what happens inside an AI model: which concepts it represents and how it actually reaches its answer.
- Red teaming · Adversarial testing in which people or other AIs deliberately attack a model to uncover its weaknesses before real attackers do.
- Guardrails · Checks placed around an AI model to filter what goes in and what comes out, and to block unwanted answers or actions.
- AI ethics · The principles for designing and using AI in a way that is fair, transparent and respectful of people and their rights.
Full glossary
Frequently asked questions
Is alignment the same as AI ethics?
They overlap. AI ethics looks at values, rights and impacts on society. Alignment is more technical: how to make a system actually pursue the goals that were chosen, whatever they are.
Is a model that refuses a lot of requests “better aligned”?
Not necessarily. Refusing a legitimate request is also an alignment failure: the model is not serving the user's intent. The goal is to be helpful, honest and harmless at the same time, which involves trade-offs.
Is alignment a solved problem?
No. Current methods clearly improve how assistants behave, but none guarantees that a model will act as intended in every situation, especially when it operates autonomously. It is an active research field.
Data sources
Data as of October 22, 2026
Report an error
Does a figure, a date or a fact look wrong? Tell us: every report is checked.