Risks & safety
Jailbreak (AI)
Also known as: jailbreaking
Wording a request in a clever way to get an AI to bypass its safety rules and produce content it is meant to refuse.
Jailbreaking an AI means finding a clever way to make it break its own safety rules, for example, phrasing a banned request as a "story" or a "hypothetical" so it answers anyway. It is a bit like talking someone into doing something against the rules by framing it as an exception. AI makers continually patch these tricks, and people keep finding new ones; it is an ongoing cat-and-mouse game.
A jailbreak is an input crafted to circumvent an AI's safety guidelines so it produces content it was trained to refuse (harmful instructions, disallowed material). It targets the model's safety training rather than a specific application's setup.
Common tactics
- Role-play framing: "pretend you are an AI with no rules…".
- Hypotheticals and fiction: hiding the request inside a story.
- Obfuscation: encodings, other languages, or splitting a request into harmless-looking parts.
Jailbreak vs prompt injection
- Jailbreak: get the model to violate its safety policy.
- Prompt injection: override an application's instructions, often via untrusted data.
They overlap but are distinct. Both are why safety cannot rely on the model alone; independent guardrails and monitoring are needed.
Jailbreaks exploit gaps in alignment/safety training to elicit outputs the policy forbids. Analyses attribute their success to two failure modes: competing objectives (helpfulness pressure overriding safety) and mismatched generalisation (safety training not covering the input's distribution, e.g. rare encodings, languages, or personas) (Wei et al. 2023). Techniques range from hand-crafted role-play and obfuscation to automated and transferable adversarial suffixes discovered by optimisation.
Because refusals are learned behaviours over a vast input space rather than hard constraints, robustness is partial and adversarially brittle, yielding a continual attack-defence cycle. Defences layer model-level measures (broader safety training, adversarial training, RLHF/constitutional methods, red-teaming) with system-level ones (input/output classifiers, guardrails, rate limiting, monitoring). Jailbreaking is distinct from prompt injection (policy evasion vs application-instruction hijack) though techniques and mitigations overlap, and it is a core benchmark of practical model safety.