Tools & agents
Guardrails
Also known as: AI guardrails
The safety checks and limits placed around an AI system to keep its inputs and outputs within acceptable bounds.
Guardrails are the safety rails built around an AI to keep it in bounds, checks that block harmful requests, filter unsafe answers, and stop it doing risky actions. Just as guardrails on a road keep a car from going over the edge, AI guardrails catch problems the model itself might miss. They are added around the model, so safety does not depend on the model behaving perfectly.
Guardrails are the controls placed around an AI system, separate from the model, to enforce safety and policy. Because a model alone can be wrong, jailbroken or injected, guardrails add independent checks.
What they do
- Input checks: block disallowed or malicious requests before they reach the model.
- Output checks: filter or rewrite unsafe, off-topic or non-compliant responses.
- Action limits: for agents, restrict which tools can be used and require confirmation for risky actions.
- Structure enforcement: ensure outputs match a required format.
Why "around", not "inside"
Model-level alignment reduces bad behaviour but cannot guarantee it. Guardrails are defence-in-depth: a second, independent layer, part of good AI safety practice.
Guardrails are system-level controls, distinct from model weights, that constrain inputs, outputs and actions to enforce safety, security and policy, implementing defence-in-depth around inherently probabilistic models. Typical components: input classifiers and filters (toxicity, PII, injection detection), output validators (safety classifiers, schema/format checks, groundedness/citation verification), topical and tool-use policies, and, for agents, least-privilege scoping plus human-in-the-loop confirmation for irreversible actions.
They are essential precisely because model alignment is not a hard boundary, jailbreaks and prompt injection demonstrate that refusals and instruction-following can be subverted, so independent enforcement is required. Design trade-offs include false positives (over-blocking, degraded UX) versus false negatives (unsafe leakage), latency and cost of added checks, and the fact that classifier-based guards are themselves fallible and attackable. Frameworks such as the NIST AI RMF situate guardrails within broader governance (risk assessment, monitoring, incident response), reflecting that robust deployment combines technical controls with process and oversight.