Capabilities
Structured Output
Also known as: JSON mode, constrained decoding
Making an AI return its answer in a strict, machine-readable format (like JSON) so other software can use it reliably.
Structured output means getting the AI to answer in a tidy, fixed format instead of free-flowing prose, for example a neat list of fields a program can read. If you want an app to use the AI's answer automatically (put it in a spreadsheet, trigger an action), it needs to come back in a predictable shape. Structured output guarantees that shape.
Structured output is when a model returns data in a defined format, most commonly JSON matching a schema, rather than plain text. This is essential whenever software, not a human, will consume the answer.
Why it is needed
- Reliability: free text is hard for programs to parse; a fixed schema is not.
- Integration: it lets the AI feed results directly into apps, databases and workflows.
- It is closely tied to tool use: calling a function requires well-formed arguments.
How it is achieved
Options range from simply asking (via prompt engineering) to constrained decoding, where the system forces the output to conform to a schema, guaranteeing valid structure.
Structured output constrains generation to a formal schema (typically JSON Schema), producing machine-consumable results and underpinning reliable tool/function calling and agentic pipelines. Approaches span the spectrum from soft (prompting and few-shot exemplars, best-effort) to hard constrained decoding, where a grammar or schema mask restricts the next-token distribution at each step so only schema-valid continuations are sampled, guaranteeing parseable output.
Trade-offs: constrained decoding ensures validity but can interact with quality (over-constraining may distort content or suppress useful reasoning), so a common pattern separates a free reasoning phase from a final structured emission. Schema design, required vs optional fields, enums, and validation with retries on failure, matters for robustness. Structured output is foundational to production LLM systems because it converts probabilistic text into a dependable interface between the model and deterministic software.