Structured Output — Transcript AI Lab

Why this matters

Downstream code can't read prose. It needs {"intent": "interested", "amount": 500000} — every time, with exactly those keys, types and allowed values. At 100,000 calls a day, a 1% malformed-output rate is 1,000 silent failures.

is a contract (a ) plus enforcement (validation, retries, and where available ). The contract guarantees *shape*; only evaluation tells you whether the *values* are right.

Core concept

Technical explanation + simple intuition + example

The schema is the contract

A schema says: these fields exist, these are required, this one is an integer ≥ 0, this one is one of three exact strings, nothing else is allowed. Write it before the prompt — it forces you to decide what you actually need.

Constrained decoding vs validate-and-retry

Two ways to get valid JSON: (1) constrain generation so the model can only pick tokens that keep the output valid; (2) let it generate, then validate, retrying or routing to review on failure. Constrained decoding guarantees syntax; neither guarantees the answer is right.

The 10 questions

Every important concept should answer all of these

1What is it?

Making the model emit machine-readable output (usually JSON) that conforms to a declared schema.

2Why does it exist?

Automation needs predictable fields, types and allowed values.

3What problem does it solve?

It eliminates parsing heuristics and silent failures between the LLM and downstream code.

4How does it work internally?

Declare a JSON Schema, instruct the model (or use JSON/tool mode), validate every response, retry once, then fall back to review; optionally constrain decoding.

5What does it look like on real transcript data?

Amounts like '5 lakh' tempt models into strings; enums drift in case ('Interested'); long reasoning fields hit max_tokens and truncate the object.

6What can go wrong?

Invalid syntax, wrong types, enum violations, missing required fields, extra fields, markdown wrappers, truncation — and schema-valid but factually wrong values.

7How do I evaluate it?

Format validity rate (parse + schema), per-field accuracy against gold, and truncation/stop-reason rates.

8When should I use it?

Every LLM output consumed by code.

9When should I NOT use it?

Free-text summaries for humans (still consider a light schema with a summary field).

10How does it affect downstream systems?

Schema design fixes the interface between stages; enum choices must match the taxonomy and the annotation guideline.

Interactive lab: JSON extraction lab

Edit a JSON Schema and a model output and validate live with a real validator. Diagnose eight deliberately malformed outputs. Step through constrained decoding token by token.

LIVE COMPUTATIONPRECOMPUTED EXAMPLE

Loading lab…

Try it yourself

Conceptual (A), applied (B) and engineering (C) exercises

Q1Type B · AppliedBeginner

Output: {"loan_type": "personal", "amount": "5 lakh", "interest_rate_mentioned": true, "intent": "interested"}. The schema says amount is an integer. What's wrong?

Q2Type C · EngineeringIntermediate

Design a JSON Schema for extracting loan amount, tenure and interest rate. Requirements: all three keys required; amount integer ≥ 0 or null; tenure_months integer or null; interest_rate number between 0 and 50 or null; no extra fields.

Q3Type A · ConceptualIntermediate

With grammar-constrained decoding enforcing the intent enum, which failure can STILL happen?

Real-world application

A nightly pipeline logged “0 interested customers” for three days. The model had started returning "intent": "Interested" after a provider model update; the downstream filter compared against the lowercase enum. No exception was thrown. The team added schema validation with enum, alerting on validity-rate drops, and pinned the model version.

Common mistakes

Parsing with regex instead of validating against a schema

Silent failures: wrong types and enum drift pass through.

Fix: Validate every output; alert on validity rate.

Omitting keys for unknown values

Downstream code breaks or treats missing as false.

Fix: Require all keys; use null for unknown.

Believing valid output is correct output

The schema can't know that 5 lakh ≠ 50,000.

Fix: Add grounding checks and field-level evaluation.

Unbounded retries

Cost and latency explode on systematic failures.

Fix: One retry, then route to review and log.

Engineering decision

Constrained decoding, JSON mode, or validate-and-retry?

Use it when

Constrained decoding/strict tool schemas when your provider or runtime supports them
Validate-and-retry always, as a safety net
Both for high-volume pipelines

Avoid it when

Relying on 'please return JSON' alone at scale

Trade-offs

Strict schema vs Flexibility

additionalProperties: false surfaces improvisation but requires schema updates for new fields.

Reasoning field vs Truncation & cost

Long 'reason' fields help audits but raise token cost and truncation risk; cap length.

Quiz

Pass mark 60%. Missed questions go to your mistake log and spaced review.

0/5 answered

Q1Beginner

What does "additionalProperties": false do?

Q2Intermediate

An output ends with "interest_rate_men and no closing brace. Most likely cause?

Q3Intermediate

Unknown amount should be represented as…

Q4Intermediate

Which belong in a production structured-output pipeline? (Select all)

Select all that apply.

Q5Advanced

How does grammar-constrained decoding enforce an enum?

Advanced challenge

Type C · EngineeringAdvanced

Write the final-output schema for the transcript system: fields intent (enum of 3), entities (object with amount integer|null), evidence (string), confidence (number 0–1), reason (string), needs_review (boolean). All required; no extra fields at top level.