Hallucination
Fabricated entities, invented numbers, overconfident inferences — and how to catch each type.
Why this matters
A in transcript analysis is any output content not supported by — or contradicting — what was said. It's dangerous because it's fluent: “customer works at Infosys” sounds as factual as “customer works at an IT company in Pune”.
Different hallucinations have different causes and different defences. A fabricated bank name needs a quote-existence check; “5 crore” instead of “50 lakh” needs deterministic number normalization; “will apply next month” from “might consider” needs an abstain option and entailment checks. Categorizing is the first step to preventing.
Core concept
Technical explanation + simple intuition + example
A taxonomy of transcript hallucinations
- Fabricated entity: a name, bank or employer that was never said.
- Unsupported claim: an extra fact with no basis ("already has a loan elsewhere").
- Incorrect inference: a conclusion stronger than the evidence ("might" → "will").
- Missing evidence: the cited support doesn't actually support (agent's leading question).
- Contradiction: the transcript says the opposite (objection later resolved).
- Numerical hallucination: wrong numbers or units (lakh/crore, 10× errors).
Supported, unsupported, contradicted, insufficient
Judge the answer against the transcript: - Supported — everything in the answer is backed. - Unsupported — it adds specifics that aren't there. - Contradicted — the transcript says otherwise. - Insufficient evidence — its conclusion goes beyond what the transcript establishes.
The 10 questions
Every important concept should answer all of these
1What is it?
Model output content that is not supported by the source transcript (or contradicts it).
2Why does it exist?
LLMs generate plausible continuations; plausibility isn't truth, especially under pressure to answer.
3What problem does it solve?
Detecting and preventing it keeps downstream decisions tied to what customers actually said.
4How does it work internally?
Prevention: abstain options, evidence requirements, low temperature, deterministic parsing. Detection: quote checks, verifiers, claim decomposition, ranges.
5What does it look like on real transcript data?
Indian number units (lakh/crore), vague employer mentions, third-party references and hedges are hallucination magnets.
6What can go wrong?
Verifiers that share the generator's blind spots, checks that only test existence (not entailment), and treating 'no hallucination detected' as 'correct'.
7How do I evaluate it?
Label a sample of outputs per claim (supported/unsupported/contradicted/insufficient), report rates by category and slice.
8When should I use it?
Always check for hallucination on customer-facing or decision-driving outputs.
9When should I NOT use it?
Don't rely on the generator to self-report hallucinations without verification.
10How does it affect downstream systems?
Hallucinated amounts misprice offers; fabricated employers corrupt eligibility; invented intent creates false leads.
Interactive lab: Hallucination spotting
Eleven model answers to real transcripts. Judge each as Supported / Unsupported / Contradicted / Insufficient Evidence, classify the hallucination type, then reveal the evidence.
Try it yourself
Conceptual (A), applied (B) and engineering (C) exercises
Graded by a transparent keyword rubric (not an LLM) — it checks for the key ideas, then shows a model answer for self-assessment.
Real-world application
An auto-generated CRM note said “Customer employed at TCS, salary 12 LPA”. The call only said “IT company” and never mentioned salary. The note was used for eligibility pre-checks. After an audit found 7% of notes contained fabricated specifics, the team switched to extracting fields with verbatim evidence spans, rejected any field whose evidence didn't contain the value, and let employer be null when unspecified.
Common mistakes
Pressure to fill → plausible fabrication.
Fix: Allow null / insufficient evidence explicitly.
A real quote can still fail to support the claim.
Fix: Add entailment verification per claim.
Unit errors (lakh/crore) are common.
Fix: Extract the span; normalize numbers deterministically in code.
Engineering decision
Where do you invest in hallucination defence?
- Fields that drive money or compliance (amounts, rates, consent)
- Free-text notes that humans will trust
- Outputs shown to customers
- Don't build heavy verifiers for low-stakes aggregate analytics — sample-audit instead
Quiz
Pass mark 60%. Missed questions go to your mistake log and spaced review.