RAG and Agents: Do You Need Them?
Most transcript-analysis problems need neither. Know when they help — and when a deterministic pipeline wins.
Why this matters
and are powerful patterns, and they're heavily marketed. But intent classification, entity extraction and evidence selection all operate on information that's *already in the transcript*. Retrieval adds a component that can miss; agents add nondeterministic control flow that's hard to test, bound and audit.
This lesson is deliberately short. The goal is one skill: answering “do I actually need RAG or an agent for this?” — and usually answering *no*, for good reasons.
Core concept
Technical explanation + simple intuition + example
When RAG helps
RAG = look things up, then answer. You need it only when the answer depends on knowledge that isn't in the input and is too big or too changeable to put in the prompt. If the knowledge is a table with exact keys, don't use similarity search — use a database query.
When agents help
An agent decides its own next step, calls tools and loops. That's useful when you genuinely can't write the flowchart in advance. If you can draw the steps, build the steps — a fixed pipeline is cheaper, faster, testable and auditable.
The 10 questions
Every important concept should answer all of these
1What is it?
RAG injects retrieved external knowledge into prompts; agents let a model choose actions/tool calls in a loop.
2Why does it exist?
Some tasks need knowledge beyond the input or dynamic multi-step tool use.
3What problem does it solve?
RAG handles large/changing knowledge; agents handle open-ended procedures.
4How does it work internally?
RAG: index → retrieve top-k → generate with citations. Agents: model ↔ tools loop with limits.
5What does it look like on real transcript data?
Transcript intent/extraction tasks rarely need external knowledge; rate/policy checks usually need exact lookups, not semantic retrieval.
6What can go wrong?
Retrieval misses look like hallucinations; irrelevant retrieved text distracts; agents loop, overspend or take wrong actions; both are harder to evaluate.
7How do I evaluate it?
RAG: retrieval recall@k and answer faithfulness separately. Agents: task success, steps, cost, harmful-action rate on trajectory test sets.
8When should I use it?
RAG: large, unstructured, changing knowledge. Agents: tool-dependent, non-enumerable procedures with bounded risk.
9When should I NOT use it?
When everything needed is in the transcript; when knowledge is small/stable (put it in the prompt) or structured (query it); when steps are known (pipeline).
10How does it affect downstream systems?
Unneeded RAG/agents add latency, cost, failure modes and evaluation burden to every decision.
Interactive lab: Do I need RAG or an agent?
Seven realistic scenarios: pick the simplest architecture that works (deterministic code, a single call/fixed pipeline, RAG, or an agent) and read why. Then walk the RAG and agent decision trees.
Try it yourself
Conceptual (A), applied (B) and engineering (C) exercises
Graded by a transparent keyword rubric (not an LLM) — it checks for the key ideas, then shows a model answer for self-assessment.
Graded by a transparent keyword rubric (not an LLM) — it checks for the key ideas, then shows a model answer for self-assessment.
Real-world application
A team built a RAG system over 2,000 past transcripts to “help classify intent”. Retrieval surfaced similar calls; the model then sometimes copied *their* outcomes. Accuracy dropped 3 points versus classifying the transcript alone, latency doubled, and the vector index needed maintenance. Removing RAG and adding four carefully chosen few-shot examples beat the RAG system on every metric.
Common mistakes
The transcript already contains the evidence.
Fix: Ask: is the needed knowledge outside the input? If not, no RAG.
Similarity isn't exact lookup; you get 'close' wrong rows.
Fix: Extract keys and query the table.
Nondeterminism without benefit.
Fix: Build the known sequence as a pipeline or router.
Engineering decision
Does this transcript problem need RAG or an agent?
- RAG: answers depend on large, unstructured, changing external knowledge
- Agent: next steps depend on tool results in ways you can't enumerate — with limits and audits
- The transcript contains everything needed
- Knowledge is small and stable (prompt it) or structured (query it)
- The procedure is known (pipeline)
Quiz
Pass mark 60%. Missed questions go to your mistake log and spaced review.