Glossary

134 terms, each with a simple definition, a technical definition, an example, why it matters, and related concepts. Terms are clickable throughout the lessons.

Accuracy

Evaluation

Simple definition: Fraction of all predictions that were correct.
Technical definition: (TP + TN) / N; dominated by the majority class under imbalance.
Example: Predicting 'not interested' for everyone at 1% prevalence → 99% accuracy.
Why it matters: Misleading for rare events.

Class imbalance F1 score Lesson: Classification Metrics →

Adjacency pair

NLP Fundamentals

also: adjacency pairs

Simple definition: Two utterances that naturally go together, like a question and its answer.
Technical definition: A conversation-analysis unit of two turns by different speakers where the first part makes a specific second part relevant (question→answer, offer→accept/decline, greeting→greeting).
Example: Agent: 'aap salaried ho?' → Customer: 'haan'.
Why it matters: A bare 'haan' is only interpretable through the question it answers.

Dialogue act Turn-taking Lesson: Dialogue and Discourse Structure →

Agent

LLM Engineering

also: agents, agentic

Simple definition: An LLM system that decides its own next steps and calls tools in a loop.
Technical definition: A control loop where a model selects actions (tool calls), observes results and iterates until a goal or stop condition; non-deterministic control flow.
Example: An LLM that searches a CRM, schedules a callback and writes notes.
Why it matters: Powerful but harder to test, cost-bound and audit than a fixed pipeline.

Retrieval-Augmented Generation (RAG)Multi-stage pipeline Lesson: RAG and Agents: Do You Need Them? →

Annotation guideline (codebook)

Evaluation

also: codebook, guidelines

Simple definition: The rulebook annotators follow when labelling data.
Technical definition: A document specifying label definitions, inclusion/exclusion criteria, positive/negative/borderline examples, edge-case rulings and escalation procedures.
Example: INTERESTED: customer explicitly expresses willingness to pursue the product.
Why it matters: Label quality — and therefore every metric — depends on it.

Inter-annotator agreement (IAA)Gold set Insufficient evidence Lesson: Annotation Guidelines →

ASR / Speech-to-text

NLP Fundamentals

also: stt, speech to text, speech-to-text

Simple definition: Software that converts audio into text.
Technical definition: Automatic Speech Recognition maps acoustic signals to token sequences, typically with an encoder-decoder or CTC model. Output contains substitutions, deletions, insertions and no reliable punctuation.
Example: Audio 'loan' transcribed as 'lone'.
Why it matters: Every downstream stage inherits ASR errors — design for them.

Word Error Rate (WER)Speaker diarization

Base rate (prevalence)

Evaluation

also: prevalence, base rate, base rates

Simple definition: How common the positive class is in reality.
Technical definition: The prior probability P(y=1). Precision = sens·prev / (sens·prev + (1−spec)·(1−prev)) depends on it directly.
Example: A 95%-sensitive, 95%-specific detector at 1% prevalence has precision ≈ 16%.
Why it matters: The same model has very different precision on different traffic.

Class imbalance Precision Lesson: Class Imbalance →

Batching

LLM Engineering

also: batch processing

Simple definition: Processing many requests together instead of one at a time.
Technical definition: Either server-side dynamic batching for throughput, or asynchronous batch APIs that trade latency (hours) for a discount.
Example: Submitting last night's 200k calls as one batch job.
Why it matters: Cheaper for offline analytics; unsuitable for real-time needs.

Throughput Prompt caching Lesson: Prompt Caching and Batching →

BERTopic

Information Extraction

Simple definition: A topic modelling method built from embeddings + clustering + keyword extraction.
Technical definition: Pipeline: sentence embeddings → UMAP dimensionality reduction → HDBSCAN clustering → class-based TF-IDF to describe each cluster with keywords.
Example: Topic 3: 'rate, interest, high, kam'.
Why it matters: Strong default for discovering themes in short texts.

UMAP HDBSCAN Class-based TF-IDF (c-TF-IDF)Clustering Lesson: Topic Modeling and Clustering →

BIO tagging

NLP Fundamentals

also: iob

Simple definition: A labelling scheme marking the Beginning, Inside and Outside of entities.
Technical definition: Each token gets B-TYPE (first token of an entity), I-TYPE (continuation) or O (outside), converting span extraction into token classification.
Example: 'personal/B-LOAN loan/I-LOAN of/O 5/B-AMOUNT lakh/I-AMOUNT'.
Why it matters: How classic NER models and many annotation tools represent spans.

Named Entity Recognition (NER)Entity span Lesson: Named Entity Recognition →

Bootstrap

Evaluation

Simple definition: Estimating uncertainty by resampling your evaluation set many times.
Technical definition: Draw B resamples with replacement, recompute the metric on each, and take percentiles as a confidence interval.
Example: 1,000 resamples of 300 calls → F1 CI [0.71, 0.80].
Why it matters: Works for any metric, including F1 and kappa.

Confidence interval Paired test Lesson: Statistical Significance →

Brier score

Evaluation

Simple definition: Mean squared error of predicted probabilities.
Technical definition: BS = (1/N) Σ (p_i − y_i)²; a proper scoring rule combining calibration and discrimination.
Example: Predicting 0.9 for a negative case adds 0.81.
Why it matters: Rewards honest probabilities.

Calibration Lesson: Calibration →

Byte-Pair Encoding (BPE)

NLP Fundamentals

also: byte pair encoding

Simple definition: A tokenizer that learns to glue together the most frequent character pairs into bigger chunks.
Technical definition: Starting from bytes/characters, BPE iteratively merges the most frequent adjacent pair in a training corpus, producing a merge table. Encoding applies merges in learned priority order. Byte-level BPE guarantees any string is representable.
Example: If ' loan' appears often in training data it becomes a single token; a rare misspelling like ' lon' may not.
Why it matters: Explains why frequent English words are cheap and rare spellings, scripts or code-mixed text are more expensive.

Tokenization Subword tokenization Vocabulary SentencePiece Lesson: Tokenization →

Calibration

Evaluation

Simple definition: Whether a model's confidence matches how often it's right.
Technical definition: A model is calibrated if P(y = ŷ | confidence = c) = c for all c; measured with reliability diagrams, ECE and Brier score.
Example: Among predictions with confidence 0.8, about 80% are correct.
Why it matters: Confidence thresholds for auto-approve vs human review only work if confidence is calibrated.

Expected Calibration Error (ECE)Reliability diagram Brier score Lesson: Calibration →

Chance agreement

Evaluation

also: expected agreement

Simple definition: How often annotators would agree just by guessing with their usual label frequencies.
Technical definition: Expected agreement p_e computed from the product of annotators' marginal distributions (Cohen) or pooled distribution (Fleiss/Scott).
Example: If both label 'interested' 90% of the time, p_e ≈ 0.82.
Why it matters: The reason raw agreement ≠ agreement beyond chance.

Cohen's kappa Lesson: Inter-Annotator Agreement →

Chunking

NLP Fundamentals

also: chunk, chunks

Simple definition: Splitting a long text into smaller pieces that fit the model's context.
Technical definition: Partitioning a document into segments under a token budget, ideally on semantic boundaries (turns, segments) with overlap, then processing each chunk and aggregating results.
Example: A 30,000-token call split into 11,000-token chunks at turn boundaries with 2 turns of overlap.
Why it matters: Naive fixed-size chunks cut numbers and question→answer pairs apart, losing evidence.

Context window Token Conversation segmentation Lesson: Tokenization →

Class imbalance

Evaluation

also: imbalance, imbalanced data

Simple definition: When one label is much more common than others.
Technical definition: A skewed prior over classes; affects metric interpretation, sampling, thresholding and training.
Example: 1% of calls contain a compliance violation.
Why it matters: Accuracy becomes meaningless; precision collapses at low prevalence.

Base rate (prevalence)Accuracy Stratified sampling Lesson: Class Imbalance →

Class-based TF-IDF (c-TF-IDF)

Information Extraction

also: ctfidf

Simple definition: TF-IDF computed per cluster, to find the words that describe each cluster.
Technical definition: Treats all documents in a cluster as one document and computes term importance relative to other clusters.
Example: Cluster keywords: 'emi, zyada, high'.
Why it matters: Gives clusters human-readable names.

BERTopic TF-IDF Lesson: Topic Modeling and Clustering →

Closed-set classification

Information Extraction

also: closed set

Simple definition: Predict only from a fixed, predefined list of labels.
Technical definition: Classification where the label space is fixed at design time; unknown categories must map to an existing label or 'other'.
Example: Objection ∈ {rate, eligibility, timing, documentation, other}.
Why it matters: Consistent, governable, comparable over time.

Open-set extraction Taxonomy Lesson: Open-set vs Closed-set Classification →

Clustering

Information Extraction

Simple definition: Grouping similar items without predefined labels.
Technical definition: Unsupervised partitioning of points by similarity (k-means, hierarchical, density-based like HDBSCAN).
Example: Objection snippets grouping into rate, eligibility and timing concerns.
Why it matters: Discovers themes to feed taxonomy design.

k-means HDBSCAN BERTopic Lesson: Topic Modeling and Clustering →

Code-switching

NLP Fundamentals

also: code-mixing

Simple definition: Switching between languages within a conversation or even a sentence.
Technical definition: Alternation between two or more languages at inter- or intra-sentential boundaries.
Example: 'EMI kitna aayega for 3 years?'
Why it matters: Language ID, spelling correction and NER must handle mixed spans.

Hinglish Romanization Lesson: Text Normalization →

Cohen's kappa

Evaluation

also: kappa, cohen's kappa, cohen kappa

Simple definition: Agreement between two annotators after removing agreement expected by chance.
Technical definition: κ = (p_o − p_e) / (1 − p_e), where p_o is observed agreement and p_e = Σ_k p_A(k)·p_B(k) is chance agreement from each annotator's marginal label distribution.
Example: p_o = 0.90, p_e = 0.82 → κ = 0.44.
Why it matters: Raw agreement looks great when one label dominates; κ reveals it.

Inter-annotator agreement (IAA)Fleiss' kappa Chance agreement Lesson: Inter-Annotator Agreement →

Confidence interval

Evaluation

also: ci, confidence intervals

Simple definition: A range of plausible values for a metric given sampling noise.
Technical definition: An interval procedure that covers the true parameter with a stated long-run frequency (e.g., 95%); for proportions, Wilson intervals are preferred over the normal approximation.
Example: Accuracy 0.84, 95% CI [0.78, 0.89] on 200 examples.
Why it matters: Two prompts with overlapping CIs may not really differ.

Statistical significance Bootstrap Lesson: Statistical Significance →

Confusion matrix

Evaluation

Simple definition: A table of predicted vs actual labels.
Technical definition: A K×K count matrix where cell (i, j) counts instances with gold label i predicted as j; binary case yields TP, FP, FN, TN.
Example: TP=40, FP=10, FN=20, TN=930.
Why it matters: Every classification metric is derived from it.

Precision Recall F1 score Lesson: Classification Metrics →

Constrained decoding

LLM Engineering

also: constrained generation, grammar-constrained decoding

Simple definition: Only letting the model pick tokens that keep the output valid.
Technical definition: At each decoding step, logits for tokens that would violate a grammar/schema are masked to −∞, guaranteeing syntactically valid output (e.g., grammar-constrained or JSON-mode generation).
Example: After '"intent": "' only tokens that can continue one of the enum values are allowed.
Why it matters: Guarantees format, not truth — a valid enum can still be the wrong label.

JSON Schema Structured output Lesson: Structured Output →

Context engineering

LLM Engineering

Simple definition: Deciding exactly what goes into the model's context, in what order.
Technical definition: The design of the full input — instructions, rules, examples, retrieved evidence, conversation state — under token, cost and attention constraints.
Example: Putting the rulebook first (cacheable), the transcript next and the question last.
Why it matters: The same model performs very differently depending on what it's shown.

Context window Lost in the middle Prompt bloat Lesson: Context Engineering →

Context rot

LLM Engineering

Simple definition: Quality degrading as the context gets longer and noisier.
Technical definition: Informal term for performance decline with increasing input length and irrelevant content, even well within the nominal context window.
Example: Adding 30 unrelated policy pages makes the intent label less reliable.
Why it matters: More context is not free — it can hurt accuracy.

Lost in the middle Prompt bloat Lesson: Context Engineering →

Context window

LLM Engineering

also: context length, context limit

Simple definition: How much text (in tokens) the model can consider at once.
Technical definition: The maximum number of tokens (input + output) a model can attend over in a single forward pass sequence. Content beyond it must be truncated, summarized or retrieved.
Example: A 128k-token window holds roughly a few hundred pages of English, fewer of Hinglish.
Why it matters: Determines whether a long call plus rulebook fits in one request and what it costs.

Token Lost in the middle Prompt bloat Lesson: Context Engineering →

Conversation segmentation

NLP Fundamentals

Simple definition: Splitting a call into phases like opening, discovery, offer, objection and closing.
Technical definition: Partitioning a dialogue into contiguous topical or functional segments, e.g., via dialogue-act patterns or topic-shift detection.
Example: Opening → Discovery → Qualification → Offer → Objection handling → Closing.
Why it matters: Lets you look for intent evidence in the right segment and measure agent behaviour per phase.

Dialogue act Topic shift Lesson: Dialogue and Discourse Structure →

Cosine similarity

Information Extraction

also: cosine

Simple definition: How closely two vectors point in the same direction.
Technical definition: cos(u, v) = u·v / (‖u‖‖v‖), in [−1, 1]; for normalized embeddings equals the dot product.
Example: cos('EMI too high', 'rate is high') = 0.62 (model-dependent).
Why it matters: The standard similarity for embeddings; absolute values are not comparable across models.

Embedding Lesson: Embeddings and Semantic Similarity →

Dense model

LLM Engineering

Simple definition: A model where every parameter is used for every token.
Technical definition: Standard transformer where all feed-forward weights participate in each forward pass; compute scales with total parameters.
Example: A 7B dense model runs all 7B parameters per token.
Why it matters: Simpler serving; cost and latency scale with size.

Mixture-of-Experts (MoE)Lesson: Model Selection →

Dialogue act

NLP Fundamentals

also: dialogue acts, speech act

Simple definition: What an utterance is doing in the conversation — asking, answering, refusing…
Technical definition: A functional label for an utterance's communicative intent (e.g., DAMSL/ISO 24617-2 categories such as question, inform, request, commit).
Example: 'kitne ka chahiye?' → Question; 'around five lac' → Answer.
Why it matters: Lets you find where evidence lives (answers to qualification questions) and segment calls.

Adjacency pair Turn-taking Conversation segmentation Lesson: Dialogue and Discourse Structure →

Edit distance

NLP Fundamentals

Simple definition: A family of measures counting how many edits separate two strings.
Technical definition: Generic term for string metrics defined by minimum-cost edit sequences (Levenshtein, Damerau-Levenshtein with transpositions, Hamming for equal-length strings).
Example: Damerau counts 'lnoa' → 'loan' as 1 transposition; Levenshtein counts 2.
Why it matters: Choice of edit operations changes what counts as 'close'.

Levenshtein distance Lesson: Fuzzy Matching →

Embedding

Information Extraction

also: embeddings, sentence embedding, vector

Simple definition: A list of numbers representing the meaning of a piece of text.
Technical definition: A dense vector from a neural encoder, trained so that semantically similar inputs map to nearby points (e.g., sentence-transformers with contrastive objectives).
Example: 'borrow money' and 'personal loan' have nearby vectors despite no shared words.
Why it matters: Powers semantic search, clustering, deduplication and retrieval.

Entailment

LLM Engineering

Simple definition: When one statement logically follows from another.
Technical definition: In natural language inference (NLI), a premise entails a hypothesis if a typical reader would infer the hypothesis is true given the premise; alternatives are contradiction and neutral (insufficient evidence).
Example: 'I will apply tomorrow' entails 'the customer intends to apply'.
Why it matters: Separates 'supported' from 'not mentioned' from 'contradicted'.

Natural Language Inference (NLI)Grounding Insufficient evidence Lesson: Grounding and Faithfulness →

Entity span

NLP Fundamentals

Simple definition: The exact stretch of text that makes up an entity.
Technical definition: A contiguous character or token range [start, end) with a type label; evaluation compares predicted and gold spans by exact or partial overlap.
Example: Is the amount '5 lakh' or 'around 5 lakh'? Guidelines must decide.
Why it matters: Boundary disagreements lower strict-match scores even when the extraction is useful.

Named Entity Recognition (NER)BIO tagging Lesson: Named Entity Recognition →

Error analysis

Evaluation

Simple definition: Systematically studying where and why a model fails.
Technical definition: Slicing performance by metadata, clustering failures, inspecting examples and forming/verifying root-cause hypotheses.
Example: F1 drops from 0.90 on clean to 0.62 on noisy transcripts.
Why it matters: Turns a metric into an engineering plan.

Slice Root-cause analysis Lesson: Error Analysis →

Error propagation

NLP Fundamentals

Simple definition: An early mistake causing more mistakes later in the pipeline.
Technical definition: In a sequential pipeline, the output distribution of each stage is conditioned on upstream outputs; upstream errors shift downstream inputs outside what downstream stages were validated on.
Example: Wrong speaker → wrong evidence → wrong intent → wrong sales action.
Why it matters: End-to-end accuracy hides where the failure originated.

Per-stage evaluation Multi-stage pipeline Lesson: Speaker Attribution from Text →

Evidence span

LLM Engineering

also: evidence

Simple definition: The exact quote from the transcript that supports an answer.
Technical definition: A contiguous substring (or turn range) of the source cited as justification for a claim; verifiable by string matching and entailment.
Example: "main kal tak apply kar dunga" supports intent=interested.
Why it matters: Makes outputs auditable and lets reviewers check the model quickly.

Grounding Entailment Lesson: Grounding and Faithfulness →

Expected Calibration Error (ECE)

Evaluation

also: expected calibration error

Simple definition: The average gap between confidence and accuracy.
Technical definition: ECE = Σ_b (n_b/N)·|acc(b) − conf(b)| over confidence bins b.
Example: Bin 0.9: accuracy 0.72 → gap 0.18.
Why it matters: One-number summary of miscalibration (sensitive to binning).

Calibration Reliability diagram Lesson: Calibration →

F1 score

Evaluation

also: f1 score, f-score, f-beta

Simple definition: A single number that balances precision and recall.
Technical definition: Harmonic mean F1 = 2PR/(P+R); F-beta weights recall β times as much as precision.
Example: P=0.8, R=0.67 → F1 ≈ 0.73.
Why it matters: Punishes a model that trades one metric away entirely.

Precision Recall Macro F1 Lesson: Classification Metrics →

Faithfulness

LLM Engineering

Simple definition: Whether an output accurately reflects the source, without adding or distorting.
Technical definition: The degree to which generated content is entailed by the input context; distinct from factuality with respect to the world.
Example: Summary says 'customer wants 5 lakh' but the customer never gave an amount → unfaithful.
Why it matters: The core property you verify in transcript analysis.

Grounding Hallucination Lesson: Grounding and Faithfulness →

Few-shot prompting

LLM Engineering

also: few shot, k-shot

Simple definition: Showing the model a few worked examples inside the prompt.
Technical definition: Including k input→output demonstrations in context so the model infers the mapping (in-context learning). Example choice, order and label balance affect results.
Example: Two labelled transcripts: one 'interested', one 'insufficient_evidence'.
Why it matters: Great for teaching boundaries — and for accidentally teaching biases (label imbalance, surface cues).

Zero-shot prompting In-context learning Lesson: Prompt Engineering →

Fleiss' kappa

Evaluation

also: fleiss kappa

Simple definition: Kappa for more than two annotators.
Technical definition: Generalizes Scott's π to a fixed number of raters per item: κ = (P̄ − P̄_e)/(1 − P̄_e), with P̄ the mean per-item pairwise agreement and P̄_e from pooled category proportions.
Example: Three annotators label 20 calls → κ = 0.51.
Why it matters: Standard for crowdsourced labelling with multiple raters.

Cohen's kappa Krippendorff's alpha Lesson: Inter-Annotator Agreement →

Fuzzy matching

NLP Fundamentals

Simple definition: Matching strings that are similar but not identical.
Technical definition: Approximate string matching using a similarity function and a decision threshold, optionally over a candidate dictionary.
Example: Matching 'persnal lon' to the catalogue entry 'personal loan'.
Why it matters: Thresholds trade false positives against false negatives.

Levenshtein distance Jaro-Winkler similarity Jaccard similarity Lesson: Fuzzy Matching →

Gazetteer

NLP Fundamentals

Simple definition: A lookup list of known entity names.
Technical definition: A curated dictionary of entity surface forms (bank names, cities, products) used for rule-based matching or as model features.
Example: A list of all bank names you partner with.
Why it matters: Cheap, precise, but misses unseen variants unless combined with fuzzy matching.

Named Entity Recognition (NER)Fuzzy matching Lesson: Named Entity Recognition →

Gold set

Evaluation

also: golden set, gold standard

Simple definition: A carefully labelled set of examples treated as the reference answer.
Technical definition: A held-out, high-quality labelled dataset (often adjudicated by multiple annotators) used to measure system performance.
Example: 300 calls labelled by two annotators with disagreements adjudicated.
Why it matters: If the gold set is wrong or leaked, every conclusion is wrong.

Train / dev / test split Annotation guideline (codebook)Lesson: Gold Set Design →

Greedy decoding

LLM Engineering

Simple definition: Always pick the single most likely next token.
Technical definition: Decoding that selects argmax at each step; equivalent to the T→0 limit. Not guaranteed to find the most likely full sequence.
Example: Always output 'interested' if it's the top token.
Why it matters: Maximizes repeatability, but real APIs can still vary slightly between calls.

Temperature Lesson: Sampling Parameters →

Grounding

LLM Engineering

Simple definition: Making sure the model's answers are based on the provided source text.
Technical definition: Constraining generation so each claim is attributable to supplied evidence, usually by requiring citations/spans and verifying them.
Example: Intent 'interested' must cite a customer utterance that shows interest.
Why it matters: Ungrounded outputs cannot be audited or trusted.

Faithfulness Evidence span Entailment Lesson: Grounding and Faithfulness →

Hallucination

LLM Engineering

also: hallucinations

Simple definition: When a model states something not supported by its input or by reality.
Technical definition: Generated content that is unfaithful to the source (intrinsic: contradicts it; extrinsic: adds unverifiable information) or factually wrong.
Example: Output says 'rate agreed: 11%' when no rate was discussed.
Why it matters: Silent, fluent, confident — so it must be detected with verification, not intuition.

Faithfulness Grounding Lesson: Hallucination →

Hamming loss

Evaluation

Simple definition: Fraction of individual label decisions that were wrong in multi-label classification.
Technical definition: Mean over instances and labels of the XOR between predicted and gold binary label vectors.
Example: 3 labels × 2 examples, 1 wrong decision → 1/6.
Why it matters: Gives partial credit where subset accuracy gives none.

Multi-label classification Lesson: Text Classification →

HDBSCAN

Information Extraction

Simple definition: A clustering method that finds dense groups and labels the rest as noise.
Technical definition: Hierarchical density-based clustering that extracts stable clusters across density thresholds; does not require k and marks outliers as −1.
Example: Twelve snippets fit no cluster → noise.
Why it matters: Honest about outliers; sensitive to min_cluster_size.

BERTopic Clustering Lesson: Topic Modeling and Clustering →

Hierarchical classification

NLP Fundamentals

also: hierarchical labels

Simple definition: Labels arranged in a tree, from general to specific.
Technical definition: Classification over a label hierarchy (tree/DAG) where predictions must be consistent with ancestor relations; can be solved top-down, flat with constraints, or globally.
Example: Loan → Personal Loan → Interested.
Why it matters: Lets you back off to a parent label when evidence for the child is weak.

Taxonomy Multi-class classification Lesson: Text Classification →

Hinglish

NLP Fundamentals

Simple definition: Mixed Hindi and English speech, usually written in Latin script.
Technical definition: A code-mixed variety combining Hindi grammar/lexicon with English lexicon, common in Indian call-centre conversations.
Example: 'mujhe personal loan chahiye but rate kam hona chahiye'.
Why it matters: Many NLP tools assume one language per utterance and fail on mixed input.

Code-switching Romanization Lesson: Text Normalization →

In-context learning

LLM Engineering

also: icl

Simple definition: The model picking up a task from examples in the prompt, without retraining.
Technical definition: Conditioning a pretrained model on demonstrations in its input so that its predictions adapt to the task; no gradient updates occur.
Example: After three Hinglish examples the model handles 'chahiye' correctly.
Why it matters: Cheaper and faster to iterate than fine-tuning.

Few-shot prompting Lesson: Prompt Engineering →

Insufficient evidence

Evaluation

Simple definition: The honest label for 'the transcript doesn't tell us'.
Technical definition: An explicit abstention class used when neither the positive nor negative label is entailed by the evidence.
Example: 'Maybe someday' for purchase intent.
Why it matters: Without it, annotators and models are forced to guess, which destroys label quality.

Entailment Annotation guideline (codebook)Lesson: Grounding and Faithfulness →

Inter-annotator agreement (IAA)

Evaluation

also: inter-annotator agreement, inter annotator agreement

Simple definition: How consistently different people label the same data.
Technical definition: Statistics quantifying agreement between annotators, ideally corrected for chance (Cohen's κ, Fleiss' κ, Krippendorff's α).
Example: Two annotators agree on 85% of calls, κ = 0.62.
Why it matters: An upper bound on how well any model can be measured against those labels.

Cohen's kappa Fleiss' kappa Krippendorff's alpha Lesson: Inter-Annotator Agreement →

Jaccard similarity

NLP Fundamentals

Simple definition: Overlap between two sets divided by their union.
Technical definition: J(A,B) = |A∩B| / |A∪B|, applied to token sets or character n-gram sets.
Example: Tokens {personal, loan} vs {loan, personal} → 1.0 (word order ignored).
Why it matters: Robust to reordering, blind to order-dependent meaning.

n-gram Fuzzy matching Lesson: Fuzzy Matching →

Jaro-Winkler similarity

NLP Fundamentals

Simple definition: A similarity score that rewards strings sharing the same beginning.
Technical definition: Jaro similarity counts matching characters within a window and transpositions; Winkler adds a bonus l·p·(1−J) for a common prefix of length l ≤ 4 (p usually 0.1).
Example: 'Rahul' vs 'Rahool' scores high because they share 'Rah'.
Why it matters: Good for short names; can over-match short strings with a shared prefix.

Levenshtein distance Fuzzy matching Lesson: Fuzzy Matching →

JSON Schema

LLM Engineering

Simple definition: A formal description of what valid JSON must look like.
Technical definition: A vocabulary (draft 2020-12 etc.) for declaring types, required properties, enums, ranges and additionalProperties that validators can check.
Example: "intent": {"enum": ["interested", "not_interested", "insufficient_evidence"]}
Why it matters: Turns 'the model usually returns the right shape' into a checkable contract.

Structured output Constrained decoding Lesson: Structured Output →

k-means

Information Extraction

also: kmeans

Simple definition: Splits points into k groups around k centre points.
Technical definition: Iteratively assigns points to the nearest centroid and recomputes centroids to minimize within-cluster squared distance; requires choosing k and assumes roughly spherical clusters.
Example: k=3 on objection embeddings.
Why it matters: Simple and fast; forces every point into a cluster (no noise label).

Clustering Silhouette score Lesson: Topic Modeling and Clustering →

Krippendorff's alpha

Evaluation

also: krippendorff alpha, alpha

Simple definition: A flexible agreement statistic that handles any number of annotators, missing labels and different data types.
Technical definition: α = 1 − D_o/D_e, the ratio of observed to expected disagreement computed from a coincidence matrix, with a distance metric for nominal, ordinal, interval or ratio data.
Example: Some annotators skipped some calls; α still works.
Why it matters: Useful when building reliable human-labelled datasets with real-world messiness.

Cohen's kappa Fleiss' kappa Lesson: Inter-Annotator Agreement →

Latency

LLM Engineering

Simple definition: How long you wait for a response.
Technical definition: End-to-end response time, typically decomposed into time-to-first-token (prefill) and per-token generation time (decode) plus network/queueing.
Example: p95 latency of 2.4s for a 300-token output.
Why it matters: Real-time agent assist has hard latency budgets; batch analytics does not.

Time to first token (TTFT)Throughput Lesson: Model Selection →

Levenshtein distance

NLP Fundamentals

also: levenshtein

Simple definition: The minimum number of single-character edits to turn one string into another.
Technical definition: Edit distance with unit-cost insertions, deletions and substitutions, computed by dynamic programming in O(mn).
Example: 'persnal' → 'personal' = 1 (insert 'o').
Why it matters: The basic building block of fuzzy matching against product names, names and keywords.

Edit distance Jaro-Winkler similarity Fuzzy matching Lesson: Fuzzy Matching →

LLM-as-a-judge

LLM Engineering

also: llm judge, llm-judge

Simple definition: Using an LLM to grade other outputs.
Technical definition: Prompting a model to score or compare candidate outputs against a rubric or reference; a scalable but biased measurement instrument that must be validated against human labels.
Example: Judge compares answer A vs B for faithfulness.
Why it matters: Cheap evaluation at scale — but not ground truth.

Position bias Verbosity bias Self-preference bias Lesson: LLM-as-a-Judge →

Logits

LLM Engineering

Simple definition: The raw scores the model gives each possible next token.
Technical definition: Unnormalized real-valued outputs of the final layer; softmax converts them into a probability distribution.
Example: interested: 4.1, insufficient: 2.9, not_interested: 1.2.
Why it matters: Temperature and constrained decoding operate on logits.

Softmax Temperature Lesson: Sampling Parameters →

Lost in the middle

LLM Engineering

Simple definition: Models often use information at the start or end of a long prompt better than in the middle.
Technical definition: An empirically observed U-shaped retrieval accuracy curve over position in long contexts (Liu et al., 2023). Magnitude varies by model and has decreased in newer models, but position effects remain worth measuring.
Example: The key 'nahi chahiye' buried at turn 60 of 120 is more likely to be missed.
Why it matters: Place critical instructions and evidence deliberately; measure on your own model.

Context window Context rot Lesson: Context Engineering →

Macro F1

Evaluation

also: macro averaging, macro f1

Simple definition: Average the F1 of each class equally, regardless of size.
Technical definition: Macro-F1 = (1/K) Σ_k F1_k; each class weighted equally.
Example: Rare 'Ambiguous' class counts as much as 'Interested'.
Why it matters: Exposes failure on rare classes that micro/accuracy hide.

Micro F1 Weighted F1 Lesson: Macro vs Micro Averaging →

Max tokens

LLM Engineering

also: max_tokens

Simple definition: The cap on how many tokens the model may generate.
Technical definition: A hard limit on output length; generation stops (with a length stop reason) when reached, possibly mid-structure.
Example: max_tokens=20 cuts a JSON object in half.
Why it matters: Too low → truncated/invalid JSON; too high → cost and latency risk.

Structured output Lesson: Sampling Parameters →

McNemar's test

Evaluation

also: mcnemar's test

Simple definition: A significance test for comparing two models on the same examples.
Technical definition: A paired test on discordant counts b (A right, B wrong) and c (A wrong, B right): χ² = (|b−c|−1)²/(b+c), or an exact binomial test.
Example: b=12, c=25 → B is better, p ≈ 0.049.
Why it matters: Uses the pairing, so it's more powerful than comparing two independent accuracies.

Statistical significance Paired test Lesson: Statistical Significance →

MECE

Information Extraction

Simple definition: Mutually Exclusive, Collectively Exhaustive: categories don't overlap and together cover everything.
Technical definition: A property of a partition: every item belongs to exactly one category. Requires an explicit 'other/insufficient' category in practice.
Example: {Interested, Not interested, Insufficient evidence} for a single dimension.
Why it matters: Non-MECE label sets make single-label annotation ambiguous.

Taxonomy Lesson: Taxonomy Design →

Metaphone

NLP Fundamentals

Simple definition: A smarter phonetic code that uses English pronunciation rules.
Technical definition: A rule-based phonetic encoding (Lawrence Philips, 1990) handling digraphs (PH→F, TH→0, SH→X), silent letters and context-dependent consonants.
Example: 'Phone' and 'Fone' → FN.
Why it matters: More accurate than Soundex for English; still English-centric for Indian names.

Soundex Phonetic matching Lesson: Phonetic Matching →

Micro F1

Evaluation

also: micro averaging, micro f1

Simple definition: Pool all decisions together, then compute F1.
Technical definition: Sum TP, FP, FN over classes and compute F1 from totals; for single-label multi-class this equals accuracy.
Example: Dominated by the 80% 'Interested' class.
Why it matters: Reflects per-instance performance; hides rare-class failures.

Macro F1 Accuracy Lesson: Macro vs Micro Averaging →

Mixture-of-Experts (MoE)

LLM Engineering

also: mixture of experts

Simple definition: A model made of many 'expert' sub-networks where only a few run for each token.
Technical definition: Sparse architecture where a router activates k of N expert feed-forward blocks per token; total parameters ≫ active parameters, lowering compute per token for a given capacity while keeping memory requirements high.
Example: A model with 8 experts, 2 active per token.
Why it matters: Can offer large-model quality at lower per-token compute; memory and serving complexity remain.

Dense model Latency Lesson: Model Selection →

Multi-class classification

NLP Fundamentals

also: multiclass

Simple definition: Choose exactly one label from several.
Technical definition: Each instance receives exactly one label from K > 2 mutually exclusive classes; typically softmax output.
Example: Interested / Not interested / Insufficient evidence.
Why it matters: Forces mutual exclusivity — wrong when labels can co-occur.

Multi-label classification Hierarchical classification Lesson: Text Classification →

Multi-label classification

NLP Fundamentals

also: multilabel

Simple definition: Each example can have any number of labels at once.
Technical definition: Each instance receives a subset of labels; usually modelled as K independent binary decisions (sigmoid outputs) and evaluated with per-label or example-based metrics.
Example: {Interested, Existing customer, Needs follow-up}.
Why it matters: Needed when a call contains several facts at once; evaluation is more complex.

Multi-class classification Hamming loss Lesson: Text Classification →

Multi-stage pipeline

LLM Engineering

also: pipelines

Simple definition: Breaking a task into several steps, each handled by a component.
Technical definition: A DAG of components (deterministic code, classifiers, LLM calls) with explicit interfaces; enables per-stage evaluation, routing and parallelism.
Example: Normalize → NER → Classify → Extract evidence → Verify.
Why it matters: Decomposition improves debuggability but adds latency, cost and error compounding.

Router Error propagation Lesson: Multi-stage Pipeline Design →

n-gram

NLP Fundamentals

Simple definition: A sequence of n consecutive characters or words.
Technical definition: Contiguous subsequences of length n used as features for similarity, language modelling and indexing.
Example: Character bigrams of 'loan': lo, oa, an.
Why it matters: Character n-grams make similarity robust to small spelling errors.

Named Entity Recognition (NER)

NLP Fundamentals

also: named entity recognition

Simple definition: Finding and labelling things like amounts, names, dates and products in text.
Technical definition: Sequence labelling that identifies entity spans and assigns types, typically framed as token classification with BIO tags or as span/LLM extraction.
Example: '5 lakh' → AMOUNT; '10.5 percent' → RATE.
Why it matters: Entities fill structured fields used for routing and decisions.

Entity span BIO tagging Gazetteer Lesson: Named Entity Recognition →

Natural Language Inference (NLI)

LLM Engineering

Simple definition: Deciding if a claim is supported, contradicted, or neither, given a text.
Technical definition: A three-way classification task (entailment / contradiction / neutral) over premise–hypothesis pairs, often used to verify generated claims.
Example: Premise: 'I might consider it'. Hypothesis: 'Customer will apply'. → neutral.
Why it matters: A reusable verification step in extraction pipelines.

Entailment Lesson: Grounding and Faithfulness →

Open-set extraction

Information Extraction

also: open set

Simple definition: Let the model describe categories in its own words and discover new ones.
Technical definition: Generation or clustering where categories are not fixed a priori; outputs must be normalized/clustered to be aggregated.
Example: Model outputs 'wants lower EMI' and 'EMI too high' as separate reasons.
Why it matters: Discovers unknown issues but drifts and fragments without governance.

Closed-set classification Clustering Lesson: Open-set vs Closed-set Classification →

p-value

Evaluation

Simple definition: How surprising your result would be if there were truly no difference.
Technical definition: P(test statistic at least as extreme as observed | null hypothesis). Not the probability the null is true.
Example: p = 0.03 → such a gap would arise by chance ~3% of the time if the prompts were equal.
Why it matters: Commonly misinterpreted; pair it with effect sizes and CIs.

Statistical significance Lesson: Statistical Significance →

Paired test

Evaluation

Simple definition: A comparison that uses the fact both systems saw the same inputs.
Technical definition: Tests on per-item differences (McNemar, paired bootstrap, sign test) that remove between-item variance.
Example: Both prompts run on the same 300 calls.
Why it matters: Detects real differences with far fewer examples.

McNemar's test Bootstrap Lesson: Statistical Significance →

PCA

Information Extraction

also: principal component analysis

Simple definition: Finds the directions along which the data varies the most, to plot it in 2D.
Technical definition: Principal Component Analysis: projects data onto top eigenvectors of the covariance matrix; linear and variance-preserving.
Example: 384-dim embeddings projected to 2D for visualization.
Why it matters: Quick, deterministic visualization; can hide non-linear structure.

UMAP Embedding Lesson: Embeddings and Semantic Similarity →

Per-stage evaluation

Evaluation

also: per stage evaluation, component evaluation

Simple definition: Measuring each pipeline step separately, not just the final answer.
Technical definition: Evaluating each component on its own gold labels (and optionally with gold upstream inputs) to attribute end-to-end errors to stages.
Example: Speaker attribution 0.93, NER 0.88, classification given gold inputs 0.91.
Why it matters: Tells you what to fix first.

Error propagation Error analysis Lesson: Per-Stage Evaluation →

Phonetic matching

NLP Fundamentals

Simple definition: Matching words by how they sound instead of how they are spelled.
Technical definition: Encoding strings into phonetic keys and comparing keys (or key similarity) instead of surface forms.
Example: 'Mohammed' and 'Muhammad' share Soundex M530.
Why it matters: Handles transliteration variation in names; produces false matches for different words that sound alike.

Soundex Metaphone Lesson: Phonetic Matching →

Position bias

LLM Engineering

Simple definition: A judge preferring whichever answer is shown first (or second).
Technical definition: Systematic dependence of pairwise judgments on presentation order, detectable by swapping order and measuring inconsistency.
Example: A wins when shown first; B wins when order is swapped.
Why it matters: Always evaluate both orders.

LLM-as-a-judge Lesson: LLM-as-a-Judge →

Precision

Evaluation

also: ppv

Simple definition: Of the items the model flagged, how many were actually right?
Technical definition: Precision = TP / (TP + FP); the positive predictive value.
Example: Model flags 50 interested customers, 40 truly are → 0.80.
Why it matters: High precision means sales teams don't waste calls on false leads.

Recall F1 score Confusion matrix Lesson: Classification Metrics →

Prompt bloat

LLM Engineering

Simple definition: Prompts growing with rules and examples until they are expensive and confusing.
Technical definition: Accretion of instructions, redundant examples and irrelevant context over iterations, increasing cost/latency and often introducing contradictory rules.
Example: A 12,000-token rulebook where 90% is irrelevant to intent classification.
Why it matters: Every token is paid for on every call.

Context rot Prompt caching Lesson: Context Engineering →

Prompt caching

LLM Engineering

also: caching, prefix caching

Simple definition: Reusing the already-processed beginning of a prompt across many calls.
Technical definition: Providers store the attention key/value state for a prompt prefix; later requests with an identical prefix skip recomputation and are billed at a discounted cached-input rate (cache writes may cost a premium; caches expire).
Example: A 5,000-token rulebook shared by 100,000 calls.
Why it matters: Large cost and latency savings — only if the static content comes first and is byte-identical.

Time to first token (TTFT)Batching System prompt Lesson: Prompt Caching and Batching →

Prompt overfitting

Evaluation

also: test set overfitting, winner's curse

Simple definition: Tweaking a prompt until it scores well on your test set but not on new data.
Technical definition: Selection bias from repeatedly evaluating prompt variants on the same labelled set; the best observed score is an upward-biased estimate of true performance (winner's curse).
Example: Visible test 0.91 → hidden test 0.83.
Why it matters: Makes every reported number untrustworthy.

Train / dev / test split Lesson: Gold Set Design →

Recall

Evaluation

also: sensitivity, tpr, true positive rate

Simple definition: Of the items that were actually positive, how many did the model find?
Technical definition: Recall = TP / (TP + FN); sensitivity / true positive rate.
Example: 60 truly interested customers, model finds 40 → 0.67.
Why it matters: High recall means you don't miss genuine opportunities.

Precision F1 score Lesson: Classification Metrics →

Reliability diagram

Evaluation

also: calibration curve

Simple definition: A chart of predicted confidence vs actual accuracy.
Technical definition: Plot of per-bin empirical accuracy against mean confidence; the diagonal is perfect calibration; below the diagonal = overconfident.
Example: Points under the diagonal at high confidence mean the model is overconfident.
Why it matters: The first plot to draw before trusting confidence scores.

Calibration Expected Calibration Error (ECE)Lesson: Calibration →

Retrieval-Augmented Generation (RAG)

LLM Engineering

also: retrieval augmented generation

Simple definition: Looking up relevant documents and adding them to the prompt before the model answers.
Technical definition: A pattern where a retriever (lexical, dense or hybrid) selects passages from an external corpus and injects them into the context for grounded generation.
Example: Retrieving the current rate sheet to check whether the agent quoted the correct rate.
Why it matters: Only needed when the answer depends on knowledge not already in the input.

Agent Semantic search Embedding Lesson: RAG and Agents: Do You Need Them? →

Romanization

NLP Fundamentals

also: transliteration, romanized hindi

Simple definition: Writing a language like Hindi using the Latin alphabet.
Technical definition: Transliteration of a non-Latin script into Latin characters. Informal romanization has no standard spelling, producing many variants for one word.
Example: 'chahiye', 'chaiye', 'chahie', 'chahye' all mean 'need'.
Why it matters: Exact-match rules and keyword lists silently miss variants.

Text normalization Hinglish Code-switching Lesson: Text Normalization →

Root-cause analysis

Evaluation

also: root cause

Simple definition: Finding the underlying reason for failures, not just where they appear.
Technical definition: Iteratively testing hypotheses (e.g., controlling for confounders) to identify the causal factor behind a slice's degraded performance.
Example: Agent B's low F1 is really caused by Agent B handling more noisy calls.
Why it matters: Prevents fixing the wrong thing.

Error analysis Slice Lesson: Error Analysis →

Router

LLM Engineering

also: routing

Simple definition: A step that sends each input down the appropriate path.
Technical definition: A classifier or rule that dispatches inputs to specialized sub-pipelines (e.g., cheap model for simple cases, strong model or human for hard ones).
Example: Short, clean calls → small model; ambiguous → larger model + human review.
Why it matters: Cuts cost while protecting quality on hard cases.

Multi-stage pipeline Lesson: Multi-stage Pipeline Design →

Sample size

Evaluation

Simple definition: How many examples you evaluate on.
Technical definition: N determines standard error (∝ 1/√N) and therefore CI width and power.
Example: Going from 100 to 400 examples halves the CI width.
Why it matters: Plan it before running experiments.

Statistical power Confidence interval Lesson: Statistical Significance →

Self-preference bias

LLM Engineering

Simple definition: A model judging its own outputs (or its family's) more favourably.
Technical definition: Judges assigning higher scores to outputs with lower perplexity under the judge model, often those generated by the same model family.
Example: Model X rates Model X's summaries higher than humans do.
Why it matters: Use a different family as judge or validate against humans.

LLM-as-a-judge Lesson: LLM-as-a-Judge →

Semantic search

Information Extraction

Simple definition: Finding text by meaning rather than exact words.
Technical definition: Ranking documents by embedding similarity to a query embedding, often with approximate nearest-neighbour indexes; commonly combined with lexical search (hybrid).
Example: Query 'customer can't afford the EMI' finds 'bahut mehenga hai'.
Why it matters: Handles paraphrase and some cross-lingual variation.

Embedding Retrieval-Augmented Generation (RAG)Lesson: Embeddings and Semantic Similarity →

SentencePiece

NLP Fundamentals

Simple definition: A tokenizer toolkit that treats text as a raw stream (including spaces), so it works for any language.
Technical definition: A language-independent subword tokenizer library implementing BPE and Unigram-LM; it encodes whitespace as a visible symbol (▁) and does not require pre-splitting on spaces.
Example: '▁personal▁loan' — the ▁ marks where a space was.
Why it matters: Common in multilingual models; it handles scripts without spaces and code-mixed text more uniformly.

Byte-Pair Encoding (BPE)Tokenization Subword tokenization Lesson: Tokenization →

Silhouette score

Information Extraction

Simple definition: How well each point fits its own cluster versus the nearest other cluster.
Technical definition: s = (b − a) / max(a, b), where a is mean intra-cluster distance and b is mean distance to the nearest other cluster; averaged over points, in [−1, 1].
Example: k=3 gives 0.41, k=6 gives 0.22.
Why it matters: A heuristic for choosing k — not a substitute for reading clusters.

k-means Lesson: Topic Modeling and Clustering →

Slice

Evaluation

also: slicing, data slice

Simple definition: A subset of data defined by some attribute.
Technical definition: A metadata-defined subpopulation (agent, language, quality, segment) over which metrics are computed separately.
Example: All noisy Hinglish outbound calls.
Why it matters: Aggregate metrics hide slice-level failures.

Error analysis Lesson: Error Analysis →

Softmax

LLM Engineering

Simple definition: Turns scores into probabilities that sum to 1.
Technical definition: softmax(z)_i = exp(z_i) / Σ_j exp(z_j).
Example: Logits [2, 1, 0] → [0.67, 0.24, 0.09].
Why it matters: Explains why temperature changes the whole distribution, not just the top token.

Logits Temperature Lesson: Sampling Parameters →

Soundex

NLP Fundamentals

Simple definition: An old algorithm that gives similar-sounding names the same code.
Technical definition: Keeps the first letter, maps consonants to 6 digit classes (B/F/P/V=1, C/G/J/K/Q/S/X/Z=2, D/T=3, L=4, M/N=5, R=6), drops vowels/H/W/Y, collapses adjacent duplicates, pads to 4 characters.
Example: Rahul, Raahul, Rahool → R400.
Why it matters: Useful for name matching despite spelling variation; designed for English surnames.

Metaphone Phonetic matching Lesson: Phonetic Matching →

Speaker attribution

NLP Fundamentals

Simple definition: Deciding which person (agent or customer) said each line.
Technical definition: Assigning speaker identities or roles to utterances, from audio diarization and/or text cues (questions, politeness markers, domain phrases).
Example: 'aap salaried ho?' is almost certainly the agent.
Why it matters: If the agent's words are attributed to the customer, the system fabricates customer intent.

Speaker diarization Error propagation Lesson: Speaker Attribution from Text →

Speaker diarization

NLP Fundamentals

Simple definition: Figuring out 'who spoke when' in audio.
Technical definition: Segmenting audio into speaker-homogeneous regions and clustering them into speaker identities; distinct from assigning roles (Agent vs Customer).
Example: Diarization says 'Speaker 1' and 'Speaker 2'; role attribution decides Speaker 1 is the Agent.
Why it matters: Diarization errors swap who said what — a major source of wrong evidence.

Speaker attribution ASR / Speech-to-text Lesson: Speaker Attribution from Text →

Specificity

Evaluation

also: tnr

Simple definition: Of the truly negative items, how many did the model correctly leave alone?
Technical definition: TN / (TN + FP); true negative rate; 1 − false positive rate.
Example: 930 of 940 uninterested customers correctly not flagged → 0.989.
Why it matters: Often high by default under imbalance; watch precision instead.

Precision Recall Lesson: Classification Metrics →

Statistical power

Evaluation

also: power

Simple definition: The chance your experiment detects a real difference if one exists.
Technical definition: 1 − β: probability of rejecting the null when a specified alternative is true; increases with sample size and effect size.
Example: Detecting a 2-point accuracy gain needs thousands of examples.
Why it matters: Small eval sets can't distinguish close prompts.

Sample size Lesson: Statistical Significance →

Statistical significance

Evaluation

also: significance

Simple definition: Evidence that a difference is unlikely to be pure chance.
Technical definition: A result where the p-value under a null hypothesis (e.g., no difference) falls below a pre-chosen α; says nothing about effect size or practical importance.
Example: Prompt B beats A by 3 points, p = 0.21 → not significant.
Why it matters: Stops you shipping noise.

p-value Confidence interval McNemar's test Lesson: Statistical Significance →

Stratified sampling

Evaluation

also: stratification

Simple definition: Sampling so each important group is represented in known proportions.
Technical definition: Partition the population into strata (by label, language, agent…) and sample within each; use sampling weights to recover population estimates.
Example: Sample 50 from each of 'clean' and 'noisy' transcripts.
Why it matters: Guarantees enough rare examples to measure them.

Class imbalance Gold set Lesson: Class Imbalance →

Structured output

LLM Engineering

also: json output

Simple definition: Making the model return machine-readable data like JSON.
Technical definition: Constraining or instructing generation to follow a schema (JSON Schema, function/tool signatures), optionally enforced by constrained decoding and always validated after generation.
Example: {"intent": "interested", "amount": 500000}
Why it matters: Downstream code needs reliable fields, types and enums.

JSON Schema Constrained decoding Lesson: Structured Output →

Subword tokenization

NLP Fundamentals

Simple definition: Breaking rare words into smaller known pieces instead of treating them as unknown.
Technical definition: A vocabulary strategy between character-level and word-level models: frequent strings become single units, rare strings decompose into multiple units, eliminating out-of-vocabulary failures.
Example: 'chahiye' → 'ch' + 'ahi' + 'ye'.
Why it matters: Noisy and code-mixed text never breaks the tokenizer, but it costs more tokens.

Byte-Pair Encoding (BPE)Token Lesson: Tokenization →

System prompt

LLM Engineering

also: system instruction

Simple definition: Standing instructions that set the model's role and rules for the whole conversation.
Technical definition: A privileged message placed before user turns that conditions all subsequent generation; often holds role, policy, output format and constraints.
Example: 'You are an auditor of loan-sales calls. Only use evidence spoken by the customer.'
Why it matters: Stable instructions belong here — and form a cacheable prefix.

Few-shot prompting Prompt caching Lesson: Prompt Engineering →

Taxonomy

Information Extraction

Simple definition: An organized set of categories, often a tree.
Technical definition: A controlled vocabulary of labels with defined relations (parent/child, exclusivity) and definitions, used consistently for annotation and prediction.
Example: Loan → {Personal, Home, Vehicle}.
Why it matters: Bad taxonomies create unresolvable annotation disagreements and meaningless metrics.

MECE Hierarchical classification Open-set extraction Lesson: Taxonomy Design →

Temperature

LLM Engineering

Simple definition: A knob that changes how strongly the model prefers its most likely next token.
Technical definition: Logits are divided by T before softmax: p_i ∝ exp(z_i/T). T<1 sharpens the distribution, T>1 flattens it; T→0 approaches greedy argmax decoding.
Example: At T=0.2 the model almost always outputs 'interested'; at T=1.5 lower-ranked labels appear.
Why it matters: Controls variability; low temperature suits classification and extraction.

Top-p (nucleus) sampling Softmax Logits Lesson: Sampling Parameters →

Temperature scaling

Evaluation

Simple definition: A simple fix that stretches or squashes confidence scores to be more honest.
Technical definition: Post-hoc calibration dividing logits by a scalar T fitted on a validation set (minimizing NLL); preserves ranking.
Example: T=1.8 softens an overconfident classifier.
Why it matters: Cheap recalibration when you have logits or scores.

Calibration Temperature Lesson: Calibration →

Text normalization

NLP Fundamentals

also: text normalization

Simple definition: Cleaning text into a more consistent form before processing.
Technical definition: Deterministic or learned transformations — case folding, Unicode normalization, punctuation/whitespace canonicalization, spelling correction, transliteration, abbreviation expansion — that map surface variants to a canonical representation.
Example: 'personal lon ke liye' → 'personal loan ke liye'.
Why it matters: Improves matching and consistency, but every step can destroy information (negation, emphasis, casing).

Romanization Levenshtein distance Lesson: Text Normalization →

TF-IDF

Information Extraction

also: tfidf

Simple definition: Weights words by how often they appear in a document, discounted by how common they are overall.
Technical definition: tf(t,d)·log(N/df(t)); produces sparse lexical vectors; similarity depends on shared surface forms.
Example: 'loan' is common (low idf); 'foreclosure' is rare (high idf).
Why it matters: A strong, cheap baseline — but blind to synonyms.

Embedding Class-based TF-IDF (c-TF-IDF)Lesson: Embeddings and Semantic Similarity →

Throughput

LLM Engineering

Simple definition: How much work the system completes per unit time.
Technical definition: Requests or tokens processed per second across concurrent requests; improved by batching at the cost of per-request latency.
Example: 10,000 transcripts/hour.
Why it matters: Batch workloads optimize throughput and cost, not latency.

Batching Latency Lesson: Model Selection →

Time to first token (TTFT)

LLM Engineering

Simple definition: How long until the first word of the answer appears.
Technical definition: Latency of the prefill phase (processing the prompt) plus queueing; grows with input length and is reduced by prompt caching.
Example: A 6,000-token rulebook prompt has a noticeably longer TTFT than a 500-token prompt.
Why it matters: Dominant latency term for long-input, short-output tasks like classification.

Latency Prompt caching Lesson: Prompt Caching and Batching →

Token

NLP Fundamentals

also: tokens

Simple definition: The unit of text a language model actually reads — often a word piece, not a whole word.
Technical definition: An integer-indexed entry in a model's vocabulary produced by a tokenizer (e.g. byte-level BPE). Models consume and emit sequences of token IDs; pricing and context limits are measured in tokens.
Example: 'personal' may be one token while 'persnal' may be split into 'pers' + 'nal'.
Why it matters: Cost, latency, context limits and chunking are all computed in tokens, not words or characters.

Tokenization Byte-Pair Encoding (BPE)Context window Token ID Lesson: Tokenization →

Token ID

NLP Fundamentals

Simple definition: The number that represents a token inside the model.
Technical definition: An integer index into the vocabulary/embedding matrix. The model never sees characters — only these IDs.
Example: ' loan' → 11057 (the actual ID depends on the tokenizer).
Why it matters: Debugging tokenization means looking at IDs and their decoded strings.

Token Vocabulary Lesson: Tokenization →

Tokenization

NLP Fundamentals

also: tokenizer

Simple definition: Splitting text into the pieces a model understands.
Technical definition: A deterministic mapping from a string to a sequence of vocabulary IDs, typically via pre-tokenization (regex splitting) followed by a subword algorithm such as BPE or Unigram LM.
Example: 'mujhe loan chahiye' → ['m','ujhe',' loan',' ch','ahi','ye'] (exact splits depend on the tokenizer).
Why it matters: Different scripts and spellings produce very different token counts for the same meaning.

Token Byte-Pair Encoding (BPE)SentencePiece Subword tokenization Lesson: Tokenization →

Top-p (nucleus) sampling

LLM Engineering

also: nucleus sampling, top p

Simple definition: Only sample from the smallest set of tokens that together cover p of the probability.
Technical definition: Sort tokens by probability, keep the minimal prefix whose cumulative mass ≥ p, renormalize, sample.
Example: With p=0.9, rare tail tokens are never chosen.
Why it matters: Cuts off unlikely nonsense while keeping some diversity.

Temperature Lesson: Sampling Parameters →

Topic modeling

Information Extraction

also: topic model, topic modelling

Simple definition: Automatically finding the main themes in a collection of texts.
Technical definition: Unsupervised discovery of latent themes, classically LDA (word co-occurrence) and more recently embedding-based methods like BERTopic.
Example: Themes across 50k calls: rate concern, documentation, timing.
Why it matters: Feeds open-set discovery and taxonomy revision.

BERTopic Clustering Lesson: Topic Modeling and Clustering →

Topic shift

NLP Fundamentals

Simple definition: The moment the conversation moves to a new subject.
Technical definition: A boundary where the discourse topic changes, often signalled by markers ('aur', 'by the way', 'ek aur baat').
Example: 'ok aur credit card ka bhi bata do' shifts from top-up to credit card.
Why it matters: A single call can contain multiple intents for multiple products.

Conversation segmentation Lesson: Dialogue and Discourse Structure →

Train / dev / test split

Evaluation

also: data split, dev set, test set, validation set

Simple definition: Separate data for building, tuning, and final checking.
Technical definition: Disjoint partitions: train (fit/prompt examples), development (iterate/tune), test (one-shot final estimate). Repeated use of the test set turns it into a dev set.
Example: Few-shot examples come from train; prompt iterations use dev; test is touched once.
Why it matters: Prevents fooling yourself with optimistic numbers.

Gold set Prompt overfitting Lesson: Gold Set Design →

Turn-taking

NLP Fundamentals

Simple definition: How speakers alternate in a conversation.
Technical definition: The organization of speaker changes, including overlaps, interruptions and back-channels ('haan', 'ji').
Example: Back-channel 'hmm' from the customer while the agent explains is not agreement.
Why it matters: Transcripts merge or split turns incorrectly; downstream logic must be robust to it.

Adjacency pair Speaker diarization Lesson: Dialogue and Discourse Structure →

UMAP

Information Extraction

Simple definition: A method to squash high-dimensional vectors into 2–5 dimensions while keeping neighbours close.
Technical definition: Uniform Manifold Approximation and Projection: builds a fuzzy k-NN graph and optimizes a low-dimensional layout preserving local structure.
Example: 384-dim embeddings → 5 dims before clustering.
Why it matters: Makes density clustering work better; distances in the plot are not globally meaningful.

BERTopic PCA Lesson: Topic Modeling and Clustering →

Verbosity bias

LLM Engineering

Simple definition: A judge preferring longer answers even when they're not better.
Technical definition: Correlation between judge scores and output length independent of quality.
Example: A padded, confident explanation beats a short correct label.
Why it matters: Can push your system toward long, costly, less precise outputs.

LLM-as-a-judge Lesson: LLM-as-a-Judge →

Vocabulary

NLP Fundamentals

Simple definition: The fixed list of tokens a model knows.
Technical definition: The finite set of symbols (typically 32k–200k+) the tokenizer can output; each entry maps to an embedding row in the model.
Example: o200k_base has roughly 200,000 entries.
Why it matters: Larger vocabularies usually compress multilingual text better, at the cost of larger embedding tables.

Token Token ID Lesson: Tokenization →

Weighted F1

Evaluation

also: weighted f1

Simple definition: Average per-class F1 weighted by how common each class is.
Technical definition: Σ_k (n_k/N) F1_k; a compromise between macro and micro.
Example: Close to the majority class's F1 on skewed data.
Why it matters: Often reported by default (e.g., sklearn 'weighted avg') — know what it hides.

Macro F1 Micro F1 Lesson: Macro vs Micro Averaging →

Word Error Rate (WER)

NLP Fundamentals

Simple definition: The percentage of words the transcription got wrong.
Technical definition: WER = (S + D + I) / N, where S, D, I are substitutions, deletions, insertions from a minimum edit alignment and N is the number of reference words.
Example: Reference 'personal loan chahiye', hypothesis 'personal lone chahiye' → WER = 1/3.
Why it matters: Transcript quality is a key slicing dimension in error analysis.

ASR / Speech-to-text Levenshtein distance

Zero-shot prompting

LLM Engineering

also: zero shot

Simple definition: Asking the model to do a task with instructions only, no examples.
Technical definition: Task specification purely via natural-language instructions without in-context demonstrations.
Example: 'Classify the customer's intent as interested, not_interested or insufficient_evidence.'
Why it matters: Fast to write; weaker on ambiguous boundaries and custom label definitions.

Few-shot prompting In-context learning Lesson: Prompt Engineering →