Level 2: Generative AI — Cheat Sheet

Key terms

Generative AI

Machine learning that creates new content (text, images, audio, video, code) by learning the deep statistical structure of its training material and sampling new examples from it. Think of it as the composer to predictive AI's music critic: same training material, radically different output.

Predictive AI

The Level 1 style of AI that judges content rather than making it, outputting a label or number like "spam or not" or "what price." The shift from judging to creating is what made AI explode into public consciousness in 2022.

Diffusion

The image-generation technique where a model learns to predict what noise was added to an image, then runs backwards from pure noise, step by step, until an image emerges. Prediction repeated becomes generation.

Multimodal AI

A single model handling text, images, and audio together, like GPT-4o, Claude, or Gemini. It is the same core idea — learned structure plus iterative prediction — wearing different clothes. (see AI-052)

Hallucination

Fluent, confident, fabricated output. It's the truth-optional failure mode that classifiers never had, which is why verification shifts to the human and judgment became the scarce skill. (see AI-060)

Provenance

The question of whether content was made by a person or a machine. Watermarking and detection remain unsolved, and deepfakes make this a live societal issue.

Deepfake

Machine-generated media convincingly imitating a real person's face or voice. A direct consequence of generation quality reaching photorealism and voice-cloning fidelity.

Transformer

The 2017 architecture behind essentially all modern AI. GPT's T stands for it. Its two winning properties are attention (distance-free access between words) and parallel-friendly training that scales to trillions of words.

Attention

The mechanism where each token asks "which other words should influence my interpretation, and how much?", computing a relevance score against every other token and blending their information in proportion. For "it" in the trophy sentence, attention scores "trophy" high, so the blended representation of "it" now means trophy. (see AI-013)

Attention Head

One of dozens of parallel attention patterns per layer, each free to track a different relationship: grammar, coreference, tone. Heads stack across dozens of layers, with early layers linking neighbors and deep layers connecting whole-document themes.

RNN (Recurrent Neural Network)

The pre-2017 approach that read text left to right, compressing everything into one running summary, like meeting notes passed down a line of people, with nuance fading over distance. Its sequential reading also made web-scale training impractically slow.

Parallelism

The transformer's other half: every word attends to every word simultaneously, so an entire document trains in one parallel step, mapping perfectly onto GPUs. No parallelism, no LLMs: it unlocked the scaling laws. (see AI-009)

Positional Encoding

The position marker each token carries, needed because all words are processed at once rather than in order. Without it the model could not tell "dog bites man" from "man bites dog."

Context Window

The size of the roundtable: how many tokens can attend to each other at once. Attention's cost grows fast with table size, which is why long context is expensive. (see AI-061)

Embedding

The number-vector each token is mapped to before entering the attention layers, capturing meaning in a form the network can process. (see AI-014)

Query

The "what am I looking for?" vector each token sends out, like bringing a search request to a library. It is created by multiplying the token's embedding by a learned weight matrix.

Key

The "here's what I'm about" vector each token broadcasts, the spine label every book advertises. Queries are scored against all keys to find relevant tokens.

Value

The actual information a token hands over when its key matches a query well. The new representation of a token is a percentage-weighted blend of other tokens' values — "it" becomes 60% mat, 25% cat.

Multi-Head Attention

Running many parallel attention patterns per layer (32 to 128 in large models), each with its own query/key/value matrices. Heads specialize (subject↔verb agreement, pronoun links, quote pairing) without anyone assigning roles; specialization emerges from training pressure.

Causal Masking

The constraint that each token may attend only to tokens before it, so a generator never peeks at the future. This single restriction is what makes left-to-right generation possible; embedding models that only read skip it. (see AI-014)

Softmax

The normalization step that turns raw relevance scores into percentages summing to 1, so value blending has well-defined weights.

Quadratic Cost

Attention's scaling problem: every token scores against every other, so 100,000 tokens means 10 billion comparisons. It is the engineering reason context windows were once tiny and long context is priced steeply. (see AI-061)

KV Caching

Storing every token's keys and values so each newly generated token only computes its own. It's the optimization that makes chatbot streaming affordable, and the reason long conversations consume growing GPU memory.

Embedding

A list of numbers, typically ~1,536 learned axes, positioning any content on a map of meaning where similar things land close together. "Puppy" sits beside "dog," across town from "carburetor," and finding similar things becomes finding nearby dots.

Cosine Similarity

The cheap arithmetic (one multiplication per axis, summed) that measures how close two vectors are in meaning. Your instant ranking of "a delicious dinner was prepared" as closest to "the chef cooked a wonderful meal" is what cosine similarity computes.

Semantic Search

Search where query and documents both become vectors and results are nearest neighbors, finding "reset my password" when you typed "can't log in," no shared keywords needed. It still misses exact identifiers, which is why production systems add hybrid keyword search. (see AI-015)

Vector Arithmetic

The party trick that directions carry meaning: king − man + woman ≈ queen, Paris − France + Japan ≈ Tokyo. Relationships are arrows and categories are neighborhoods, and stereotype directions ride along too. (see AI-062)

Chunking

Splitting documents into paragraph-sized pieces before embedding, because whole documents blur into mushy vectors. Chunk strategy is a core retrieval-quality lever. (see AI-016)

Embedding Model

The model that converts content into vectors: one API call (OpenAI, Cohere, Voyage) or a free local sentence-transformers model. Queries and documents must use the same model; mixing models breaks similarity entirely.

Dimension

One learned axis of the embedding space, uninterpretable individually but collectively capturing topic, tone, syntax, and subtler shades. More dimensions mean finer meaning but more storage and slower search. (see AI-015)

Vector Database

A store that holds millions of embeddings and answers "find the most similar vectors to this one" in milliseconds. It's the search infrastructure that turns embeddings into products. When your chatbot "searches the company docs," a vector database answered. (see AI-014)

Approximate Nearest Neighbor (ANN)

The core trick: accept near-perfect results (~95–99% recall) in exchange for cutting a million comparisons down to a tiny fraction. The smart librarian who walks you to the right neighborhood instead of checking every shelf.

HNSW

The dominant ANN index, a multi-layer "highway network" over the vectors, with coarse express links on top and local streets below. Queries ride the highways to the right region, then explore locally, giving millisecond search over millions of vectors.

Recall

The fraction of true nearest neighbors your index actually finds: the universal dial traded against speed. Production teams tune it with measurements on their own data, not guesses. (see AI-025)

Metadata Filtering

Combining similarity with structured filters, like "nearest neighbors where team = legal and year ≥ 2024." Real queries almost always need it, and databases differ sharply in filtering without wrecking recall.

Hybrid Search

Running vector and keyword search together and merging results, often with a reranker. Vectors miss exact identifiers like "error TS-4102"; keywords miss paraphrases — every real corpus needs both.

pgvector

A free Postgres extension that became the default vector store for startups, the boring option already in your stack. The honest advice: start with pgvector or Chroma, and graduate to Pinecone/Weaviate/Qdrant/Milvus only when scale demands it.

Chunk

A ~200–500 token passage split from a document before embedding, small enough to be precise, big enough to carry meaning. The first step of every vector-search pipeline: chunk → embed → index → query. (see AI-016)

Retrieval-Augmented Generation (RAG)

The open-book exam for LLMs: retrieve the most relevant passages from your knowledge base, paste them into the prompt, and instruct the model to answer from them with citations. It fixes knowledge cutoff, private-data access, and hallucination in one architecture, making it the most deployed LLM pattern in industry.

Retrieval

The "R" stage: embed the question, pull the top-k nearest chunks from the vector database. Most bad RAG answers are retrieval misses: the right passage never reached the prompt, so debug this stage first. (see AI-015)

Grounding

Anchoring the model's answer to supplied passages rather than its weights, enforced with prompt guards like "Answer only from the provided context; if it doesn't contain the answer, say so." (see AI-060)

Reranker

A second model that reorders the retrieved top-50 into a sharper top-5, one of the highest-leverage fixes for retrieval misses.

Chunking

Splitting the knowledge base into passages that respect paragraph and section boundaries, with overlapping edges and titles attached. A passage cut mid-sentence carries broken meaning. Bad chunks are a diagnosable RAG failure stage.

Query Rewriting

Transforming a vague user question ("what about part-timers?") into an explicit, retrievable one before embedding it — a standard lever when conversational follow-ups miss.

Fine-tuning

The alternative knowledge path that retrains model weights — days of work versus minutes to re-index a document. The standard guidance: RAG for knowledge and freshness, fine-tuning for style and behavior; they compose. (see AI-051)

Groundedness

The evaluation metric for the generation stage: did the answer actually come from the retrieved passages? Distinct from retrieval recall; measure both separately or you will fix the wrong stage. (see AI-025)

AI Agent

An AI system that receives a goal rather than a question, then reasons, plans, uses tools, and performs multiple actions until the goal is achieved. Unlike an assistant that answers once and stops, an agent runs the reason→act→observe loop with minimal human guidance.

Agent Loop

The repeating cycle Goal → Reason → Plan → Choose Tool → Act → Observe → Evaluate → Repeat-or-Finish. The flight-booking worked example runs it in six steps with two tools, and shows every place it can derail without checks and gates.

Planning

Breaking a large goal into smaller executable tasks, like "prepare presentation" becoming collect data → summarize → create slides → review. Poor planning is the most common way agents produce poor outcomes.

Tool

An external capability the agent can invoke: web search, calculator, calendar, database, code execution. Without tools an agent can only reason; with tools it can act. (see AI-018)

Memory

The agent's retention of previous conversations, decisions, retrieved documents, and intermediate results across steps, which prevents repeating work within a task.

Reflection

The agent evaluating its own work after a step: did I achieve the goal, is information missing, should I try another approach? It creates an iterative improvement loop.

Multi-Agent System

Multiple specialized agents collaborating, such as Planner → Research → Writer → Reviewer, each owning one responsibility. Contrast with a single agent doing the entire task.

Step Budget

A maximum-iterations cap on the loop, without which agents can spin forever on impossible tasks. One of the basic guardrails alongside limited tool permissions, action logs, and approval gates for high-impact actions. (see AI-023)

Function Calling

The capability where a model, instead of writing prose, outputs a structured request like {"tool": "get_weather", "args": {"city": "Chennai"}} for your application to execute. The critical division of labor: the model only ever writes JSON; your code does the actual calling.

Function

A reusable piece of software with defined inputs and outputs: get weather, send email, create invoice. Like the restaurant waiter, the LLM coordinates which function to call but never cooks the food itself.

Parameter

A named input a function requires, like Title, Date, and Participants for create_calendar_event(). The LLM extracts these values from natural language, and it fills fields exactly as well as the schema describes them: "date (YYYY-MM-DD, must be future)" beats "date".

Schema

The declared name, parameters, and descriptions of a tool the model can call. Vague schemas produce guessed or hallucinated arguments; registering fifty tools at once dilutes selection accuracy. (see AI-020)

API (Application Programming Interface)

The interface a function typically calls behind the scenes: Calendar API, Jira API, Payment API. The AI decides which to use; the application performs the secure communication.

Validation

Treating model-generated arguments as untrusted user input: validate, sanitize, and authorize before executing. Applications, never the model, remain responsible for authentication, authorization, and error handling. (see AI-027)

Round Trip

The full flow: user request → model emits a call → your code executes it → result appended to the conversation → model composes the final answer. Two model calls, one tool execution.

Error Path

Returning a structured error when an API fails so the model can recover or apologize, instead of observing a false "success" that sends the reasoning off a cliff.

Model Context Protocol (MCP)

An open standard, the USB-C for AI tools, that lets AI applications connect to external tools, data sources, and applications through one consistent protocol instead of bespoke integrations. It standardizes tool discovery, descriptions, execution, resource access, prompts, and errors.

MCP Client

The component inside the AI application (Claude Desktop, an AI IDE, a custom agent) that discovers available servers, requests resources, executes tools, and exchanges messages with MCP Servers.

MCP Server

A domain-focused service exposing Tools, Resources, and Prompts: a GitHub server, Jira server, filesystem server. One well-built server serves many clients: the worked example's ticket server powers chat, editor, and overnight triage bot from a single integration.

Resource

A readable data source an MCP Server exposes — documentation, logs, configuration, source code. Unlike tools, resources are read rather than executed.

Prompt (MCP)

A reusable prompt template a server can expose, like a Code Review Prompt or Security Checklist, enabling consistent AI workflows across teams.

M×N Problem

The integration explosion MCP collapses: M AI apps × N data sources means M×N custom integrations without a standard, but only M+N with one. That arithmetic is the protocol's entire value proposition.

Tool Scoping

Exposing narrow, purpose-built operations instead of overly powerful ones — a run_sql tool hands the model your whole database. The server enforces permissions, never the model. (see AI-027)

Function Calling

The model-side mechanism for emitting structured tool requests; MCP is the standardized plumbing that delivers tools and resources to any client. Confusing the two is the most common mistake in this lesson. (see AI-018)

Context Engineering

The discipline of deciding what enters the model's window (instructions, tools, knowledge, history) and what stays out. Prompt engineering asks "how do I phrase this?"; context engineering asks the bigger question, "what information should the AI have before it starts thinking?"

Context Window

The model's working memory limit: the maximum tokens it can process at once. When context overflows, older information is dropped and important details get lost, so good engineering ensures the most valuable information makes the cut. (see AI-061)

Context Assembly

The pipeline that runs before every response: retrieve documents (RAG) → call tools → read memory → load system instructions → assemble one carefully prepared package → send to the LLM. The sprint-review example gathers backlog, Jira updates, velocity, and templates before generating a word.

Context Budget

Allocating the window in defended slots — the worked example fits a support assistant into 14K tokens: 900 for system prompt, 1,100 for four tool schemas, 4,000 for top-3 reranked chunks, 5,000 for history, 2,000 output reserve. Every row is a testable decision.

Context Pyramid

The layered sources of context: system instructions, user request, conversation history, retrieved knowledge, external tool results, and memory. Together these layers form the complete context presented to the model.

Selection

The principle that curated context beats stuffed context: retrieving just page 7 outperforms pasting the whole 10-page document, at lower cost and latency. Irrelevant tool schemas and bloated history dilute attention quality. (see AI-016)

Memory

Stored user preferences, previous decisions, and long-term context retrieved into future interactions — layer 6 of the pyramid, distinct from within-conversation history.

Grounding

Supplying specific, verified information in the context so responses anchor to facts rather than model memory, reducing hallucination. (see AI-060)

Token

The smallest unit of text an LLM reads or writes, a common chunk of characters, roughly 4 characters or three-quarters of an English word. Every cost, limit, and latency figure in the LLM world is denominated in tokens. (see AI-009)

Tokenizer

The component that splits raw text into tokens using a fixed learned vocabulary. Every model family ships its own tokenizer, so the same text can produce different token counts on different models.

Byte-Pair Encoding (BPE)

The most common tokenizer training algorithm: start from single characters and repeatedly merge the most frequent adjacent pairs until the vocabulary reaches target size. It is why "strawberry" becomes [str][aw][berry] and letter-counting is genuinely hard for models.

Vocabulary

The fixed set of all tokens a model knows, typically 50,000 to 200,000 entries learned from training data. English-heavy vocabularies are why Tamil or Japanese can cost 3–10× more tokens per sentence.

Token Budget

The deliberate allocation of a context window across system prompt, retrieved context, history, and expected output, the worked example's 4,400-token support turn. The core discipline of prompt cost engineering. (see AI-020)

Input Tokens

The tokens you send (prompt, context, history), billed at the base rate. Trimming a redundant 800-token system prompt to 400 saves real money every month at scale.

Output Tokens

The tokens the model generates, billed at 3–5× the input rate because generation is sequential, one forward pass per token. They dominate bills for long-form generation. (see AI-008)

Tokens per Word

The density ratio used for estimation: about 1.3 for English prose, higher for code (brackets and indentation each cost tokens), much higher for many non-English languages.

Hallucination

A confident, fluent, fabricated statement: the student who never leaves an answer blank, writing plausible fiction in the same confident handwriting as fact. Produced because models optimize plausible next tokens, not truth, with no internal flag separating recall from invention. (see AI-009)

Grounding

Anchoring answers to provided source material (retrieved documents, databases, tools) instead of training-weight memory. The single biggest hallucination lever, and the "R" in RAG. (see AI-016)

Faithfulness

How accurately output reflects the source it was given: a summary adding claims the document never makes has low faithfulness even if those claims happen to be true. One of the five hallucination types alongside fabrication, fake citations, instruction drift, and capability claims.

Fabricated Citation

A perfectly formatted reference to a paper, case, URL, or book that does not exist, the type that sanctioned the 2023 New York lawyer whose six ChatGPT-cited court cases were all invented. The most dangerous type for professional work.

Knowledge Cutoff

The date the training data ends: the model has nothing after it but will still answer questions about later events, inviting fabrication. (see AI-007)

Groundedness Metric

The eval score measuring what fraction of claims are supported by provided sources, the core number hallucination evals track before and after every prompt change. (see AI-025)

Verification Pass

A second model call checking each claim of the first against the sources, flagging unsupported statements before users see them: a cheap catch for faithfulness drift.

Temperature

The sampling randomness dial: higher values raise creativity and hallucination risk together, so factual Q&A should run near zero. (see AI-009)

Context window

The fixed maximum number of tokens a model can process in a single request. It is the model's entire working memory, shared by the system prompt, documents, history, user message, and the generated response.

Lost in the middle

The measured tendency of models to recall information at the beginning and end of a long context far better than information buried in the middle.

Working budget

A deliberately smaller context allocation an application designs to (for cost, latency, and quality) rather than the model's maximum window.

Context compaction

Replacing older conversation turns with a running summary so long chats keep fitting in the window. Most chat products do this invisibly.

Sliding window

A history strategy that keeps the system prompt plus the most recent N turns and drops everything older.

Prompt caching

A provider feature that caches a long unchanging prompt prefix so repeated calls stop paying full input price for it.

Map-reduce summarization

Handling documents larger than the window by summarizing each chunk separately, then summarizing the summaries.

max_tokens

The API parameter capping response length; it reserves output room inside the shared context window.

Reinforcement Learning

A branch of machine learning where an agent learns to maximize a reward signal through trial and error, with no labeled correct answer provided. (see AI-002)

Agent

The learner or decision-maker in a reinforcement learning system, such as a language model policy being trained with RLHF.

Policy

The agent's strategy for choosing actions, adjusted during training to maximize expected cumulative reward.

Reward Model

A model trained on human preference rankings to predict how much a person would prefer one response over another, used as an automatic reward signal in RLHF.

RLHF

Reinforcement Learning from Human Feedback — a three-stage pipeline (supervised fine-tuning, reward model training, RL fine-tuning) that aligns a pretrained LLM into a helpful assistant. (see AI-051)

PPO

Proximal Policy Optimization — the reinforcement learning algorithm used in the original RLHF pipeline to update a policy against a reward model while limiting how far it drifts per update.

DPO

Direct Preference Optimization — a lighter-weight alignment technique that optimizes a policy directly on human preference pairs, without training a separate reward model or running PPO.

Constitutional AI

Anthropic's variant of RLHF where a model critiques and revises its own outputs against a written set of principles, generating part of its own preference data for harmlessness training.

Supervised Fine-Tuning (SFT)

The first RLHF stage, where a pretrained model is fine-tuned on human-written example responses using ordinary supervised learning. (see AI-051)

Reward Hacking

When a policy over-optimizes against a reward model's blind spots, producing outputs that score well on the reward model but that a human would not actually prefer.

Test-Time Compute

Additional inference-time computation, such as a reasoning model's thinking tokens, that RL training can be used to make productive rather than wasted. (see AI-063)