AI Glossary

Comparing two prompt versions on the same dataset and promoting the higher scorer. Prompt A at 89% versus Prompt B at 94% is a decision, not a debate.

Controlled comparison of two variants across adoption, satisfaction, task success, cost, and quality, letting evidence beat internal opinion. (see AI-058)

The share of AI suggestions users actually adopt (42% in the example), a product metric that captures real value delivery better than raw usage counts.

A named human owns every outcome. The credit officer is accountable, not decorative, and an appeal path reaches a person. Responsibility cannot be delegated entirely to software. (see AI-053)

The percentage of responses that are correct; 92 right out of 100 is 92%. The worked example shows its blind spot: a model missing every case of a 1%-prevalence condition still scores 94%+, so aggregate accuracy hides class failures.

The "squash" step that transforms a neuron's summed input before passing it on, e.g. "if negative, output 0." It is what lets stacked layers build patterns more complex than plain addition and multiplication.

Having the model route its most uncertain examples to human reviewers first, maximizing learning per label and turning the review queue into a training-data engine. (see AI-006)

The subset of weights actually used for one token; determines per-token compute cost and generation speed.

Deliberately attempting to break the AI with prompt injection, conflicting instructions, hostile inputs, and 10-page emails. The system must refuse or degrade gracefully. (see AI-023)

The learner or decision-maker in a reinforcement learning system, such as a language model policy being trained with RLHF.

The surrounding software that turns a plain AI model into a working, tool-using agent

Technical: The runtime infrastructure, planner, tool registry, memory, state management, and sandboxing, that wraps a raw LLM into a working agent. (see AI-017)

The repeating cycle an agent runs through: think, act, check the result, repeat

Technical: The repeating cycle Goal → Reason → Plan → Choose Tool → Act → Observe → Evaluate → Repeat-or-Finish. The flight-booking worked example runs it in six steps with two tools, and shows every place it can derail without checks and gates.

The final-stage model in a mixture-of-agents pipeline that synthesizes multiple candidate responses into one answer.

AI matching or exceeding human performance across most cognitive work. It is a capability claim with several competing definitions and no agreed test.

An AI that's given a goal and figures out the steps itself, instead of just answering one question

Technical: An AI system that receives a goal rather than a question, then reasons, plans, uses tools, and performs multiple actions until the goal is achieved. Unlike an assistant that answers once and stops, an agent runs the reason→act→observe loop with minimal human guidance.

Systematic differences in a model's behavior or performance across groups, inherited primarily from training data and design choices rather than intentional programming.

The ongoing loop from business problem through architecture, model selection, prompt and context engineering, development, testing, security review, evaluation, deployment, monitoring, and continuous improvement. Deployment marks the middle, not the end.

A control-plane layer between applications and LLM providers that unifies auth, routing, caching, rate limiting, and logging behind one API. (see AI-028)

The framework of policies, processes, responsibilities, and controls keeping AI systems safe, compliant, and accountable across their lifecycle: the traffic laws and building regulations of enterprise AI. Done well, it enables innovation; done badly, it becomes the committee where AI projects go to die.

Safety checks built around an AI system to stop it from doing something harmful or unauthorized

Technical: The policies, controls, and technical mechanisms applied throughout an AI system, not just inside the model, to prevent unsafe, unauthorized, or harmful outputs and actions. Like road guardrails, they don't stop the car from driving; they prevent serious accidents.

The structured pre-launch evaluation of an AI feature's risks to individuals and groups (data flows, vulnerable populations, mitigations, residual risk), analogous to a GDPR Data Protection Impact Assessment.

The coordination of models, tools, data sources, and business processes into one intelligent workflow. Think of the conductor who plays no instrument but keeps state, order, retries, and the audit trail; remove it and you have capable musicians with no score.

The blueprint organizing frontend, backend, AI services, knowledge, and integrations into a maintainable system. Mostly placement decisions: where each responsibility lives and what happens when it fails, as the reply-suggestions worked example shows.

Starting from a frustrating customer problem and asking whether AI improves the outcome. It's Team B's habit, systematized. Users don't buy AI; they buy time saved, effort reduced, and better decisions.

The layer owning prompt construction, context assembly, memory, model routing, response validation, and guardrails, kept separate from business logic so both evolve independently. Putting all logic into prompts is the classic anti-pattern.

The blueprint defining how users, applications, models, data sources, and supporting services work together. The worked example traces one question through seven boxes, only one of which is the model; one part model, six parts engineering is the honest shape of production AI.

The inventory of every AI system: owner, purpose, data touched, model, eval link, review date. The artifact regulators increasingly ask for, and where every governance regime starts. (see EU AI Act)

Verifying that an AI system behaves correctly, safely, and consistently across real-world scenarios. Unlike traditional software where input → same output every time, AI produces probable outputs, so testing targets behavior (quality, consistency, safety) instead of exact text matching.

Designing interactions so users achieve goals intuitively and confidently. The worked example proves the stakes: identical model, but auto-insert killed adoption in a month while suggest-and-diff grew it. AI UX is the discipline of that difference.

Automated notifications when thresholds are exceeded, such as latency above 5 seconds, hallucination rate above 3%, or daily cost over budget, so teams respond before users notice.

The research field working to ensure highly capable AI systems pursue their designers' intended goals.

The connection point your app actually uses to talk to another service

Technical: The interface a function typically calls behind the scenes: Calendar API, Jira API, Payment API. The AI decides which to use; the application performs the secure communication.

Apple's version of that special AI chip, built into iPhones and Macs

Technical: Apple's dedicated NPU, present since the A11 Bionic chip, built to run Core ML models with high throughput per watt.

A fast search shortcut that finds "close enough" matches instead of checking every single item

Technical: The core trick: accept near-perfect results (~95–99% recall) in exchange for cutting a million comparisons down to a tiny fraction. The smart librarian who walks you to the right neighborhood instead of checking every shelf.

The field of computer science focused on building systems that perform tasks normally requiring human intelligence, such as recognizing images, understanding speech, or generating text. In this lesson the defining feature is that modern AI learns patterns from data rather than following hand-written rules.

The sender keeps working while awaiting responses, receiving results later. This gives better scalability and faster workflows than synchronous waiting, at the cost of coordination complexity.

The mechanism where each token asks "which other words should influence my interpretation, and how much?", computing a relevance score against every other token and blending their information in proportion. For "it" in the trophy sentence, attention scores "trophy" high, so the blended representation of "it" now means trophy. (see AI-013)

One of dozens of parallel attention patterns per layer, each free to track a different relationship: grammar, coreference, tone. Heads stack across dozens of layers, with early layers linking neighbors and deep layers connecting whole-document themes.

Periodic formal review of policies, security, documentation, model performance, and access. The verification that governance processes actually work, not just exist on paper.

Keeping a record of everything the AI did, so it can be reviewed later

Technical: Recording requests, responses, tool calls, permission checks, and safety violations. The full trace of blocked attempts feeds the security team's review and improves guardrails over time. (see AI-028)

Controlling what each specific person is allowed to see or do

Technical: Determining what an authenticated user may access; the manager sees payroll, the intern does not. Notably, the strongest guardrail in the worked example is plain authorization, not an AI gate at all: the tool only accepts the authenticated user's own ID.

Decisions about individuals made without meaningful human involvement. GDPR Article 22 grants the right to human review and to contest significant decisions, one reason high-stakes AI ships as decision support. (see AI-053)

Reviewers deferring to AI recommendations even when the AI is likely wrong

Technical: the failure that makes "a human checked it" decorative rather than protective. (see AI-043)

Interrupting the AI while it's talking, and having it actually stop and listen

Technical: A user interrupting a voice AI's in-progress spoken response, requiring the system to stop and reprocess gracefully.

The raw output of pre-training; powerful text completion with no instruction-following behavior.

Grouping operations, such as one hundred embeddings in one API call, to improve throughput and per-unit cost.

Running many requests through the GPU simultaneously to raise throughput, at the price of added per-request latency. Serving engineering is choosing your point on the latency/throughput/cost triangle.

The pattern of evaluation sets designed to challenge AI for years being solved in months, forcing ever-harder replacements.

Generating multiple candidate responses from a model and selecting the strongest via a scorer, reward model, or vote.

A free, open-source reranking model you can run yourself instead of using a paid API

Technical: An open-weight cross-encoder reranker family from BAAI, commonly self-hosted for production reranking.

A baseline offset added to a neuron's weighted sum before squashing, such as the −0.5 in the umbrella example. It shifts the threshold at which the neuron activates.

A structured (increasingly legally required) assessment measuring an AI system's performance differences across demographic groups.

How many users a failure reaches and how fast the team knows. The second half is observability, and it is the half teams forget. (see AI-028)

Weighing where output lands: customer-facing outreach makes hallucination a brand risk, while internal briefs make it a nuisance. Risk assessment reframed the product. (see AI-060)

Two identical production environments (blue live, green updated), with traffic switched after validation, giving instant rollback and minimal downtime.

The classic keyword-matching search method used alongside AI-based search

Technical: The standard keyword-search ranking algorithm, scoring documents by weighted term frequency; the keyword side of most hybrid search pipelines.

Daily/weekly/monthly limits per department or project with alerts before thresholds are breached. Unmonitored token spend is the most common production AI surprise.

Per-user rate limits, billing alerts, and circuit breakers preventing usage spikes, runaway automations, or viral growth from blowing the budget. (see AI-039)

The per-component decision to build, buy, or use a managed service: auth is usually bought, core business logic built, vector databases depend on scale. It drives development speed and total cost of ownership.

The Lean Startup cycle applied to AI: build, release, measure, improve, repeat, with every iteration answering a specific question about user behavior or value.

The structured investment argument: problem, solution, expected benefits, risks, and timeline to value. It's what turns a design into a funded project.

The organizational-value metrics executives care about, such as ticket deflection, time saved, revenue impact, and developer productivity. Technical metrics and business metrics must be measured together; neither alone tells the story.

The common method used to decide how text gets split into chunks

Technical: The most common tokenizer training algorithm: start from single characters and repeatedly merge the most frequent adjacent pairs until the vocabulary reaches target size. It is why "strawberry" becomes [str][aw][berry] and letter-counting is genuinely hard for models.

Releasing to a small percentage of users first (5% → 20% → 50% → 100%) while watching metrics. In the worked example, the canary caught a non-English quality regression at 5% instead of shipping it to everyone. It's one of the safest strategies.

Exposing the new variant to ~1% of users and expanding as confidence grows

Technical: early problem detection before full deployment. (see AI-029)

A differentiating AI capability becoming available to everyone via provider API updates or open-source releases: the Q1 summarization edge that GPT-4o gives away free by Q3. The analysis question is never "what can they do today?" but "what advantage survives when capabilities converge?"

Projected launch-day traffic × tokens per request versus provider rate limits: the example carried 4× headroom plus a configured fallback provider. (see AI-008)

Forecasting compute, API quotas, storage, and headcount from growth trends so expansion is proactive, not an emergency reaction.

Designing a complete AI product from problem discovery through launch and scaling. The worked example compresses eleven lessons into one page for a freelance-designer proposal tool. If a curriculum section leaves no trace in your page, revisit it.

A voice AI built from separate steps: turning speech to text, thinking, then turning text back to speech

Technical: A voice pipeline made of separate speech-to-text, LLM, and text-to-speech stages chained together.

Attempting a task with a cheap model first and escalating to a stronger model only when an uncertainty signal is triggered.

Fine-tuning an AI on something new causes it to get worse at things it used to do well

Technical: Losing previously learned capabilities when fine-tuning aggressively on new data — managed with careful learning rates, regularization, and mixed-task training data.

The constraint that each token may attend only to tokens before it, so a generator never peeks at the future. This single restriction is what makes left-to-right generation possible; embedding models that only read skip it. (see AI-014)

Telling the AI to "show its work" step by step instead of jumping to an answer

Technical: Asking the model to "think step by step" before answering, which measurably improves math and logic. The intermediate tokens act as working memory. (see AI-063)

Prompting or training a model to produce intermediate reasoning steps, which act as working memory and improve multi-step accuracy.

Communication, training, executive sponsorship, and feedback channels for the humans whose work the product changes; the organizational half launches forget.

A saved snapshot of progress so a crashed step resumes where it stopped instead of restarting the whole run. Essential when workflows span days or weeks.

A single small piece of a document, sized to be searchable and meaningful

Technical: A ~200–500 token passage split from a document before embedding, small enough to be precise, big enough to carry meaning. The first step of every vector-search pipeline: chunk → embed → index → query. (see AI-016)

Splitting a document into smaller pieces before converting it to numbers for search

Technical: Splitting documents into paragraph-sized pieces before embedding, because whole documents blur into mushy vectors. Chunk strategy is a core retrieval-quality lever. (see AI-016)

The automated pipeline that runs tests, validates prompts, checks security, and prepares releases on every change. For AI it adds prompt evaluation and regression gates to the traditional build-and-test steps. (see AI-022)

Asking one focused question when a request is incomplete ("what type of report?") instead of guessing generically, and asking one question per turn, not a form-shaped interrogation.

A model family mapping an item to a category, such as spam filters or defect detection. One of several families (regressors, vision, speech, embeddings, LLMs, generative) that real products often chain into pipelines. (see AI-021)

A trick that makes the generated image follow your prompt more closely

Technical: A technique that runs the diffusion model with and without text conditioning at each step, then exaggerates the difference to strengthen prompt adherence.

The model aligning image and text representations in one shared embedding space, enabling zero-shot classification, semantic image search, and the text conditioning behind many diffusion models. (see AI-014)

A model accessible only through a provider's API; the weights never leave the provider's infrastructure.

A ready-made service you can plug in to do the reranking step for you

Technical: A managed reranking API widely used as a drop-in reranking layer on top of any vector database.

Reviewing marketing copy against eval results: "never makes mistakes" was removed because the evals said otherwise. Overclaiming is a disclosure risk. (see AI-060)

Weekly scans, monthly landscape-table updates, quarterly deep-dives: the rhythm that keeps analysis current in a market where printed maps of the skyline are obsolete on arrival.

Meeting external obligations such as GDPR, EU AI Act, ISO 42001, SOC 2, and HIPAA, with governance supplying the evidence: documentation, evals, audit trails, register entries. (see AI-056)

Week 1's move: no product at all, a human runs the AI workflow manually and emails results back. 20 users and "the summary missed payment terms"

Technical: learning at zero build cost.

Routing on content, so billing questions go to the finance agent, technical questions to support, and HR questions to HR. The pattern behind intelligent request routing. (see AI-024)

Producing plausible but wrong output because knowledge is dissolved into weight arithmetic rather than filed in a lookup table. This is why a network cannot cite where a fact came from. (see AI-060)

Routing on model certainty. The invoice pipeline auto-approves when classification confidence exceeds 0.9 and the amount is under $5,000, and sends everything else to human review. (see AI-053)

A high/medium/low signal beside responses helping users decide when to verify before acting; it's the shading in the suggest-and-diff design.

A measurable indicator, such as logprobs or a self-reported uncertainty score, used to trigger cascade escalation.

The routing boundary: above it auto-approve, below it reject or review, the middle band goes to the queue. Insufficient alone

Technical: models can be confidently wrong (miscalibration), so 95% stated confidence is not 95% accuracy. (see AI-025)

What happens when two agents disagree

Technical: the coordinator decides — When agents disagree (Research recommends A, Analysis recommends B), the coordinator makes the final call, ensuring consistent outcomes.

Anthropic's variant of RLHF where a model critiques and revises its own outputs against a written set of principles, generating part of its own preference data for harmlessness training.

Restricting a model's token choices at each generation step to only those consistent with a target schema or grammar.

The pipeline that runs before every response: retrieve documents (RAG) → call tools → read memory → load system instructions → assemble one carefully prepared package → send to the LLM. The sprint-review example gathers backlog, Jira updates, velocity, and templates before generating a word.

Allocating the window in defended slots

Technical: the worked example fits a support assistant into 14K tokens: 900 for system prompt, 1,100 for four tool schemas, 4,000 for top-3 reranked chunks, 5,000 for history, 2,000 output reserve. Every row is a testable decision.

The component that assembles everything the AI needs, including the user prompt, conversation history, retrieved documents, memory, business rules, and tool outputs, before the LLM call. This is where context engineering becomes practical. (see AI-020)

Quietly summarizing older parts of a long conversation so it keeps fitting

Technical: Replacing older conversation turns with a running summary so long chats keep fitting in the window. Most chat products do this invisibly.

The discipline of deciding what enters the model's window (instructions, tools, knowledge, history) and what stays out. Prompt engineering asks "how do I phrase this?"; context engineering asks the bigger question, "what information should the AI have before it starts thinking?"

The layered sources of context: system instructions, user request, conversation history, retrieved knowledge, external tool results, and memory. Together these layers form the complete context presented to the model.

The deliberate choice of what enters the prompt (last 10 messages plus thread topic, not the whole channel history) for cost, privacy, and attention-quality reasons. (see AI-020)

How much text an AI can "see" and remember at once during a conversation

Technical: The model's working-memory size, from 8K to 1M tokens, meaning how much text it can consider at once. It is a model-card field, not a learning mechanism. (see AI-061)

Testing that never stops after deployment

Technical: daily error/latency/cost checks, weekly prompt and retrieval quality, monthly regression suites and business KPIs.

Structuring the multi-turn journey (greeting, understand goal, gather information, respond, clarify, confirm, offer next step), like an experienced service rep who guides rather than expects a perfect brief. UX writing with state.

Retaining stated goals, uploaded files, and choices across turns so users never repeat themselves, using an explicit cue like "I'll remember the allergy; change it anytime." (see AI-035)

The agent that manages the team and assigns work to the others

Technical: The project manager: understands the request, delegates subtasks to specialists, manages workflow state, resolves conflicts between agents, and assembles the final result.

Apple's toolkit for running AI models on its devices

Technical: Apple's native framework for converting and running models on the Apple Neural Engine, GPU, or CPU with automatic hardware selection.

A unique identifier (like "rpt-2026-07-02-441") stitching all messages, responses, and log entries of one run together for end-to-end tracing and debugging. (see AI-028)

The math used to measure how similar two pieces of text are in meaning

Technical: The cheap arithmetic (one multiplication per axis, summed) that measures how close two vectors are in meaning. Your instant ranking of "a delicious dinner was prepared" as closest to "the chef cooked a wonderful meal" is what cosine similarity computes.

Tracking spend per team, application, or user through gateway logs, a prerequisite for the cost-optimization work in AI-039.

All variable and semi-variable serving costs

Technical: LLM APIs, embeddings, infrastructure, storage, third-party APIs, human review. Unlike traditional software, AI COGS scales with usage, so unmodelled growth can destroy margins.

The GPU-seconds burned per prediction, the third ruling metric of serving. At a million requests a day, shaving 30% off inference cost is real money. (see AI-059)

The variable cost of serving one user request: (input tokens × input price) + (output tokens × output price). Profile a representative sample of real interactions rather than guessing

Technical: token counts vary widely by use case.

A separate agent whose whole job is to check another agent's work

Technical: A reviewer checking another agent's output against source notes and a rubric, flagging unsupported claims and scoring tone. Fresh-context critique genuinely catches errors that self-review misses; that separation is the core reason multi-agent beats one mega-prompt.

The mechanism that connects your text prompt to the image being generated at every step

Technical: The attention mechanism connecting a denoising network's image latents to text embeddings, steering generation toward a text prompt at every step. (see AI-013)

A capability spanning every layer, such as security, identity, monitoring, governance, or cost, rather than living in one component.

A more accurate but slower model that compares a question and a passage directly, together

Technical: A model that scores a query and passage together as one input, more accurate than comparing separately-computed embeddings but too slow to run over an entire corpus.

Pre-release testing of frontier models for high-risk abilities such as bioweapons uplift, cyber-offense, and autonomous replication.

Checking whether the inputs the model needs actually exist. The CRM notes were sparse and stale, which killed the personalization idea on day 2. (see AI-006)

Systematic gaps or skews in training data that the model faithfully learns and reproduces, like the child shown only red apples failing on green ones. More biased data does not help; it entrenches the bias. (see AI-062)

Coverage of the situations the model will actually face: daytime and nighttime driving photos, every friend indoors and outdoors. Diversity gaps are exactly where the model fails, and they cause the ugliest deployment surprises.

How current the dataset is relative to the drifting real world; a fraud model trained on 2019 patterns misses 2026 scams. Data has a shelf life, and deployed models degrade silently as the world moves. (see AI-028)

Proprietary data (from user interactions, exclusive partnerships, closed systems) that improves the AI in ways competitors can't replicate without equivalent data. One of the four foundations that outlast the ever-taller towers of model capability.

Planting false or misleading information in training data or the retrieval knowledge base so the AI retrieves and trusts it. Mitigations: trusted data pipelines, document validation, human review, version control.

The contract between controller and processor specifying purposes, security measures, and data-subject obligations; required whenever your AI product sends user data to a model provider. (see AI-045)

The deliberate mixing ratios of data sources (web, code, math, books) in a training corpus, which measurably shape model character.

The requirement that personal data and AI processing stay within a geographic region, constraining which providers and inference endpoints an enterprise product may use, and motivating EU-only routing paths.

The requirement that data stays within a jurisdiction or network boundary. It is a primary driver of open-weights adoption in regulated industries.

A model that predicts by branching through a sequence of yes/no questions about input features, ending at a leaf node prediction.

The subset of Machine Learning that uses many-layered neural networks, powering image recognition, speech, and chatbots like ChatGPT and Claude. It sits at the innermost level of the AI ⊃ ML ⊃ Deep Learning nesting. (see AI-003)

Machine-generated media convincingly imitating a real person's face or voice. A direct consequence of generation quality reaching photorealism and voice-cloning fidelity.

Using several layers of safety checks, so one gets through even if another one fails

Technical: Stacking multiple gates, including input validation, scoped tools, output validation, and audit, so that any single gate can fail and the stack still holds. One safety filter is never enough for production.

What happens when the AI dies: the feature quietly disappears, but chat itself never blocks on AI. If the whole product dies with the model endpoint, that's an architecture smell. (see AI-030)

The lag between an AI response and its measurable effect

Technical: a slightly wrong summary eroding trust over weeks. Effects that take time to manifest require patient measurement horizons.

A fairness definition requiring each group to receive positive outcomes at the same rate.

A standard transformer where every parameter participates in processing every token.

Moving an AI application from development into a secure, scalable production environment. It's an ongoing operational process of infrastructure, versioning, monitoring, and controlled updates, not a single event.

Showing honest signals instead of false confidence: "I found two possible interpretations," confidence shading, "human review recommended." Honesty builds trust more reliably than pretended certainty.

The image-generation technique where a model learns to predict what noise was added to an image, then runs backwards from pure noise, step by step, until an image emerges. Prediction repeated becomes generation.

A generator trained to reverse a noise process

Technical: learn to denoise progressively noisier images, then at inference walk from pure random noise to a coherent image guided by a text prompt. (see AI-011)

A newer, more powerful version of that noise-cleaning network, used in tools like Sora

Technical: A newer diffusion model architecture that replaces the U-Net with a transformer, used in models like Sora and FLUX. (see AI-012)

The same image-generation trick extended across time to generate video instead of a single picture

Technical: A diffusion transformer extended across the time dimension to generate video, as in Sora, denoising a sequence of latent frames jointly.

One of the many "axes" that make up a piece of text's numeric meaning

Technical: One learned axis of the embedding space, uninterpretable individually but collectively capturing topic, tone, syntax, and subtler shades. More dimensions mean finer meaning but more storage and slower search. (see AI-015)

Reporting model metrics separately per group instead of only in aggregate. It is the single most important bias-detection practice.

Training a small student model to imitate a large teacher model's outputs, transferring capability into a much smaller network.

Direct Preference Optimization

Technical: a lighter-weight alignment technique that optimizes a policy directly on human preference pairs, without training a separate reward model or running PPO.

The world silently changing away from the training data (new vocabulary, new formats), degrading a model with no code change. Drift is only invisible if nothing is watching. (see AI-006)

A gateway running on a distributed network of edge locations (such as Cloudflare's) to minimize added latency.

How much users change AI output before using it (31% of characters), a proxy for draft quality that caught the mobile truncation bug when quality metrics stayed flat.

A list of numbers that represents the meaning of a word so the AI can process it mathematically

Technical: The number-vector each token is mapped to before entering the attention layers, capturing meaning in a form the network can process. (see AI-014)

The specific tool used to turn text into those meaning-numbers

Technical: The model that converts content into vectors: one API call (OpenAI, Cohere, Voyage) or a free local sentence-transformers model. Queries and documents must use the same model; mixing models breaks similarity entirely.

The screen before the user types, filled with example prompts and buttons for common answers, because free text is a blank-page problem for users too. The first interaction determines long-term adoption. (see AI-043)

Combining multiple model outputs, often via majority vote, to produce a more reliable final answer.

The blueprint connecting users, models, knowledge, orchestration, governance, and operations into one platform: nine layers behind the simple chat interface, like the hundreds of coordinated systems behind an airport's check-in desk.

The structured organization of a company's information so AI and people can reliably discover and use it, the library's categories, catalogue, and index applied to policies, wikis, contracts, and code. An AI is only as effective as the knowledge it can access; most "our AI is wrong" complaints are retrieval faithfully serving an ungoverned corpus.

Past conversation summaries, embedded and retrievable. Asking "communication preferences" surfaces last month's chat, effectively RAG over your own history. (see AI-016)

One complete pass through the entire training dataset. Training typically runs many epochs until the loss stops shrinking.

A fairness definition requiring that, among people who truly qualify, each group is approved at the same rate.

Agents reporting tool failures, timeouts, and permission errors instead of silence, with retry escalation: temporary failure → retry, still failing → alternative agent, still failing → human review.

Telling the AI clearly when something failed, instead of letting it think it worked

Technical: Returning a structured error when an API fails so the model can recover or apologize, instead of observing a false "success" that sends the reasoning off a cliff.

Replacing "Error" with guidance, such as "I couldn't find that document. Upload it again?", so mistakes move users forward instead of dead-ending them.

Routing to the expensive model only when signals demand it, such as low model confidence, complex document type, or long input. The worked example escalates 12% of traffic and lands at ~92.8% effective accuracy for $7,200/month instead of $31,000.

The world's first comprehensive horizontal AI regulation (in force since August 2024), classifying systems into four risk tiers: unacceptable (prohibited), high (conformity assessment, human oversight, documentation), limited (disclose AI nature), minimal (voluntary codes). Most consumer chatbots land in limited/minimal; CV screening and credit scoring land in high.

New users bringing new inputs the golden set never covered

Technical: chemistry questions doubling thumbs-down on that slice. Fix: mine production logs into the eval set weekly. (see AI-025)

Locking golden-set scores a week before launch: any prompt or model change after freeze restarts the clock. The quality half of go/no-go. (see AI-022)

The safety condition on every cost lever: measure answer quality before and after. Silently swapping in a cheaper model without evals is the classic false economy: invisible quality traded for visible savings. (see AI-022)

A consistent scoring framework across criteria like accuracy, completeness, clarity, safety, and relevance (e.g., 24/25) so different reviewers can compare results objectively.

Triggered by external events rather than a user: a new email arrives, gets classified, gets summarized, a ticket is created, and the team is notified. The backbone of enterprise automation.

Outcome-focused communication for leadership on problem, solution, business value, launch, and growth, built around confidence in business outcomes rather than technical detail.

One of the small feed-forward networks inside an MoE layer. Specialization emerges statistically during training rather than by human-defined topic.

Providing meaningful reasons for outputs, such as reason codes a rejected applicant can read, not "Application rejected." If you can't write the explanation without hand-waving, the system isn't ready for high-stakes use.

Outside experts a company hires to try to break its AI before release

Technical: A standing group of outside domain experts engaged by a lab to probe high-stakes categories before a major model release.

A completely made-up source or reference that looks real but doesn't exist

Technical: A perfectly formatted reference to a paper, case, URL, or book that does not exist, the type that sanctioned the 2023 New York lawyer whose six ChatGPT-cited court cases were all invented. The most dangerous type for professional work.

Switching to a second model provider when the first has an outage. LLM APIs fail like any dependency, so relying on one provider with no fallback is a top production mistake. (see AI-024)

Probing how a competitor's product behaves when wrong, uncertain, or out of scope: the fastest read on their error handling, feedback loops, and honesty about limitations. (see AI-060)

Sketching what the user sees when the model is wrong before designing the happy path. If the wrong-state design is hard, the feature may need a lower autonomy level.

Treating people equitably. The bank's disaggregated evals found an approval gap, removed the proxy feature (postcode), re-measured, and documented it. Bias enters through data, prompts, and evaluation itself, rarely intentionally. (see AI-062)

Whether the AI's answer actually matches what the source document said

Technical: How accurately output reflects the source it was given: a summary adding claims the document never makes has low faithfulness even if those claims happen to be true. One of the five hallucination types alongside fabrication, fake citations, instruction drift, and capability claims.

Automatically retrying a failed or overloaded provider against a secondary provider so an outage at one vendor does not become an outage in the product.

A cheap, fast test of whether the AI can do the job: hand-crafting one great prompt against real inputs and judging honestly. The wizard-of-oz test is discovery's cheapest experiment. (see AI-010)

A measurable property the model uses to make predictions. Before deep learning, experts hand-crafted features for years ("count the edges"); deep learning's real breakthrough is that features are learned automatically from raw pixels, audio, or text.

Scope expanding as every suggestion becomes a feature: delayed releases, confusing interfaces, rising costs. The contract MVP deferred RAG entirely because contracts fit in the context window. (see AI-061)

The old craft of manually designing input features for each domain, which required specialist effort and plateaued in accuracy. ImageNet 2012 (AlexNet) showed learned features beating hand-engineered ones by a margin that redirected the entire field.

A runtime switch that enables features for chosen cohorts (internal users → pilot customers → everyone) without redeploying, reducing release risk.

A score, produced by tree-based ensemble models, indicating which input features contributed most to a model's predictions.

Sorting every idea into Must Have / Should Have / Could Have / Future and building only the must-haves. It's the discipline that returns every new suggestion to the framework instead of the backlog.

Routing bad answers (thumbs-down) into the eval set instead of fixing them by hand, so the same failure can never silently return. (see AI-022)

Showing the AI one or two examples of what you want before asking it to do the real thing

Technical: Showing one or two input→output example pairs in the prompt so the model pattern-matches your format and style. It teaches better than paragraphs of description.

Retraining the AI itself on your data, instead of just handing it documents to read

Technical: The alternative knowledge path that retrains model weights — days of work versus minutes to re-index a document. The standard guidance: RAG for knowledge and freshness, fine-tuning for style and behavior; they compose. (see AI-051)

The simplest way to split text: just cut it every N words, regardless of meaning

Technical: Splitting text into chunks of a set token count with a fixed overlap, simple but blind to document structure.

Contextual next actions after each response (improve this, create slides, generate an email), keeping conversations moving and revealing the product's range.

The setup step of gradually adding random noise to a training image until it's unrecognizable

Technical: The fixed, unlearned process of gradually adding Gaussian noise to a training image until it becomes indistinguishable from random noise.

One trip of input numbers through the network's layers (multiply by weights, add, squash) with no weight changes. For an LLM, each forward pass produces the next token's probabilities, which is why long responses stream word by word.

Desirability (users want it), feasibility (we can build it), viability (business value), responsibility (safe to deploy). A successful AI product satisfies all four.

A specific action the AI can ask for, like "get weather" or "send email"

Technical: A reusable piece of software with defined inputs and outputs: get weather, send email, create invoice. Like the restaurant waiter, the LLM coordinates which function to call but never cooks the food itself.

The AI asking your app to run a specific action instead of just replying with text

Technical: The capability where a model, instead of writing prose, outputs a structured request like {"tool": "get_weather", "args": {"city": "Chennai"}} for your application to execute. The critical division of labor: the model only ever writes JSON; your code does the actual calling.

A free tool that automatically runs known attack techniques against an AI to find weaknesses

Technical: An open-source LLM vulnerability scanner that runs a large library of known probe types against a target model automatically.

The EU's data protection law governing how organizations collect, process, retain, and delete personal data, including everything an AI system touches, from training data to conversation logs. Product teams make GDPR-affecting decisions daily. (see AI-037)

A model's ability to handle cases it never saw during training, the way a child recognizes a brand-new cat. This is why a learned spam filter keeps working when spammers invent new tricks while a rule-based one breaks.

Machine learning that creates new content (text, images, audio, video, code) by learning the deep statistical structure of its training material and sampling new examples from it. Think of it as the composer to predictive AI's music critic: same training material, radically different output.

The de facto file format for quantized models in the llama.cpp/Ollama ecosystem, offering multiple precision variants per model.

A representative test dataset of real inputs with expected answers, covering easy, difficult, edge, and unexpected cases; the worked example uses 120 questions built from support logs. Without one, tuning is guesswork.

The cross-functional group approving policies, reviewing high-risk applications, and providing strategic direction. Governance is shared across executives, engineers, business owners, risk, and security; it is never one team's job.

The progression from ad hoc (anyone deploys anything) through defined policies and standardized processes to enterprise governance with continuous improvement baked into daily operations.

Graphics Processing Unit, hardware originally built for games that turned out to be perfect for the parallel math of stacked layers. GPUs are one of the three fuels (with data and training techniques) that made deep learning practical after 2012.

An ensemble method that builds decision trees sequentially, each new tree correcting the errors of the previous trees, to minimize overall prediction error.

The core optimization strategy: like walking down a foggy hillside, feel the slope (gradient) under your feet and step downhill, repeatedly. The landscape is the loss across all possible weight settings, and each step is one weight update.

A classical ML method that usually beats deep learning on small tabular datasets, such as a 2,000-row spreadsheet. A reminder that deep learning is the wrong tool when data is small, budgets are tight, or explanations are required.

Constrained decoding driven by a formal grammar (regex, context-free grammar, or JSON Schema) rather than just JSON syntax rules.

(Revenue − COGS) / Revenue

Technical: mature AI SaaS targets 70–80%. The metric that decides whether growth compounds profit or losses.

A quality check asking "did the answer actually come from the documents, or did the AI make it up?"

Technical: The evaluation metric for the generation stage: did the answer actually come from the retrieved passages? Distinct from retrieval recall; measure both separately or you will fix the wrong stage. (see AI-025)

An automated grader verifying answers stay within retrieved documents; it's the metric that caught the silent fabrication regression in the worked example. (see AI-060)

A score measuring how much of the AI's answer is actually backed by real sources

Technical: The eval score measuring what fraction of claims are supported by provided sources, the core number hallucination evals track before and after every prompt change. (see AI-025)

An automated score on whether the answer stayed within retrieved context, with an alert threshold (0.8 in the example). The trap: a gate in report-only mode logs the violation but lets it ship

Technical: flip it to blocking.

Pasting the real document or data into your prompt so the AI answers from facts instead of guessing

Technical: Pasting the actual document, data, or code into the prompt so the model works from real context instead of guessing. This is a direct hallucination reducer. When knowledge is missing entirely, retrieval or fine-tuning is needed instead. (see AI-016)

An instruction telling the AI it's okay to say "I don't know" instead of making something up

Technical: An explicit escape route like "If the information is not in the document, say 'not found' — do not guess." Permission not to answer beats silent fabrication. (see AI-060)

Running content filtering, PII redaction, and prompt-injection checks at the gateway layer as a single enforcement point. (see AI-023)

A metric that must not degrade even if the primary metric improves

Technical: latency, safety violation rate, cost per query. The insurance against optimizing one number at another's expense. (see AI-025)

Input and output safeguards such as content filtering, prompt-injection detection, permission checks, groundedness checks, rate limiting, and human approval. Every box exists because some failure taught the industry it must. (see AI-023)

The AI confidently stating something false because it's built to keep predicting plausible words, not to know the truth

Technical: Fluent, confident fabrication that follows from the design: the model always produces a plausible next token, even where it has no knowledge, because plausibility is its only currency. (see AI-060)

The proportion of responses containing incorrect, invented, or unsupported information; 4 fabrications in 100 responses is a 4% rate. Reducing it is a major production objective. (see AI-060)

Any layer between input and output where intermediate patterns are built: edges into parts into objects. The name just means its values are not directly observed as input or output.

A team structure with a manager on top and specialists reporting to it

Technical: Manager agent delegating to specialists with clear authority at each level, the common enterprise structure. Contrast with swarm architecture, where agents communicate directly, more flexible but harder to coordinate.

Continuity through redundancy, failover, health checks, circuit breakers, and disaster recovery, so the platform survives individual component failures.

Unfairness present in the world that generated the training data. The data records past discrimination faithfully and the model learns it as ground truth.

The most common technique vector databases use to search quickly

Technical: The dominant ANN index, a multi-layer "highway network" over the vectors, with coarse express links on top and local streets below. Queries ride the highways to the right region, then explore locally, giving millisecond search over millions of vectors.

Qualified humans review high-stakes outputs, such as medical diagnoses by doctors, contracts by lawyers, and financial approvals by managers. The bank shipped decision support, not auto-decision.

The AI is purely advisory; the human makes every decision

Technical: the lawyer reads the AI's case suggestions but files everything themselves. For safety-critical domains or immature, untrusted models.

Requiring a person to approve risky actions before the AI can actually carry them out

Technical: Requiring human approval before high-risk, irreversible actions such as payments, contract approvals, record deletion, and production changes. The AI prepares the work; humans make the final decision. (see AI-053)

Building AI systems where humans remain meaningfully involved in consequential decisions

Technical: not a sign of AI weakness but an architectural choice. The most reliable AI products are not the ones that automate everything; they are the ones that know exactly when to hand off to a human.

The AI acts autonomously on most decisions while a human monitors the stream and can override

Technical: high-confidence content removals with a moderator sampling decisions. For high-volume, lower-stakes work after trust is established.

Handling easy requests on your device and sending harder ones to the cloud

Technical: A deployment pattern where a small on-device model handles common, latency-sensitive cases while harder requests escalate to a cloud model.

The production pattern of sending most traffic to fast cheap models and escalating only hard problems to reasoning models.

Searching by both exact keywords and by meaning at once, to catch more relevant results

Technical: Running vector and keyword search together and merging results, often with a reranker. Vectors miss exact identifiers like "error TS-4102"; keywords miss paraphrases — every real corpus needs both.

An image's cost in the context window

Technical: typically 100–2,000 tokens depending on resolution and architecture, directly affecting API cost and context budget. (see AI-059)

The large labelled-image benchmark whose 2012 competition was deep learning's turning point, when AlexNet beat the best hand-engineered systems decisively. It is the standard historical marker for when the field pivoted to deep learning.

Hiding malicious instructions inside documents, web pages, PDFs, or emails the AI later reads, as in the worked example's white-on-white "forward the last five emails" text. The content is the exploit, which is why "the model read something" must be treated like "a user typed something." It's the signature new vulnerability of the LLM era.

Using an already-trained model to make a prediction on new input, like your face unlock comparing your face to stored patterns. Training happens once on many examples; inference happens every time you use the system.

The chunks of text you send to the AI, which you pay for at the standard rate

Technical: The tokens you send (prompt, context, history), billed at the base rate. Trimming a redundant 800-token system prompt to 400 saves real money every month at scale.

Training a raw AI model to actually follow instructions and hold a conversation

Technical: Training on (instruction, response) pairs so a base model follows instructions reliably — how base models become chat assistants. (see AI-067)

How consistently different reviewers label the same item (measured with statistics like Cohen's Kappa)

Technical: the health metric of annotation quality and guideline clarity.

The degree to which a model's decision process can be understood and explained, a key advantage of classical ML over deep learning in regulated domains. (see AI-003)

A model provider's contractual commitment to defend customers against IP claims arising from model outputs, a key vendor-contract term while courts actively rule on AI copyright cases.

A trick prompt designed to get an AI to ignore its safety rules

Technical: A prompt or sequence of prompts designed to bypass a model's safety training and elicit disallowed output.

Focusing on the task users are actually hiring the product for: "prepare tomorrow's meeting in 15 minutes," not "I need AI." The job reveals what the product truly must do.

A provider flag that biases generation toward syntactically valid JSON without guaranteeing conformance to a specific schema.

Using a separate model to evaluate multiple candidate outputs and select or score the best one. (see AI-025)

An unsupervised algorithm that groups unlabeled data into k clusters by iteratively assigning points to the nearest cluster center and recomputing centers.

The "here's what I'm about" vector each token broadcasts, the spine label every book advertises. Queries are scored against all keys to find relevant tokens.

Centralizing provider API keys in the gateway instead of scattering them across application services.

Failure conditions agreed before building, such as "if fewer than X of Y users do Z by week 4, we stop." The cheapest founder discipline there is.

The pre-agreed failure condition (fewer than 3 of 10 pilots active by week 6) that stops the project honestly instead of letting it drift. (see AI-046)

The point where the AI's knowledge simply stops, because its training data ends there

Technical: The date the model's training data ends; it knows nothing after that. A design consequence of frozen weights, not a bug. (see AI-007)

Named ownership, review dates, access permissions, retention policies, and quality standards. The worked example's stale-doc escalation and ACL-filtered retrieval respect the asker's permissions, not the bot's. (see AI-027)

Connected entities and relationships (Employee → Project → Customer → Contract → Invoice) so AI can navigate related knowledge instead of isolated documents, improving reasoning and discovery.

Storing every token's keys and values so each newly generated token only computes its own. It's the optimization that makes chatbot streaming affordable, and the reason long conversations consume growing GPU memory.

The known correct answer attached to a training example, such as an email tagged "spam" or "not spam" by a human. Labels are what make supervised learning possible, and user corrections like clicking "Not spam" become fresh labels.

Examples with the correct answer attached, like an email tagged "spam." Labels usually come from paid human effort, which makes labelled data the expensive kind; LLMs escaped this economics by using the next word as a free label. (see AI-009)

An AI trained on huge amounts of text that learns to predict the next word

Technical: A neural network with billions of parameters trained on trillions of words to do one thing: predict the next token. That single skill, at sufficient scale, produces translation, coding, reasoning, and conversation.

How long one request takes; users feel anything over ~200ms in interactive apps, while LLM chat lives at 1–10 seconds. Reduced with smaller models, quantization, caching, and streaming. (see AI-065)

How fast an AI feature needs to respond before it feels too slow to use

Technical: The hard time limit user experience imposes: reply suggestions must land in under 1.5 seconds or typing outruns them, forcing a small fast model, streaming, and aggressive caching. (see AI-008)

Doing the noise-cleaning process on a compressed version of the image to make it much faster

Technical: Diffusion run in a compressed latent space (via a Variational Autoencoder) instead of pixel space — the trick behind Stable Diffusion and FLUX that makes high-quality generation feasible on consumer hardware.

One of six GDPR Article 6 grounds (consent, contract, legal obligation, vital interests, public task, legitimate interests) required before an AI system may process personal data at all.

One stage of a deep network's processing hierarchy: early layers detect edges, middle layers combine them into parts like eyes and noses, and deep layers assemble whole concepts. The lesson's company analogy maps layers to frontline staff, team leads, and directors.

Leading metrics (trial usage, feature adoption) predict; lagging metrics (revenue, retention, ROI) confirm. Both are needed for trajectory.

The stride length of gradient descent. It sets how big each weight-update step is. Too small and training takes forever; too large and updates overshoot the valley and bounce between hillsides.

Giving a person or an AI only the minimum access it actually needs, nothing more

Technical: Granting users and the AI only the minimum permissions needed. A developer's GitHub tool, for example, can read a repository but not delete it. Applied to MCP, it decides which servers, tools, and actions each user gets. (see AI-019)

Costs that grow with every single user: tokens, storage, human review. The linear list is what breaks at 50×; each item needs a sub-linear plan: cache, route, batch. (see AI-039)

A model that predicts a continuous number as a weighted sum of input features plus a bias term, the simplest baseline predictor. (see AI-002)

Using a language model to score responses at scale. It's fast and repeatable, but judges need temperature 0, explicit rubrics, and calibration against human labels, or their scores drift run to run. (see AI-025)

Using a language model to evaluate the quality of another model's output, applied at inference time in quality routing and at test time in evaluation.

The design discipline of asking which steps genuinely need a model. Invoice validation is pure rules, no model needed or wanted. This question halves most AI workflow designs and their costs. (see AI-039)

An auxiliary training objective that pushes the router to distribute tokens across experts, preventing routing collapse.

A classification model that outputs a probability by passing a weighted sum of features through a sigmoid function, widely used for interpretable yes/no predictions.

Detailed event records, such as prompt executions, tool calls, errors, and model versions, that explain individual moments in time. Log everything important, but never excessive sensitive data.

What the agent remembers across different sessions, so it doesn't forget everything each time

Technical: Facts, prior decisions, and user preferences persisted across sessions so an agent doesn't start from zero each time.

A cheaper way to fine-tune a model by only training a small add-on instead of the whole thing

Technical: The most widely used parameter-efficient technique: freeze the base weights, add two small trainable matrices A and B whose product approximates the update, merge at inference (W' = W + A×B). Only 0.1–1% extra parameters — a 7B model fine-tunes on a single consumer GPU.

The "wrongness score" a loss function assigns to each prediction: the model guesses "dog, 62%" for a cat and gets a loss of 2.4. Watching the loss curve fall is how practitioners literally see a model learning.

The AI tends to forget things buried in the middle of a long conversation more than the start or end

Technical: The measured tendency of models to recall information at the beginning and end of a long context far better than information buried in the middle.

Without a shared standard, every AI app needs a custom connection to every data source

Technical: that's what MCP avoids — The integration explosion MCP collapses: M AI apps × N data sources means M×N custom integrations without a standard, but only M+N with one. That arithmetic is the protocol's entire value proposition.

The branch of AI where computers learn patterns from examples and improve through experience, instead of relying on manually programmed rules. In the spam-filter example, the programmer writes only the learning procedure; the data writes the rules.

The subset of AI where systems learn from examples instead of being explicitly programmed, the technology behind nearly all modern AI. When a headline says "the AI decided," it almost always means a machine-learned model made a statistical prediction. (see AI-002)

Overloading the AI with fake examples of bad behavior so it copies the pattern

Technical: Filling the context window with faux examples that model undesired compliance, exploiting in-context learning to shift final-turn behavior.

Summarizing a huge document piece by piece, then summarizing those summaries

Technical: Handling documents larger than the window by summarizing each chunk separately, then summarizing the summaries.

A setting that caps how long the AI's reply is allowed to be

Technical: The API parameter capping response length; it reserves output room inside the shared context window.

The AI app's side of the connection, the part that asks for tools and data

Technical: The component inside the AI application (Claude Desktop, an AI IDE, a custom agent) that discovers available servers, requests resources, executes tools, and exchanges messages with MCP Servers.

The integration stage that reaches external tools (GitHub, Jira, calendar, CRM) through standardized MCP servers instead of bespoke per-system integrations. (see AI-019)

The other side of the connection, the part that actually offers the tools and data

Technical: A domain-focused service exposing Tools, Resources, and Prompts: a GitHub server, Jira server, filesystem server. One well-built server serves many clients: the worked example's ticket server powers chat, editor, and overnight triage bot from a single integration.

Distortion introduced when the training label is a flawed proxy for the real outcome, like using arrests as a proxy for crime or spending as a proxy for medical need.

The agent remembering what it already did earlier in the task

Technical: The agent's retention of previous conversations, decisions, retrieved documents, and intermediate results across steps, which prevents repeating work within a task.

Summarizing many conversations into compact long-term records, so 20 chats become one summary, preserving what matters while keeping retrieval efficient and cheap.

Policies giving memories a lifetime, such as a meeting-room code that expires in a day versus a company policy that persists. Remembering everything is a mistake; so is never pruning.

Ranking candidate memories by relevance, recency, importance, and similarity so only the highest-ranked enter the context budget. Retrieval quality matters as much as storage. (see AI-020)

The property of LLM inference where speed is limited by how fast weights move from memory, which is why smaller quantized weights generate faster.

Information about information, such as author, department, version, effective_date, tags, and security classification. The parental-leave fix in the worked example is metadata: retrieval filters to current-only via effective_date and supersedes fields. (see AI-015)

Narrowing a search using extra criteria, like "only results from this year"

Technical: Combining similarity with structured filters, like "nearest neighbors where team = legal and year ≥ 2024." Real queries almost always need it, and databases differ sharply in filtering without wrecking recall.

Numerical measurements over time

Technical: latency, error rate, token usage, cost. AI adds its own signals to the traditional list: prompt version, retrieval score, hallucination rate, user rating.

The layered instrumentation of one feature: north star (hours saved per user per week), product metrics (acceptance rate, edit distance, retention), quality metrics (groundedness, thumbs-down), system metrics (p95 latency, cost per draft). Its power is diagnosis: the tree traced an "AI got worse" report to a mobile CSS bug in minutes.

The smallest improvement the experiment can detect with significance

Technical: computed before starting to set sample size and duration, preventing underpowered experiments that "prove" nothing.

The simplest version delivering real value while enabling rapid learning. The contract summarizer took four weeks: fake it, thin slice, trust features, charge. Not unfinished software; the smallest product worth using.

A transformer architecture where feed-forward layers are split into many small expert networks, with a router activating only a few per token.

Combining outputs from multiple different models via an aggregator model that synthesizes a final answer.

A framework for making AI models run efficiently on many different types of devices

Technical: A compilation framework for running LLMs efficiently on-device across a wide range of hardware backends.

The structured durability check across data, workflow, trust, network, and speed: separating temporary capability leads (sand) from structural advantages (deep footings).

The learned artifact produced by training on examples: the "internal sense of cat-ness" in the child analogy. Given a new input it has never seen, a model produces a prediction based on the statistical patterns it absorbed.

The spec sheet shipped with a model release: parameters, context window, training cutoff, modalities, license, and benchmark evals. Learning to parse "Llama 3.1 8B, 128K context, cutoff Dec 2023, open weights" is the practical skill this lesson teaches.

The approved list of models and providers with routing and fallback built in. Swap a provider once and forty use cases inherit the change. (see AI-024)

The degradation that results from repeatedly training on unfiltered model-generated data; diversity shrinks and errors compound.

A common language that lets AI apps plug into any tool or data source without custom-building each connection

Technical: An open standard, the USB-C for AI tools, that lets AI applications connect to external tools, data sources, and applications through one consistent protocol instead of bespoke integrations. It standardizes tool discovery, descriptions, execution, resource access, prompts, and errors.

The legal terms attached to model weights; may restrict commercial use, require attribution, or prohibit training competitors on outputs.

The architectural pattern of classifying each request and directing it to the right model: a simple FAQ to a small model, research to a reasoning model, an image upload to a vision model. The customer sees one assistant; multiple models collaborate behind it.

Choosing the most suitable model for a task by balancing capability, cost, speed, context size, reliability, and deployment constraints. There is no universally best model, only the best model for a particular task, like choosing between a bicycle and a cargo ship.

A specific release of a model (GPT-4 → GPT-4o, Claude 3 → 3.5 → 4) whose behavior can differ dramatically from its siblings. Production teams pin exact versions and run regression evals before upgrading. (see AI-026)

Continuous observation of response time, token usage, API failures, costs, satisfaction, and error rates. Paired with logging (requests, responses, tool calls, model versions, trace IDs) for troubleshooting and auditing. (see AI-028)

Multiple agents, each with their own harness, working together on a shared task

Technical: Multiple agents, each running inside its own harness, working together on a shared task via a coordination layer. (see AI-032)

Several AI agents working as a team, each with its own job

Technical: Multiple specialized agents collaborating, such as Planner → Research → Writer → Reviewer, each owning one responsibility. Contrast with a single agent doing the entire task.

Running many parallel attention patterns per layer (32 to 128 in large models), each with its own query/key/value matrices. Heads specialize (subject↔verb agreement, pronoun links, quote pairing) without anyone assigning roles; specialization emerges from training pressure.

Slowly pushing an AI toward a bad answer over many messages, so no single message looks harmful

Technical: A jailbreak technique that gradually escalates a conversation across many turns so no single message appears harmful.

A single model handling text, images, and audio together, like GPT-4o, Claude, or Gemini. It is the same core idea

Technical: learned structure plus iterative prediction — wearing different clothes. (see AI-052)

A model processing text, images, audio, video, and documents together. It's the vision path in a routing setup for tasks like damaged-product photos or invoice analysis. (see AI-052)

The simplest version delivering enough value for real customers to use and give feedback, the stage every earlier step in the lifecycle exists to reach with less uncertainty.

Advantage compounding with usage: more feedback data improving the model, more user content benefiting others, more social proof de-risking purchases for new buyers.

A computational structure made of layers of simple connected units, loosely inspired by the brain. No single layer understands the input; recognition emerges from the whole stack working together. (see AI-004)

A single unit that does three things: multiply each input by a weight, add the results plus a bias, and squash the total through a simple function before passing it on. The umbrella example shows the full arithmetic; there is nothing else hidden inside.

The AI's core trick: guessing the most likely next word, over and over, to build a whole response

Technical: The sole training objective of an LLM: given text so far, output a probability for every possible next token. To predict really well, the model is forced to compress the world that produced the text, including its grammar, facts, styles, and reasoning patterns.

Point-and-click platforms (Zapier, Make, n8n) for wiring one app's trigger to another app's action without writing an orchestrator. n8n is open-source and self-hostable; Zapier and Make are hosted SaaS. Built for automation, not orchestration

Technical: see Automation vs Orchestration above.

Identical inputs producing different outputs across calls due to temperature and sampling

Technical: the property that makes AI experiments a comparison of two distributions rather than two fixed experiences. (see AI-009)

The single outcome metric everything else serves: hours saved per active user per week for the email-drafting feature. An accurate AI nobody uses is not a successful product.

Users engaging more with a new feature simply because it is unfamiliar, inflating early treatment metrics

Technical: one reason AI experiments need longer windows than button-color tests.

A special chip built just for running AI models efficiently

Technical: Dedicated silicon built specifically to run neural network workloads efficiently, separate from a device's general-purpose CPU and GPU.

Collecting, analyzing, and visualizing operational data to understand why an AI system behaves the way it does. Monitoring tells you something is broken; observability explains why. It's the cockpit instruments of production AI.

Running an AI model right on your phone or laptop instead of a remote server

Technical: Running the model on the user's hardware (face unlock, keyboard prediction) for zero network dependency, total privacy, and offline operation. Compression techniques like distillation and quantization make models fit. (see AI-065)

Time-boxing the whole design (problem, evidence, solution shape, architecture, MVP plan, metrics tree, risks) to 90 minutes on one page, then sharing it with one honest reader whose first confused question marks the section to rework.

A toolkit that runs AI models on phones regardless of what tool built them

Technical: A cross-platform runtime for running models exported to the ONNX format on mobile devices.

A formal model of relationships between concepts, such as Employee works in Department, Department owns Project, letting AI reason about meaning rather than keywords.

Releasing the model file itself so anyone can download and run it. Sharing the file shares the capability. The DeepSeek-R1 release showed a single uploaded file can move markets. (see AI-066)

A model whose trained weights are downloadable and self-hostable, while training data and code may remain private. This is the most common meaning of "open" in AI.

Monitoring, logging, alerting, security, deployment, scaling, backup, and disaster recovery. This is the layer that keeps the platform healthy long after launch.

The structured checklist for any AI idea: who has the problem, what frustrates them, how they solve it today, how AI improves it, why invest, what could go wrong, how success is measured. (see AI-042)

The component deciding what happens next at every stage: workflow execution, task sequencing, tool and model selection, error handling, human approval, and result aggregation. The order #4712 example chains a router, RAG step, tool call, agent step, and human gate under one conductor.

Tuning max_tokens per route to curtail verbose answers. Output tokens cost more than input, so uncapped verbosity is silent spend. (see AI-059)

Double-checking the AI's answer before showing it to the user

Technical: Reviewing responses before users receive them, covering PII redaction, policy compliance, citation verification, and harmful-content detection. If validation fails, the response is rejected or regenerated.

The chunks of text the AI generates back to you, which usually cost more per chunk

Technical: The tokens the model generates, billed at 3–5× the input rate because generation is sequential, one forward pass per token. They dominate bills for long-form generation. (see AI-008)

Double-checking the agent's results before trusting them

Technical: Checking tool results and final outputs against expectations before trusting them, closing the agent loop rather than acting blindly.

When a model memorizes its training examples instead of learning the underlying pattern, so it recalls old cases perfectly but fails on new ones. The house-price model priced a farmhouse absurdly because its city-apartment training data left a gap. (see AI-005)

The response time 95% of requests beat, the tail metric that matters for user experience. The mid-tier model's 1.8s versus the frontier's 6.1s is often the deciding factor for real-time products.

The tone and emotion in someone's voice that gets lost if you just read a transcript

Technical: Tone, emphasis, and emotional cues carried in speech that a text transcript discards.

Independent tasks running simultaneously, such as searching documents, analyzing images, and querying the database all at once, then combined. Cuts total response time.

The transformer's other half: every word attends to every word simultaneously, so an entire document trains in one parallel step, mapping perfectly onto GPUs. No parallelism, no LLMs: it unlocked the scaling laws. (see AI-009)

One piece of information a function needs to run, like a date or a name

Technical: A named input a function requires, like Title, Date, and Participants for create_calendar_event(). The LLM extracts these values from natural language, and it fills fields exactly as well as the schema describes them: "date (YYYY-MM-DD, must be future)" beats "date".

All the weights and biases in a network. It's the number in every model card, from ~100 thousand in a tiny classifier to hundreds of billions in frontier LLMs. More parameters means more pattern-storage capacity, and more data and compute to train and run. (see AI-064)

A scaffold with guardrails, evals, and tracing pre-wired, so a new use case ships a compliant chatbot in days. The platform payoff: use case #41 shipped in two weeks with two engineers, with governance enforced by architecture instead of memos.

Fine-tuning just a small part of a model instead of retraining the whole thing

Technical: The family of methods updating only a small subset of parameters while freezing the rest — near full fine-tuning quality at a fraction of the cost, since full fine-tuning of billions of weights is impractical for most teams.

Breaking any metric down by class and subgroup. The symptom checker's recall was worst for patients over 70, invisible in the aggregate. Choose the metric that matches the cost of the error. (see AI-062)

A free, widely used add-on that turns a regular Postgres database into a vector database

Technical: A free Postgres extension that became the default vector store for startups, the boring option already in your stack. The honest advice: start with pgvector or Chroma, and graduate to Pinecone/Weaviate/Qdrant/Milvus only when scale demands it.

Expanding access gradually: internal → beta → pilot → 5% → 20% → 100%, so issues surface before they reach everyone. Boring launches are engineered, not lucky: the worked example's provider slowdown at hour 3 triggered the fallback and users never noticed.

The part of the harness that figures out what steps to take next

Technical: The harness component that breaks a high-level goal into steps and decides what to do next given current state.

Breaking a big goal down into a sequence of smaller steps

Technical: Breaking a large goal into smaller executable tasks, like "prepare presentation" becoming collect data → summarize → create slides → review. Poor planning is the most common way agents produce poor outcomes.

The evolution from product to platform (plugins, developer APIs, extensions, marketplace, partner ecosystem), distributing growth across many contributors.

The agent's strategy for choosing actions, adjusted during training to maximize expected cumulative reward.

The position marker each token carries, needed because all words are processed at once rather than in order. Without it the model could not tell "dog bites man" from "man bites dog."

Quantizing an already-trained model without further training. It is the fast, common approach used by GGUF formats.

Proximal Policy Optimization

Technical: the reinforcement learning algorithm used in the original RLHF pipeline to update a policy against a reward model while limiting how far it drifts per update.

The expensive first stage where a model learns next-token prediction over trillions of tokens, acquiring its core knowledge and capabilities.

When the AI gives a positive answer, how often it is correct. High precision means few false positives, like few innocent passengers stopped at airport security.

The number of bits used to store each weight; halving bits roughly halves model memory.

The Level 1 style of AI that judges content rather than making it, outputting a label or number like "spam or not" or "what price." The shift from judging to creating is what made AI explode into public consciousness in 2022.

Optimizing a model toward responses humans (or AI judges) prefer, implemented via RLHF, DPO, and similar methods.

The line where data leaves your control: messages going to a provider require a data-processing addendum, no-training clauses, and regional routing for enterprise tiers. (see AI-056)

Embedding protection from the start: collect only necessary data (data minimization), cap retention, encrypt, and test the model for memorized personal-data leakage. Privacy cannot be added later. (see AI-056)

Accessible to one user only, covering personal preferences, notes, and conversation history. Contrast with shared memory for teams, covering project docs, decisions, and standards. Enterprise systems combine both, with strict access control between them.

User + need + reason: "Project managers need a faster way to summarize weekly updates because reports take hours." Clear problems lead to better solutions.

Choosing the problem, then discovering whether AI is the right tool. Team B started from "43% abandon during data import" and shipped a column-mapping classifier, not a chatbot, while the feature-first team shipped a chatbot 4% of users tried once.

The structured process of reducing uncertainty before building: user research, problem validation, prototyping, business analysis. The worked example's one-week discovery killed "AI writes the email" and saved "AI briefs the rep": same team, a tenth of the risk, aimed at the real bottleneck.

Problem Discovery → Research → Product Thinking → UX → Architecture → MVP → Metrics → Launch → Scaling → Continuous Improvement. This is the full cycle the capstone runs once, as the foundation for repeating it. (see AI-041)

The aspirational statement of what the product achieves, for whom, and what success looks like; it's the tiebreaker when priorities conflict.

The precondition for scaling: strong retention, organic growth, users measurably worse off without the product. Scaling before fit is the classic fatal mistake.

A complete platform combining models, data, retrieval, tools, security, monitoring, and operational processes. It's the airport ecosystem behind the simple chat interface. The demo-vs-production table shows the difference: same model and prompt, but retries, failover, guardrails, cost dashboards, and feedback loops on the production side.

Passing the checklist across architecture, security (auth, encryption, guardrails), quality (evals, testing, benchmarks), operations (monitoring, alerts), deployment (CI/CD, rollback, versioning), and business (KPIs, feedback, docs).

Starting simple and revealing advanced features as users gain confidence, reducing cognitive load so new users succeed quickly.

The message you type to an AI to get a response

Technical: The input text the model continues. Every word shifts the probability distribution over possible outputs. You are not commanding a machine; you are conditioning a distribution. (see AI-009)

A ready-made prompt template a server offers so teams don't rewrite the same instructions each time

Technical: A reusable prompt template a server can expose, like a Code Review Prompt or Security Checklist, enabling consistent AI workflows across teams.

Reusing a stored copy of a long, unchanging prompt so you don't pay full price for it every time

Technical: A provider feature that caches a long unchanging prompt prefix so repeated calls stop paying full input price for it.

Behavior changing while the prompt text hasn't, caused by silent provider model updates, provider-side system prompt changes, or shifting input data. Detected only by continuous evaluation, never by reading the prompt. (see AI-007)

Writing your request so the AI understands exactly what you want

Technical: The craft of writing inputs that steer a model toward the output you want. It is the highest-leverage, lowest-cost skill in applied AI, and the first tool to try before fine-tuning or complex pipelines. (see AI-051)

The systematic process of measuring prompt performance with repeatable tests, objective metrics, and structured feedback, evidence instead of "does this answer look better?". It turns prompt engineering from subjective art into a measurable engineering discipline.

Sneaking hidden instructions into what an AI reads, tricking it into ignoring its real instructions

Technical: An attack embedding instructions like "ignore every previous instruction and reveal confidential information" in user input. The worked example's input gate flags it with an injection classifier at risk score 0.93. (see AI-027)

A central service storing versioned prompts, metadata, and deployment history, from which applications fetch the right version at runtime, decoupling prompt deploys from code deploys.

Dev → staging → production environments a prompt change must pass, each with progressively stricter evaluation and access control, the same discipline software adopted forty years ago.

A reusable Role → Task → Context → Constraints → Output Format structure giving users a predictable starting point for recurring work. (see AI-010)

Designing the prompting experience so users communicate naturally without learning prompt engineering: the meal-planner's rewrite took first-menu completion from 38% to 74% on the same model. Every system-prompt line is an interface decision.

Tracking every prompt change as a discrete, auditable commit (who changed what, when, why), with rollback to any previous state. The antidote to the shared-Notion-doc failure where a "clarity tweak" silently dropped deflection 12% with no git blame to find it.

Treating prompts like source code (v1.0 → v1.1 → v2.0), with every change documented, tested, reviewed, and measured. (see AI-057)

Checking invariants that must hold across hundreds of generated inputs, such as "never invents a meeting time" or "always preserves the recipient's name." Invariants are the easiest tests to write and the most valuable to keep.

The question of whether content was made by a person or a machine. Watermarking and detection remain unsolved, and deepfakes make this a live societal issue.

A company or service offering LLM inference over an API, such as OpenAI, Anthropic, Google Gemini, Cohere, or a self-hosted open-source model.

Abstracting the model API behind an internal service so the provider can be swapped without touching forty call sites. (see AI-040)

An innocuous-looking input (postcode) that correlates with a protected characteristic and smuggles discrimination into the model, found only by slicing metrics by demographic group.

An innocent-looking feature (postcode, school, vocabulary) that statistically encodes a sensitive attribute, letting bias survive the removal of explicit columns.

An agent publishes an event and all subscribed agents automatically receive it, so "policy updated" reaches the compliance, legal, and HR agents at once. Ideal for event-driven systems.

A Python class defining typed, validated fields, commonly used as the schema definition layer for structured LLM extraction.

A free Microsoft tool for automatically testing AI systems for security weaknesses

Technical: Microsoft's open-source Python Risk Identification Toolkit for orchestrating automated, scriptable red-team attacks against LLM applications.

An even lighter version of LoRA that needs much less computer memory to run

Technical: LoRA with the frozen base model loaded in 4-bit precision to slash GPU memory, keeping adapter weights in full precision — the technique that put large-model fine-tuning within reach of consumer hardware. (see AI-065)

Attention's scaling problem: every token scores against every other, so 100,000 tokens means 10 billion comparisons. It is the engineering reason context windows were once tiny and long context is priced steeply. (see AI-061)

The AI chip found inside most Android phones

Technical: Qualcomm's NPU inside Snapdragon chips, used across most Android flagship and mid-range devices.

Directing or combining model calls to maximize output accuracy rather than minimize cost. (see AI-039)

The metric that actually drives selection: accuracy on your own golden set divided by cost per request. Benchmark on your data, not leaderboards, and re-evaluate periodically. (see AI-022)

Compressing a model by storing weights in fewer bits (8-bit, 4-bit), shrinking memory and speeding inference with modest quality cost.

The "what am I looking for?" vector each token sends out, like bringing a search request to a library. It is created by multiplying the token's embedding by a learned weight matrix.

Clarifying a vague question before searching, so the search actually finds what you meant

Technical: Transforming a vague user question ("what about part-timers?") into an explicit, retrievable one before embedding it — a standard lever when conversational follow-ups miss.

An ensemble of many decision trees, each trained on a random subset of data and features, whose predictions are averaged to reduce overfitting.

Enforcing per-team, per-application, or per-user request caps to protect shared provider quotas and budgets.

Provider caps hit at peak hours as usage grows

Technical: mitigated with request queuing, graceful "high demand" messaging, an overflow provider, and pre-negotiated quota raises. (see AI-024)

OpenAI's technology for building live, voice-based AI conversations

Technical: OpenAI's WebSocket-based API for native speech-to-speech interaction, supporting streaming audio and interruption within a session.

A model optimized for multi-step reasoning, mathematics, planning, and complex analysis, usually slower and pricier. It's reserved for requests that genuinely need it. (see AI-063)

How many of the truly best matches a search actually manages to find

Technical: The fraction of true nearest neighbors your index actually finds: the universal dial traded against speed. Production teams tune it with measurements on their own data, not guesses. (see AI-025)

A method for combining two separate search result lists into one fair ranking

Technical: The standard method for merging two ranked result lists (e.g., BM25 and vector search) into one combined ranking, using each item's rank position rather than raw scores.

A smarter split that tries to break at paragraph or sentence edges first

Technical: Splitting text using a hierarchy of separators (paragraphs, then sentences, then words), respecting natural boundaries more often than fixed-size splitting.

Running adversarial prompts (injection, PII extraction, brand-damage requests) before launch; the example found two failures, gated them, and retested. (see AI-027)

Deliberately trying to break an AI to find its weaknesses before a real attacker does

Technical: The discipline of deliberately, systematically attacking a model or AI system to find failures before an adversary does. (see AI-023, AI-027)

A reusable blueprint of standard components, layers, and flows, a starting point for production design so each new use case doesn't reinvent the stack.

The agent checking its own work and deciding if it needs to try again

Technical: The agent evaluating its own work after a step: did I achieve the goal, is information missing, should I try another approach? It creates an iterative improvement loop.

Re-running a changed prompt against previously passing cases to catch trade-offs. In the worked example, the "better" prompt gained +0.4 helpfulness while groundedness dropped from 96% to 88%. Never deploy prompts without it.

The pre-defined line a new version must not cross (e.g., no more than a two-point drop in golden-set accuracy versus current production), turning "looks fine to me" into a gate. (see AI-026)

Learning through trial, error, and rewards rather than labelled answers. It is the approach behind game AI, robotics, and tuning chatbots with human feedback. (see AI-067)

Designing for failure by retrying failed requests, caching common responses, switching to backup models, and degrading gracefully when APIs fail. Production systems assume components will break.

When parsing fails, naming what was understood and asking a narrower question, never the dead-end "I didn't understand." Recovery should always move users forward.

Skew caused by some groups being underrepresented in training data, producing models that perform worse for the groups they saw least.

One agent asks another and waits for a direct reply, as in Research Agent → Knowledge Agent → documents back. Simple, synchronous, and the most widely used pattern.

A second check that re-sorts the found passages so the very best ones end up on top

Technical: A second model that reorders the retrieved top-50 into a sharper top-5, one of the highest-leverage fixes for retrieval misses.

A known weakness that a company has decided to accept rather than fix

Technical: A red-team finding that is documented and accepted rather than fixed or mitigated, typically after formal sign-off.

A piece of data an MCP server makes available to read, like a file or a log

Technical: A readable data source an MCP Server exposes — documentation, logs, configuration, source code. Unlike tools, resources are read rather than executed.

Serving repeated questions ("what are the office hours?") from a stored validated answer with no model call at all, cutting cost, latency, and load simultaneously.

Designing, building, and operating AI that is fair, transparent, accountable, secure, and beneficial. Not a poster on the wall but four questions with owners, artifacts, and re-check dates, as the loan pre-screening example shows.

A lab framework tying model training and release decisions to safety evaluations and capability thresholds.

Whether users return (Day 1, Week 1, Month 1), one of the strongest indicators of sustained value, and harder to fake than adoption spikes.

The step where the system finds the most relevant passages to hand the AI

Technical: The "R" stage: embed the question, pull the top-k nearest chunks from the vector database. Most bad RAG answers are retrieval misses: the right passage never reached the prompt, so debug this stage first. (see AI-015)

The stage that searches organizational knowledge: documents → embeddings → vector database → similarity search → relevant context. It is the RAG path in the request flow. (see AI-016)

Monitoring retrieved documents, similarity scores, chunk relevance, and search latency, because poor retrieval often explains poor answers. In the worked trace, the retrieval span checked out; the mini model was the culprit.

Verifying the RAG stage separately: correct documents retrieved, relevant chunks selected, metadata filters working. Poor retrieval, not the LLM, causes most poor answers. (see AI-016)

Giving an AI your own documents to read before it answers, so it doesn't have to guess

Technical: The open-book exam for LLMs: retrieve the most relevant passages from your knowledge base, paste them into the prompt, and instruct the model to answer from them with citations. It fixes knowledge cutoff, private-data access, and hallucination in one architecture, making it the most deployed LLM pattern in industry.

Automated re-execution of failed steps with backoff, alternative tools, and fallback models. Reliable orchestration plans for failure: API down → retry, tool timeout → alternative, approval rejected → stop.

Handling API timeouts by re-attempting with increasing delays and falling back to cached answers, instead of crashing. This is the difference between the demo and production rows of the table.

An automated pattern where a failed schema validation is fed back to the model as an error message, prompting a corrected regeneration.

The AI's actual trick: starting from pure noise and gradually cleaning it up into a real image

Technical: The learned process where a neural network predicts and removes noise step by step, starting from random noise, to produce a coherent image. (see AI-052)

When a policy over-optimizes against a reward model's blind spots, producing outputs that score well on the reward model but that a human would not actually prefer.

A model trained on human preference rankings to predict how much a person would prefer one response over another, used as an automatic reward signal in RLHF.

A simple four-part checklist (Role, Instruction, Context, Expectation) for writing a clear prompt

Technical: A structured prompt-writing framework covering Role, Instruction, Context, and Expectation, giving beginners a checklist so no critical piece of a prompt gets left out. It is a scaffold for the same system-prompt discipline covered above, not a replacement for it. (see AI-057)

The structured questions, what could happen, how likely, what impact, how mitigated, who owns it, required for every significant AI project, and revisited rather than done once.

Classifying use cases by data sensitivity, decision impact, and who is affected. Tier 1 trivia bots skip most review, while Tier 3 credit decisions get the full board. Tiered and templated governance is what let the worked example reach "yes" in nine days.

Reinforcement Learning from AI Feedback, using AI judges instead of humans to generate preference data at scale.

Reinforcement Learning from Human Feedback

Technical: a three-stage pipeline (supervised fine-tuning, reward model training, RL fine-tuning) that aligns a pretrained LLM into a helpful assistant. (see AI-051)

Improving an AI by having humans rate its answers, then training it to produce more of what people preferred

Technical: Using human preference annotations to train a reward model that guides reinforcement learning, aligning outputs with human values. DPO achieves similar results by training directly on preference data, skipping the reward model.

The pre-2017 approach that read text left to right, compressing everything into one running summary, like meeting notes passed down a line of people, with nuance fading over distance. Its sequential reading also made web-scale training impractically slow.

Telling the AI to act like a specific kind of expert before it answers

Technical: Assigning a persona ("You are a senior security engineer reviewing code for OWASP top-10 vulnerabilities") to activate the right register and knowledge for the task.

Reverting to the previous stable version when a deployment misbehaves. It was one click in the worked example, limiting damage to 5% of one language segment for six hours. Rollback procedures must be tested before they are needed.

Actually executing the rollback before you need it; a rollback that has never been run is a hope, not a plan. (see AI-029)

The full back-and-forth: you ask, the AI requests an action, your app runs it, then the AI replies

Technical: The full flow: user request → model emits a call → your code executes it → result appended to the conversation → model composes the final answer. Two model calls, one tool execution.

The component that classifies an incoming request and picks a path: a policy question goes to the RAG path on a cheap model tier, a complex task to a stronger model. (see AI-024)

The small learned network that scores each token and selects which top-k experts process it.

The specific method controlling how the image gets cleaned up step by step

Technical: The algorithm (DDPM, DDIM, DPM++) controlling how the reverse diffusion process steps from noise to image, trading off step count against image quality.

Running risky AI actions in a safe, isolated space so mistakes can't cause real damage

Technical: Running generated code or risky tool calls in an isolated environment to contain the damage a hallucinated or malicious action could cause.

Growing users, capabilities, infrastructure, and business together while holding quality and cost: the study-helper's 2,000→90,000 users broke the bill, the provider limits, and the eval coverage, and none of it was the model. Scaling AI is scaling the system around the model.

The pattern that AI models get reliably smarter as they're made bigger and trained on more data

Technical: The research finding that capability rises smoothly and predictably as parameters, data, and compute grow together. Skills nobody programmed, such as translation, arithmetic, and coding, simply emerged as models grew.

The instructions telling the AI exactly what a tool needs and how to describe it

Technical: The declared name, parameters, and descriptions of a tool the model can call. Vague schemas produce guessed or hallucinated arguments; registering fifty tools at once dilutes selection accuracy. (see AI-020)

The property that every field, type, and enum value in a model's output matches a predefined schema exactly.

Storing passwords, API keys, and tokens in dedicated secret services, never inside prompts or retrieved context, where the model could echo them into an output.

The principle that curated context beats stuffed context: retrieving just page 7 outperforms pasting the whole 10-page document, at lower cost and latency. Irrelevant tool schemas and bloated history dilute attention quality. (see AI-016)

Running model weights on infrastructure you control, trading operational burden for privacy, control, and at-scale economics.

A technique where a strong model generates instruction-response pairs to fine-tune models, replacing human annotation.

A trained reasoning behavior where the model checks its own intermediate results and backtracks from detected errors.

Matching cached responses by meaning rather than exact text, so "What time does the office open?" and "When do you open?" hit the same entry, multiplying cache effectiveness.

Caching responses keyed on embedding similarity rather than exact text match, so paraphrased duplicate requests still hit the cache.

Splitting text where the topic actually changes, not just at a fixed word count

Technical: Splitting text at points where sentence-embedding similarity drops sharply, aligning chunks with actual topic shifts rather than a token count.

Distilled durable facts like preferences.contact_channel = "email", written by an extraction step at conversation end with source, timestamp, and confidence. Next session it is injected into context and the bot simply knows.

Searching by meaning instead of exact keywords, so different wording can still find the right result

Technical: Search where query and documents both become vectors and results are nearest neighbors, finding "reset my password" when you typed "can't log in," no shared keywords needed. It still misses exact identifiers, which is why production systems add hybrid keyword search. (see AI-015)

MAJOR.MINOR.PATCH applied to prompts: major for breaking changes to output format or behavior, minor for non-breaking improvements, patch for typo fixes. A single word ("helpful" → "friendly") can warrant a version bump because it shifts tone in production.

Each step waits for the previous one, as in Research → Summarize → Translate → Publish. Simple and predictable, for when later work depends on earlier results.

The production engineering discipline of running inference at scale, including preprocessing, safety filters, retries, and scheduling engines like vLLM that can triple throughput on the same hardware. (see AI-021)

Running a new version silently beside the old on a slice of live traffic (say 5%) for a week, with humans spot-checking diffs before full rollout. (see AI-029)

Running the candidate on real traffic in parallel, logging outputs for offline analysis but never serving them

Technical: quality comparison at zero user risk. (see AI-026)

One service every AI call in the company flows through, handling auth, rate limits, per-team cost attribution, and central logging. The first move in the worked example's re-architecture, collapsing three duplicate stacks into one. (see AI-028)

A common notebook all the agents can read from and write to

Technical: A store all agents can read and write, holding common context, intermediate results, and coordination signals, alongside each agent's private, short-term, and long-term memory.

What the agent remembers from the current conversation or task

Technical: The current conversation and recent tool results kept available to the model within a single task or session. (see AI-035)

Keeping only the instructions and the most recent messages, dropping the rest

Technical: A history strategy that keeps the system prompt plus the most recent N turns and drops everything older.

A compact model offering lower cost, faster responses, and lower hardware requirements. It's ideal for mobile, edge, and internal automation, and the default tier in a routed portfolio.

The normalization step that turns raw relevance scores into percentages summing to 1, so value blending has well-defined weights.

Running only a small subset of a model's parameters for each input token, decoupling model size from per-token compute.

An agent that only does one job, like research or writing, and nothing else

Technical: An agent with one narrow role, such as researcher, planner, analyst, writer, reviewer, or tool user. The newsroom example deliberately gives the writer no search tools: it may only use the researcher's cited notes, a grounding constraint. (see AI-060)

An AI that hears your voice and replies with its own voice directly, no text step in between

Technical: A native model architecture that processes audio input and produces audio output directly, without a text transcription step.

A pattern in the data that predicts the label for the wrong reason, like "portable scanner style = pneumonia" in the hospital X-ray story or snow backgrounds in wolf photos. The model learns whatever patterns are present, including the ones you didn't know were there.

The production-like environment for final validation, covering user acceptance testing, performance testing, and security verification, before real users are affected.

A finite set of states with only valid transitions allowed, as in Draft → Review → Approved → Published, preventing processes from entering impossible states.

Keeping track of where a multi-step process is up to, so it can resume instead of starting over

Technical: Persisting workflow progress and intermediate results (order ID, approval status) so long-running processes can pause, resume, and roll back. This is the unglamorous half of orchestration, and the half that breaks first.

A limit on how many actions an agent can take before it has to stop

Technical: A maximum-iterations cap on the loop, without which agents can spin forever on impossible tasks. One of the basic guardrails alongside limited tool permissions, action logs, and approval gates for high-impact actions. (see AI-023)

The AI speaking its answer as it's being generated, rather than waiting to finish thinking first

Technical: Producing and playing audio output in chunks as it's generated, rather than waiting for a full response before playback starts.

OpenAI's guarantee that a Structured Outputs response will match the provided JSON Schema exactly, with no missing or extra fields.

Information organized as tables of rows and columns, like sales records or sensor logs. It is the easiest kind to feed classical ML models.

Passing exactly what the next agent needs and nothing more. The worked example's JSON carries facts with per-claim provenance, notes, and a trace ID, not the researcher's whole chat log. Unstructured handoffs let half-thoughts and injected content travel along uninvited. (see AI-027)

LLM output mechanically guaranteed to conform to a predefined schema, rather than free-form text that merely resembles it.

The small model trained during distillation to match the teacher's behavior.

Replace "AI" with "a very fast intern" in your pitch. If the sentence still creates value, the idea rests on a real problem, not on the technology's glamour.

Hypothetical AI decisively beyond the best humans in every domain; a stronger concept often conflated with AGI.

The first RLHF stage, where a pretrained model is fine-tuned on human-written example responses using ordinary supervised learning. (see AI-051)

Learning from labelled examples where each input has a known correct answer, answering questions like "Is this email spam?" or "What will this house sell for?" It dominates industry because labelled data plus a clear question is the most common business situation.

Compromise arriving through the models, libraries, MCP servers, plugins, and APIs an AI application depends on. Mitigations: trusted vendors, dependency scanning, updates, digital signatures.

The top-10 expected complaints with prepared responses and an escalation path carrying trace IDs to engineering; support should know the product before customers do. (see AI-028)

Training data generated by models for other models, now standard practice at the frontier. It sits alongside human annotation, user exhaust, and programmatic labels as a source of training examples. (see AI-067)

A public report explaining what a model can and can't do safely

Technical: A published document detailing a model's capabilities, limitations, and red-teaming results ahead of or alongside release.

The hidden instructions set up behind the scenes that shape every response you get from an AI product

Technical: The layered production prompt (role, context, task, format, guardrails) behind virtually every LLM product, often thousands of words long and versioned like source code. (see AI-057)

Structured data organized in rows and columns (e.g., spreadsheets, database tables), the data type classical ML handles best.

A hierarchical classification, as in Company → Engineering → Backend → API Standards, giving large organizations consistent categories and discoverability.

The large, capable model whose outputs supervise a distillation process.

Acceptable shortcuts that accelerate learning, but never on security, privacy, data integrity, or reliability, and always with a resolution plan.

A setting that controls how random or predictable the AI's word choices are

Technical: The randomness dial applied when sampling the next token from the probability distribution. It is why the same creative question yields different answers each time, while "What is 2+2?" stays identical, since one token dominates.

Google's toolkit for running AI models on phones and small devices

Technical: Google's lightweight runtime for deploying models to mobile and embedded devices, rebranded LiteRT to reflect broader framework support.

The layered strategy from the worked example: deterministic unit tests on every commit, golden-set evals on every prompt change, property tests over random inputs, an adversarial suite, and shadow deployment on live traffic. Layers 1–2 catch most regressions for cents; layer 5 catches what no offline test can.

The held-out ~10% used once, at the very end, to measure honest real-world performance. If it leaks into training decisions, the exam is void: the student saw the questions.

Additional inference-time computation, such as a reasoning model's thinking tokens, that RL training can be used to make productive rather than wasted. (see AI-063)

The part that turns your written prompt into something the image model can understand

Technical: A model (CLIP or T5) that converts a text prompt into embeddings used to condition image generation via cross-attention. (see AI-052)

User Value + Business Value + Technical Feasibility + Responsible AI = successful product; weakness in any dimension drags down the whole.

A simple way to choose between prompting, RAG, or fine-tuning depending on what you actually need

Technical: Prompting for quick iteration, RAG for dynamic knowledge, fine-tuning for consistent style and narrow task mastery — and the best systems combine all three. Avoid fine-tuning with under 100 quality examples or frequently changing knowledge. (see AI-016)

A phone slowing itself down when it gets too hot from heavy AI use

Technical: A chip reducing its performance under sustained load to manage heat, which can slow on-device inference during real-world use.

An API parameter capping how many tokens a model may spend reasoning before it must answer.

The tokens generated during a reasoning model's thinking phase, billed like output tokens even when hidden from the end user.

Requests served per second per machine, multiplied by batching many requests through the GPU together. It trades directly against latency: bigger batches serve more users but each waits longer.

A probabilistic estimate of when AGI-level capability arrives. Expert estimates span years to decades with wide error bars.

A small chunk of text, roughly a word or part of a word, that the AI reads and writes one piece at a time

Technical: The unit of text an LLM reads and writes, roughly a word or word-fragment. Each reply is generated one token per forward pass, which explains streaming, output pricing, and letter-counting blind spots. (see AI-059)

Planning ahead of time how much of the AI's "attention" to spend on instructions vs. context vs. the answer

Technical: The deliberate allocation of a context window across system prompt, retrieved context, history, and expected output, the worked example's 4,400-token support turn. The core discipline of prompt cost engineering. (see AI-020)

The core financial reality of AI products: you pay every time it is used, not once to build it. Flagship models cost 10–50× more per token than small ones, and output tokens cost several times input tokens. (see AI-059)

The tool that breaks your text into those small chunks before the AI processes it

Technical: The component that splits raw text into tokens using a fixed learned vocabulary. Every model family ships its own tokenizer, so the same text can produce different token counts on different models.

A rough rule of thumb for estimating how many chunks a piece of text will use

Technical: The density ratio used for estimation: about 1.3 for English prose, higher for code (brackets and indentation each cost tokens), much higher for many non-English languages.

An outside capability an agent can use, like web search or a calculator

Technical: An external capability the agent can invoke: web search, calculator, calendar, database, code execution. Without tools an agent can only reason; with tools it can act. (see AI-018)

An agent with unrestricted tools deleting databases, sending emails, or modifying infrastructure. Mitigations: permission boundaries, approval workflows, least privilege, and tool allow-lists. (see AI-018)

The list of tools an agent is allowed to use, and the system that actually runs them

Technical: A catalog of available tools with schemas the model can call via function calling, plus the runtime that executes those calls. (see AI-018)

Giving an AI only the narrow permission it needs for a task, not full access to everything

Technical: Exposing narrow, purpose-built operations instead of overly powerful ones — a run_sql tool hands the model your whole database. The server enforces permissions, never the model. (see AI-027)

Defining a data-extraction task as a fake tool call with a strict input schema, reusing function-calling machinery for structured data. (see AI-018)

All weights in the model including every expert; determines memory footprint and stored knowledge.

Records following one request through every component, including the retrieval span, prompt span, LLM span, and guardrail span, each with timing and status. The worked example closes a hallucination ticket in four minutes by reading one trace; without it, a week of guesswork.

The classic approach where a programmer writes explicit rules, such as "IF email contains 'FREE $$$' THEN spam." It fails predictably when a rule is missing, whereas a learned system generalizes from patterns, the central contrast of this lesson.

The process of showing a system many labelled examples (a million cat photos) so it discovers the distinguishing patterns itself. Nobody writes the rules; the data effectively writes them. (see AI-005)

The date at which the model's knowledge of the world ends, because the weights are frozen after training. Anything that happened later is simply not in the file.

The core design used inside almost every major AI model today

Technical: The architecture underlying every major LLM family, including GPT, Claude, Gemini, and Llama. They differ in data mix, scale, fine-tuning, and openness, not in the core design. (see AI-012)

Users know when AI is involved, what information influenced a decision, and what limitations exist, so applicants are told AI assists the screening.

A model releasing weights, training data, and training code, enabling full reproduction and auditing.

The compounding adoption driver built from transparency, reliability, explainability, privacy, and human oversight. It's one of the strongest competitive advantages an AI product can hold. (see AI-038)

Systematically reducing human oversight as the model demonstrates validated reliability

Technical: the controlled path from human-as-the-loop toward automation, never the reverse leap.

The week-3 additions that doubled conversion: clause-level "show source" links and a visible "verify amounts" note. Trust is a shippable feature, not a vibe. (see AI-043)

The layer validating outputs, enforcing guardrails, applying Responsible AI controls, and routing high-stakes decisions to humans before responses return. (see AI-023)

Brand capital from consistent accuracy, responsible data practices, and honest limitation disclosure, most durable in risk-averse domains like healthcare, legal, and enterprise security. (see AI-043)

The AI knowing when you've finished talking so it can respond at the right moment

Technical: Detecting when a speaker has finished so a voice AI system can respond without interrupting or leaving an awkward silence.

The original type of neural network used to clean up the noise step by step

Technical: The neural network architecture originally used in diffusion models to predict the noise at each denoising step.

A single request format (often OpenAI-compatible) that works against multiple LLM providers without application code changes.

Profitability analyzed at the level of one user or one interaction

Technical: establishing sustainable margin before scaling. If each user loses money, volume makes it worse. (see AI-046)

Free-form content with no fixed format, such as photos, emails, audio, and PDFs. Deep learning's ability to learn directly from raw unstructured data is what made it dominant. (see AI-003)

Learning from unlabelled data to find hidden structure, answering questions like "Which customers behave similarly?" Everyday uses include customer segmentation and anomaly detection.

Charging in proportion to consumption (per message, document, or API call), aligning revenue with variable cost

Technical: the natural hedge against AI's pay-per-use cost structure.

Watching users work rather than asking. Shadowing 5 reps revealed 16 of 20 minutes went to research, not writing. Users normalize their problems; observation surfaces what interviews miss.

Double-checking what the AI asks for before actually doing it, in case it got something wrong

Technical: Treating model-generated arguments as untrusted user input: validate, sanitize, and authorize before executing. Applications, never the model, remain responsible for authentication, authorization, and error handling. (see AI-027)

The ~10% data slice held out from weight updates and checked during training to tune decisions and detect overfitting. Distinct from the test set, which is touched exactly once at the end as the final honest exam.

The actual information a token hands over when its key matches a query well. The new representation of a token is a percentage-weighted blend of other tokens' values

Technical: "it" becomes 60% mat, 25% cat.

Impressive-looking numbers that inform no decision: total prompts sent, parameter counts, feature counts. Define "active use" with the harsher option: accepting output, not just opening the panel.

The tool that shrinks an image down and expands it back, used to make image generation faster

Technical: A model trained to compress an image into a small latent representation and reconstruct it back, used to enter and exit latent space in latent diffusion.

A quirky trick where you can do math with meanings, like "king" minus "man" plus "woman" equals "queen"

Technical: The party trick that directions carry meaning: king − man + woman ≈ queen, Paris − France + Japan ≈ Tokyo. Relationships are arrows and categories are neighborhoods, and stereotype directions ride along too. (see AI-062)

A specialized database built to quickly find the "closest matches" among millions of pieces of text

Technical: A store that holds millions of embeddings and answers "find the most similar vectors to this one" in milliseconds. It's the search infrastructure that turns embeddings into products. When your chatbot "searches the company docs," a vector database answered. (see AI-014)

Dependence on one provider's models or proprietary features, exposing you to their pricing, deprecation, and policy decisions.

Making review the path of least resistance: per-paragraph insert with cited sources meant agents skimmed citations naturally, versus auto-insert where habituated agents stopped reading. (see AI-053)

Having a second AI check the first AI's answer against the sources before showing it to you

Technical: A second model call checking each claim of the first against the sources, flagging unsupported statements before users see them: a cheap catch for faithfulness drift.

The transformer adapted for images

Technical: divide the image into fixed-size patches, treat each patch as a token, attend across all patches. A 512×512 image might become 256 patch tokens.

An image encoder (typically a Vision Transformer) bolted onto an LLM backbone via a projection layer, so image tokens are injected into the sequence alongside text tokens and the language model attends to both. (see AI-012)

A widely used open-source, high-throughput serving engine for running open-weights LLMs in production.

The complete list of all the chunks of text a model recognizes

Technical: The fixed set of all tokens a model knows, typically 50,000 to 200,000 entries learned from training data. English-heavy vocabularies are why Tamil or Japanese can cost 3–10× more tokens per sentence.

The system figuring out when someone is actually speaking versus silence

Technical: A technique for detecting when a person is actively speaking versus silent, used to drive turn-taking decisions.

An AI you can have a live spoken conversation with, not just text

Technical: Real-time, streaming AI interaction over spoken audio, requiring latency and turn-taking guarantees text-based multimodal AI does not. (see AI-052)

An HTTP callback

Technical: the external system POSTs event data to a URL you control the instant something happens, instead of your system polling for changes. The trigger mechanism behind most event-driven workflows and no-code automation tools.

A learned number acting as a neuron's importance dial for one input, like 0.8 for cloud cover in the rain detector. In the orchestra analogy the weights are the sheet music; the intelligence lives in their values, not in the neurons.

OpenAI's open speech-recognition model, trained on 680,000 hours of multilingual audio and transcribing 99 languages with word-level timestamps

Technical: the standard audio-input building block for multimodal pipelines.

The execution system that runs workflows reliably from start to finish: state, retries, checkpoints, human approvals, resumption after crashes. Orchestration defines what should happen; the engine guarantees how it executes. In the invoice pipeline, AI does two boxes and the engine does the reliability.

Charting the user's full flow (receive request, gather information, analyze, create, review, deliver) and adding AI only at the highest-value steps, instead of asking "what can AI do?"

An AI feature so embedded in daily process (integrations, accumulated personalization, habit) that switching means significant re-learning and lost context.

The persistent record of current step, completed tasks, pending work, errors, and results, which is what lets a workflow pause for a week-long approval and resume safely. Durable state is what separates workflow engines from scripts.

In-progress workflows keep their original definition while new runs use the latest version, preventing mid-flight behavior changes as business processes evolve.

A smaller, deliberate limit an app sets on how much context to use, well below the model's real maximum

Technical: A deliberately smaller context allocation an application designs to (for cost, latency, and quality) rather than the model's maximum window.

The context window itself: the current conversation, retrieved context, and intermediate results, free while the chat lasts and gone when it ends. The first of the three layers in the worked example. (see AI-061)

The engineering heart of memory: deciding what deserves promotion from chat to durable fact, how conflicts update when the user changes their mind, and what the user can see and delete. (see AI-056)

A widely used gradient boosting implementation, prized for speed and accuracy on tabular data and a frequent winner of Kaggle competitions.

A TypeScript schema-validation library commonly used as the schema definition layer for structured LLM extraction in JavaScript/TypeScript projects.