Level 4: Enterprise AI — Cheat Sheet

Key terms

AI Orchestration

The coordination of models, tools, data sources, and business processes into one intelligent workflow. Think of the conductor who plays no instrument but keeps state, order, retries, and the audit trail; remove it and you have capable musicians with no score.

Orchestrator

The component deciding what happens next at every stage: workflow execution, task sequencing, tool and model selection, error handling, human approval, and result aggregation. The order #4712 example chains a router, RAG step, tool call, agent step, and human gate under one conductor.

Sequential Workflow

Each step waits for the previous one, as in Research → Summarize → Translate → Publish. Simple and predictable, for when later work depends on earlier results.

Parallel Workflow

Independent tasks running simultaneously, such as searching documents, analyzing images, and querying the database all at once, then combined. Cuts total response time.

Conditional Workflow

Routing on content, so billing questions go to the finance agent, technical questions to support, and HR questions to HR. The pattern behind intelligent request routing. (see AI-024)

Event-Driven Workflow

Triggered by external events rather than a user: a new email arrives, gets classified, gets summarized, a ticket is created, and the team is notified. The backbone of enterprise automation.

State Management

Persisting workflow progress and intermediate results (order ID, approval status) so long-running processes can pause, resume, and roll back. This is the unglamorous half of orchestration, and the half that breaks first.

Retry Logic

Automated re-execution of failed steps with backoff, alternative tools, and fallback models. Reliable orchestration plans for failure: API down → retry, tool timeout → alternative, approval rejected → stop.

Multi-Agent System

An architecture where multiple specialized agents collaborate on problems too complex for one agent, like a house built by an architect, electrician, and plumber rather than one person. Each agent's context stays small and focused, and each role can run on a different model tier. (see AI-024)

Coordinator Agent

The project manager: understands the request, delegates subtasks to specialists, manages workflow state, resolves conflicts between agents, and assembles the final result.

Specialist Agent

An agent with one narrow role, such as researcher, planner, analyst, writer, reviewer, or tool user. The newsroom example deliberately gives the writer no search tools: it may only use the researcher's cited notes, a grounding constraint. (see AI-060)

Critic Agent

A reviewer checking another agent's output against source notes and a rubric, flagging unsupported claims and scoring tone. Fresh-context critique genuinely catches errors that self-review misses; that separation is the core reason multi-agent beats one mega-prompt.

Hierarchical Architecture

Manager agent delegating to specialists with clear authority at each level, the common enterprise structure. Contrast with swarm architecture, where agents communicate directly, more flexible but harder to coordinate.

Shared Memory

A store all agents can read and write, holding common context, intermediate results, and coordination signals, alongside each agent's private, short-term, and long-term memory.

Conflict Resolution

When agents disagree (Research recommends A, Analysis recommends B), the coordinator makes the final call, ensuring consistent outcomes.

Human-in-the-Loop

A human gate on critical outputs, where the legal agent drafts and the lawyer approves. Two failed critic checks send the newsroom draft back; a pass releases it to a human editor. (see AI-053)

Request-Response

One agent asks another and waits for a direct reply, as in Research Agent → Knowledge Agent → documents back. Simple, synchronous, and the most widely used pattern.

Publish-Subscribe

An agent publishes an event and all subscribed agents automatically receive it, so "policy updated" reaches the compliance, legal, and HR agents at once. Ideal for event-driven systems.

Asynchronous Communication

The sender keeps working while awaiting responses, receiving results later. This gives better scalability and faster workflows than synchronous waiting, at the cost of coordination complexity.

Structured Handoff

Passing exactly what the next agent needs and nothing more. The worked example's JSON carries facts with per-claim provenance, notes, and a trace ID, not the researcher's whole chat log. Unstructured handoffs let half-thoughts and injected content travel along uninvited. (see AI-027)

Correlation ID

A unique identifier (like "rpt-2026-07-02-441") stitching all messages, responses, and log entries of one run together for end-to-end tracing and debugging. (see AI-028)

Shared Memory

A store multiple agents read and write, holding customer profiles, project data, and intermediate results, reducing duplicated retrieval at the cost of version conflicts and access control. Enterprise platforms usually combine it with direct messaging.

Conflict Resolution

When agents disagree (Finance rejects, Travel Policy approves), the coordinator compares evidence and decides. Every enterprise workflow needs explicit rules for it.

Error Communication

Agents reporting tool failures, timeouts, and permission errors instead of silence, with retry escalation: temporary failure → retry, still failing → alternative agent, still failing → human review.

Workflow Engine

The execution system that runs workflows reliably from start to finish: state, retries, checkpoints, human approvals, resumption after crashes. Orchestration defines what should happen; the engine guarantees how it executes. In the invoice pipeline, AI does two boxes and the engine does the reliability.

Workflow State

The persistent record of current step, completed tasks, pending work, errors, and results, which is what lets a workflow pause for a week-long approval and resume safely. Durable state is what separates workflow engines from scripts.

State Machine

A finite set of states with only valid transitions allowed, as in Draft → Review → Approved → Published, preventing processes from entering impossible states.

Checkpoint

A saved snapshot of progress so a crashed step resumes where it stopped instead of restarting the whole run. Essential when workflows span days or weeks.

Retry Logic

Distinguishing temporary failures (API timeout → retry, model unavailable → switch model) from permanent ones (→ escalate to human), and recovering automatically wherever possible.

Confidence Branching

Routing on model certainty. The invoice pipeline auto-approves when classification confidence exceeds 0.9 and the amount is under $5,000, and sends everything else to human review. (see AI-053)

LLM/Code/Human Labeling

The design discipline of asking which steps genuinely need a model. Invoice validation is pure rules, no model needed or wanted. This question halves most AI workflow designs and their costs. (see AI-039)

Workflow Versioning

In-progress workflows keep their original definition while new runs use the latest version, preventing mid-flight behavior changes as business processes evolve.

Working Memory

The context window itself: the current conversation, retrieved context, and intermediate results, free while the chat lasts and gone when it ends. The first of the three layers in the worked example. (see AI-061)

Episodic Memory

Past conversation summaries, embedded and retrievable. Asking "communication preferences" surfaces last month's chat, effectively RAG over your own history. (see AI-016)

Semantic Memory

Distilled durable facts like preferences.contact_channel = "email", written by an extraction step at conversation end with source, timestamp, and confidence. Next session it is injected into context and the bot simply knows.

Memory Consolidation

Summarizing many conversations into compact long-term records, so 20 chats become one summary, preserving what matters while keeping retrieval efficient and cheap.

Memory Scoring

Ranking candidate memories by relevance, recency, importance, and similarity so only the highest-ranked enter the context budget. Retrieval quality matters as much as storage. (see AI-020)

Memory Expiration

Policies giving memories a lifetime, such as a meeting-room code that expires in a day versus a company policy that persists. Remembering everything is a mistake; so is never pruning.

Private Memory

Accessible to one user only, covering personal preferences, notes, and conversation history. Contrast with shared memory for teams, covering project docs, decisions, and standards. Enterprise systems combine both, with strict access control between them.

Write Path

The engineering heart of memory: deciding what deserves promotion from chat to durable fact, how conflicts update when the user changes their mind, and what the user can see and delete. (see AI-056)

Enterprise Knowledge Architecture

The structured organization of a company's information so AI and people can reliably discover and use it, the library's categories, catalogue, and index applied to policies, wikis, contracts, and code. An AI is only as effective as the knowledge it can access; most "our AI is wrong" complaints are retrieval faithfully serving an ungoverned corpus.

Metadata

Information about information, such as author, department, version, effective_date, tags, and security classification. The parental-leave fix in the worked example is metadata: retrieval filters to current-only via effective_date and supersedes fields. (see AI-015)

Taxonomy

A hierarchical classification, as in Company → Engineering → Backend → API Standards, giving large organizations consistent categories and discoverability.

Ontology

A formal model of relationships between concepts, such as Employee works in Department, Department owns Project, letting AI reason about meaning rather than keywords.

Knowledge Graph

Connected entities and relationships (Employee → Project → Customer → Contract → Invoice) so AI can navigate related knowledge instead of isolated documents, improving reasoning and discovery.

Hybrid Search

Combining keyword, semantic, and vector search. Usually better than any single method for enterprise corpora with both exact identifiers and paraphrased questions. (see AI-015)

Knowledge Governance

Named ownership, review dates, access permissions, retention policies, and quality standards. The worked example's stale-doc escalation and ACL-filtered retrieval respect the asker's permissions, not the bot's. (see AI-027)

Provenance

Every answer citing document and version, so employees and auditors can verify the source. The trust dividend of governed knowledge. (see AI-037)

AI Governance

The framework of policies, processes, responsibilities, and controls keeping AI systems safe, compliant, and accountable across their lifecycle: the traffic laws and building regulations of enterprise AI. Done well, it enables innovation; done badly, it becomes the committee where AI projects go to die.

Risk Tiering

Classifying use cases by data sensitivity, decision impact, and who is affected. Tier 1 trivia bots skip most review, while Tier 3 credit decisions get the full board. Tiered and templated governance is what let the worked example reach "yes" in nine days.

AI System Register

The inventory of every AI system: owner, purpose, data touched, model, eval link, review date. The artifact regulators increasingly ask for, and where every governance regime starts. (see EU AI Act)

Risk Assessment

The structured questions, what could happen, how likely, what impact, how mitigated, who owns it, required for every significant AI project, and revisited rather than done once.

Governance Board

The cross-functional group approving policies, reviewing high-risk applications, and providing strategic direction. Governance is shared across executives, engineers, business owners, risk, and security; it is never one team's job.

Compliance

Meeting external obligations such as GDPR, EU AI Act, ISO 42001, SOC 2, and HIPAA, with governance supplying the evidence: documentation, evals, audit trails, register entries. (see AI-056)

Audit

Periodic formal review of policies, security, documentation, model performance, and access. The verification that governance processes actually work, not just exist on paper.

Governance Maturity

The progression from ad hoc (anyone deploys anything) through defined policies and standardized processes to enterprise governance with continuous improvement baked into daily operations.

Responsible AI

Designing, building, and operating AI that is fair, transparent, accountable, secure, and beneficial. Not a poster on the wall but four questions with owners, artifacts, and re-check dates, as the loan pre-screening example shows.

Fairness

Treating people equitably. The bank's disaggregated evals found an approval gap, removed the proxy feature (postcode), re-measured, and documented it. Bias enters through data, prompts, and evaluation itself, rarely intentionally. (see AI-062)

Explainability

Providing meaningful reasons for outputs, such as reason codes a rejected applicant can read, not "Application rejected." If you can't write the explanation without hand-waving, the system isn't ready for high-stakes use.

Accountability

A named human owns every outcome. The credit officer is accountable, not decorative, and an appeal path reaches a person. Responsibility cannot be delegated entirely to software. (see AI-053)

Transparency

Users know when AI is involved, what information influenced a decision, and what limitations exist, so applicants are told AI assists the screening.

Human Oversight

Qualified humans review high-stakes outputs, such as medical diagnoses by doctors, contracts by lawyers, and financial approvals by managers. The bank shipped decision support, not auto-decision.

Privacy by Design

Embedding protection from the start: collect only necessary data (data minimization), cap retention, encrypt, and test the model for memorized personal-data leakage. Privacy cannot be added later. (see AI-056)

Proxy Feature

An innocuous-looking input (postcode) that correlates with a protected characteristic and smuggles discrimination into the model, found only by slicing metrics by demographic group.

Model Routing

Sending each request to the cheapest model that can handle it. The biggest lever in the worked example, saving $18,900 of a $42,000 bill by routing 70% of queries to a mini model, gated by an eval so quality stayed measured. (see AI-024)

Prompt Caching

Reusing an identical prompt prefix (like a 3,000-token system prompt) across calls at a fraction of the price. The cheapest optimization in modern LLM serving, worth $6,200/month in the example. (see AI-061)

Response Cache

Serving repeated questions ("what are the office hours?") from a stored validated answer with no model call at all, cutting cost, latency, and load simultaneously.

Semantic Cache

Matching cached responses by meaning rather than exact text, so "What time does the office open?" and "When do you open?" hit the same entry, multiplying cache effectiveness.

Output Caps

Tuning max_tokens per route to curtail verbose answers. Output tokens cost more than input, so uncapped verbosity is silent spend. (see AI-059)

Eval Gate

The safety condition on every cost lever: measure answer quality before and after. Silently swapping in a cheaper model without evals is the classic false economy: invisible quality traded for visible savings. (see AI-022)

Batch Processing

Grouping operations, such as one hundred embeddings in one API call, to improve throughput and per-unit cost.

Budget Control

Daily/weekly/monthly limits per department or project with alerts before thresholds are breached. Unmonitored token spend is the most common production AI surprise.

Enterprise AI Architecture

The blueprint connecting users, models, knowledge, orchestration, governance, and operations into one platform: nine layers behind the simple chat interface, like the hundreds of coordinated systems behind an airport's check-in desk.

Reference Architecture

A reusable blueprint of standard components, layers, and flows, a starting point for production design so each new use case doesn't reinvent the stack.

Shared Gateway

One service every AI call in the company flows through, handling auth, rate limits, per-team cost attribution, and central logging. The first move in the worked example's re-architecture, collapsing three duplicate stacks into one. (see AI-028)

Model Catalog

The approved list of models and providers with routing and fallback built in. Swap a provider once and forty use cases inherit the change. (see AI-024)

Paved-Road Template

A scaffold with guardrails, evals, and tracing pre-wired, so a new use case ships a compliant chatbot in days. The platform payoff: use case #41 shipped in two weeks with two engineers, with governance enforced by architecture instead of memos.

Trust Layer

The layer validating outputs, enforcing guardrails, applying Responsible AI controls, and routing high-stakes decisions to humans before responses return. (see AI-023)

Cross-Cutting Concern

A capability spanning every layer, such as security, identity, monitoring, governance, or cost, rather than living in one component.

High Availability

Continuity through redundancy, failover, health checks, circuit breakers, and disaster recovery, so the platform survives individual component failures.

AI Gateway

A control-plane layer between applications and LLM providers that unifies auth, routing, caching, rate limiting, and logging behind one API. (see AI-028)

Unified API

A single request format (often OpenAI-compatible) that works against multiple LLM providers without application code changes.

Model Routing

Sending a request to a specific model based on cost, latency, or task complexity, often cheap-first with escalation to a frontier model. (see AI-039)

Semantic Caching

Caching responses keyed on embedding similarity rather than exact text match, so paraphrased duplicate requests still hit the cache.

Fallback

Automatically retrying a failed or overloaded provider against a secondary provider so an outage at one vendor does not become an outage in the product.

Rate Limiting

Enforcing per-team, per-application, or per-user request caps to protect shared provider quotas and budgets.

Provider

A company or service offering LLM inference over an API, such as OpenAI, Anthropic, Google Gemini, Cohere, or a self-hosted open-source model.

Key Management

Centralizing provider API keys in the gateway instead of scattering them across application services.

Guardrail Enforcement

Running content filtering, PII redaction, and prompt-injection checks at the gateway layer as a single enforcement point. (see AI-023)

Edge-Deployed Gateway

A gateway running on a distributed network of edge locations (such as Cloudflare's) to minimize added latency.

Cost Attribution

Tracking spend per team, application, or user through gateway logs, a prerequisite for the cost-optimization work in AI-039.

Agent Harness

The runtime infrastructure, planner, tool registry, memory, state management, and sandboxing, that wraps a raw LLM into a working agent. (see AI-017)

Planner

The harness component that breaks a high-level goal into steps and decides what to do next given current state.

Tool Registry

A catalog of available tools with schemas the model can call via function calling, plus the runtime that executes those calls. (see AI-018)

Short-Term Memory

The current conversation and recent tool results kept available to the model within a single task or session. (see AI-035)

Long-Term Memory

Facts, prior decisions, and user preferences persisted across sessions so an agent doesn't start from zero each time.

State Management

Tracking an agent's progress through a multi-step task across turns, so interruptions do not force a restart from scratch.

Sandboxing

Running generated code or risky tool calls in an isolated environment to contain the damage a hallucinated or malicious action could cause.

Output Verification

Checking tool results and final outputs against expectations before trusting them, closing the agent loop rather than acting blindly.

Agent Loop

The conceptual observe-plan-act-repeat cycle a harness executes on behalf of the model. (see AI-017)

Multi-Agent Coordination

Multiple agents, each running inside its own harness, working together on a shared task via a coordination layer. (see AI-032)

Red-Teaming

The discipline of deliberately, systematically attacking a model or AI system to find failures before an adversary does. (see AI-023, AI-027)

Jailbreak

A prompt or sequence of prompts designed to bypass a model's safety training and elicit disallowed output.

Prompt Injection

Hiding attacker instructions inside content a model processes, such as a document or tool result, so the model follows them instead of the user's intent.

Multi-Turn Erosion

A jailbreak technique that gradually escalates a conversation across many turns so no single message appears harmful.

Many-Shot Jailbreaking

Filling the context window with faux examples that model undesired compliance, exploiting in-context learning to shift final-turn behavior.

PyRIT

Microsoft's open-source Python Risk Identification Toolkit for orchestrating automated, scriptable red-team attacks against LLM applications.

garak

An open-source LLM vulnerability scanner that runs a large library of known probe types against a target model automatically.

System Card

A published document detailing a model's capabilities, limitations, and red-teaming results ahead of or alongside release.

Residual Risk

A red-team finding that is documented and accepted rather than fixed or mitigated, typically after formal sign-off.

External Red-Team Network

A standing group of outside domain experts engaged by a lab to probe high-stakes categories before a major model release.

Quality-Based Routing

Directing or combining model calls to maximize output accuracy rather than minimize cost. (see AI-039)

Best-of-N Sampling

Generating multiple candidate responses from a model and selecting the strongest via a scorer, reward model, or vote.

Mixture-of-Agents

Combining outputs from multiple different models via an aggregator model that synthesizes a final answer.

Cascade Routing

Attempting a task with a cheap model first and escalating to a stronger model only when an uncertainty signal is triggered.

Judge-Based Routing

Using a separate model to evaluate multiple candidate outputs and select or score the best one. (see AI-025)

Ensemble Voting

Combining multiple model outputs, often via majority vote, to produce a more reliable final answer.

Confidence Signal

A measurable indicator, such as logprobs or a self-reported uncertainty score, used to trigger cascade escalation.

Reward Model

A model trained to score candidate outputs by quality, commonly used to select among best-of-n samples.

LLM-as-Judge

Using a language model to evaluate the quality of another model's output, applied at inference time in quality routing and at test time in evaluation.

Aggregator Model

The final-stage model in a mixture-of-agents pipeline that synthesizes multiple candidate responses into one answer.