Large Language Models

Why This Matters

LLMs are the technology of the decade: ChatGPT, Claude, Gemini, Copilot, and thousands of products are all LLMs in different wrappers. Prompting, RAG, agents, fine-tuning, everything covered after this lesson builds on the mental model you form here. The single most clarifying fact is also the strangest: underneath the apparent intelligence is just next-word prediction, scaled beyond intuition.

Everyday Analogy

Play "finish the sentence": "The capital of France is ___." You said Paris, and not because you queried a database. A lifetime of reading makes Paris overwhelmingly the most plausible continuation. Now: "I'm sorry to hear about your loss, please accept my ___." You produced "condolences": grammar, social context, and empathy compressed into one plausible next word.

An LLM plays this game with superhuman reading experience, trillions of words, and it plays it one token at a time, every time. Everything it does is this game.

What "Large" Means

Three dimensions, each staggering:

• Parameters: billions to hundreds of billions of learned weights (AI-004); GPT-2 had 1.5B; frontier models have hundreds of billions (often MoE, AI-064).
• Training data: trillions of tokens (AI-059), essentially the digitized public text of humanity (AI-067 covers the pipeline).
• Compute: months on tens of thousands of GPUs; frontier runs cost tens of millions (AI-005).

The shocking research finding, known as the scaling laws, is that capability rises smoothly and predictably with all three. Skills nobody programmed (translation, arithmetic, coding) simply emerged as the models grew.

How a Reply Is Actually Generated

You type: "Write a haiku about rain." The model:

1. Tokenizes your text into ~8 tokens (AI-059).
2. Runs a forward pass (AI-008) producing a probability for every token in its vocabulary as the next token. Perhaps: "Soft" 12%, "Rain" 9%, "Gentle" 7%…
3. Samples one token, say "Soft," influenced by a randomness dial called temperature.
4. Appends it and repeats. The input is now your prompt + "Soft"; predict again. And again. One forward pass per token, until a stop signal.

That is the entire mechanism. It explains the phenomena you observe: replies stream word by word (each token is a fresh computation), output costs more than input (AI-059), the same question yields different answers (sampling), and long responses take longer (more passes).

Why Next-Word Prediction Produces "Understanding"

Here is the deep idea: to predict the next word really well, you are forced to model the world that produced the text. Predicting the next line of a Python program requires learning Python's rules. Predicting the end of a detective story requires tracking who knew what. Predicting a chemistry answer requires compressing chemistry. Prediction pressure, applied to humanity's text at scale, distills a compressed model of grammar, facts, styles, and reasoning patterns into the weights.

Compressed, not perfect. The same design guarantees the limitations:

• Hallucination (AI-060): the model always produces a plausible next token, even where it has no knowledge, because plausibility is the only currency it has.
• Knowledge cutoff: it knows nothing after its training data ends (AI-007).
• No memory between calls: each conversation is replayed into the context window every turn (AI-061).
• Tokenizer blind spots: letter counting and long arithmetic stumble on token boundaries (AI-059).

Base models are then shaped into assistants through fine-tuning and preference training (AI-067). That's why ChatGPT chats instead of just completing your sentence.

The LLM Landscape

All are transformers (AI-012) trained on the same recipe; they differ in data mix, scale, fine-tuning, and openness.

Real-World Showcase

• ChatGPT reached 100M users in two months, the fastest-adopted consumer product in history at the time, powered entirely by next-token prediction.
• GitHub Copilot writes a meaningful share of committed code at millions of companies. It's an LLM predicting the next token of code.
• Customer-service transformation: Klarna reported its LLM assistant handling the workload of hundreds of human agents within months of launch.

Try It Yourself

1. Be the LLM: take any book, read a sentence aloud, and have a friend guess the next word before turning the page. Score them over 20 sentences. Where they succeed and fail (common phrases: easy; specific facts: hard) mirrors LLM strengths and weaknesses exactly.
2. Watch the sampling: ask a chatbot the same creative question three times ("Give me a startup name for a plant shop"). Different answers each time = temperature and sampling at work. Then ask "What is 2+2?" three times and you get identical answers, because one token dominates the probability.
3. Catch the streaming: watch a long chatbot answer appear word by word. You are literally watching one forward pass per token, live.

Common Mistakes

• Thinking the LLM looks things up. There is no database; there are only weights (AI-007).
• Treating fluency as truth: plausibility is the objective, and correctness is a frequent byproduct (AI-060).
• Believing it learns from your chats, when context is not training (AI-061).
• Dismissing it as "just autocomplete." That's technically true and profoundly misleading, since at scale, prediction forced real structure into the weights.
• Ignoring the wrapper. Products add system prompts, tools, and safety layers around the raw model (AI-010, AI-021).

Key Takeaways

• An LLM does one thing: predict the next token. Everything else emerges from doing it at scale.
• Replies are generated one token per forward pass, which is why streaming, sampling variety, and output pricing all work the way they do.
• Prediction pressure compressed a world-model into the weights, though imperfectly: hallucination and cutoffs are design consequences, not bugs.
• Scaling laws made capability predictable; assistants are base models shaped by fine-tuning.
• Every major chatbot is this same architecture with different data, scale, and polish.