Data

Why This Matters

Ask any ML practitioner where their time goes: not algorithms, but data. Collecting it, cleaning it, labelling it, fixing it. "Garbage in, garbage out" is the oldest saying in computing, and ML industrialized it: a mediocre algorithm with excellent data beats a brilliant algorithm with bad data, almost every time. Every model failure you will ever debug traces back, more often than not, to the data.

Everyday Analogy

A chef is only as good as their ingredients. Give a world-class chef wilted vegetables and spoiled fish, and no technique rescues the dish. Give a decent home cook pristine ingredients, and dinner is good. Models are chefs; data is the ingredients. Most kitchens spend more on sourcing than on knives.

The Data Types You'll Encounter

The pairing matters: supervised learning (AI-002) needs labelled data, and labels usually come from paid human effort, which is why labelled data is the expensive kind. LLMs cracked this economics by pre-training on unlabelled internet text, where the "label" is simply the next word (AI-009).

What Makes a Dataset Good?

Four properties determine everything:

1. Quantity. Deep models are data-hungry: thousands of examples for simple tasks, millions for vision, trillions of tokens for LLMs.
2. Quality. Wrong labels teach wrong lessons. A dataset where 10% of "cat" labels are actually dogs caps your accuracy at ~90% before training even starts.
3. Diversity. The dataset must cover the situations the model will face. Trained only on daytime driving photos? The model is blind at night, literally.
4. Freshness. The world drifts. A fraud model trained on 2019 patterns misses 2026 scams. Data has a shelf life (AI-028 covers drift monitoring).

Worked Example: How Data Problems Become Model Problems

A hospital builds a pneumonia detector from 100,000 chest X-rays. It scores 94% in testing. Deployed at a different hospital, accuracy collapses. Why?

Investigation finds the training X-rays from the sickest patients were taken with portable scanners (patients too ill to move), which produce a subtly different image style. The model had partly learned "portable scanner style = pneumonia", a spurious correlation in the data, invisible in the aggregate accuracy. At the new hospital with different equipment, the shortcut failed.

This really happened (documented in published research), and it is the canonical data lesson: the model learns whatever patterns are in the data, including the ones you didn't know were there. The bias lesson (AI-062) is this same principle with human stakes.

Where Labelled Data Comes From

• Human annotation: armies of labellers drawing boxes and tagging text; entire companies (Scale AI, Appen) exist for this.
• User exhaust. Every "Mark as spam" click, every star rating, every skipped song is free labelling by users.
• Programmatic labels, where rules or older models pre-label data that humans then verify.
• Synthetic data: models generating training data for other models, now standard at the frontier (AI-067).

Real-World Showcase

• ImageNet. 14 million hand-labelled images, assembled by a professor most people doubted, became the dataset that ignited deep learning (AI-003). The lesson: the dataset was the breakthrough.
• Common Crawl, a nonprofit's scrape of the public web, quietly became the backbone of nearly every LLM's pre-training diet.
• Recaptcha: those "select all traffic lights" puzzles labelled millions of street images for free for over a decade. You have personally labelled training data.

Try It Yourself

1. Audit a dataset in your life: open your photo gallery and imagine training a "my friends" recognizer on it. Check diversity: do you have photos of each friend indoors and outdoors, close-up and far? The gaps you find are exactly where the model would fail.
2. Feel labelling costs: time yourself labelling 20 emails as spam/not-spam. At your pace, price out one million labels at $15/hour. (Typical answer: tens of thousands of dollars, and that is for a small dataset.)
3. Find the spurious correlation: a model trained on photos labelled "wolf" vs "husky" achieved high accuracy by learning that wolf photos had snow backgrounds. What would you add to the dataset to break that shortcut? (Answer: wolves without snow, huskies in snow.)

Common Mistakes

• Fixating on model architecture while the dataset rots. Practitioners improve data first.
• Trusting aggregate accuracy: always slice metrics by subgroup and situation (AI-062, AI-025).
• Assuming more data fixes everything, when more of the same biased data just entrenches the problem.
• Ignoring freshness. A deployed model degrades silently as the world drifts away from its training data.
• Forgetting privacy and consent, since data is regulated (GDPR, AI-056) and reputationally explosive.

Key Takeaways

• The model IS its data: capabilities, blind spots, and biases all inherit from the training set.
• Labelled data is the expensive kind; whole industries exist to produce it.
• Good datasets need quantity, quality, diversity, and freshness; diversity failures cause the ugliest surprises.
• Models learn hidden patterns too (spurious correlations); the X-ray scanner story is the eternal warning.
• Data work is 80% of applied ML, and it is where debugging almost always leads.