4.6 Training Data and Testing Data

You split your data: most of it to learn from, some held back to test on. Never test on what you trained on.

Chapter 4: Introduction to Machine Learning (ML)

If you study using the exact questions that will be on the exam, a perfect score doesn't prove you understand anything. Same with AI.

So data gets split. The training set (often around 70–80%) is what the model learns from. The test set (the rest) is locked away and only used at the end to check real performance. Many projects also keep a validation set for tuning along the way.

If a model aces training data but flops on test data, it's overfitting - it memorised instead of learning. The opposite problem, underfitting, is when it's too simple to learn the pattern at all.

Real talk

Overfitting is the AI version of memorising flashcards word for word and then panicking when the exam rephrases the question.

Key takeaways

Train on one part, test on a separate part

Typical split: ~70% train / 15% validation / 15% test

Overfitting = memorising, not learning