8.7 Model Accuracy and Performance
Accuracy alone can lie. Precision, recall and the confusion matrix show what kinds of mistakes a model makes.
Chapter 8: Data and AI Algorithms
Accuracy = how often the model is right overall. Sounds perfect, but it can be misleading. If only 1% of emails are spam, a model that says "not spam" every time is 99% accurate and completely useless.
So we use a confusion matrix to see the four outcomes: true positives, false positives, false negatives and true negatives. From that we get precision (when it says "spam", how often is it right?) and recall (of all the real spam, how much did it catch?).
Which matters more depends on the job. For a cancer screening tool, missing a case (low recall) is worse. For a spam filter, sending a real email to spam (low precision) is the bigger problem.
Key takeaways
Accuracy can be misleading on unbalanced data
Confusion matrix shows the types of mistakes
Precision = how trustworthy a "yes" is; recall = how many real cases are caught