Optimise a metric and watch it stop meaning anything, fool a classifier with changes no human can see, then learn to evaluate models without deceiving yourself.
3 units, about 3 hours. Free.
Demonstrate Goodhart's law in code and explain why optimising a proxy destroys its meaning.
2Construct an adversarial example against a linear classifier and explain why high dimensions make models fragile.
3Demonstrate how test contamination inflates accuracy and apply the discipline of clean held-out evaluation.