Handling Imbalanced & Messy Real-World Data

> A classifier that never once predicts a failure scores 90% accuracy on a 10%-failure-rate dataset — and oversampling the exact same data one step too early silently turns 47.7% of a "test" set into duplicates of what the model was trained on.