CourseML System Design · 0221 minPremium
Data and labels
Where training examples come from and what makes them true: logging impressions so negatives exist, implicit versus explicit feedback, defining a label with a window, labels that arrive weeks late, selection bias, the many shapes of label leakage, splitting by time, negative downsampling with the calibration fix, class imbalance and building a training set of billions of rows.
Premium
Part of the ML System Design course, on the Premium plan
Readers with Lifetime Access read it in full. Already bought it? Sign in from the top bar.
What’s inside
- 1. Where examples come from
- 2. Log the impression, or you have no negatives
- 3. Defining the label
- 4. Labels that arrive late
- 5. Selection bias: you only see what you allowed
- 6. Label leakage
- 7. Sampling and class imbalance
- 8. Building the training set at scale
- 9. Human labels
- 10. Privacy and retention
- What goes wrong
- Drills
- Check yourself