Model Evaluation & the Bias-Variance Tradeoff

In November 2013 the American College of Cardiology and the American Heart Association published new cholesterol guidelines built around a risk calculator. Its pooled cohort equations turn six risk factors, among them age, cholesterol and blood pressure, into the probability of a heart attack or stroke within ten years, and the guidelines recommended considering a statin once that probability reached 7.5%. Applied to a national health survey, the new rules would have made 12.8 million more American adults eligible for statins. The same month, Paul Ridker and Nancy Cook of Brigham and Women's Hospital ran the calculator on three large cohorts and found that it overestimated risk by 75% to 150%.