Base Rates Change Everything
Same tool. Same score. Different population.
A risk assessment tool does not operate in a vacuum. Its predictions depend heavily on the outcome’s base rate: how common the outcome is for a population.
Risk scores are not universal constants—they are tied to populations and outcomes.
What a Base Rate Means
A base rate is simply how common an outcome is for a population.
The base rate of a probation population will likely be lower than the base rate of a parole population.
Same Tool, Different Meaning
The figure below shows a simple example.
Both groups use the same risk tool. Both individuals receive the same score. But the predicted probabilities are very different because the underlying outcome rates differ across populations.
Base rates change score meaning
The same tool and same score can correspond to different predicted probabilities when the outcome is more or less common in different populations.
A score only makes sense in relation to the population and outcome it is tied to.
How the Scores Differ
- In one population, a score of 50 might correspond to about 15% risk
- In another population, the exact same score might correspond to about 45% risk
Same tool. Same score. Different population.
Why This Matters
- Risk estimates depend on the sample the tool was developed on
- Tools may behave differently across jurisdictions
- Base rates affect how useful scores are for decision-making
Why Revalidation Matters
This is one reason tools often need revalidation and recalibration when moved to new settings.
If your state or agency has adopted a tool developed elsewhere, has it been evaluated recently?
Bottom Line
Risk scores are not universal constants. They are tied to populations, outcomes, and the settings where they are used.
Zachary Hamilton
Professor
My research centers on innovation in risk and needs assessment development.