Predictive Shrinkage
Tools do not always transport cleanly.
Risk assessment tools often perform best where they were originally developed. But what happens when agencies apply them somewhere else?
Risk tools are not universally stable—their predictive performance can shrink across populations and applications.
What Predictive Shrinkage Means
The figure below shows an example from Hamilton et al. (2025) examining predictive shrinkage in two widely used tools:
- LS/CMI; LSI-R
- ORAS
In both cases, predictive performance (AUC) declined as the tools were applied outside their original development settings.
Predictive performance can shrink
Tools may lose predictive accuracy when applied outside the populations or settings where they were originally developed.
Predictive shrinkage reflects declines in performance across populations, agencies, or jurisdictions.
Why Shrinkage Happens
Jurisdictions differ in:
- base rates
- demographics
- correctional practices
- supervision structures
- outcome patterns
As a result, tools may not transport cleanly across agencies or regions.
Why This Matters
- Agencies may rely on tools that perform differently in their local population
- Predictive accuracy can decline over time or across jurisdictions
- Revalidation and recalibration may be necessary to maintain performance
Why Local Evaluation Matters
If your state or agency has adopted a tool developed elsewhere, has its AUC been evaluated for predictive shrinkage?
Bottom Line
Risk tools are not universally stable. Their predictive performance can shrink across populations, jurisdictions, and applications.
Zachary Hamilton
Professor
My research centers on innovation in risk and needs assessment development.