Predictive Shrinkage

Mon, 25 May 2026 00:00:00 +0000

Module 9 · Risk Tool Lessons

Tools do not always transport cleanly.

Risk assessment tools often perform best where they were originally developed. But what happens when agencies apply them somewhere else?

Key takeaway

Risk tools are not universally stable—their predictive performance can shrink across populations and applications.

The figure below shows an example from Hamilton et al. (2025) examining predictive shrinkage in two widely used tools:

In both cases, predictive performance (AUC) declined as the tools were applied outside their original development settings.

The idea: Predictive performance can shrink when tools are applied in new populations or jurisdictions.

Illustration

Tools may lose predictive accuracy when applied outside the populations or settings where they were originally developed.

Predictive shrinkage reflects declines in performance across populations, agencies, or jurisdictions.

Jurisdictions differ in:

As a result, tools may not transport cleanly across agencies or regions.

If your state or agency has adopted a tool developed elsewhere, has its AUC been evaluated for predictive shrinkage?

Risk tools are not universally stable. Their predictive performance can shrink across populations, jurisdictions, and applications.