Probability Calibration and Reliability
Two models can rank every row in the same order while reporting very different probabilities. ROC-AUC treats them as identical, but a cost calculation does not. A reported risk of $0.8$ must behave like an $80\%$ event if that number drives a decision.
This agreement between predicted probability and observed frequency is calibration. This page explains how to inspect it, measure it, and repair it without discarding a useful ranking.
What Calibration Means
A classifier is calibrated when events assigned probability $q$ occur at rate $q$:
$$\Pr\big(Y=1\mid \hat p=q\big)=q,$$For example, among many rows assigned $0.7$, about $70\%$ should be positive. Calibration does not say whether those rows were ranked above lower-risk rows.
A second property is sharpness, the ability to produce probabilities away from the overall base rate when the features contain useful information. A model that always reports the exact positive rate is calibrated in the population, but it has no discrimination because every row receives the same value. Useful probability forecasts need both separation between risks and agreement with observed frequencies.
- Calibration-in-the-large checks only whether the average predicted probability matches the overall positive rate.
- Conditional calibration checks whether predicted and observed rates agree across the probability range.
A model can pass the first check while being too confident at both ends. One average cannot reveal that shape.
Calibration Intercept and Slope
A compact diagnostic fits one logistic regression using the old model's logit as the only input:
$$\operatorname{logit}\Pr(Y=1\mid \hat p)=\alpha+\gamma\,\operatorname{logit}(\hat p).$$Perfect calibration gives $\alpha=0$ and $\gamma=1$.
- With $\gamma$ fixed at $1$, $\alpha>0$ means the event is more common than predicted, while $\alpha<0$ means it is less common than predicted.
- A slope $\gamma<1$ pulls extreme old logits toward zero, indicating that the old probabilities were too confident.
- A slope $\gamma>1$ spreads the logits outward, indicating that the old probabilities were too close to the middle.
When both values are estimated together, they can partly compensate for each other, so the intercept-only and slope diagnostics are also useful separately. Platt scaling fits the same intercept-and-slope form, but uses it as a correction for future scores rather than only as a report.
The Reliability Diagram
A reliability diagram estimates conditional calibration in four steps:
- Predictions are grouped into probability bins.
- The mean predicted probability is computed in each bin.
- The observed positive fraction is computed in the same bin.
- Observed frequency is plotted against mean prediction.
The diagonal represents perfect agreement. A point below it means the model predicted too high in that bin. A point above it means the model predicted too low. Low-risk points above the line and high-risk points below it form the usual pattern of overconfidence. The reverse pattern indicates underconfidence.
▶ Building the Reliability Diagram and Accumulating Calibration Error
Five bins over a fixed validation set of 800 predictions. Each step plots one bin at its mean predicted probability against its observed positive frequency, draws the gap to the diagonal, and adds the weighted gap to the expected calibration error. Observed frequencies and the error are computed at runtime.
In the animation, the lowest bin predicts $0.10$ but has an observed rate of $0.18$. The highest predicts $0.90$ but has an observed rate of $0.75$. Both ends are too extreme. Expected calibration error is $0.084$, and the largest bin gap is $0.15$. Yet the overall mean prediction is $0.490$ and the observed rate is $0.476$. The average is close even though the probability levels are not.
Measuring Calibration Error
Expected calibration error (ECE) averages the absolute bin gaps, weighted by the number of rows in each bin. Maximum calibration error (MCE) reports the largest displayed bin gap:
$$\mathrm{ECE}=\sum_{b=1}^{B}\frac{n_b}{N}\,\big|\bar p_b-\bar y_b\big|,\qquad \mathrm{MCE}=\max_{b}\,\big|\bar p_b-\bar y_b\big|,$$Here, $n_b$ is the bin size, $\bar p_b$ is its mean prediction, and $\bar y_b$ is its positive rate. ECE and MCE depend on the bin boundaries. Wide bins can hide local error; narrow bins contain fewer rows and become noisy. Therefore the binning method and bin count belong beside the reported value. MCE is the worst observed bin, not a guarantee about every possible probability level.
The Brier score has a useful grouped decomposition:
$$\text{Brier}=\underbrace{\sum_b\frac{n_b}{N}\big(\bar p_b-\bar y_b\big)^2}_{\text{reliability (lower better)}}-\underbrace{\sum_b\frac{n_b}{N}\big(\bar y_b-\bar y\big)^2}_{\text{resolution (higher better)}}+\underbrace{\bar y\,(1-\bar y)}_{\text{uncertainty}}.$$- Reliability measures squared calibration gaps. Zero is ideal.
- Resolution rewards groups whose outcome rates differ from the overall rate. It measures useful separation in risk.
- Uncertainty depends only on the overall positive rate and cannot be improved by the model.
A strictly increasing recalibration preserves score order. Isotonic regression can merge adjacent scores into ties, so empirical ranking or resolution can change slightly.
Why Models Miscalibrate
Several forces pull probabilities away from the diagonal, and recognizing them points to the right fix.
| Cause | Effect on calibration |
|---|---|
| Strong regularization | Shrunken logits can make probabilities too close to the middle. |
| Class weighting or resampling | An altered class prior shifts probabilities unless an intercept correction or recalibration restores the population prior. |
| Model family | Tree ensembles, margin scores, and neural networks can produce useful rankings without calibrated probabilities. |
| Missing model structure | An omitted curve or interaction can make probabilities systematically wrong in one region. |
| Distribution shift | Changes after training can break calibration even when much of the ranking survives. |
Logistic regression has one limited training-set guarantee. If its intercept is present and unpenalized, the intercept score equation gives $\sum_i p_i=\sum_i y_i$. The mean fitted probability equals the training positive rate, even when the slopes are regularized. This says only that one average matches. It does not guarantee conditional calibration or held-out calibration. Class weighting changes the equality to a weighted one, and penalizing the intercept removes it.
Post-hoc Recalibration
When ranking remains useful, post-hoc recalibration learns a new mapping from score to probability on held-out data. The base model stays unchanged.
Platt scaling fits a one-input logistic regression:
$$\hat p_{\text{cal}}=\sigma(a\,s+b),$$The slope $a$ and intercept $b$ correct both spread and overall level. With a positive slope, ranking is preserved. Two parameters make Platt scaling relatively stable on modest calibration sets, but its sigmoid shape cannot match every reliability curve.
Isotonic regression fits a non-decreasing step function. Adjacent score groups that violate monotonicity are pooled until their fitted frequencies increase with score. It can represent many shapes, but its flexibility needs more calibration data and can create tied outputs.
Temperature scaling divides a logit by one positive value:
$$\hat p_{\text{cal}}=\sigma\!\left(\frac{z}{T}\right),$$A value $T>1$ softens probabilities toward the middle, while $T<1$ makes them sharper. Temperature scaling preserves ranking exactly and is common for neural-network logits. Because it has no intercept, it cannot by itself correct a pure change in the base rate.
Splits, Cross-Validation, and Leakage
A calibrator evaluated on the same rows that fitted it looks too good. A simple pipeline has separate roles:
- The training split fits the base model.
- A calibration split fits the score-to-probability map.
- Validation predictions select any threshold.
- An untouched test split evaluates the frozen chain once.
Calibration and threshold selection can share one validation split when data are limited, provided the final test set remains untouched.
Cross-validated calibration produces an out-of-fold score for every development row. The calibrator sees only those out-of-fold scores, not optimistic in-sample predictions. Hyperparameter tuning requires this procedure inside the training side of each outer evaluation fold. Patient, account, group, and time boundaries must remain intact during splitting.
Shift, Decay, and Recalibration
Calibration belongs to a particular population and time period. Under pure prior shift, the feature distribution inside each class stays the same while prevalence changes from $\pi_{\mathrm{old}}$ to $\pi_{\mathrm{new}}$. Only an intercept correction is needed:
$$\operatorname{logit}(p_{\mathrm{new}})=\operatorname{logit}(p_{\mathrm{old}})+\operatorname{logit}(\pi_{\mathrm{new}})-\operatorname{logit}(\pi_{\mathrm{old}}).$$A changed calibration slope indicates more than a base-rate move, such as changed feature effects or measurements. Platt or isotonic scaling can repair the mapping while ranking remains useful. If ranking also decays, the underlying model needs refitting rather than another probability patch.
Production calibration is estimated from recent representative labels, checked on a later time window, and versioned with the base model and threshold.
Edge Cases and the Implementation
- Binning. ECE and MCE need the bin count and binning rule beside them.
- Leakage. Training-set predictions are too optimistic for fitting or evaluating a calibrator.
- Small calibration sets. Isotonic regression can overfit when each score region contains few outcomes.
- Ranking. Positive-slope Platt and temperature scaling preserve order. Isotonic scaling preserves order but can create ties.
The C++ calibration section builds reliability bins, calibration-error summaries, and post-hoc mappings. A dedicated evaluation and calibration chapter is also planned.