Leave-One-Out (LOOCV)

September 2026 · 9 min read

Leave-one-out cross-validation (LOOCV) is K-fold CV with one validation example per fold. With $N$ examples, fit $N$ models, each using the other $N-1$ examples, and collect one held-out prediction for every row.

Each fit uses almost the entire dataset. That makes leave-one-out attractive for some small-data problems, but the resulting estimate is not automatically more reliable than a five- or ten-fold estimate. The main tradeoffs involve training size, model stability, metric choice, and computation.

Five points, five fitted lines

Take independent observations with feature values 0, 1, 2, 3, and 4, and target values 1, 2, 2, 4, and 8. Call them A–E. These are constructed cross-sectional examples, not a time series. Our recipe fits an ordinary least-squares line: an intercept plus a slope times the feature.

When E is held out, the line fitted to A–D is $0.9+0.9x$. At E’s feature value $x=4$, it predicts 4.5. E’s actual target is 8, so the error is 3.5 and its squared error is 12.25. E’s target did not participate in fitting that line.

Collect the held-out mistakes

Repeat the fit with each point held out. Mean squared error (MSE) squares each prediction mistake and averages the five squared errors:

Write $\hat y_i^{(-i)}$ for the prediction for example $i$ from the model fitted without that example. The superscript reminds us which row was excluded; it is not an exponent. With $N$ examples, the estimate is

A separate line fitted on all five points has training MSE 1.12. That is a different quantity: the model helped fit the targets whose errors it measures. The larger held-out error here exposes how poorly some points are predicted when they cannot influence their own fit.

What using almost all the data buys

For a fixed recipe and independent identically distributed examples, each leave-one-out prediction tests a model trained on $N-1$ examples. The final model would normally use $N$. That small training-size difference can make LOOCV closer to the intended training regime than a split that leaves out a large fraction.

It still does not evaluate the exact final fitted model. Nor is “nearly all the data” a universal guarantee of low bias for every target of evaluation. The comparison depends on the learning procedure, sample size, and what population or training-set size the reported risk is meant to describe.

In this five-point example, holding out E removes the largest feature value and an unusually large target. The remaining line must extrapolate to E’s location. The large error is informative about this dataset; averaging it away through a flattering choice of split would hide that difficulty.

One-point folds need a compatible score

Squared error and absolute error are defined for one held-out observation, so averaging them across leave-one-out folds is straightforward. Classification accuracy can likewise average the individual correct/incorrect outcomes.

Other metrics need a set of observations. R² uses variation around the evaluation targets’ mean; a single target has no such variation, and scikit-learn does not define R² for a one-example test set. ROC AUC needs examples from both classes, so it cannot be evaluated separately in a one-point fold.

For regression, choose an explicit loss scorer instead of relying on an estimator’s default score. For ranking or class-comparison metrics, consider folds large enough to support the metric. Computing a metric on pooled out-of-fold predictions is a separate aggregation choice; it is not generally an average of per-fold scores, and the predictions come from different fitted models.

Reproduce the fits directly

The explicit loop makes the exclusion rule visible. Every prediction is stored at its original row index, and the coverage check ensures that all five rows were evaluated once.

import numpy as np
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import LeaveOneOut

X = np.arange(5, dtype=float).reshape(-1, 1)
y = np.array([1, 2, 2, 4, 8], dtype=float)
predictions = np.full(len(y), np.nan)
seen = np.zeros(len(y), dtype=int)

for train, valid in LeaveOneOut().split(X):
    model = LinearRegression().fit(X[train], y[train])
    predictions[valid] = model.predict(X[valid])
    seen[valid] += 1

assert np.all(seen == 1)
squared_errors = (y - predictions) ** 2
print("Held-out predictions:", predictions)
print("LOO MSE:", squared_errors.mean())

# A separate fit for use after assessment.
final_model = LinearRegression().fit(X, y)
print("Full-data training MSE:", np.mean((y - final_model.predict(X)) ** 2))

Cost and stability limit the appeal

A direct implementation performs $N$ nearly full-data fits for one candidate. With 1,000 examples and ten candidates, that means 10,000 fits before a final refit. Repeating leave-one-out with a deterministic model provides the same set of exclusions again; there is no random partition left to explore.

Some models support shortcuts. RidgeCV has an efficient leave-one-out mode that avoids naively refitting a ridge model once per row. Such an estimator-specific shortcut is not a general property of arbitrary preprocessing pipelines.

Model stability also matters. Removing one influential example can noticeably change a fitted predictor. Meanwhile, all leave-one-out training sets overlap heavily, so their errors are dependent. Do not treat $N$ fold errors as $N$ independent experiments or claim that LOOCV always has lower variance. The choosing-k chapter separates training fraction, score variability, and fit cost.

Choose the unit that must be unseen

Leaving out one row is suitable only when one row is the intended unseen unit. With several measurements per patient, the other measurements may still identify the held-out patient. Group-aware validation, including leave-one-group-out, changes the exclusion unit to match that question.

For forecasting, fitting on every other row would include the future. Use chronological forecast origins instead. For a genuinely small independent dataset and an inexpensive, stable recipe, leave-one-out can be a useful assessment choice.

If these scores select among recipes, preserve a further evaluation boundary as in nested CV. After assessment, fit the selected recipe on the intended full training data. The five fold models above are assessment artifacts; the code’s final_model is a separate fit.

References and further reading

• Scikit-learn: LeaveOneOut — singleton validation folds, equivalence to K-fold with N folds, and computational cost.
• Scikit-learn: evaluating estimator performance — leave-one-out tradeoffs and metric aggregation for held-out predictions.
• Scikit-learn: RidgeCV — the estimator’s specialized efficient leave-one-out implementation.
• Scikit-learn: R² score — the minimum evaluation sample requirement.