Deletion: Listwise and Pairwise

September 2026 · 9 min read

Deletion analyzes the values already present without filling gaps. Its consequences depend on which columns the analysis needs, which rows survive, and what quantity you want to estimate.

Two common forms are listwise deletion, which keeps rows complete on all required columns, and pairwise deletion, which uses a separate available subset for each pair of columns. Neither changes a missing value into an observed one. Both can change which cases contribute to the result.

Listwise deletion starts with the required columns

Suppose an analysis requires A, B, and C. A row survives only if all three are observed. In the constructed table below, rows 1 and 3 survive. If the analysis requires only A and B, row 5 can also be used: its missing C is irrelevant to that particular calculation.

Dropping rows also changes alignment. Keep row IDs or a shared Boolean mask so that features, targets, sample weights, and group IDs all retain the same examples in the same order.

Small per-column missing rates can remove many rows

For a planning calculation, assume each of $d$ required columns is independently missing with probability $p$. A given column is present with probability $1-p$. A row survives only when all $d$ are present, so multiply those probabilities:

With a 5% missing rate in each of 20 independently missing columns, the retained fraction is $0.95^{20}\approx0.3585$. That means roughly 359 survivors per 1,000 rows on average, even though no individual column has many gaps.

The independence assumption matters. If the same rows are missing all 20 columns, the retention pattern is very different. Measure actual row completeness and its breakdown by class, group, and date; do not infer it from a list of column percentages alone.

Validity depends on the quantity being estimated

Under MCAR, complete cases form a random subset with respect to the relevant data, which supports many ordinary complete-case estimates while sacrificing information. It is a useful sufficient condition, not a universal necessary condition for every analysis.

For example, consider a correctly specified regression with fully observed predictors and missing outcomes. If an outcome’s chance of being observed depends only on those predictors, the conditional outcome distribution at a fixed predictor value is unchanged by selection. The regression relationship can remain estimable where there is adequate observed support, even though the overall distribution of observed predictors changes.

If selection still depends on the unseen outcome after conditioning on the predictors, that reasoning no longer applies. A label such as MAR or MNAR should be connected to the actual variables and estimand; it is not a blanket approval or rejection of deletion.

Pairwise deletion uses a different subset for each relationship

In the first table, the A–B correlation uses rows 1, 3, and 5; A–C uses rows 1, 2, and 3; B–C uses rows 1, 3, and 4. Each pair has three available cases, but they are different cases.

This can be useful for an exploratory table when you also report each pair’s sample size. It does not produce a completed feature matrix for a model. It can also produce a collection of correlations that could not all arise from one common set of complete observations.

Row   A   B   C
1     1   1   —
2     2   2   —
3     1   —   1
4     2   —   2
5     —   1   2
6     —   2   1

For prediction, report who receives a prediction

A service that rejects incomplete inputs needs an explicit fallback or abstention policy. Its accuracy among accepted inputs does not describe all incoming requests.

If production must score incomplete inputs, train and evaluate a procedure that handles those inputs. Appending a serving-time imputer to a model assessed only on complete cases does not establish its performance on imputed inputs. Compare a coherent imputation pipeline, a model that handles missing features, or an explicit fallback using representative held-out data.

C++: return retained row indices

The implementation checks only the requested columns. Returning indices makes it possible to apply the same selection to every aligned array.

#include <cmath>
#include <iostream>
#include <limits>
#include <stdexcept>
#include <vector>

std::vector<std::size_t> completeRows(
    const std::vector<std::vector<double>>& X,
    const std::vector<std::size_t>& columns) {
    std::vector<std::size_t> kept;
    for (std::size_t i = 0; i < X.size(); ++i) {
        bool complete = true;
        for (std::size_t j : columns) {
            if (j >= X[i].size())
                throw std::invalid_argument("column outside row");
            if (std::isnan(X[i][j])) complete = false;
        }
        if (complete) kept.push_back(i);
    }
    return kept;
}

int main() {
    const double NA = std::numeric_limits<double>::quiet_NaN();
    std::vector<std::vector<double>> X = {
        {1, 10, 100}, {2, NA, 200}, {3, 30, 300},
        {NA, 40, 400}, {5, 50, NA}, {6, NA, NA}
    };
    std::vector<int> labels = {0, 1, 1, 0, 1, 0};
    auto kept = completeRows(X, {0, 1, 2});
    for (std::size_t i : kept)
        std::cout << "row " << i + 1 << ": label " << labels[i] << "\n";
    // Reuse these same indices for X, labels, weights, and row IDs.
}

row 1: label 0
row 3: label 1

Before adopting deletion, record which columns require completeness, how many cases are lost, what changes in the retained population, and what happens to incomplete inputs at deployment. Those checks make the decision interpretable without turning a convenient data-cleaning operation into an unsupported statistical assumption.

References and further reading

• Van Buuren: listwise and pairwise deletion — information loss and the risk of incompatible pairwise covariance estimates.
• Van Buuren: when complete-case analysis is useful — why some conditional regression analyses remain valid under weaker assumptions than MCAR.
• R statistics manual: correlation and covariance — pairwise-complete observations and the positive-semidefiniteness limitation.