Data Preparation: Foundations

September 2026 · 12 min read

Data preparation turns recorded observations into inputs a model can use, while preserving a meaningful evaluation. The two tasks belong together. A perfectly numeric table can still contain information that would not exist when a prediction is made.

This page follows one small example from raw columns to model inputs. Use the series map afterward for the step your own data needs; the chapters supply the deeper explanations and implementations.

First decide what a prediction represents

Name the target, the unit being predicted, and the time at which the prediction will be made. For example: “Predict whether this customer cancels next month, using information available today.” A cancellation confirmation recorded next month cannot be an input, however accurately it predicts the label.

Choose the split to match that question before fitting data-dependent transformations. New-customer prediction may require keeping a customer’s records together. Forecasting future observations requires respecting time. A random row split answers a different question when the same person, event or near-duplicate can appear on both sides. Start with splitting and stratification.

Follow a small preprocessing pipeline

Our example has a numeric measurement and a category. The training measurements are 2, missing, 6, with categories A, B, A. Two test rows have measurements missing, 8 and categories C, B. These five rows demonstrate the mechanics; they are too few for a meaningful model evaluation.

The numeric branch will fill gaps with the training median, then standardize the filled values. The category branch will create one indicator column for each training category. The branches are joined before the model. This is one suitable arrangement for the example; different columns and models can need different steps.

Fit learns the rule; transform applies it

The observed training measurements, 2 and 6, have median 4. Filling the training gap gives [2, 4, 6]. That completed column has mean 4 and population standard deviation √(8/3), about 1.633. Scaling is fitted after imputation in this pipeline, so these are the values it sees.

Let $x$ now mean a measurement after filling any gap, and $z$ its standardized value. The saved transformation is:

The missing test measurement becomes 4 and then 0. The observed test value 8 becomes about 2.449. If it changes to 100, we still subtract 4 and divide by 1.633; we do not recalculate the rule from the test rows. Otherwise the held-out observations would influence the representation used to train the model.

The category vocabulary is also fitted: it contains A and B. Their codes are [1, 0] and [0, 1]. For this example, an unseen category such as C receives [0, 0], matching OneHotEncoder(handle_unknown="ignore") without dropping a category. That is an explicit fallback, not a claim that C has no effect on the target. The one-hot chapter discusses the choice in more detail.

import numpy as np
from sklearn.compose import ColumnTransformer
from sklearn.impute import SimpleImputer
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import OneHotEncoder, StandardScaler

X_train = np.array([[2, "A"], [np.nan, "B"], [6, "A"]], dtype=object)
y_train = np.array([0, 1, 0])
X_test = np.array([[np.nan, "C"], [8, "B"]], dtype=object)

numeric = Pipeline([
    ("impute", SimpleImputer(strategy="median")),
    ("scale", StandardScaler()),
])
prepare = ColumnTransformer([
    ("numeric", numeric, [0]),
    ("category", OneHotEncoder(handle_unknown="ignore", sparse_output=False), [1]),
])
model = Pipeline([
    ("prepare", prepare),
    ("predict", LogisticRegression(random_state=0)),
])
model.fit(X_train, y_train)
fitted = model.named_steps["prepare"]
print(np.round(fitted.transform(X_test), 4))
# Columns: standardized measurement, is_A, is_B.
# [[0.0000, 0.0000, 0.0000],
#  [2.4495, 0.0000, 1.0000]]
predictions = model.predict(X_test)  # Uses the same fitted preparation.

Cross-validation repeats the fitting boundary

Cross-validation creates several training/validation partitions. Each partition needs its own fitted preprocessing. A median computed once before creating the folds already contains information from every validation fold, even though no target labels were used.

A pipeline helps enforce this boundary when given the correct rows. It cannot remove a future-derived feature or fix a split that mixes records from the same customer. Those decisions still come from the prediction question. Some target-based transformations also need internal out-of-fold values for training rows; see target encoding.

Find the chapter for the decision in front of you

These twelve subseries (plus the dedicated Data Imbalance series) form the full data-preparation collection. Read them as needed; a dataset does not require every transformation.

• Splitting and stratification — choose what “unseen” means, including groups and time.
• Data cleaning — parse types, reconcile units and investigate duplicates or recording errors.
• Missing values — understand why data is absent before choosing deletion or imputation.
• Categorical encoding — represent categories, including rare and previously unseen values.
• Feature scaling — choose units and relative feature scales that suit the model.
• Cross-validation — compare choices while respecting the data’s grouping and time structure.
• Pipelines and reproducibility — compose, save and reuse the complete fitted procedure.
• Feature transformation — change numeric shapes with log, power or quantile transforms.
• Feature engineering — build interactions, aggregates, time features and text representations.
• Outliers — distinguish unusual observations from errors and choose how to handle them.
• Feature selection — choose a subset of the original measurements.
• Class imbalance — connect sampling, weighting, thresholds and evaluation to the mistakes that matter.
• Dimensionality reduction — create fewer coordinates for reconstruction, prediction or exploration.

For implementation practice, the C++ data-preprocessing companion builds the underlying operations explicitly. For a project, start with a simple valid split and baseline, then add a transformation when it addresses a concrete issue and improves the appropriate validation result.

References and further reading

• scikit-learn: data leakage and inconsistent preprocessing — the distinction between learning transformations and applying the saved rules.
• scikit-learn: pipelines and composite estimators — sequential stages, column-specific branches and integration with model selection.
• scikit-learn: StandardScaler — the fitted mean and population-variance convention used in the example.
• scikit-learn: OneHotEncoder — the category vocabulary and handling of unknown categories.
• scikit-learn: cross-validation — validation strategies for independent, grouped and time-dependent observations.