From Scratch · C++ · Machine Learning · Data Preparation and Preprocessing

Data Preprocessing from Scratch in C++

June 2026 · Series overview

The model posts in this series all start from a tidy numeric matrix that is already shuffled and split. Real data almost never looks like that. This sub-series implements the everyday preprocessing steps from scratch in plain C++, the same standard-library-only style as the rest of the series, and links each step to the model that consumes its output. For the conceptual treatment (the why behind each step, with animations), see the companion Deep Learning · Data Preparation and Preprocessing series.

A useful way to think about preprocessing is as two jobs: shaping the data into a clean numeric matrix (encoding categories, filling missing values, scaling features) and partitioning it so the model can be trained and evaluated honestly (splitting, cross-validation).

The golden rule: fit on train, apply to test. Every statistic a transform needs, a column mean for imputation, a min and max for scaling, the set of categories for one-hot, must be learned from the training data only and then applied unchanged to the test data. Peeking at the test set while preparing the data leaks information and quietly inflates the score. Each post below follows this rule.

The Steps

Train/Test Split & Shuffling — the shuffle-and-cut split, and a stratified split that preserves class ratios on imbalanced data. Feeds every model's 80/20 split.
Feature Scaling — standardization (z-score) and min-max normalization, with statistics fit on the training set. Matters for distance- and gradient-based methods such as gradient boosting.
Encoding Categorical Features — turning text labels into the numbers a decision tree can split on, via label and one-hot encoding.
Handling Missing Values — mean and median imputation from training-set column statistics.
K-Fold Cross-Validation — a lower-variance accuracy estimate than a single split, demonstrated on the decision tree and a natural companion to the forest's out-of-bag error.

None of these needs a library: a split is a shuffle and a cut, scaling is a mean and a standard deviation, encoding is a lookup table. Building them by hand makes the golden rule above concrete, because you can see exactly which numbers come from the training set.