Feature Scaling in C++

June 2026 · 12 min read

When one feature ranges over thousands and another over hundredths, any method that compares features by magnitude, distances, dot products, gradient steps, is dominated by the large-scale feature. Scaling puts every feature on a comparable footing. This post implements the two standard scalers from scratch, and shows the one rule that matters: fit the scaler on the training set, then apply it to the test set.

Two Scalers

Standardization (z-score) centers each feature to mean $0$ and scales it to standard deviation $1$:

Min-max normalization linearly maps each feature into $[0, 1]$:

Z-score is the usual default; it tolerates outliers better than min-max, whose range is set by the two most extreme points. In both, the statistics ($\mu, \sigma$ or $\min, \max$) are learned once on the training data.

A Standardizer

The scaler is a tiny object with fit (compute per-feature mean and std) and transform (apply them). Separating the two is what enforces the golden rule: fit sees only training data, transform runs on both.

#include <bits/stdc++.h>
using namespace std;
using Matrix = vector<vector<double>>;

struct Standardizer {                 // z-score: (x - mean) / std, per feature
    vector<double> mean, std;

    void fit(const Matrix& X) {
        int N = X.size(), D = X[0].size();
        mean.assign(D, 0);
        std.assign(D, 0);
        for (auto& row : X)
            for (int f = 0; f < D; ++f) mean[f] += row[f];
        for (int f = 0; f < D; ++f) mean[f] /= N;
        for (auto& row : X)
            for (int f = 0; f < D; ++f)
                std[f] += (row[f] - mean[f]) * (row[f] - mean[f]);
        for (int f = 0; f < D; ++f)
            std[f] = sqrt(std[f] / N) + 1e-12;            // guard against zero variance
    }

    Matrix transform(const Matrix& X) const {
        Matrix out = X;
        for (auto& row : out)
            for (int f = 0; f < (int)row.size(); ++f)
                row[f] = (row[f] - mean[f]) / std[f];
        return out;
    }
};

Min-max is the same shape: fit records each feature's min and max, and transform returns (x - min) / (max - min).

Running It

Make two features on wildly different scales, feature 0 around $50 \pm 10$ and feature 1 around $0 \pm 0.01$, fit on train, and check the transformed statistics:

int main() {
    mt19937 rng(7);
    normal_distribution<double> f0(50, 10), f1(0, 0.01);
    Matrix Xtr, Xte;
    for (int i = 0; i < 400; ++i) Xtr.push_back({f0(rng), f1(rng)});
    for (int i = 0; i < 100; ++i) Xte.push_back({f0(rng), f1(rng)});

    Standardizer scaler;
    scaler.fit(Xtr);                      // learn mean/std from TRAIN only
    Matrix Ztr = scaler.transform(Xtr);
    Matrix Zte = scaler.transform(Xte);   // TEST uses the same train stats
    // ... print per-feature mean and std of Xtr, Ztr, Zte ...
}

After scaling, both training features have mean $0$ and standard deviation $1$, so feature 1 is no longer drowned out by feature 0. The test set does not come out at exactly mean $0$, std $1$, and that is correct: it is transformed with the training mean and standard deviation, so its own statistics land close to but not exactly on $0$ and $1$. Re-fitting on the test set would be a leak.