Normalization vs Standardization

September 2026 · 10 min read

“Normalize the data” is an incomplete instruction. It might mean map each feature into a range, express each feature in standard deviations, or give every example unit length. These operations can produce very different inputs to a model.

In this series, standardization means the usual column-wise z-score transform. Min-max normalization means rescaling columns using their training minima and maxima. Scikit-learn’s Normalizer means row-wise unit-norm normalization. When reading another source, check its actual operation rather than relying on the word alone.

One feature is enough to see the difference

Take three training examples with one numerical feature: 2, 4, and 8. Their mean is about 4.667 and their population standard deviation is about 2.494. The training minimum is 2 and the maximum is 8.

A z-score subtracts 4.667 and divides by 2.494. Min-max scaling subtracts 2 and divides by the range, 6. For L2 row normalization, each row has only one coordinate: its length is that number’s absolute value. Every positive row therefore becomes 1.

Which information should the model retain?

The two column transforms retain all distinctions among 2, 4, and 8. Their order is unchanged, and the gap from 4 to 8 remains twice the gap from 2 to 4. With the saved parameters, either transform can recover the original values. This assumes a nonconstant reference column and no clipping.

Row normalization deliberately removes length. In our one-feature example, that removes every difference between positive examples. With multiple features, it retains direction: [3, 4] and [6, 8] both become [0.6, 0.8], as the unit-circle figure shows. Use it when direction is the representation you want, and decide whether to retain magnitude separately.

Array shape is therefore part of the meaning. [[2], [4], [8]] is three examples with one feature. [[2, 4, 8]] is one example with three features. A row normalizer performs a different calculation on these two inputs.

Choose the property you need

These are descriptions of behavior, not a ranking of prediction quality. The appropriate choice depends on what distance, regularization, or optimization means for the model. The when-to-scale chapter develops those connections. Compare candidate preprocessing choices using the same validation design.

Write an instruction someone can reproduce

Replace “normalize the features” with a specific instruction, such as: fit a mean and population standard deviation for each numerical column on the training rows, then reuse them to center and scale future rows.

Also state how missing values, constant columns, clipping, and sparse input are handled when they occur. These choices belong in the preprocessing configuration. During cross-validation, each training fold supplies its own fitted statistics; see the leakage notes for the evaluation boundary.

Two other meanings to keep separate

A normal distribution is a bell-shaped probability distribution. Neither z-scoring nor min-max scaling makes arbitrary data normally distributed. The before-and-after distribution plot shows why: an affine change of units preserves the shape.

Probabilities are another requirement. Mapping a column to [0, 1] does not make its entries a probability distribution or make a score calibrated. For nonzero, nonnegative rows, L1 normalization makes the row sum to 1; that arithmetic property alone does not establish that its entries estimate event probabilities.

Read the API name and the input shape together

The following code reproduces the figure. The two scalers learn from the three training rows. Normalizer exposes the same fit/transform interface but is stateless: it calculates a norm from each row independently.

import numpy as np
from sklearn.preprocessing import StandardScaler, MinMaxScaler, Normalizer

# Shape (3, 1): three examples, one feature.
train = np.array([[2.], [4.], [8.]])
future = np.array([[10.]])
for transform in (StandardScaler(), MinMaxScaler(), Normalizer(norm="l2")):
    transform.fit(train)
    values = transform.transform(np.vstack([train, future]))
    print(type(transform).__name__, values.ravel().round(4).tolist())

StandardScaler [-1.069, -0.2673, 1.3363, 2.1381]
MinMaxScaler [0.0, 0.3333, 1.0, 1.3333]
Normalizer [1.0, 1.0, 1.0, 1.0]

For the arithmetic from scratch, use the focused C++ implementations in standardization, min-max scaling, and column scaling versus row normalization. They keep the same distinction visible in code.

References and further reading

• Scikit-learn: preprocessing data — separate sections for standardization, scaling to a range, and normalization.
• Scikit-learn: StandardScaler — column statistics, variance convention, and behavior of fitted transforms.
• Scikit-learn: MinMaxScaler — training extrema, feature ranges, and optional clipping.
• Scikit-learn: Normalizer — independent sample normalization and the stateless fit interface.