Feature Transformation
A feature transformation changes what differences in a measurement mean to a model. A logarithm makes ratios easier to express; a quantile map expresses position within a distribution. The aim is a useful representation for the task, not a normal-looking histogram for every column.
This series follows feature scaling. It introduces each transformation through small inputs and visible changes in spacing, then explains what must be fitted and what information can be lost. Detailed algebra and implementations sit in optional sections.
Scaling changes units; a nonlinear map changes relative gaps
Take the training values [1, 2, 10, 100]. Dividing by the training maximum, 100, multiplies every gap by the same factor. A logarithm or a rank map changes the gaps by different amounts:
For a distance-based model, those gaps affect which observations look close. For a linear predictor, using log(x) instead of x changes the response curve it can express in the original units. The power-transform chapter develops that connection.
“Scaling” and “transformation” overlap as general terms. Here, the useful distinction is between rescaling every column gap equally and changing relative gaps. The terminology note also separates those column operations from normalizing each row’s length.
Choose a chapter by the question you need answered
| Your question | Chapter |
|---|---|
| What does a long tail tell me, and does it matter for this model? | Skewness: what it measures and when it matters |
| Should equal ratios count as equal changes? How do I handle zeros? | Log transforms |
| What do a root, square or reciprocal do to order, gaps and sign? | Fixed power transforms |
| Can I learn a smooth shape change for a strictly positive feature? | Box–Cox |
| How can a fitted power family include negative values and zero? | Yeo–Johnson |
| How can I represent a value by its distribution position? | Quantile and rank transforms |
For a first pass, read skewness, log transforms and fixed powers, then compare the three fitted methods. Each fitted-method chapter includes a Python example, a focused C++ implementation and a figure you can use to check the numerical behavior.
Separate a chosen formula from learned quantities
A natural logarithm has no parameter to estimate. A Box–Cox transformer learns a parameter from training values. A quantile transformer learns a collection of landmarks. That distinction tells you what must be saved and reused on future inputs.
A fixed formula can still be a model-selection choice. If you compare log, square root and no transformation, that comparison belongs in validation. The final test set is not where you choose among them.
Scikit-learn’s PowerTransformer also standardizes its output by default. This is a second fitted step after the nonlinear map. Its examples in this series set standardize=False when demonstrating the raw family, so the displayed numbers match the equations.
Make the choice in four small steps
If the only problem is inconsistent units, revisit when to scale. If values are measurement errors, a nonlinear map will not correct their meaning. If the issue is a target transformation, also read why an inverse-transformed mean is not usually the original-scale mean.
Ask what the model can still distinguish
A strictly increasing map preserves order but can change distances. A square can erase sign when both positive and negative inputs are allowed. Quantile clipping can merge distinct future extremes. Those are different information tradeoffs, shown in the individual chapters.
Likewise, a transform fitted to training data does not guarantee a particular distribution on a future population. Keep the input units and fitted reference consistent. If the data distribution changes, evaluate the whole model and preprocessing together rather than silently fitting new coordinates beneath an existing model.
Continue to preprocessing pipelines to connect these choices with validation and saved artifacts. Return to this overview whenever the main question is which representation to try; use the method chapters for the formulas and worked examples.
References and further reading
- Scikit-learn: nonlinear transformations — power families, distribution mappings and changes in feature relationships.
- Scikit-learn: PowerTransformer — the fitted shape parameter and separate standardization step.
- Scikit-learn: QuantileTransformer — stored landmarks, interpolation and output bounds.