Column Transformer

September 2026 · 12 min read

A numerical age and a city name need different preprocessing. Scaling the whole table cannot encode a city, and one-hot encoding every value would discard the numerical meaning of age. ColumnTransformer selects columns, applies a separate transform to each selection, and joins the resulting feature blocks.

Use a Pipeline when one operation must consume the previous operation’s output. Use ColumnTransformer when different column groups need different operations. A common model contains both: a ColumnTransformer with small pipelines inside its branches, followed by an estimator.

Define the reference table and the output features

We have three training rows. Age is numerical; city is categorical. The record identifier is present for tracing rows but is deliberately excluded from this model.

The numerical branch first fills missing ages with the observed training mean, 30. It then standardizes [20, 30, 40], learning center 30 and standard deviation √(200/3), approximately 8.165. The categorical branch learns the ordered one-hot vocabulary [Delhi, Pune].

We list the numerical branch first, so the output columns will be [standardized age, city is Delhi, city is Pune]. The output order follows the transformer list, then the columns emitted by each branch.

Parallel column branches, sequential steps inside a branch

Putting an imputer and scaler next to each other in the ColumnTransformer list does not chain them. If both select age, both start from the original age column and each emits its own output block. The scaler never sees the imputer’s result.

Here “parallel” describes independent branches. It does not require simultaneous execution on multiple CPU cores.

Build the numerical pipeline inside ColumnTransformer

A branch is a three-part tuple: (name, transformer, columns). The name identifies its fitted state and parameters. The column selection tells it which raw input to receive.

import numpy as np
import pandas as pd
from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import StandardScaler, OneHotEncoder

train = pd.DataFrame({
    "age": [20., np.nan, 40.],
    "city": ["Delhi", "Pune", "Delhi"],
    "record_id": ["u1", "u2", "u3"],
})
numeric = Pipeline([
    ("impute", SimpleImputer(strategy="mean")),
    ("scale", StandardScaler()),
])
prep = ColumnTransformer([
    ("num", numeric, ["age"]),
    ("cat", OneHotEncoder(handle_unknown="ignore", sparse_output=False), ["city"]),
], remainder="drop")
train_features = prep.fit_transform(train)
future = pd.DataFrame({
    "age": [50., np.nan],
    "city": ["Pune", "Mumbai"],
    "record_id": ["u4", "u5"],
})
print(prep.get_feature_names_out().tolist())
print(train_features.round(4).tolist())
print(prep.transform(future).round(4).tolist())
print(prep.output_indices_["num"], prep.output_indices_["cat"])

['num__age', 'cat__city_Delhi', 'cat__city_Pune']
[[-1.2247, 1.0, 0.0], [0.0, 0.0, 1.0], [1.2247, 1.0, 0.0]]
[[2.4495, 0.0, 1.0], [0.0, 0.0, 0.0]]
slice(0, 1, None) slice(1, 3, None)

The encoder returns dense output here to keep the tiny result easy to inspect. Its handle_unknown="ignore" policy preserves output width for Mumbai by emitting zeros. Those zeros do not identify which unseen city occurred. The one-hot chapter compares unknown and rare-category policies.

Select the shape each transformer expects

["age"] selects a two-dimensional table with one column, suitable for an imputer or scaler. A scalar selector such as "document" passes a one-dimensional sequence, suitable for a text vectorizer that expects one string per example. The square brackets are meaningful.

Names also avoid relying on accidental column positions. Keep the required input names, units, and types consistent. Selecting a string-typed age column does not by itself parse or validate the ages.

Inspect the output schema instead of guessing it

get_feature_names_out() names the emitted columns. The default branch prefixes distinguish num__age from the two cat__city_… columns. output_indices_ records that the numerical branch occupies slice 0:1 and the categorical branch slice 1:3.

Inspect fitted internals through prep.named_transformers_["num"] or prep.named_transformers_["cat"]. For example, the numerical pipeline’s named_steps["scale"] exposes the saved mean and standard deviation.

With remainder="drop", unspecified columns such as record_id are excluded. remainder="passthrough" instead appends remaining columns seen at fit time. That can accidentally include an identifier or even a target column left inside the input frame. Select deliberately; keep targets separate from feature inputs.

One sparse branch does not guarantee sparse output

With a wide categorical vocabulary, dense one-hot output can be expensive. Setting OneHotEncoder(sparse_output=True) makes that branch sparse, but ColumnTransformer still decides how to store the combined matrix.

If at least one branch is sparse, the combined density and sparse_threshold determine the output representation during fitting. The default threshold is 0.3. If every branch returns dense data, the result is dense regardless of that threshold. Inspect prep.sparse_output_ and the actual transformed object.

Wrap the assembled features and estimator in one pipeline

After importing Ridge from sklearn.linear_model, use the composition as part of the predictor, for example Pipeline([("prep", prep), ("model", Ridge())]), then fit that full pipeline on raw training features and targets. The model will receive the same ordered feature blocks during training and prediction.

Nested parameters follow the route through the objects. prep__num__impute__strategy reaches the numerical imputer; model__alpha reaches ridge. Pass the entire pipeline to cross-validation so each training fold learns its own replacement values, scales, vocabulary, and estimator.

C++: route selected fields and join saved transforms

This compact transform uses the fitted values from our example. It allocates one numerical coordinate followed by one coordinate per saved city. It deliberately does not read record_id or add unseen cities to the vocabulary. It assumes ages are finite or NaN and that the saved divisor is positive.

#include <cmath>
#include <iomanip>
#include <iostream>
#include <limits>
#include <string>
#include <vector>

struct Row { double age; std::string city, record_id; };
struct SavedColumns {
    double fill, center, scale;
    std::vector<std::string> cities;
    std::vector<double> transform(const Row& row) const {
        std::vector<double> out(1 + cities.size(), 0.0);
        const double age = std::isnan(row.age) ? fill : row.age;
        out[0] = (age - center) / scale;
        for (std::size_t j = 0; j < cities.size(); ++j)
            if (row.city == cities[j]) out[1 + j] = 1.0;
        return out; // Unknown city leaves the category block zero.
    }
};
int main() {
    // State already fitted on the three training rows in the article.
    const SavedColumns prep{30, 30, std::sqrt(200. / 3), {"Delhi", "Pune"}};
    const double missing = std::numeric_limits<double>::quiet_NaN();
    std::cout << std::fixed << std::setprecision(4);
    for (const Row& row : std::vector<Row>{{50,"Pune","u4"}, {missing,"Mumbai","u5"}}) {
        const auto values = prep.transform(row);
        std::cout << values[0] << " " << values[1] << " " << values[2] << "\n";
    }
}

2.4495 0.0000 1.0000
0.0000 0.0000 0.0000

The code illustrates routing and concatenation, not fitting. The preceding C++ pipeline example shows sequential fitting; the one-hot implementation shows vocabulary-based output construction.

References and further reading

• Scikit-learn: ColumnTransformer API — column selectors, output order, remainder behavior, feature names, and sparse stacking.
• Scikit-learn: heterogeneous data composition — column-specific branches and their combination with pipelines.
• Scikit-learn: mixed-type example — a numerical pipeline and categorical branch combined with an estimator.
• Scikit-learn: OneHotEncoder — learned categories, unknown-category policy, and output representation.