From Scratch · C++ · Machine Learning · Data Preparation and Preprocessing

Encoding Categorical Features in C++

June 2026 · 11 min read

The decision tree splits on numeric thresholds: it asks x[f] < t. A column of strings like {"red", "green", "blue"} has no such ordering, so it has to be turned into numbers first. The two standard ways are label encoding and one-hot encoding, and the difference between them is whether you accidentally invent an order that is not there.

Label Encoding

Label encoding maps each distinct category to an integer id. It is compact, one column stays one column, and it is the natural fit for the target labels a classifier predicts. Fitting it once records the mapping so the same ids are reused everywhere:

#include <bits/stdc++.h>
using namespace std;

struct LabelEncoder {
    map<string,int> index;            // category -> integer id (sorted order)

    void fit(const vector<string>& col) {
        set<string> uniq(col.begin(), col.end());
        int id = 0;
        for (auto& c : uniq) index[c] = id++;
    }
    int transform(const string& c) const { return index.at(c); }
    int size() const { return index.size(); }
};

The hidden trap. Label encoding a feature as red=2, green=1, blue=0 tells a distance- or gradient-based model that green sits halfway between blue and red, an order that does not exist. Tree models are immune (a threshold on the ids just partitions the categories into two groups), but for most other models an unordered feature should be one-hot encoded instead.

One-Hot Encoding

One-hot encoding replaces a $K$-category feature with $K$ binary columns, exactly one of which is $1$. No category is numerically larger than another, so no false order is implied. The cost is width: $K$ categories become $K$ columns.

int main() {
    vector<string> col = {"red", "green", "blue", "green", "red"};
    LabelEncoder enc;
    enc.fit(col);

    printf("categories:");
    for (auto& [c, id] : enc.index) printf("  %s=%d", c.c_str(), id);
    printf("\n");

    printf("label:   ");
    for (auto& c : col) printf(" %s->%d", c.c_str(), enc.transform(c));
    printf("\n");

    printf("one-hot:\n");
    int K = enc.size();
    for (auto& c : col) {
        vector<int> oneHot(K, 0);
        oneHot[enc.transform(c)] = 1;     // flip on this category's column
        printf("  %-6s ->", c.c_str());
        for (int v : oneHot) printf(" %d", v);
        printf("\n");
    }
}

categories: blue=0 green=1 red=2 label: red->2 green->1 blue->0 green->1 red->2 one-hot: red -> 0 0 1 green -> 0 1 0 blue -> 1 0 0 green -> 0 1 0 red -> 0 0 1

The label row collapses each category to a single id; the one-hot block gives each its own column, so red, green, and blue are equidistant. Because the encoder is fit once and stored, a category seen at training time always maps to the same column at test time, and an unseen category can be detected (here index.at would throw) rather than silently mis-encoded.

Where this feeds in. Run this before the decision tree or any model whenever a feature is text: trees are happy with the compact label encoding, while distance- and gradient-based models want the one-hot form. Numeric features that simply have gaps, on the other hand, are handled by imputation.