Label Encoding

September 2026 · 10 min read

Label encoding assigns each category a stable integer ID. The ID can identify a class, a vocabulary entry, or a lookup-table row. Whether that integer is a sensible model input depends on how the model uses it.

There are two questions to separate: how do we name categories with numbers? and what relationships does the model infer from those numbers? A label map answers the first. It does not automatically answer the second.

A label map is a reversible dictionary

Take the class labels [fox, cat, dog, fox]. A sorted vocabulary is [cat, dog, fox], giving IDs [0, 1, 2]. Replacing each label with its ID produces [2, 0, 1, 2]; decoding recovers the original labels.

A different fixed mapping could work equally well for class identifiers, provided training, prediction, probability columns, and decoding all agree. Sorting is one convention; a fixed domain vocabulary is another. Avoid rebuilding the mapping independently for each batch.

Why class IDs work as classification targets

Scikit-learn’s LabelEncoder is intended for the target vector y, not the input-feature matrix X. Many scikit-learn classifiers already accept string class labels. Explicit IDs are useful when an interface requires integer targets or when coordinating a fixed output vocabulary.

For a neural classifier trained with multiclass cross-entropy, the target ID selects the probability assigned to the correct class. If fox has ID 2, the loss reads the fox probability; it does not calculate a numerical distance between 2 and the predicted class ID.

For example, probabilities [0.10, 0.25, 0.65] in cat, dog, fox order give fox the largest probability. Position 2 identifies the fox column. Training a regressor to minimize squared error on the IDs would impose a different, usually unintended problem: errors between IDs would have numerical magnitudes.

import numpy as np
from sklearn.preprocessing import LabelEncoder

labels = np.array(["fox", "cat", "dog", "fox"])
encoder = LabelEncoder().fit(labels)
ids = encoder.transform(labels)
assert encoder.classes_.tolist() == ["cat", "dog", "fox"]
assert ids.tolist() == [2, 0, 1, 2]
assert encoder.inverse_transform(ids).tolist() == labels.tolist()

# A constructed probability row, with its class-column order explicitly given.
# In a fitted sklearn classifier, read this order from model.classes_.
model_class_ids = np.array([0, 1, 2])
probabilities = np.array([[0.10, 0.25, 0.65]])
predicted_ids = model_class_ids[probabilities.argmax(axis=1)]
print(encoder.inverse_transform(predicted_ids).tolist())

try:
    encoder.transform(["owl"])
except ValueError:
    print("unseen class: explicit handling required")

['fox']
unseen class: explicit handling required

The class-column order belongs to the fitted classifier. Read model.classes_ rather than assume an argmax column position always equals the encoded class ID. This matters when a model was trained on a subset of a larger vocabulary.

An integer input may become a numerical measurement

Now suppose color is an input feature, with red = 0, green = 1, and blue = 2. A distance-based model sees red and blue as twice as far apart as red and green. A linear model using one coefficient for this feature constrains its contribution to change in equal steps across those codes.

Those relationships come from the representation, not the category names. Standardizing the IDs changes the scale but preserves the arbitrary arrangement. For unordered features, compare one-hot encoding or a model with explicit categorical handling. For genuinely ordered categories, read ordinal encoding, including the separate question of spacing between levels.

Ordinary numeric trees are affected too

A numerical tree split such as color_id > 1.5 can only separate contiguous groups along the code order. Imagine the desired split is “blue versus the other colors.” Move blue’s ID in the figure:

A sufficiently deep tree can represent the blue-only case with either mapping. The point is that the required partitions and depth change, which can affect a constrained learned tree. “Trees are unaffected by arbitrary integer codes” is therefore too broad.

Some tree estimators support categorical features explicitly. In that case, follow their categorical interface and tell the estimator which columns are categorical. Passing an integer column to a numeric-threshold tree does not automatically activate that behavior. The histogram gradient-boosting documentation distinguishes these representations.

Freeze the vocabulary and define unknown-label behavior

LabelEncoder.transform() raises an error for an unseen target label. It has no handle_unknown option like feature encoders do. Silently mapping a new target class to an existing ID would corrupt its meaning, while refitting a separate encoder could change every ID.

A fixed class vocabulary can be specified from the task definition before training. Including a class name in that vocabulary does not give the model examples of that class. If an evaluation class never appeared in training, address the coverage issue explicitly; a new integer alone cannot teach a closed-set classifier to recognize it.

For input categories that may be unseen at serving time, use a feature encoder with the required policy. OrdinalEncoder can reserve an unknown code, and OneHotEncoder can use an unknown-category strategy. Their behavior and geometry are covered in the dedicated chapters. Unknown categories and genuinely missing values also need distinct meanings when the task requires them.

Store the map with the model

The vocabulary is part of the model artifact. Version it alongside the weights, preserve the class order in exported probability arrays, and decode predictions with that same mapping. A change from fox = 2 to fox = 0 is not a harmless data-cleaning update to a model whose output column 2 still means fox.

Integer IDs are also suitable indices into embedding tables when the downstream model uses a lookup. The ID selects a learned vector; its raw magnitude is not the similarity measure. This is another example of why the consumer of the code determines its meaning.

C++: a fixed map with explicit rejection

This implementation sorts and deduplicates the training labels once, then supports encoding and decoding. A lookup never inserts an unknown label. The constructor takes time dominated by sorting; subsequent hash-map encodings take expected constant time, with decoding by vector index.

#include <algorithm>
#include <iostream>
#include <stdexcept>
#include <string>
#include <unordered_map>
#include <utility>
#include <vector>

class LabelMap {
    std::vector<std::string> labels;
    std::unordered_map<std::string, std::size_t> ids;
public:
    explicit LabelMap(std::vector<std::string> trainingLabels)
        : labels(std::move(trainingLabels)) {
        if (labels.empty()) throw std::invalid_argument("empty label vocabulary");
        std::sort(labels.begin(), labels.end());
        labels.erase(std::unique(labels.begin(), labels.end()), labels.end());
        for (std::size_t i = 0; i < labels.size(); ++i) ids.emplace(labels[i], i);
    }
    std::size_t encode(const std::string& label) const {
        const auto found = ids.find(label);
        if (found == ids.end()) throw std::invalid_argument("unseen label");
        return found->second;
    }
    const std::string& decode(std::size_t id) const {
        return labels.at(id);
    }
};

int main() {
    const LabelMap map({"fox", "cat", "dog", "fox"});
    std::cout << map.encode("fox") << " " << map.encode("cat") << " "
              << map.encode("dog") << "\n";
    std::cout << map.decode(2) << "\n";
    try { map.encode("owl"); }
    catch (const std::invalid_argument&) { std::cout << "unseen label\n"; }
}

2 0 1
fox
unseen label

References and further reading

• Scikit-learn: LabelEncoder — target-label encoding, class vocabularies, and inverse transforms.
• Scikit-learn: OrdinalEncoder — integer feature codes and explicit unknown-value handling.
• Scikit-learn: categorical features in histogram gradient boosting — native categorical splits versus numeric and one-hot representations.