Ordinal Encoding

September 2026 · 10 min read

Ordinal encoding represents ordered categories with ordered numbers: for example, low → 0, medium → 1, high → 2. It preserves a ranking supplied by the domain.

The subtlety is that order does not establish distance. Knowing that high is above medium does not tell us whether the change from medium to high is the same size as the change from low to medium. The model determines whether the numerical gaps matter.

Write down the order before assigning codes

Imagine a feature called risk band with the documented levels low, medium, and high. Its order is meaningful even if those words sort alphabetically as high, low, medium. Specify the domain order explicitly.

Do not infer order from frequency, alphabetical position, or the order labels happened to appear in a CSV. Those are reproducible sorting rules, but none establishes domain ranking. Unordered labels belong to the category-ID discussion and usually need a different treatment as numerical inputs.

Scikit-learn’s OrdinalEncoder can assign integer feature codes even when categories are unordered. The class name does not certify that your feature has a real order. Use its categories argument when order is part of the feature definition.

A linear model also uses the spacing

Suppose the encoded level is $c$. A linear predictor uses an intercept $a$ and coefficient $b$ to produce a numerical prediction $\hat y$:

With codes 0, 1, and 2, adjacent levels change the prediction by the same amount, $b$. That is an additional modeling assumption. For a logistic model, equal steps apply to the linear score, or log-odds, rather than to the probability itself.

A distance-based model also uses these gaps. A numerical threshold tree can express the same ordered partitions under either strictly increasing recoding: low versus the rest, or low-and-medium versus high. The label-encoding tree example shows the different problem caused by changing the category order itself.

An ordered input does not force a monotone prediction

Ordinal codes describe the input ranking. They do not say that the target must increase with the rank. An unconstrained tree can assign predictions 10, 30, and 20 to low, medium, and high. A linear coefficient can be positive or negative.

If the task requires predictions to increase with the level while other inputs stay fixed, use an appropriate monotonic model or constraint and check its behavior. Encoding the rank alone does not enforce that requirement, and a predictive monotonicity constraint is not a causal claim.

Keep known levels, unseen categories, and missing values distinct

A domain vocabulary may contain a valid level absent from a particular training sample. In the following example, medium is a known level even though the sample contains only low, high, and missing entries. Urgent is outside the declared vocabulary, while NaN means the value was not supplied.

import numpy as np
from sklearn.preprocessing import OrdinalEncoder

# The domain defines the order, including a valid level absent from this sample.
train = np.array([["high"], ["low"], [np.nan], ["high"]], dtype=object)
encoder = OrdinalEncoder(
    categories=[["low", "medium", "high", np.nan]],
    handle_unknown="use_encoded_value", unknown_value=-1,
    encoded_missing_value=-2,
).fit(train)

new = np.array([["medium"], ["urgent"], [np.nan]], dtype=object)
codes = encoder.transform(new)
assert codes.ravel().tolist() == [1., -1., -2.]
print(codes)

[[ 1.]
 [-1.]
 [-2.]]

The explicit category list puts NaN last, as required by this API. Standardize the source’s missing markers before encoding. The reserved codes −1 and −2 are representation markers; they do not mean “less than low.” A numeric model will nevertheless process them as numbers unless the pipeline handles that meaning.

Possible policies include retaining NaN for a compatible estimator, using separate missing/unknown indicators, or choosing a representation with explicit extra categories. Pick a policy with the downstream model in mind; reserving a code only solves the vocabulary lookup.

Preserve the definition across splits and serving

A domain-defined order can be fixed before the split without using target outcomes. If the vocabulary or rare-level grouping is learned from data, learn it within training folds. Do not rebuild separate mappings on validation or test data.

Store the ordered vocabulary and missing/unknown rules with the model. Adding a new level between medium and high can change the existing codes, so it requires a compatible model update. Do not silently shift a trained model’s input meanings.

Being able to encode a known but unobserved training level does not guarantee reliable predictions there. With no medium examples, a linear model interpolates according to its chosen spacing; a tree routes the code according to its fitted thresholds. Evaluate coverage of the levels the model will actually encounter.

C++: preserve the supplied order

This small map assigns codes in the explicitly supplied order, rejects duplicate labels, and distinguishes an absent value from an unknown string. It assumes a vocabulary small enough for integer codes. The downstream interpretation of the reserved negative codes remains a separate design choice.

#include <iostream>
#include <optional>
#include <stdexcept>
#include <string>
#include <unordered_map>
#include <vector>

class OrdinalMap {
    std::unordered_map<std::string, int> codes;
public:
    // Explicit domain order. Do not sort these labels alphabetically.
    explicit OrdinalMap(const std::vector<std::string>& orderedLevels) {
        if (orderedLevels.empty()) throw std::invalid_argument("empty order");
        for (std::size_t i = 0; i < orderedLevels.size(); ++i)
            if (!codes.emplace(orderedLevels[i], static_cast<int>(i)).second)
                throw std::invalid_argument("duplicate level");
    }
    int encode(const std::optional<std::string>& value) const {
        if (!value) return -2;  // Missing input, not a level below 'low'.
        const auto found = codes.find(*value);
        return found == codes.end() ? -1 : found->second;
    }
};

int main() {
    const OrdinalMap encoder({"low", "medium", "high"});
    std::cout << encoder.encode(std::string("medium")) << " "
              << encoder.encode(std::string("urgent")) << " "
              << encoder.encode(std::nullopt) << "\n";
}

1 -1 -2

References and further reading

• Scikit-learn: OrdinalEncoder — explicit category order, unknown codes, and encoded missing values.
• Scikit-learn: categorical feature encoding — integer and one-hot representations and their intended uses.
• Scikit-learn: monotonic constraints — separate constraints on model response to numerical features.