Rare-Category Grouping

September 2026 · 10 min read

Rare-category grouping keeps well-supported category names and combines selected low-frequency names into one pooled category, often called “other.” An encoder can then assign that group one representation.

This reduces the number of separate effects the model must estimate. It also gives up distinctions between the grouped names. Rarity describes the evidence available in the reference pool, not whether a category is unimportant.

Choose which names retain their own representation

Our training pool contains twenty orders: A appears 10 times, B 5 times, C 3 times, D once, and E once. Set a minimum count of 3. A, B, and C remain separate; D and E become one pooled group.

The rule is “keep counts at least the minimum.” A category exactly at the threshold stays. Move the threshold to see both the membership and the number of output groups change:

Pooling shares an effect; it does not prove similarity

After grouping, a one-hot linear model gives D and E the same category contribution. Two observations support that shared effect instead of one observation supporting each separate effect. This can reduce unstable estimates, at the cost of hiding differences between D and E.

If D and E have systematically different outcomes, grouping can hurt. More rows in “other” do not guarantee a coherent population, and a rare category can carry a strong signal. Compare the grouping with an ungrouped baseline and examine the affected rows.

Grouping also differs from frequency encoding. Here frequent A, B, and C retain distinct names. Frequency encoding would replace every name with its prevalence, potentially merging any categories that happen to have equal counts.

Save membership from training data

Fit the grouping rule using the training pool. Apply that saved membership to validation and future inputs. D stays in “other” even if it appears repeatedly in a later prediction batch.

Re-counting each validation or serving batch would change the feature definition. To revise membership after data accumulates, fit a new grouping-and-model version. Keep the entire fitting process inside each training fold when comparing thresholds.

Rare, unseen, and missing are different cases

A rare category has some training support. An unseen category has none. A missing value means no category value was supplied. You may map some of them to the same group, but that is a modeling decision rather than an equivalence between the cases.

Scikit-learn’s handle_unknown="infrequent_if_exist" sends an unseen name to the infrequent group if one was created during fitting. If every training name met the minimum, no such group exists and an unseen name produces an all-zero feature block. The option does not manufacture a trained fallback group.

If missingness needs a distinct meaning, preserve a missingness indicator or explicitly keep a missing category. A missing marker that occurs rarely can otherwise be pooled with ordinary rare names. Custom reserved group names must also be distinguishable from legitimate input values.

Python: grouping and one-hot encoding together

OneHotEncoder can learn the grouping and output columns in one fitted step. The tiny example uses dense output for readability; keep sparse output for wide real datasets.

import numpy as np
from sklearn.preprocessing import OneHotEncoder

train = np.array([["A"]] * 10 + [["B"]] * 5 + [["C"]] * 3
                 + [["D"], ["E"]])
encoder = OneHotEncoder(
    min_frequency=3,
    handle_unknown="infrequent_if_exist",
    sparse_output=False,
).fit(train)

print(encoder.get_feature_names_out(["shop"]).tolist())
new = np.array([["C"], ["D"], ["Z"]])
encoded = encoder.transform(new)
print(encoded.astype(int).tolist())
assert encoded.tolist() == [[0, 0, 1, 0], [0, 0, 0, 1], [0, 0, 0, 1]]

['shop_A', 'shop_B', 'shop_C', 'shop_infrequent_sklearn']
[[0, 0, 1, 0], [0, 0, 0, 1], [0, 0, 0, 1]]

C activates its own column. D activates the infrequent column because it was rare in training. Z activates that same column under the chosen unknown policy. From this encoded block alone, the model cannot distinguish D from Z. The one-hot note explains how dropping a reference column can introduce another collision.

Minimum count and maximum width answer different questions

An integer min_frequency sets a row-count threshold. A fraction sets a threshold relative to the number of fitting rows: with twenty rows, min_frequency=0.2 uses a threshold of four. A fixed count controls minimum support; a fraction follows the reference pool’s size.

max_categories caps the number of output categories including the pooled group. In our example, min_frequency=3, max_categories=3 keeps A and B and pools C, D, and E. C met the minimum count but the width cap still limits the retained names.

Measure the tradeoff where grouping has an effect

Compare overall performance and performance on retained, pooled, and truly unseen categories. Use a split that reflects whether deployment will face new entities or future data. If outcomes or domain rules determine which categories get merged, validate that fitted grouping step too.

The pooled population may change over time even when its name remains “other.” Track how many inputs use it and which original names dominate it. Save the full training vocabulary separately when you need to distinguish previously rare names from new ones in those reports.

Grouping reduces output width. It does not guarantee a vocabulary-free encoder: scikit-learn retains information about the original fitted categories. Compare hashing if vocabulary memory is the main constraint, or supervised encoding when labels can support the effects and its stricter fitting boundaries are appropriate.

C++: a fitted count-threshold grouper

This implementation retains names meeting the minimum and records whether a pool exists. It sends other names to that pool when available. Otherwise it returns an empty optional, leaving the caller to apply an explicit unseen-category policy. It implements the count threshold, not the separate maximum-width cap.

#include <cstddef>
#include <iostream>
#include <optional>
#include <stdexcept>
#include <string>
#include <unordered_map>
#include <unordered_set>
#include <vector>

class RareGrouper {
    std::unordered_set<std::string> kept;
    bool has_pool = false;
public:
    RareGrouper(const std::vector<std::string>& reference, std::size_t minimum) {
        if (reference.empty() || minimum == 0)
            throw std::invalid_argument("nonempty pool and positive minimum required");
        std::unordered_map<std::string, std::size_t> counts;
        for (const auto& name : reference) {
            if (name == "<OTHER>")
                throw std::invalid_argument("reserved group name in input");
            ++counts[name];
        }
        for (const auto& item : counts) {
            if (item.second >= minimum) kept.insert(item.first);
            else has_pool = true;
        }
    }
    std::optional<std::string> group(const std::string& name) const {
        if (kept.count(name)) return name;
        if (has_pool) return std::string("<OTHER>");
        return std::nullopt;  // Unseen, and no pooled group was learned.
    }
};

int main() {
    std::vector<std::string> reference;
    reference.insert(reference.end(), 10, "A");
    reference.insert(reference.end(), 5, "B");
    reference.insert(reference.end(), 3, "C");
    reference.push_back("D");
    reference.push_back("E");
    const RareGrouper grouper(reference, 3);
    for (const std::string name : {"C", "D", "Z"})
        std::cout << name << ": " << grouper.group(name).value_or("<UNSEEN>") << "\n";
}

C: C
D: <OTHER>
Z: <OTHER>

References and further reading

• Scikit-learn: OneHotEncoder — minimum frequency, output-category caps, fitted infrequent categories, and unseen-input behavior.
• Scikit-learn: categorical encoding guide — grouping infrequent categories and using the resulting indicator columns.
• Scikit-learn: data leakage — learning preprocessing choices from the permitted training data.