Weight of Evidence

September 2026 · 12 min read

Weight of evidence (WOE) encodes a category by comparing how often it appears among two outcome classes. It uses target labels, so it is a supervised encoding. The result is a log ratio, not an event probability.

Let the event be a repeat purchase: label 1 means yes and label 0 means no. Throughout this note, positive WOE means evidence toward label 1. Some sources reverse the ratio and its sign, so always state the label and ratio convention.

Compare shares within each class

Suppose a reference pool contains 20 events and 80 non-events. Category A occurs in 8 of the events and 8 of the non-events. Although the two counts are equal, their shares of the two class populations differ: 8/20 = 0.4 versus 8/80 = 0.1.

A is four times as prevalent among events as among non-events. Taking the natural logarithm gives WOE(A) = log(4), approximately 1.386. Equal raw counts did not mean equal evidence because the class totals were different.

For category $c$, write $n_{1c}$ and $n_{0c}$ for its event and non-event counts. Write $N_1$ and $N_0$ for the corresponding totals across the reference pool. With positive counts, the unsmoothed encoding is

A ratio above 1 produces positive WOE; equal class shares produce zero; a ratio below 1 produces negative WOE. The denominator inside each class share is that class’s total, not the number of rows in category $c$.

Evidence is relative to the starting event rate

In category A, 8 of 16 rows are events, giving an observed event rate of 50%. Across the whole pool, only 20% are events. A raises the event rate relative to that baseline. Switch the background pool below while keeping A’s rows fixed:

WOE = 0 means the category supplies no change in odds relative to the reference population. It does not mean an event probability of zero, or necessarily 50%.

Zero cells need a stated smoothing rule

A category with events but no non-events gives an infinite unsmoothed WOE. Two observed events and zero non-events are not enough evidence for an infinitely strong feature. Smoothing supplies small pseudo-counts so the ratio stays finite.

Here we add $\alpha=1$ to every category’s count within each class. With two categories, that adds 2 to each class total. A separate example makes the arithmetic visible:

Smoothing changes estimates; it does not create observations or remove label leakage. Compare rare-category behavior and the smoothing strength on the intended validation task.

Build each training feature without that row’s label

WOE uses labels just as target encoding does. Fitting a WOE map on all training rows and transforming those same rows lets each row’s label influence its own feature. Rare categories make this especially easy to exploit.

Use the cross-fitting procedure already explained in the target-encoding note: fit the map on other inner folds to encode a held-out inner fold. Its labels must be excluded from both the category counts and the class totals. Outer validation remains separate from all of this.

Group and time constraints apply to the reference pool too. If labels arrive later, a historical feature can use only labels available before that prediction. The ordinary WOEEncoder.fit_transform interface does not itself provide this cross-fitting boundary.

Python: fit one permitted reference map

This function implements the stated pseudo-count rule. It requires both classes and assumes non-missing category names. It supplies a map for other rows; the fold construction belongs around this operation. For an unseen name, the example deliberately returns zero: no added evidence relative to the baseline.

from collections import defaultdict
from math import isfinite, log

def fit_woe(reference_x, reference_y, alpha=1.0):
    if len(reference_x) != len(reference_y) or not len(reference_x):
        raise ValueError("use equally sized, nonempty reference inputs")
    if not isfinite(alpha) or alpha <= 0:
        raise ValueError("this implementation requires positive smoothing")
    counts = defaultdict(lambda: [0, 0])
    for name, label in zip(reference_x, reference_y):
        if label not in (0, 1):
            raise ValueError("labels must be 0 or 1")
        counts[name][label] += 1
    n0 = sum(pair[0] for pair in counts.values())
    n1 = sum(pair[1] for pair in counts.values())
    if not n0 or not n1:
        raise ValueError("both classes must occur in the reference pool")
    k = len(counts)
    return {name: log((n1c + alpha) / (n1 + k * alpha))
                  - log((n0c + alpha) / (n0 + k * alpha))
            for name, (n0c, n1c) in counts.items()}

# This permitted reference pool supplies a map for OTHER rows.
reference_x = ["A"] * 8 + ["B"] * 12 + ["A"] * 8 + ["B"] * 72
reference_y = [1] * 20 + [0] * 80
mapping = fit_woe(reference_x, reference_y)
new_x = ["A", "B", "unseen"]
encoded = [mapping.get(name, 0.0) for name in new_x]
print([round(value, 4) for value in encoded])

[1.3157, -0.4098, 0.0]

These values differ from the raw WOE because smoothing is enabled. Once the training rows have their cross-fitted features, fit a final map on the full permitted training pool for future inputs. Save the label convention, smoothing rule, mapping, and unknown policy with the downstream model.

A readable number still needs evidence

One WOE value per input feature compresses category effects onto a supervised numerical scale. A positive value describes association in the reference data; it is not a causal effect or a guarantee of future outcomes.

Numeric features are often binned before this encoding. If labels influence the bin boundaries or category grouping, that fitting step must follow the same exclusions as the WOE counts. Inspect support in both classes, stability across time, and held-out performance rather than choosing bins for extreme training separation.

For a regression target or several classes, this binary ratio is not directly the required encoding. Use a method designed for that target type instead of silently collapsing the outcome into two classes.

C++: encode a reference count table

The counts below describe the same original reference pool as the Python example. The constructor applies the same smoothing rule; lookups reuse the saved values and give unknown categories zero. It does not build cross-validation folds.

#include <cmath>
#include <cstddef>
#include <iomanip>
#include <iostream>
#include <stdexcept>
#include <string>
#include <unordered_map>
#include <vector>

struct CategoryCounts {
    std::string name;
    std::size_t events;
    std::size_t non_events;
};

class WoeMap {
    std::unordered_map<std::string, double> mapping;
public:
    explicit WoeMap(const std::vector<CategoryCounts>& counts, double alpha = 1) {
        if (counts.empty() || !std::isfinite(alpha) || alpha <= 0)
            throw std::invalid_argument("invalid pool or smoothing");
        double events = 0, non_events = 0;
        for (const auto& c : counts) {
            if (c.events == 0 && c.non_events == 0)
                throw std::invalid_argument("category has no observations");
            events += c.events;
            non_events += c.non_events;
        }
        if (events == 0 || non_events == 0)
            throw std::invalid_argument("both classes are required");
        const double added = alpha * counts.size();
        for (const auto& c : counts) {
            const double w = std::log((c.events + alpha) / (events + added))
                           - std::log((c.non_events + alpha) / (non_events + added));
            if (!mapping.emplace(c.name, w).second)
                throw std::invalid_argument("duplicate category");
        }
    }
    double encode(const std::string& name) const {
        const auto found = mapping.find(name);
        return found == mapping.end() ? 0.0 : found->second;
    }
};

int main() {
    const WoeMap encoder({{"A", 8, 8}, {"B", 12, 72}});
    std::cout << std::fixed << std::setprecision(4);
    for (const std::string name : {"A", "B", "unseen"})
        std::cout << name << ": " << encoder.encode(name) << "\n";
}

A: 1.3157
B: -0.4098
unseen: 0.0000

References and further reading

• Category Encoders: WOEEncoder — binary targets, configurable missing/unknown policies, and regularization.
• Category Encoders: WOE implementation — ratio orientation, class totals, regularization, and singleton handling.
• Scikit-learn: cross-fitting for supervised encodings — the label-leakage mechanism motivating separate reference folds.