Turn Anomaly Scores into an Investigation Queue

September 2026 · 12 min read

Before you start: Isolation Forest scores and telemetry row and exposure definitions.

A detector scores individual feature rows; reviewers investigate cases. Turning row scores into a review queue requires three operations: an admission rule (cutoff vs. capacity cap), deduplication into cases before applying the review budget, and local explanations whose units match the reported score.

A score cutoff and a capacity limit answer different questions

Given $N$ eligible rows sorted by score $s_1 \ge s_2 \ge \dots \ge s_N$ (with ties broken by row ID ascending), compare four admission policies:

• Fixed cutoff ($\tau = 0.70$): admits $\{i : s_i \ge 0.70\}$. Queue volume fluctuates with daily traffic.
• Top-$K$ ($K = 3$): admits $\{1, \dots, \min(K, N)\}$. Fixes daily volume, even when all scores are low.
• Top-$p\%$ ($p = 25\%$): admits the top $\lceil 0.25 N \rceil$ rows ($2$ rows when $N=8$; $3$ rows when $N=12$).
• Capped cutoff ($\tau = 0.70,\; K = 3$): admits $\{i \le K : s_i \ge 0.70\}$, bounding workload while enforcing a minimum score floor.

If $\lambda$ cases arrive per day and reviewers resolve $\mu$ cases per day, admitting $\lambda > \mu$ grows the unresolved backlog by $\lambda - \mu$ cases/day (e.g., $8 - 3 = 5$ cases/day on the busy cohort). Select thresholds on resolved historical cohorts and audit unreviewed traffic separately (see Evaluate Graph Risk).

Group rows into cases before applying the budget

Partition the day's rows by an investigation key $g(i)$ (such as an app–tenant pair within a seven-day window) into candidate cases $G_k = \{i : g(i) = k\}$, and score each case by its maximum member score:

On the busy day, the top three rows (`B01 = 0.93`, `B02 = 0.90`, `B03 = 0.87`) all belong to case `K1`. Taking the top $3$ rows first and merging afterward yields only 1 case (`K1`). Grouping into cases first and then taking the top $K = 3$ cases admits `K1` ($0.93$), `K2` ($0.84$), and `K3` ($0.78$) while retaining all six underlying member rows as evidence.

def select_cases(rows, budget, minimum_score=None):
    ranked = sorted(rows, key=lambda row: (-row["score"], row["id"]))
    groups = {}
    for row in ranked:
        groups.setdefault(row["group"], []).append(row)
    cases = []
    for key, members in groups.items():
        representative = members[0]  # Highest score, then lowest ID.
        if minimum_score is None or representative["score"] >= minimum_score:
            cases.append({"group": key, "score": representative["score"],
                          "representative": representative["id"],
                          "members": [row["id"] for row in members]})
    return cases[:budget]

When evaluating queue quality, match the metric denominator to the unit reviewers handle: case-level precision counts confirmed positive cases divided by admitted cases, so three duplicate rows inside `K1` count as one investigation rather than three separate hits.

Additive explanations in path space vs. score space

For a model output $f(x)$ and a background reference dataset $X_{\text{ref}}$, an additive attribution (Lundberg & Lee, 2017) decomposes the prediction into the reference expectation plus feature contributions $\sum_{j=1}^M \phi_j(x) = f(x) - \mathbb{E}[f(X)]$. When $M = 1$ (a single feature), the entire difference belongs to $\phi_1(x)$:

In tree explainers such as SHAP's `IsoTree`, the tree leaf values are corrected path lengths $h_t(x)$ rather than normalized anomaly scores $s(x)$. Using $X_{\text{ref}} = \{1, 2, 3, 4, 20\}$ from chapter 1 gives two different additive decompositions:

• Path-space explanation ($f(x) = \bar{h}(x)$): $$\mu_{\text{path}} = \frac{1.6667 + 3.0000 + 4.0000 + 4.0000 + 1.3333}{5} = \mathbf{2.8000}, \qquad \phi_{\text{path}}(x) = \bar{h}(x) - 2.8000$$ Because $s(x) = 2^{-\bar{h}(x)/c(5)}$ decreases with path length, a negative $\phi_{\text{path}}(x)$ means a shorter path and therefore a higher anomaly score.
• Score-space explanation ($f(x) = s(x)$): $$\mu_{\text{score}} = \frac{0.6087 + 0.4092 + 0.3038 + 0.3038 + 0.6722}{5} = \mathbf{0.4595}, \qquad \phi_{\text{score}}(x) = s(x) - 0.4595$$

Because $g(z) = 2^{-z / c(5)}$ is nonlinear, you cannot exponentiate $\mu_{\text{path}}$ and $\phi_{\text{path}}(x)$ separately; you must sum the path components first and then apply the transform:

Keep one compact, reproducible case record

When admitting a case to the queue, persist the grouping key, member rows, admission policy, feature snapshot, and peer baseline version alongside an append-only outcome log:

{
  "case_id": "K3",
  "representative_row": "B06",
  "member_rows": ["B06"],
  "score": 0.78,
  "score_name": "higher_is_more_unusual",
  "admission_rank": 3,
  "queue_policy": "group-max-then-top3-v1",
  "grouping_window": "declared-seven-day-window",
  "model_version": "recorded-model-version",
  "feature_snapshot": "recorded-as-of-snapshot",
  "peer_reference": "recorded-peer-version",
  "evidence_ids": ["event-group-41"],
  "outcome": "pending"
}

References and further reading

• Lundberg & Lee: A Unified Approach to Interpreting Model Predictions — Additive feature attributions, the reference output, and local reconstruction.
• SHAP tree-explainer source: IsoTree — The Isolation Forest adapter’s corrected path-length representation.
• SHAP: TreeExplainer — Output-space and background semantics.
• scikit-learn: IsolationForest — Raw scores, offsets, and the library’s outlier labels.