Data Imbalance in Deep Learning: Reading Guide
Eleven connected chapters on class imbalance and long-tailed learning: from the 99%-accuracy trap and imbalance evaluation to random oversampling, random undersampling, SMOTE, ADASYN, class weights, Effective-Number and Focal Loss, decoupled two-stage training, threshold moving, and prior-shift recalibration.
Real-world datasets rarely have equal counts per class: fraud transactions, medical lesions, and rare traffic hazards often make up less than $1\%$ of training examples. When standard cross-entropy is minimized on a skewed distribution, head classes dominate the mini-batch gradient, push tail decision boundaries inward, and inflate overall accuracy while minority recall collapses.
Eleven chapters, one progression
Understand class imbalance through a $99\%$-accurate baseline, clickable confusion counts, explicit error costs, and the separate roles of sampling, weights, and thresholds.
Separate operating decisions, ranking, and calibration; explore prevalence interactively, compare ROC with PR curves, and compute tie-aware average precision.
Follow source identities through sampling with replacement, see which rows are repeated, and understand why a larger resampled dataset does not add independent observations.
Compare balanced subsets that retain different boundary evidence and calculate the probability of dropping an informative majority region.
Choose minority parents, interpolate a synthetic point between them, and inspect domain constraints that valid endpoints satisfy while their midpoint violates.
Use all-class neighbors to allocate a synthesis budget toward harder boundary seeds, then interpolate along minority-only neighbors.
Calculate balanced class weights and vary their influence interactively.
Derive Cui et al.’s effective number of samples $E_n=(1-\beta^n)/(1-\beta)$ from random subset coverage, and derive Focal Loss $-\alpha_t(1-p_t)^\gamma\log p_t$ and its exact logit gradient.
Separate backbone learning from classifier calibration, inspect why head classifier weights have larger $\ell_2$ norms $\|w_k\|_2$, and compare cRT, LWS, and $\tau$-normalization.
Tune a decision cutoff on fixed validation scores, inspect alert and error costs interactively, and derive the Bayes probability threshold $p^*=C_{\text{FP}}/(C_{\text{FP}}+C_{\text{FN}})$.
Prove why oversampling, undersampling, and class weights warp predicted probabilities, and derive the exact Elkan/Saerens Bayes prior-shift recalibration formula.
Where each intervention acts
| Stage | What it changes | Representative methods | Chapters |
|---|---|---|---|
| Problem & Evaluation | Replaces accuracy and FPR with minority-sensitive denominators and PR ranking | Confusion counts, Precision, Recall, $F_1$, PR-AUC, Average Precision | Ch. 01–02 |
| Resampling | Changes the class counts in the training partition via repetition or subset selection | Random Oversampling, Random Undersampling | Ch. 03–04 |
| $k$-NN Synthesis | Interpolates synthetic minority vectors along same-class neighbor segments | SMOTE, ADASYN | Ch. 05–06 |
| Loss Weighting | Rescales each example’s gradient by class rarity or prediction hardness | Class Weights, Effective Number $(1-\beta^n)/(1-\beta)$, Focal Loss | Ch. 07–08 |
| Two-Stage Head | Trains backbone on natural data ($q=1$), then recalibrates linear head | cRT, $\tau$-Normalization, Learnable Weight Scaling (LWS) | Ch. 09 |
| Post-Hoc Decision | Moves the operating score threshold or corrects prior odds at inference time | Validation Threshold Moving, Cost Cutoff $p^*$, Prior-Shift Correction | Ch. 10–11 |
Notation that stays consistent
Across the series, a dataset of $N=\sum_{k=1}^K n_k$ examples has class counts $n_1\ge n_2\ge\cdots\ge n_K$ and empirical class priors $\pi_k=n_k/N$. The Imbalance Factor is $\text{IF}=n_1/n_K$. A backbone $f_\theta(x)\in\mathbb R^d$ feeds a linear classifier with weights $w_k\in\mathbb R^d$ and biases $b_k\in\mathbb R$ to produce logits $z_k=w_k^\top f_\theta(x)+b_k$ and softmax probabilities $p_k=e^{z_k}/\sum_{j=1}^K e^{z_j}$.