Handling Imbalanced Datasets

⚠ The Imbalance Problem

Class imbalance occurs when the distribution of target labels is skewed — one class vastly outnumbers another. This is not an edge case: it is the default condition in almost every real-world classification problem of practical interest. Fraud, disease, intrusion, defect — the things we most want to detect are always rare.

🔄 Resampling Techniques

Resampling modifies the training data distribution to balance the classes. The goal is to give the model equal opportunity to learn patterns from both minority and majority classes during training — while evaluating on the true (imbalanced) distribution.

How SMOTE Works

SMOTE (Synthetic Minority Over-sampling TEchnique) generates new minority-class samples by:

• For each minority sample, find its k nearest neighbours (default k=5) within the minority class
• Randomly select one neighbour
• Create a new sample on the line segment between the original and the selected neighbour: new = original + λ × (neighbour − original) where λ ∈ [0,1]
• Repeat until the desired ratio is reached
• Result: more diverse minority examples that fill in the feature space, rather than just duplicates

import pandas as pd
import numpy as np
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split, StratifiedKFold
from imblearn.over_sampling import SMOTE, ADASYN, RandomOverSampler
from imblearn.under_sampling import RandomUnderSampler, TomekLinks
from imblearn.combine import SMOTETomek
from imblearn.pipeline import Pipeline as ImbPipeline
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import classification_report

# Create severely imbalanced dataset (1:50 ratio)
X, y = make_classification(
    n_samples=5000, n_features=20, n_informative=10,
    weights=[0.98, 0.02], random_state=42
)
print(f"Class distribution: {np.bincount(y)}")  # ~4900 vs ~100

# CRITICAL: Split BEFORE resampling (never resample before the split!)
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y  # stratify preserves ratio
)

# ---- SMOTE ----
smote = SMOTE(k_neighbors=5, random_state=42, sampling_strategy="minority")
X_train_sm, y_train_sm = smote.fit_resample(X_train, y_train)
print(f"After SMOTE: {np.bincount(y_train_sm)}")  # balanced

# ---- ADASYN (adaptive) ----
adasyn = ADASYN(n_neighbors=5, random_state=42)
X_train_ad, y_train_ad = adasyn.fit_resample(X_train, y_train)

# ---- SMOTETomek (oversample + clean boundary) ----
smote_tomek = SMOTETomek(random_state=42)
X_train_st, y_train_st = smote_tomek.fit_resample(X_train, y_train)

# ---- Best practice: use imbalanced-learn Pipeline ----
# Resampling ONLY happens during fit, NOT during transform/predict
pipeline = ImbPipeline([
    ("oversample", SMOTE(random_state=42)),
    ("classifier", RandomForestClassifier(n_estimators=100, random_state=42))
])

# Cross-validation with stratified folds
cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
pipeline.fit(X_train, y_train)
y_pred = pipeline.predict(X_test)

print(classification_report(y_test, y_pred, target_names=["majority", "minority"]))

🧠 Algorithm-Level Approaches

Rather than modifying the data, algorithm-level approaches modify how the model trains or predicts to handle imbalance. These are often simpler to implement and less risky than resampling.

import numpy as np
from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import RandomForestClassifier, BaggingClassifier
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from sklearn.metrics import (classification_report, roc_curve,
                              precision_recall_curve, average_precision_score)
import matplotlib
matplotlib.use("Agg")  # non-interactive backend

X, y = make_classification(weights=[0.95, 0.05], n_samples=10000, random_state=42)
X_train, X_test, y_train, y_test = train_test_split(X, y, stratify=y, test_size=0.2)

# ---- class_weight='balanced' ----
lr_balanced = LogisticRegression(class_weight="balanced", max_iter=1000)
lr_balanced.fit(X_train, y_train)
print("Balanced LR:\n", classification_report(y_test, lr_balanced.predict(X_test)))

# ---- Custom class weights (explicit ratio) ----
lr_custom = LogisticRegression(class_weight={0: 1, 1: 20}, max_iter=1000)
lr_custom.fit(X_train, y_train)

# ---- Threshold tuning via ROC curve ----
rf = RandomForestClassifier(n_estimators=100, class_weight="balanced", random_state=42)
rf.fit(X_train, y_train)
y_proba = rf.predict_proba(X_test)[:, 1]

# Find threshold that maximises F1 for minority class
from sklearn.metrics import f1_score
thresholds = np.arange(0.1, 0.9, 0.05)
best_thresh, best_f1 = 0.5, 0
for thresh in thresholds:
    y_pred_t = (y_proba >= thresh).astype(int)
    f1 = f1_score(y_test, y_pred_t, pos_label=1)
    if f1 > best_f1:
        best_f1, best_thresh = f1, thresh

print(f"Best threshold: {best_thresh:.2f}, F1 minority: {best_f1:.3f}")
y_pred_tuned = (y_proba >= best_thresh).astype(int)
print(classification_report(y_test, y_pred_tuned))

# ---- BalancedBaggingClassifier (ensemble for imbalance) ----
from imblearn.ensemble import BalancedBaggingClassifier, EasyEnsembleClassifier
bbc = BalancedBaggingClassifier(
    estimator=RandomForestClassifier(n_estimators=10),
    n_estimators=10, random_state=42
)
bbc.fit(X_train, y_train)
print("BalancedBagging:\n", classification_report(y_test, bbc.predict(X_test)))

📏 Evaluation Metrics for Imbalanced Data

Choosing the right metric is as important as choosing the right model. Using accuracy on imbalanced data doesn't just give a misleading number — it actively hides the problem from you.

✅ Practical Checklist