Unsupervised Consensus-Based Anomaly Detection for Spatiotemporal Malaria Incidence in Ghana

Summary§

The paper introduces a consensus anomaly detection framework applied to monthly malaria incidence data from Ghana (2014-2023) to identify atypical transmission patterns. The approach combines multiple unsupervised anomaly detection algorithms (e.g., Isolation Forest, Local Outlier Factor, One-Class SVM) and aggregates their outputs via majority voting to produce a robust binary label (anomalous vs. normal) for each space-time point (district-month). The key innovation is the separation of anomaly burden (cumulative cases during anomalous months) from anomaly frequency (proportion of months flagged as anomalous), revealing that high-burden areas (e.g., Tamale) are not necessarily those with the most frequent anomalous transmission (e.g., Ashanti districts). The analysis shows that anomalous months form a statistically distinct group with large effect sizes (Cohen's $d = 3.252$ for case counts, $d > 1.2$ for seasonal deviations).

Core Methodology§

The framework consists of three steps:

1. Preprocessing: Monthly case counts per district are normalized and detrended to remove seasonal and long-term trends. For each district $i$ and month $t$, the anomaly score $z_{i,t}$ is computed as the deviation from the historical mean $\bar{x}_i$ (over same calendar month) divided by the standard deviation $\sigma_i$: $$z_{i,t} = \frac{x_{i,t} - \bar{x}_i}{\sigma_i}$$

2. Ensemble Anomaly Detection: Four base detectors (Isolation Forest, Local Outlier Factor, One-Class SVM, and a simple Z-score threshold) are trained on the normalized data. Each detector outputs a binary label $y_{i,t}^{(k)}$ for $k=1,\dots,K$.

3. Consensus Voting: The final label $y_{i,t}$ is determined by majority voting: $$y_{i,t} = \begin{cases} 1 & \text{if } \frac{1}{K}\sum_{k=1}^K y_{i,t}^{(k)} > 0.5 \\ 0 & \text{otherwise} \end{cases}$$

Implementation Details§

The data set comprises monthly malaria cases for 216 districts from January 2014 to December 2023 (120 months). Missing values are imputed using linear interpolation. The ensemble is trained on the full data set (unsupervised), and anomaly scores are standardized across detectors to ensure fair voting. Hyperparameters for each detector (e.g., contamination rate for Isolation Forest) are set to default values from scikit-learn. Statistical tests (Cohen's $d$) compare anomalous vs. normal months.

Code Snippet§

import numpy as np
from sklearn.ensemble import IsolationForest
from sklearn.neighbors import LocalOutlierFactor
from sklearn.svm import OneClassSVM
from scipy.stats import zscore

def consensus_anomaly_detection(X, detectors=None, threshold=0.5):
    """
    X: numpy array of shape (n_samples, n_features) after normalization
    Returns: binary array of shape (n_samples,) where 1 = anomaly
    """
    if detectors is None:
        detectors = [
            IsolationForest(contamination=0.1, random_state=42),
            LocalOutlierFactor(contamination=0.1, novelty=True),
            OneClassSVM(nu=0.1),
            ZScoreAnomalyDetector(threshold=2.0)
        ]
    
    votes = np.zeros((X.shape[0], len(detectors)), dtype=int)
    for i, det in enumerate(detectors):
        if hasattr(det, 'fit_predict'):
            labels = det.fit_predict(X)
            votes[:, i] = (labels == -1).astype(int)
        else:
            det.fit(X)
            pred = det.predict(X)
            votes[:, i] = (pred == -1).astype(int)
    
    consensus = (np.mean(votes, axis=1) > threshold).astype(int)
    return consensus

class ZScoreAnomalyDetector:
    def __init__(self, threshold=2.0):
        self.threshold = threshold
    def fit(self, X):
        self.mean_ = np.mean(X, axis=0)
        self.std_ = np.std(X, axis=0)
    def predict(self, X):
        z = np.abs((X - self.mean_) / (self.std_ + 1e-8))
        return np.where(np.any(z > self.threshold, axis=1), -1, 1)

Evaluation Metrics§

• Anomaly Burden: Total cases during anomalous months per district.
• Anomaly Frequency: Percentage of months flagged anomalous per district.
• Effect Size: Cohen's $d$ for case counts and seasonal deviation (average absolute Z-score normalized by calendar month).

Conclusion§

The consensus framework provides a robust, unsupervised method to identify spatially structured anomalies. The separation of burden and frequency offers actionable insights for public health surveillance, enabling targeted interventions in both high-burden and high-frequency anomaly areas.