Tracing the Heart: An Evidence-Linked Pipeline for Heart-Failure Feature Engineering

Overview§

This paper introduces the Nimblemind Multi-Agent System (nMAS), an evidence-linked, rubric-grounded pipeline for automated feature engineering from electronic health records (EHRs) in the context of heart failure. The system addresses the bottleneck of manual feature engineering (39-45% of data-scientist workload) by combining rule-based modularity with LLM-driven reasoning and evidence traceability. nMAS operates on 500 dummy patient records across nine EHR source tables, producing 132 structured features and 70 aggregated features that are scored against clinical rubrics. Adding these features improves held-out AUROC from 0.895 to 0.963 for HFrEF and from 0.870 to 0.910 for HFpEF phenotyping.

Methodology§

nMAS is a multi-agent system where specialized LLM agents handle distinct stages of feature engineering:

1. Extraction Agent: Parses heterogeneous EHR tables and extracts clinically meaningful entities (e.g., medications, lab values, vitals) with provenance links.
2. Feature Generation Agent: Proposes candidate features based on guideline-based clinical reasoning, mapping them to rubric criteria (e.g., evidence support, methodological soundness, reproducibility).
3. Aggregation Agent: Combines structured and rubric-scored features into aggregated representations, using temporal windows and summary statistics (mean, slope, variability).
4. Audit Agent: A restricted LLM verifies feature provenance, structural integrity, and rubric compliance, flagging hallucinations or untraceable values.

Each feature is stored with a provenance chain: source table, row identifiers, transformation steps, and the exact evidence snippet from clinical guidelines. The rubric scoring uses a linear combination:

$$ \text{RubricScore}(f) = \sum_{i=1}^{N} w_i \cdot \text{criteria}_i(f) $$

where $w_i$ are clinical-domain weights and $\text{criteria}_i$ are LLM-assessed binary or ordinal indicators (e.g., "is the feature derived from a current guideline?", "is the aggregation window clinically justified?"). The aggregated features are then fed into a downstream classifier (e.g., gradient-boosted trees) for HFrEF/HFpEF phenotyping.

Implementation Details§

A simplified PyTorch-style illustration of the aggregation logic (not the full system, but the core feature-construction pattern) is shown below:

import pandas as pd
import numpy as np

def aggregate_feature(patient_df, window_days=90):
    """Aggregate lab values over a time window with provenance."""
    # Sort by timestamp
    df = patient_df.sort_values('timestamp')
    # Restrict to clinical window
    df = df[df['timestamp'] >= df['timestamp'].max() - pd.Timedelta(days=window_days)]
    
    # Compute aggregated statistics
    agg = {
        'mean': df['value'].mean(),
        'slope': np.polyfit(np.arange(len(df)), df['value'], 1)[0],
        'variability': df['value'].std(),
        'max_relative_change': df['value'].pct_change().max(),
    }
    # Provenance: record source tables and row indices
    provenance = {
        'source_table': df['source'].unique().tolist(),
        'row_ids': df['row_id'].tolist(),
    }
    return agg, provenance

In the actual nMAS, the LLM agents generate the exact aggregation functions and rubric-scoring logic, rather than using a fixed Python function. The audit agent verifies that each aggregated feature has a valid provenance chain and that no synthetic or ungrounded values were introduced.

Results and Discussion§

The system produced 132 structured features (e.g., medication counts, ejection fraction categories) and 70 aggregated features (e.g., 90-day slope of NT-proBNP, variability in systolic blood pressure). An independent LLM rubric assessment scored the features at 81.5% of the maximum possible points, indicating strong evidence support and methodological soundness. The AUROC improvement was consistent across both heart-failure subtypes, demonstrating that automated, auditable feature engineering can replace manual pipeline construction.

Limitations include reliance on dummy records, single-institution evaluation, and the need for external validation on real EHR data. Future work should explore generalizability to other diseases and integration with existing clinical data warehouses.