ConceptSMILE: Auditing the Trustworthiness of Concept-Based Explainable AI

Overview§

ConceptSMILE audits the trustworthiness of concept-based explanations by systematically perturbing input regions and measuring shifts in concept responses. It fits a local XGBoost surrogate with locality weighting to approximate concept behavior, then quantifies reliability via five metrics: attribution accuracy, surrogate fidelity, faithfulness, stability, and consistency.

Methodology§

Perturbation and Concept Response§

Given an input image $\mathbf{x}$ and a concept encoder $f$ that produces concept scores $\mathbf{c} = f(\mathbf{x}) \in \mathbb{R}^K$, ConceptSMILE generates a set of perturbed versions $\mathbf{x}^{(j)}$ by masking or adding noise to localized regions. For each perturbation $j$, the concept response shift is $\Delta \mathbf{c}^{(j)} = f(\mathbf{x}^{(j)}) - f(\mathbf{x})$.

Locality Weighting§

To focus on local behavior, each perturbation is assigned a weight $w_j$ based on the distance between the perturbed region and the region of interest (e.g., using a Gaussian kernel): $$w_j = \exp\left(-\frac{d_j^2}{2\sigma^2}\right)$$ where $d_j$ is the spatial distance and $\sigma$ controls locality.

Surrogate Modeling§

An XGBoost regressor $g$ is trained to predict concept response shifts $\Delta \mathbf{c}^{(j)}$ from perturbation features $\mathbf{z}^{(j)}$ (e.g., binary mask of perturbed pixels). The loss is weighted by $w_j$: $$\mathcal{L} = \sum_j w_j \cdot \text{MSE}\left(\Delta \mathbf{c}^{(j)}, g(\mathbf{z}^{(j)})\right)$$ Surrogate fidelity is measured using weighted $R^2$: $$R_w^2 = 1 - \frac{\sum_j w_j (\Delta c^{(j)} - \hat{\Delta c}^{(j)})^2}{\sum_j w_j (\Delta c^{(j)} - \bar{\Delta c}_w)^2}$$ where $\bar{\Delta c}_w = \frac{\sum_j w_j \Delta c^{(j)}}{\sum_j w_j}$.

Reliability Metrics§

• Attribution Accuracy: Correlation between surrogate feature importances and ground-truth concept attribution masks.
• Surrogate Fidelity: $R^2_w$ from the surrogate.
• Faithfulness: How well the surrogate's predictions align with actual concept changes under strong perturbations (e.g., occlusion).
• Stability: Variance of concept scores under small random perturbations.
• Consistency: Agreement of explanations across different perturbation types.

Code Snippet§

import numpy as np
import xgboost as xgb

def concept_smile_audit(image, concept_encoder, perturbation_type='mask', n_perturb=100, sigma=5.0):
    # Generate perturbations (simplified: random masks)
    perturbations = np.random.binomial(1, 0.1, (n_perturb, *image.shape))  # binary masks
    orig_concept = concept_encoder(image[np.newaxis, ...])  # shape (1, K)
    concept_shifts = []
    weights = []
    for mask in perturbations:
        perturbed = image * mask  # masked version
        new_concept = concept_encoder(perturbed[np.newaxis, ...])
        shift = (new_concept - orig_concept).flatten()
        concept_shifts.append(shift)
        # locality weight: assume center of mask is known
        d = np.linalg.norm(np.array([image.shape[0]//2, image.shape[1]//2]) - np.array([0, 0]))  # placeholder
        w = np.exp(-d**2/(2*sigma**2))
        weights.append(w)
    
    # Feature matrix (flattened mask)
    X = perturbations.reshape(n_perturb, -1)
    # Target for first concept (index 0)
    y = np.array([s[0] for s in concept_shifts])
    w = np.array(weights)
    
    # Train weighted XGBoost
    dtrain = xgb.DMatrix(X, label=y, weight=w)
    params = {'objective':'reg:squarederror', 'eval_metric':'rmse'}
    model = xgb.train(params, dtrain, num_boost_round=50)
    
    # Evaluate fidelity
    preds = model.predict(dtrain)
    ss_res = np.sum(w * (y - preds)**2)
    ss_tot = np.sum(w * (y - np.average(y, weights=w))**2)
    r2_weighted = 1 - ss_res / ss_tot
    return r2_weighted, model.feature_importances_

Evaluation§

ConceptSMILE was evaluated on retinal fundus images comparing MedSAM-derived visual concepts (based on spatial segmentation) with VLM-based semantic concepts (language descriptions). Results showed MedSAM achieved higher surrogate fidelity ($R^2 = 0.8503$, $R_w^2 = 0.8465$) while VLM had stronger vessel faithfulness and stability under certain artifacts.

Key Findings§

• Concept reliability is not uniform across concepts or explanation methods.
• Framework provides an independent audit layer without requiring ground-truth concept labels.
• Perturbation-based approach is model-agnostic and can be applied to any concept-based XAI system.