Rethinking Neural Thickets Without Weight Perturbations

Core Claim§

The Neural Thickets hypothesis claims pretrained LMs sit in dense neighborhoods of task experts in weight space, surfaced by Gaussian weight perturbations $\theta + \epsilon, \epsilon \sim \mathcal{N}(0, \sigma^2 I)$. This paper shows the reported gains are not a geometric property of weight space but an emergent effect of output diversity + validation selection + voting — reproducible with a fully frozen model.

Key Empirical Findings§

1. Single-block sufficiency. Perturbing just one decoder block recovers most of the improvement from full-model perturbation, undermining a "dense neighborhood" story that should require coordinated moves across all weights.
2. Logit bias matches weights. Adding a random bias $\delta$ directly to output logits (model frozen) matches or beats full-weight perturbations on most reasoning benchmarks.
3. First-order analysis. For small perturbations,

$$f(\theta+\epsilon) \approx f(\theta) + J_\theta\,\epsilon$$ so the effect on predictions flows entirely through the induced logit shift $J_\theta\epsilon$. The specific weight-space direction is immaterial; only the resulting output perturbation matters. This motivates perturbing logits directly.

Perturb-Select-Vote (PSV)§

A general procedure unifying Thickets with sampling, self-consistency, and verifier-guided decoding:

1. Perturb: generate $N$ stochastic predictors (weight noise or logit bias).
2. Select: keep the top-$k$ members by accuracy on a small held-out validation set.
3. Vote: aggregate predictions (majority or self-consistency) across selected members.

The Thicket "experts" are simply a committee {p_i}; quality comes from diversity of p_i plus selection, not from proximity in $\theta$.

Implementation Sketch§

import torch

def psy_logit_perturb(model, x, n_members=10, sigma=0.5):
    # Frozen model; perturb only output logits
    with torch.no_grad():
        base_logits = model(x)                       # [B, T, V]
        logits = base_logits + sigma * torch.randn(n_members, 1, 1, base_logits.size(-1),
                                                    device=x.device)
    return logits                                    # candidate logits per member

def select_and_vote(logits, val_acc):
    k = max(1, len(val_acc) // 2)
    keep = torch.topk(val_acc, k).indices
    preds = logits[keep].argmax(-1)                  # [k, B, T]
    return torch.mode(preds, dim=0).values           # majority vote

For weight perturbation, one instead samples $\epsilon$ per member and applies theta += sigma*eps, but the paper argues logit perturbation dominates.

Vision Extension§

The PSV framing transfers: logit PSV substantially improves accuracy under domain shift (e.g., Earth observation), while full-weight perturbations yield little extra benefit — confirming gains stem from output diversity plus selection, not weight geometry.

Practitioner Takeaways§

• Full-weight perturbation ensembles can be replaced by training-free logit noise on a frozen model.
• Keep a small validation set to select members; selection drives most of the gain.
• Thickets relate to self-consistency / verifier-guided selection — reframe ensembles accordingly.