VAD: Attributing Visual Evidence for Target Reconstruction in Multimodal On-Policy Distillation

Overview§

Multimodal on-policy distillation (OPD) supervises a student's autoregressive generation using a privileged-view teacher that observes additional visual evidence. The teacher's next-token corrections are source-mixed: they conflate genuine visual evidence with linguistic priors and teacher-specific biases. VAD addresses this by decomposing the teacher correction into a visually attributable component and a residual.

Method§

At each prefix $x_{<t}$, VAD evaluates the fixed teacher twice: once with visual evidence $E$ and once with it removed (e.g., masked image). Let $\log p_t^E$ and $\log p_t^{E^-}$ be the teacher's log-probabilities. The visual evidence direction is defined as the centered difference:

$$u_t = \log p_t^E - \log p_t^{E^-} - \frac{1}{V}\sum_{y}(\log p_t^E(y) - \log p_t^{E^-}(y))$$

This vector is a signed proxy: positive values indicate tokens supported by the evidence, negative values indicate tokens refuted.

Let $r_t$ be the teacher's original correction logits. VAD projects $r_t$ onto $u_t$:

$$\alpha_t = \frac{r_t \cdot u_t}{\|u_t\|^2}, \quad a_t = \alpha_t u_t$$

The aligned component $a_t$ captures only the part of the teacher's correction that is visually grounded. The residual $r_t - a_t$ is discarded.

Target Reconstruction & Training§

The student's own logits $s_t$ serve as the anchor. The reconstructed target distribution is:

$$\tilde{p}_t = \text{softmax}\left(s_t + \tau \, a_t\right)$$

where $\tau$ is a temperature. This target is used as the primary supervision signal (e.g., cross-entropy). The original teacher logits $r_t$ only contribute a weak KL regularizer:

$$\mathcal{L} = \mathrm{CE}(\tilde{p}_t, q_t) + \lambda \, \mathrm{KL}(p_t^E, q_t)$$

Implementation Sketch§

# teacher forward with and without visual evidence
logp_E = teacher(prefix, evidence).log_softmax(-1)
logp_Em = teacher(prefix, removed_evidence).log_softmax(-1)
u = logp_E - logp_Em
u -= u.mean(dim=-1, keepdim=True)  # center

# teacher correction logits (original)
r = teacher(prefix, evidence)

# project between correction and visual evidence direction
alpha = (r * u).sum(-1, keepdim=True) / (u * u).sum(-1, keepdim=True)
aligned = alpha * u

# student-anchored target
target_logits = student_logits + tau * aligned
target_probs = target_logits.softmax(-1)

student_log_probs = student_logits.log_softmax(-1)
loss = -(target_probs * student_log_probs).sum(-1) + lam * kl_div(teacher_logprobs, student_log_probs)

Results§

At 4B and 9B scales across six fine-grained visual benchmarks, VAD outperforms direct privileged-view distillation and visual-advantage weighting. Token-level analyses confirm the aligned component is enriched in task-relevant visual corrections and produces stronger target shifts, especially when evidence refutes plausible but incorrect answers.