$π\mathbf{R}^2$: Reactive Real-time Flow Policies

Technical Overview§

πR² addresses the latency-reactivity tradeoff in flow-based robot manipulation policies. Standard action-chunking flow policies (e.g., GR00T-N1.7) run open-loop within a chunk because the perception-to-action pipeline (large vision-language backbone + multiple denoising steps) is too slow. πR² enables closed-loop replanning at ~25 Hz by: (1) splitting conditioning into a fast proprioception channel and a slow vision-language channel, and (2) using a latency-adaptive flow schedule that treats in-flight actions as inpainting conditioning and produces an action in a single denoising step.

Dual-Channel Conditioning§

Let the policy be a conditional flow matching model. Denote the state at time step $t$ as $x_t$, the action chunk as $a_{t:t+H-1}$, and the observation as $o_t = (p_t, v_t)$ where $p_t$ is proprioception (joint angles, velocities) and $v_t$ is visual observation. In standard flow matching, the denoising function $\epsilon_\theta(a, \sigma, o_t)$ conditions on the entire observation. πR² splits conditioning into:

• Fast channel: proprioception $p_t$, updated every tick (40 ms).
• Slow channel: vision-language features $f_t = \phi(v_t)$ from a frozen backbone, updated asynchronously (e.g., every 200 ms).

The denoising function becomes $\epsilon_\theta(a, \sigma, p_t, f_{\lfloor t/\tau \rfloor})$ where $\tau$ is the slow channel update period. This allows the policy to react to proprioception mid-chunk while tolerating stale visual features.

Latency-Adaptive Flow Schedule§

Define the action chunk duration $H \cdot \Delta t$ (e.g., 10 steps × 40 ms = 400 ms). During inference, the policy must output an action $a_t$ at every tick. Instead of full denoising, πR² uses a flow inpainting formulation. Let $\mathbf{a} = [a_t, a_{t+1}, ..., a_{t+H-1}]$ be the current chunk. Some actions $a_{t+1}, ..., a_{t+H-1}$ may already be committed from the previous chunk. These are treated as known inpainting regions. At tick $t$, the policy receives fresh proprioception $p_t$ and stale visual features $f$. It samples noise $\mathbf{a}_\sigma = \mathbf{a} + \sigma \mathbf{n}$ and performs one denoising step from noise level $\sigma_{\text{start}}$ to $\sigma_{\text{end}}$ (typically from high to low noise, often $\sigma_{\text{start}}=1$, $\sigma_{\text{end}}=0$):

$$\mathbf{a}_{\text{new}} = \mathbf{a}_\sigma - (\sigma_{\text{start}} - \sigma_{\text{end}}) \cdot \epsilon_\theta(\mathbf{a}_\sigma, \sigma_{\text{start}}, p_t, f).$$

The inpainting region enforces that already-committed actions remain unchanged. The resulting action $a_t$ is executed. This single-step denoising is possible because diffusion forcing adapts the noise schedule per position.

Implementation Details§

πR² is fine-tuned from a pretrained GR00T-N1.7 policy. Training uses standard flow matching loss with the dual-channel conditioning and a modified noise schedule. The slow channel features are cached and updated every $k$ ticks (e.g., $k=5$). The single-step denoising schedule is learned by training with a fixed number of denoising steps (usually 1).

import torch
import torch.nn as nn

class PiR2Policy(nn.Module):
    def __init__(self, backbone, action_dim, chunk_len):
        super().__init__()
        self.backbone = backbone  # vision-language encoder (slow)
        self.proprio_encoder = nn.Linear(6, 64)  # example
        self.denoiser = Denoiser(action_dim, chunk_len)  # flow matching denoiser

    def forward(self, obs, prev_actions, noise_level, slow_feat=None):
        # obs: dict with 'proprio' and 'vision'
        p = self.proprio_encoder(obs['proprio'])  # fast
        if slow_feat is None:
            slow_feat = self.backbone(obs['vision'])  # slow, cached
        # Concatenate conditioning
        cond = torch.cat([p, slow_feat], dim=-1)
        # Inpainting mask: known actions (committed) get conditioned with small noise
        action_noisy = prev_actions + noise_level * torch.randn_like(prev_actions)
        noise_pred = self.denoiser(action_noisy, noise_level, cond)
        return action_noisy - (noise_level - 0.0) * noise_pred  # one step denoising

Results§

On a real xArm6+XHand platform with A5000 GPU, πR² achieves ~25 Hz replanning (every 40 ms) compared to ~6 Hz for the base policy. In simulation, success rate improves by up to 23%; in real-world tasks, up to 30% over strong baselines. Minimal architectural modifications allow fine-tuning from existing pretrained policies.