PAC-MAN: Perception-Aware CBF-RL for Whole-Body Safety in Humanoid Dodgeball

Overview§

PAC-MAN couples control barrier functions (CBFs) with reinforcement learning (RL) to train a whole-body dodgeball policy for humanoid robots. The key novelty is making the safety structure perception-aware: the policy is trained with CBF guidance that assumes different levels of observability, and deployment uses only segmentation-masked depth from a head-mounted camera. The authors show that the effectiveness of CBF guidance hinges on perceptual observability—joint-level barriers that excel with privileged state access degrade under fixed-camera observations, whereas a per-link barrier variant transfers robustly to the real world, achieving a 95% dodge success rate on the Unitree G1.

Methodology§

They define safety per body link $i$ using a clearance function:

$$h_i(x) = \|p_i - p_{\text{ball}}\| - r_{\text{safe}}$$

where $p_i$ is the link's position and $r_{\text{safe}}$ is a safety margin. The CBF condition, $\dot{h}_i(x, u) + \alpha h_i(x) \ge 0$, is incorporated into the RL reward as a penalty:

$$r = r_{\text{task}} - \lambda_{\text{cbf}} \sum_i \max(0, -\dot{h}_i - \alpha h_i)$$

During training, a semantic segmentation network masks the ball from the depth image, producing a masked depth observation $D_t \odot M_t$. A CNN encoder $E_\theta$ maps this to a latent state $z_t$, which is concatenated with proprioception $q_t$ to form the policy input. The policy outputs whole-body joint targets, and an adversarial motion prior (a discriminator) regularizes the policy to produce natural evasive behaviors rather than brittle, overfitted movements.

Two CBF variants are compared:

• Joint-CBF: A single barrier function over the full joint configuration, requiring accurate estimates of all link states.
• Link-CBF: Independent, lightweight barrier functions per body link, using only local clearance information.

The authors find that Joint-CBF achieves near-oracle performance with privileged ball state, but its guidance becomes harmful when observations come from a fixed camera. A ball-tracking gimbal or a privileged runtime filter restores its performance. Link-CBF, by contrast, is robust to partial observability because each link's safety constraint depends only on locally perceptible geometry.

Implementation Details§

Training uses a distributed PPO setup in simulation, with the CBF penalty computed from simulator ground-truth distances. In deployment, the policy runs at real-time on the Unitree G1, using the same perception stack (masked depth + encoder) as in training, with no privileged information. A simplified training loop is:

import torch
import torch.nn as nn

# CBF condition: h_dot + alpha * h >= 0
class CBFModule(nn.Module):
    def __init__(self, alpha=1.0):
        super().__init__()
        self.alpha = alpha
    
    def forward(self, h, h_dot):
        condition = h_dot + self.alpha * h
        return torch.relu(-condition).mean()

def train_step(policy, critic, encoder, batch):
    depth = batch["masked_depth"]
    z = encoder(depth)
    proprio = batch["proprio"]
    a, logp = policy(z, proprio)
    v = critic(z, proprio)
    
    # CBF safety loss
    h = batch["clearance"]
    h_dot = batch["clearance_deriv"]
    cbf_loss = CBFModule()(h, h_dot)
    
    # Combined PPO + CBF loss
    ppo_loss = compute_ppo_loss(a, logp, v, batch["returns"])
    loss = ppo_loss + 0.1 * cbf_loss
    loss.backward()

At inference, only the encoder, policy, and a lightweight semantic segmentation model are required. The CBF penalty is removed, so the policy's learned evasive reflexes are what remain. This zero-shot deployment succeeds on 95% of real throws, demonstrating that perception-aware CBF guidance during training produces policies that are both safe and robust to real-world sensing limitations.