HemiDiff enhances robustness by using an asymmetric routing mechanism to mask degraded modalities and dynamically adjust multimodal contributions without retraining, while a coordination energy expert preserves physically consistent inter-arm coordination.
Simulation in RLBench2
We conduct simulation experiments using RLBench2, an extension of RLBench specifically designed for bimanual manipulation. The benchmark includes six tasks: Lift Ball, Lift Tray, Push Box, Push Buttons, Put Item into Drawer and Straighten Rope.
Real-World Tasks
The real-world platform consists of two UR5e robotic arms equipped with PGI grippers and four flexible piezoresistive tactile arrays, each containing 16×16 sensing units with a spatial resolution of 2 mm. The visual system comprises three Intel RealSense cameras from different viewpoints with a resolution of 640 × 480. We evaluate three categories of challenging tasks: (i) precision assembly, including Hammer into Slot and Insert Peg into Hole; (ii) occlusion handling, including Retrieve Occluded Bag and Retrieve Transparent Object, and (iii) bimanual coordination, including Handover Bread to Plate and Place Food on Plate. Each task is evaluated over 20 randomized initializations.
Comparisons with Baselines
In the real world, we compare HemiDiff with five SOTA methods: ACT, DP, DP3, DP (RGB+Tactile) with Feature Concat, and VQ-BET. HemiDiff achieves a 76% average success rate and outperforms the strongest baseline by 19 percentage points.
Generalization
To evaluate the generalization capability of HemiDiff, we introduce five representative real-world perturbations on a real robotic platform: (i) Human interference by changing the target object position at inference time; (ii) Random visual occlusion of a camera view; (iii) Single-sided tactile sensor failure; (iv) Viewpoint perturbation by translating the camera by 3 cm and rotating it by 3°; (v) Lighting variation.
Abstract
Bimanual manipulation requires both adaptive multimodal perception and physically consistent inter-arm coordination. Yet existing imitation policies typically couple all sensory inputs through fixed early-fusion structures, which can amplify modality conflicts and fail under asymmetric observations or local perception degradation. We introduce Hemispheric Diffusion (HemiDiff), a compositional diffusion policy for coordinated bimanual manipulation. HemiDiff decomposes multimodal conditioning into independent modality experts and recomposes their denoising predictions through an asymmetric perception router that assigns arm-specific modality weights at inference time. A coordination energy expert further guides the denoising process toward demonstration-consistent relative motion, preserving inter-arm coordination under decoupled perception. Experiments on RLBench2 and a real bimanual robot demonstrate that HemiDiff achieves a 76% average success rate on real-world tasks, outperforming the strongest baseline by 19 percentage points while remaining robust to occlusion, sensor degradation, and viewpoint perturbations.
Overview
Overview of HemiDiff. The HemiDiff framework consists of three core components. Independent modality experts extract visual, tactile, and point cloud features; an asymmetric perception router dynamically generates modality weights and masks for the left and right arm action subspaces based on global context, enabling perceptual decoupling in asymmetric tasks; and a coordination energy expert injects demonstration-consistent relative-motion guidance during denoising, preserving perceptual independence while enforcing physically consistent relative motion between the two arms.
