Overview
BimanualShift adapts frozen pretrained unimanual policies to bimanual manipulation through arm-conditioned residual skill transfer. Instead of learning a full bimanual policy from scratch, it aligns shared bimanual observations to arm-specific inputs and learns lightweight residual coordination for synchronization, spatial avoidance, and contact compensation under few bimanual demonstrations.
BimanualShift Framework
The framework treats bimanual manipulation as arm-conditioned residual skill transfer over frozen unimanual policy priors. Arm-Conditioned Visual Projection converts the shared workspace into policy-compatible arm-specific observations, Residual Skill Modulation adds lightweight residual actions to frozen policy outputs for dual-arm coordination, and Retrieval-Conditioned Refinement reuses successful executions as residual contexts for long-horizon correction rather than direct trajectory replay.
Simulation in RLBench2
We evaluate BimanualShift on six representative bimanual manipulation tasks from RLBench2 using two Franka Panda arms and six noise-free RGB-D cameras at 256 × 256 resolution. To reflect real-world data-collection costs, the tasks are split into complex 30-demonstration tasks and simpler 100-demonstration tasks. BimanualShift-PerAct achieves the best average success rates in both settings, reaching 31.7% on the complex tasks and 50.7% on the single-action tasks.
Push Two Buttons
Straighten Rope
Lift a Ball
Pick Up a Plate
Sweep DustPan
Take Tray Out of Oven
Real-World Tasks
The evaluation involves eight real-world tasks covering diverse challenges: Flower Arrangement and Pouring Water require coordinated stability and smooth motion control; Toasting Completion and Vegetable Sorting test synchronous grasping and timing; Quilt Folding and Cable Routing demand robust control under deformable object dynamics; while Toaster Activation and Block Threading involve heterogeneous actions and high-precision spatial alignment in constrained spaces. With only 15 bimanual demonstrations per task, BimanualShift adapts PerAct and Diffusion Policy backbones pretrained on separate unimanual data. BimanualShift-PerAct achieves an average success rate of 85.6%, BimanualShift-DP reaches 65.0%, and the end-to-end DP baseline trained from scratch reaches 11.9%.
Vegetable Sorting
Pouring Water
Cable Routing
Block Threading
Toaster Activation
Toasting Completion
Flower Arrangement
Quilt Folding
Generalization
To evaluate real-world generalization, we conduct a comprehensive study on the Block Threading task using BimanualShift-PerAct. Seven settings are considered: unseen object color, unseen object shape, lighting change, left-right task exchange, unseen background, extreme glare, and camera perturbation. Under standard execution, BimanualShift-PerAct achieves 98.3% average stage-wise success and 95.0% insertion success, compared with 63.3% and 30.0% for PerAct-LF. Across appearance, configuration, and sensing changes, BimanualShift-PerAct consistently maintains stronger stage-wise success rates.
Unseen Object Color
Unseen Object Shape
Lighting Change
Left-Right Task Exchange
Unseen Background
Extreme Glare
Camera Perturbation
Long-Horizon Execution
To evaluate data efficiency and retrieval-conditioned residual refinement, we test BimanualShift under data-scarce and long-horizon settings. With only 5 demonstrations, BimanualShift-PerAct maintains 85.0%, 65.0%, and 55.0% success on Vegetable Sorting, Block Threading, and Pouring Water, while PerAct-LF drops to 25.0%, 5.0%, and 0.0%. For the long-horizon toast preparation task, which combines Insert, Press, Grasp, and Place, BimanualShift-PerAct reaches 75.0% overall success under limited demonstrations.