RoboTwin 2.0 Benchmark

Benchmark Setting:

All policies are trained on the Aloha-AgileX embodiment using 50 demo_clean demonstrations for each single task (specifically, all pre-trained policies (e.g. Pi0) are fine-tuned on the single task based on their open-source weights), and evaluated 100 times under the demo_clean (Easy) and demo_randomized (Hard) settings. All training details are provided in the Appendix of the RoboTwin 2.0 Paper.

News:
Pending: G3Flow, DexVLA, TinyVLA, LLaVA-VLA, OpenVLA-oft, Chain-of-Action
RDT Pi0 ACT DP DP3
Easy Hard Easy Hard Easy Hard Easy Hard Easy Hard
Adjust Bottle 81% 75% 90% 56% 27% 14% 97% 0% 99% 3%
Beat Block Hammer 77% 37% 43% 21% 0% 0% 42% 0% 72% 8%
Blocks Ranking RGB 3% 0% 19% 5% 0% 0% 0% 0% 3% 0%
Blocks Ranking Size 0% 0% 7% 1% 0% 0% 1% 0% 2% 0%
Click Alarmclock 61% 12% 63% 11% 11% 0% 61% 5% 77% 14%
Click Bell 80% 9% 44% 3% 4% 2% 54% 0% 90% 0%
Dump Bin Bigbin 64% 32% 83% 24% 2% 0% 49% 0% / /
Grab Roller 74% 43% 96% 80% 66% 6% 98% 0% 98% 2%
Handover Block 45% 14% 45% 8% 0% 0% 10% 0% 70% 0%
Handover Mic 90% 31% 98% 13% 0% 0% 53% 0% 100% 3%
Hanging Mug 23% 16% 11% 3% 0% 0% 8% 0% 17% 1%
Lift Pot 72% 9% 84% 36% 7% 2% 39% 0% 97% 0%
Move Can Pot 25% 12% 58% 21% 0% 0% 39% 0% 70% 6%
Move Playingcard Away 43% 11% 53% 22% 3% 0% 47% 0% / /
Move Stapler Pad 2% 0% 0% 2% 0% 0% 1% 0% 12% 0%
Open Laptop 59% 32% 85% 46% 32% 0% 49% 0% 82% 7%
Open Microwave 37% 20% 80% 50% 0% 0% 5% 0% 61% 22%
Pick Diverse Bottles 2% 0% 27% 6% 3% 0% 6% 0% 52% 1%
Pick Dual Bottles 42% 13% 57% 12% 4% 0% 24% 0% 60% 1%
Place A2B Left 3% 1% 31% 1% 0% 0% 2% 0% 46% 2%
Place A2B Right 1% 1% 27% 6% 0% 0% 13% 0% 49% 0%
Place Bread Basket 10% 2% 17% 4% 0% 0% 14% 0% 26% 1%
Place Bread Skillet 5% 1% 23% 1% 0% 0% 11% 0% 19% 0%
Place Burger Fries 50% 27% 80% 4% 0% 0% 72% 0% 72% 18%
Place Can Basket 19% 6% 41% 5% 0% 0% 18% 0% 67% 2%
Place Cans Plasticbox 6% 5% 34% 2% 0% 0% 40% 0% 48% 3%
Place Container Plate 78% 17% 88% 45% 0% 0% 41% 0% 86% 1%
Place Dual Shoes 4% 4% 15% 0% 0% 0% 8% 0% 13% 0%
Place Empty Cup 56% 7% 37% 11% 0% 0% 37% 0% 65% 1%
Place Fan 12% 2% 20% 10% 0% 0% 3% 0% 36% 1%
Place Mouse Pad 1% 0% 7% 1% 0% 0% 0% 0% 4% 1%
Place Object Basket 33% 17% 16% 2% 0% 0% 15% 0% 65% 0%
Place Object Scale 1% 0% 10% 0% 0% 0% 1% 0% 15% 0%
Place Object Stand 15% 5% 36% 11% 0% 0% 22% 0% 60% 0%
Place Phone Stand 15% 6% 35% 7% 0% 0% 13% 0% 44% 2%
Place Shoe 35% 7% 28% 6% 0% 0% 23% 0% 58% 2%
Press Stapler 41% 24% 62% 29% 17% 1% 6% 0% 69% 3%
Put Bottles Dustbin 21% 4% 54% 13% 0% 0% 22% 0% 60% 21%
Put Object Cabinet 33% 18% 68% 18% 0% 0% 42% 0% 72% 1%
Rotate QRcode 50% 5% 68% 15% 0% 0% 13% 0% 74% 1%
Shake Bottle Horizontally 84% 51% 99% 51% 54% 19% 59% 18% 100% 25%
Shake Bottle 74% 45% 97% 60% 38% 0% 65% 8% 98% 19%
Stack Blocks Three 2% 0% 17% 0% 0% 0% 0% 0% 1% 0%
Stack Blocks Two 21% 2% 42% 1% 0% 0% 7% 0% 24% 0%
Stack Bowls Three 51% 17% 66% 24% 0% 0% 63% 0% 57% 5%
Stack Bowls Two 76% 30% 91% 41% 0% 0% 61% 0% 83% 6%
Stamp Seal 1% 0% 3% 4% 0% 0% 2% 0% 18% 0%
Turn Switch 35% 15% 27% 23% 7% 1% 36% 1% 46% 8%
Move Pillbottle Pad 8% 0% 21% 1% 0% 0% 1% 0% 41% 0%
Average 35% 14% 47% 17% 6% 1% 28% 1% 55% 4%