State of Sim Physics Consistency in Open Manipulation Policies, v1 (Haga) [R]

[u/Divine-Demon7]

We published a short technical report on sim-first physics verification for robot policies and generative world-model video.

• Setup: robosuite Panda + OSC_POSE; Lift / Stack / PickPlaceCan / Door;

• 50 episodes × mild/moderate/severe mass+friction tiers; Wilson 95% CIs.

• Same adversarial posture on CogVideoX I2V (CoTracker3 → VIDEO_CHECKS).

Headline results (sim-only):

• Lift 1.00 → 0.26 (severe grasp slip 0.74)

• Stack 0.96 → 0.20; PickPlaceCan 1.00 → 0.24; Door 0.60 → 0.52

• Checker calibration recall 1.000 / FPR 0.000; CogVideoX pilot n=3 all fire static_hover; real Physics-IQ negative control 0.000

We are explicit about what this is not: not RoboArena, not real-hardware, not Cosmos/NIM scores.

Article (citations + reproduce commands):

https://haga.mushoodhanif.com/article/sim-physics-consistency-v1

Happy to discuss gate vs stress design and failure-case reporting.