From Physical Intelligence, with authors including Kevin Black, Danny Driess, Chelsea Finn, Karol Hausman, Brian Ichter, and Sergey Levine.
The problem
"In order for robots to be useful, they must perform practically relevant tasks in the real world, outside of the lab. While vision-language-action (VLA) models have demonstrated impressive results for end-to-end robot control, it remains an open question how far such models can generalize in the wild."
The paper frames this as the field's central gap: "open-world generalization represents one of the biggest open problems in physical intelligence" for "embodied systems such as robotic arms, humanoids, and autonomous vehicles."
It also states the baseline it is departing from: prior VLA demonstrations are "typically less than a minute in length and often with relatively low success rates."
The method
π0.5 "uses co-training on heterogeneous tasks to enable broad generalization," drawing on "data from multiple robots, high-level semantic prediction, web data, and other sources."
The training examples are "hybrid multi-modal examples that combine image observations, language commands, object detections, semantic subtask prediction, and low-level actions" — mixing levels of abstraction within a single example rather than training a perception stack and a control stack separately. The paper's experiments are aimed at showing "that this kind of knowledge transfer" works.
Post-training builds on π0's design, "which represents the action distribution via flow matching," where "the tokens corresponding to actions receive the partially denoised actions from the previous step."
The result
"π0.5 can control a mobile manipulator to clean kitchens and bedrooms in new homes that were not present in the training data, performing complex multi-stage behaviors with durations of 10 to 15 minutes."
Both quantities are the point of the paper against its stated baseline: the environments are unseen, and the horizon is one to two orders of magnitude longer than the sub-minute demonstrations it cites. Named tasks include "putting all of the dishes in the sink or picking all of the clothing off the floor of a new bedroom."
Relationships
- supports: AI Robotics — a leading result on open-world generalization in manipulation
- related: Embodied AI vs AGI (China's Race Redirect) — bears on whether embodiment is a separate problem or a transfer problem
- related: Embodied Perception (Tacit Knowledge vs. Pattern Recognition), Scaling Laws