MSD / Research / RESEARCH EDITION

RT-2: from image and sentence to robot action

RT-2 combines image and language knowledge with robotic action output. Semantic cues should be transferable to the manipulation.

Jetzt eintauchen ↓RECHERCHESTAND 12.11.2026 · FAKTEN + EINORDNUNG + SZENARIEN
RESEARCH NOTE / 2023Research

Robotics & Physical AI

External research · MSD classification
Keine Behauptung eines eigenen Versuchs.

01 / EINORDNUNG

What this finding means.

Grenzen des Nachweises

Knowledge about an object does not mean that a robot reliably knows its weight, friction and secure grip points.

Our derivation for a pilot operation

Measure semantic understanding and physical success rate separately.

This derivation is an editorial recommendation from MSD. It is not a further result of the study cited.

Source and version

Brohan et al. · RT-2 · Publikations-/Bezugsjahr 2023. Recherchiert am 12.11.2026.

Read primary source ↗

External research, not an MSD customer reference. We will only publish our own experiments with their setup, date, measurements and documented failed attempts.

02 / LINKS

Think ahead. Keep checking.

Robotics & Physical AI / Research / 2023

Bewegungen als Verteilung statt einzelner Befehl ↗

Diffusion Policy learns action sequences for robotic manipulation. The project page shows experiments with visual observations and various tasks.

Chi et al. · Diffusion Policy
Robotics & Physical AI / Dataset / 2023

Many robots, one common data room ↗

Open X-Embodiment bundles data sets from different robotic setups. Shared data is intended to facilitate transfer between tasks and platforms.

Open X-Embodiment Collaboration
Robotics & Physical AI / Research / 2024

Octo: an open generalist control ↗

Octo provides an open robotics policy that can be adapted for new observations and action spaces.

Octo Model Team