VLA Models
Multimodal foundation models that take images and text instructions as input and output sequences of motor actions. The core of 2024-2025 generalist robots.

A VLA model (Vision-Language-Action) is a multimodal foundation model that takes images and a text instruction as input and outputs a sequence of motor actions a robot can execute — joint torque vectors, X/Y/Z displacements, gripper opening. The term was introduced by Google DeepMind in the RT-2 paper (July 2023) and now refers to the main architecture behind generalist robots: OpenVLA (Stanford, 2024), π0 (Physical Intelligence, 2024), Helix (Figure AI, 2025), GR00T N1 (NVIDIA, 2025), Gemini Robotics (DeepMind, 2025). VLAs are the "brain" of modern embodied AI.
Typical architecture
A VLA is generally built in three layers:
Vision encoder — extracts a latent representation from one or more camera images. Common building blocks: SigLIP (Google), DINOv2 (Meta), a pre-trained ViT. Recent models use several cameras (head view, wrist views) with cross-fusion.
Language encoder / VLM backbone — a pre-trained VLM provides high-level reasoning. Models used: PaLI-X and PaLM-E for RT-2, Llama 2 for OpenVLA, PaliGemma (Gemma + SigLIP) for π0, Eagle 2 for GR00T N1. Web pre-training brings the common-sense knowledge (objects, relationships, instruction semantics) that pure robotics datasets sorely lack.
Action head — converts the joint representation into motor commands. Several schemes exist: discrete action tokenization (RT-2, OpenVLA, like a vocabulary of 256 entries per dimension), diffusion of continuous actions (π0 uses flow matching to produce smooth trajectories at 50 Hz), or a specialized autoregressive decoder.
Dual-system architecture (2025)
The 2025 models (Figure's Helix, NVIDIA's GR00T N1, partly π0.5) adopt a dual-system architecture inspired by Kahneman's System 1 / System 2:
- System 2 (slow, deliberative) — a VLM reasons about the scene, plans the sub-task and formulates the intent. Runs at 5-10 Hz and can be offloaded to a high-end GPU.
- System 1 (fast, reflexive) — a compact policy network converts the intent into high-frequency motor commands (200 Hz) on the robot's embedded compute.
This split resolves a dilemma: a large VLM is too slow for the motor loop, and a small model is too limited for semantics. Coupling the two delivers the robustness of large models with the responsiveness physical control requires.
Training and data
VLAs are trained in two phases. Pre-training on massive web corpora (text + images), inherited from the VLM backbone, provides general semantics. Robotics fine-tuning on episode datasets (image + instruction → action) draws on:
- Open X-Embodiment (Stanford + 21 institutions, 2024) — 1M+ episodes, 22 embodiments. The community's common foundation.
- DROID (Stanford, 2024) — 76,000 demonstrations across 564 scenes, a household dataset.
- Proprietary data from manufacturers (Tesla collects Optimus trajectories internally, Figure leverages its industrial partnerships).
The dominant data sources are teleoperation (a human drives the robot while the system records the trajectories) and simulation. The sim/real ratio varies by approach — π0.5 bets on the diversity of real data, GR00T N1 on the volume of simulation.
Current limitations
Three research fronts dominate in late 2025.
Inference latency and cost — a 7B-parameter VLA typically draws 100-200 W during inference at 10 Hz, versus 30-50 W for a classic controller. Distillation into more compact models and dedicated chips (NVIDIA Thor, Tenstorrent) are active areas.
Fine manipulation — long-duration bimanual dexterity (assembling, folding laundry, handling cables) remains a sticking point. Datasets lack diversity on these tasks.
Safety and certification — VLAs are probabilistic foundation models: they can't be formally certified (IEC 61508, ISO 13482). Industrial deployments pair them with symbolic verifiers or a human supervisor.
Standards and Specifications
EU AI Act — GPAI models
General-purpose VLAs (cross-task, cross-embodiment) fall under the 'general-purpose AI models' category of Regulation (EU) 2024/1689 — transparency obligations on training, and systemic risk management above 10²⁵ FLOPs
Open X-Embodiment
Shared dataset and benchmark (1M+ episodes, 22 embodiments) — the de facto training and evaluation base for modern VLAs (OpenVLA, π0, GR00T)
Frequently Asked Questions
What is the difference between a VLA and a multimodal LLM like GPT-4o?
GPT-4o, Claude and Gemini are VLMs (Vision-Language Models): they take image + text as input and produce text as output. They can't drive an actuator.
A VLA takes the same inputs (image + instruction) but outputs a sequence of motor actions — joint torque vectors, X/Y/Z displacements, gripper opening. The typical recipe: start from a pre-trained VLM (PaLI-X for RT-2, PaliGemma for π0, Eagle 2 for GR00T N1) and replace or extend the text-generation head with an action head, fine-tuned on millions of robot episodes.
Why is the term 'VLA' so recent?
It was introduced by Google DeepMind in the RT-2 paper (July 2023). Before RT-2, robot policies were either symbolic (planners) or learned end to end, task by task (BC-Z, RT-1). RT-2 was the first to show that a VLM pre-trained on the web and then fine-tuned on robot data inherited emergent capabilities — for example, executing a command like 'throw away the trash' without ever having seen the word 'trash' in the robot data, by drawing on the VLM's internet knowledge.
Since then, OpenVLA (Stanford, June 2024), π0 (Physical Intelligence, October 2024), Helix (Figure AI, February 2025), GR00T N1 (NVIDIA, March 2025) and Gemini Robotics (Google DeepMind, 2025) have consolidated the approach.
Can a VLA run locally on a robot?
Yes, but it's tight. OpenVLA has 7B parameters: it can run on a mobile RTX 4090 or a Jetson AGX Orin with quantization, but only at 5-10 Hz — not enough for direct motor control. Most current architectures (Helix, GR00T) therefore offload the slow module (reasoning) to a high-end GPU at 5-10 Hz and keep a smaller, fast local module at 200 Hz for the motor loop.
Cloud latency remains out of the question: you can't close a 200 Hz control loop through a remotely hosted model. Future distilled VLAs (1-3B params) and dedicated chips (NVIDIA Thor, Tenstorrent, Etched) aim to run the whole stack embedded.
Which VLAs are open source?
Fully open source (weights + code): OpenVLA (Stanford, 2024), π0 (Physical Intelligence, October 2024) and π0.5 (unveiled in April 2025, weights released in September 2025), under the Apache 2.0 license, GR00T N1 (NVIDIA, 2025, the first open humanoid foundation model).
Closed: RT-2 and Gemini Robotics (Google DeepMind), Helix (Figure AI). Open models now dominate academic research and startup experimentation. For industrial deployment, the gap is closing: its authors present π0.5 as the first end-to-end learned system able to carry out long, dexterous tasks (cleaning a kitchen or a bedroom) in homes it has never seen.