Foundation Models for Autonomous Driving

Abstract will be announced closer to the workshop date.

Speaker bio will be announced closer to the workshop date.

Abstract will be announced closer to the workshop date.

Speaker bio will be announced closer to the workshop date.

Abstract will be announced closer to the workshop date.

Speaker bio will be announced closer to the workshop date.

Abstract will be announced closer to the workshop date.

Speaker bio will be announced closer to the workshop date.

Abstract will be announced closer to the workshop date.

Speaker bio will be announced closer to the workshop date.

Abstract will be announced closer to the workshop date.

Speaker bio will be announced closer to the workshop date.

AI agents are rapidly making the physical world executable, but much of human expertise lies beyond static data: in scientific ideas, expert workflows and interactions. I will describe autonomous discovery for autonomy and physical AI, through a series of systems that progressively expand what AI agents can do: from turning research ideas into fully executable code, to discovering and composing new methods, to verifiable experimentation on physical data and hardware. I will also explore how, as agents gain capabilities, humans remain effective participants with creativity, trust and resilience.

Manmohan Chandraker is a professor in the CSE department of University of California, San Diego. His interests are in computer vision and machine learning, with applications in physical AI including self-driving, augmented reality and robotics.

Abstract will be announced closer to the workshop date.

Speaker bio will be announced closer to the workshop date.

Autonomous driving has long scaled through road miles: larger fleets, more data, and more physical testing. World models offer a complementary path, turning compute into a scalable engine for simulation and evaluation. This talk traces their evolution from realistic scene generation to interactive, multimodal environments that place the driving policy inside the loop. We will highlight GAIA-4, Wayve’s latest world model, and explore how closed-loop camera and radar generation can produce counterfactual futures grounded in real journeys, enabling repeatable evaluation of end-to-end driving behaviour.

Vassia Simaiaki is passionate about taking research into application and has spent the last decade of her career in computer vision and deep learning. She leads AI research and development at Wayve, focusing on pushing the boundaries of generative simulation and world modelling continuing to accelerate the driving performance and evaluation of autonomous vehicles.

In this talk I will introduce Alpamayo, NVIDIA's open platform for reasoning-based autonomous vehicle development. Alpamayo contains a family of open-weight reasoning VLA models that produce interpretable chain-of-thought traces alongside trajectories. It also includes the critical tooling around them: among others, 1,700 hours of curated driving data; and AlpaSim, a neural reconstruction-based end-to-end simulator. Finally, I will talk about a next frontier: closed-loop RL post-training of large VLA or WAM models using AlpaGym, our distributed closed-loop RL framework built on top of AlpaSim and Cosmos RL.

Peter Karkus is a Senior Research Scientist at NVIDIA, at the Autonomous Systems and Physical AI Research Group. He obtained his PhD from the National University of Singapore, and has previously held visiting research appointments at MIT, CMU, Google, and DeepMind. His research interests include autonomous driving, robotics and machine learning, with particular focus on end-to-end vision-language-action models, world models, closed-loop evaluation and reinforcement learning.

Abstract will be announced closer to the workshop date.

Speaker bio will be announced closer to the workshop date.

Abstract will be announced closer to the workshop date.

Speaker bio will be announced closer to the workshop date.

The nuReasoning Challenge is here — the world’s first reasoning-centric challenge focused on long-tail autonomous driving scenarios. Join us in pushing the boundaries of Physical AI by advancing reasoning, decision-making, and planning in complex, real-world autonomous driving. We invite researchers and industry teams worldwide to build, evaluate, and benchmark the next generation of driving foundation models on some of the most challenging real-world driving scenarios. Reasoning is essential for autonomous driving (AD) in long-tail scenarios, where vehicles must apply commonsense knowledge, understand spatial relations, infer agent interactions, and make safe decisions. However, existing AD datasets and benchmarks mainly target perception, prediction, or planning, and provide limited supervision for reasoning over realistic long-tail driving scenes. We introduce nuReasoning, a large-scale real-world dataset and benchmark for reasoning-centric AD. Following the lineage of nuScenes and nuPlan, nuReasoning advances real-world AD datasets and benchmarks toward structured reasoning in long-tail driving scenarios. It contains 20K 20-second clips (247K reasoning annotations) collected across multiple cities, with synchronized multi-camera images, LiDAR, HD maps, object annotations, and complementary forms of human-verified reasoning annotations: Spatial Reasoning, Decision Reasoning, and Counterfactual Reasoning. Unlike prior datasets that focus primarily on visual question answering, nuReasoning supports both reasoning evaluation and planning evaluation, enabling a direct study of how reasoning supervision affects driving performance. Experiments show that fine-tuning VLMs on nuReasoning substantially improves driving-specific question answering, while incorporating reasoning supervision into VLA training improves planning performance even when textual reasoning outputs are disabled at inference time. These results establish nuReasoning as a foundation for evaluating and improving robust, interpretable, reasoning-driven AD systems in realistic long-tail settings.

Tony Qi is an AI industry innovation leader with more than a decade of experience advancing autonomous systems across autonomous driving, robotics, reasoning, and Physical AI. His work lies at the intersection of foundation models, multimodal reasoning, embodied intelligence, and real-world AI deployment. He is currently a Senior Principal Engineer and Team Lead at Motional leading the driving foundation model effort. Tony is particularly passionate about building AI systems that move beyond perception to reason, make decisions, and act safely in the physical world. He focuses on translating advances in foundation models and reasoning into scalable production systems, spanning the full lifecycle from technical strategy, model and system architecture, and large-scale training to evaluation, validation, release, and real-world deployment in robotaxi and consumer robotics products. He has led AI teams and technology initiatives at Motional, Amazon Consumer Robotics, Toyota Research Institute, and GM, with work spanning perception, mapping, prediction, planning, reinforcement learning, and, more recently, next-generation driving foundation models, including Vision-Language-Action (VLA) models, world models, and agentic AI systems. As a technical and people leader, Tony has built and scaled high-performing AI teams and mentored engineers and applied scientists across both frontier research and product development. He is also an active technical contributor to autonomous driving and robotics, with more than 30 publications and 20 U.S. patents spanning AI, autonomous systems, and robotics. Tony is the originator and project lead of nuReasoning, a pioneering reasoning-centric autonomous driving initiative created to advance how foundation models understand, reason about, and make decisions in complex and long-tail driving scenarios.

Abstract will be announced closer to the workshop date.

Speaker bio will be announced closer to the workshop date.

Abstract will be announced closer to the workshop date.

Speaker bio will be announced closer to the workshop date.

Abstract will be announced closer to the workshop date.

Speaker bio will be announced closer to the workshop date.

End-to-end models that map multimodal inputs directly to future trajectories/maneuvers have emerged as an increasingly prominent research paradigm in autonomous driving. This class of models includes both Vision-Language-Action models and trajectory-generative planners. Unlike classic machine learning applications, autonomous vehicles operate in safety-critical and interaction-intensive environments where traditional open-loop imitation of expert demonstrations is not sufficient to ensure reliability. In particular, small execution errors can accumulate over time, while recovery behaviors are scarce in training data. In addition, long-horizon objectives such as safety and driving comfort are not captured by pointwise labels either. These limitations have motivated a shift toward post-training techniques, which further refine driving policies beyond pure imitation. This survey presents a unified view of post-training for autonomous driving by defining its scope and organizing the existing literature into four major families based on the form of supervision they use. For each family, we discuss its capabilities, limitations, and open challenges. We aim to facilitate a systematic understanding of this emerging area and stimulate future research on reliable and efficient post-training for autonomous driving.

Speaker bio will be announced closer to the workshop date.

Video generation for autonomous driving cannot follow the web-scale route: driving data is expensive to collect, bound by privacy requirements, and cannot be scraped at will, so models must make the most of a fixed corpus. We present a systematic scaling-law study of video diffusion models trained from scratch on driving data: a family of models from 1M to 9B parameters, trained at different exposures on up to 5,500 hours of driving. Validation loss follows consistent power laws in both model size and training exposure, answering the questions that shape a training budget: whether compute is better spent on longer training or on a larger model, and whether more data is needed. Loss improves much faster with training exposure than with model size, making longer training the most effective way to improve a fixed model under limited compute. However, larger models continue to achieve lower asymptotic loss, so compute-optimal scaling still favors increasing model size when sufficient compute and data are available. Guided by these laws, we train a 9B-parameter model, to our knowledge the largest video diffusion model trained from scratch on driving data: it sets a new open-source state of the art for driving video generation, as measured on nuScenes. Our code and pretrained models are available at https://github.com/valeoai/VATIX. NATIX is separately releasing the underlying driving data in stages.

Speaker bio will be announced closer to the workshop date.

Current world models operate at a single level of abstraction, with most prioritizing perceptual fidelity while lacking the spatial reasoning and semantic understanding required for real-world downstream tasks. We present a hierarchical driving world model that factorizes future prediction across two levels operating at distinct temporal and abstraction scales: a high-level predictor that forecasts coarse scene structure over extended temporal horizons, and a low-level generator that produces detailed predictions conditioned on the high-level output. This decomposition yields high perceptual fidelity while also capturing strong spatial and semantic representations. We further show that pretraining with a diffusion forcing objective yields substantially richer internal representations than the standard teacher forcing objective, while teacher forcing — predicting only the next frame from clean context — produces more stable autoregressive rollouts. We therefore introduce a generic two-stage training paradigm that pretrains the model with diffusion forcing and fine-tunes with teacher forcing, combining the representational benefits of the former with the rollout stability of the latter. Our approach achieves state-of-the-art results across the standard suite of driving world model evaluations on established benchmarks, including long-horizon generation fidelity, steering responsiveness evaluated on counterfactual scenarios, and internal representation quality.

Speaker bio will be announced closer to the workshop date.

Existing autonomous driving datasets have enabled major progress, but fall short in sensor fidelity, map completeness, or geographic diversity. We present KITScenes Multimodal, a European dataset built around high-fidelity sensors and maps. Our fully synchronized sensor suite combines high-resolution global-shutter cameras, long-range lidar beyond 400 meters, 4D imaging radar, and redundant GNSS/INS localization. Our HD maps are, to our knowledge, the most complete of any sensor dataset, verified through autonomous driving trials on open-source software. For the first time in a public dataset, all driving-relevant traffic elements, such as traffic lights, are mapped in 3D to a reprojection-accurate level with full topological connectivity. Recorded in cities with irregular street layouts and mixed traffic modes, our dataset complements existing datasets by broadening the available geographic diversity. We also introduce four benchmarks, each advancing spatial learning for embodied AI: online HD map construction, long-range depth estimation, novel view synthesis, and end-to-end driving.

Speaker bio will be announced closer to the workshop date.

Unsupervised 3D instance segmentation of outdoor LiDAR scans has traditionally relied on handcrafted geometric priors such as density-based clustering, motion cues, or projected 2D detections. In this work, we investigate whether a frozen, self-supervised point transformer already contains the structural information required to isolate object instances without any handcrafted geometric prior. Using this transformer purely as a feature extractor, we probe its internal representations across the SemanticKITTI, nuScenes, and Waymo Perception datasets. Our analysis yields four core insights: (1) the instance signal concentrates in the attention queries and keys rather than in the values or final output features; (2) output features semantically collapse, merging adjacent same-class objects that the queries and keys keep distinct; (3) this instance signal is bimodal in depth, strongest at the shallowest and deepest encoder stages; and (4) this signal is driven predominantly by the rotary position encoding (RoPE), whose removal collapses its advantage. We put these findings into our method TokenGraph3D, a training-free segmenter that groups points via connected components on a key-similarity graph, using neither density-based clustering nor proximity priors. Under identical prior-free conditions, we substantially outperform output-feature baselines, making the emergent 3D instance structure visible.

Speaker bio will be announced closer to the workshop date.