跳到主要导航 跳到搜索 跳到主要内容

UniDriveDreamer: A Single-Stage Multimodal World Model for Autonomous Driving

  • Guosheng Zhao
  • , Yaozeng Wang
  • , Xiaofeng Wang
  • , Zheng Zhu*
  • , Yu Tingdong
  • , Guan Huang
  • , Yongchen Zai
  • , Ji Jiao
  • , Changliang Xue
  • , Xiaole Wang
  • , Zhen Yang
  • , Futang Zhu
  • , Xingang Wang*
  • *此作品的通讯作者
  • GigaAI
  • CAS - Institute of Automation
  • BYD Company Ltd.

科研成果: 书/报告/会议事项章节 › 会议稿件 › 同行评审

摘要

World models have demonstrated significant promise for data synthesis in autonomous driving. However, existing methods predominantly concentrate on single-modality generation, typically focusing on either multi-camera video or LiDAR sequence synthesis. In this paper, we propose UniDriveDreamer, a single-stage unified multimodal world model for autonomous driving, which directly generates multimodal future observations without relying on intermediate representations or cascaded modules. Our framework introduces a LiDAR-specific variational autoencoder (VAE) designed to encode input LiDAR sequences, alongside a video VAE for multi-camera images. To ensure cross-modal compatibility and training stability, we propose Unified Latent Anchoring (ULA), which explicitly aligns the latent distributions of the two modalities. The aligned features are fused and processed by a diffusion transformer that jointly models their geometric correspondence and temporal evolution. Additionally, structured scene layout information is projected per modality as a conditioning signal to guide the synthesis. Extensive experiments demonstrate that UniDriveDreamer outperforms previous state-of-the-art methods in both video and LiDAR generation, while also yielding measurable improvements in downstream driving tasks.

源语言英语
主期刊名Computer Vision – ECCV 2026 - 19th European Conference, 2026, Proceedings
编辑Paolo Favaro, Zuzana Kukelova, Atsuto Maki, Anna Rohrbach, Konrad Schindler, Federico Tombari
出版商Springer Science and Business Media Deutschland GmbH
页360-377
页数18
ISBN(印刷版)9783032373137
DOI
出版状态已出版 - 2026
已对外发布是
活动19th European Conference on Computer Vision, ECCV 2026 - Malmö, 瑞典
期限: 8 9月 2026 → 12 9月 2026

丛书

姓名Lecture Notes in Computer Science
卷17047 LNCS
ISSN(印刷版)0302-9743
ISSN(电子版)1611-3349

会议

会议19th European Conference on Computer Vision, ECCV 2026
国家/地区瑞典
市Malmö
时期8/09/26 → 12/09/26

学术指纹

探究 'UniDriveDreamer: A Single-Stage Multimodal World Model for Autonomous Driving' 的科研主题。它们共同构成独一无二的学术指纹。

引用此