跳到主要导航 跳到搜索 跳到主要内容

Optimization-Guided Diffusion for Interactive Scene Generation

  • Shihao Li*
  • , Naisheng Ye
  • , Tianyu Li
  • , Kashyap Chitta
  • , Tuo An
  • , Peng Su
  • , Boyang Wang
  • , Haiou Liu
  • , Chen Lv
  • , Hongyang Li
  • *此作品的通讯作者
  • Beijing Institute of Technology
  • The University of Hong Kong
  • Nanyang Technological University
  • KE:SAI – Kyutai ELLIS Scalable Autonomous Intelligence
  • NVIDIA
  • Yinwang Intelligent Tech. Co. Ltd.

科研成果: 书/报告/会议事项章节 › 会议稿件 › 同行评审

摘要

Realistic and diverse multi-agent driving scenes are crucial for evaluating autonomous vehicles, but safety-critical events which are essential for this task are rare and underrepresented in driving datasets. Data-driven scene generation offers a low-cost alternative by synthesizing complex traffic behaviors from existing driving logs. However, existing models often lack controllability or yield samples that violate physical or social constraints, limiting their usability. We present OMEGA, an optimization-guided, training-free framework that enforces structural consistency and interaction awareness during diffusion-based sampling from a scene generation model. OMEGA re-anchors each reverse diffusion step via constrained optimization, steering the generation towards physically plausible and behaviorally coherent trajectories. Building on this framework, we formulate ego–attacker interactions as a game-theoretic optimization in the distribution space, approximating Nash equilibria to generate realistic, safety-critical adversarial scenarios. Experiments on nuPlan and Waymo show that OMEGA improves generation realism, consistency, and controllability, increasing the ratio of physically and behaviorally valid scenes from 32.35% to 72.27% for free exploration capabilities, and from 11% to 80% for controllability-focused generation. Our approach can also generate 5× more near-collision frames with a time-to-collision under three seconds while maintaining the overall scene realism.

源语言英语
主期刊名Computer Vision – ECCV 2026 - 19th European Conference, 2026, Proceedings
编辑Paolo Favaro, Zuzana Kukelova, Atsuto Maki, Anna Rohrbach, Konrad Schindler, Federico Tombari
出版商Springer Science and Business Media Deutschland GmbH
页228-244
页数17
ISBN(印刷版)9783032371669
DOI
出版状态已出版 - 2026
活动19th European Conference on Computer Vision, ECCV 2026 - Malmö, 瑞典
期限: 8 9月 2026 → 12 9月 2026

丛书

姓名Lecture Notes in Computer Science
卷17033 LNCS
ISSN(印刷版)0302-9743
ISSN(电子版)1611-3349

会议

会议19th European Conference on Computer Vision, ECCV 2026
国家/地区瑞典
市Malmö
时期8/09/26 → 12/09/26

学术指纹

探究 'Optimization-Guided Diffusion for Interactive Scene Generation' 的科研主题。它们共同构成独一无二的学术指纹。

引用此