Adv-BMT: Bidirectional Motion Transformer for Safety-Critical Traffic Scenario Generation

TL; DR

Adv-BMT augments real-world driving logs with realistic and diverse collision interactions.

Adv-BMT contains three steps: (1) adversarial initialization (2) reverse motion prediction, and (3) rule-based fallbacks.

Adv-BMT is an adversarial scenario generator for closed-loop learning: continuously produces diverse accident interactions targeting at the ego vehicle.

Adv-BMT

Adv-BMT first initializes diverse collision states between a new adversary agent and ego vehicle; then, it reconstructs the adversarial trajectories via BMT’s reverse predictions. A rule-based fallback mechanism is used to reject candidate adversarial trajectories. In the output scene, the new adversarial agent maintains realistic interactions with surrounding traffic.

BMT for Bidirectional Motion Prediction

BMT employs two sets of motion tokens for forward and reverse predictions to generate the next-step token for each agent. All predictions are conditioned only on the map and the one-step current state of all predicted agents.

Diverse Adversarial Behaviors

Real-world Rendering

We leverage Dreamland for rendering Adv-BMT scenarios into real-world accident videos.

Safer Agent via Adversarial Learnings

Adversarial RL Evaluations in Waymo Environments

Related Works

MetaDrive (TPAMI 2021): An open-source platform for large-scale traffic scenario simulation and modeling.

CAT (CoRL 2022): Closed-loop adversarial training for safe end-to-end driving.

Dreamland (Arxiv 2025): An open-source generative model for real-world rendered videos.

Reference

Bidirectional Motion Transformer for Safety-Critical Traffic Scenario Generation (NeurIPS 2025):

@inproceedings{
liu2025bidirectional,
title={Bidirectional Motion Transformer for Safety-Critical Traffic Scenario Generation},
author={Yuxin Liu and Zhenghao Peng and Xuanhao Cui and Bolei Zhou},
booktitle={The Thirty-ninth Annual Conference on Neural Information Processing Systems},
year={2025},
url={https://openreview.net/forum?id=avZ01E4aYt}
}