MetaUrban: A Simulation Platform for Embodied AI in Urban Spaces
Wayne Wu , Honglin He , Yiran Wang , Chenda Duan , Jack He , Zhizheng Liu , Quanyi Li , Bolei Zhou
University of California, Los Angeles
TL;DR
-
MetaUrban is a compositional simulation platform for Embodied AI research in urban spaces. It will be publicly available to enable more research opportunities for the community, and foster generalizable and safe embodied AI in urban spaces.
Abstract
Public urban spaces like streetscapes and plazas serve residents and accommodate social life in all its vibrant variations. Recent advances in Robotics and Embodied AI make public urban spaces no longer exclusive to humans. Food delivery bots and electric wheelchairs have started sharing sidewalks with pedestrians, while diverse robot dogs and humanoids have recently emerged in the street. Ensuring the generalizability and safety of these forthcoming mobile machines is crucial when navigating through the bustling streets in urban spaces. In this work, we present MetaUrban, a compositional simulation platform for Embodied AI research in urban spaces. MetaUrban can construct an infinite number of interactive urban scenes from compositional elements, covering a vast array of ground plans, object placements, pedestrians, vulnerable road users, and other mobile agents' appearances and dynamics. We design point navigation and social navigation tasks as the pilot study using MetaUrban for embodied AI research and establish various baselines of Reinforcement Learning and Imitation Learning. Experiments demonstrate that the compositional nature of the simulated environments can substantially improve the generalizability and safety of the trained mobile agents. MetaUrban will be made publicly available to provide more research opportunities and foster safe and trustworthy embodied AI in urban spaces.Procedural Generation Pipeline
![](/assets/img/metaurban/procedural_generation.png)
Urban Scene Gallery
![](/assets/img/metaurban/urbanscene_gallery.gif)
Parade of Dynamic Agents
Sensors
![](/assets/img/metaurban/sensors_grid.gif)
Benchmarks
![](/assets/img/metaurban/benchmark.png)
Results
The results of a PPO policy trained in MetaUrban environments on the social navigation task. We demonstrate success cases that can avoid collision with objects and other agents. However, there are still many interesting failure cases, which indicate the complexity of MetaUrban environments and the significant room for improvement of embodied agents in urban spaces.Success Cases
Failure Cases
User Interface for Demonstration
Impacts
Embodied AI. MetaUrban contributes to advancing areas such as robot navigation, social robotics, and interactive systems. It could facilitate the development of robust AI systems capable of understanding and navigating complex urban environments.Economy. MetaUrban could be used in businesses and services operating in urban environments, such as last-mile food delivery, assistive wheelchairs, and trash-cleaning robots. It could also drive innovation in urban planning and infrastructure development by providing simulation tools and insights into how spaces are utilized, thereby enhancing the economic and societal efficiency of public urban spaces like sidewalks and parks.
Society. By enabling the safe integration of robots and AI systems in public spaces, MetaUrban could support the development of assistive technologies that can aid in accessibility and public services. Using AI in public spaces might foster new forms of social interaction and community services, making urban spaces more livable and joyful.
Release Plan
- Demo video: June 12, 2024
- Code of MetaUrban - tiny version: June 12, 2024
- Project page: July 5, 2024
- Code of MetaUrban - official version 1.0: September 15, 2024
Acknowledgement
The project is supported by the NSF Grants CCRI-2235012, RI-2339769, and POSE-2346267, and Intel Rising Star Faculty Award.
Reference
@article{wu2024metaurban,
title={MetaUrban: A Simulation Platform for Embodied AI in Urban Spaces},
author={Wu, Wayne and He, Honglin and Wang, Yiran and Duan, Chenda and He, Jack and Liu, Zhizheng and Li, Quanyi and Zhou, Bolei},
journal={arXiv preprint arXiv:2407.08725},
year={2024}
}