Wayve has introduced a new scene reconstruction model, Prism-1, that enables the realistic simulation of 4D scenes (3D in space plus time) from video data to improve the testing and training of its ADAS and AD technology. This model provides a flexible scene representation, expanding the variety of scenarios that can be simulated with the Ghost Gym system.
Introduced in December 2023, Ghost Gym is a closed-loop neural simulator tailored for autonomous driving (AD), enabling consistent testing conditions to evaluate and validate driving models. Wayve says Ghost Gym accelerates development cycles by enabling rapid iteration and testing of algorithms.
It can also support scene augmentation by changing vehicle viewpoints, positions or speeds to create novel scenarios for evaluation, which is essential to ensure safety.
A key aspect of an effective simulator is the realism of the environment. According to the company, Prism-1 improves the realism of simulation environments, powering the next generation of Ghost Gym scenarios.
Simulating dynamic urban scenes
Simulation plays a crucial role in developing AD technologies by providing a safe, repeatable and cost-effective method to test and refine driving models. Real-world testing poses challenges due to variable conditions, even on the same road. Prism-1 aims to address these issues through controlled model evaluation and exploration of various hypothetical scenarios.
“Prism-1 bridges the gap between the real world and our simulator,” said Jamie Shotton, chief scientist at Wayve. “By enhancing our simulation platform with accurate dynamic representations, Wayve can extensively test, validate and fine-tune our AI models at scale.”
Simulating urban environments for AD involves accurately representing dynamic elements such as pedestrians, cyclists and vehicles. These elements exhibit unpredictable behaviors and varied appearances, posing technical challenges and safety concerns.
Traditional scene graphs, used to organize scene elements hierarchically, struggle with dynamic interactions and complex environmental conditions, leading to error propagation and increased complexity in dense urban settings.
Building on an end-to-end approach to AD, Wayve says Prism-1 is designed to improve how autonomous systems interact with and respond to complex urban landscapes.
Prism-1
Key features of the simulator include self-supervised scene disentanglement, which separates static and dynamic elements within scenes to eliminate the need for explicit labels or predefined models. This enables adaptability to different camera setups without additional sensors or explicit 3D information.
Prism-1 can also remain efficient as scene complexity escalates, minimizing engineering effort and curtailing error propagation typically arising from conventional modeling techniques.
The simulator employs a flexible framework to identify and track changes in the appearance of scene elements over time. This capability enables precise re-simulation of complex dynamic scenarios where elements change shape and move throughout the scene.
Wayve says Prism-1’s ability to reconstruct detailed, high-fidelity scenes meets the demands of closed-loop simulation and model training. This enables quick iteration at scale, focusing on resimulating driving scenes while generative models like GAIA-1 generate entirely new scenarios.
“We are building embodied AI technology that generalizes and scales,” added Shotton. “To achieve this, we continue to advance our end-to-end AI capabilities, not only in our driving models, but also through enabling technologies like Prism-1. We are also excited to publicly release our WayveScenes101 data set, developed in conjunction with Prism-1, to foster more innovation and research in novel view synthesis for driving.”
Read Wayve’s full blog post outlining all of Prism-1’s simulations of static and dynamic scenes here.