Over the development period of its automated driving services, Alphabet-owned Waymo has created a software solution internally dubbed SimulationCity, which it states helps to improve its AD evaluation tools.
According to the company, SimulationCity automatically synthesizes entire journeys to assess the Waymo Driver’s performance. This capability is enabled by the more than 20 million autonomous miles worth of data the Waymo Driver has collected on public roads, third-party data such as NHTSA Crash Data Systems and Naturalistic Driving Study Data, Waymo’s access to parent company Alphabet’s technical infrastructure, and the culmination of advancements in sensor simulation and machine learning techniques in areas such as intelligent agents, object trajectory generation, and high-quality automated data labeling.
Waymo said, “With the power of SimulationCity, we can significantly accelerate the development and deployment of the Waymo Driver – an ‘everything engine’ that allows us to safely bring up new vehicle platforms, test them in new geographies and refine our operations so we can bring our technology to more people in more places.”
For simulation to be an effective tool, it has to closely represent the environment it’s emulating to provide accurate insights into a system’s performance. One-way Waymo says it ensures statistical realism in its simulated world is by creating realistic conditions for the autonomous driving technology to experience. For example, if the Waymo Driver is going through spring showers in Detroit at sunset in SimulationCity, it can recreate raindrops on the vehicle sensors and even simulate other minute details such as the dimming light and solar glare.
The company also wants to understand how its system reacts to the full distribution of behaviors it will encounter in the real world. It gives the example of a tailgating scenario at an intersection. It wants to understand as many various outcomes as possible and their likelihood to happen to evaluate the Waymo Driver’s behavior. If a random tailgating scenario were to be picked, the chances are the tailgater would brake in time. But how would the Waymo Driver behave if the tailgater doesn’t brake in time, such as when that driver is distracted or inattentive? It notes that with ever more variations of the same scenario, a convergence of the distribution of outcomes between what is observed in simulation and the real world starts to be realized.
Creating high-fidelity simulation for Class 8 trucks requires additional nuances, such as the variable payloads in the trucks’ free-moving trailers, which all benefit from additional simulation. That’s why, in addition to scripting the situations Driver encounters in the real world, Waymo also models the various payloads its vehicles haul. Usefully, the testing is platform-agnostic – many of the functions and features designed for passenger vehicle simulation transfer over to the simulations that operate on Class 8 trucks, and vice versa.
Waymo notes that most simulation involves short targeted-scenario segments of real driving data to evaluate isolated events, like whether the autonomous driving technology successfully made an unprotected left turn. While this short-form simulation allows its engineers to optimize and refine isolated skills, such as nudging around a double-parked vehicle, it says it is important to evaluate how an autonomously driven system’s capabilities work together over full-length trips.
As Waymo’s simulation becomes more sophisticated, it states it is able to assess harder behaviors and aggregate important trip-level statistics about the Waymo Driver’s performance. The process is also cumulative, so for example, when the company introduced its fifth-generation hardware suite, SimulationCity allowed it to incorporate all of the learning from previous iterations.
Waymo also highlights that SimulationCity is especially beneficial as it expands its operating domains. For example, even if it hasn’t autonomously driven in a specific location, a map can be uploaded, a Waymo vehicle inserted into the scene, and its bank of data used to make an estimation of expected behavioral data.