New procedures and technology to better assess ADAS and ADS are being explored by Euro NCAP and NHTSA in a bid to ensure test protocols keep up with today’s safety systems, discovers Trinity Francis, in an exclusive feature first published in the September 2024 issue of ADAS & Autonomous Vehicle International magazine.
In the 10 years since Euro NCAP introduced autonomous emergency braking (AEB), lane-keeping assist and lane-departure warning testing, ADAS has become commonplace in modern cars. As the technology has evolved, Euro NCAP has adapted its framework to include separate vulnerable road user (VRU) and safety-assist scores within its five-star rating.
Euro NCAP testing protocols are now set to be overhauled every three years to keep up with advancing technologies and safety standards. The next framework will be introduced in 2026, with future ratings focusing on four distinct phases of an accident: safe driving, crash avoidance, crash protection and post-crash safety. However, although the uptake of ADAS solutions is a positive move for road safety, it has increased the amount of testing required.
Adriano Palao Bernal, ADAS technical manager at Euro NCAP, says, “It’s impossible to cover everything. Our tests can barely scratch the surface of the number of situations you can encounter in the real world.” To find the right balance, Euro NCAP is exploring real-world testing, variations on existing tests and the implementation of new technologies.
Types of testing
Traditionally, ADAS testing has employed some manual methods that rely on interpretation by engineers or human inputs. This can be time-consuming and difficult to scale reliably. Equipment such as steering and pedal robots has remedied some of these repeatability issues, but Mario Torres, CEO of Ivex, argues, “The current limitation is that many OEMs rely too much on test tracks and manual methods. I believe there should be much more of a data-driven approach, where you collect all the data that you can, then derive your analysis from this data.”
Euro NCAP is focused on increasing real-world testing to make its programs representative of what a driver would typically experience on the road. Ivex has been contracted to work in collaboration with the safety organization to develop real-world test protocols.
Torres says, “We’re helping Euro NCAP to include more representative testing by driving 2,000-5,000km on a route that OEMs don’t know about, in different countries. Our software will track everything the car sees every 100ms.”
The test vehicle will be fitted with sensors, which include, at a minimum, a camera pointed at the dash, a contextual camera outside of the vehicle, GPS and an accelerometer. Lidar sensors can also be added. Ivex collates the data from these sensors and processes it using AI and machine learning to produce results categorized by scenario.
“There are two ways to look at the data that’s collected. One way is a closed box and the other is an open box. When we talk about testing and validation, it’s much better to look at the vehicle as a closed box,” Torres says. In this sense, Ivex does not want to rely on the car reporting what it has done. Instead, external testing equipment is used to independently verify what happened.
Currently, real-world testing is restricted to evaluating speed-limit information. With much longer routes, the vehicle is likely to encounter many different scenarios that can test multiple ADAS features. Real-world testing provides a platform for more complex encounters with multiple vehicles or VRUs that would be challenging to recreate in a controlled track environment.
However, Bernal stresses, “We’re not going to provoke tough safety situations; we will still reserve them for proving grounds.”
Dr James Buck, an ADAS test technical specialist at Horiba Mira, agrees that there will still be a requirement for controlled tests in addition to real-world evaluation. “The operator might drive tens of thousands of miles but they might only encounter 80% of situations that we want to test. What about the other 20%?”
Horiba Mira is a fully accredited Euro NCAP laboratory for active safety, and partially accredited for passive safety testing. The company is also working with Polestar, IPG Automotive, Connected Places Catapult and Coventry University on a project called CERTUS. The collaboration was set up to reduce testing costs for automated driving systems by 40%, using technology to make testing quicker and more efficient.
“The idea behind CERTUS is to develop a toolchain that can take that real-world data and work out what the gaps are; then you can test those in the test environment,” notes Buck. Hence, testing appears to be trending toward a multipronged approach that still relies heavily on physical testing but can be separated between controlled testing and real-world driving.
The next generation of Euro NCAP tests are also set to become less prescriptive. As the number of scenarios and the abilities of ADAS increase, the organization will adopt a different standpoint. “We want to change our approach and talk about ranges,” Bernal explains. “We want to ask the OEMs what their systems are capable of doing within a range. We want to see that their systems are functional in a range of speeds, and we will reserve the right to test randomly selected cases.”
According to Buck, these ranges will extend beyond just testing at different speeds: “We’ve done some research where we’ve taken conventional test scenarios and done what’s known as fuzzing. The basic concept is taking the protocols and making relatively small changes to understand what can make a difference and what doesn’t.”
This could be as simple as adding different clothing to a VRU soft target, such as a high-visibility jacket or a long, flowing skirt.
Horiba Mira is developing computer models that use data analytics in real time to create new tests. “A search- based optimization program can do some evaluation and suggest a test to do next that might give you a more interesting result. It uses statistical analysis to point you in the right direction as to what parameters to tweak, so you can quickly home in on interesting points,” says Buck.
Automated driving systems
Employing more software-based solutions for ADAS testing is likely to accelerate future testing programs for automated driving systems (ADS). In the short term, defining the circumstances under which these features can function will inform testing requirements.
DRI was recently awarded five NHTSA vehicle safety research contracts, with the aim of advancing the safe deployment of ADAS and automated driving technologies. Nadine Wong, the company’s director of track testing, says, “There may be some difference in scenarios from ADAS to ADS testing because currently the autonomous vehicles that are available have a very limited operational domain.”
In this sense, ADS testing will be influenced by regulations around where these vehicles are permitted to operate, initially narrowing the focus of test protocols. However, Torres argues that the testing burden increases as the vehicle takes on more responsibility. “I would expect to see much more data from OEMs making ADS,” he says. “When you have a driver, you can say very clearly where the responsibility of the driver is and where the responsibility of the OEM starts. When you have ADS, you just see vehicle behavior.”
This is where Wong, Torres, Bernal and Buck agree that simulation will need to play a bigger role so testing can be conducted in a thorough and timely manner. However, Buck notes, “The problem with simulation is that the only people who can do in-depth simulation are the vehicle manufacturers and the Tier 1s, because they are the people who have access to all the code and the brains of how it works. While that’s still very valuable, it can never, by definition, in my view, be truly independent. If you want to be entirely independent, you must have some degree of physical testing.”
Like with ADAS, a mixed approach is expected to be the best solution for ADS testing, but the proportion of physical testing compared with simulated testing remains to be seen.
Buck concludes, “As we transition from ADAS to ADS, we need to make sure we have robust validation and verification strategies to give consumers confidence in the systems. The bottom line is, if we can’t test them, they will never be on the roads.”
HGV testing
Euro NCAP is expanding its usual testing remit to include trucks. HGV testing is expected to begin later this year, with the first results published in early 2025. Testing is set to focus on AEB, lane-keeping support, driver monitoring, speed assistance, rescue information and vision. Trucks will be rated depending on their suitability for the application they are designed for. If a vehicle is intended to operate in a city environment, it needs to perform highly on direct vision, nearside-turn AEB and truck-to-VRU AEB. Trucks that are designed for highway operation will need to show good lane-keeping functionality and truck-to-car AEB.
Horiba Mira is involved in the development of new HGV testing protocols. The company’s Dr James Buck explains that there are similarities to car testing but equipment limitations must be overcome to achieve accurate results. “There are some physical differences. For example, we don’t want to drive trucks over our flying-carpet platforms for soft targets because these vehicles are far heavier. AB Dynamics is developing truck panels that can be driven over. Because the ADAS landscape is evolving so rapidly, equipment is constantly changing.”
Alongside a requirement for sturdier test equipment, measuring instruments must be fitted in different places to achieve high levels of accuracy. “An inertial measurement unit that sits inside of a car can tell us where it is to within 2cm of accuracy. If you put that in an HGV cab, when the vehicle brakes, the cab moves forward,” Dr Buck explains. Hence, the process of developing robust test protocols for HGVs is more complex than merely scaling up existing methods.
Like Euro NCAP’s car tests, HGV testing will be overhauled every three years. From 2027, test protocols are expected to include truck-to-VRU reversing AEB and direct occupant status monitoring to detect signs of driver fatigue or distraction. This year, driver monitoring tests will be limited to seatbelt use and indirect driver monitoring such as journey duration, driver inputs and vehicle control.
This feature was first published in the September 2024 issue of ADAS & Autonomous Vehicle International magazine – you can find our full magazine archive, here. All magazines are free to read. Read the article in full in its original format here. Never miss an issue: subscribe now!
