In an exclusive interview with Autonomous Vehicle International, Dr Jewel Li, COO of AutoX, reveals how the company continues to accumulate thousands of miles of real-world data from its robotaxi operation in some of China’s most densely populated cities.
Backed by Chinese e-commerce giant Alibaba, as well as leading OEMs such as SAIC Motor and Dongfeng Motor, AutoX is seen by many as leading the race to deploy fully autonomous vehicle technology in China, with its efforts currently spearheaded by its several hundred-strong robotaxi operation spread across Shenzhen, Shanghai, Wuhan and other major cities.
The company’s ambition is to build a self-driving AI platform capable of handling the most challenging traffic scenarios in China, for use across various vehicle platforms, including robotaxis and robotrucks. Although it currently operates its own robotaxi fleet, its long-term goal is to develop and provide self-driving technology to vehicle manufacturers.
Headquartered in Shenzhen, where it was the first company to test AVs on Chinese public roads without the presence of safety drivers, it also has a US base in San Jose, and became only the third company (following Waymo and Nuro) to receive a driverless permit from the state of California, in July 2020.
The extremely rich and diverse data derived from such trials, and its commercial operation of ride-hailing robotaxi services in some of China’s most densely populated mega-cities, gives AutoX a clear advantage over its rivals, according to COO Dr Jewel Li.
“Not every mile is equal,” she says. “If you have 10 million miles on the highway, that’s completely different from 10 million miles from a city like Shenzhen, where you have far more objects to trace, as well as more difficult scenarios involving aggressive driving and four-way intersections with no traffic lights. We have worked out self-driving in the city is 40 times harder than autonomous driving in suburban areas. On average, there are about 300 to 400 vehicles, cyclists and pedestrians every single kilometer [in a city like Shenzhen], so it’s a very high volume,” notes Li.
“Globally, there are probably 10 or a dozen leading autonomous driving companies, with some focused on the China market, and some on the US,” she continues. “There are big differences between these two markets – the driving style, culture, landscape, legislation, regulation – everything is different. AutoX is one of the key players absolutely focused on China, and more generally, Asia. We are very advanced in terms of our technical capabilities, and the ecosystem that we have built for the market.”
A good indicator of just how advanced was provided by the authorities’ willingness to grant AutoX the first ever licence to operate a fully driverless ride-hailing service on public roads in China. As a result, the company began driverless operation in downtown Shenzhen in January 2021, after an initial trial, marking the first time a completely autonomous robotaxi fleet without safety drivers or remote operators had been deployed in China.
“We were the first ever company allowed to have a truly empty robotaxi running on the public road [in China],” says Li. “The car can have nobody inside at all – not in the passenger seat, in the driver seat, or in the co-driver seat.”
AutoX became only the second company in the world to operate such as service, following hot on the heels of US rival Waymo, which first offered safety-driver-free robotaxis to the public in Chandler, Arizona, in October 2020.
Competitive environment
That’s not to say that there aren’t other robotaxis already on the road in China. Baidu has been operating in Beijing since April 2021, however a safety employee must ride along in the passenger seat, with services limited to clearly defined zones and closed campuses. Similarly, WeRide’s service in Guangzhou also requires a safety driver in the vehicle.
Meanwhile, AutoX has since been joined by Pony.ai in running a fully driverless robotaxi operation (no safety driver) after it began offering driverless vehicles to ride-hailing customers in the Nansha district of Guangzhou, from June 2021.
Despite all the competition, Li continues to emphasise the richness of AutoX’s data as a key differentiator in developing the company’s AI Driver platform to ensure it can handle the most dynamic and challenging traffic conditions.
“We were the first to begin autonomous driving in a downtown environment in China, starting in Shenzhen at the end of 2017,” she says. “Everybody else was doing testing in suburban or rural areas, but we went directly for the heart of the city. We have done that from 2017, all the way to now,” Li continues, while pointing out its rivals only started entering urban CBDs after 2020.
Shenzhen also happens to be one of the most densely populated cities in China – hence the data gleaned so far is of exceptional value: “We call that the ‘density’ of the data – for every mile or kilometre, the number of vehicles, pedestrians, or bicycles or anything else that you see is far greater,” explains Li. “And it is not enough to just see them, the vehicle must also be able to interact with and negotiate with all these objects. This is valuable for not only perception, but also prediction and decision planning across all of the software stacks.”
All of these interactions are fed back to dedicated operations centers in the cities AutoX serves. When it first opened in April 2020, the company claimed its Shanghai Operations Center was the largest self-driving car data hub in Asia, let alone China, collecting ‘petabytes of real-world data’ from the city’s robotaxi fleet each week.
The 80,000ft2 facility houses simulation cloud computers running ever more varied and complex scenarios 24/7, as well as a dedicated car turntable surrounded by 360° of calibration panels to ensure sensor accuracy and performance.
AutoX has a similar center in Shenzhen, equipped with high-speed, fiber-optic broadband to ensure passenger support and seamless communication with its roadside assistance teams, as well as a back-up power system. “Our Shenzhen robotaxi operation is larger than Shanghai now,” says Li. “Operations assistants in each robotaxi operating zone are able to help passengers if needed.”
AI Driver
AutoX operates a mixed fleet comprising vehicles from Honda, Ford, Chrysler, Dongfeng, SAIC, BYD and Chery, all fitted with its ‘AI Driver’ autonomous driving platform.
“The majority of our robotaxis are installed with our new Gen5 [AI Driver] system,” confirms Li. “However, there is a small fleet in Shanghai installed with Gen3 – this is because Shanghai permits are related to each generation.”
The company launched its fifth-generation AI Driver system in July – a full-stack Level 4 autonomous driving platform combining both software and hardware, including what Li describes as a “self-developed” 2,200 TOPS capable ‘XCU’ vehicle control unit (featuring two NVIDIA Ampere GPUs that deliver 900 TOPS each) to crunch the data derived from some 50 sensors in total.
These include 28 cameras capable of capturing a total of 220 million pixels per second, six high-resolution lidar offering 15 million points per second, 4D radar from Arbe (see below) with 0.9° resolution, as well as GPS and IMU inputs
“We have lidar and camera fusion systems on four sides of the vehicle,” notes Li. “These are for the blind spots. We have six lidar in total – two on the top and one for each side. We also have over 20 purpose-built, high resolution cameras; and 4D radar for 360° coverage of the vehicle – many of our competitors are still using more traditional 3D radars.”
ON Semiconductor is providing AutoX with 28 AR0820AT 8MP image sensors and four SiPM lidar sensor arrays, to ensure the sensor fusion necessary for full surround view with zero blind spots.
The Arizona-based company claims its technology extends AI Driver’s practical sensing distance to beyond 300m to enable autonomy at freeway speeds, where objects or signs must be recognized farther away from the vehicle to enable sufficient reaction time.
Li says the company insists on making its own vehicle control units to ensure they not only satisfy the high compute required for L4 operation, but also the safety, durability, reliability and redundancy standards demanded by automotive applications: “I always say that the hardware is the bottom line – no matter how good or how advanced the software is, if you don’t have reliable hardware, then safety cannot be assured.”
On that note, AutoX’s vehicles also feature a fully redundant drive-by-wire system for extra reassurance: “We worked with FCA for over a year to ensure the on-vehicle AD system [AI Driver] is able to drive the vehicle when one drive-by-wire system is not functioning, and that it can also interact with the vehicle’s thermal management and battery systems.”
All the sensors and hardware also undergo extensive in-house testing, including being subjected to high-pressure water jets to simulate extreme weather, as well as humidity testing and de-fogging, and high-temperature endurance.
Testing environment
The emphasis on safety is hardly surprising, but Li says she never ceases to be amazed by what supposedly cautious human road users can get up to: “We’ve seen all kinds of weird stuff. If you test with hundreds of vehicles on the road every day, you get to see a lot. We’ve seen people suddenly running out of nowhere and doing martial-arts-style flips in the road, and we’ve seen pedestrians lying in the middle of the street for no reason. It’s this abnormal behaviour that is of most interest to us – anyone building an autonomous driving system must avoid making false assumptions. Sometimes it’s easy as a human and an engineer to make assumptions of what could happen on the road, but that would be very dangerous.”
The data captured from these more unusual events, as well as the day-to-day challenges of Chinese urban driving, help to sharpen AutoX’s simulation modeling. “It’s not the average case that we’re worried about – it’s the corner cases that you have to handle,” notes Li. “From the very beginning, to avoid making false assumptions, we wanted true downtown driving data. We’ve been to 13 cities globally, and it’s still going on, it’s still expanding. We need a variety of data to break our own assumptions; and we do tons of simulation using that data. We also acquire data from other vehicles [third-party sources] to mirror what happens in the real world, and we use randomness in our simulation to ensure the AI Driver platform has seen plenty of stuff before hitting the road.”
As for the future, where beyond an autonomous vehicle’s own sensing capabilities there is also the potential for its software stacks to receive data captured and sent via V2X technology, Li says there’s still a lot of work to be done.
“We welcome any technology that can potentially benefit the deployment of driverless vehicles, especially V2X, which could be another pair of eyes,” she says. “But there are certain pain points, currently – who is going to pay for it, and who is going to be responsible to maintain it? We do so much testing and we invest so much time and money to make sure our own systems are reliable. But what we see in the current space for V2X is that nobody is investing much energy and resource, and nobody is maintaining them, so they become an unreliable source of information. And once it’s unreliable, it’s better not to have it. We need to think about exactly where and how V2X will be of most benefit, so it can be selectively deployed, instead of just installing it at every single intersection across China, which is going to cost a huge amount of money.”
4D imaging radar for greater safety
AutoX is using a 4D imaging radar platform, supplied by Arbe, for its robotaxi vehicles, as well as other autonomous driving projects. Over the next five years, AutoX will integrate 400,000 Arbe-based ultra-high resolution radar systems into its L4 fleet. Multiple radar units feature as an integral component of the sensor suite fitted to AutoX’s robotaxi fleet, for safety application development, AI-based perception algorithms, and sensor fusion.
Arbe is providing AutoX with the entire Arbe Imaging Radar platform, which it says enables an image 100 times more detailed than any radar on the market, via a proprietary technology with 2K resolution – the highest channel count in the industry, according to the Tel Aviv-based company.
Arbe says its radar technology addresses the core issues that have caused recent autonomous vehicle and autopilot accidents such as detecting stationary objects, identifying vulnerable road users, and eliminating false alarms without radar ambiguities, while insisting its 4D imaging radar provides ultra-high resolution in any weather or lighting condition: “The Arbe-AutoX partnership will transform the level of safety for L4 vehicles, elevating the standards of safety and leading the path for the entire industry,” said Roman Levi, VP of sales and general manager of Arbe APAC.