How Safe Is Safe Enough?

That was the overarching question a group of 180 experts discussed last week at the ISO 26262 & SOTIF conference for four days during #FuSaWeek2023 in Berlin. “How Safe is Safe Enough” is also the title of Prof. Koopman’s book from September 2022. I mentioned him in my blog “Are We Too Hard On Artificial Intelligence For Autonomous Driving?” Prof. Koopman was referenced often in Berlin, and he will give the Thursday keynote titled “Defining Safety for Shared Human/Computer Driver Responsibility” next week at AutoSens Detroit.

Safety is a hotly debated item, and we discussed many related topics. How does security relate? How do the various standards overlap? How many of them can the automotive design chain feasibly support from an ROI perspective? Is there any hope for safe AI/ML? What is the power dynamic from OEMs through Tier 1 system providers, software developers, semiconductor vendors, and their IP providers?

Why are we doing all this?

The best vision of mobility and motivation for what conveniences the future may bring came from Continental AG’s Danila da Costa Ribeiro. I have stared at the below picture for future mobility a lot since last week and find new items every time. From autonomous drones delivering items via air and street, autonomous cars safely stopping for pedestrians to humans interacting with robots in buildings. The future that the team Continental imagines looks pretty convenient.

On a good day, that is.

A vision of future mobility. Source: Continental AG, https://bit.ly/40TLRXv

It is easy to imagine the flip side. Falling drones, out-of-control cars having to decide whom to hit. You get the gist.

So what does it take to make the autonomous future safe and secure?

Standards will play a key role. Gareth Price from Eatron Technologies talked about the pain points around standards. The cartoon below was on one of his slides next to some examples – ISO 26262 (Road vehicle, Functional Safety), SOTIF, UL4600 (Safety for the Evaluation of Autonomous Products), ISO TS 5083, ISO 21434 (Automotive Security), IEEE P2846, and 1234567890.

The last one is my favorite, clearly.

Source: XKCD.com, https://bit.ly/3Lu3GGT

To make my head spin even more, Riccardo Vincelli gave an excellent overview of the standards landscape, and his summary slide had eleven (11!) line entries with status updates. He added to the list above SAE J3016 / ISO/SAE PAS 22736 (Taxonomy for ADAS), ISO 21488 (Safety of the Intended Function), UN R157 (Automated Lane Keeping), ISO TS 5083 (Safety for ADAS), ISO 9839 (Predictive Maintenance), ISO/IEC TR5469 (Functional Safety and AI Systems) and ISO PAS 8800 (Safety and Artificial Intelligence).

That’s a lot of standards. They relate, interact, overlap, and depend on each other.

What is reasonable?

ISO 26262 defines “Functional Safety” as “the absence of unreasonable risk due to hazards caused by malfunctioning behavior of Electrical/Electronic systems.” Many discussions centered on what “reasonable” means here, and some participants linked it to “acceptable societal norms.” Well, not only do they change over time (hey, we could standardize how to measure them), but one immediately enters the area of consumer preferences and “acceptable behaviors.”

One of the last slides of the week reminded the audience to consider three things, all driven by what is considered “reasonable”:

• Think about the people you love
• Think about your legal protection
• Think about your profit

Recent studies in this area left me concerned. Last year, the Insurance Institute for Highway Safety / Highway Loss Data Institute surveyed 600 active users of partially automated cars. They found that many U.S. drivers treat partially automated cars as self-driving. 53% of Super Cruise, 42% of Autopilot, and 12% of ProPILOT Assist owners “said they were comfortable treating their vehicles as fully self-driving.” Ouch. Convenience seems to strike safety concerns. Adoption may be unreasonable, ahead of what the technology can do.

On the other hand, when optimizing for ROI, we as an industry need to make more than reasonable efforts to create better than “minimally viably safe and secure products.” With representatives from all areas of the design chain at the conference, this aspect became a central discussion point. What standards need to be supported and why? How are requirements communicated, traced, and checked? How can organizations embed design for safety and security into their development processes and achieve related culture changes?

These are only some questions that will keep us busy for quite some time.

What are the benefits?

While the safety and security discussions often center on technology and how it can fail – many presentations set up their motivation by mentioning disasters that happened – we often forget to take a step back and consider the balance with human driving behavior.

If aliens would arrive and assess how to reduce traffic-related deaths, they would most certainly take humans off the streets. Analysis of the National Motor Vehicle Crash Causation Survey, conducted by the National Highway Traffic Safety Administration (NHTSA), shows that driver error is a factor in 94% of crashes.

What’s next?

Advanced Driver Assistance Systems have the potential to improve roadway safety significantly. Autonomy adds much additional convenience. Still, safety and security must be key considerations during development, holistically. We must understand how system requirements at the vehicle level trickle down to semiconductor IP providers like Arm, Imagination, SiFive, and us at Arteris – all had representation at the ISO 26262 & SOTIF conference last week, and some of us will see each other again at AutoSens Detroit this coming week. Some pragmatism is needed here, too, and the industry needs to define the proper steps to take while maintaining the long-term vision that full autonomy promises. Arteris CEO Charlie Janac recently wrote about this in his article “How to Avoid Fall in Expectations for Automated Driving.”

Bottom line – as an industry, we will need more time to get safety and security right. When asked how we would rate the success rate of the ISO 26262 standardization in terms of acceptance and “penetration,” the interactive result came out at 3.4 out of 5. See below. Pretty good, but room to grow.

And, of course, let’s not forget all these other standards!

By the way, my compliments to Riccardo Vincelli from Renesas and Franck Galtie from NXP, who served as moderators throughout the week, managing to keep us mostly on time while always leaving room for discussions. It was one of the most interactive conferences I have ever attended. They leave some big shoes to fill for next year!