Fatal Tesla Crash: That’s Not All, Folks

EE Times; Junko Yoshida, Chief International Correspondent

The NTSB’s report contains a few surprises, including how and where Tesla’s captured data is stored, routed and saved inside a vehicle.

The National Transportation Safety Board (NTSB) last week released, after more than a year of suspense, a 500-page document or “docket,” about a fatal highway crash involving a Tesla S and a tractor-semitrailer truck.

For an automotive industry now fixated on the development of self-driving cars — among which the Tesla S is one of the grand experiments — the NTSB probe is a treasure trove of data. However, it falls short of determining who, or what, is to blame for the death of the driver.

The NTSB docket includes reports on highway design, vehicle performance, human performance, and motor carrier factors. The crash reconstruction report, included in the docket, describes the crash sequence, along with interview transcripts and summaries, photographs, and other investigative details.

The NTSB emphasized that the docket contains “only factual information collected by NTSB investigators” and it “does not provide analysis.” It said no conclusions about how or why the crash occurred should be drawn from the docket. The NTSB is planning to release its own analysis, findings, recommendations, and a judgment about probable cause “at a later date.”

Using system performance data downloaded from the passenger car, the report states that its speed just prior to impact with the trailer was 74 mph. The data also revealed that the driver was operating the car using automated vehicle control systems, a Traffic-Aware Cruise Control (TACC) and the Autosteer lane-keeping system.

Driver Assistance System
Among the documents released last week, the one entitled “Driver Assistance System” poses particular interest for EE Times readers.

The report delves into details of how Tesla’s driver assistance system — consisting of a Bosch radar system, Mobileye image capture & processing system, an ultrasonic sensor system and gateway electronic control unit — works. It literally reads like a teardown of the Model S driver-assistance system.

It also contains a few surprises, including how and where Tesla’s captured data is stored, routed and saved inside a vehicle, and how it’s sent to Tesla’s server using a virtual private network connection established via Wi-Fi, or using the vehicle’s 3G cellular data capability.

The report says that Tesla S stores non-geo-located data in-vehicle in non-volatile memory using a removable SD card installed within the Gateway ECU.

Really, an SD card? What role does the SD card play?

No Event Data Recorder?
After reading the report, Mike Demler, a senior analyst at The Linley Group, told EE Times, “I find the description of some of Tesla’s control and data-recording systems to be interesting.” In particular, he said, “The statement in the report that says, ‘This SD card is large enough to typically maintain a complete record of all stored data for the lifetime of the vehicle’ is interesting.” He asked: “How could they determine how much data will be generated over the lifetime of the vehicle?”

Unfortunately, the NTSB report doesn’t answer such questions.

But one thing is clear. The NTSB apparently sees this “removable” SD card as a proxy for an event data recorder (EDR). Because current NTSB specs do not require an EDR (it’s completely voluntary), the NTSB appears to conclude that Tesla did enough.

Danny Kim, director and partner of Vision Systems Intelligence (VSI), told EE Times, “The Tesla Model S involved in this crash did not, nor was it required to by regulation, contain an event data recorder.” Even had it included an EDR, he noted that “The current EDR, outlined by the regulation, is outdated and it does not reflect what autonomous vehicles can do.”

Kim said, however, that sooner or later there will probably be a new regulation including an EDR mandate, per NHTSA’s recommendations last fall.

To Tesla’s credit, when NHTSA sent Tesla an extensive list of questions, the automaker was able to offer much more than what a current EDR can provide, Kim observed. “The most notable points include a detailed list of all of Tesla’s Autopilot capable vehicles including the VIN, model, model year, total mileage with Autosteer on, total number of ‘Hands on Wheel’ Autosteer warnings records, etc.”

Hands on Wheel
Now, of course, the most contentious issue that emerged from the fatal crash last year was if the driver kept his hands on the wheel while using the autopilot mode.

Based on recovered data, the NTSB confirmed that during a 37-minute period when the driver was required to have his hands on the wheel, he did so for just 25 seconds.

The report said Autopilot was activated through most of his trip. It gave the driver visual warnings seven separate times that said “Hands Required Not Detected.”

Six times, the system sounded a chime along with the “Hands Required” warning.

Phil Magney, founder and principal at VSI, told EE Times, “Tesla Auto Pilot monitors hands on state continuously. If the driver is in active autopilot mode and does not have his/her hands on the wheel the vehicle will chime and flash to gain the attention of the driver. After repeated warnings, if the driver refuses to grab the wheel the system will disable itself for the duration of the drive cycle.”

The problem, as Magney acknowledged, is that “It is easy for the Tesla driver to fool the system by casually resting their hands on the wheel.”

Magney noted, “For Tesla Auto Pilot systems (AP 1 and AP 2) the primary method to measure driver engagement is through the steering wheel sensors.  We don’t think that is an adequate means of measuring driver attentiveness. And it is for this reason that GM (SuperCruise) is using driver monitoring ‘facial recognition’ via camera to determine if the driver is awake and attentive.”

Magney explained that the GM system does not require hands on the wheel. This is why GM claims to be the first “hands-free” automated solution. “In my opinion, the GM solution is a better method to handle the task of measuring driver engagement,” Magney  said.

Strategy Analytics analyst Angelos Lakrintis also told us after reading the report, “It genuinely sounds like the autopilot failed to prevent a collision. Whether right or wrong, we would expect it to work before it is broadly deployed as Autopilot, DrivePilot, ProPilot etc.”

Linley Group’s Demler concluded, “In any event, this reinforces the thinking that Level 3 (although Tesla describes Autopilot as Level 2) ADAS is fraught with danger, since it relies on handover to human drivers.”

Tesla last fall unveiled improvements in Autopilot, adding new limits on hands-off driving and other features that likely would have prevented the crash death. The updated system temporarily prevents drivers from using the system if they do not respond to audible warnings to take back control. But the system still depends on steering wheel sensors to judge if a driver is paying attention to the road.

Data flow and data storage
The NTSB’s report explains how Tesla stores its captured data.

The report noted, “As part of Forward Collision Warning (FCW) and Automatic Emergency Braking (AEB) event data, the vehicle supports the acquisition and storage of image data captured by the forward facing camera. This system can buffer 8 frames of image data sampled at one second intervals centered upon a triggering FCW/AEB event. These frames are initially captured in volatile memory in the Camera ECU and then stored in non-volatile memory at the end of the drive. From there it is transferred via CAN bus to the Gateway ECU where it is stored on the internal SD card.” Then, “data from the SD card is episodically data-linked to Tesla servers,” the report says.

Although the data flow described in the report might be a revelation to some, it goes no further about the types of memory used.

VSI, which did a deep dive into Tesla’s Model S a few months ago, shared a few more details and educated guesses.

According to VSI’s Kim, Nvidia’s Tegra SoC, which is part of the MCU motherboard, features volatile memory SK Hynix DDR3 SDRAM (4Gbytes – there are four 4GB slots) and 8GB flash memory. The Nvidia PCB also has on the board “connectivity modules, microcontroller, FPGA, Ethernet Switch, and the 4GB SD memory (for EDR) along with others,” he explained.

Kim said VSI believes that inside the vehicle all data is recorded in real-time. VSI speculates that real-time recording would be writing to the 8GB onboard flash drive (not the removable SD card) on the Nvidia Tegra SOC because it has the fastest write speed. Once data is saved on the 8GB flash memory, perhaps after extra processing, VSI suspects that “some of that data is copied to the SD card for long term storage, and to be more easily accessed.”

VSI also believes that “perhaps not all of the raw data is copied to the SD card, but perhaps only certain data points the system deems important.” For example, such data could be certain key time interval, or certain data elements like timestamps, vehicle speed data, Auto Pilot engagement, and actuation data (steering angle, brakes). Excluded would be larger sensor data (camera frames and radar). This would allow a Tesla engineer to be able to quickly see the most important information and a summary of events, and if necessary know where to look for more raw data.

What recovered data did not show
While industry analysts were most curious about what exactly Tesla’s recovered data revealed about the crash, they were disappointed that the NTSB report was not able to find what the camera saw.

The report said, “Image data from the vehicle’s camera was recovered from the SD card. By design, this data does not contain any timestamp information. The recovered image data did not contain information consistent with the crash.” The report speculates that “these images were likely triggered by an earlier FCW/AEB event and written to the SD card prior to the last trip.”

The Linley Group’s Demler observed, “It is very curious that they couldn’t recover any images from the Mobileye camera.”

Tesla’s OTA capabilities
Given Tesla’s ability to link up with its servers and to store data in the car that expands the information contained on the servers, VSI’s Magney was most impressed with Tesla’s Over-the-Air (OTA) capabilities. He noted, “Based on our examinations, it is my opinion that the Tesla vehicle architecture is a proxy for future vehicle platforms.” He noted, “Albeit Tesla is a maverick in this space, their OTA architecture plus event handling and data recording is vital for proper Autonomous Vehicle management.”

Magney added, “Frankly, I am little bit surprised other OEMs are nowhere near as advanced as Tesla in terms of their ability to manage their vehicle remotely. This industry has been talking about this for years but only Tesla manages their vehicles as a proper digital device.”

— Junko Yoshida, Chief International Correspondent, EE Times