The Challenge

The European New Car Advisory Program (Euro NCAP) is the auto industry regulator that awards “Safety Stars” for vehicle performance. For the first time, Euro NCAP will assess the performance of vehicle detection of driver drowsiness and distraction.

Given the wide variety of vehicles and sensors, and the limited time to study them, how can validation transparently, fairly, and effectively assess all the cars in a new, $3.2 billion industry?

My Contribution

I was the project team’s only continual member; all other participants were executives with shifting availability. Strategy was decided via group consultation and my duty was execution.

I handled almost all day-to-day project duties: attending international phone calls, designing research, analyzing data, writing the experimental protocols, etc.

The Result

After over 2 years of engagement, NCAP’s DMS protocol testing accounts for cell phone distraction; actual drowsiness (not acted); microsleeping; driver demographics (age, height, complexion, etc.); noise factors (e.g: lighting, head coverings, makeup, etc.); individual gaze differences (lizard vs. owl behavior); & both single-glance and multi-glance behavior.

These features will reduce worldwide road accident severity and ensure driver safety features are more than a checkbox.

The Methods

  • Literature Review: I identified the driver behaviors with the highest crash risk and reviewed existing distraction algorithms to find common features. This successfully persuaded NCAP of the necessity of thorough human-centered testing and an invitation to contribute to the testing protocol.
  • Illustration of Test Behaviors: Utilizing the philosophy of “Design for Extremes”, we listed driver testing behaviors all DMS systems should be able to detect (i.e.: failure to detect texting and driving is unacceptable, regardless of your technological approach).
    To alleviate concerns over repeatability and the required test time, I wrote a text-to-speech timing script, recorded myself performing all of the behaviors, and edited it into short animations. This showed that one participant can perform all behaviors in under 15 minutes.
  • Eye Closure Analysis: The common assumption for drowsiness is that it is a matter of eye closure. Reviewing our Guardian fleet monitoring database, I found that non-drowsy eye closures (e.g: squinting, laughing, etc.) account for ~90% of cases. If simple eye closure is used to measure drowsiness, the result is an unusably noisy user experience
  • Digital Survey of Phone Usage: For phone testing, we were unsure how drivers would hold their phones. I ran an international survey via Qualtrics and mTurk to find that drivers prefer to hold their phones on the passenger side, in the middle range of the wheel, and that it was not dependent on handedness nor the side of the road they drove on.
  • Assessment of Multiglance Standard: While single glance distraction’s risks are well-researched, time-sharing glance behavior can vary widely. I found a demonstrable increase in crash risk for on-road gaze proportions exceeding 1/3and demonstrated how the definition harmonized the three competing methods used in research literature.
  • Test Matrix Proposal: With the wide range of demographics, test behaviors, and noise factors, we needed to show that such a test was logistically possible. Collaborating with a company founder, we developed a study design framework that showed that an informative study was possible with 24 participants that could cover all behaviors and noise factors.