Len Klie

Nuance and DFKI to Work on Conversational AI for Autonomous Driving

Nuance Communications is expanding its partnership with Deutsches Forschungszentrum für Künstliche Intelligenz (DFKI ), the German research center for artificial intelligence (AI), to further advance cognitive and conversational AI innovation across several joint research initiatives around in-car systems, healthcare systems and omnichannel customer care.

Nuance and DFKI have already begun work on conversational and cognitive AI for self-driving cars, where automotive assistants in these environments must be intelligent and collaborative to effectively engage passengers when it’s their turn to drive.

To better understand the many scenarios where transfer of control will need to happen and how, Nuance and DFKI conducted a usability study to identify the most effective ways to grab someone’s attention. Study participants were placed in a simulated autonomous car and asked to do several activities, such as reading, listening to music, composing an email, and watching a movie.  While they were engaged in these activities, the autonomous system would alert them through vibration (haptic), visual, and auditory cues to see which senses responded the fastest to take the wheel.  The study included a variety of scenarios, such as inclement weather conditions, system diagnostic warnings, sensor defects, and traffic jams.

The study found the following:

  • Drivers prefered integrated, multimodal user interfaces leveraging voice, touch, and visual cues.
  • Drivers weren’t as responsive or pleased with notifications using the same modality as their current activity and would rather a combination of alerts adapted to their current activity.
  • Independent of the current driver activity, sound is considered more pleasant and effective than visual cues, leading to faster reactions than simply vibrations or haptic alerts.
  • Drivers trust audible and haptic responses from the automotive assistant more than visual cues alone.

“Our partnership with DFKI is focused on advancing the state of the art of AI solutions for physicians, healthcare organizations, automakers, and enterprises,” said Vlad Sejnoha, chief technology officer at Nuance, in a statement. “Nuance and DFKI share the vision that advanced personal assistants and cognitive technologies can effectively amplify human intelligence, and in the process, transform patient and customer care, and make our experience with smart devices and the connected car vastly more productive and rewarding.”

“Cognitive and conversational AI are the key technologies driving the second wave of digitalization that is based on deep machine understanding of digital data. Partnering with Nuance allows us to put innovation into action, taking our research and directly applying it to the systems that people use every day and addressing the real-world challenges and complexities of connected smart service platforms.  With a Nuance office at the DFKI Campus in Saarbruecken, where research groups on autonomous driving, deep learning, multimodal dialogue, and language understanding are located, we will continue to push boundaries on the conversation between humans and smart environments, and ultimately bring to market the next generation of cars, bots, assistants, and smart objects that simply make everyday life better and safer,” said Wolfgang Wahlster, CEO of DFKI, in a statement.

The partnership between Nuance and DFKI is also advancing NLU’s mapping of words to meaning to further develop conversational and cognitive AI interfaces between people and virtual assistants across a number of vertical markets, including healthcare and enterprise omnichannel customer care.   The DFKI team will work to localize Deep NLU into German.