“The broad goal is to innovate the whole space of conversational technology,” said Dr. David Nahamoo, IBM fellow and speech chief technology officer at IBM Watson. “We’re creating a knowledge-rich system that humans can converse with. This is an exciting time for those involved. We are grateful to the Business Engagement Center for spearheading the conversations between IBM and the University of Michigan in order to bring this project to life.”

In an effort to help solve one of the grand challenges of artificial intelligence, the University of Michigan and IBM have launched a $4.5 million collaboration to develop a new class of conversational technologies that will enable people to interact more naturally and effectively with computers. In Project Sapphire, IBM and the U-M Artificial Intelligence Lab will develop a cognitive system that functions as an academic advisor for undergraduate computer science and engineering majors at the university. The system will allow researchers to explore how smart machines interact with people in goal-driven dialogues.

Satinder Singh Baveja, U-M professor of computer science and engineering and director of the Artificial Intelligence Lab draws a diagram to illustrate the Turing test, developed by Alan Turing in 1950 as a test of machine intelligence. For a computer to pass it, it must converse with a human interrogator so fluidly that the interrogator believes it is human. Also pictured are David Nahamoo, (center) chief speech technology officer in IBM Research's Watson Group and  Emily Mower Provost, U-M assistant professor of computer science and engineering.  University of Michigan faculty members are working with IBM to turn the company's Watson cognitive computing platform into a truly conversational system that could pass or redefine the Turing test.  University of Michigan North Campus.   Photo by Marcin Szczepanski/Senior Multimedia Producer, University of Michigan, College of Engineering

Satinder Singh Baveja, U-M professor of computer science and engineering and director of the Artificial Intelligence Lab draws a diagram to illustrate the Turing test, developed by Alan Turing in 1950 as a test of machine intelligence. For a computer to pass it, it must converse with a human interrogator so fluidly that the interrogator believes it is human. Also pictured are David Nahamoo, (center) chief speech technology officer in IBM Research’s Watson Group and Emily Mower Provost, U-M assistant professor of computer science and engineering. Photo by Marcin Szczepanski/Senior Multimedia Producer, University of Michigan, College of Engineering

To do this, the team will capture and annotate large volumes of approved recorded human-to-human conversations between undergraduates and their advisors on topics such as course selection, career advice, extracurricular recommendations and homework resources. The team will use these conversations to train the system on how to respond to interactions with students, and ultimately learn how to automatically navigate and successfully reply in conversations with those using the system. Satinder Singh Baveja (center), U-M professor of computer science and engineering, is leading a project with IBM to develop a conversational cognitive computing system that would initially serve as an automated advisor for a segment of undergrad students.“Human-to-machine interactions, similar to human-to-human conversations, are rarely confined to one question and one answer. They involve multiple turns of a conversation with responses that can be imprecise and unclear, making it difficult to simulate the human experience,” said David Nahamoo, IBM fellow and chief technologist for conversational systems, IBM Watson. “By partnering with the University of Michigan, we have an enormous opportunity to apply AI technologies in new ways and transform human-machine communication. “This collaboration marks the next chapter in a longstanding relationship between the university and IBM to place the power of cognitive technologies into the hands of the next generation of thinkers.” Unlike rules-based interactive virtual assistant systems that are programmed with hard-coded responses and scripted replies, Project Sapphire will apply probabilistic and statistical methods of reasoning that understand conditions and context, the researchers say. By employing deep learning, machine learning, reinforcement learning, natural language understanding, knowledge representation, emotion analysis and software technologies, the cognitive system will be trained using the recorded human-to-human conversations and will continue to learn and improve with increased interaction. “Natural conversations bring in so many different aspects of human intelligence—knowledge, context, goals and emotion, for instance. In many ways, to build a versatile conversational system is a grand challenge for artificial intelligence,” said Satinder Singh Baveja, professor of computer science and engineering and director of U-M’s AI Lab. “We look forward to taking it on with this partnership.” Through the partnership, eight computer science and engineering faculty members, along with graduate students and postdoctoral researchers, will work with IBM scientists. Within the next couple years, they expect their system to be up and running. The digital adviser isn’t meant to replace human professionals who guide students along the academic and emotional journey that is college. Once it’s developed, students can choose to talk to it for simple or routine questions, or to complement a meeting with a person. Students might tell the system their preferences and receive course recommendations that advance them toward their degree. They could define broad career goals and get a good list of electives. They could hear an estimate of how many homework hours their class load might require, or be directed to extracurriculars that might help them land the kind of job they’re after. The system would record the conversations and at any point, it could hand the session over to a human. One of the researchers involved—Emily Mower Provost, assistant professor of computer science and engineering, studies emotional cues. She’ll work on enabling Project Sapphire to recognize when students need a real shoulder, even if they’re not asking for one. Satinder Singh Baveja, U-M professor of CSE, illustrates the Turing test, the famous test of machine intelligence developed by Alan Turing in 1950. Also pictured are David Nahamoo, (center) IBM, and Emily Mower Provost, U-M CSE.The automated advisor is one application of what the team envisions to be a platform technology. Project Sapphire’s resulting innovations could be embedded into cognitive systems across many industries to improve how they learn and codify human expertise, understand a user’s intent and context, and deliver appropriate responses that direct conversations towards a stated goal. “What we are building has the potential to revolutionize how we interact with our computers and other devices such as our cars and our appliances,” Baveja said. “These conversational systems become cognitive advisors that can assist us in a variety of personal, professional, and enterprise tasks such as advising for personal finance, helping employees in scheduling meetings and travel arrangement, and providing technical support to customers of an enterprise.” The U-M students on the project are excited to work with an industry leader. “I want to build something that will not only benefit students, but will also benefit their children,” said Ananda Narayan, a U-M doctoral student studying reinforcement learning. “This is the future and it is happening here at the University of Michigan.” Project Sapphire is another step forward in IBM’s history of innovation in conversational systems. Most recently, IBM added its latest dialog management capabilities to its Watson platform, which is accessible to a community of 77,000 start ups and innovators in established businesses who are tapping these capabilities in the form of API services to pilot, test and deploy new products and applications that can converse with users. U-M and IBM have a history of collaboration. IBM has assisted with cognitive computing courses, worked with the university to discover new ways to apply Watson, and enabled the U-M Solar Car Team to use cognitive solar forecasting technology to predict solar radiation and cloud movement during the 2015 World Solar Challenge.