Except there's one thing: Ellie is not a person. She is a virtual character created by the Institute of Creative Technologies (ICT) through the University of Southern California (USC). Using a Microsoft Kinect, a video camera, and a microphone, Ellie is able to pick up activity levels and stress responses through body language, facial expressions, and voice cues, and is able to adjust questions accordingly. Ellie’s goal is to track potential signs of depression, which can be a symptom of a variety of mental disorders – something that the system is able to catch 90% of the time.
But Ellie comes with a lot of questions, both ethical and practical.
“A lot of what I do in my work is [address] why would you want a virtual person,” said Jonathan Gratch, a research associate professor of computer science and psychology at USC and associate director of virtual human research at ITC. “What are its advantages and what does it need to do?”
Funded through the Defense Advanced Research Projects Agency through the Department of Defense, ICT’s mission as a research institution has been to create virtual training software to be used for a variety of military requirements, such as interviewing and negotiating, before putting soldiers with actual people. Ellie was one of the developments that came from the initiative, but her mission would be very different. Instead of training, the focus would be on diagnosis and treatment.
The machines that monitor nonverbal communication are paired with some of the technology that was prevalent in ELIZA, or a chatbot from the 1960s that looked for keywords when talking to patients to map out a response. Gratch, who is overseeing the data collection and evaluation aspects of Ellie, said that the project is one step up from that, using certain behaviors and vocal cues to determine the next question. Unlike ELIZA, Ellie is programmed to have clinical protocol and not ask as many open-ended questions. The nonverbal element is also vital, as a person looking away or a pitch contour can often say more than just the words themselves.
Ellie was based off of an experienced clinician, which helped create a lot of the mannerisms that the program uses. There is backtracking, or sounds of acknowledgment, that a normal therapist would make to help connect. Gratch says that the rapport that Ellie builds, such as leaning in when a person is trying to lean back or smiling or nodding at key points, helps the patient feel more connected and willing to divulge information.
Ellie, being a virtual program, may have certain advantages that a human may not have in tracking conditions. Whereas a patient’s facial expressions and body language may be forgotten by a human psychotherapist, Ellie is capturing everything and analyzing it, comparing it to other patients and producing data. Also, a patient may be more comfortable disclosing behaviors to a computer program that they may not want to tell to an actual person but is vital information, such as risky sex and illegal drugs. It could be one variation of telemedicine.
However, the idea that Ellie would replace an actual human psychotherapist is a far-off concept, according to Gratch. He says the technology hasn’t advanced to that point, and even if it was, it leads to a variety of questions. These questions include whether using Ellie over a long period is the best use of resources or if it’s ethical to keep using her over an extended period. Rather, Gratch sees Ellie more as an augmentation of what people can do to treat patients in mental healthcare.
“It’s a better version of a self-help book, which is one way that people who are resistant to care seek care,” he says. He adds that he feels that it is bridging a gap that may exist in human care, not replacing it. And Gratch adds that Ellie is not able to give an exact diagnosis; rather, it’s a determination. He compares it to tracking a fever, but not understanding the cause of it. Depression, like a fever, can be co-present with many different mental conditions.
Gratch sees potential for Ellie beyond depression, such as helping in autism research, social disorders, and elder care. Also, he points out there is interest in seeing how a virtual character like Ellie can detect pain.
But until then, there is still data that needs to be collected. Gratch is hoping to put their technology in a kiosk in the Veterans Administration in order to collect in the field. Also, an independent group at the University of California, Los Angeles, will be doing individual research to help improve Ellie. And the program is still working to develop other characters similar to Ellie, in a different gender as well as various races, who could help make patients more comfortable. The hope is to eventually use this technology to help soldiers and track behavior changes pre- and post-deployment.
Reina V. Slutske is the assistant editor for MD+DI.