“Good morning, Pepper”
“Good morning, Lewis”
“What can you tell me today.”
“You slept for nine hours and two minutes last night and you got out of bed at 7:45 am.”
“Now, did I? What else? How are my vital signs?”
“Your heart rate is 70 beats per minute, your temperature is ninety-eight point six degrees, and you have walked 127 steps this morning.”
“Very good, Pepper.”
“I have an alert.”
“Yes, Pepper?”
“Your gait is irregular.”
“Oh dear. All right, I’ll get the walker. (pause) How is my gait now?"
“You gait is steady, Lewis. It’s time to take your medications."
“Alright, alright, I’m on my way to the kitchen now. But you know that. Thank you, Pepper.”
“You’re welcome, Lewis."
Dr. Arshia Khan, an associate professor of computer science, and Yumna Anwar, her graduate assistant, are busy programing their newest robot, Pepper, to become a personal caregiving assistant.
They are hoping to see conversations like the one with Dr. Lewis occur as Pepper helps individuals. Their work is centering around those with mild cognitive impairment, either in the early stages of Alzheimer’s disease or with vascular dementia.
The biggest difference between Pepper and a black box sitting on a counter recording data is its human-like behavior. It’s the emotional response from humans that Arshia and Yumna are most interested in.
Pepper, who is about 4-feet tall, uses cameras and sensors to detect a person’s facial expression, tone of voice, body movements, and gaze. Pepper can make eye contact, and can provide a sense of comfort, as if it’s watching over a person. “Pepper is designed specifically to make people happy, enhance lives, and facilitate relationships,” Arshia says.
In the first stage of dementia diagnosis, individuals experience cognitive impairment, but can often carry out their daily activities. “Robots like Pepper can assist those suffering from dementia by allowing them to maintain a quality of life and function independently,” says Arshia. A robot such as Pepper can learn a person's regular patterns and schedules and they can watch for danger.
Yumna, from Karachi, Pakistan, is in her second year of the computer science master’s degree program at UMD, and she’s a perfect fit for the research project with Pepper. Her thesis, which used a study of 44 human subjects on campus, investigates how emotional changes are reflected in physiological signals. She says, “After meeting Dr. Khan I didn’t consider working with anyone else.” Their interests in Pepper, especially the emotional response component, were in sync.
How Pepper Works
An individual wears a watch, a wireless device, that can monitor multiple inputs at once. Pepper records all the data from the watch and relays it to loved ones, a care facility, or call center. Pepper continually records the individual’s pulse and body temperature. It can go beyond an exercise tracker that monitors sleep and steps, and watch for falls, side to side gait, and balance indicators. I
t can record physical activity by classification, right down to sleep analysis, energy expenditure, and hand tremors. By monitoring posture transitions, Pepper take many actions, such as sending an alert for a caregiver to check for bed sores. Air temperature, inside and out, can be tracked and watched for conditions that would lead to freezing or overheating. And, in an area of great concern for loved ones, Pepper can watch for wandering.
Ashia has collaborated with many agencies on her research including the University of Minnesota Medical School - Duluth, St. Luke’s Medical Center, and the Veterans Administration hospitals. Her work with Pepper is part of a study with the Essentia Health System.
Outside Interests
Yumna and Arshia share more than work. They are both involved in the Muslim Student Association and actively participate with the Women in Computing (WIC) Club. The UMD WIC recently hosted an Association for Computing Machinery conference for MinneWIC for the Upper MidWest in Duluth.
Yumna says when she came to UMD she was surprised to receive so much encouragement and appreciation from professors. “It’s really helpful for students to maintain interest in what they’re doing,” says Yumna.
Arshia encouraged Yumna to continue her academic career and that encouragement has made all the difference. Yumna has been accepted at the University of Iowa in the Ph.D. program with full support.
About the Women in Computing Club
About the UMD Computer Science Department