Exploring Autonomy:

Intelligent Health Assistant for Older Adults

From August 2018 to December 2019, I worked under assistant professor Aqueasha Martin-Hammond in her research lab as a Research Assistant. The project was designing a voice user interface to be used in smart speakers such as Alexa or Google to help users over the age of 50 manage their health and wellness goals. Studies have found that an increasing amount of older adults are living by themselves thanks to smart technology, giving them more independence and comfort. However, no store-bought devices have advanced health voice user interfaces (VUIs) that can assist the elderly. The goal of the project was to design a health-based voice user interface to assist older adults with basic and advanced health needs. My task was to identify and discover features that would help users unfamiliar with a smart speaker device.

Role : Undergraduate Research Assistant

Task : Transcriptions, thematic analysis, competitive research, interviews, and design.

Goal : Design a health-based VUIs to assist older adults with improved lifestyles and independent living.

Team : Aqueasha Martin-Hammond, Karen Bonilla, Naomi Khenglawt, Phani Nallam

Publication: Link | Download

Data Analysis:

Before I arrived, The team had conducted 10 semi-structured interviews alongside Wizard-of-Oz user testing in order to perceive the user’s current knowledge of VUI technology and observe the user’s interactions with the device. The team constructed a prototype using cardboard and a speaker, and used 2 medicine-based scenarios to mimic a every day interaction with the device. All participants were between the ages of 60 and 70, and 7 of them used the Internet to find health information outside of a doctor’s office.

The first task I was given was transcribing recordings of the interviews. Once transcribed, we organized the questions and answers into an Excel Sheet to analyze themes and patterns individually. For me, I focused on highlighting keywords, actions, descriptions, and positive and negative comments. I wanted to find what the participants thought of the idea of a health-focused voice assistant, and how beneficial it would be to their lives. When we were all done with the analysis, we created an affinity diagram of the different themes and ideas we found. The diagramming took a few sessions. This helped us find the story and data behind the interviews.

From our analysis, we found that participants had a lot of expectations for the device. One segment wanted it to be integrated into their lives completely; games, music, lifestyle, and the medical assistant. Another segment wanted the device to know as little as possible, only waking when called upon and only for knowledge about basic medical information and symptoms.

We also saw another theme emerge with data management preferences, where 6 of the 10 participants were found with inputting personal information into the device themselves or with help from a family member. However, 4 of the participants were apathetic about their information, and would be fine with a third party managing everything for them.

We identified this as an issue because older adults are increasingly vulnerable to information security issues, and while a trusted third-party would be ideal, having no idea where their information was going put them at a higher risk for identity theft and could potentially damage their trust with technology.

Re-aligning Goals & Interviews:

We re-aligned our focus and continued the project with new goals:

Design a VUI that induced less apathy from users and promoted transparency,
Discover how technology is accessible to seniors in an everyday settings
Craft the design to be inclusive
Create a voice search system to better understand the user’s voice inquiries regarding medicine

We brought two more people on the team temporarily to help because two of our team members had graduated. These team members would help with our security goals, develop a VUI for understanding various medical terms, and develop a prototype with raspberry pi. I was in charge of leading the research on privacy and security issues with another undergraduate, Karen. The one theme I noticed from our affinity diagramming was a lack of understanding of what security and privacy settings were. Words like “hacked” were used frequently, and while preventing hacking issues were important, there wasn’t a lot of clarity from the participants on their definition of “hacked”, how they were protecting their information, and what kind of protection they were looking for. My colleagues and I brainstormed ways to clearly and transparently disclose privacy settings to voice assistant users.

To help us, we researched current security and privacy terms and conditions on Google and Amazon’s websites to better understand how the user’s information is protected. We found that the pages do a great job explaining how their security works and how private data is stored as shared, but it took digging from both of us to find; there was no way one of our users would find the information. We also found that the jargon, while easy for us, might be complicated for others. We considered this as we put together a script for the device.

Solution A was to have the assistant clearly state when and how the information would be used. So for instance, if a user asked the device to tell them their blood pressure readings last month, the device would have to let the user know it was using personal data from a database for the results. Users could choose to turn this feature on or off, depending on the type of data (ex: medical history, demographic). This was to be a hands-on and direct way of giving data control to the users. But this method could be seen as intrusive and confusing if the user is already fearful of data security.

Solution B was to have informational descriptions programmed into the device. So if a user wanted to know more about where their information was going, they could ask. This was a less intrusive idea aimed at giving the user the option to learn in-depth about their data. The issue with it was that because the user would have to ask to get the knowledge, most would not know that it was accessible.

We chose to go with Solution B because of the intrusive factor that came with the first. To bring attention to the access of knowledge, we included key phrases that would be added on to answers in order to help discovery.

That summer we also conducted a second round of interviews with the new goals of discovering how users feel about having devices in public settings vs. non-public settings, how often they will use the device and privacy concerns. There was no prototype this time, and we were working with 15 participants to interview, all from the same senior center in Indianapolis and over the age of 50.

We ran into a snag right at the beginning of the interviews; the sign-up sheet for participants did not mention the requirements of having experience using voice devices. More than half of our participants never even used the device! To fix the issue, we interviewed the participants who did have a solid voice device experience, including experience with Siri. We also extended the time frame of the interviews, delaying our goals to be pushed back to the fall once everyone returns from summer break.

Second Round of Research:

When we returned from summer break, we dove right into transcribing the interviews. Karen and I also continued with wrapping up our security design research by creating a paper storyboard and interactive prototype following the process of a user learning security and privacy measure. We used the information we learned from our research to create both artifacts and presented our designs to the team.

From here on out, I handed off the finishing touches of the project to Karen to take lead as I focused on two other projects, one of which is the content analysis for Health Freedom Inc.

Paper Submission:

While we worked on our analysis, we were also writing research papers to submit to three conferences: ACM CHI, ACM DIS, and ACM IUI. The paper I worked on with Aqueasha was accepted and we went to ACM IUI 2019 to present our findings in a paper/poster presentation.

This was such an exciting time for me because it was my first academic conference! And I was PRESENTING! I talked with a lot of my peers, and was a bit shocked to learn I was one of a handful of undergrads also presenting. It gave me a lot of confidence in the work I was doing and that I was headed in the right direction.

Lessons Learned:

Looking back, while I conducted various levels of research and data analysis, I would have liked to experience a little more in design. Much of the design work was given to the graduate students on the team, which was understandable considering their seniority. For future work, I’ll be more proactive with content development and assisting with timed goals and deadlines. But this project taught me more about the research process than any textbook could.

If you want to learn more about where this project is now, contact Aqueasha Martin-Hammond at aqumarti @ iupui . edu (just tell her I sent you~).