Summary: In this evaluative study, we designed a new prototype system for managing personal data and tested it with 18 participants, using interviews, a set of scenarios, and two card sorting exercises. Most participants had positive reactions to the prototype but we also identified some usability issues and aspects to improve.
My role as a researcher
- I was the main researcher leading the project end-to-end.
- I planned and led the user evaluation with 18 participants: wrote the study plan and script, coordinated recruitment, conducted most of the interviews, analyzed the data, wrote the usability report and research paper with results from the study.
Team: I worked with Janet Chen, an undergraduate research assistant that I directly supervised + Dr. William Odom and Dr. Joanna McGrenere, professors in HCI.
The problem
People struggle with managing their personal data across devices and platforms. They also have different preferences for how to do it.
Most people who use technology have to manage a growing amount of personal data, from photos, documents, and files to mobile applications and location history data. Often, personal data is spread across many different platforms and devices.
Managing data becomes a challenging task because it’s hard to know what data is stored where and how to best organize it. Some people like to keep everything and organize all their data neatly, others might be messier or might want to get rid of unnecessary items. So, how can we support people’s needs and individual preferences for managing their data?
Research goals and questions
Can centralization and customization help users better manage their data?
In this study, we wanted to understand if two design approaches could help people in better managing their data.
- The first design approach is to centralize data in a single location, providing an aggregated overview of items from different platforms and devices. We wanted to know whether centralization could help people.
- The second design approach is to use customization to create a system that can better adapt to people’s individual preferences for how to manage their data. We wanted to know whether customization might help to create a system that resonates with a wide range of users.
To answer these two key questions, we designed Data Dashboard, a centralized and personalized system for managing data. Then, we evaluated a prototype of the system in a user study with 18 participants.
The Data Dashboard prototype
Data Dashboard is a centralized system that gives users an overview of personal data and a set of personalized filters for sorting through their items
We designed Data Dashboard as a centralized system that shows an aggregated overview of data stored on different devices or cloud platforms (e.g., Dropbox, Google Drive, iCloud) and also provides a set of customizable filters for curating data. There are four sections in the system:
- Activity shows data based on recent activity, grouped by type.
- Explore Your Data shows all data broken down by type and also has a set of filters for sorting through data.
- Quick Actions provides a set of recommendations for quick management actions and also has the same filters found in Explore Your Data.
- Settings let users change how the filters work and add or remove platforms and devices.
Evaluation methodology
We structured the prototype evaluation in three parts:
- An introductory interview, where we asked participants about their general attitudes and behaviors in data management.
- A scenario-based interaction with the prototype.
- A debriefing interview with two card sorting exercises.
Scenarios of use
After the introductory interview, we asked participants to interact with the prototype and prompted them to think out loud as they went through five different scenarios of use:
1 The space on your computer is running out. You want to find some data to discard. You are not sure where to start looking, but you know that you do not care too much about old documents.
2 It is a rainy day. You have set aside some time for doing a regular cleanup of your devices. You usually do this every few months. You want to review your data and make sure everything is organized in your preferred way.
3 You have 5 to 10 minutes in between meetings and errands. You decide to take a look at your recent data to get a sense of anything that needs taking care of.
4 You have heard about a data leak from a popular cloud storage platform that exposed personal information to hackers. You want to review what data you have stored on different cloud platforms that might pose a privacy risk in the future.
5 You are in the process of buying a new computer. You want to make sure that you are not going to lose any of the data you care about. You want to ensure that everything is stored in more than one place.
We used these scenarios to cover possible data management behaviors and situations.
Debriefing and card sorting
Then, after going through the scenarios, we had a debriefing interview where we asked participants clarifications and overall impressions about the prototype. In this last part of the study, we also had participants go through two short card sorting exercises:
- In the first card sorting exercise, we asked participants to rank the scenarios based on how relevant they were to their experience. We used this exercise to understand what parts of the prototype worked best in different situations.
- In the second card sorting exercise, we asked participants to rank the Data Dashboard prototype against other similar tools that they had used in the past. Here, we wanted to understand how the prototype would integrate into their existing flows and how it compared to existing tools.
Key results
The majority of participants (12/18) had positive reactions to the prototype, saying that it was smart, intuitive, user-friendly, and would save their time. Several participants also preferred the system when comparing it to other tools they had used in the past. The rest of the participants had mixed or negative reactions, thinking that some aspects of the system were unclear or unnecessary, although they still thought that in some specific situations and with some changes it could be useful.
Participants thought that centralizing data in a single location presented some privacy risks, but the customization options offered by Data Dashboard could help offset these risks. The different sections of the system proved useful in different scenarios of use, with the first three scenarios being the most relevant for participants.
Based on participants’ interactions during the scenarios, we noticed that the prototype had some issues in discoverability, usability, and clarity. Some of our key design recommendations focus on making key functions more discoverable, simplifying how the different sections work, and promoting consistency within similar sections or functions.
Overall, the results of the evaluation show that our approach of combining centralization and customization in a single system has promise. That said, there is more work to do in this space to fully support people’s needs and Data Dashboard can provide a good jumping point for future system development.
In the paper we virtually presented at the DIS 2020 conference, you can read more about the design decisions that went into the prototype and the reactions from participants framed against the conceptual metaphor of data boundaries.
