User Research Program
FTI Technology (2014–2018)
Project goals
I saw an opportunity at FTI to do more user research, and specifically to involve external users and users of competitive products. Through this research I thought that we could to gain insight into how and why people use e-discovery software (not just our product), while exposing the entire development team to first-hand accounts and experiences.
I started with pilot sessions and coworkers saw the value and began seeking more user observations and testing for their projects. This resulted in improvements across the product.
I interviewed and observed participants in addition to conducting think-aloud usability testing. I sought out both experienced and novice users to gain a broad perspective.
Contributions
- Recruiting representative and varied participants
- NDA and compensation logistics
- Study design including:
- formalizing study goals
- writing tasks
- defining interview protocol
- Prototyping
- Running studies (site visits or remote via screen sharing and phone)
- Creating reports and a shared archive
- Presenting results to the entire development team
Results
Coworkers in design, development, quality assurance, and product management have attended user testing sessions and research presentations. Attending these sessions helped the whole team think about their biases and gaps in understanding, leading to more reflection on our product and the desire to continue learning about our users.
Case study
I planned and moderated usability testing for the Conditional Coding feature. This feature was new and involved administrative users creating templates (including logical conditions) that others would use to tag documents that they reviewed. The development team (product owner, designer, and engineers) wanted to know if users could easily complete key tasks and were also open to new feature ideas that might emerge during testing.
Results
We agreed on ways to address bugs and points of confusion that were uncovered. We made plans to conduct a follow-up study after those changes were implemented.
One specific example of a problem we addressed is that users were able to access a particular feature whether or not that they had taken a step that was a precondition to actually using that feature. We discussed either disabling the feature until it was applicable, or actually changing its behavior. In addition to noting potential problems, I also make note of what was clear to users and worked well, so that we consider those aspects when making any changes.
Following the testing sessions and discussions with the team, I gave a brief presentation to show the broader team what we had learned and what changes we planned to make. That presentation is available upon request.
Process
For this study, I began by meeting with the product owner and designer on the project to talk about their questions and goals. I then wrote a study plan that I shared with project team for feedback. I later developed a detailed guide for the testing sessions, including specific tasks to test and prompts for the participants.
This study was conducted using a working prototype version of the features. Before beginning a study, I run through the testing guide with the prototype and then also do one or two practice sessions with coworkers to make sure that everything goes smoothly with our participants.
For this particular study I was able to work with our product owner to recruit four internal users, from two slightly different user types. All participants were remote, so the testing was conducted via phone and screen sharing. The development team was invited to listen in and later discuss the sessions.
User research influences
I studied user research methods and cognitive psychology as part of the design master's program at Carnegie Mellon University. In addition, there are many practitioners whose work I admire and have learned from. One researcher whose work I've found especially helpful is Steve Portigal, specifically his book Interviewing Users and his podcast Dollars to Donuts, which consists of interviews with user research leaders.