Conducting a User Study

Now that the stage is set, it's time to deliver the act.

Days or weeks of hard work, boils down to this one hour with the user. Conducting a good user interview can be deceptively hard. Making the user feel comfortable, refraining from asking leading questions, removing personal bias, preventing the user from digressing etc, are all hard earned but crucial interview skills. They take time to master, and discipline to manage.‌

To make things easier, we break down the act into the three parts that follow.

Act 1: The Exposition

Like all good plays, the first act is about setting the premise — gauging the users background, introducing them to the subject and creating the right atmosphere for what’s to come. Unlike all good plays, we don’t want this to be exciting — comfort is key.

Unfortunately, the nature of user studies is such that they can be quick and intense. While we may be pressed for time, we don’t want our users to sense this. By talking slowly and patiently, we signal to them that they are our priority and this encourages them to open up to us.

To ease our users into the study, we begin with a warm introduction and explain what the purpose of the study is. We then ask them for permission to record the interview and explain how this data will be used.

If our users have any questions for us, we clarify them and assure them that we have their best interests at heart. Once we know they are reasonably comfortable, we ask them to introduce themselves and begin the interview with a few opening questions from our script that leads into Act 2.

Act 2: The Complication

The complication is where all the action lies. It’s the part where the protagonist — our user — is faced with difficult choices, learns to navigate tricky situations and reveals their true selves to us.

Our aim is to ask users to perform tasks or answer important questions to understand their mental models and gauge their understanding of the product.

As we transition to the tasks, we ask for their permission before we begin the activity to reinforce a small sense of control for the user. While introducing a prototype, we tell users that we haven't built the product, we’re simply testing it. This makes them less likely to withhold criticism.

While they perform the activity, we encourage them to think out loud and remind them that there are no right or wrong answers. Sometimes these tasks can feel like an uncomfortable farce. So, to encourage them and to make them feel heard, we nod, take notes, offer acknowledgements and make frequent eye contact.

Sometimes users may have questions about our designs. When this happens, we simply redirect the questions back to our users. For example, if they look at an icon and say “What does this mean?”, we simply reply by asking “What do you think it means?”

This discomfort may also manifest itself in pockets of silence. We do our best to embrace this — by allowing long pauses we’re giving our users a chance to elaborate on topics without being prompted. If this doesn’t work, we probe a little deeper with open ended questions like “Can you tell me more about that?”.‌

Act 3: The Resolution

After the intensity of the Complication, it is time for the Resolution. In this act, all the main tensions are brought to a close and we have a rich understanding of who our protagonists really are. If done well, users walk away feeling like they’ve made a meaningful contribution.

We begin by going through our debrief questions and tasks to summarize our users reaction to the product. Then, we ask them if they have any questions for us to create a sense of closure.

In addition to our climax, we want the user study to have a happy ending. So, we thank the participant for their time, give them the incentive picked out earlier and walk them out.

However, if an interview is unproductive, we don’t hesitate to end it early. This is not only to save our time, but also theirs — users can sense when things aren’t going well and it’s best to end on a positive note.

When we have followed this process with 4-5 users, we use our recordings and notes to then synthesize.

Synthesizing the user study