Preparing for a User Study

User studies are often a little bit like improv. While a loose script does exist, all involved are constantly adjusting to the words and reactions of the protagonist — the user. Improv is tricky, but user studies can get even trickier because you deal with not 1 protagonist, but 5-6 of them, back to back. It's almost like repeating the same play, but with a different storyline every single time. The challenge though isn't having to deal with different stories, but maintaining a consistent set of situations for users so as to be able to draw patterns. Such an ambiguous situation obviously calls for a few prerequisites:

1. The Screenplay

2. The Actors

3. The Stage

Part 1: The Screenplay

By now, we already have a research goal, user tasks and a complementary prototype ready. Our next step, is to seamlessly weave them together into a 3 part act —

Act 1: The Exposition

In theatre, The Exposition sets the premise of the play. Likewise, our goal is to understand our users background, get them comfortable and introduce them to the purpose of the study.

We begin by crafting a warm introduction and a few opening questions. Our aim is to get the users comfortable so these questions need to be impersonal and casual.

Next, like most good plays, we need an inciting incident — a catalyst that leads us into Act 2. For example, if we are working on an events app, we may ask “So what do you do in your free time?” and follow this up with “How often do you attend events?”. This creates a segue from a casual conversation to the subject of the study.

Act 2: The Complication

With our Exposition ready, we need to flesh out our narrative. In Act 2, we orchestrate several tasks for our protagonist to perform and explore.

Our aim here is to ensure that users stay natural. Over time, we’ve realized that the answer to doing this is the same as making a relatable play — recreating reality.

To do this, we turn to scenario-based tasks. If possible, we try to get them to enact the task, for example —

An experiment - Use QR code to transfer money. Task - You just purchased a few things from a street vendor. He would like you to pay through UPI.

To add a touch of reality, we complement this activity with a QR Code card. We then design several other tasks similarly and knit them together as a story. Now, we ask several open ended questions — even when the answers are binary (yes/no). For example — "Do you like this feature?" could be rephrased to "What about this feature feels good or bad?". This encourages conversation and keeps our users engaged.

The Resolution

Finally, it’s time for the climax — a summary to tie together the entire session. To quickly sift through the entire session, we ask a couple of debrief questions like the ones that follow:

🤔How does this compare to what you do now?

🧐How would you describe this product to a friend?

😐How would you compare both of these products? (If you’re testing multiple versions)

Another approach is a cool down co-creation activity. We often go on the field with extra paper, voting stickers and print outs of our concepts. With these, we either ask users to help us design screens, suggest names for titles or vote on different user flows. Doing so gives us a nuanced view of what they need, want and understand.

With our 3 acts ready, we rehearse and edit the screenplay to make it tighter.

Part 2: The Actors

We need to find 3 types of actors to complement a user study script —

1. The Protagonist/Users

To find the right protagonists, we begin by defining their demographic, their interactions with the product and their goals/concerns.

Using this as a base, we source and filter users from our clients, our personal networks or a recruiting agency. Regardless of the source, we make sure to get on a call with them to verify if they’re the right fit before scheduling an interview.

Since our users’ time is valuable, we choose an appropriate incentive to compensate them for it.

2. The Narrator/Interviewer

It is no easy task to conduct the interview. It requires unending patience, tactful conversational skills and a ton of willpower. To ease this pressure, the interviewer prints out the script and conducts a trial study with any one of our team members. This helps them anticipate certain situations and plan for them accordingly.

3. Secondary Characters

This role depends on the script but can range from a note-taker to someone to role-play with our user. If possible, we try to take someone who is not a designer or researcher to help them empathize with our users.

Part 3: The Stage

With our actors and screenplay in place, now we set the stage.

Is the user going to be using the design when alone or surrounded by others, subject to interruptions? If the user is likely to be using the design in a crowded space for example in a hospital, we prefer to conduct the test in their actual surroundings.

We use our phone to record audio, video and screens. To recreate reality, we may use props such as print outs or physical devices for our users to role play with.

If we are conducting it in our office, we occupy 2 conference rooms — one for the interview and one to observe from. Using our laptop, we project the interview to the observation room with a Zoom call. We may also use AudioHijack and Screenflow to record the laptop audio and screen.

Conducting a user study