What's a scenario walkthrough?
A scenario walkthrough is a fast, flexible, and inexpensive method for evaluating the things you build, including applications, documents, web content, and processes for service delivery. In it, a member of your team will adopt the perspective of one of your constituents and work through a realistic scenario. For example, they may try to complete an application, upload a document, or get an answer to a question. Your team will take notes on the problems that come up, and you’ll use your findings to improve your project. In this way, scenario walkthroughs let you rapidly catch issues, iterate, and improve before you release a project or feature to constituents.
When and why conduct scenario walkthroughs?
Conduct scenario walkthroughs early and often. Aim for them to be a typical part of your team’s work, especially during moments where you are releasing new features or making substantive changes. For example, you might conduct a scenario walkthrough at:
- The beginning of a design or redesign phase to identify work to be done and inform your strategy
- Key decision points, such as prior to a release or pilot
- The end of a “sprint” to evaluate a prototype and inform your future plans
- Before you release a major revision to a document
You can set up and conduct a scenario walkthrough in a few hours.
Preparing for a scenario walkthrough
Define what you want to learn from the test (learning objectives)
First, agree on what you want to evaluate and what you hope to learn. This should be specific enough that you can design a “walk through” around it. For example, if you are about to launch a new webpage (or website!), you’ll need to be more specific than, “Do people have a good experience using this webpage?” Instead, settle on something like:
- Does the organization of this webpage meet people’s expectations?
- Do people get stuck trying to fill out this form?
- Can people use this calculator to find out if they are eligible for benefits?
- Can people enter their information into this form using a keyboard only (no mouse)?
Audience you’ll focus on
Identify a specific person or persons you’ll focus on — emphasis on specific. “A member of the general public” is not enough.
You can start with a group of people, e.g. “small business owners,” but the walkthrough will be more productive if you have a specific person in mind. If you’ve defined personas for your project, you can use these. You can also pick a specific person you’ve engaged with previously, such as someone who visited a field office or who called your call center. If you don’t have these, make up a person based on your assumptions.
Here is an example written for testing new web content for getting a Real ID Driver’s License:
Priya is a 34-year-old HR Manager at a tech company. She recently got married and wants to update her license to reflect her new last name. At the same time, she wants to upgrade to a Real ID.
Try to add a few details, such as job, life situation, and age. These help the role player adopt realistic motivations and decisions during the walkthrough.
Important: Taking on the role of a constituent is not a substitute for actually speaking with them. Real people will have perspectives and experiences you can’t simulate in a scenario walkthrough. However, testing with real people can be time consuming and expensive. This is why it’s good for both scenario walkthrough and constituent interviews to inform your project plans.
Design a scenario your persona will “walk through”
A scenario is a meaningful, realistic situation in which one or more people attempt to achieve a goal. It is not a technical use case. That is, it should reflect how the persona you've chosen would think about what they're doing and why.
Once you’ve named your scenario, identify and describe the key tasks or steps that the person would have to do to achieve their goal.
Most of the time, you will want to define success criteria, too. For example, imagine you're testing if someone can learn if they are eligible for something. You'll want the "audience member" to say during the test if they believe they are or aren't eligible (or if they can't tell). You'll also want to know if this answer matches the truth.
Design your scenario to make sure the role player focuses on what you want them to focus on. For example, imagine you are testing if someone can use a calculator. It may be helpful to agree that the scenario begins after the person has already found your calculator or completed any prior requirements to using it. This will prevent the scenario wandering into territory like "Can you find the calculator through a search engine."
You can also update your persona in a way that shapes your scenario. For example, you might stipulate that they "hate to use the phone if they can avoid it," or "feel skeptical of interacting with government." This helps narrow the paths your role player can take.
You may need multiple audience/scenario pairs
Some objectives may require multiple scenarios. For example, imagine you want to learn if people can fill out an application. You could write personas and scenarios someone who:
- Only uses the keyboard
- Uses a cell phone (and not a desktop)
- Doesn't speak much English
Alternatively, you can narrow your objective so that you don't need to include as many audiences and scenarios at a time.
Assemble your team
You’ll need:
- Facilitator: This person will lead the review session. They’ll give instructions to participants, describe the ground rules, pause the exercise to prompt team reflection, and ask questions to keep the review moving through the scenario.
- Role player: This person will act as the person you came up with and try to complete your scenario in pursuit of their goal. While they do this, they will “think aloud,” narrating their experience. People who have experience interacting with your audience are often good candidates for role players.
- (Optional) Additional reviewers: Observers of the review session who will record problems the role player experiences and participate in reflections.
Running your scenario walkthrough
Prepare the team
Explain the rules to everyone involved, including:
- Who the person testing the scenario is
- What the scenario is, including the steps to complete it
- What procedures you want reviewers to follow for capturing issues and improvement opportunities
You can adapt the review to your needs, though you should tell the team to avoid prioritizing, debating, or solving issues during the review. Remember, the purpose of the review is to get a better understanding of a person’s experience with your project.
You should define the key questions that observers should attend to at each step in the scenario. For example, you might ask them to focus on how easily a person moves toward their goal, if they seem to understand the information they encounter, or where they get stuck.
Walk through the scenario
The facilitator leads the role player through the walkthrough, reminding them whenever necessary to act as if they are the person your team has agreed upon. The facilitator should also encourage the role player to “think aloud,” preferably as the character and in first person.
The role player
The role player tries to achieve their goal while thinking the way their persona would think. This means:
- Act using an appropriate level of prior knowledge. For example, would your persona understand a technical term they encounter?
- Try to respond the way the persona would. If you encounter a technical term you don't know, does it make you frustrated? Angry?
- Make decisions the way the persona would. Would they read text carefully or skim? If they aren't sure about something, would they pause to reflect or would they plow ahead?
The role player should voice as much of this as possible. This allows the team to take notes and identify key moments. They should comment on things like:
- What they’re paying attention to
- What they’re thinking
- Questions they have
- What they expect to encounter, especially in comparison with what they are encountering
- How they feel along the way
The facilitator
The facilitator has multiple jobs during the walkthrough.
- Make sure the role player is "thinking aloud." You can accomplish this with gentle prompts, such as, "I noticed you reread that paragraph. Can you say what you're thinking?"
- If the role player does something out of character, remind them who they're "playing" or what their actual goal is.
- Keep the role player on track. For example, imagine you want to test if someone can figure out if they're eligible. The role player spends a long time researching one specific term. The facilitator might say, "I think we can agree this is a confusing term. Let's keep going through the eligibility criteria so we don't run out of time."
- The facilitator may sometimes pause the walkthrough and ask the reviewers and role player (who can temporarily step out of character) to briefly share the issues they’re noticing. You can do this every few minutes or at key points in the walkthrough.
Review your results
When the walkthrough is done, you should:
- Organize and synthesize the issues you’ve identified
- Highlight any opportunities you see to improve your project
- Prioritize issues so that you can take action
Tip: It’s helpful to assign scales to issues, e.g.
- Showstopper issue: Something that blocks someone from succeeding
- Major issue: Makes it difficult to succeed
- Minor issue: Small issue that it would be nice to fix
Scenario walkthroughs are not a substitute for speaking to constituents
Scenario walkthroughs will not match real people’s experiences. You should not assume that conducting them will reveal all or even most problems with your products and services. However, they will help you catch obvious issues, and they keep you grounded in the experiences of the people you serve. This is essential for teams who want to release useful, usable, and used products and services.