How to test paper prototypes with users

Dear Optimal Workshop

I’m working on my first project with paper prototypes and I’m not quite sure what the best method is to test these on users. Any suggestions?

Liz

Hi Liz,

Excellent question! Paper prototypes are an awesome way to introduce research early in the design process. They’re cheap, useful, and testing them is actually kind of fun! It might seem strange to test digital interactions with bits of paper and pens, but you’ll soon realize that they can in fact generate plenty of great feedback for you to work with.

Testing paper prototypes is a bit of a departure from traditional usability testing, and it can take a while to figure out the best way to test something that is not fully functional. The best way to test your prototypes is largely determined by what you’re trying to test, who you want to test, and how much time you have to get it done. Here are three approaches you may want to consider:

Testing with your user, in person

Moderated, in-person tests are ideal for testing interactions. If you’re using prototypes to see what actions your users perform between different screens, being in the same room as your participant has many benefits.

From a practical standpoint, by playing the role of the ‘human computer’, you’re able to walk your participant through your interactions, and swap out screens after your users perform certain actions. From a research perspective, testing in person allows you to address any questions or hesitations that your users may have during the session.

Since you’re testing an incomplete experience, it’s very likely that your session will drift away from your script, or that the flow of actions you intended your participant to follow will not be as seamless as you would have hoped. These instances can be a great thing, as they challenge any assumptions you may have made about how your user will interact with your prototype. Depending on the sophistication of your prototype, it can be more (or less) important to follow a strict script to make sure you are testing what you intended. Following a set flow of events can be especially useful when testing low-fidelity prototypes, where not all interactions are possible to your participant. By being in the same room, you’re on hand to answer any questions, and keep your test moving along when things don’t go according to plan.

Testing with your user, remotely

Running moderated tests remotely is a great option if your participants are hard to reach geographically, or if you don’t have the possibility of inviting them to an in-person session.

Ideally, when running a remote test, you want a solid internet connection, and good quality audio and video. Scans of your prototypes can be uploaded to something as basic as a slideshow or PDF file, where your screens are sequentially chained together. Alternatively, you can use a simple prototyping tool to create a more realistic experience for your users to click through (we recently used Atomic, and it was very easy to set up!).

Remote user tests are harder to execute than in-person tests, and careful preparation is key. Technical difficulties can often interfere with getting quality insights. Not being in the same room as your participant also gives you less control over the session, and fewer opportunities to observe more subtle indicators that something may not be working as it should.

The legendary Steve Krug recently acknowledged some of challenges of remote testing during a talk at our Wellington office: “One thing that is true about remote testing is you have to pay really close attention, because you may not know whether they fell silent because they’re thinking about something, or they fell silent because their dog just came into the room.”

A few tips for setting up your moderated tests…

Focus on expectations

Ask your participants what they think may happen after a certain action is performed, because it can often reveal areas where your design may misalign with what your users would expect. Rather than moving immediately to a new screen, give your participant the time to explain what they imagine would happen after performing a certain action.

Don’t take it too seriously

Sometimes it’s good to admit to your participant the limitations of your design before you begin your session. This puts everyone at ease, and helps your users ‘play along’ with the tasks you’ve assigned. Paper prototyping is a great technique, but it’s a pretty unusual experience, so don’t treat your sessions too seriously! If something is confusing or unclear, ask your participant to draw or sketch their own ideas for how things could be improved.

Testing remotely, with many users

If you’re testing certain elements of your prototype, and answering questions such as “Will users understand this button?”, or “Do users know how to locate the search box?” an unmoderated test with a variety of users may give you the answers you’re looking for.

Testing paper prototypes with Chalkmark

Chalkmark is a remote click-testing tool that makes it quick and easy for you to test your screenshots and visual designs, and make informed design decisions. Setting up a Chalkmark study couldn’t be easier. Simply scan (or take a good photograph) of your prototype, and upload your image. Give your participants a set of tasks that relate to the part of your design you’re testing, set the areas that contain the correct answer, and you’re good to go!

Once your results roll in, Chalkmark will show you exactly where your participants clicked to find the answer to your question. From here, you can work on refining your design to make sure you’re pointing your users in the right direction, and minimizing their chances of clicking down the wrong path.

A few tips for unmoderated tests…

Word your tasks carefully

Avoid using language found in your UI when writing your tasks. It’s likely your participant will simply skim the image trying to find the relevant words corresponding to your question. Since your users will generally visit your website with a specific goal in mind, make your tasks action-orientated to give them more context.

Look at reaction times

Reaction times can tell you a lot about how difficult it was for your users to find the answers to their questions. Even if they successfully navigated to the right section in the end, consider making relevant information more prominent in your design to minimize the time it takes for your users to find what they’re looking for.

Leave space for additional comments

Since opportunities for diving deeper into participant responses are limited during unmoderated tests, adding follow-up questions to your Chalkmark study can go a long way. Use your post-survey questions as a space for any comments, ideas, or elements that were unclear or confusing.

A few final words…

So in summary, in-person tests are great for evaluating your current ideas and identifying issues with your design. They also tend to be more generative. The dialogue between you and your participant can often help you discover new ideas and insights to feed into your prototypes. Conducting your in-person tests remotely is also an option, if your participants are scattered around different locations.

Unmoderated tests performed remotely with a larger group of users are generally more evaluative. They can give you information on where people expect to click to find certain pieces of information, but they won’t give you as much qualitative insight as in-person tests.

Thanks for the question Liz! I hope you’ll find these approaches useful 🙂