A comprehensive look at usability testing

Usability testing has an important role in UX and if you’re new to it, this article gives you a solid introduction to it with practical tips, a checklist for success and a look at our remote testing tool, Treejack.

Concepts Of Usability Testing

Usability testing is the process of evaluating a product or service with users prior to implementation. The goal of usability testing is to identify any usability issues before the product or service is released into the world for use. Usability testing is a research activity that results in both quantitative and qualitative insights and can be used to gauge user satisfaction.

A typical usability testing session is moderated and involves a participant, a facilitator and an observer. The facilitator leads the session and the observer takes notes while the participant completes the task based scenario. While this is common, usability testing is scalable and the possible approaches are endless giving you the flexibility to work with the resources you have available—sometimes one person performs the role of facilitator and observer!

Location also varies for usability testing. For example, you might conduct your testing in a lab environment or you might talk to users in a specific environment. It also worth noting that not all usability testing sessions are moderated—more about this later.

Usability testing usually occurs multiple times during the design process and can be conducted anytime you have a design you would like to test. User research activities like a focus group for example, are conducted early in the design process to explore and gain understanding before ideas are generated. Usability testing is about testing those ideas as early and as often as possible. From a fully functioning digital prototype to a simple hand drawn wireframe on paper, nothing is too unrefined or too rough to be tested.

Developing a Usability Test Plan

Before you start a round of usability testing, you need to develop a usability test plan. The usability test plan will keep you organised and is an opportunity to define roles and set clear expectations upfront. The first step in developing this is to hold a meeting with your team and stakeholders to discuss what you are going to do and how you plan to achieve it. Following this meeting, a document outlining the usability test plan as it was discussed is created and shared with the group for review. Any changes suggested by the group are then added to the final document for approval from the relevant stakeholders.

What to include in your usability test plan:

• The goal, scope and intent of the usability testing
• Constraints impacting upon testing
• Details on what will be tested eg wireframes
• Schedule and testing location
• Associated costs eg participant recruitment
• Facilitator and observer details for each session
• Session details
• Participant recruitment approach
• Equipment
• Details of any documentation to be produced eg a report

Usability Testing Questions

Once you have developed your test plan, you need to create a list of questions and task based scenarios for the testing session. These form the structure for your testing and provide a framework of consistency across all testing sessions in the study.

The questions serve as a warm up to ease the participant into the session and can also provide insights on the user that you may not have had before. These questions can be a combination of　open and closed questions and are especially useful if you are also developing personas for example. Some examples of what you might ask include:

• Tell me about a recent experience you had with this product/service
• Do you currently use this product/service?
• Do you own a tablet device?

The purpose of the task based scenarios is to simulate a real life experience as closely as possible. They provide a contextual setting for the participant to frame their approach and they need to be realistic—your participant needs an actionable starting point to work from. A good starting point for task based scenario development would be to look at a use case.

It is also important that you avoid using language that provides clues to the solution or leads your participant as this can produce inaccurate results. An example of a task based scenario would be:

You’re planning a Christmas vacation to New Zealand for your family of two adults and 4 children. Find the lowest priced airfares for your trip.

Usability Testing Software: Treejack

Treejack is a remote information architecture (IA) validation tool that shows you exactly where users are getting lost in your content. Knowing this will enable you to design a structure for your website that makes sense to users before moving on to the user interface (UI) design.

Treejack works like a card sort in reverse. Imagine you have just completed a card sort with your users to determine your IA and you are now working backwards to test that thinking against real world task based scenarios. Treejack does this using a text-based version of your IA that is free from distracting visual aids such as navigation and colour allowing you to determine if your structure is usable from the ground up. A Treejack study is structured around task based scenarios and comes with the option to include pre and post study questionnaires.

As a remote testing tool, Treejack is unmoderated and provides the opportunity to reach a much larger audience because all you have to do is share a link to the study with your participants to gain insights. You also have the option of handing the task of　targeted participant recruitment over to Optimal Workshop.

Once launched and shared with participants, Treejack takes care of itself by recording the results as they come in giving you the freedom to multitask while you wait for the testing to finish.

The results produced by Treejack are not only detailed and comprehensive but are also quite beautiful. The story of your participants’ journey through your testing activity is told through pietrees. A pietree is a detailed pathway map that shows where your participants went at each fork in the road and their destinations. They allow you to pinpoint exactly where the issues lie and are a powerful way to communicate the results to your team and stakeholders.

Treejack also provides insights into where your participants landed their first click and records detailed information on pathways followed by each individual participant.

Usability Testing Checklist

The following checklist will help ensure your usability testing process runs smoothly:

• Meet with team and stakeholders
• Determine goals, scope and intent of usability testing
• Decide how many sessions will be conducted
• Create usability testing schedule
• Select facilitators and observers for each session if applicable
• Develop and complete a usability test plan
• Determine test questions and scenarios
• Recruit participants for testing
• Gather equipment required for testing if applicable
• Book testing location if applicable
• Keep a list of useful contact details close by in case you need to contact anyone during testing
• Complete a dry run of a testing session with a team member to ensure everything works before actual testing begins
• Organise debrief meetings with observers after each testing session
• Set aside time to analyse the findings
• Document and present findings to team and relevant stakeholders