Tree testing in the design process — Part 1: The research phase

When we’re designing a website, nothing happens in a vacuum. Every facet of design — research, information architecture, interaction design, visual design, content strategy — affects everything else.

In Part 1 of this series, we’ll look at:

• Where Information Architecture (in general) and tree testing (in particular) fit into the overall design process
• How to best use tree testing alongside other methods in the research phase of your project

How does IA fit into design?

Information Architecture (IA) concerns itself with the very broad (overall site structure and navigation) and the very detailed (labeling and content).

Early in the design process, information architects look at the scope and state of their content, and ask questions such as:

• How much of this content are we keeping/deleting/adding?
• Which content is up to date? Which needs reworking for our target audiences?
• How will we organize the content? What do our users mentally group this material?
• Can we standardize terms, and if so, which terms should we use?

Later in the design process, information architects shift to thinking about how users will find what they’re looking for, including basic elements such as:

• Global navigation (global and contextual)
• Search (content and metadata)

Once a prototype or working site is up and running, information architects want to see how all of these factors combine with interaction design, visual design, and actual content to get users to the answers they want. Typically they find out by using traditional UX evaluation methods such as:

• Usability testing (in-person or remote)
• Analytics (page visits, abandons, etc.)

When it’s done properly, IA doesn’t just happen at the beginning of the project; it progresses from beginning to middle to end.

How does tree testing fit into design?

We mentioned four basic IA elements above — organization, labeling, navigation, and search. These are taken from the classic “polar-bear book” – Information Architecture For the Web and Beyond by Rosenfeld, Morville, and Arango.

In general, we tend to deal with the first two (organization and labeling) as foundations before addressing the second two (navigation and search), which segue into UI design.

Because tree testing is only concerned with the first two elements (organization and labeling), it makes sense that tree testing should be done early in the design process, not later.

To get more specific, tree testing typically happens at two points early in the design of a website:

1. During the research phase, before new site structures are even conceived.
2. Early in the design phase, when our research has suggested ways to structure the site, and we have started creating site trees to exercise those ideas.

Below, we’ll look at the research phase. (In Part 2 of this series, we’ll look at the design phase.)

The research phase

Of the three phases of user experience (research, design, and testing), research is the one that tends to get abbreviated (or skipped entirely) when budgets and proposals are drawn up. Experienced designers know that this introduces big risks to the success of a new site, because it’s harder to design something good when we don’t really know who the users are, what they know, what they need, and what causes them the most pain.

While the first method discussed below (contextual inquiry) does take an investment of time and money, the other two methods (card sorting and baseline tree testing) can be done quickly and cheaply. So, even on a shoestring research budget we should be able to get some useful insights to help us design a better site structure.

Using contextual inquiry

When we don’t know enough about our users, one of the best methods of finding out more is contextual inquiry; a fancy term for “watch, listen, then ask”. Wikipedia describes it as:

“…an approximately two-hour, one-on-one interaction in which the researcher watches the user do their normal activities and discusses what they see with the user.”

Ideally, we visit the user in their natural habitat (home, office, commute, whatever makes sense for our purposes) and observe them doing tasks related to our website. During or after the observation, we ask them questions about what they did and why, so we can get a clear idea of what they knew, how they behaved, what they wanted, which issues they encountered, and so on.

As information architects, we pay particular attention to the IA-related elements:

• which content they were after, and why
• how they mentally grouped different kinds of content
• which terms they used (and which terms they knew)
• how they browsed or searched for items

For example, if we’re researching how cyclists buy bike gear online, we might observe 20 people and discover the following:

• Content
  Most are looking for parts and accessories, not bikes (perhaps because they prefer examining and trying bikes in person at a store).
• Grouping
  Non-experts prefer items to be grouped by topic (for example parts, clothing, etc.), not by brand.
• Terms
  Most understand Parts versus Accessories, but do not know the difference between one-speed and fixed-gear bikes.
• Browsing/searching
  They all start by browsing the menus unless they already know the exact model name/number to search for.

Contextual inquiry takes time and some practice, but it’s great for discovering what users really care about, how they behave, and why. It’s a fundamental user research method that all designers should know and use.

For more on contextual inquiry, see:

• “Usability Body of Knowledge: Contextual Inquiry”
• “User and Task Analysis for Interface Design” by JoAnn Hackos and Ginny Redish
• “Contextual Design: Defining Customer-Centered Systems” by Hugh Beyer and Karen Holtzblatt

Using card sorting to generate ideas

Card sorting is probably the best known (and most used) method in information architecture, and with good reason — it’s a great way to find out how users think about content.

Early in a project, when we’re looking for ideas for structuring our site, we can run an open card sort to see how our users mentally organize topics.

Whether it’s done in person using index cards, or online using web apps, the fundamentals are the same:

• We create about 40 cards that represent a range of topics on our site.
• We ask participants to sort these cards into groups.
• We ask participants to name the groups they’ve created.

The findings from a card sort can fundamentally change how we structure our site. For example, suppose we are designing a recipe website.

We might have initially thought about grouping the recipes by time of day:

If the card sort revealed that most of our participants grouped the recipes by cuisine, we should really reconsider our main headings:

Card sorts are not restricted to top-level headings. We can also run card sorts on subsets of our content, to generate ideas for the next few levels down.

For any medium- or large-sized website, we recommend running open card sorts as part of the research that happens before we jump into design.

The definitive book on card sorting is Donna Spencer’s “Card Sorting: Defining Usable Categories”. For a quick primer, see her 2004 Boxes & Arrows article.

Baselining the existing tree

During the research phase, we also recommend running a tree test, even before we’ve created draft site structures.

How can we test a tree if we don’t have a tree yet?

If we’re redesigning an existing site, we do have a tree (our existing site structure). We should definitely run a tree test on it, even if we know the existing tree isn’t very effective.

Why? Because it gives us something to measure our new ideas against. Remember, we don’t just want to create a new site structure; we want to create a better one. And the way we ensure that is by baselining the old tree and later measuring it against our new ideas.

For example, one of the first organizations to use tree testing was ACC, a public health service in New Zealand. When they redesigned their website a few years ago, they did a baseline test before they revised their site tree, and another tree test afterward. They were able to quantify an substantial improvement in findability:

The scores are only part of the story. Baselining the old tree also helps us find out:

• Which parts of the old tree are problems that we can fix or rethink
• Which parts of the old tree are actually working well, so we can reuse those in the new site. We don’t want to throw the baby out with the bathwater.

Over and over, we see redesign projects head off in the wrong direction because the stakeholders think they know what to change. Doing proper research (such as contextual inquiry, card sorts, and baseline tree testing) helps to point us in the right direction before we start designing in earnest.