Six ways to run closed card sorts like a boss

Closed card sorting is a versatile research technique with virtually limitless application, and we thought it was time someone sung its praises from the skies. So here’s a post that suggests a few ways you can use it to find out cool stuff about your customers, your teams, your products, and your ideas.

A quick word on how closed card sorting is different to open card sorting

Closed card sorting has sometimes been used to validate an information architecture, like so: You run an open card sort to generate category labels, and then you test the effectiveness of these labels by getting people to place your content in the categories. And it seems to follow that if most of your participants place the same cards into one category, then replicating this grouping on your website will make your information easy to find. But as Donna Spencer pointed out in 2005, this approach can have drawbacks:

Categorizing information and finding it are two entirely different tasks, with entirely different cognitive processes. The only way to test whether a classification will allow people to find information is to ask them to find information (or at least ask where they would look). You don’t learn it by asking them to place information in the classification.

Closed card sorting is a ‘content-centric’ technique, as Donna says. You could use it to find out where people would place content under categories you’ve generated from running an open card sort, which will show give you data on how many people agree on what goes where. But to find out if the relationship between your content is logical to people, and if they can find information easily and quickly, your time is probably better spent on a task-centric technique like tree testing.

Closed card sorting can be simple for your participants to complete

Closed card sorting is good because, well, it’s an easy activity for people to complete. This of course depends on your categories, and the complexity of the terms you have on your cards, but generally, participants are asked to do something that comes naturally. Research has shown that ‘organizing the chaos of the surrounding world into categories is one of the brain’s key functions’. And this function is so ingrained that some scientists have been able to pinpoint exactly where these ‘semantic neighbourhoods’ are located in the brain. And although open card sorting taps into the brain’s same tendencies, the practise of grouping items and labelling groups (creating a taxonomy, essentially) may not come easily for everyone, and might need to be explained beforehand.  In contrast, when you ask people to arrange content into preset categories — like ranking items from Important to Unimportant — they’re more likely to have the mental model of the task already and find it easy. What this comes down to is that your results are likely to be based on peoples’ quick, intuitive decisions.

Six ideas for closed card sorts

The potential for closed card sorting is essentially only limited by your imagination, so keep in mind that these six ideas are just the tip of the iceberg. Each idea has a brief explanation, and an image illustrating how your study might look. What you might notice as you scroll is that website analytics can answer a few of the same questions we’re suggesting you can answer with closed card sorting. But though analytics can tell you what people actually do, it can’t tell you what people think they do or want to do. Studies like the ones you’ll see below can. In the first example below, I’m sharing the set up and results of a closed card sort we ran on Yelp’s search filters, and what the results from 40 participants look like in OptimalSort. The other examples are suggestions only — but if you see one you’d desperately like us to run, let me know!

1. Establish which tags or filters people consider to be the most important

Create cards with your search filters or tags, and have participants rank them according to how important they consider them to be. Analytics can tell you half of the story (where people actually click), so the card sort can give another side: a better idea of what people actually think or want. In the Yelp study, we asked participants to sort 47 cards into four categories: So that people understood what we meant by our category names, I added more detail in the instructions they see before they start sorting cards. To find out more about our participants’ experience using Yelp, we asked three simple pre-activity questions that gave us more than enough context: The results overview shows that 40 people sorted all 47 cards in an average of 3.03 minutes, which suggests that people found the sort simple and intuitive. (In contrast, and to illustrate the difference in cognitive effort between closed and open card sorting, participants in our latest open card sort have taken an average of 12.6 minutes to sort 44 cards.) The simplicity of a closed card sort (a set of cards, a set of categories, and no outliers or new labels to deal with) can make for heart-soaringly simple data to analyze. For example, this is what the Similarity Matrix looks like: And this is a glimpse at the top of the Results Matrix (with the darker the blue, the more times the card was placed in that category): And the Cards table, which shows that the top four cards were only sorted into ‘Very important’ and ‘Somewhat important’, and never into ‘Not important’ or ‘I don’t know what this means’.

2. Reduce content on landing pages to what people access regularly

Before you run an open card sort to generate new category ideas, you can run a closed card sort to find out if you have any redundant content. Say you wanted to simplify the homepage of your intranet. You can ask participants to sort cards (containing homepage links) based on how often they use them. You could match this card sort data with the analytics data from your intranet from the last month or two and see if peoples’ actual behaviour and self-perception are well aligned. The results you get from this task could give you a great starting point for next questions to answer. For example, if 95% of participants place ‘Knowledge wiki’ in the ‘Never column’, it might mean that

• they know everything they need to know
• they don’t know it’s on the homepage, or it’s not prominent
• the wiki sucks.

And if 100% of people place ‘Employee access’ in ‘At least once a day’ and the remaining cards in ‘Once or Twice a month’, you might decide to make an employee access form that is prominent and dominant, and place the other categories to the side.

3. Crowd-source the values you want your team/brand/product to represent

Card sorting is a well-established technique in the ‘company values’ realm, and there are some great resources online (and a few very cool products) to help you and your team brainstorm the values you represent. These ‘in-person’ brainstorm sessions are great, and you can run a remote closed card sort to support your findings. And if you want feedback from more than a small group of people (if your company has, say, more than 15 staff) you can run a remote closed card sort on its own. In this example, participants are asked to sort cards containing words from  Microsoft’s Reaction Card Method

4. Do some market research to help you decide what to work on next

This is a fun way for you to engage your customers in market research. You could have product types on your cards, and categories labelled ‘Buying this month’, ‘Buying this year’, and ‘Never buying’, for example. In the following example, participants would be asked to sort the dresses based on whether or not they’re ‘on trend’. (Keep in mind that I have no idea what ‘on trend’ actually means, so this is purely a hypothetical).

5. Find out what actions people take across time

You could create categories that represent time, and ask people to sort the content according to how they go through their days. For example, your categories could be ‘January to March’, ‘April to June’, ‘July to September’, and ‘October to December’, and you could ask participants to sort according to the time of year they’re most likely to do certain things (go on vacation, do their taxes, make big purchases, and so on). And in this example, you would ask participants ‘At what time of day do you do these things the most?’. Obviously there are other more arduous and more accurate methods for gathering this data, but sometimes you just want something useful somewhat faster.

6. Gather quantitative data on prioritizing project tasks or product features

Closed card sorting can give you data that you might usually gather in team meetings or in post-its on the wall, or that you might get through support channels. You can model your method on other prioritization techniques, including Eisenhower’s Decision Matrix, for example:

7. Well, this one’s up to you…

As I mentioned at the start, the possibilities are endless. These a just a few examples of the kinds of closed card sorts you could run. Let us know about the closed card sorts you’ve run, and what you might do next.