Online card sorting: The comprehensive guide

When it comes to designing and testing in the world of information architecture, itâs hard to beat card sorting. As a usability testing method, card sorting is easy to set up, simple to recruit for and can supply you with a range of useful insights. But thereâs a long-standing debate in the world of card sorting, and thatâs whether itâs better to run card sorts in person (moderated) or remotely over the internet (unmoderated).
This article should give you some insight into the world of online card sorting. We’ve included an analysis of the benefits (and the downsides) as well as why people use this approach. Let’s take a look!
How an online card sort works
Running a card sort remotely has quickly become a popular option just because of how time-intensive in-person card sorting is. Instead of needing to bring your participants in for dedicated card sorting sessions, you can simply set up your card sort using an online tool (like our very own OptimalSort) and then wait for the results to roll in.
So whatâs involved in a typical online card sort? At a very high level, hereâs whatâs required. Weâre going to assume youâre already set up with an online card sorting tool at this point.
- Define the cards: Depending on what youâre testing, add the items (cards) to your study. If you were testing the navigation menu of a hotel website, your cards might be things like âHomeâ, âBook a roomâ, âOur facilitiesâ and âContact usâ.
- Work out whether to run a closed or open sort: Determine whether youâll set the groups for participants to sort cards into (closed) or leave it up to them (open). You may also opt for a mix, where you create some categories but leave the option open for participants to create their own.
- Recruit your participants: Whether using a participant recruitment service or by recruiting through your own channels, send out invites to your online card sort.
- Wait for the data: Once youâve sent out your invites, all thatâs left to do is wait for the data to come in and then analyze the results.
Thatâs online card sorting in a nutshell â not entirely different from running a card sort in person. If youâre interested in learning about how to interpret your card sorting results, weâve put together this article on open and hybrid card sorts and this one on closed card sorts.
Why is online card sorting so popular?
Online card sorting has a few distinct advantages over in-person card sorting that help to make it a popular option among information architects and user researchers. There are downsides too (as there are with any remote usability testing option), but weâll get to those in a moment.
Where remote (unmoderated) card sorting excels:
- Time savings: Online card sorting is essentially âset and forgetâ, meaning you can set up the study, send out invites to your participants and then sit back and wait for the results to come in. In-person card sorting requires you to moderate each session and collate the data at the end.
- Easier for participants: Itâs not often that researchers are on the other side of the table, but itâs important to consider the participantâs viewpoint. Itâs much easier for someone to spend 15 minutes completing your online card sort in their own time instead of trekking across town to your office for an exercise that could take well over an hour.
- Cheaper: In a similar vein, online card sorting is much cheaper than in-person testing. While itâs true that you may still need to recruit participants, you wonât need to reimburse people for travel expenses.
- Analytics: Last but certainly not least, online card sorting tools (like OptimalSort) can take much of the analytical burden off you by transforming your data into actionable insights. Other tools will differ, but OptimalSort can generate a similarity matrix, dendrograms and a participant-centric analysis using your study data.
Where in-person (moderated) card sorting excels:
- Qualitative insights: For all intents and purposes, online card sorting is the most effective way to run a card sort. Itâs cheaper, faster and easier for you. But, thereâs one area where in-person card sorting excels, and thatâs qualitative feedback. When youâre sitting directly across the table from your participant youâre far more likely to learn about the why as well as the what. You can ask participants directly why they grouped certain cards together.
Online card sorting: Participant numbers
So thatâs online card sorting in a nutshell, as well as some of the reasons why you should actually use this method. But what about participant numbers? Well, thereâs no one right answer, but the general rule is that you need more people than youâd typically bring in for a usability test.
This all comes down to the fact that card sorting is whatâs known as a generative method, whereas usability testing is an evaluation method. Hereâs a little breakdown of what we mean by these terms:
Generative method: Thereâs no design, and you need to get a sense of how people think about the problem youâre trying to solve. For example, how people would arrange the items that need to go into your websiteâs navigation. As Nielsen Norman Group explains: âThere is great variability in different people’s mental models and in the vocabulary they use to describe the same concepts. We must collect data from a fair number of users before we can achieve a stable picture of the users’ preferred structure and determine how to accommodate differences among usersâ.
Evaluation method: Thereâs already a design, and you basically need to work out whether itâs a good fit for your users. Any major problems are likely to crop up even after testing 5 or so users. For example, you have a wireframe of your website and need to identify any major usability issues.
Basically, because youâll typically be using card sorting to generate a new design or structure from nothing, you need to sample a larger number of people. If you were testing an existing structure, you could get by with a smaller group.
Where to from here?
Following on from our discussion of generative versus evaluation methods, youâve really got a choice of 2 paths from here if youâre in the midst of a project. For those developing new structures, the best course of action is likely to be a card sort. However, if youâve got an existing structure that you need to test in order to usability problems and possible areas of improvement, youâre likely best to run a tree test. Weâve got some useful information on getting started with a tree test right here on the blog.
Try Online Card Sorting
Try card sorting now with OptimalSort. Find out how people think your content should be organized and categorized.