97 tags, 37 categories, 172 remote participants: How we're taming our blog with card sorting

taming_blog.png

Do you know that we've published a whole bunch of very cool content over the last six or so years? No? Well, I don't blame you, esteemed reader, because neither our blog homepage nor the post pages themselves show you how to find it. So we're creating a new blog structure to bring our archives to light and to house all the new writing we have lined up.  We've run an open card sort using OptimalSort to gather data on what how people group and categorise our blog tags (or topics). As of right now, 172 people have completed the card sort (what a thrill!) so we've got plenty of data to help us make evidence-based decisions. This piece is a look behind the scenes of our project, including a focus on:

  • what a 'blog' actually is, and how blog purpose and information architecture are linked
  • how four well-established online publications arrange their content
  • what cards we included in the card sort
  • the most interesting, relevant results from the card sort itself and the questions we asked participants
  • what comes next.

What do you think of when you hear the word 'blog'?

Updating our blog has got me reflecting on what the word 'blog' actually represents. I've concluded that it's a chronically ambiguous word that has wildly contrasting connotations — from 'authoritative industry voice' to 'angsty, narcissistic rants', from Medium-chic 'thought-leader shares thoughts' to anything that comes under the dubious genre of 'content marketing'. And this ambiguity can makes setting a blog strategy quite tricky (and very important). The responses to the pre-study question on our blog IA card sort illustrates this ambiguity. Some objective and apt responses include:

'A site or section of a site with fresh content that is produced by an individual or individuals reflecting their interests or the nature of the work they do.' A series of short, narrative articles about timely updates to a theme, typically arranged reverse chronologically. Usually tagged at a high level but not very searchable.' 'Articles. Last one first. Comments. Interaction between writer and readers. Share options. Not meant to last: news like.' — 'Brief updates and missives in the voice of the organization, or individual people within the organization. More personable and first-hand than press releases or official statements. More lighthearted, timely, and informal.' 'A thought piece more substantial than a tweet or FB status but less involved than a New Yorker article.'

And a few that betrayed a wariness towards the practice of blogging:

'Too much to read, a bit self promoting, sometimes boring' 'Weblog personal fragments. Mostly a waste of time. Sometimes extremely valuable.' 'The online diary of a mostly pathetic person or where companies can place advertisements disguised as helpful articles.' 'I fucking hate that word.'

Well, OK then. (I hear ya brother *wink*)

Information architecture can be your blog purpose

If you write a company blog, you'll have your overall strategy (to amplify your brand, draw people to you, be a part of the community, and so on). And you could write this explicitly on your landing page so that reader's know where they're at. But another way you can make clear your purpose is to limit your categories and tags and make it easy for readers to glance through them quickly. Information architecture in this case — how you label and structure the content on your blog — is purpose. Your labels and categories can only represent the type of content you publish. You won't create a category called 'Gardening in Denim' if you haven't (or don't plan to) publish blogs related to that topic. The same goes the other way. If 40% of your articles are about beekeeping, it follows that one of your main category labels will have the word 'Bees' in it somewhere. If you get your taxonomy right, it does two things: it tells your readers who you are and what you're about. And it helps you to keep your editorial strategy in line. A couple of our industry's most widely-admired online publications show how this works in practise. Smashing Magazine arranges their content into just six main categories: Coding, Design, Mobile, Graphics, UX Design, and Wordpress. Though each category contains content tagged with more than four topics, they've chosen to display only the top four tags for each category on the left menu. smashmag1 And each category has its own landing page that contains a brief description of the content it holds, and links to the main tags under that category (damn I love this page): smasg Nielsen Normal Group publish content in a section called 'Articles', and their menu on the left contains three categories which, instead of providing an umbrella for content, offer readers the chance to browse by Topic, Author, or Most Popular article. nng1 UXPA Magazine landing page reflects how printed magazines present their content. You see the latest theme, and it's clear that the content you see is a part of that one issue. uxpa1 And when you scroll further down, you can start browsing by content. uxpa2 Boxes and Arrows also have a useful list of categories on the left. boxes and arrows And then on the landing page of the post, you can see more information about the author, and can click on links to more of their posts, or to the Most-commented articles on their website. Boxes and Arrows2 Now, of course we could debate the terms and taxonomy they've chosen to represent their content from now until earth is vaporized by the sun in 7 billion years. But what's clear is that at some point they've said 'We're done. This is us. This works, for us and our readers.' And that's what we want to achieve for our blog.

Creating the card sort and exploring the results

First, an admission. Our blog currently has 97 tags, and (wait for it...) 37 categories. How intense is that?! So before we created the cards we removed irrelevant and dated tags, and any that were strangely specific (like 'Cute design' haha). To refine the tags further, we created a draft study with 61 cards, and asked a few colleagues to check out the preview and tell us if they thought any other tags should be excluded. So our final study had just 44 cards (not our final set, but a good start point nonetheless). card sort cards Though it seems a small-ish number of cards, we knew we needed an audience clued-up on the UX industry because of the terminology. (Mila, one of our kick-ass developers, pointed out to me that for her, trying to sort these particular tags into categories would be like me trying to sort Ruby-on-Rails terminology). My partner-in-UX-crime, Alan, sent out the request via email and Twitter, and because the sort would require some effort, offered an incentive.

A glimpse into the quantitative results of our card sort

We were really lucky to receive so many responses from people, especially considering it took the 172 people an average of 12.6 minutes. So thanks to those of you who contributed  — awesome! I went straight to the Categories section of the results to see what ideas people came up with, and found myself face to face with 1,133 of the things! Hardcore. But standardizing the categories was pretty simple for the most part. For example, I called this category 'Business' (yeah, clever, I know). results 2 After standardizing, I explored the Participant-Centric Analysis, which shows the top three IAs with the most agreement between participants (and three alternative category labels suggested by other participants who agree with the same grouping. This one shows that 62/172 participants created IAs that were similar, and could be a useful starting point for the categories we create. PCA card sort The Actual Agreement Method dendrogram shows cards that were actually grouped together by a percentage of participants. A few interesting results include that fact that 53% (represented by the vertical light gray line) of participants grouped these cards together into the corresponding categories: dendrogram11 92% of participants grouped these two cards cards together: dendrogram12 67% of participants grouped these cards together: dendrogram13 And 52% of participants grouped these cards together: dendrogram14 These agreements are reflected in the Similarity Matrix as well, which shows the percentage of participants who agree with each pairing. The darker the shade of blue, the higher the agreement, so even a quick glance shows us that some cards are destined to be together: similarity matrix 6

A few insights from people about the activity and the cards themselves

On top of the rigorous quantitative data we've gathered, we also received many considered responses to the post-activity question (which we deliberately left open). These responses are absolutely invaluable, and I'm pleased we were able to strike a great balance between big numbers and what people actually think. Here's a few responses that will influence both how we move forward with this project and how we could improve on the card sort.

How was your experience of doing this card sort? Do you have any thoughts you'd like to share about the topics and/or the groups you've created?

"I always find it difficult when there are really broad terms included that I want to use as labels. I never know what to do with those (e.g. user research methods)."

"Many of the cards were waaay ambiguous, like "business" I wouldn't know what kind of posts would be found under a tag like that, so I can't imagine that it would be a useful one to use. many others were similar."

"I wasn't sure what the developer content was about - was it about working with developers? Articles FOR developers? I left that out in its own group for now."

"It was more challenging for me to sort these items than I had expected. One reason is because some of the cards were titles I would have given the categories, so it was strange for me to have a grouping called 'information Architecture" with a card of the same name within the category."

"I grouped the topics as if I were to group all the topics into teams working towards a product. The range in topics was strange. There were a lot of things on UI/UX-InfoArchitecture, and then there were a lot things that didnt have much in common with it."

"The weight and scope of some items seemed inconsistent/nonparallel, which complicated sorting. Also, the lack of descriptions for any item left some open to interpretation/ambiguity. But, no doubt, those are some of the real-world challenges you're facing when trying to categorize blog posts, which often are applicable to multiple categories or are one-off subjects."

"There were some labels that were not clear. Like reviews. Did you mean reviews of interfaces or did you mean article reviews?"

"It was difficult because there were so many topics. I also felt that I really big one was missing. Usability! and Moderated and Unmoderated. Also Best practices. Usability Research, Usability Testing."

"Challenging to sort when terms involve different dimensions/facets such as an industry sector (banking), function (e.g., marketing), research techniques (e.g., card sorting) and website structures (e.g., information architecture). Ultimately, I suspect what is needed in a multi-faceted taxonomy with filtered search capabilities."

"That wasn't easy :-) Good luck!" We'll take that luck with bells on!

So...what will our new blog structure look like?

Well, we're not sure yet, but it will involve a synthesis of three key things.

  • Our overall strategy to publish practical, high-quality writing within a taxonomy designed for our readers
  • The quantitative and qualitative results from our open card sort of 172 people
  • The bits we love from the best online publications in our industry (it's called inspiration, not stealing)

Feel free to write any ideas or thoughts (even suggested category names) in the comments. And whichever direction we take, we'll always be keen to hear your feedback (good and bad). We're creating this new blog structure for you. Hang tight for a follow up blog on the launch of our new design.

Published on Feb 24, 2015
Kathryn Reeves

Blogs you might also enjoy