Information architecture (IA) is everywhere. It’s not all about digital — the term does after all refer to the structural organization of any shared information environment. Shared information environments are all around us: the way the shelves are organized in the supermarket, the way exhibits are organized in a museum and the carefully considered design of a shopping mall. Another great example is a library. There are several systems that libraries around the world follow to structure their content and in this two part series, we’re going to take a closer look at two of the most common: Dewey Decimal Classification (DDC) system and Library of Congress Classification (LCC) system. There are a number of differences between the two and usage depends entirely upon the size of a library’s collection.
The DDC system is great for organizing smaller collections like those seen in public libraries or schools, while the LCC system works best with large and diverse collections such as university libraries. Check out this awesome infographic that compares the two systems.
The DDC system was designed in 1876 to broadly organize all the world’s knowledge into 10 classes. Each DDC class has up to 10 divisions underneath and each one of those divisions has up to 10 sections. Classes are like website categories and the divisions and sections are like subsets of categories. The DDC system also arranges content by discipline rather than by subject — so a topic like ‘shoes’ would appear in multiple classes based on disciplinary context, e.g., footwear design would be in Class 700 while content covering social customs related to footwear would appear in Class 300. While the DDC system has many classes, divisions and sections they are still quite broad and high level which limits the number of unique call numbers that can be assigned to individual works. Call numbers are used to identify individual works and show exactly where they are kept in the system down to the section level.
From an IA size perspective, the DDC system is 10 categories wide and only 3 levels deep while the LCC system is 21 categories wide and up to 7 levels deep in some parts. Taxonomically speaking, the DDC system’s broader classes mean that multiple high level subjects get grouped together into some classes while in the LCC system, many of these are a class of their own. For example, content related to ‘Medicine’ in the DDC system lives under: Class 600: Technology > Division 610: Medicine & Health, while under the LCC system it’s its own class: Class R - Medicine and has 17 subclasses underneath.
Since IA isn’t just about digital, neither are the Optimal Workshop suite of tools. They’re actually quite flexible and can be used to research many different types of IAs. I’m going to talk you through what happened when I ran a closed card sort on the DDC system using OptimalSort — with a twist.
IA: Dewey Decimal Classification (DDC) system
Research scope: The DDC system
Not in scope: The physical library environment
For this study, I decided to use OptimalSort to run a closed card sort study with image based cards rather than text based labels. OptimalSort is our online card sorting tool that shows how your users expect your content to be grouped (taxonomy).
A closed card sort seemed like a good fit because it only has 10 classes and I wasn’t looking to redesign it — I was curious to see how an information system that is almost 150 years old would perform in a digital world where books aren’t always made of paper and found on a shelf. Personally, I love libraries and print books: the weight of them, the scent of the pages and the feeling of collapsing into a large comfy, velvet upholstered chair to revel in the crisp sound of page turning. But I do recognize we live in a world where we consume this type of content in many, many different ways.
I chose image based cards for this study because I wanted to see where participants thought specific books would belong in this system and the cover art of these books presented a great opportunity to communicate this visually. Instead of asking them to place cards with the title and author in text, I thought it would be more interesting and relevant to the subject matter to use an image of the cover. I also wondered just how much of an impact the cover art would have on their responses.
Preparing for the study
Choosing the categories
Because this is a closed card sort, the categories are predetermined and in this case they were the 10 DDC classes.
Through desktop research, I found out that there are a number of inconsistencies around the class names between different libraries and different sources and ended up needing to make a choice for the purposes of this study. After comparing several sources, I settled on the below 10 classes for my categories because they seemed to be the most common:
- Class 000: Computer science, information & general works
- Class 100: Philosophy & psychology
- Class 200: Religion
- Class 300: Social sciences
- Class 400: Language
- Class 500: Science
- Class 600: Technology
- Class 700: Arts & recreation
- Class 800: Literature
- Class 900: History & geography
Choosing the cards
Choosing which books to include as cards in this study was an interesting process that required a lot of cross checking. I had to be sure I covered all 10 classes and the book titles I chose had to be non-fiction works to fit into the system — works of fiction are out of scope for this study because they organized in a separate part of the library and in alphabetical order by author surname rather than under the DDC system.
I started scouring the internet’s numerous bestseller and top 100 lists and chose titles that were likely to be well known. I wanted to avoid adding in an extra layer of ambiguity if possible and stuck to popular titles.
I copied and pasted them into a spreadsheet (see below) and cross referenced the titles with OCLC’s (Online Computer Library Center) WorldCat look up tool to obtain the OCLC identification number. That number was then entered into the OCLC’s Classify tool to obtain the call number to ensure I was getting good coverage of the classes.
My book card spreadsheet
When choosing cards for a card sort, it’s a good idea to include between 30 and 60 cards. Anything lower than 30 just isn’t worth it and anything over 60 is just going to annoy people and boost the study’s abandonment rate which you really don’t want. I decided to include 40 book cards in this study to strike a good balance. As for those purple squares you’re seeing in my spreadsheet above, those were the cards that I suspected might trip people up because they look like they could belong in multiple classes.
With the Optimal Workshop suite of tools, you can add post-study questions that appear at the end of the study after participants have completed the activity. I include them whenever I want to gain more information or some insight into the activities and thoughts of my participants. It’s no substitute for talking to users in person, but it can be useful for gaining additional context. For this study, I decided to ask one post-study question: ‘When was the last time you visited a library?’, because I was curious to see just how recently my participants had set foot in a library. I set it as a multiple choice radio button response based question and included an ‘Other’ option in case anyone wanted to provide more information. Participant responses were not marked as ‘required’ giving people a choice in whether they wanted to share that information.
Building the study
All the tools in the Optimal Workshop suite have a guided study-build process that make it quick and easy to get your research off the ground and out into the world. Because this study had picture cards, I needed to source images of the cover art for each card. Some of these books are very old and have been published numerous times over, so in most cases I had a quite a bit of choice around which cover to use. I focused my efforts on finding cover art that communicated the title of each book clearly and with good contrast to ensure my study was as accessible as possible. All images were sourced via Google Images and were a mix of JPEG and PNG files.
Once I had my images, I uploaded them into OptimalSort and gave each one a descriptive label (see below). For image based cards, these labels are hidden by default but you can toggle the feature off and show them to your participants. I decided to keep them hidden and let my images do the talking — if someone handed you a book and told you to shelve it, there wouldn’t be an additional label accompanying it! Useful for some cases, just not this one. I gave each card a label that makes sense to me because they’ll make the results analysis process much easier!
Image based cards and their labels in OptimalSort during the build process
Next stop was the Categories tab to set the card sort type and copy and paste the 10 DDC classes in as my categories (see below).
From here I made some minor tweaks to OptimalSort’s default participant messaging. They’re great but they mention that the study is researching a website which isn’t the case this time. It’s an easy edit to make and you can tweak the text to suit whatever you’re researching without having to think too hard.
Before launching any study, I always check it over in preview mode (see below) first — just to be sure I haven’t made any embarrassing errors!
Preview mode for this OptimalSort study
Preview mode shows you exactly what your participants will see while completing the study, but it doesn’t record any data. I didn’t spot any errors in this study preview and the picture cards were working exactly the way they should!
For this closed card sort study on the DDC system, I used the Optimal Workshop recruitment service which can be accessed via the Recruitment tab in OptimalSort. This service has proven to be very reliable for me and usually has a very quick turnaround time — at most, it’s only ever taken a few hours for the completed participant responses to land in my dashboard. The service allows me to specify age, gender, location and more, providing easy access to research participants from my user group that I wouldn’t be able to obtain for myself in the same timeframe and with the same level of certainty around suitability. The recruitment brief for this study was 50 participants with an equal mix of genders all residing in the United States.
My OptimalSort closed card sort study on the DDC system had 59 participants in total — 51 completed the study and 8 abandoned it. Abandonments are recorded when participants close the tab or window of the study without submitting it. It can happen for a number of reasons – sometimes people will open it and then close it down to come back to it later! High abandonment rates (upwards of 25%) can indicate that the study had too many cards or would take participants too long to complete but in this case, a 13.5% abandonment rate is normal and nothing to worry about.
Before jumping into results analysis, I like to take a quick look at the Participants tab in OptimalSort. It allows me to flick through each individual participant sort and gain an overall sense of the results. It’s also an opportunity to exclude any of the participants from the results, such as people who lumped everything together into one category and didn’t actually sort the cards! For this study, I did not see any evidence of this irritating behavior, so it’s time to interpret the results!
Interpreting the card sort results: Results Matrix and the Popular Placements Matrix
When interpreting the results of a closed card sort in OptimalSort, there are two data visualization matrices that do a lot of the heavy lifting for you: the Results Matrix and the Popular Placements Matrix. The Results Matrix (see below) shows the number of times each card was sorted into the corresponding category and highlights the numbers in various shades of blue. The darker the shade of blue, the more times that card was sorted into a particular category.
OptimalSort’s Results Matrix
I find the Results Matrix easy to read because I can skim over it quite quickly and interpret the results.
The other results visualization in OptimalSort that I find really useful is the Popular Placements Matrix (see below 2 images).
Popular Placements Matrix in OptimalSort, top half of the diagram
Popular Placements Matrix in OptimalSort, scrolled to show the bottom half of the diagram
The Popular Placements Matrix shows the percentage of participants that sorted each card into a particular category and makes it easy to spot the clusters by shading the most popular placements in blue. It also highlights the categories that didn’t get a lot of card sorting love and in this study that would be Class 600: Technology — more on that one soon. The percentages used in the Popular Placements Matrix make it quick and easy to communicate findings to stakeholders; nothing says we need to conduct more user research like “X% of participants sorted Card 1 into Category A when that content currently exists in Category Z”.
A really cool feature about the Optimal Workshop suite of tools that I haven’t told you about is the results sharing functionality that can be accessed via the ‘Sharing’ tab (see below) in all of the tools.
The ‘Sharing’ tab for this OptimalSort study
It allows you to share your findings and the demonstrate the results analysis options to stakeholders and team members in a really meaningful way - they get to see what the researchers see! Sharing is turned off by default but you can go in there and choose from a range of sharing options ranging from public to password protected links. You can also decide how much you’d like to share. I’ve decided to share all of my tabs and publicly here so you can have a play with the results from this study yourself and see what I mean!
Key findings in this study
Diving into the Results Matrix, I can instantly see that those cards I flagged in purple did indeed trip people up but they weren’t the only ones!
One of my favorite things about the DDC system is the existence of Class 000: Computer science, information & general works. It includes encyclopedias, magazines, newspapers and works that you usually can’t leave the building with, but the coolest part (in my opinion at least) are the divisions covering information systems and library science! Naturally, I included two cards in this study on the topic of libraries: “This book is overdue! How librarians and cybrarians can save us all” by Marilyn Johnson and “The meaning of the library: a cultural history” by Alice Crawford. In this study, Johnson’s book was sorted into 7 different categories (see below) with the most popular of those being Class 800: Literature which it was sorted into by 39% of participants. Only 9 people (18%) sorted that card into Class 000.
Results Matrix in OptimalSort
I can easily see how participants would connect a title containing the word ‘book’ to a category called ‘Literature’ but it does make me wonder if people are aware of that magical Class 000 that library nerds like me so admire. As for Crawford’s book, in this study it was scattered across 9 of the classes and its most popular placement was in Class 900: History & geography. Given that the DDC system arranges content by discipline rather than subject, it’s not too much of a stretch that a book on the cultural history of libraries would be in the history division. I looked up the call number just to be sure and it is: 027.009 placing it in Class 000. If my understanding is correct, perhaps there are some exceptions to the discipline rule.
Another interesting finding that caught my eye was the placement of “Steal Like an Artist: 10 Things Nobody Told You About Being Creative” by Austin Kleon. With the call number: 153.35, its current DDC system home is Class 100: Philosophy & Psychology. It’s a really interesting book on creative thinking and is shaped around the the things the author wished he’d been told when he was starting out. In this study, this particular card was sorted into 6 different categories and the most popular was Class 700: Arts & recreation with 67% of participants placing it there. Much like with Crawford’s book, ‘artist’ belongs in ‘Arts and recreation’ right? It looks like it belongs there, but successful use of the DDC system requires a deeper level of understanding around its structure. It seems like a lot of pattern matching came into play when participants were deciding on card placements in this particular study.
As for the DDC classes themselves, through the OptimalSort results matrices it’s clear that ‘Technology’ wasn’t a popular category choice for any card in this study. Anytime this occurs in the results, it’s possible that there is some ambiguity around that particular category name. This study contained 5 book cards taken from this particular library class:
- “The Making of the Atomic Bomb” by Richard Rhodes
- “Alcoholics Anonymous” by Alcoholics Anonymous
- “Gray's Anatomy” by Henry Gray
- “Nigella Bites” by Nigella Lawson
- “My Life in France” by Julia Child
As for where participants sorted them, they were mostly scattered across literature, history and science. “The Making of the Atomic Bomb” was primarily split between Class 500: Science and Class 900: History & geography with 18 and 21 placements respectively. Alcoholics Anonymous’ “Big Book” was also mainly split across 2 classes with 23 placements in Class 300: Social Science and 17 in Class 100: Philosophy & psychology.
Human anatomy textbook, “Gray's Anatomy”, was sorted into Class 500: Science by 78% of participants (40 times). Lastly, Lawson’s book landed in Class 800: Literature 24 times and Child’s book shared a similar fate with 30 placements in that same class.
As someone familiar with the Dewey Decimal Classification system I know that ‘Technology’ can be a confusing because it contains a wide variety of content that includes topics on medicine and food science — it’s a bit of a catch all for content that doesn’t quite fit into the other classes. As mentioned earlier, some libraries use slightly different names for their classes and I’ve seen this one appear as ‘Technology & applied sciences’. These results appear to support the case for at least exploring the merits of that alternative further!
Other findings from this study
- The 3 books in this study taken from Class 200: Religion were the least scattered across the classes and achieved some of the highest levels of agreement – participants agreed with their current home in the real world. Potentially due to keywords in the titles.
- 82% of participants in this study placed Truman Capote’s “In Cold Blood” into Class 800: Literature when it currently resides in Class 300: Social Sciences.
- “I Know Why the Caged Bird Sings” by Maya Angelou was the card that scored the highest level of agreement in this study with 96% of participants also sorting it into Class 800: Literature when it actually belongs in Class 900: History & geography.
- 90% of participants in this study had no trouble correctly placing my trusty university sidekick “Bescherelle: Bescherelle 12 000 Verbs. Complete Guide to Conjugating French Verbs” into Class 400: Language.
Post-study question results
My post-study questionnaire revealed that 47.1% (see below) of my participants last visited a library over a year ago and that just 16 of my 51 participants have been within the last 3 months. I’m not surprised by these results; life is just so busy, but in hindsight I wish I had included further questions to try to uncover why this was the case.
Next steps for a study like this one
This study was a lot of fun to run but it also left me with quite a few cliffhangers that really illustrate the value and need for running studies like this one in parallel to face to face moderated user research.
A lot of my questions went unanswered. I have no way of measuring the impact that the image based cards had on my participant responses and I don’t know exactly why ‘Technology’ was left out in the cold. I also don’t really know what impact this might have on the library experience; are people wandering around the shelves and not being able to find what they need? How dependent are library users on cataloguing tools? To what extent are the valuable services of librarians being utilized?
I also don’t know why my participants haven’t been visiting libraries. I have my theories but I could be wrong and I could also be missing a key piece of contextual information that would change everything. That’s not to say there isn’t value in conducting studies like this one or closed card sorts in general, just that like my results, the process is incomplete. This study is a fantastic starting point but there is more work to be done.
Using this study as an example, to complete the picture you would need to run some in person card sorts with printed image based cards. You could print the book covers out and then use OptimalSort’s printed card option and stick the barcodes on the back of the covers so you can take advantage of the tool’s analysis functions without distracting your participants. Or you use actual books instead of cards and just slip the barcodes inside! An even better idea would be to increase the scope of the research by heading to an actual library, picking say 10 books off the shelves and then asking your participants to shelve them in that contextual environment. You would shadow your participants along the way and observe how they interact with the physical library space and ask questions along the way to build your understanding of their thinking. You could find out exactly why ‘Technology’ was so overlooked and understand reasons behind it. No more unsatisfying cliffhangers — and that’s just this study. You could easily apply the thinking for this example to just about any design process involving an IA.
5 key IA lessons we can all take away from this study
The DDC is 141 years old and has endured as one of the most commonly used library classification systems in the world. The results of this one closed card sort study isn’t going to change it but that’s not to say there isn’t anything to be learned from this study beyond satisfying my curiosity. Here are some key IA lessons we can all learn from this one and apply to our work:
- Be mindful of the use of potential ‘catch all’ categories and conduct thorough user research both remotely and in person to ensure this is the best possible solution for your users
- Always be open to the possibility of alternate labels and invest time in developing a robust ontology for your IA
- Remote user research studies definitely add value but should be run in conjunction with face-to-face moderated user research to gain context, ask questions and understand user needs, wants, behaviors and goals
- Non-digital IAs can be researched using the same tools and methods you would to research digital ones by slightly adapting your approach and ensuring a balance between remote and face-to-face studies
- Closed card sorts are best used as part of an overall program of user research as their ability to gather user insights is limited. Use them as a starting point to help formulate questions to be explored and answered through further research or as a final check up before launching a live product.
Keep an eye out for Part 2 where we will be discussing what happened when the LCC system and 10 of the book based cards from this study met up with Optimal Workshop’s Treejack.