Two days ago I presented this poster to the crowd at the IA Summit 2011 in Denver, Colorado for the Poster Session. We're trying to address two issues with Card Sort Analysis and this poster is a discussion piece for a proposed new algorithm for analysis. The two issues:
- Current methods for Card Sort Analysis are essentially qualitative. Although this is very useful, there are times when it is desirable to use a larger data set. Quantitative Card Sort Analysis using current methods is difficult, or damn near impossible with hundreds or thousands of results.
- Current visualizations for presenting Card Sort Analysis (dendrograms and similarity matrices) are not very helpful at showing alternate popular mental models that might come through in the raw data. Understanding alternate models can help you decide what to put in a sidebar or footer (for example) or provide valuable insights for second tier navigation or even copy writing. Traditionally you would need to wade through a spreadsheet to uncover these insights.
In short, we test each card sort result against all the others and come up with an "acceptability score" which represents the degree to which each participant agrees with the other results. In this way we can establish which particular results is most acceptable to the population, and from there, we can answer the question: "Of those who do not agree with this particular IA, how would they prefer to group the cards?".
We have already developed a working prototype of our Participant Centric Analysis Method and hope to integrate the new visualization into OptimalSort in the near future. We'd love to hear any feedback you might have on this new method.
Download the poster (PDF)