According to Information Architecture for the World Wide Web, card sorting "can provide insight into users' mental models, illuminating the way that they often tacitly group, sort and label tasks and content within their own heads."
Is a card sort right for you?
If you're an Information Architect or User Experience professional tasked with organizing a large system, or developing the navigation for a website, card sorting can help. Card sorting provides quantifiable data about how your users understand and group content.
• Provides data to aid in decision making
• Fast and cheap method of user testing labels and structure
• Helps vet navigation schemes
• Input into content type definition and information architecture for large Content Management Systems
Open vs Closed Card Sort
The basic premise of a card sort is that you provide users a list of categories and a list of items - allowing them to group the items in a way that makes sense to them. There are primarily two types of card sorts – "Open" and "Closed". Open card sorts allow you to illicit item grouping and labeling ideas, and closed sorts help test and fine-tune a known taxonomy.
Open Card Sort
For an open card sort you don't define any categories to test – only the items. This allows you to see where users logically group the items. Open card sorts are useful when you want to see where users think the items should go without leading them. You can also allow users to name the groups they create, which may provide some guidance when you start creating your taxonomy. I normally use open card sorts as a preliminary input to closed card sorts, but will often just start with the closed card sort.
Closed Card Sort
For a closed card sort you and your team define the categories and the items. Your main goal with a closed card sort is to test the categories and groupings.
Traditionally card sorts were conducted using index cards and a group of test participants in a room. Paper based card sorting allows users to work together and talk while grouping the index cards. This can be beneficial for observing the thought process of users and often eliminates some skewed results when people are able to think together about the system.
While paper testing is still a great way to gather insightful data, many Information Architects are moving card sorts online. Online card sorting allows you to test more users in remote locations and also allows for faster collection and analysis of results. For the purposes of this article, we'll focus on online testing.
We currently prefer to use the online sorting tool Websort.net. They provide free access for three studies up to ten participants. Upgrading to the paid version allows for an unlimited number of participants. I normally recommend testing between 30 and 100 users depending on the complexity of the sort and their knowledge of the subject matter.
Conducting the Card Sort
Conducting a card sort is easy, but the art comes in choosing what you are going to test and analyzing the results. Before you can setup your experiment you need to define the categories (for a closed card sort) and items you want to test.
Do your research first!
• Conduct interviews with stakeholders and users about how they do their job, use the existing system and their current pain points
• Analyze the existing content (painful, but very important) through content inventories and audits
• Study any available analytics (search logs, page hits, etc…)
Take the time to brainstorm with your UX team and stakeholders about how you feel the content goes together. Many times you'll find that once you've done your research the content will start to self-organize. At the very least you'll have a place to start.
Picking the categories and items
For a closed card sort we look to test between 5 – 9 categories and somewhere in the neighborhood of 30-60 items. The number of categories and items can vary, but try to create a test that won't take too long, or be overly complex. Users tend to start losing interest after about 15-30 minutes for an online card sort. For a good barometer – take the test yourself.
For an open card sort the users are helping you define categories by grouping like items together. For a closed card sort you can define the categories based on your previous research. Example categories for a company Intranet might look something like Human Resources, About the Organization, News, Support Services, Application Tools.
To select the items you want to test look for primary use cases, existing pain points and common tasks. We like to include a few baseline items that logically seem to fit existing categories and also a few tough items, but the goal isn't to trick the users, or to create great results you can show-off to your client. You want to test your categories and create a logical taxonomy that your users will understand. When choosing items don't lead the user buy including the name of a category in the item name. Example items for a company intranet might look something like Company Locations, History of Company, Management Information, Benefits, Cafeteria Menu.
If you are using an online tool like Websort enter your categories and items into the tool and choose your test parameters in the settings.
The tool is fairly straightforward, but to choose if you want an open or closed card sort you need to select the correct option on the categories tab. Selecting "Allow creation of new categories" will give you an open card sort.
The card sort is now ready to send to your users. Websort provides a link that you can email with a short message about how they conduct the test. We normally try to personalize the message to the type of system and user base.
Try to send the test to between 10 and 50 users, depending on the complexity of the study. Be sure to select people who represent the demographic of your users. If most of your system's users are older and don't have much experience with computers - don't send the test to all the young interns.
Analyzing the Results
Analyzing your results and making adjustments to your study takes practice, but one of the nice things about using a tool like Websort is that it provides several views of your results. You're looking for categories and category items that aren't testing well.
Two useful result views are the Category to Item view and the Item to Item view. The Category to Item view shows you what percentage of time a user places an item in a specific category, and the Item to Item view shows you the percentage of time users are placing items together.
What to look for
• Heat maps – darker areas are testing well, while lighter areas aren't
• Categories that users don't seem to understand
• Item labels that may be misleading users to place them in different categories
• Categories with too many items
• Categories with no items
Depending on the results of your sort you may need to refine the categories or items and do a retest (preferably with different users), or you might find that your categories tested well. In any event, you now have quantifiable metrics to help you make decisions.
Hopefully this post has given you a better understanding of card sorting. Over my years in UX, it's definitely been one of my weapons of choice when organizing large amounts of content or trying to test a new navigation scheme.