Open vs Closed vs Hybrid Card Sorting: Which to Use
Open, closed and hybrid card sorting each suit different research goals. Open sorts reveal natural category names and groupings; closed sorts measure agreement with predefined categories; hybrid sorts test your categories while staying open to alternatives. Here's what each tells you, when to use it, and how the analysis differs.
Quick Answer by Type
Open card sorting
When to use it: You're starting from scratch and need to discover how users think about your content. No existing categories.
What you get: User-created category names, natural groupings, cluster analysis showing patterns.
Closed card sorting
When to use it: You have candidate categories and want to validate them. Measures how well your structure matches user mental models.
What you get: Placement agreement rates, cards that don't fit, categories that are too broad or too narrow.
Hybrid card sorting
When to use it: You have some categories but want to stay open to alternatives. Balances validation with discovery.
What you get: Validation of existing categories plus any new categories participants create when yours don't fit.
Free plan: 3 studies. Unlock the full analysis for a single study for $15, or choose Starter at $19/mo.
1,800+ users
Comparison Table
| Open | Closed | Hybrid | |
|---|---|---|---|
| Participants create categories | Yes | No | Optional |
| Categories provided | None | All | Some |
| Best for discovery | Yes | No | Partial |
| Best for validation | No | Yes | Partial |
| Analysis output | Cluster analysis | Placement agreement | Both |
Decision Flow
Here's a practical sequence for most projects:
- Start with an open card sort if you're building new navigation or redesigning from scratch. Let participants create their own categories and groupings without bias.
- Analyse the results: Standardise category names, read the similarity matrix, and interpret the dendrogram to identify strong clusters.
- Design candidate categories from the open sort clusters. These become your navigation structure proposals.
- Run a closed card sort (optional) to validate your categories with a separate group of participants. Measure placement agreement to confirm your structure matches user mental models.
- Then tree test your final design to measure actual findability. Card sorting informs structure; tree testing validates it.
For redesigns where you're keeping some existing structure, consider hybrid card sorting: provide your current categories but let participants create new ones when yours don't fit.
Running Open Then Closed
Many teams run both methods in sequence on the same content:
- Open sort first: Reveals natural groupings, category names, and mental models without researcher bias.
- Analyse and design: Turn open sort clusters into candidate categories. Choose clear labels based on participant language.
- Closed sort second: Tests whether your interpretation of the open sort actually matches how users think. Measures placement agreement with a separate participant group.
Use separate participant groups for each sort to avoid bias. Someone who created categories in an open sort will have artificially high agreement with those categories in a closed sort.
Hybrid Card Sorting Tips
Hybrid sorting (some predefined categories, participants can create new ones) is useful when:
- You have some confident categories but suspect they don't cover everything
- You're redesigning and want to test existing structure while staying open to improvements
- You want validation data but also discovery data in one study
How to analyse it:
- Treat predefined categories like a closed sort: calculate placement agreement for each card.
- Treat participant-created categories like an open sort: look for patterns in the new groupings.
- If many participants create the same new category, that signals your predefined list was incomplete or poorly labelled.
Free plan: 3 studies. Unlock the full analysis for a single study for $15, or choose Starter at $19/mo.
1,800+ users
Frequently Asked Questions
Should I run an open sort or closed sort first?
Run an open sort first to discover natural groupings and category names without bias. Then, if you need to validate specific categories, run a closed sort with a separate group of participants. The open sort informs better category choices for the closed sort.
Can I run both open and closed sorts on the same content?
Yes, and this is common practice. The open sort reveals mental models; the closed sort measures whether your interpretation of those models (expressed as predefined categories) actually matches how users think. Use separate participants for each to avoid bias.
When should I use hybrid card sorting?
Use hybrid sorting when you have some confident category candidates but suspect they may not cover everything. This method lets you validate your categories while catching gaps. It's useful for redesigns where you're keeping some existing structure but want to stay open to improvements.
How do I analyse hybrid card sort results?
Treat the predefined categories like a closed sort (calculate placement agreement), and treat participant-created categories like an open sort (look for patterns in the new groupings). If many participants create the same new category, that's a signal your predefined list was incomplete.