When to Use Ranking
Ranking questions are ideal when you need to:- Identify priorities among options
- Force differentiation between items
- Understand relative preferences
- Determine what matters most
“Please rank these features from most important (1) to least important (5):” Drag to reorder:
- Price
- Quality
- Brand reputation
- Convenience
- Customer service
Ranking vs. Rating
| Aspect | Ranking | Rating (Matrix) |
|---|---|---|
| Forces choice | Yes - items must be ordered | No - all can be rated equally |
| Shows priorities | Clear hierarchy | Items may tie |
| Cognitive effort | Higher | Lower |
| Best for | True prioritization | Absolute evaluation |
Creating Ranking Questions
1
Add a Ranking Question
Use AI chat (“Add a ranking question for feature priorities”) or add manually.
2
Write the Question Text
Clearly explain what participants should rank and the ranking direction (most to least, etc.).
3
Add Items to Rank
Enter each item. These will appear as draggable elements.
4
Configure Settings
Set initial order, randomization, and whether all items must be ranked.
Best Practices
Limit the Number of Items
Ranking becomes difficult with too many items. Stick to 5-7 items maximum.| Number of Items | Difficulty |
|---|---|
| 3-4 | Easy |
| 5-7 | Manageable |
| 8-10 | Difficult |
| 10+ | Too hard - avoid |
Write Clear Instructions
Tell participants:- What they’re ranking
- The direction (1 = most or least?)
- How to perform the ranking (drag and drop)
“Drag the items below to rank them. Place the most important feature at the top (1) and the least important at the bottom.”
Use Comparable Items
All items should be:- At the same level of specificity
- Reasonably comparable
- Relevant to the participant
Consider Initial Order
The starting order can influence rankings (position bias). Options:- Randomize: Best for reducing bias
- Alphabetical: Neutral and consistent
- Fixed: Only if order is meaningful
Allow Partial Ranking
For longer lists, consider allowing “Top 3” ranking instead of ranking all items.“Drag your top 3 choices in order of preference”This reduces cognitive load while still capturing key priorities.
Interpreting Ranking Data
Average Rank
The most common analysis: calculate the average rank position for each item across all respondents. Example results:| Item | Average Rank |
|---|---|
| Price | 1.8 |
| Quality | 2.1 |
| Convenience | 3.2 |
| Brand | 3.9 |
| Service | 4.0 |
First Choice Analysis
Look at what percentage of respondents ranked each item #1. Example:- Price: 45% ranked first
- Quality: 35% ranked first
- Convenience: 15% ranked first
- Other: 5% ranked first
Rank Distribution
See how rankings spread across positions:| Item | #1 | #2 | #3 | #4 | #5 |
|---|---|---|---|---|---|
| Price | 45% | 25% | 15% | 10% | 5% |
| Quality | 35% | 30% | 20% | 10% | 5% |
Common Use Cases
Feature Prioritization
“Rank these potential new features:”
- Mobile app
- Faster shipping
- More payment options
- Extended warranty
- Loyalty program
Purchase Decision Factors
“When buying a laptop, rank these factors by importance:”
- Price
- Performance
- Battery life
- Design/weight
- Brand
Message Testing
“Rank these headlines from most to least appealing:”
- “Save money today”
- “Premium quality guaranteed”
- “Trusted by millions”
- “Free shipping always”
Competitive Preference
“Rank these brands from your most to least preferred:”
- Brand A
- Brand B
- Brand C
- Brand D
Combining with Other Questions
Ranking + Follow-up
After a ranking question, ask about the top choice:- “Rank these features…” (Ranking)
- “You ranked [top choice] as most important. Why is this feature most important to you?” (Open text)
Ranking + Rating
Use both to understand priorities AND absolute evaluations:- “Rank these features by importance…” (Ranking)
- “Rate your satisfaction with each feature…” (Matrix)
Common Mistakes
Too many items to rank
Too many items to rank
More than 7 items creates cognitive overload. Split into multiple questions or use “top 3” ranking.
Unclear direction
Unclear direction
Always specify whether 1 is best or worst. Ambiguity leads to reversed responses.
Non-comparable items
Non-comparable items
Items at different levels of abstraction can’t be meaningfully ranked. Keep items parallel.
Not randomizing start order
Not randomizing start order
Fixed starting positions create bias. Randomize the initial order.
Mobile Considerations
Ranking questions work differently on mobile:- Drag and drop may be harder
- Touch targets need to be large enough
- Consider simpler interactions for mobile respondents