Skip to content
ConductExperience
Menu

Ranking questions: when to rank and when to rate

A ranking question asks people to put a list of options in order, the first being the one they would choose or care about most. It shows which option a person puts ahead of another. A rating may not, since a person can rate every option highly.

Use a rating when you want how much people like each option, and a ranking when you want to know which they would choose first.

Try a ranking

This is the question as a participant answers it. A person moves an option with its Move up and Move down buttons, so a ranking needs no mouse, or drags it by the handle. The first move is the answer.

A ranking of 4 options

When you choose a coffee, what matters? Put these in order, the one that matters most first.

Put these in order, first at the top.

  1. Taste
  2. Price
  3. Strength
  4. Aroma

This is the question as a participant sees it. Nothing you answer here is saved or sent.

The row it writes

One row of the answers table, without its first columns, which name the person and the times. A blank cell means not answered.

matters__1Tastematters__2Pricematters__3Strengthmatters__4Aroma
blankblankblankblank

When to rank and when to rate

CompareRankingRating
What it showsWhich option a person puts ahead of another.How much a person likes each option.
TiesNone: every option has its own place.Allowed: several options can get the same rating.
Best whenPeople must choose among options, such as which features to build first.Each option matters on its own, or people may like them all.
Work for the personGrows with the list: each option added is one more to place.Small: one answer for each option, in a grid or a rating scale.
Rating scales, and scales of several statements

How many options

A ranking question takes up to 300 options. A short list is easier to order: each option added is one more to place, and the work grows quickly. If your list is long, ask for a rating of each option in a grid, or for a favorite in a single choice.

Where people start

Each person starts from the order you wrote, or from a random order of their own, which you choose with Shuffle options. A random start keeps the order you wrote from leading people.

A person who agrees with the order shown presses Keep this order, so every ranking is an answer the person gave.

What the download holds

Each questionnaire has an answers table with a row each time a person takes it. After the columns that name the person, the part and the time, it holds:

  • One column for each option, named <key>__<number>, such as matters__1 for the first option. It holds that option's rank: 1 for first, 2 for second, and so on.
  • A blank cell means the question was not answered or not shown.
  • data_dictionary.csv describes every column, and the download has SPSS syntax and readers for R and Python.

What Results shows

Results lists each option with its mean rank, with 1 for first, and the number and share of people who put it first, the best mean rank first. Filter it by version, group, status and start date. Excluded people are left out unless you include them, and one line gives the counts for the paper: started, finished, excluded by reason and analyzed.

Build a ranking question

  1. Start a study on the survey template.
  2. Add a questionnaire part and a Ranking question, and write its options.
  3. Choose Shuffle options if each person should start from a random order.
  4. Preview the study as a participant, and try the ranking with a keyboard.
  5. Publish the study and share the link.
The editor and its other question types

Limits

  • A ranking takes up to 300 options.
  • A ranking has every option once: no ties, and no option left out. A person is asked to put every option in order.
  • A ranking runs wherever a questionnaire runs: in the lab, by link, on a tablet and in check-ins.

Related

  • Likert scale

    Statements on one ordered scale, added up into a score.

  • Visual analog scale

    A line from 0 to 100 that people mark, and the number it makes.

  • Survey software

    The question types, rules, scores and languages of the questionnaire editor.

  • Data export

    The answers table for each questionnaire, a data dictionary, and files for SPSS, R and Python.

Planning a study?

Start one now, or tell us what you want to measure.