MaxDiff Analysis

MaxDiff (Maximum Difference Scaling) solves a fundamental problem in survey research: asking people to rank or rate a long list of items produces unreliable data. When faced with 15 or 20 features, participants struggle to differentiate between items in the middle of the list, and rating scales suffer from response biases where everything gets a 4 out of 5. MaxDiff sidesteps this by showing respondents small subsets — typically 4 or 5 items at a time — and asking them to select only the most important and the least important item from each set. Each respondent sees multiple subsets, and the experimental design ensures every item appears an equal number of times across all subsets.

The result is a rank-ordered list of preferences derived from many simple, cognitively easy choices rather than one cognitively demanding ranking exercise. Because the method forces trade-offs (you must pick a best and a worst), it produces cleaner differentiation between items than direct rating. The statistical output is a utility score for each item on a ratio scale, meaning you can say not just that Feature A is preferred over Feature B, but by how much.

MaxDiff requires specialized survey tooling with dedicated question types — platforms like Sawtooth Software or Qualtrics offer built-in MaxDiff modules that handle the experimental design and analysis. Running MaxDiff in a generic survey tool is technically possible but error-prone. For a broader discussion of structured decision methods in UX research, see Section 14.3 of UX Research: Building Blocks for Impact in the Age of AI by Marc Busch.