A study structure where different groups of participants test different conditions. Group 1 tests only Version A; Group 2 tests only Version B. Eliminates order effects but requires more participants.
Definition: A study structure where different groups of participants test different conditions. Group 1 tests only Version A; Group 2 tests only Version B. Eliminates order effects but requires more participants.
A Between-Subjects Design assigns different, independent groups of participants to each condition of the study. For example, Group 1 tests only Prototype A, while a separate Group 2 tests only Prototype B.
This design is the correct choice when there is a high risk that seeing one version would unfairly influence a participant's perception or performance on another—known as an Order Effect.
A user who has just learned how to complete a task in Prototype A will likely be much faster when they see a similar task in Prototype B, not because B is better, but because they have had practice. Between-subjects design eliminates this problem.
Benefit: Eliminates order effects, ensuring a clean comparison
Drawback: Requires a larger total number of participants to achieve statistical power. Inherent differences between the participants in each group can introduce statistical "noise" that may mask the true effect of the design change.
If you need to compare two designs and learning or fatigue could contaminate results, use between-subjects. Budget for roughly double the participants you would need for a within-subjects design.
The choice between between-subjects and within-subjects design is one of the most important decisions in comparative research.
A study structure where the same participants test all conditions. Every participant interacts with both Version A and Version B. Statistically powerful but requires counterbalancing to control order effects.
A technique for controlling order effects in within-subjects designs by varying the sequence of conditions across participants. Half test A→B; half test B→A.
This term is referenced in the following articles:
The idea that you only need five users is one of the most famous, and most misunderstood, heuristics in UX research. Here is what the numbers actually mean and when they apply.
The structure of your study, who sees what, and in what order, determines what conclusions you can draw. Understanding the trade-offs between study designs is fundamental to research craft.