A measure of the magnitude of a finding—how big the difference is between conditions, not just whether it exists. Essential for determining practical significance beyond statistical significance.
Definition: A measure of the magnitude of a finding—how big the difference is between conditions, not just whether it exists. Essential for determining practical significance beyond statistical significance.
Effect size quantifies the magnitude of a finding—how much better one version is over another, not just whether the difference exists.
Statistical significance tells you a difference is probably real. Effect size tells you if it is big enough to matter.
A study might find a statistically significant difference that is too small to justify the cost of implementation. Conversely, a practically meaningful difference might not reach statistical significance in a small sample. You need both pieces of information.
Raw difference: The actual units of measurement (e.g., "12.5 SUS points higher")
Cohen's d: A standardized measure expressing the difference in terms of standard deviations:
Standardized effect sizes allow comparison across different scales and studies.
When reporting findings, always include both:
This practice is central to driving impact—it moves beyond a simple p-value to tell stakeholders whether a difference justifies the cost of development.
A determination that an observed result is unlikely to have occurred by random chance alone. Conventionally indicated by a p-value below 0.05, meaning less than 5% probability of the result being a fluke.
The probability of observing your data (or something more extreme) if there were truly no effect. Widely used, widely misunderstood, and never sufficient on its own to make a decision.
Research focused on numerical measurement with the goal of generalizing findings from a sample to a broader population. Answers 'how much,' 'how many,' and 'how often.'
This term is referenced in the following articles:
Validity is not one thing. It is a family of concepts, each addressing a different way your research can go wrong. A comprehensive guide to study design validity, measurement validity, qualitative trustworthiness, and modern unified frameworks.
An interactive sample size calculator for UX research, with the statistical foundations explained — from binomial problem discovery to power analysis.
The most powerful insights rarely come from a single source. They emerge from the strategic partnership between UX research and Data Science, fusing deep contextual understanding with patterns identified at massive scale.