A controlled experiment comparing two variants by randomly splitting users between them. The only reliable way to measure the causal impact of a specific change on user behavior.
Definition: A controlled experiment comparing two variants by randomly splitting users between them. The only reliable way to measure the causal impact of a specific change on user behavior.
A/B testing splits your users into two groups—one sees version A, the other sees version B—and measures which performs better on a defined metric. It is the gold standard for causal inference in product decisions.
A/B tests answer "which is better" but not "why." If your conversion rate drops 15%, an A/B test tells you the new design caused it. It does not tell you what confused users. You need qualitative research for that.
A/B tests also require sufficient traffic. If your sample size is too small, results will not reach statistical significance and you are guessing with extra steps.
The percentage of users who complete a desired action (e.g., purchase, sign-up) out of the total number of visitors.
A determination that an observed result is unlikely to have occurred by random chance alone. Conventionally indicated by a p-value below 0.05, meaning less than 5% probability of the result being a fluke.
The number of participants in a research study. Appropriate sample size depends on research goals, method type (qualitative vs. quantitative), the precision required, and the number of distinct user segments being studied.
This term is referenced in the following articles:
An interactive sample size calculator for UX research, with the statistical foundations explained — from binomial problem discovery to power analysis.
One of the most common points of friction is not about budget or methods, it is about timing. Your core job is to reframe research from a single, disruptive event into a continuous, value-adding loop.
UX research is not a luxury or a checkbox, it is a systematic process for reducing uncertainty and driving measurable business outcomes. Here is how to frame its value.
Market research, UX research, CX research, product research, are these different things? At their core, they are all related methods for gathering data to reduce uncertainty. The key is understanding what each is best suited for.
Rather than a sharp divide, qualitative and quantitative research exist on a continuum. The most powerful insights come from combining both, understanding why something happens and measuring how often.
There are two fundamentally different ways we gather data. Research we design and control, and data users generate without our prompting. Most teams over-rely on one and misunderstand the other.