Reliability
The consistency of a research method—whether it produces similar results when repeated under the same conditions. About precision, not accuracy. A method can be reliable without being valid.
Definition: The consistency of a research method—whether it produces similar results when repeated under the same conditions. About precision, not accuracy. A method can be reliable without being valid.
Reliability refers to the consistency of a research method. A reliable method produces similar results if repeated under the same conditions. It is about precision: are your measurements consistent and free from random error?
Reliability vs. Validity
A method can be reliable without being valid—this is a crucial concept:
- Reliable: Your measurements are consistent (darts cluster together)
- Valid: Your measurements are accurate (darts hit the bullseye)
If your method consistently produces the same answer, it is reliable. But if that answer is consistently wrong, you have reliability without validity. Your darts cluster in the same spot, but it is not the center of the target.
Why Reliability Matters
Unreliable methods produce random noise. If the same study run twice gives wildly different results, you cannot trust either outcome. Reliability is the foundation—without it, you cannot even begin to assess validity.
Achieving Reliability
Reliability comes from standardization:
- Consistent protocols across participants
- Clear, unambiguous questions and tasks
- Trained moderators who follow scripts
- Documented procedures that others can replicate
The question to ask: "If someone else ran this study following my protocol, would they get similar results?" If yes, your method is reliable.
Related Terms
Validity
Whether a research method measures what it claims to measure. About accuracy, not precision. A method can be reliable (consistent) but not valid (accurate) if it consistently measures the wrong thing.
Objectivity
The degree to which research findings are independent of who conducts the study. If two researchers follow the same protocol and get different results, you have an objectivity problem.
Systematic Error
Consistent, predictable bias that skews results in a known direction. Manageable because you can account for it in interpretation—far better than random, unsystematic error.
Bias
Systematic deviation from the true value in research findings. Cannot be eliminated, only managed through standardization and awareness. The goal is systematic bias (manageable) over unsystematic bias (chaos).
Mentions in the Knowledge Hub
This term is referenced in the following articles:
UX Measurement Instruments: Scales, Scores, and What They Actually Measure
Standardized measurement instruments provide benchmarks and comparability. But using them effectively requires understanding what each one actually measures, and what it does not.
Research Quality and Managing Bias
You will always introduce bias into your research, that is unavoidable. The goal is not elimination but management. Understanding the difference between systematic and unsystematic error is what makes findings trustworthy.