If you’ve ever had a supervisor scrawl “but is this valid?” in the margin of your methodology chapter, you already know that validity and reliability are the two pillars every research instrument is judged against. Get either one wrong and your findings — no matter how interesting — won’t survive peer review, a thesis defense, or an NGO funder’s due diligence. This guide breaks down exactly what each term means, how they differ, and how to actually test for them in your own study.

Validity vs. Reliability: The Core Difference

Reliability is about consistency. If you gave the same test to the same person under the same conditions, would you get the same result? Validity is about accuracy. Does the instrument actually measure the concept it claims to measure?

The classic analogy is a bathroom scale. If it reads 68 kg every time you step on it within the same minute, it’s reliable — it’s consistent. But if your true weight is 63 kg, the scale is not valid — it’s consistently wrong. This is the single most important idea in this entire guide: a measure can be reliable without being valid, but it can never be valid without first being reliable.

Why Validity and Reliability Matter for Your Research

Whether you’re running a regression in SPSS, coding interview transcripts for a thematic analysis, or designing a survey for a social impact assessment, three audiences will scrutinize your instrument’s validity and reliability before they trust your conclusions:

• Thesis and dissertation committees, who will ask you to justify your instrument in the methodology chapter.
• Peer reviewers, who routinely reject otherwise strong papers over unreported or weak psychometric properties.
• NGO funders and policy stakeholders, who need to know your evaluation tools actually measure the outcomes you’re claiming to have achieved.

In short: strong validity and reliability aren’t academic box-ticking. They’re what separates evidence from opinion.

The 4 Main Types of Validity (With Examples)

Validity isn’t a single yes/no property — it’s usually established through several complementary types of evidence.

1. Content Validity

Content validity asks whether your instrument covers all relevant aspects of the concept — not just part of it. If you’re building a scale to measure “job satisfaction,” it should include pay, work-life balance, relationships with colleagues, and career growth, not just salary alone. Content validity is usually established by having subject-matter experts review the instrument against the full theoretical definition of the construct.

2. Construct Validity

Construct validity is the big one — it asks whether your instrument truly captures the underlying theoretical concept it claims to measure. A depression questionnaire should measure depression itself, not general mood, self-esteem, or anxiety, even though those overlap. Construct validity is typically supported through convergent validity (correlating well with related measures) and discriminant validity (correlating poorly with unrelated ones).

3. Criterion Validity

Criterion validity compares your instrument’s results against an established, independent outcome — a “gold standard.” It has two subtypes:

• Concurrent validity — your measure and the criterion are assessed at the same point in time.
• Predictive validity — your measure is used to predict an outcome that occurs later (e.g., does a screening tool for caregiver burnout predict actual burnout six months later?).

4. Face Validity

Face validity is the weakest and most subjective form: does the instrument look, on the surface, like it measures what it claims to? It’s not a substitute for the other three types, but it’s a useful first gut-check during instrument development, and it matters for respondent buy-in — participants disengage from surveys that feel irrelevant to the stated purpose.

The 4 Main Types of Reliability (With Examples)

1. Test-Retest Reliability

This measures consistency over time by administering the same instrument to the same participants on two separate occasions and correlating the scores (typically with Pearson’s r). A well-known example: the Beck Depression Inventory-II achieves a test-retest correlation of r = .93 over a one-week interval — a strong indicator of stability.

2. Inter-Rater Reliability

Essential whenever human judgment is involved — coding qualitative interviews, scoring observational data, or rating essay responses. It measures the level of agreement between two or more independent raters, typically using Cohen’s Kappa (for categorical ratings) or the Intraclass Correlation Coefficient (for continuous ratings). Cohen’s Kappa ranges from 0 (agreement no better than chance) to 1 (perfect agreement); values above 0.60 are generally considered acceptable, and above 0.80 substantial.

3. Internal Consistency Reliability (Cronbach’s Alpha)

This checks whether the different items within a single scale — say, 10 questions all meant to measure “caregiver burnout” — correlate with one another and reliably tap the same underlying construct. It’s reported as Cronbach’s alpha, which ranges from 0 to 1:

• α < 0.60 — unacceptable
• α = 0.60–0.69 — questionable
• α = 0.70–0.79 — acceptable
• α = 0.80–0.89 — good
• α ≥ 0.90 — excellent (though values above 0.95 can signal item redundancy — your items may be asking the same question in slightly different words)

4. Parallel-Forms Reliability

Two different but equivalent versions of the same test are administered to the same group, and scores are correlated. It’s especially useful when you want to avoid practice effects that would inflate scores if the identical test were simply repeated (as in test-retest reliability).

Validity vs. Reliability: Side-by-Side Comparison

How to Test Validity and Reliability in SPSS

If you’ve already worked through SPSS for Beginners, testing reliability is a natural next step:

1. Reliability (Cronbach’s alpha): Analyze → Scale → Reliability Analysis. Move your scale items into the “Items” box and select “Alpha” as the model. SPSS returns your overall alpha plus an “alpha if item deleted” column, which flags any item dragging your reliability down.
2. Construct validity (exploratory factor analysis): Analyze → Dimension Reduction → Factor. This checks whether your items cluster into the theoretical dimensions you expect — a core piece of evidence for construct validity.
3. Criterion validity (correlation): Analyze → Correlate → Bivariate, correlating your new measure’s total score against an established, validated instrument measuring the same or a related construct.

5 Common Mistakes Researchers Make

1. Reporting reliability but never discussing validity. A high Cronbach’s alpha tells reviewers nothing about whether you’re measuring the right thing.
2. Treating face validity as sufficient evidence. “It looks right to me” is a starting point, not a conclusion.
3. Using a borrowed scale without piloting it on your own population. A tool validated on Western university students may not behave the same way with, say, rural caregivers in a different cultural context — always re-check reliability on your actual sample.
4. Chasing an alpha above 0.95 and assuming higher is always better — it often means your items are redundant rather than well-constructed.
5. Confusing internal validity with internal consistency. They sound alike but measure completely different things — one is about your study design, the other about your scale’s items.

Frequently Asked Questions