Face validity
Face validity is whether a test or questionnaire looks, to the people who take it, as if it measures what it says it measures. A depression questionnaire that asks about mood and sleep has it; one that asks about favorite colors does not.
It is a judgment of how the test looks, not evidence about what its scores measure. Mosier (1947) took a critical look at the term's several meanings, and Nevo (1985) proposed a definition with four facets and presented data suggesting that face validity can be rated validly and reliably.
Face validity and the kinds of validity evidence
| Kind | The question it asks | Evidence from |
|---|---|---|
| Face validity | Does the test look, to the people who take it, as if it measures what it says? | The people who take the test, and those who use its results |
| Content validity | Do the items cover what the test should measure, and only that? | Experts in what the test measures |
| Construct validity | Do scores behave as the idea behind the test predicts, such as rising with a related measure? | The scores of people who took the test |
| Criterion validity | Do scores agree with an outcome or a standard measured another way? | The scores, and the outcome or standard |
How to check face and content validity
- Ask a few people from the group the test is for to read each item and say what they think it asks, and whether it fits the test's purpose.
- Ask experts to rate each item's relevance from 1 (not relevant) to 4 (highly relevant), and work out each item's content validity index: the share who rate it 3 or 4.
- With three or more experts, an index of 0.78 or more can be taken as evidence of good content validity for the item (Polit, Beck and Owen, 2007). Rewrite or drop the items below it.
- Then gather evidence from data: how the scores behave, as construct and criterion validity ask.
A worked example
A panel of 6 experts rates 5 items. The index is the share who rate an item 3 or 4, and the modified kappa allows for the experts agreeing by chance.
| Item | Ratings | Rated 3 or 4 | Index | Modified kappa |
|---|---|---|---|---|
| Item 1 | 4, 4, 3, 4, 4, 3 | 6 of 6 | 1.00 | 1.00 |
| Item 2 | 4, 3, 3, 4, 2, 4 | 5 of 6 | 0.83 | 0.82 |
| Item 3 | 3, 4, 4, 4, 4, 4 | 6 of 6 | 1.00 | 1.00 |
| Item 4 | 2, 3, 2, 4, 3, 2 | 3 of 6 | 0.50 | 0.27 |
| Item 5 | 4, 4, 4, 3, 4, 4 | 6 of 6 | 1.00 | 1.00 |
The scale's index can be worked out two ways, and a paper should say which it used (Polit and Beck, 2006). As the mean of the items' indexes it is 0.87; as the share of items every expert rates 3 or 4 it is 0.60. Item 4, at 0.50, needs rewriting.
An expert panel in ConductExperience
- Build the panel's ratings as a Grid question: each item is a row, and the ratings 1 to 4 are its answers.
- Results shows each row's answered count and how many experts chose each answer, so an item's index is the number who chose 3 or 4 over the row's answered count.
- Add a Long text question after it for the experts' comments on wording.
Sources
- Mosier, C. I. (1947). A critical examination of the concepts of face validity. Educational and Psychological Measurement, 7(2), 191–205.
- Nevo, B. (1985). Face validity revisited. Journal of Educational Measurement, 22(4), 287–293.
- Polit, D. F., & Beck, C. T. (2006). The content validity index: Are you sure you know what's being reported? Critique and recommendations. Research in Nursing & Health, 29(5), 489–497.
- Polit, D. F., Beck, C. T., & Owen, S. V. (2007). Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Research in Nursing & Health, 30(4), 459–467.
Related
Cognitive interviewing
Test how people read your questions before the study, with probes for each step of answering.
Published scales
Scales in their holders' wording, scored as published.
Likert scale
Rating questions, their points and labels, and what the download holds.
Pilot study
What a pilot checks, how many people it needs, and what to report.
Planning a study?
Start one now, or tell us what you want to measure.