Implications of Empirical Bayes Meta-analysis for Test Validation

Authors

Michael T. Brannick, University of South FloridaFollow

Document Type

Article

Publication Date

1-1-2001

Abstract

Empirical Bayes meta-analysis provides a useful framework for examining test validation. The fixed-effects case in which ρ has a single value corresponds to the inference that the situational specificity hypothesis can be rejected in a validity generalization study. A Bayesian analysis of such a case provides a simple and powerful test of ρ = 0; such a test has practical implications for significance testing in test validation. The random-effects case in which ς² _ρ  > 0 provides an explicit method with which to assess the relative importance of local validity studies and previous meta-analyses. Simulated data are used to illustrate both cases. Results of published meta-analyses are used to show that local validation becomes increasingly important as ς² _ρ increases. The meaning of the term validity generalization is explored, and the problem of what can be inferred about test transportability in the random-effects case is described.

Was this content written or created while at USF?

Yes

Citation / Publisher Attribution

Journal of Applied Psychology, v. 86, issue 3, p. 468-480

Scholar Commons Citation

Brannick, Michael T., "Implications of Empirical Bayes Meta-analysis for Test Validation" (2001). Psychology Faculty Publications. 2337.
https://digitalcommons.usf.edu/psy_facpub/2337