Exploring Upper Limits of Reliability and Validity in Job Performance Ratings

Document Type

Article

Publication Date

1978

Keywords

job performance ratings using behavior type scale, upper limits of reliability & validity, college students & industrial psychologists

Digital Object Identifier (DOI)

https:///doi.org/10.1037/0021-9010.63.2.135

Abstract

Explored possible upper bound limits in reliability and validity of performance ratings by creating a nearly ideal environment for obtaining performance evaluations and then examining closely the precision of these ratings. 16 scripts describing persons performing on 2 jobs–recruiting interviewer and manager—were prepared in such a way that the performers' effectiveness on various dimensions of performance approached a preset, realistic level. Five- to 7-min performances of these scripts were videotaped, and 14 raters evaluated the effectiveness of each performer on each performance dimension. Raters were very knowledgeable about the 2 jobs, had ample opportunity to observe the performers, and used a relatively advanced behavior scale type of rating form. Thus, the opportunity for obtaining highly reliable and valid ratings was excellent. Levels of convergent and discriminant validities were high compared to other studies, but interrater agreement was far from perfect. An analysis of the performance rating process reveals the possible reasons for error in ratings, and research approaches to learn more about this process are discussed.

Was this content written or created while at USF?

No

Citation / Publisher Attribution

Journal of Applied Psychology, v. 63, issue 2, p. 135-144

Link to Full Text

Share

COinS