- > The term within-group agreement refers to the degree to which rat ings from individuals are interchangeable; that is, agreement re flects the degree to which raters provide essentially the same rating (Kozlowski & Hattrup, 1992; Tinsley & Weiss, 1975).
Page 4
- > The most fre quently encountered measure of within-group agreement in the organizationalliteratureisther^gorr^j (Jamesetal., 1984, 1993).
Page 6
- > Reliability can be considered a measure assessing the relative con sistency of responses among raters (Kozlowski & Hattrup, 1992). It is the degree to which ratings are consistent “when expressed as deviations from their means” (Tinsley & Weiss, 1975, p. 359). One can have high reliability in cases where agreement is low. For ex ample, if rater A uses only 1, 2, and 3 on a 5-point scale while rater B uses only 3, 4, and 5, then across ratees a rating of 3 from rater A will be equivalent to a rating of 5 from rater B. Agreement will be low because 3 is different from 5, but reliability will be high be cause the responses are proportionally consistent.
