Abstract: A contingency-table approach to the problem of testing for agreement among m groups of raters, each ranking k items, is presented. The approach is based on a natural decomposition of the hypothesis of agreement into a hierarchy of subhypotheses. It is argued that unless the samples are unusually large, only a small number of these subhypotheses can actually be tested. A conditional testing procedure is then recommended, but a flexible unconditional procedure is also presented. Both procedures employ the familiar chi-squared and are illustrated with numerical examples.
Key words and phrases: Conditional analysis, ranking, chi-squared test of fit.