Kappa statistic for clustered physician–patients polytomous data

Motivated by the recent advances in the kappa statistic for the clustered physician–patients dichotomous data, we extend the development for the polytomous data. For the clustered physician–patients polytomous data, based on its special correlation and covariance structure, we propose a simple and efficient data generation algorithm, and develop a semi-parametric variance estimator for the kappa statistic. An extensive Monte Carlo simulation study is then conducted to evaluate the performance of empirical coverage probability of the new proposal and alternative methods. The method ignoring dependence within a cluster underestimates the variance, and the variance estimators from new proposal and sampling-based delta method behave reasonably well for at least a moderately large number of clusters (e.g., K≥50). Moreover, the new proposal has acceptable performance when the number of clusters is small (e.g., K=25), although we need to be cautious about some bias concerns of the kappa statistic when the within-physician correlation is large as ρw>0.5. To illustrate the practical application of all the methods, a physician–patients data example from the Enhancing Communication and HIV Outcomes (ECHO) study is analyzed.