Gwet's Agreement Coefficient

The kappa paradox was starkly demonstrated for Question 7a, where 96.4% observed agreement after option collapsing yielded a Gwet's AC of 0.96 but a Fleiss' kappa of −0.02, implying agreement worse than chance. Gwet's AC was selected over Cohen's and Fleiss' kappa because it avoids the kappa paradox, in which high observed agreement can paradoxically yield low kappa values when item prevalence is skewed. Agreement benchmarks followed Landis and Koch's categories interpreted via 95% cumulative probability, providing more conservative interpretations than simply mapping a point estimate to a band. Gwet's AC1 was applied to questions with Not Applicable options (nominal data), and Gwet's AC2 with linear weighting was applied to all remaining questions and the overall judgement (ordinal data).