How does prevalence affect Cohen's kappa?

• A. If prevalence is moderate then kappa will be higher than if prevalence is low or high
• B. Higher prevalence always lowers kappa
• C. Prevalence does not affect kappa
• D. Kappa increases with larger sample size only

Answer: (A)

The key idea is that Cohen’s kappa depends on how common each category is, not just on how often the raters agree. Kappa adjusts the observed agreement by the amount of agreement you’d expect by chance, and that chance agreement (Pe) is driven by the marginal prevalence of each category. When one category is very common, the margins are highly imbalanced, so Pe is large and a lot of the agreement looks like "agreement by luck." That can pull kappa down even if percent agreement seems high. When prevalence is more balanced (moderate), Pe isn’t inflated by dominance of a single category, so the same level of observed agreement translates into a larger kappa value. In short, kappa tends to be higher with moderate prevalence and lower when prevalence is very low or very high. Larger sample size affects the precision of kappa’s estimate but not the value of kappa itself, so it doesn’t inherently make kappa higher.