Which statistic is commonly used to quantify variability between paired test results and assess agreement within or between raters?

• A. Intraclass correlation coefficient
• B. Coefficient of variation
• C. Pearson correlation coefficient
• D. Limits of agreement plots

Answer: (A)

Assessing how much measurements agree when two tests are paired or how consistently different raters score the same subjects hinges on reliability. The intraclass correlation coefficient is designed to quantify that reliability by partitioning the total variation into parts: variation between subjects (the true differences we care about) and error variation (random noise, measurement error, or rater differences). When most of the variability comes from true differences between subjects and little from measurement error, the ICC is high, indicating good agreement across paired results or across raters.

This statistic is particularly suited for paired data because it treats the multiple measurements for each subject as part of the same unit, allowing you to assess consistency or absolute agreement depending on the model you choose. A high ICC (closer to 1) means that the measurements or ratings are highly reliable and consistent across measurements or raters; a low ICC (closer to 0) signals poor reliability.

It’s worth distinguishing ICC from other measures. Pearson correlation assesses linear association but not agreement in magnitude or bias between two measurements—two methods can be perfectly correlated yet differ systematically. The coefficient of variation describes spread relative to the mean, not how closely two measurements or raters agree. Bland-Altman limits of agreement provide a useful visual and numerical summary of agreement but as a single statistic they don’t quantify reliability in the same way ICC does.

For those reasons, intraclass correlation coefficient is the standard statistic to quantify variability between paired test results and assess agreement within or between raters.