Intraobserver reproducibility and interobserver agreement of demarcation line depth measurements following corneal cross linking

PURPOSE: The aim of this study was to evaluate the intraobserver reproducibility of demarcation line depth measurement in keratoconic patients after epithelium-off corneal collagen cross linking and assess the interobserver variability/agreement among examiners of various experience levels. METHODS: 56 eyes of 56 patients undergoing corneal collagen cross linking for progressive keratoconus were enrolled. After 4 weeks, all patients underwent an anterior segment optical coherence tomography (AS-OCT; CASIA SS-1000; Tomey, Nagoya, Japan) in the cross-linked eye by four masked examiners (two medical students (S1 and S2) and two corneal fellows (F1 and F2)) in order to identify and measure the demarcation line depth in experienced and non-experienced users. The intraclass correlation coefficient and the coefficient of variation were calculated. Agreement between raters was evaluated using Bland-Altman plots, intraclass correlation coefficient and weighted-kappa statistics comparing raters in pairs. RESULTS: The average demarcation line depth of all measurements was 254.34 ± 72.3 μm, varying from 84 to 459 μm. The intraclass correlation coefficient evaluating the reproducibility of measurements for F1 was 0.9379 (95% confidence interval (CI) =0.9035-0.9619), for the second (F2), it was 0.9837 (95% confidence interval = 0.9743-0.9901), while intraclass correlation coefficient between medical students was calculated 0.844. The overall intraclass correlation coefficient among all four observers was 0.8706 (95% confidence interval = 0.8061-0.9185). The coefficient of variation for repeated measurements was 5.981 μm (95% confidence interval = 3.966-7.471) and 3.312 μm (95% confidence interval = 2.468-3.981) for F1 and F2 raters, respectively. The percentage of demarcation line detection was 90.32%. CONCLUSION: The reproducibility of demarcation line measures although very good (intraclass correlation coefficient > 0.9), yielded a difference between the two experienced raters. Furthermore, the novice raters did not reach an excellent level of agreement with the expert ones showing greater variability in their recordings.