Validating a Guidelines-Based Asthma Decision Support System: Step Two

Purpose. Previously we described a computerized asthma decision support system and measured its accuracy in classifying asthma severity using pulmonologists as the gold standard. It is possible that the computerized decision support system may outperform pulmonologists in classifying asthma severity. The purpose of this study was to determine whether the computerized decision support system or pulmonologists have better accuracy with the 2007 edition of the Global Initiative for Asthma guidelines in classifying asthma severity. Design. One hundred asthma case scenarios were developed by a medical resident and classified according to the 2007 Global Initiative for Asthma guidelines. These case scenarios were deemed the gold standard. These case scenarios were then entered into a computerized asthma decision support system and also presented to a group of 10 practicing pulmonologists. Sensitivity, specificity, and agreement were calculated for the computerized decision support system and the pulmonologists. Results. The sensitivity and specificity of the computerized decision support system was 100%. The sensitivity of the pulmonologist was 94.2% and the specificity was 70.0%. Agreement between the computerized decision support system and the gold standard was perfect with a Kappa = 1.00. The Kappa between the pulmonologists and the gold standard was equal to 0.672. Conclusion. The computerized decision support system outperformed pulmonologists in the task of assigning asthma severity according to the 2007 Global Initiative for Asthma guidelines.