Interpretation of occupational peak flow records: level of agreement between expert clinicians and Oasys-2.

BACKGROUND: Oasys-2 is a validated diagnostic aid for occupational asthma that interprets peak expiratory flow (PEF) records as well as generating summary plots. The system removes inconsistency in interpretation, which is important if there is limited agreement between experts. A study was undertaken to assess the level of agreement between expert clinicians interpreting serial PEF measurements in relation to work exposure and to compare the responses given by Oasys-2.
METHOD: 35 PEF records from workers under investigation for suspected occupational asthma were available for review. Records included details of nature of work, intercurrent illness, drug therapy, predicted PEF, rest periods, and holidays. Simple plots of PEF and the Oasys-2 generated plots were available. Experts were advised that approximately 1 hour was available to review the records. They were asked to score each work-rest-work (WRW) period and each rest-work-rest (RWR) period for evidence of occupational effect. At the end of each record scores of 0-100% were given for evidence of "asthma" and "occupational effect" for the whole record. Kappa values were calculated for each scored period and for the opinions on the whole record. The scores were converted into four groups (0-25%, 26-50%, 51-75%, 76-100%) and two groups (0-50% and 51-100%) for analysis. This is relevant to scores produced by Oasys-2. Agreement between Oasys-2 scores and each expert was calculated.
RESULTS: 24 of 35 records were analysed by seven experts in the allotted time. For whole record occupational effect, median kappa values were 0.83 (range 0.56-0.94) for two groups and 0.62 (0.11-0.83) for four groups. For asthma, median kappa values were 0.58 (0-0.67) and 0.42 (0.15-0.70) for two and four groups respectively. For all WRW and RWR periods kappa values were 0.84 (0.42-0.94) and 0.70 (0.46-0.87) respectively. Agreement between Oasys-2 and individual experts showed a median kappa value of 0.75 (0.50-0.92) for two groups and 0.50 (0.39-0.70) for four groups. Kappa values for the median expert score v Oasys-2 were 0.75 for two groups and 0.67 for four groups. Agreement was poor for records with intermediate probability, as defined by Oasys-2.
CONCLUSION: Considerable variation in agreement was seen in expert interpretation of occupational PEF records which may lead to inconsistencies in diagnosis of occupational asthma. There is a need for an objective scoring system which removes human variability, such as that provided by Oasys-2.