Intra- and inter-rater reliability for judgement of cough following citric acid inhalation.

This study investigated the inter-rater and intra-rater reliability of subjective judgements of cough in patients following inhalation of citric acid. Eleven speech-language pathologists (SLPs) currently using cough reflex testing in their clinical practice (experienced raters) and 34 SLPs with no experience using cough reflex testing (inexperienced raters) were recruited to the study. Participants provided a rating of strong, weak, or absent to 10 video segments of cough responses elicited by inhalation of nebulized citric acid. The same video segments presented in a different sequence were re-evaluated by the same clinicians following a 15-minute break. Inter-rater reliability for experienced raters was calculated with a Fleiss' generalized kappa of .487; intra-rater reliability was higher with a kappa of .700. Inexperienced raters showed similar reliability, with kappa values for inter-rater and intra-rater reliability of .363 and .618, respectively. In conclusion, SLPs demonstrate only fair-to-moderate reliability in subjectively judging a patient's cough response to citric acid. Experience in making cough judgements does not improve inter-rater reliability significantly. Further validity and reliability research, including an evaluation of the effect of training on judgement reliability, would be beneficial for guiding clinical policies.