Reliability and validity of a standardized measure of influenza vaccination coverage among healthcare personnel.

OBJECTIVE: To evaluate the reliability and validity of a standardized measure of healthcare personnel (HCP) influenza vaccination.
SETTING: Acute care hospitals, long-term care facilities, ambulatory surgery centers, physician practices, and dialysis centers from 3 US jurisdictions. PARTICIPANTS: Staff from 96 healthcare facilities randomly sampled from 234 facilities that completed pilot testing to assess the feasibility of the measure.
METHODS: Reliability was assessed by comparing agreement between facility staff and project staff on the classification of HCP numerator (vaccinated at facility, vaccinated elsewhere, contraindicated, declined) and denominator (employees, credentialed nonemployees, other nonemployees) categories. To assess validity, facility staff completed a series of case studies to evaluate how closely classification of HCP groups aligned with the measure's specifications. In a modified Delphi process, experts rated face validity of the proposed measure elements on a Likert-type scale.
RESULTS: Percent agreement was high for HCP vaccinated at the facility (99%) and elsewhere (95%) and was lower for HCP who declined vaccination (64%) or were medically contraindicated (64%). While agreement was high (more than 90%) for all denominator categories, many facilities' staff excluded nonemployees for whom numerator and denominator status was difficult to determine. Validity was lowest for credentialed and other nonemployees.
CONCLUSIONS: The standardized measure of HCP influenza vaccination yields reproducible results for employees vaccinated at the facility and elsewhere. Adhering to true medical contraindications and tracking declinations should improve reliability. Difficulties in establishing denominators and determining vaccination status for credentialed and other nonemployees challenged the measure's validity and prompted revision to include a more limited group of nonemployees.