S Mattke1. 1. the RAND Corporation, 1200 South Hayes Street, Arlington, VA 22202, USA. mattke@rand.org.
Abstract
OBJECTIVE: To document current practices on long-term maintenance of quality measures and to develop a consensus framework to guide the design of maintenance systems. STUDY DESIGN: Survey of 10 organisations developing measures and selected researchers in the USA about current policies and procedures and desirable properties for a comprehensive system for measures maintenance. Panel discussions with all respondents to arrive at consensus recommendations for a framework for maintenance of measures. PARTICIPANTS: Five measures developers, two provider and three purchaser organisations. Six were private sector organisations, two were governmental agencies, and two were accreditation institutions. PRINCIPAL FINDINGS: All organisations had procedures for measures maintenance, but the degree of formalisation of the procedures varied. Three key functions for a measures maintenance system emerged: ad hoc review to deal with unexpected problems; annual maintenance to incorporate changes in coding conventions; and regular re-evaluation to thoroughly review measures at predefined intervals. Importance, scientific soundness, feasibility and usability were universally used as evaluation criteria. The panel discussions yielded a consensus set of recommendations for relationships between maintenance functions, evaluation criteria and measures disposition. CONCLUSIONS: A sufficient degree of implicit consensus was found among leading measures developers to arrive at a consensus framework for policies and procedures for measures maintenance. Although organisations may choose to implement the framework in a way that is most consistent with their mission and structure, it provides guidance regarding which components should be included.
OBJECTIVE: To document current practices on long-term maintenance of quality measures and to develop a consensus framework to guide the design of maintenance systems. STUDY DESIGN: Survey of 10 organisations developing measures and selected researchers in the USA about current policies and procedures and desirable properties for a comprehensive system for measures maintenance. Panel discussions with all respondents to arrive at consensus recommendations for a framework for maintenance of measures. PARTICIPANTS: Five measures developers, two provider and three purchaser organisations. Six were private sector organisations, two were governmental agencies, and two were accreditation institutions. PRINCIPAL FINDINGS: All organisations had procedures for measures maintenance, but the degree of formalisation of the procedures varied. Three key functions for a measures maintenance system emerged: ad hoc review to deal with unexpected problems; annual maintenance to incorporate changes in coding conventions; and regular re-evaluation to thoroughly review measures at predefined intervals. Importance, scientific soundness, feasibility and usability were universally used as evaluation criteria. The panel discussions yielded a consensus set of recommendations for relationships between maintenance functions, evaluation criteria and measures disposition. CONCLUSIONS: A sufficient degree of implicit consensus was found among leading measures developers to arrive at a consensus framework for policies and procedures for measures maintenance. Although organisations may choose to implement the framework in a way that is most consistent with their mission and structure, it provides guidance regarding which components should be included.
Authors: Thomas Woodcock; Yewande Adeleke; Christine Goeschel; Peter Pronovost; Mary Dixon-Woods Journal: BMC Med Res Methodol Date: 2020-01-14 Impact factor: 4.615