Development of quality indicators for departments of hospital-based physiotherapy: a modified Delphi study.

BACKGROUND: International hospital accreditation instruments, such as Joint Commission International (JCI) and Qmentum, focus mainly on hospital policy and procedures and do not specifically cover a profession such as hospital-based physiotherapy. This justifies the need for a quality system to which hospital-based physiotherapy can better identify, based on a common framework of quality indicators for effective quality management.
OBJECTIVE: This study aimed to identify the most important quality indicators of a hospital-based physiotherapy department in the eyes of hospital-based physiotherapists and their managers.
METHODS: Based on input from three focus groups and a structured literature review, a first set of quality indicators for hospital physiotherapy was assembled. After checking this set for duplicates and for overlap with JCI and Qmentum, it formed the starting point of a modified Delphi procedure. In two rounds, 17 hospital-based physiotherapy experts rated the quality indicators on relevance through online surveys. In a final consensus meeting, quality indicators were established, classified in quality themes and operationalised by describing for each theme the rationale, specifications, domain and type of indicator.
RESULTS: Three focus groups provided 120 potential indicators, which were complemented with 18 potential indicators based on literature. After duplicate and overlap check and the Delphi procedure, these 138 potential indicators were reduced to a set of 56 quality indicators for hospital-based physiotherapy. Finally, these 56 indicators were condensed into 7 composite indicators, each representing a quality theme based on definitions of the European Foundation for Quality Management.
CONCLUSION: A set of 56 quality indicators, condensed into 7 composite indicators each representing a quality theme, was developed to assess the quality of a hospital-based physiotherapy department. © Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Introduction

Hospital-based physiotherapy can play a significant role in the multidisciplinary treatment of hospitalised patients by focusing primarily on functional mobility as an important part of the patient’s functional health condition, before, during and after hospitalisation.1 Quality of hospital-based physiotherapy can be defined as the degree of similarity between criteria of good care (desirable care) and the practice of care (actual care).2 This degree of similarity can be quantified through quality indicators which can be classified into structure, process and outcome indicators.2–4 In hospital care, quality indicators are being used as a tool in quality improvement cycles, for example, to decrease morbidity and mortality, or to qualify for a recognised quality approval such as Joint Commission International (JCI) or Qmentum.5–8 These globally expanding accreditation instruments focus mainly on hospital policy and procedures and do not specifically cover a profession like hospital-based physiotherapy.7 8

To determine whether the practice of care provided by hospital-based physiotherapists meets the criteria of desirable care and to stimulate continuous quality improvement, a quality system is required. Such a quality system typically consists of four components: a professional profile with core competencies, a system of standards and guidelines, external accountability and systematic quality control.9 Because accreditation instruments such as JCI and Qmentum cover only the medical and nursing staff as recognisable individual disciplines,7 8 these instruments do not allow systematic quality control of hospital-based physiotherapy departments. This justifies the need for a tailored quality system for hospital-based physiotherapy. To assess the quality of the provided hospital physiotherapy care, the alignment between performance, strategy, vision and desired outcomes needs to be established.10 Profession-specific quality assessment feedback can help physiotherapists to identify areas of professional practice that need improvement. This process has been recommended as an essential component in raising the standards of hospital-based physiotherapy care.11 12

Because organisational restructuring due to financial issues is common in multidisciplinary hospital care, a quality system for hospital-based physiotherapy should be sufficiently flexible towards organisational changes and the associated changing roles of hospital physiotherapists. This continuously changing role places an increased emphasis on higher level skills in patient care and interprofessional communication and collaboration skills, and highlights the ongoing importance of professionalism.13 When hospital restructuring takes place, physiotherapists, as part of allied healthcare, need their own conceptual model to describe the effect of hospital restructuring on their professional role.14 15 Hospitals nowadays struggle how to organise these changing roles: a move to programme management from a traditional department structure affects the professional practice of physiotherapists, reporting both positive and negative effects on professional affect, professional practice and patient care.16

A quality system for an individual profession such as hospital-based physiotherapy should be based on a common framework for effective quality management to which this specific profession can better identify, and which is independent of hospital restructuring. Because such a system is lacking, the aim of this study is to identify the most important indicators for the quality of a hospital-based physiotherapy department in the eyes of hospital-based physiotherapists and their managers, as a first step towards establishing a valid and useful quality system for the profession.

Methods

A modified RAND/UCLA (RAND Corporation/University of California at Los Angeles) Appropriateness Method Delphi study was used to establish a list of the key quality indicators of hospital-based physiotherapy.17 The study was conducted from May 2017 to January 2019 in a convenience sample of Dutch hospital-based physiotherapists and their managers, taken from the member databases of the Dutch Association of Physical Therapy in Hospitals (NVZF) and the Dutch Association of Managers of Physical Therapy (VLF). According to Dutch regulations, ethical review was not required because there were not any patients or interventions involved in this study.

Patient and public involvement

No patients or public were involved in this study because the opinion of the professional was the main topic. The results of the study will be disseminated to the members of the Delphi Panel and members of the NVZF and VLF during the annual congress of the NVZF.

The study consisted of three stages.

Stage 1: focus groups

During a scheduled meeting in May 2017, all VLF members were informed that focus groups on quality indicators for hospital-based physiotherapy would be part of the meeting’s programme. Attendees expressing interest in participating in the focus groups were divided into three equally sized focus groups. Each focus group interview was moderated by an NVZF board member experienced in the subject of quality of hospital-based physiotherapy. Every moderator received detailed moderation instructions from one of the primary researchers and was supported by a research team member who made field notes during the interviews. The focus group interviews were semistructured following a topic list (online supplementary appendix 1), concentrating on two main questions:

Which do you think are the most important elements that demonstrate the quality of a hospital-based physiotherapy department?

What do you think is a good and short definition of quality of a hospital-based physiotherapy department?

Respondents were encouraged to speak freely and to respond to each other. The interviews, planned for 1 hour, were audio recorded and transcribed. The transcripts were checked against the field notes and then returned for a member check. The two primary researchers independently analysed the transcripts using Atlas.ti V.7,18 19 to identify all the quality topics that were proposed by the participants during the group interviews. Differences were resolved by discussion until consensus was reached.

Stage 2: literature search

A structured literature search was conducted to complement the initial set of quality indicators from stage 1. PubMed was searched in February 2018 using the following search string: ‘((‘Physical Therapy Department, Hospital’ [Mesh]) AND ((((‘Quality of Health Care’ [Mesh] OR ‘Quality Assurance, Health Care’ [Mesh]))) OR quality [Title/Abstract])) AND ((((‘Organization and Administration’ [Mesh] OR ‘organization and administration’ [Subheading]))) OR ((Organization*[tiab] OR Administrat*[tiab])))’. The two primary researchers independently reviewed the retrieved titles and abstracts, and the full text of potentially relevant articles. The reference lists of retrieved full-text articles were hand-searched for potentially relevant articles. Relevant articles were independently hand-searched for additional quality indicators. Differences were discussed until the two primary researchers agreed on a set of additional quality indicators which were added to the set obtained in stage 1. The resulting set of indicators was checked for duplicates and for overlap with JCI and Qmentum quality indicators and classified by following the EFQM (the European Foundation for Quality Management) Excellence Model (figure 1).20

Figure 1

The EFQM (the European Foundation for Quality Management) excellence model (Source: www.efqm.org).

Stage 3: Delphi rounds

All 180 NVZF members, hospital-based physiotherapists and managers of hospital-based physiotherapy received written study information by email, along with an invitation to join the Delphi panel. Members who responded positively to the invitation took part in the Delphi panel. The Delphi procedure consisted of two online survey rounds and a consensus meeting.

In the first two online survey rounds, the IQ (Scientific Center for Quality of Healthcare) Consensus Tool was used.21 This tool supports groups in achieving consensus, based on the Modified RAND Delphi Method. Each quality indicator was valued by the group members using a nine-point Likert scale, ranging from very low to very high value. The tool combines the highest tertile, median and top 3 scores to arrive at a ‘selection’, ‘discussion’ or ‘no selection’ score, using the Campbell criteria.22 Group members were also asked whether indicators showed overlap with known JCI or Qmentum quality indicators and if they thought indicators were missing.

In preparation for the consensus meeting, the research team put together the first draft, assembling the indicators by theme. The meeting started with a presentation on the study purpose and setup, followed by a full-day group discussion moderated by the first author who is highly experienced in running meetings in similarly sized groups. Throughout the meeting, changes to the developing indicator set were projected on a screen in real time. The meeting was audio recorded, and comments and changes were recorded by a research team member.

First, the consensus panel members were asked to confirm the quality indicators they had selected or deleted in the two online survey rounds. Second, they discussed whether the indicators with the label ‘discussion’ should be included or deleted. All selected indicators were condensed into composite indicators, each representing a quality theme based on definitions of the EFQM. These composite indicators were operationalised by describing the rationale, specifications, domain (following EFQM) and type of indicator (structure, process or outcome). When differences in opinion hampered the process, decisions were made through consensus. When unanimous consensus appeared impossible, voting by hand raising took place, where a >75% majority was needed for adopting a proposal for amendment. If this majority was not attained, the proposal was rejected.

One week after the meeting, the resulting quality indicator list was sent to the participants by email, to allow feedback on the text. This feedback was processed by the two primary researchers, leading to the final set of themes and indicators.

Results

Focus groups

The quarterly VLF meeting in May 2017 was attended by 26 of 70 members. Characteristics of the participants are presented in table 1. Attending members were divided into three focus groups.

Table 1

Group member characteristics

	Expert group VLFN=26	Delphi group NVZFN=17
Age in years, mean (SD)	51.6 (8.9)	45.8 (11.6)
Men:Women (%)	35:65	47:53
Working or worked as a physiotherapist (%)	85	100
University:General teaching:District hospital (%)	15:39:46	18:53:29
Experience in years, mean (SD)	8.9 (5.7)	14.4 (9.7)
Position (partially) in management (%)	88	53

Analysis of the three focus groups’ transcripts resulted in 120 potential quality indicators for hospital-based physiotherapy.

Literature search

The PubMed search yielded 163 hits, 12 of which were considered relevant after reading title and abstract.11–16 23–28 These 12 articles were searched for quality indicators not found during stage 1. The resulting 18 indicators were added to the 120 potential quality indicators for hospital-based physiotherapy departments obtained in stage 1, providing a total of 138 potential indicators. Deleting duplicates and indicators present in JCI and Qmentum resulted in a list of 103 potential quality indicators of hospital-based physiotherapy (online supplementary appendix 2). These were classified according to the EFQM model.

Delphi rounds

The Delphi panel consisted of 17 of 180 NVZF members. Characteristics of the participants are presented in table 1.

In the first online survey round (October 2018) with a 100% response, 43 quality indicators were selected, 35 were labelled as ‘discussion’ and 25 were not selected. None of the group members indicated missing quality indicators or overlaps with JCI or Qmentum indicators.

In the second online survey round (November 2018) with a 100% response, the 35 ‘discussion’ indicators were reoffered to the group members. The members selected 13 indicators to be included, 7 were labelled as ‘discussion’ and 15 were not selected.

During the consensus meeting in January 2019, with a 94% attendance of group members, consensus was reached to definitively not select the seven indicators which were labelled as ‘discussion’. With this decision, 56 quality indicators for hospital-based physiotherapy were set (figure 2). The participants agreed that these 56 quality indicators could be grouped into seven themes (composite indicators, table 2). Every theme with the specific indicators was operationalised by describing the rationale, specifications, domain (according to EFQM) and type of indicator (structure, process or outcome) (online supplementary appendix 3).

Figure 2

Number of quality indicators for hospital-based physiotherapy during the three stages of the process.

Table 2

The 56 quality indicators for hospital-based physiotherapy sorted into 7 composite Indicators

Composite indicator (Quality theme)	Quality indicators
(1) The hospital physiotherapy department has a culture of continuous learning, improvement and open dialogue.	PDCA (Plan–Do–Check–Act) cycle Peer review Treatment Collegiality within team Meeting obligations Culture of feedback/open dialogue
(2) The hospital physiotherapy department ensures the promotion of staff expertise that is consistent with the demand for care.	Structure of team (Bachelor/Masters) Continuity of quality PDCA (Plan–Do–Check–Act) cycle Training plan Expertise Specialisations Attitude to delivering quality Quality passport
(3) The hospital physiotherapy department uses a planning and control cycle to work on achieving its goals in the short, medium and long term, with a policy plan that fits within the frameworks of organisational policy.	Quality plan Financial possibilities Innovation and modernisation Visibility Continuity of care Critically monitoring process indicators and acting accordingly Efficiency of operational process Service quality
(4) The hospital physiotherapy department forms an integral part of the overall patient and hospital process.	Care trajectories: forming an integral part of Demonstrable effectiveness Commitment to internal training Movement-related care Innovation and modernisation Added value of physiotherapy in the process Supplementary diagnostics Uniformity of treatment Evaluation based on clinimetrics Endpoints of treatment Multidisciplinary cooperation Care networks: role and position
(5) The hospital physiotherapy department implements a patient-oriented policy.	Contribution to patient’s ability to cope independently Provision of information to patient Patient self-determination Sufficient care Patient-oriented Patient safety Handover Opening times Accessibility
(6) The hospital physiotherapy department systematically ensures that the physiotherapeutic interventions undertaken by its employees are of the highest possible quality.	Clinical reasoning Peer review Patient file checks EBP (evidence-based practice) conditions (access to literature) EBP (evidence-based practice) Endpoints of treatment Implementation of new processes/treatment policy Protocols; topicality, management, application Guidelines Evaluation based on clinimetrics
(7) The hospital physiotherapy department collects feedback on its performance from stakeholders and staff and takes action that is based on this feedback.	Employee satisfaction Customer satisfaction Patient satisfaction

Discussion

Major findings

This study aimed to identify important quality indicators of a hospital-based physiotherapy department in the eyes of hospital-based physiotherapists and their managers providing an inside-out perspective. A RAND-modified Delphi procedure resulted in 56 indicators condensed into 7 composite indicators each representing a quality theme: (1) culture of continuous learning, improvement and open dialogue; (2) promotion of staff expertise that is consistent with the demand for care; (3) using a planning and control cycle to work on achieving its goals in the short, medium and long term, with a policy plan that fits within the frameworks of organisational policy; (4) forming an integral part of the overall patient and hospital process; (5) implementing a patient-oriented policy; (6) systematically ensuring that the physiotherapeutic interventions undertaken by employees are of the highest possible quality; (7) collecting feedback on performance from stakeholders and staff and taking action that is based on this feedback.

Meaning of the findings

These composite quality indicators can serve as the first step towards a quality system for hospital-based physiotherapy, meeting the hospital-based physiotherapy’s need of such a system. With these findings, based on a common framework, a foundation could be laid for a method of quality improvement of hospital-based physiotherapy in the Dutch situation. This method could rely on an assessment procedure in which the specific profession of hospital-based physiotherapy recognises itself better than in systems such as JCI or Qmentum. Assessment of professional performance, including both clinical and organisational performance, can be applied for summative or formative purposes. Summative assessments are used to decide on academic progress, certification or accreditation (such as JCI and Qmentum). Formative assessments are used to support continuous learning and quality improvement.29 Most indicators found in this study are structure elements, a few are process indicators and none are outcome indicators. This suggests that a formative assessment would be better in place right now.

Relation with similar studies

To our knowledge, this is the first set of quality indicators for hospital-based physiotherapy. A known set of quality indicators for a specific discipline in Dutch hospitals is one used by obstetric caregivers. The user experience of this set shows high awareness of and reasonable contribution to quality indicators,30 which supports the similar use of quality indicators for hospital-based physiotherapy. Learning from nursing initiatives concerning the use of quality indicators, it is also important that leaders empower their staff in the process of transforming towards a higher quality delivery system.31

Strengths of the study

This study has several strengths. Although the set of indicators was developed from the perspective of Dutch hospital-based physiotherapy departments, we consciously used established indicator development methodology—that is, the RAND-modified Delphi procedure—to improve its validity and generalisability.17 32 33 The response rates to the online surveys were high with 100% in the two online rounds, and 94% attendance of group members during the consensus meeting. Because members of the focus groups and the Delphi Panel represent approximately 30% of the total amount of hospitals in the Netherlands, in which organisational positioning of a department of hospital-based physiotherapy can differ strongly, we assume that the presented list of quality themes and indicators is sufficiently flexible towards organisational changes and changing roles of hospital-based physiotherapy. Research has shown that bottom-up quality improvement initiatives, such as communities of practice and professional networks focusing on collaborative learning, might hold better and more sustainable results than external, top-down regulations,34–36 because shared social and professional norms are important predictors of behaviour change.37 38 This is the main reason why we chose a bottom-up approach for our study.

Limitations

We acknowledge the following limitations. Although Delphi consensus groups can produce collective answers, the achieved consensus is not necessarily accurate; bias can occur in the consensus meeting because one individual’s opinion can be overrepresented.39 Also, the group size of 17 members was slightly larger than the ideal Delphi group size of 5–11 members.40 Since the consensus meeting was not anonymous, respondents may have felt restrained to speak freely. Although we consciously invited representatives of hospital-based physiotherapy, their scope may be limited affecting the validity and generalisability. Stakeholders from outside the profession of hospital-based physiotherapy were not involved in the development process and the set of quality indicators has not yet been subjected to external review.41

Suggestions for further research

The presented list of quality indicators, classified in quality themes, should be enriched in future research. By including key stakeholders of hospital-based physiotherapy such as patients, medical specialists and hospital management, an outside-in perspective could be provided. Combining the results from both the inside and outside perspectives could provide the ideal mix of indicators for good quality of hospital-based physiotherapy. In addition, our set of quality indicators should be further assessed for reliability, validity and acceptability. Reviewing the list by national and international hospital-based physiotherapy specialists could contribute to these points and to the issue of generalisation. It is conceivable that after these steps, a foundation could be laid for a method of quality improvement of hospital-based physiotherapy, at least in the Dutch situation.

Conclusions

This study presents a set of 56 quality indicators, condensed into 7 composite indicators each representing a quality theme based on definitions of the EFQM. These indicators are important and relevant to Dutch hospital-based physiotherapy departments and their managers. To our knowledge, this is the first time such a quality set for hospital-based physiotherapy has been presented. By involving relevant stakeholders and external reviewers, this set can be further assessed on reliability, validity and acceptability, laying a foundation for a method of quality improvement of hospital-based physiotherapy.

This study was reported following the Consolidated criteria for Reporting Qualitative research (COREQ).42