Patient feedback for safety improvement in primary care: results from a feasibility study.

OBJECTIVES: Patient involvement in safety improvement is a developing area of research. The aim of this study was to investigate the feasibility of a patient feedback on safety intervention in primary care. Specifically, the intervention acceptability, fidelity, implementation enablers and barriers, scalability, and process of systematically collecting safety data were examined. DESIGN, SETTING AND PARTICIPANTS: Mixed-methods feasibility trial with six purposively selected Australian primary care practices. INTERVENTION: The intervention comprised an iterative process with a cycle of measurement, learning, feedback, action planning and implementation period of 6 months. PRIMARY AND SECONDARY OUTCOMES: Qualitative and quantitative data relating to feasibility measures (acceptability, fidelity, enablers, barriers, scalability and process of collecting safety data) were collected and analysed.
RESULTS: A total of n=1750 patients provided feedback on safety. There was a statistically significant increase in mean patient safety scores indicating improved safety (4.30-4.37, p=0.002). Staff deemed the intervention acceptable, with minor recommendations for improvement. Intervention fidelity was high and implementation enablers were attributed to the intervention structure and framework, use of intuitive problem-solving approaches, and multidisciplinary team involvement. Practice-based safety interventions resulted in sustainable and measurable changes to systems for safety, such as increased access to care and improved patient information accuracy.
CONCLUSIONS: The findings indicate that this innovative patient feedback on safety intervention is feasible for scale-up to a larger effectiveness trial and further spread into policy and practice. This intervention complements existing safety improvement strategies and activities, and integrates into current patient feedback service requirements for Australian primary care. Further research is needed to examine the intervention effects on safety incident reduction. © Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

A feasibility study was conducted prior to the development and implementation of a large-scale effectiveness trial and wider spread and uptake into policy and practice.

Several feasibility domains were assessed including intervention acceptability, fidelity, implementation enablers and barriers, scalability, and process of systematically collecting safety data in a primary care.

A mixed-methods approach addressed each feasibility domain and included both qualitative and quantitative data collection and analysis.

A limitation is that the data collected will be mostly descriptive, and therefore, the generalisability of the findings may be limited to only one geographical area.

Introduction

Involving patients in error prevention and harm reduction activities has gained traction over the past decade.1–7 Patient engagement has been found to prevent or reduce adverse events, and increase awareness of potential safety risks.8 Much of this research has centred on hospital settings with the majority of interventions utilising patient feedback mechanisms for safety improvement.3 8–12 The evidence base regarding patient feedback on safety in primary care is considerably lacking by comparison.

In addition to reporting formal safety incidents,13–16 patient feedback about processes, systems and structures that lead to safety incidents is an essential piece of the safety intelligence ‘jigsaw’.17 Patients have demonstrated understanding and knowledge about the various conditions in the latent environment that influence safety, such as access to care; communication systems; information and care planning; and transitions between care settings.17–20 Capturing patient feedback about these contributory factors to safety incidents and using it for safety improvement work in primary care is a developing and novel field of research.21

Only one validated, real-time and theory-derived patient feedback tool for assessment of factors contributing to safety in primary care is currently available—the primary care patient measure of safety (PC PMOS).20 22 The PC PMOS aims to enhance or complement current data collection methods for patient safety in primary care.20 22 This self-administered tool is an acceptable, efficient and appropriate mechanism for engaging patients in safety improvement.11 13 17 23 The PC PMOS also facilitates primary care professionals and organisations learning, and drives implementation of real-time service improvements.20 21

The implementation and impact of interventions which use the PC PMOS tool for data-driven improvement and ongoing safety monitoring in primary care remains unexplored. Primary care, like most healthcare settings, is a complex system with multiple and multilevel factors likely to affect implementation of a patient feedback for safety improvement intervention.24 While common barriers and enablers to implementation of quality and safety improvement interventions have been published,25–27 the specific processes and outcomes of using the PC PMOS in a primary care safety improvement intervention is unknown. Advocates for complexity science and implementation science in healthcare-improvement-research recommend feasibility studies be conducted prior to the introduction of large-scale effectiveness trials or wider spread into policy and practice.24 25 28–30 Therefore, the aim of this study was to understand the acceptability, fidelity, implementation enablers and barriers, scalability, and process of systematically collecting safety data in a primary care patient feedback on safety intervention.

Methods

A detailed description of the study design and sampling frame, intervention, and primary and secondary outcome measures has been published in the study protocol.21 A brief overview is provided below.

Study design and sampling frame

This was a mixed-methods feasibility trial with six purposively sampled primary care practices from the southwest region of Victoria, Australia (online supplementary appendix 1).

Intervention

Intervention tool: PC PMOS

The PC PMOS tool is an anonymous 28 item survey covering nine latent conditions in the primary care environment influencing safety incidents including: access to care, communication, the external policy environment, information flow, organisation and care planning, patient-related factors, the physical environment, referral systems and task performance (available on request).20 22 The PC PMOS consists of a five-point Likert scale with higher scores indicating safer primary care. The PC PMOS also captures patient-reported safety incident data.

Intervention phases

The intervention comprised an iterative process with a cycle of measurement, learning, feedback, action planning and implementation period of 6 months (figure 1).

Figure 1

Intervention phases.

Patient feedback about the safety of their care was measured using the PC PMOS tool at baseline (time 1-T1). Primary care teams then used patient feedback from the PC PMOS to develop and implement specific safety interventions over a 6-month period. Patient feedback about the safety of their care was measured again (PC PMOS) at the end of the intervention period (time 2-T2).

Primary care practices were asked to form safety improvement teams (SIT). These teams comprised a minimum of three members and included any combination of practice manager, practice nurse, receptionist or administration staff or general practitioner.

SIT members participated in two learning and development workshops on teamwork, communication, implementation planning, the model for improvement’s (MfI) Plan-Do-Study-Act (PDSA) methodology31 and trial information.

PC PMOS data from each practice were collated and presented to the SIT at an action planning meeting. SIT members considered which area(s) of safety improvement to target, and developed goals, measures, ideas and PDSA cycles. SIT members were responsible for implementing and monitoring their specific safety intervention/s through application of multiple PDSA cycles over the 6-month period.

Data collection

Primary outcome

Feasibility measures included acceptability, intervention fidelity, implementation enablers and barriers, and scalability. These data were collected using three qualitative methods:

Recordings and overt observations of SIT members at workshops and action planning meetings,

Semistructured interviews with SIT members at trial conclusion,

Reflexive researcher journaling.

Audio data were transcribed verbatim. Overt participant observation data were recorded using detailed field note diaries and regular researcher discussion and reflection. Approximately 31 hours of audio was recorded with participants at workshops (2×3 hours), action planning meetings (6×1.5 hours) and semistructured interviews (16 hours—13 discrete individual or group interviews).

Secondary outcomes

Patient feedback on contributing factors to safety

Every adult (≥18 years) presenting for their appointment was invited by the practice receptionist to complete the PC PMOS over a 3-week period. Patients returned their surveys via a secure survey return box in the practice waiting room. Surveys were anonymous and completion was voluntary.

Patient-reported safety incidents and concerns

The PC PMOS contains questions for patients to report any patient safety incident. Questions were adapted from the ‘Patient Incident Reporting Tool’ used in the Patient Reporting and Action for a Safe Environment intervention.32 The PC PMOS has an ‘other comments’ free-text question which also provides patients the opportunity to report safety incidents or concerns.

Staff safety culture

The validated Agency for Healthcare Research and Quality Medical Office Survey (MOS) on patient safety33 was used to obtain data about staff safety culture perceptions at baseline (T1) (prior to patient data collection) and after the intervention (T2). All staff were invited to complete the survey and return it to the researcher via a provided prepaid envelope. Surveys were anonymous and completion was voluntary.

Safety incident reports

Practice managers provided a deidentified copy of their practice’s clinical risk management/safety incident register from the previous 12 months at T1 and T2. Due to lack of detailed data provided on the register, specific analysis or categorisation of the safety incidents was unable to be performed. However, the type of incident and any patient demographic data (age, gender) were cross checked with the patient-reported safety incidents on the PC PMOS to assess for similarities or differences.

Data analysis

Triangulation and thematic analysis techniques were employed to analyse the qualitative and content data. Both inductive and deductive approaches were used to undertake the analysis.34 Deductive approaches used the literature about healthcare culture and safety improvement, patient feedback and response theory, health service implementation science, and engagement and adaption theory.25–27 35 36 Inductive coding was also performed on qualitative and content data by three researchers (ALH, SJG and HB). The initial coding framework centred on the feasibility measures of intervention enablers, barriers, acceptability, fidelity and scalability. This framework was expanded through constant comparison with the data to create the final coding framework. Discrepancies between researchers were resolved through discussion. NVivo (QSR International) was used to support the analysis.

Intervention fidelity score

Intervention fidelity refers to the implementation of safety improvement interventions being delivered as intended.37 The number of safety interventions implemented at each practice was assessed by the research team using a three choice response option—yes, no or partially.

Quantitative data were analysed using SPSS Statistics (IBM V.24). Continuous variables were compared preintervention and postintervention using t-tests, while comparisons for non-parametric data used the Mann-Whitney U test. Categorical variables were compared using X2-tests. Results were considered statistically significant where p≤0.05.

The MOS percent positive scores for each 10 patient safety culture composites, the average score across the 10 composites, and the overall patient safety rating were calculated at T1 and T2 for each practice, and overall using t-tests.

Patient and public involvement

Patients directly participated in the priority setting of safety interventions at a local level. Specifically, patients concerns or experiences with systems for safety in the primary care environment (eg, access to care, communication, information and referral processes, organisation and care planning) were acted on by primary care teams through development and implementation of interventions which prevent safety incidents from occurring.

Informed consent

Patient consent was implied by completion and return of the PC PMOS questionnaire. This was stated on the plain language statement accompanying the PC PMOS questionnaire. All staff who participated in a semistructured interview with the researchers provided written consent to participate. Each practice manager provided written practice consent for the research to be undertaken at their practice.

Results

Primary outcome

Representative participant quotes corresponding to feasibility measures are presented in table 1.

Table 1

Key participant quotes corresponding to feasibility measures

Feasibility measure	Theme	Subtheme	Participant quote
Acceptability	Attitude towards patient feedback on safety	Value patient feedback on safety	‘It's always, the valuable ones are always the awful ones, aren't they? You know… it's really precious. Ain't often people are honest like that…’ (GP, Practice D, APM)‘It’s better to be informed about it so that you can make that change…it makes it more positive for everybody then’…(Administration Staff, Practice E, APM)
		Patient feedback on safety aligned with staff awareness of issues	‘the bits that were flagged that were in there [feedback report] were probably what we expected …’ (PN, Practice D, APM).
		Believability of the feedback	‘And I accept the [safety incident] one, because, perception is truth.’ (GP, Practice B, APM).
		Concern and empathy towards patient feedback	‘… there's one [safety incident] I was actually concerned, there's a patient who obviously feels that we haven't done our best by them.’ (PM, Practice A, APM).‘So someone had a blocked airway. That sounds really terrible, doesn't it? It's (an) emergency.’ (PN, Practice A, APM).
		Surprised or unsure how to respond when feedback differed to staff perceptions	‘I thought we have got some more negative feedback from people, which surprised me.’ (PN1, Practice C, APM).‘But I'm not quite sure about that [safety incident] one… I found that one very odd, because… probably some of the best staff we have are down that end of the building, without being horrible to others, but the doctors even say that. I just find that really odd.’ (PM, Practice B, APM).
		Dismissive towards patient feedback	‘I think sometimes it’s that lack of understanding, that they [GP] can’t come and fix the world in fifteen minutes’ (PN1, Practice C, APM).‘when you get that: ‘I can’t see the doctor that I want to see’ [patient comment]. Well we’ve been working on this for five years trying to improve things!’ (PM, Practice D, Final Interview)
	Using patient feedback to make changes	Cautious about using patient feedback for safety improvement	‘But we need to… you know, decide on what, what we think's important to change… and what's changeable. And I don't think we can do anything about this [safety incident].’ (GP, Practice A, APM)
		Largely positive feedback limited staff response	‘We didn’t have too many negatives [feedback] which is a good thing but also, it was sort of, well do we need to change that much?’ (PM, Practice A, Final interview)
		Using staff identified areas of service improvement rather than patient feedback	‘…even though it's not, it's not showing up as negative as I thought it might've, so I was really happy about that, but I think the appointment system will still [need to be addressed]… And I think that will assist the, there's less likely to be an error. So there's less likely to be a, ah, negative outcome for the patient’ (PM, Practice A, APM).
Barriers and enablers to intervention development and implementation	Developing interventions	Intuitive problem-solving process	‘We're probably doing it anyway, but we don't realize it's a model for improvement.’ (PN2, Practice C, Final interview)‘So we [other administration staff] we probably collaborate a lot. We throw ideas around. You know how to do different things. So we're probably the thinkers.’ (Admin, Practice F, Final interview)
		Disconnect between staff problem-solving process and MfI framework	‘It was a good framework. Initially, what we found was when barriers kind of ah developed, we had trouble readjusting to that [MfI framework].’ (GP, Practice F, Final interview)‘I didn't ever use a model I was just sort of like, ‘This is what I'm trying to achieve. This is how I'm going to do it’… Did it work? Didn’t it work? Which is probably the same model, but I just didn't actually outline it or ever document it. It was just in my head.’ (PN, Practice D, Final Interview)‘We are not very keen of formally doing that [sic.] things [MFI]. The simple the better.’ (GP, Practice E, Final Interview)‘…we probably were never really good at documenting that stuff. Document … as I said, in here you're kind of doing things on the run, do you know what I mean? You go, ‘Oh yeah, we'll do that.’ (PM, Practice E, Final Interview)
		Integrating and adapting problem-solving approaches	‘[The model for improvement) is a good process and it's simple but sometimes we complicate it by making it bigger than what it is’ (PN1, Practice C, Final Interview).‘ … [we] do the PDSA cycle, not necessarily super formally but we just, we identify what needs to be done and we try to make our changes small not big and then we introduce those to the practice or to specific members of the practice team who might need to know about it.’ (GP, Practice A, Final Interview)
	Implementing interventions	Multidisciplinary team	‘I just figured that it would end up falling probably on the three of us(PM, PN, Admin). Because I knew [GP] was going to be time poor… So he was there if we needed him and we would bug him.’ (PM, Practice B, Final Interview)
		Staff responsibility and ownership for intervention linked to type of improvement activity	‘I like data. I like playing with data [laughter]. I enjoyed doing a lot of the collection and stuff and seeing what you can do to make it happen…’ (PM, Practice C, Final Interview)
		Difficulty in measuring change in safety outcomes	‘It is difficult to measure outcome because if you prevent a complication, it [is] what it is’ (GP, Practice E, Workshop 2)
		Use of soft measures	‘…because there were things that we couldn't really kind of quantify. I mean, how do you quantify [staff member] stress level based on one particular aspect and you know separate it from…? That was what we had trouble with, more than anything.’ (GP, Practice F, Final Interview)
		Staff support and engagement	‘Nobody wanted to be part of the safety improvement team, like, as soon as [you] mention anything like this, everyone’s just like [pause] ‘Not again’.’ (PM, Practice D, Workshop 2).
		Time and resources	‘I felt as though we could have actually used a, ‘Alright, what's going wrong? Let's troubleshoot this and see.’ I don't think as a team, we were able to devote the time or the resources or energy to actually do that when we hit those barriers.’ (GP, Practice F, Final Interview)‘A lot's changed in the practice since we [started the trial]. A lot of fairly massive things. We've taken on 50% more students, we've got a few more extra learners, we've got a few other things going on plus we've had just some stuff, health issues, which have had a huge impact.’ (GP, Practice A, Final Interview)
Trail scalability		Increased facilitation and support from research team	‘I think the workshops were valuable. I don't know whether we can just blame the [intervention barriers], I suppose our lack of engagement with [the intervention]. Maybe if we had to engage a little bit more, it probably would have kept us on track a bit more I think… even if it was just on the phone or something.’ (PM, Practice D, Final Interview)‘I think you need somebody that's there as the overseer to keep us on track.’ (PM, Practice B, Final interview).
		Real-time electronic patient feedback processes	‘Something electronic I think we’d definitely be interested in. Even things, like the emails and text messages and stuff to people after they've been to their appointment, people don't have to do them then and there. They can sit on their couch at home and do it at night when they've actually got time… I would imagine we would get different feedback if patients were being surveyed after their appointment.’ (PM, Practice D, Final interview).

MFI, model for improvement.

Acceptability

Intervention acceptability

The majority of staff found the intervention acceptable. Staff reported that the intervention was predominantly positive and fitted within current organisational approaches to quality improvement.

Attitude towards patient feedback on safety

All staff valued patient feedback on safety. Positive feedback was welcomed and viewed as contributing to workplace morale, job satisfaction and reassurance that staff were meeting patient expectations. Feedback on safety was accepted when it aligned with staff awareness of issues. Furthermore, staff acceptance of the patient’s reality also influenced believability of the feedback.

Staff exhibited a range of responses to negative patient feedback, including: acceptance, feelings of empathy, surprise or uncertainty, or being dismissive of feedback.

Commonly mentioned reasons for dismissing patient feedback involved unrealistic patient expectations; deeming patient concerns too problematic to fix or out of the practice’s control; previous attempts to solve the problem have failed; or the patient was a known difficult patient (some staff speculated who a patient was even though the survey is anonymous).

Using patient feedback to make changes

Some staff were cautious about using the patient feedback for safety improvement activity. They contextualised the feedback in terms of where it may be coming from and how appropriate it would be to respond. Additionally, some mentioned difficulties in choosing priority areas to address due to largely positive patient scores limiting what they could respond to.

Four of the six practice teams saw this trial as a catalyst for undertaking improvements that aligned with previously identified staff priorities, and not responding directly to the patient feedback. Two practice teams attempted to link their chosen safety interventions back to domains of safety on the PC PMOS. For example, improving waiting time or availability of appointments was a focus area for staff yet the PC PMOS scores relating to access to care were largely positive. The other two practices did not attempt to link their previously identified target area to a PC PMOS domain of safety. The remaining two practices chose to address areas that were directly related to areas of concern highlighted from the patient feedback. This was either a patient-reported safety incident or a negatively scored PC PMOS domain.

Implementation of safety interventions

Intervention fidelity

The average intervention duration of 5.8 months was considered acceptable by most practice teams. Among the six practices, 25 safety improvement interventions were developed at the action planning meeting or during the implementation period. Of these, 17 (68%) were fully implemented, 2 (8%) partially implemented and 6 (24%) not implemented.

The safety priorities targeted at the six practices included improvement in the following areas: communication of patient recall and reminders, access to equipment and supplies, access to care, accuracy of patient information, management of staff time, patient experience of waiting time, and patient knowledge of registrar skills and abilities. There were no differences observed in success of interventions that addressed either relational (communication, behavioural change, etc) or transactional issues (data cleaning, equipment and supplies, etc). Other mediating and contextual factors in the practice environment were attributed to the success or failure of safety interventions by staff.

Barriers and enablers to intervention development and implementation

Developing interventions

Staff employed both intuition and problem-solving processes to develop safety interventions. This process appeared to be an enabler for practice teams. This often took the form of a rapid and informal root cause analysis where common sense and a pragmatic approach was apparent. This process did not require external facilitation and staff were easily able to identify latent conditions in the practice that contributed to the safety concern. Staff reported regular use of this approach for safety and quality improvement activities unrelated to this project, but had not recognised it as formal improvement work.

Some teams experienced challenges with translating their intuitive problem-solving approach onto the MfI framework. There was a perceived disconnect between the two problem-solving methods. This mainly related to adjusting to new habits or ways of working and adhering to a structured process. Practice teams with greater quality improvement experience were better able to integrate these approaches and adapt accordingly.

Implementing interventions

The high intervention fidelity shown in this trial was attributed to various factors. One key enabler was the multidisciplinary dynamic within the SIT. The teams largely consisted of a practice manager, administration staff member and a practice nurse. GPs adopted a more passive role in implementation. Nonetheless, GPs were engaged and supportive of the SIT and provided leadership and support when needed. Since most SITs comprised a practice manager, administration staff and a practice nurse, it was difficult to make comparisons about the effectiveness of teams that had different combination of staff roles.

Practice managers and administration staff often took primary responsibility and ownership for safety intervention implementation. As the interventions addressed the latent conditions within the primary care system that contribute to safety incidents, the corresponding activities and tasks often required input from administration staff rather than clinical staff. For example, ensuring patient demographic information was up to date or improving appointment scheduling were viewed as tasks to be undertaken by administration staff who are skilled and knowledgeable in this area.

Staff generally agreed that the MfI was a useful and familiar structure for implementing safety interventions. However, a few teams experienced some implementation challenges relating to the prescriptive nature and linear processes proposed in the model. Lack of model flexibility and adaptability were commonly cited as implementation barriers.

Staff also found measuring change difficult for various reasons. Identifying an appropriate measure directly relating to their safety intervention was challenging. For example, some staff indicated it was difficult to measure clinical outcomes or safety incidents averted. Often soft or proxy measures were used due to unavailability or inaccessibility of data.

Staff identified a number of other barriers to implementation. These were common across all practices and included lack of protected time, demanding priorities particularly for patient care, issues with staff recruitment and retention as well as staff leave, power and team dynamics, management support, and engagement from the wider practice.

Scalability

Staff recommended some improvements to the structure and components of the intervention that would enable future scale-up to a larger effectiveness trial or spread into policy and practice (box 1).

Simplification of intervention framework.

Structured and defined intervention actions plans and corresponding safety measures for each of the primary care patient measure of safety domains of safety.

Electronic data collection platforms to enable real-time patient feedback.

Increased external intervention facilitation.

Modification to questionnaire collecting patient-reported safety incidents.

Existing practice infrastructure and resources were deemed adequate for participation.

The two learning workshops and facilitated action planning meeting with the research team were viewed as important. While the majority of staff felt that this level of facilitation was adequate, others suggested additional action planning meetings throughout the intervention phase would assist with accountability and implementation progress.

Patient data collection using the PC PMOS was considered relatively straightforward by practice staff. Only one practice (Practice A) failed to complete T2 data collection. Reasons for this included staff leave and patient survey fatigue. As the PC PMOS was a paper-based survey staff felt that improvements could centre on electronic data collection to increase the efficiency of real-time patient feedback, for example, via the use of waiting room iPads or emails to patients after their consultation.

Secondary outcomes

Patient feedback on contributing factors to safety: PC PMOS scores

A total of n=1750 patients completed the PC PMOS at T1 and T2 (n=839 T1, n=911 T2), representing a practice mean of 140 and 182 at T1 and T2, respectively. The crude response rate was 10.7%, however, the average response rate across the practices was 40.6%. Patient characteristics are presented in table 2. Patients completing the PC PMOS were significantly more likely to be older and female (online supplementary appendix 2). Mean age was 56 years (SD 18.2) and mean number of visits to the practice in the previous 12 months was 8 (SD 8.6).

Table 2

Patient demographic characteristics

	Practice A		Practice B		Practice C		Practice D		Practice E		Practice F		Total
	T1	T2	T1	T2	T1	T2	T1	T2	T1	T2	T1	T2	T1	T2
Gender (n)	99	–	195	197	141	150	128	142	146	155	113	257	822	901
Male (n, %)	24 (24.2)	–	48 (24.6)	62 (31.5)	67 (47.5)	71 (47.3)	24 (18.8)	29 (20.4)	43 (29.5)	51 (32.9)	35 (31.0)	97 (37.7)	241 (29.3)	310 (35.4)*
Female (n, %)	75 (75.8)	–	147 (75.4)	135 (68.5)	74 (52.5)	79 (52.7)	104 (81.3)	113 (79.6)	103 (70.5)	104 (67.1)	78 (69.0)	160 (62.3)	581 (70.7)	591 (65.6)
Age (mean, SD)	53 (17.2)	–	55 (17.2)	55 (18.4)	63 (16.8)	61 (17.7)	47 (17.7)	50 (18.6)	57 (18.9)	59 (18.1)	55 (17.4)	54 (17.7)	55 (18.1)	56 (18.3)
Visits to practice in previous 12 months (mean, SD)	13 (15.9)	–	7 (5.9)	8 (5.9)*	7 (5.8)	8 (11.0)	8 (9.4)	9 (9.3)	7 (6.1)	8 (10.1)	8 (5.7)	8 (7.8)	8 (8.5)	8 (8.7)

T1=Time 1 (baseline).

T2=Time 2 (6 months postintervention period).

*Statistically significant difference between baseline and 6 months p<0.05.

The PC PMOS total mean scores and domain scores for each practice at both times points are presented in table 3. There was a significant increase in total mean PC PMOS score for all practices from T1 to T2 suggesting improved patient safety (4.30 (SD=0.49) to 4.37 (SD=0.47), p=0.002). There were also significant increases in mean scores for all practices from T1 to T2 for the following domains: access to care (4.09 to 4.23, p<0.001), communication (4.44 to 4.50, p=0.018), information flow (4.27 to 4.36, p=0.007) and patient-related factors (4.51 to 4.61, p<0.001). There was within and between practice variation for specific PC PMOS domain scores (table 3).

Table 3

PC PMOS overall and domain specific scores by practice

	Practice A		Practice B		Practice C		Practice D		Practice		Practice F		Total
	T1	T2	T1	T2	T1	T2	T1	T2	T1	T2	T1	T2	T1	T2
PC PMOS mean score (SD)	4.22 (0.67)	–	4.29 (0.46)	4.46** (0.40)	4.26 (0.50)	4.36 (0.46)	4.44 (0.42)	4.47 (0.44)	4.24 (0.49)	4.26 (0.49)	4.36 (0.40)	4.32 (0.52)	4.30 (0.49)	4.37* (0.47)
Accessmean (SD)	3.94 (0.84)	–	4.12 (0.64)	4.38** (0.60)	4.03 (0.67)	4.26* (0.67)	4.24 (0.71)	4.29 (0.67)	4.01 (0.76)	4.07 (0.75)	4.16 (0.68)	4.15 (0.76)	4.09 (0.71)	4.23** (0.70)
Communicationmean (SD)	4.37 (0.64)	–	4.41 (0.50)	4.56* (0.47)	4.40 (0.52)	4.47 (0.56)	4.60 (0.45)	4.60 (0.47)	4.37 (0.57)	4.44 (0.53)	4.51 (0.46)	4.45 (0.58)	4.44 (0.53)	4.50* (0.53)
External policy environmentmean (SD)	4.18 (0.82)	–	4.00 (0.88)	4.16 (0.90)	4.05 (0.97)	4.08 (1.04)	4.07 (1.08)	4.20 (1.00)	3.97 (0.94)	3.90 (1.01)	3.94 (1.05)	4.09 (1.03)	4.03 (0.96)	4.09 (1.00)
Information flowmean (SD)	4.18 (0.68)	–	4.27 (0.63)	4.46* (0.56)	4.26 (0.58)	4.42* (0.59)	4.42 (0.61)	4.41 (0.68)	4.18 (0.67)	4.29 (0.61)	4.31 (0.61)	4.25 (0.68)	4.27 (0.63)	4.36* (0.63)
Organisation and care planningmean (SD)	4.37 (0.55)	–	4.27 (0.61)	4.47* (0.62)	4.36 (0.56)	4.37 (0.57)	4.36 (0.54)	4.44 (0.49)	4.29 (0.53)	4.30 (0.66)	4.45 (0.56)	4.36 (0.65)	4.34 (0.56)	4.39 (0.61)
Patient related factorsmean (SD)	4.45 (0.78)	–	4.45 (0.65)	4.69 ** (0.54)	4.49 (0.70)	4.56 (0.71)	4.63 (0.60)	4.75 (0.53)	4.48 (0.58)	4.51 (0.73)	4.60 (0.57)	4.57 (0.65)	4.51 (0.65)	4.61* (0.64)
Physical environmentmean (SD)	4.47 (0.69)	–	4.48 (0.53)	4.65* (0.49)	4.58 (0.55)	4.57 (0.58)	4.60 (0.51)	4.63 (0.58)	4.47 (0.59)	4.50 (0.60)	4.64 (0.48)	4.47* (0.66)	4.54 (0.56)	4.56 (0.59)
Referral systemsmean (SD)	4.38 (0.6)	–	4.37 (0.56)	4.53* (0.54)	4.34 (0.57)	4.41 (0.59)	4.59 (0.54)	4.60 (0.55)	4.41 (0.49)	4.45 (0.56)	4.48 (0.53)	4.43 (0.65)	4.42 (0.55)	4.48 (0.59)
Task performancemean (SD)	4.04 (0.93)	–	4.10 (0.96)	4.10 (1.17)	4.00 (0.99)	4.01 (1.12)	4.36 (0.84)	4.20 (0.95)	3.97 (0.97)	3.70* (1.15)	3.85 (1.11)	4.02 (0.94)	4.06 (0.98)	4.01 (1.07)

T1=Time 1 (baseline).

T2=Time 2 (6 months postintervention period).

*P<0.05, **P<0.001.

PC PMOS, primary care patient measure of safety.

Patient-reported safety incidents and concerns data

Patient-reported safety incident data are presented in table 4. There were n=11 patient-reported safety incidents at T1, and n=9 at T2. The mean severity rating at T1 and T2 was 7.4 (scale 1–10 with 10 being ‘extremely serious’). The median preventability rating of these safety incidents was ‘definitely preventable’ at T1, and ‘probably preventable’ at T2.

Table 4

Frequency, preventability and severity of patient-reported incidents and concerns

	Practice A		Practice B		Practice C		Practice D		Practice E		Practice F		Total
	T1	T2	T1	T2	T1	T2	T1	T2	T1	T2	T1	T2	T1	T2
No of patient-reported incidents*	1	–	4	0	2	2	0	0	3	3	1	4	11	9
Average preventability rating (range)†	Definitely preventable	–	Probably not preventable	–	Definitely preventable, Probably preventable	Probably preventable, Don’t know	–	–	Definitely preventable	Probably preventable	Definitely preventable	Definitely preventable	Definitely preventable	Probably preventable
Average severity rating (range)‡	10 (10)	–	7.3 (6–8)	–	6.5 (3–10)	8.5 (7–10)	–	–	6.7 (6–8)	6.3 (6–7)	9 (9)	7.6 (7–9)	7.4 (3–10)	7.4 (6–10)
No of patient-reported concerns§	6	–	13	6	9	3	2	2	8	6	4	8	42	25
No of patient-reported concerns that were classified as safety incidents	3	–	7	3	1	1	1	2	3	3	2	3	17	12

T1=Time 1 (baseline).

T2=Time 2 (6 months postintervention period).

*Patient-reported incidents using Patient Incident Reporting Tool.

†Preventability scale consists of five options ‘definitely preventable’, ‘probably preventable’, ‘Probably not preventable’, ‘definitely not preventable’, and ‘don’t know’. Expressed as the median due to it being an ordinal variable.

‡Patient-Rated Severity Scale is 1–10 with 1=not serious at all and 10=extremely serious.

§Patient-reported concerns mentioned in ‘other comments’ section of the survey (total number of negative comments).

¶Patient-reported incidents mentioned in the ‘other comments’ section of the survey (PISA classification system was used to classify safety incidents.68

An additional n=17 safety incidents at T1, and n=12 at T2 were identified from the ‘other comments’ section of the PC PMOS. Therefore, the total number of patient-reported safety incidents was n=28 at T1, and n=21 at T2. The number of patient-reported concerns (negative comments that were not a patient safety incident) decreased from n=45 at T1 to n=25 at T2 (table 4).

Practice measures of safety

Staff perceptions of safety culture

A total of n=57 staff completed the MOS survey at T1, and n=61 at T2. For the total sample there was an increase in the mean percent positive score for the overall patient safety rating between T1 and T2, although not significant (72% to 74%, p=0.851). For the majority of the patient safety culture composites and the average across the ten composites there was a reduction in mean percent positive scores, with only one significant reduction for the Teamwork composite between T1 and T2 (89%–80%, p=0.029) (online supplementary appendix 3).

Safety incidents recorded on practice clinical risk management system

There was a reduction in the number of incidents recorded on practice’s clinical risk management system from T1 (n=32) to T2 (n=21) (online supplementary appendix 4). The incidents recorded on the practice clinical risk management system were different to the incidents reported by patients on the PC PMOS.

Discussion

This is the first reported patient feedback on safety intervention in Australian primary care. The findings indicate that the intervention is feasible for scale-up to a larger effectiveness trial and further spread into policy and practice. Staff deemed the intervention acceptable, with minor recommendations for improvement. Intervention fidelity was high and implementation enablers were attributed to the intervention structure and framework, use of intuitive problem-solving approaches and multidisciplinary team involvement. Barriers to implementation reflected previously reported problems undertaking quality improvement in primary care, such as lack of time and staff, demanding priorities, power and team dynamics, and wider practice support and engagement.25–27 The process of systematically collecting patient safety data was achievable with n=1750 patient surveys completed. The utility of the PC PMOS tool as a measure for safety was demonstrated through the significant increase in mean scores for all practices from T1 to T2 (4.30 to 4.37, p=0.002).

It is widely acknowledged that patient feedback is rarely used for safety and quality improvement purposes.38–46 This study identified some enablers and barriers that impacted on the intervention development and implementation including the team dynamic, improvement framework and staff attitude.

A unique aspect of this patient feedback on safety intervention was the multidisciplinary dynamic of the primary care teams, particularly administration staff leadership. This was considered a key enabler to intervention adherence and acceptability. The safety interventions targeted the contributing factors to safety incidents; as such, administration staff were ideally placed for intervention delivery. Administration staff transcended professional boundaries to generate engagement and support, and implement changes at the latent end of the primary care system. In this respect administration staff acted as change agents and innovators35 47 48 and future safety improvement work should consider their current underused role.

Although the Mfi’s PDSA cycle31 is considered an effective, adaptable and flexible framework for quality improvement in some contexts, practice staff in this study identified it as a barrier to implementation. Formalising and documenting action plans in PDSA cycles was often in disconnect to their natural problem-solving approach and routine practice. In a time, resource and capacity scarce environment it is important that safety improvement frameworks are simple, and easily integrate or mimic everyday work flow. There are several well established quality improvement models49–51 that could be utilised for this patient feedback on safety intervention, however more research is needed to identify and investigate staff acceptability and appropriateness of the different frameworks in this context.52

Staff attitude towards patient feedback on safety was similar to previous research, which reveals staff difficulty to engage with or value patient feedback.36 38 41 42 45 53–57 While staff described the value and benefit of seeking patient feedback on safety, this was not entirely reflected in action plans or translated during intervention implementation. More than half of the practice teams undertook safety interventions that were a priority for staff rather than a priority for the patient. Recommendations to improve staff action on patient feedback could centre on providing staff with structured and specific intervention examples that correspond to particular domains of safety on the PC PMOS. Moreover, such intervention examples could have explicitly linked measures of safety to each of the PC PMOS domains which may address the challenges staff experienced with creating measures of change.58

The process of systematically collecting primary care safety data from the practice, staff and patients was acceptable and feasible, yet some consideration is needed when determining appropriate measures of intervention effectiveness in a larger trial. Data about patient safety in primary care in Australia is largely absent. Australia does not have a structured or connected reporting and learning system to understand the threats to patient safety, and there is no current systematic way to collect information about safety incidents or patient harm.59 60 Using available sources of patient safety data in this study revealed some limitations; as such, objective measures of intervention effectiveness like statistical control charts,61 62 PDSA cycle evaluation tools,63 64 and record review65 66 are recommended.

The findings from this study support results from other studies which have investigated patient feedback for safety improvement. While the majority of the research centres on hospital settings, the positive effect of patient feedback has been determined.3 8–12 One systematic review identified gaps in understanding regarding the enablers and barriers for implementation of patient feedback interventions.8 The findings from this study add to the discourse in this under-researched area.

A limitation of this study was the sample. The practices were from one regional area, which may limit the generalisability of the findings. However, the diversity within the practices was considered adequate for this feasibility study. All practices had participated in one or more of the Australian Primary Care Collaborative Programme67 waves previously. Their commitment, interest and understanding of safety and quality improvement processes was potentially already elevated prior to study commencement when compared with other practices. Results suggest the merit of conducting a larger scale effectiveness-implementation trial to determine the translatability of this intervention programme and safety outcomes to primary care practices more generally.

Conclusion

This study’s findings have demonstrated the feasibility of introducing an innovative patient feedback for safety improvement intervention in primary care, as well as contextual and intervention factors that promote safety improvement. The intervention complements existing safety improvement strategies and activities, and integrates into current patient feedback service requirements for primary care. Further research is needed to examine the intervention effects on safety incident reduction.