Assessing patient experience with patient safety in primary care: development and validation of the ASK-ME-questionnaire.

OBJECTIVE: To develop and test the validity and reliability of a tool measuring patient experiences with patient safety in ambulatory care that is suitable for routine use in general practitioner and specialist practices.
DESIGN: Instrument development was based on a literature review, a 3-round Delphi survey with a multidisciplinary expert panel and cognitive interviews with patients. The instrument was piloted in 22 practices using a cross-sectional survey. Exploratory (EFA) and confirmatory factor analysis (CFA) were performed to test construct validity. Internal consistency and the ability of the questionnaire to differentiate between selected subgroups and at the level of individual practices was examined.
SETTING: General practitioner and specialist practices. PARTICIPANTS: Patients aged >18 years seeking care in ambulatory care practices between February and June 2020.
RESULTS: The final ASK-ME-questionnaire consisted of 22 items covering 5 theoretical dimensions. A total of 3042 patients (71.1%) completed the questionnaire. Median item non-response rate was 4.2% (IQR 3.4%-4.7%). EFA yielded 3 factors comprising 14 items explaining 64.8% of the variance representing contributing factors to patient safety incidents. CFA confirmed the factorial structure suggested by EFA. The model fit the data satisfactorily (comparative fit index=0.92, root mean square error of approximation=0.08, standardised root mean square residual=0.08). Internal consistency values ranged from 0.7 to 0.9. Discriminant validity was supported by significant differences between patients of different age and differences in self-reported health status. The factors distinguished well between practices.
CONCLUSION: The ASK-ME-questionnaire showed good psychometric properties. It is suitable for routine use in patient safety measurement and improvement systems in ambulatory care. Further research is required to adequately assess number and type of experienced events in routine measurements. © Author(s) (or their employer(s)) 2022. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

We used a rigorous development process with patients being involved throughout the process of drafting and consenting the content as well as testing the face validity of the instrument.

The brevity of the questionnaire facilitates routine use and thereby involvement of patients in patient safety in daily practice as well as high response rates.

The self-report measure excludes patients with insufficient reading and language skills who are known to be at risk for patient safety events.

The self-selection of both practices and patients can lead to systematic differences between the participants and the non-participants.

Background

Ambulatory care is the first point of contact with the healthcare system for most patients and the context where the majority of these contacts take place. Research and debate on patient safety have long focused on inpatient care, not least because the risks were considered higher and the need for action greater than in ambulatory care. However, available research on the prevalence of patient safety events in primary care indicates that they occur in 2%–10% of the consultations.1 2 The main safety-related areas in primary care refer to medication safety,3 diagnosis4 and communication between healthcare professionals and communication with patients.1

It is increasingly acknowledged that involving patients in the safety of their care is an important aspect in systematically identifying safety problems in primary care.5 6 Available evidence indicates that patients are able to identify factors that correlate with the occurrence of safety incidents.6 7 Their reports are reliable8 9 and offer unique information that cannot be gathered otherwise.10 11 Existing measures for patient safety in primary care mostly focus on specific aspects of care or use the reports from health professionals.12 13 Instruments measuring patient safety in primary care from the patients’ perspective do either assess factors that contribute to patient safety events but not their occurrence or may not lend themselves to routine measurement due to their length.14–16

The study presented here was part of a project funded by the German Ministry of Health. The project’s aim was threefold: the first aim was to develop a questionnaire measuring factors that contribute to the occurrence of patient safety events from the patients’ perspective as well as the frequency of those events. The measure should be applicable in all practices (generic), suitable for routine use (short), for internal quality management (actionable) and available in the public domain. The second aim was to generate evidence on the status quo of patient experience with patient safety in German ambulatory care and using these first results to inform policy makers on potential improvement strategies. The third aim was to develop a feedback mechanism that would enable an automated and user-friendly feedback in future surveys, thereby fostering the systematic involvement of patients in the prevention of adverse events in ambulatory care settings. In this article, we report on the development and validation of the questionnaire.

Methods

Item development

A patient safety event (PSE) was defined as an occurrence (incident, process, procedure or outcome) that increases the risk for an adverse event or actually leads to an adverse event.17 18 A contributing factor is ‘a factor, circumstance or influence that is thought to have played a part in the origin or development, or to increase the risk, of an incident’.18 A literature search on relevant dimensions and existing tools to measure PSE and contributing factors in ambulatory care from the patient perspective was conducted. Based on these results, a pool of potential survey items was compiled for discussion and consensus in a modified three-round Delphi process. Experts were asked via email to review the dimensions and items in terms of their relevance to patient safety (rounds 1 and 2) and whether the respective items should be included in the questionnaire (rounds 1–3). The resulting questionnaire was assessed by cognitive interviews with patients. Participants were asked to comment on clarity, comprehensibility, ambiguity, redundancy and relevance of questions and answer options. Based on the results of the interviews, the questionnaire was finalised.

Data collection: study design, setting, sample, recruitment

A cross-sectional study with a mixed-mode survey design was used to collect data from patients receiving ambulatory care. Practice staff handed out the questionnaires (including cover letter and return envelope) to consecutive patients. The cover letter contained information on the study, the URL and access code for the online survey. Staff informed patients that participation is voluntary, that no disadvantages would arise from non-participation and that they could decide against participation at any time. Completion of the questionnaire was considered as consent. Patients could complete the questionnaire on paper or online either on-site or at home. Patients were asked to fill in the questionnaire only once. To ensure identification of possible duplicates, the access code corresponded to the questionnaire ID. For collection of questionnaires that were answered on-site, a poll box was installed in the waiting room of each participating practice. In order to analyse the non-response bias, an additional ‘non-responder-page’ was attached to each questionnaire. Gender, age group and educational level of the participant were noted on this page, collected in the practice and sent to the project team.

Given the threefold aim of the study, 50 practices should be recruited between December 2019 and March 2020. Recruitment was undertaken by the researchers and supported by the regional Association of Statutory Health Insurance Physicians (Westfalen-Lippe) and the German Coalition for Patient Safety. The participating practices received an expense allowance of €200 per practice. Sample size was calculated on the basis that in the intended survey period of 6 weeks an average of 800 patients are seen in the practices. Assuming a variance of 70%, a sampling error of 10% and a confidence level of 95%, the sample size for this population was 74 patients per practice. In order to achieve this value with the assumed response rate of 25%, 250 questionnaires needed to be issued per practice. All patients with a minimum age of 18 years and sufficient proficiency of the German language were eligible to participate in the survey.

Data analysis

Descriptive statistics were generated for each item to evaluate the distribution of the scores. Items were excluded from psychometric evaluation if there were >15% missing values (either due to omitted answers, ‘does not apply’ answers or due to filtering questions).19 Three items were reverse-coded, so that a higher score always indicated a more positive response.

For psychometric evaluation, the data were randomly split into two equal groups. To investigate the pattern underlying the responses, exploratory factor analysis (EFA) using maximum likelihood estimation and promax rotation was performed on dataset A. Items with low communality (<0.3), measure of sample adequacy <0.6 or multiple factor loadings >0.4 were considered for elimination. The number of factors to extract was based on the minimum average partial test, Scree-plot and interpretability of the factors extracted. For assessing the fit of the factorial structure identified by EFA, confirmatory factor analysis (CFA) was performed on dataset B. Guidelines for testing model fit followed Gärtner et al20: a comparative fit index (CFI) ≥0.90, a standardised root mean square residual (SRMR) ≤0.08 and a root mean square error of approximation (RMSEA) ≤0.08. Values of ≥0.4 for standardised factor loadings were considered acceptable.

Using the whole sample, internal consistency was assessed by item total correlation and Cronbach’s α coefficient. Alpha values of 0.7 or higher were considered to indicate good reliability. Items with an item total correlation <0.4 or items that would lead to a higher α when removed were considered for elimination.21

The ability of the questionnaire to differentiate between selected subgroups and at the level of individual practices was examined by analysing mean differences depending on the scale level using the appropriate significance tests. In extant literature, patient characteristics such as subjective health status and age, sometimes also gender and education22 23 have been found to be associated with patient experience. Methodological issues such as place of data collection (on-site vs mail-back) have also been reported to be associated with patient feedback of care experience.24 Finally, the ability of the questionnaire to distinguish between practices was determined. Data analyses were performed using SPSS V.26 and AMOS V.26.

Patient and public involvement

The development of the questionnaire was supported by experts from different fields, including patients and physicians, within a Delphi procedure. To check for comprehensibility and feasibility of the questionnaire, patients were involved through cognitive interviews.

Results

Based on the scoping review, 75 items in 5 dimensions (access, communication, medication safety, care coordination and experience of PSE) were compiled for expert review in the three-round Delphi process. From the 11 patient safety experts (4 researchers, 1 member from the German Coalition for Patient Safety, 4 patients, 2 clinicians), all participated in round 1, 10 experts took part in round 2 and 9 in the final round. Items considered for inclusion by at least half of the experts were kept in round 3. Items considered for inclusion by at least two-thirds of the experts in round 3 were included in the questionnaire. From the 50 items relating to factors contributing to PSEs 21 were included in the pretest version of the questionnaire. For the 25 questions referring to experience of actual PSEs and resulting harm, experts decided after extensive discussion to dispense with these detailed questions altogether. Instead, three generic questions were included, introduced by a short text explaining the nature of potential PSEs. The main reason for this decision was the aim of developing a generic, self-report questionnaire suitable for routine use in general practitioner as well as specialist practices where a reasonable questionnaire length is critical. Due to the lack of opportunity to clarify questions in self-report measures, it was thought that a detailed recording of PSEs would have required a comprehensive list of PSE examples. However, in settings where ambulatory care is provided by specialists in private practices as well as in general practitioner practices this list would have had to be either practice-specific or very extensive. To ensure an appropriate questionnaire length, it was thus decided to cover experience of PSEs and resulting harm with three generic questions.

Cognitive interviews were conducted with seven female and seven male patients, aged between 24 and 71 years from different educational backgrounds, with five suffering from a chronic disease. Interviews lasted between 50 and 90 min. Interviewees thought the questionnaire overall as easy to understand and complete, relevant, well-structured and comprehensive. They understood the majority of the questions as intended. For three items the wording was revised and for one item an additional ‘does not apply’ answer was added. The question about whether risks and benefits of the treatment or alternative treatment options were clearly explained was split into two questions because none of the interviewees had so far experienced the situation that alternative treatment options had been proactively explained by their respective physicians.

The pretest version of the questionnaire consisted of 22 items covering 5 dimensions (access, communication, medication safety, care coordination and experience of PSE). Response options were on a 5-point Likert scale ranging from ‘always’ to ‘never’. For eight questions, a ‘does not apply’ answer was provided. Five questions captured respondents’ demographic characteristics.

Sample

A total of 22 practices were recruited for the study. Where reasons for declining to participate were given, they mainly referred to time restraints, involvement in other projects and doubts about the validity and usefulness of patient feedback on patient safety. Data collection was intended to take place between February and April 2020. Due to the outbreak of the COVID-19 pandemic in March 2020, the survey period was extended until June 2020. On average, practices handed out 194 (26–250) questionnaires. The overall response rate, based on the number of distributed questionnaires, was 71% (23%–99%). From the 3042 questionnaires 28 had to be excluded from analyses due to a patient age <18 years, thus feedback from 3014 patients was available for analysis. Characteristics of practices and patients can be seen in table 1. The majority of the patients (96%) made use of the paper and pencil option, 77% completed the questionnaire on-site.

Table 1

Characteristics of survey respondents and practices

Characteristic	Total sample (%)	EFA sample (%)	CFA sample (%)
	N=3014	N=1531	N=1483
Age (years)
18–35	20.7	20.8	20.8
36–50	23.1	23.8	23.8
51–65	28.2	28.5	28.5
>65	22.8	22.0	22.0
Missing	5.2	4.9	4.9
Gender
Female	63.5	63.9	63.2
Male	31.5	30.7	32.3
Divers	0.5	0.4	0.7
Missing	4.4	5.0	3.8
Education
University degree	18.0	16.9	19.2
Vocational training degree	37.2	37.0	37.3
Grammar school (A -level)	8.3	7.8	8.7
Intermediate secondary school	13.6	14.0	13.1
Secondary general school	15.1	15.2	14.9
No degree	3.0	3.5	2.5
Missing	4.9	5.4	4.3
Condition existing >3 months
Yes	53.9	54.3	53.4
No	41.0	39.9	42.2
Missing	5.1	5.7	4.4
Health status
Excellent	7.7	6.3	9.0
Very good	21.2	21.1	21.4
Good	39.8	40.9	38.6
Fair	22.3	22.6	22.0
Poor	4.7	4.4	4.9
Missing	4.3	4.6	4.0
Practice (n (%) patients)
General practitioner (n=9)	33.6	35.5	31.6*
Specialist† (n=13)	66.4	64.5	68.4*

*P>0,05.

†Obstetrician and gynaecologists (n=3), ear, nose and throat specialists (n=2), surgeons (n=2), cardiologists (n=2), internists (n=1), neurologists (n=1), ophtalmologists (n=1), gastroenterologists and oncologists (n=1).

CFA, confirmatory factor analysis; EFA, exploratory factor analysis.

The ‘non-responder page’ was collected for 3915 of the 4276 distributed questionnaires. Thus, information is available for 71% (873 of 1234). Non-responders were older (56 vs 51 years, p>0001) and less likely to have an A-level or university degree (19% vs 28%, p<0001). No difference was found between the two groups in terms of gender.

Acceptability

All items had answers that included the full range of the response scales. Median item non-response rate was 4.2% (IQR 3.4%–4.7%), which is considered low to moderate.25 Missing data did not increase with question number.

Exploratory factor analysis

For the purpose of exploring and confirming the factorial structure of the questionnaire, the data were randomly split into two nearly equal subsamples. ‘Does not apply’ answers were coded as missing data. The proportion of missing data per item for each dataset is provided in online supplemental file 1. Both samples differed slightly in the number of patients per practice specialty (table 1).

Items with >15% missing data (either due to omitted or ‘does not apply’ answers (Q14, Q18, Q19, Q20) were not incorporated in the EFA. For thematical considerations an exception was made for Q15.2 with 16.4% missing data, leaving 18 items for initial analysis. Three items (Q01, Q02, Q16) were excluded from the final model due to communalities <0.3 and one item (Q12) was excluded due to multiple factor loadings >0.4 (online supplemental file 2). No item had to be excluded due to a measure of sample adequacy <0.6, leaving 14 items in the final solution. The Kaiser-Meyer-Olkin measure of sample adequacy of 0.94 and the highly significant Bartlett’s test of sphericity (p<0.001) supported the data’s factorability. For content-related reasons, the eight items excluded from EFA were kept in the questionnaire (online supplemental file 2).

EFA yielded three factors explaining 64.8% of the variance. The factor loadings and communalities are shown in online supplemental file 2. The factors reflect the themes communication and information (factor 1, 6 items), rapport and participation (factor 2, 4 items) and medication safety (factor 3, 4 items).

Confirmatory factor analysis

To assess construct validity of the measure, CFA was performed with dataset B. The CFA confirmed that the three-factor structure provided the best fit for the data with goodness-of-fit indices within the limits proposed by Gärtner et al20 for patient-reported experience measures (CFI=0.92, RMSEA=0.08, SRMR=0.08). Table 2 shows the standardised factor loadings, the item R2 values and the goodness-of-fit indices.

Table 2

Standardised factor loadings, item R2 and goodness-of-fit indices

Factor	Item	Standardised factor loading	Item R2
Communication and information	Q03	0.7*	0.55
	Q04	0.8*	0.64
	Q05	0.8*	0.62
	Q06	0.8*	0.61
	Q07	0.7*	0.5
	Q08	0.7*	0.5
Rapport and participation	Q09	0.7*	0.53
	Q10	0.8*	0.6
	Q11	0.7*	0.52
	Q13	0.8*	0.62
Medication safety	Q15.1	0.8*	0.64
	Q15.2	0.6*	0.3
	Q15.3	0.7*	0.51
	Q17	0.5*	0.27

Model fit indices: χ2(74)=10.7, p<0001; RMSEA=0.08, p=0.01; CFI=0.92; SRMR=0.04.

*P<0.001.

CFI, comparative fit index; RMSEA, root mean square error of approximation; SRMR, standardised root mean square residual.

Internal consistency

Using the whole sample, reliability analyses were performed for each item group within the factors defined by the factor analyses. Table 3 shows descriptive reliability statistics for the subscales.

Table 3

Descriptive statistics, Spearman’s correlation and internal consistency reliability (whole sample (n=3014))

	Factor
	Communication and information	Rapport and participation	Medication safety
N	2930	2883	2538
Mean (SD)*	1.39 (0.52)	1.67 (0.71)	1.52 (0.65)
Skewness	1.84	1.33	1.67
ITC	0.7 (0.6, 0.8)	0.6 (0.6, 0.7)	0.5 (0.4, 0.6)
Cronbach’s α	0.9	0.8	0.7

*Range: 1 (most positive experience) to 5 (most negative experience).

ITC, item total correlation.

The composite subscale scores were calculated by averaging the item scores within the same composite for each respondent. For cases where more than one item had a missing value, no score was calculated. All subscales had means in the upper half of the range meaning that positive experiences were more frequent than negative experiences. All subscales were skewed to the right. The item total correlation were ≥0.5 for all items except one (Q17), which is considered as high.26 Cronbach’s α values ranged from 0.9 to 0.7, indicating very good to acceptable internal consistency. Eliminating further items would not have led to improvement for any of the three subscales.

Discriminant validity

All calculations were performed on both data subsets and the whole dataset. Results shown are those for the whole dataset. Interscale correlation was assessed by Spearman’s rank-order correlations between factors with a value <0.85 for the composite (table 4) to be considered to have discriminant validity.27

Table 4

Discriminant validity indicated by Spearman’s correlation

Factor	Factor
	1	2	3
1. Communication and information
2. Rapport and participation	0.73
3. Medication safety	0.59	0.61

All Spearman’s rank-order correlations are statistically significant (P<0.01)).

Discriminant validity was further assessed by analysing mean differences between factor scores and subgroup characteristics that have been shown to be associated with patient experience. Kruskal-Wallis test and Dunn-Bonferroni test for post hoc comparisons were performed to examine differences between scale scores and subjective health status, age and educational status, respectively. Mann-Whitney U test was conducted to assess differences between scale scores and gender and survey mode, respectively. Cohen’s d was calculated to measure the size of the effect.

The Kruskal-Wallis test showed that patients with better self-reported health status were more likely to report positive experiences for each of the three factors (factor 1: H(4)=28.93, p<0.001, factor 2: H(4)=45.68, p<0.001, factor 3: H(4)=37.52, p<0.001). More positive experiences on all three factors were also found for older patients (factor 1: H(4)=50.22, p<0.001, factor 2: H(4)=28.77, p<0.001, factor 3: H(4)=24.83, p<0.001). Patients with lower educational levels reported more positive experiences with factor 2 and 3 (factor 2: H(5)=14.50, p=0.013, factor 3: H(5)=20.67, p<0.001). The post hoc tests showed that differences were predominantly between groups at the respective ends of each answer scale. Cohen’s d indicated weak to moderate effects for the observed differences with the most pronounced effects for patients with excellent compared with poor health and the oldest age group (>65 years) compared with the youngest age group (18–24 years) (online supplemental file 3).

The Mann-Whitney U test demonstrated that women were more likely to report positive experiences with factor 2 (U=9 01 166, Z=1.99, p=0.05) and more negative experiences with factor 3 (U=6 46 711, Z=−2.06, p=0.04), although the effect of gender on experience was small (Cohen’s d=0.07 and 0.08, respectively). Patients who completed the questionnaire at home reported more negative experiences with factor 2 (U=7 87 036, Z=2.80, p=0.005, Cohen’s d=0.10), for the other two factors no differences were observed.

Questionnaire as tool in internal quality and safety management

To be suitable for internal and external quality and safety management, the questionnaire needs to be able to distinguish between different practices. Thus, mean values of the scales of the practices were compared with the overall mean of all participating practices (‘average practice’). Nineteen practices had enough respondents (n≥25) to enable a reasonable analysis. The results for the scale communication and information are shown in figure 1, data for all three scales can be found in online supplemental file 4.

Figure 1

Practice means compared with overall mean (factor communication and information). Scale values: 1 to 5 (1=most positive experience, 5=most negative experience). Black bars: p<0.05.

Discussion

The ASK-ME-questionnaire has been developed as a self-report tool for measuring factors that contribute to the occurrence of patient safety events from the patients’ perspective and the frequency of those events. The instrument has been intended to be suitable for routine use in general practitioner as well as specialist practices by being generic, of reasonable length, freely available and by providing actionable feedback that also allows for benchmarking purposes. The present questionnaire is the first instrument to capture this perspective on patient safety in ambulatory care fulfilling these requirements.

With a total of 22 items, the ASK-ME-questionnaire is considerably shorter than existing instruments15 16 and therefore well suited for routine use. It showed very good acceptability (demonstrated by the survey response rate and median item non-response rate), good construct validity, very good to acceptable levels of internal consistency and good discriminant validity. The dimensionality of the questionnaire differs from existing instruments,15 16 however, the identified multi-item subscales cover three core dimensions of patient safety coupled with reasonable questionnaire length. Communication and information covered by the first subscale are seen as one of the key factors influencing the occurrence of PSEs.28 29 Miscommunication can lead to delays in treatment, misdiagnosis or physical and psychological harm. Patients who understand the information regarding their condition and treatment are more likely to share information for accurate diagnosis, follow advice and adhere to the prescribed treatment.16 However, communication is more than just conveying information on relevant facts. It is a prerequisite for a good interpersonal rapport between healthcare professionals and patients, aspects of which are covered by the second subscale rapport and participation. Patients need to understand information and have to agree with their doctor on a treatment plan. A plan that is understood and agreed on enhances the disclosure and identification of relevant information, of health outcomes16 and reduces the risk of error.18 Since patients are often the only ones who recognise and report on communicative aspects,19 the integration of these aspects into a patient questionnaire is essential. Finally, medication-related events are among the most common and serious events in ambulatory care.20 Poor education about the medication being administered and lack of involvement in the decision-making process increase the likelihood of the occurrence of an adverse medication event.21–24 The dimensions access and care coordination were assessed by single items and did not result in a factor. Given that these dimensions are also considered highly relevant for patient safety and patients are well placed to report on these dimensions, they were kept in the questionnaire.30 31 An argument can be made for strengthening the single item scales by adding further items covering these dimensions. However, this would be a trade-off between questionnaire length and detail that also needs to be considered with regard to practicability and acceptability.

All three subscales were skewed, indicating predominantly positive care experience. This positive evaluation tendency is common for surveys that capture patients’ perspectives on their medical care.32 33 Therefore, the results of the patient survey should be interpreted with caution, as they may present an overoptimistic picture. Nevertheless, since all subscales showed good ability to distinguish between practices, the questionnaire is suitable for benchmarking purposes.

All three subscales showed good discriminatory ability. The differences by age and health status described in the literature22 29 are also reflected in the present analyses. Further differences, for example, according to gender or educational level, show different effects depending on the respective subscale. These results are also consistent with the literature, where results on these effects are heterogenous.22

In contrast to existing measures,2 15 experience of PSEs and resulting harm was assessed with two generic questions. Both had to be excluded from further psychometric analyses because >15% of the patients had either not answered the question or were not sure whether what they had experienced represented a PSE. The proportion of patients who did experience a PSE was low (2.9%) compared with studies using more detailed measures. However, these studies did either not use a self-report measure2 34 or had high proportions of patients skipping the detailed section on PSE experience and a low overall response rate.29 Further research is needed to adequately assess number and type of experienced events in routine measurements.

Strengths and limitations

A strength of the present study is the rigorous development process with patients being involved throughout the process of drafting and consenting the content as well as testing the face validity of the instrument. Although data collection took place in 22 instead of 50 practices, the target sample size could be realised. The average response rate of 71% is substantially higher than the response rate of similar studies29 35 and demonstrates, together with the low proportion of missing values, the practicability and acceptability of the questionnaire.

The self-selection of both practices and patients can lead to systematic differences between the participants and the non-participants, thus biasing the results. Non-responders in this study were older and less likely to have an A-level or university degree. The literature on patient-reported experience measures suggests that younger patients report less positive experiences whereas lower educational levels are associated with more positive experiences in some studies.22 Self-selection of practices may have led to an overestimation of positive experiences. Practices with a high interest in patient safety measures and interventions and those who already have specific safety strategies in place may have been more likely to participate. Further potential biases might result from patient selection criteria: included were only patients who visited the practice within the survey period (which also included the restrictions due to the outbreak of the COVID-19 pandemic) and patients who had sufficient reading and German language skills. Language barriers are considered a risk factor for safe patient care.28 Future research should therefore investigate how to include these groups in the assessment of PSEs.

Missing values were not imputed and four items with >15% missing data were therefore excluded from the multivariate analyses. The remaining items, however, showed low levels of missing values (<5%), thus the risk of bias due to listwise deletion in the multivariate analyses is considered to be low.

The subscales demonstrated good ability to distinguish between different practices. However, given that the sample of participating practices was small and occurrence of errors is more likely in some specialties,2 this needs to be further explored with a larger sample where analyses can be stratified by practice specialty.

Conclusion

The ASK-ME-questionnaire for patient experience with patient safety in ambulatory care is a newly developed self-report tool with good psychometric properties. It is suitable as one of the important components of patient safety measurement and improvement systems where various approaches are needed to provide a complete picture of harm in ambulatory care.36 The ASK-ME-questionnaire, which is provided in online supplemental file 5, is freely available to facilitate its use and thereby promoting the integration of patients’ experience in patient safety strategies.