Patient safety risk factors in minimally invasive surgery: a validation study.

This study was conducted to adapt and validate a patient safety (PS) framework for minimally invasive surgery (MIS) as a first step in understanding the clinical relevance of various PS risk factors in MIS. Eight patient safety risk factor domains were identified using frameworks from a systems approach to patient safety. A questionnaire was drafted containing 34 questions. Three experts in the field of patient safety critically reviewed the questionnaire on clinical relevance and completeness. The questionnaire was distributed among known patient safety experts in person and also sent electronically. A total of 41 questionnaires were distributed and the response rate was 71%. The intraclass correlation coefficient was 0.42 representing moderate agreement. For seven of nine risk domains, Cronbach's alpha was sufficient (α > 0.7). Mean scores of the risk domains showed the following order of influence on patient safety from high to low: surgeon's experience [6.6, standard deviation (SD) 0.5], technical skills surgeon (6.6, SD 0.7), technology (5.9, SD 1.1), complications (5.9, SD 1.2), social interaction (5.0, SD 1.0), leadership surgeon (5.4, SD 1.2), blood loss (5.0, SD 1.2), length of surgery (5.0, SD 1.3), surgical team (4.9, SD 1.3), fallibility (4.9, SD 1.3), patient (4.5, SD 1.5), safety measures (4.4, SD 1.5), and finally environment(3.9, SD 1.5). This study is an initiative to give insight into clinical relevance of the maze of PS risk factors in MIS. All investigated risk domains were considered to be of noticeable influence on PS. Nevertheless, it is possible to prioritize various risk domains. In fact, experience and technical skills of the surgeon, technology, and complications are rated as the most important risk factors, closely followed by social interaction and leadership of the surgeon. Patient, safety measures, and environment are rated as the least important risk factors.

Background

Over the past years, society has become more quality driven. Also, in health care, the demand for the highest quality possible has become an important center of attention, especially in the operating room. This urgent request for high quality has made patient safety (PS), as a quality parameter, a very important research topic. Research concerning “patient safety” has expended explosively since 1999 and “patient safety” in the “operating room” has been increasing since 2004. Research concerning “patient safety” combined with “laparoscopy” only recently started to increase, whereas research on “laparoscopy” alone has been increasing fairly stable. All the while, research with the keyword “operating room” as well as the total number of yearly publications has been stable for years.

Recently, many PS issues considering minimally invasive surgery (MIS), especially laparoscopy, have been brought to our attention. In The Netherlands, PS issues about MIS (especially laparoscopy) brought to the attention by the Dutch Inspectorate of Healthcare even led to a discussion in the Dutch parliament about the desirability of MIS [1]. Despite the great amount of research about PS and laparoscopy in general, only a few publications concern both items.

In general, patient safety is a complex multidimensional concept and the clinical relevance of its various dimensions is not clearly understood. To comprehend the clinical relevance of various PS risk factors in surgery, the degree to which a patient is exposed to all of these risk factors during surgery should be studied. Yet, studies that try to measure risk factors during surgery usually focus on specific parts of the multidimensional concept. Nonetheless, it has been widely accepted that a wide range of factors influence PS in surgery and a number of important studies [2-5] have addressed this by developing frameworks according to the systems approach to quality and safety in surgery. The aim of this study is to adapt and validate these frameworks for MIS as a first step in understanding the clinical relevance of various PS risk factors in MIS.

Methods

Existing frameworks that were developed according to a systems approach to quality and safety in surgery [2-5] were adapted for MIS. The adapted framework consisted of the following risk domains:

Surgeon: risk factors regarding functioning of the surgeon,

Surgical team: risk factors regarding functioning of the scrub or circulating nurse,

Technology: risk factors regarding the availability and functioning of technology,

Social interaction: risk factors regarding teamwork and communication,

Environment: risk factors that potentially cause distraction or disruptions of the surgical process,

Patient: patient-related risk factors,

Fallibility: risk factors regarding factors that influence the fallibility of the surgeon,

Safety measures: items regarding (compliance of) safety protocols, and

Result: items regarding the result of the procedure.

For each risk domain, risk factors were defined and incorporated in a questionnaire (Table 1). At the end of the questionnaire, there was free space for comments on missing risk factors or other issues. The questionnaire was critically reviewed on clinical relevance and completeness by three independent experts, who did not participate in the development of the questionnaire, i.e., the president of the commission on patient safety of the Dutch society of obstetrics and gynecology, the president of the commission on patient safety of the association of surgeons of the Netherlands who is also a pioneer in MIS, and the president of the Dutch society of endoscopic surgery who is also a gynecologist with a research line in patient safety in MIS. This led to a minor addition to the questionnaire.

Table 1

Categorization of risk factors

Risk domains	Risk factors
Surgeon	Experience of the surgeon
	Technical skills of the surgeon
	Leadership of the surgeon
Surgical team	Qualified staffing
	Experience of the scrub nurse
	Scrub nurse’s knowledge of the procedure
	Experience of the circulating nurse
Technology	All instruments are present
	All instruments work properly
	It is known how to handle all instruments
	All equipment is present
	All equipment works properly
	The OR team knows how to handle all equipment
	The surgeons knows how to handle all equipment
Social interaction	Communication between OR team members
	Failure of professional communication
	Communication of important issues at shift changes
	Collaboration between OR team members
Environment	Disruptions of the surgical process
	Distractions (e.g., questions not relating to the patient)
	Number of people in the OR
Patient	Patient’s BMI
	Patient’s ASA score
	Previous surgeries
Fallibility	Time of day surgery takes place (e.g., daytime, nighttime)
	Workload
	Number of procedure (e.g., first or last of that day)
	Fatigue
Safety measures	Universal safety protocols
	Briefing according to WHO checklist
	Compliance of safety measures (protocol)
Result	Intraoperative complications
	Amount of blood loss
	Length of surgery

BMI body mass index, OR operating room, ASA score American Society of Anesthesiologists Physical Status Scores

Next, a sample of international gynecological PS experts was asked to rate all PS risk factors. A PS expert was defined as a gynecologist specialized in laparoscopic surgery, who is well recognized by their peers and that had either published a minimum of two articles on PS-related topics and/or is actively involved in a commission on PS. All experts were members of the European Society for Gynaecological Endoscopy and/or the American Association of Gynecologic Laparoscopists. The questionnaire was distributed internationally both directly among PS experts during a gynecological MIS conference (N = 12) and electronically by email (N = 29). The risk factors were rated according to their potential impact on PS in MIS on a 6-cm Visual Analogue Scale (VAS) with at the endpoints “no impact” and “huge impact”. Because of technological restrictions, the electronic version of the VAS was converted to a 13-point scale with, similar to the VAS, at the endpoint 0 “no impact” and at the endpoint 12 “huge impact” and every intermediate point corresponded with half a centimeter on the VAS. For conventional reasons and ease of interpretation, both scales were converted to a seven-point scale after all data were collected. The mean scores of the VAS and 13-point scale were first analyzed separately showing similar results for both rating scales. As such, it was decided to combine the two rating scales to report the results of this study.

Data analysis

All data were analyzed with SPSS 16.0 software package (SPSS, Chicago, IL, USA). An overall inter-rater agreement was determined for the complete questionnaire with the intraclass correlation coefficient (ICC). The ICC is exactly identical to the weighted kappa with quadratic weights [6] and is an appropriate method to determine an inter-rater agreement between multiple raters of multiple questions with a large rating scale as is the case in the current study. The ICC values were interpreted according to Landis and Koch’s [7] guidelines for the interpretation of kappa (kappa values <0 poor; 0 to 0.2 slight; 0.2 to 0.4 fair; 0.4 to 0.6 moderate; 0.6 to 0.8 substantial; and >0.8 almost perfect agreement).

For each PS risk factor domain (containing a number of separate risk factors), internal consistency was determined using Cronbach’s alpha. If internal consistency of a PS risk factor domain was accepted (Cronbach’s alpha >0.7) [8], further analysis was performed for that domain as a whole. If internal consistency was insufficient (Cronbach’s alpha <0.7), further analysis was performed for every risk factor separately. Mean scores and 95% confidence intervals (95% CI) were calculated per PS risk domain (or single risk factors if internal consistency was insufficient) and plotted. Statistical significant difference between PS risk domains was determined with one-way analysis of variance (ANOVA) and Bonferroni post hoc analysis (p value < 0.05).

Findings

A total of 41 questionnaires were distributed of which 29 were completed and returned, resulting in a response rate of 71%. Out of the 29 questionnaires, 28 were filled out completely and one incomplete (five items at the last page were not completed). An agreement over the complete questionnaire was moderate (ICC = 0.42). The internal consistency of PS risk domains surgical team, technology, social interaction, environment, patient, fallibility, and safety measures were all acceptable (Cronbach’s alpha >0.7) (Table 2). The PS risk domains surgeon and result did not reach sufficient internal consistency (Cronbach’s alpha 0.54 and 0.14, respectively) and therefore further analysis was done for every risk factor separately. ANOVA analysis showed significant differences between groups (p < 0.001). Further analysis with Bonferroni post hoc test will be described in more detail below.

Table 2

Cronbach’s alpha per risk domain

Patient safety risk domain	Cronbach’s alpha
1	Surgeon	0.54
2	Surgical team	0.74
3	Technology	0.90
4	Social interaction	0.81
5	Environment	0.78
6	Patient	0.70
7	Fallibility	0.72
8	Safety measures	0.93
9	Result	0.14

Mean ratings and 95% CIs are plotted in Fig. 1. This figure illustrates the relative importance of the PS risk factor domains towards each other and shows that on average the experience and technical skills of the surgeon are rated highest [6.6, standard deviation (SD) 0.5 and 6.6, SD 0.7, respectively], whereas leadership of the surgeon was rated significantly less important (mean 5.4, SD 1.2). Although all risk factors are rated relatively high (right side of the seven-point scale), the mean rating of the PS risk domain environment (3.9, SD 1.5) is significantly lower than all other risk domains except for the mean rating of patient and safety measures (4.5, SD 1.5 and 4.4, SD 1.5, respectively). Both technology and social interaction are rated among the most important risk domains (mean 5.9, SD 1.1 and mean 5.5, SD 1.0, respectively). Within the risk domain result, both length of surgery and blood loss were rated relatively lower [mean 5.0, SD 1.3 and 5.0, SD 1.2, respectively] than complications (mean 5.9, SD 1.2), although not significant. On average, the risk domains surgical team (4.9, SD 1.3), fallibility (4.9, SD 1.3), patient, and safety measures did not differ significantly.

Fig. 1

Mean scores and 95% confidence interval (95% CI) per risk domain

There were no missing risk factors reported in the free space section. There were a few comments, which will be further discussed in the next section.

Discussion

In this study, an adapted framework of PS risk factors in MIS was validated as a first step in understanding the clinical relevance of various PS risk factors in MIS. All investigated risk domains were rated of noticeable influence on PS, confirming the multidimensionality of the concept PS. This also implies that in order to completely assess PS in MIS, all of the proposed risk factors should be considered.

The results of this study should be interpreted carefully. The selection of PS experts comprises only gynecologists who specialize in MIS; therefore, the results can only be interpreted as valid for gynecological MIS. Further research is needed to examine whether these results can be extrapolated to other specialisms and whether other professionals that participate in MIS (i.e., anesthesiologists and OR nurses) share this opinion. Furthermore, it should be taken into account that there are differences in agreement on the importance of various risk factors. This is illustrated by the differences in width of the 95% CI of mean (Fig. 1). A small 95% CI should be interpreted as strong agreement and a wide 95% CI should be interpreted as weak agreement. As such, agreement on the importance of technology, experience of the surgeon, and social interaction is strongest and therefore the average rating can be interpreted as more reliable than the average rating of length of surgery, leadership of the surgeon, complications, and blood loss, of which the 95% CIs are wider (e.g., agreement is less strong). Yet, overall agreement of the complete questionnaire was moderate (ICC = 0.42) and the widest 95% CI is 1.0 point wide. Together with the response rate of 71%, this is considered satisfactory.

The free space section in the questionnaire gave experts the possibility to comment on the questionnaire. There were no missing risk factors reported; however, there were few relevant comments made. There was one comment on the importance of the first assistant’s experience by directly affecting safety and by adding experience to a difficult procedure. We agree, however, we considered this as part of the qualified staff. Another expert commented that the 13-point scale in the electronic version of the questionnaire gives too much choice and could lead to clustering of the answers in the middle of the scale. Because of software restrictions, converting the VAS to a Likert-type scale for the electronic questionnaire was imperative. We chose to let every half a centimeter on the VAS correspond to a point on the Likert-type scale, hence the origin of the 13-point scale. Furthermore, the results show an arrangement of the risk domains in given rating; therefore, it can be stated that clustering of the answers due to the large scale is no issue in the current questionnaire.

It is possible to prioritize certain risk domains over others. As such, experience and technical skills of the surgeon, technology, and complications have the highest ratings, closely followed by social interaction and leadership of the surgeon. The use of advanced technology in MIS easily explains the experts’ view on the importance of good functioning instruments and equipment. The magnitude of technology-related problems during MIS has previously been illustrated in several studies [9, 10]. Also, in gynecological MIS procedures, many technology-related problems, with potentially dangerous consequences, have been observed [11]. In these studies, technology-related incidents mainly led to delay and extra work. The importance of technology in MIS is supported by the fact that the experts rate this domain among the most important risk domains in MIS. Surgeons could be facilitated in the optimal use of advanced technology in laparoscopic operating units by checklists. Checklists have already been shown to improve the use of the available technology and reduce technology-related incidents [9, 12]. Problems in social interaction (such as communication and teamwork) have also been shown to occur frequently during surgery [13, 14]. For example, communicational difficulties have been reported to occur in approximately 30% of team exchanges [13]. About a third of these communicational failures resulted in visible effects that can influence PS [14]. The importance of social interaction for PS in MIS is supported by the fact that the experts also rate this domain among the most important risk domains in MIS.

Many studies concerning PS in the operating room have been focusing on disruptions and distractions from the surgical process (environment) because they are believed to influence the surgeon’s concentration or are perceived as stressful events [15-20]. Remarkably, the experts rated this risk domain (environment) as the least influential of all risk domains. The results of an observational study are in line with the experts’ opinion as they found frequent disruptions, however, the stressfulness of disruptions were least severe compared to other (e.g., technical) incidents [16].

Conclusion

This study is an initiative to give insight into clinical relevance of the maze of PS risk factors in MIS. All investigated risk domains were considered to be of noticeable influence on PS. Nevertheless, it is possible to prioritize various risk domains. In fact, experience and technical skills of the surgeon, technology, and complications are rated as the most important risk factors, closely followed by social interaction and leadership of the surgeon. Although in general, patient safety measures and environment are among the most widely studied risk factors, experts in the field rate them as the least important risk factors. Although the effect of environmental factors should not be underestimated, experts in the field indicate that it should not be overestimated as well.

More research is needed to give us better insight in the pitfalls of patient safety as perceived in clinical practice to guide us in future research. As the results of this study can only be interpreted as valid for gynecological MIS, it would be interesting to examine whether experts of other specialisms (i.e., surgeons, anesthesiologists, and OR nurses) share the same view on patient safety. A focal group session including experts from different specialisms could give us more insight in the real need and concerns of experts on patient safety in minimally invasive surgery.