Best-Worst Scaling Study to Identify Complications Patients Want to Be Informed About Prior to Abdominal Aortic Aneurysm Surgery.

BACKGROUND: Surgeons must discuss the most severe surgical complications with their patients while making a treatment decision. However, it is unclear which complications patients deem most severe. This study aimed to have patients classify potential complications following abdominal aortic aneurysm surgery based on severity using best-worst scaling.
METHODS: Dutch patients with an abdominal aortic aneurysm, either under surveillance or following surgery, received a survey with 33 potential surgical complications. The survey presented these complications in sets of three. Patients had to classify one of three complications as most severe and one as least severe. After all participants had completed the survey, the number of times a complication was classified as most severe was subtracted from the number of times that the complication was classified as least severe, thus resulting in a best-worse scaling score. Complications with the lowest scores were ranked as more severe.
RESULTS: Fifty out of 79 participating patients completed the survey in full. Patients classified the following ten complications as most severe: Below-ankle amputation, aneurysm rupture, stroke, renal failure, type 1 endoleak, spinal cord ischaemia, peripheral bypass surgery, bowel lesion, myocardial infarction and heart failure. Haematoma was ranked as the least severe complication.
CONCLUSION: This best-worst scaling study enabled patients to classify complications following abdominal aortic aneurysm surgery based on severity. Vascular surgeons should discuss the ten complications deemed most severe with their patients and help their patients to effectively weigh the benefits of surgery against the harms patients themselves deem important, thereby improving shared decision making.

Key Points for Decision Makers

Introduction

Prior to surgery, vascular surgeons discuss the benefits and potential surgical complications with their patients. However, the complications patients are informed about often differ with each vascular surgeon [1, 2]. Surgeons are supposed to discuss the most frequently occurring complications and those major complications that a reasonable person in the patient’s position would deem of significance [3]. In a previous study, we reached consensus among vascular surgeons about major complications following abdominal aortic aneurysm (AAA) surgery [4]. However, it remains unclear which complications patients deem as most severe and would like to have discussed prior to surgery.

Understanding which complications following surgery patients deem most severe allows surgeons to obtain a true informed consent. It also ensures that all patients are informed in a similar manner regarding complications that are important to them. Patients will then be able to effectively weigh the benefits and harms of the available treatment options with their surgeon. Helping patients weigh the benefits and harms important to them is a crucial aspect of shared decision making [5]. Shared decision making between surgeons and patients has shown to improve patient satisfaction and quality of care, while also reducing overtreatment [5-7].

Weighing the benefits and harms of available treatment options is especially important for patients with an AAA. At some time during their disease progress, these patients may need to decide between continuing surveillance and undergoing endovascular or open surgery [8]. As patients with an AAA are usually asymptomatic and not every AAA will rupture, it continues to be difficult to predict which patients will benefit from surgery, that is, in which patients aneurysm rupture is prevented [9]. Thus, it is important for patients to understand that by undergoing surgery they may risk going from having no symptoms at diagnosis, to living with the consequences of a complication in order to reduce the risk of aneurysm rupture.

Therefore, this study aimed to have patients classify potential complications based on severity, to clarify the major complications patients want to be informed about prior to AAA surgery.

Methods

Study Design

The study design entailed a cross-sectional survey using best–worst scaling (BWS). Finn and Louviere first introduced BWS in the field of marketing research to elicit the preferences of participants [10]. They concluded that more information can be gained from having participants trade off one item against another than by asking participants to rate the importance of several items [11, 12]. The authors used the reporting guideline for survey research from Kelley et al. [13] to report this study (Appendix A, see electronic supplementary material).

Participants

Participants in this study were Dutch patients diagnosed with an AAA. These patients were either under surveillance or had already undergone endovascular or open surgery to treat their AAA. The researchers and patient partner identified potential participants via the Dutch patient organisation for people with cardiovascular diseases (Harteraad) and the outpatient clinic of the Amsterdam University Medical Centres—location Academic Medical Center. Members of the patient organisation, who were registered as having an AAA and at registration agreed to be open to participation in survey research, were invited by the patient partner to participate in the study. All consecutive patients who visited the outpatient clinic of the Amsterdam University Medical Centres—location Academic Medical Center between April 2018 and September 2018 for surveillance or follow-up of their AAA were invited by the researchers to participate in the study. If patients agreed to participate, they received the survey either electronically using Google forms (Mountain View, California, USA) or on paper, depending on the patients’ preference. A week after the survey was sent, one of the researchers would contact the patient via phone to see if further assistance was required to complete the survey. If assistance was required, the researcher would read aloud the survey to the participant and help them comprehend the complications presented in the survey. If patients had not returned the survey within 4 weeks after it had been sent, one of the researchers would contact the patient one last time to help patients complete the survey.

Calculating a sample size for a BWS study requires a significance level, a statistical power level, the statistical model and initial beliefs about parameter values [14]. No information or initial beliefs were present with regard to parameter values. In addition, we decided upon count analysis as the statistical model. This meant it was not possible to calculate a sample size, especially since the aim of our study was not to prove an effect or difference. Therefore, our sample was based on the expected availability and response rate of participants. A total of 140 patients were available between April 2018 and September 2018; 38 participants from the patient organisation and 100 participants from the outpatient clinic of the Amsterdam University Medical Centres—location Academic Medical Center. A response rate of 35% was expected based on previous literature for this combination of self-completion surveys and assistance via phone, resulting in a sample of 49 patients [13]. Thus, the researchers halted the search for new participants upon receiving 50 correctly filled-out surveys.

Survey Development

The survey used to elicit the opinion of patients with regard to the most severe complications following AAA surgery was developed for this specific research question. Development of this survey entailed the extraction and presentation of complications, the design of the survey and pilot testing.

Extraction of Complications

The complications presented in the survey included all complications discussed in reference articles of the Cochrane systematic review by Paravastu et al. [15]. Two researchers independently extracted 33 potential complications from these reference articles.

Presentation of Complications

The severity of complications may depend on their consequences. Therefore, the researchers provided each complication with a consequence of moderate severity based on the Society for Vascular Surgery (SVS) reporting standards [16]. For example, myocardial infarction may have a mild consequence if there are little or no hemodynamic consequences. It may have a moderate consequence if percutaneous transluminal coronary angioplasty is necessary. The complication becomes severe if resuscitation due to cardiac arrest occurs. To prevent bias, our survey presented patients with the moderate level of severity of each complication. Death was not included as a complication in the survey as the authors are of the opinion that the possibility of death following surgery must always be discussed with patients prior to surgery.

Vascular surgeons must discuss the most frequently occurring complications and the most severe complications with their patients. The most frequently occurring complications are usually those complications that occur in more than one out of 100 patients. These associated risks are fairly well known from local or international data sources. As stated, vascular surgeons must also discuss the most severe complications irrespective of their associated risks. Therefore, the researchers explicitly decided not to provide patients with the associated risks in our survey to assess their opinion about which actual complications patients deem most severe.

The survey presented the complications to patients using nonprofessional terms and also provided additional information about how these consequences would affect patients in their daily functioning. For example, a superficial wound infection would require patients to rinse the wound at home by themselves at least three times a day for about 3 weeks. Appendix B provides a complete description of complications used in this study (see electronic supplementary material).

Survey Design

The survey consisted of 15 questions. The first three questions concerned the patients’ sex, age and treatment status. Patients who had previously undergone endovascular aortic aneurysm repair (EVAR) or open surgery were also asked to fill out whether they had developed a specific complication following this surgery.

The 11 questions thereafter each presented three potential complications following AAA surgery. The researchers used the BWS case 1 design to present and analyse the relative importance of these complications [11]. For our survey, this meant that of the three complications presented per question, patients had to classify one complication as most severe and one complication as least severe. For each question, one complication was not classified as most severe or least severe—the classification box of this complication remained blank. Every patient received a different combination of three complications and a different order in which the survey presented the complications. The researchers used the ‘uniform division randomisation’ function in Microsoft Excel 2016 (Redmond, Washington, USA) to randomise the order in which the survey presented the complications. Each patient scored 33 complications.

The 15th and final question asked patients whether they missed any complications in the survey. An example of the survey used in this study is available in Appendix B (see electronic supplementary material).

The department of patient education at the Amsterdam University Medical Centres—location Academic Medical Centre ensured that the survey was designed and the questions were presented in a manner that participating patients would easily be able to understand.

Pilot Testing

Eight patients, who are members of a patient organisation, formed a workgroup to test the usability and readability of the survey. The electronic version and paper version were piloted separately in this workgroup upon request of the participants. All participants were able to fill out the questionnaire. However, some small adjustments to the cover letter were suggested to clarify the use of the survey. These changes were made accordingly.

Patient Partner

The patient partner, a representative of the Dutch patient organisation for people with cardiovascular diseases, played an important part in this study. The patient partner took part in designing the survey. In addition, the patient partner chaired the patients participating in the pilot study and actively reached out to other patients via the patient organisation to participate in the study. Finally, the patient partner critically reviewed the outcomes of the study.

Data Analysis

Patient demographics are summarised using descriptive statistics. Normally distributed outcome measures are expressed as means with standard deviations. Differences between participants and non-participants were studied using unpaired samples t tests and Chi-square analysis. Following the completion of 50 surveys, the researchers analysed the trade-offs between the complications in accordance with the BWS case 1 study design count analysis. This means that for each complication, the number of times patients classified it as most severe was subtracted from the number of times patients classified it as least severe. Since all 50 patients classified each of the 33 complications once, BWS scores could range between − 50 and 50. Complications with a BWS score close to − 50 were classified as the most severe complications (major), whereas complications with a BWS score close to 50 were classified as the least severe complications (minor). These results were presented in quantitative and narrative forms, including tables and figures, to aid in data presentation where appropriate.

Additional Analysis

The researchers also looked at whether the complications that were deemed most severe or least severe differed between patients with an AAA who were under surveillance, patients who had undergone EVAR and patients who had undergone open surgery. The researchers observed these potential differences by looking at the order in which these complications were ranked based on their BWS scores.

Results

Survey Responses

The researchers had sent the survey to 79 patients, before obtaining 50 correctly filled-out surveys (63% response rate). Forty-one patients were able to return the survey on their own. Nine patients required assistance via phone. Of the remaining 29 patients who did not return the survey, 11 patients had filled out the survey incorrectly. For example, they classified all three complications as most or least severe instead of just two out of the three complications presented per question. Four patients stated they found it difficult to classify complications they never had themselves and preferred not to participate. Five patients stated they did not have the time to complete the survey. Of the nine remaining patients, it was unclear as to why they refrained from filling out the survey, as they did not respond to follow-up inquiries.

Patient Demographics

Table 1 shows the demographics of the 50 participating patients. Forty-seven participants (94%) were men with a mean age of 72 years. Thirteen patients were (still) under surveillance. Of the 37 patients who had undergone surgery, 22 patients had undergone endovascular repair and 15 patients open surgery (41%). Eighteen patients reported they had suffered from complications following AAA surgery. Examples of these complications were wound infection, endoleaks, erectile dysfunction, renal insufficiency and ostomy dependency.

Table 1

Patient demographics

	Participants	Non-participants	p value
Total, N	50	19
Sex, n (%)
Men	47 (94)	16 (84)	0.20
Women	3 (6)	3 (16)
Age, years
Mean	72	75	0.08
Standard deviation	± 5.5	± 6.0
Treatment, n (%)
Active surveillance	13 (26)	4 (21)	0.63
Endovascular repair	22 (44)	7 (37)
Open surgery	15 (30)	8 (42)
Number of participants with a postoperative complication, n (%)
Yes	18 (49)	7 (47)	0.90
No	19 (51)	8 (53)

Table 1 also shows the patient demographics of patients who had not (correctly) filled out the survey. The researchers obtained the information on patient demographics from the survey itself. Nineteen out of 29 non-participants filled out this section of the questionnaire, which entailed all eleven patients who wrongly filled out the rest of the survey, all four patients who found it difficult to classify complications they never had themselves and four out of five patients who did not have time to complete the survey. This information could not be obtained from the nine patients who did not fill out the survey and could not be contacted thereafter. There were no statistically significant differences between the participants and non-participants based on our patient demographics.

Best–Worst Scaling

Figure 1 shows the BWS scores of all 33 complications based on the classifications made by the 50 participating patients. Sixteen of 33 complications received a negative score as these complications were more frequently considered most severe than least severe. Seventeen of 33 complications were more frequently considered least severe than most severe. Below-ankle amputation following a thromboembolic event was the highest ranking, most severe complication, with 43 of 50 patients (86%) considering this the most severe complication, followed by aneurysm rupture and stroke. Forty of 50 patients (80%) considered haematoma the least severe complication. Table 2 presents a detailed overview of the number of times patients classified a complication as most or least severe.

Fig. 1

Overall best–worst scaling scores

Table 2

Overview of patient classifications per complication

Complication	Classified as most severe	Not classified as most or least severe	Classified as least severe	BWS score
Below-ankle amputation following thromboembolic event	43	6	1	− 42
Aneurysm rupture	43	4	3	− 40
Stroke	41	7	2	− 39
Renal failure requiring dialysis	37	11	2	− 35
Type-1 endoleak requiring open surgery	29	16	5	− 24
Spinal cord ischaemia	29	15	6	− 23
Bypass surgery following thromboembolic event	29	14	7	− 22
Bowel lesion	24	17	9	− 15
Myocardial infarction	23	18	9	− 14
Heart failure	23	17	10	− 13
Leg artery occlusion requiring reintervention	21	17	12	− 9
Allergic reaction	19	21	10	− 9
Type-1 endoleak requiring endovascular treatment	18	20	12	− 6
Vascular graft migration	17	21	12	− 5
Pulmonary embolus	18	18	14	− 4
Technical deployment problem vascular graft	17	19	14	− 3
Pneumonia	11	25	14	3
Buttock claudication	11	25	14	3
Cicatricial hernia	13	18	19	6
Renal insufficiency requiring follow-up	12	19	19	7
Bowel ischaemia	10	23	17	7
Sepsis	9	21	20	11
Deep venous leg thrombosis	10	18	22	12
Vascular graft infection	7	22	21	14
Superficial wound infection	7	16	27	20
Urine retention	3	20	27	24
Ureteral lesion	5	15	30	25
Postoperative haemorrhage	4	17	29	25
Erectile dysfunction	7	10	33	26
Type-2 endoleak requiring follow-up	5	14	31	26
Deep wound infection	2	20	28	26
Pseudoaneurysm	3	15	32	29
Haematoma	0	10	40	40

BWS Best − worst scaling

Figure 2 shows the BWS scores of all 33 complications stratified for the treatment patients had undergone. The order in which this figure presents the complications is similar to the order of presentation in Fig. 1. Since 13 patients were under surveillance, the BWS scores for this group ranged between − 13 and 13. Likewise, the endovascular repair group had a BWS range between − 22 and 22, and the open surgery group between − 15 and 15. Besides these smaller ranges for the surveillance and open surgery groups, no important differences in severity-based ranking were observed between the three treatment groups.

Fig. 2

Best–worst scaling scores per treatment group

All three treatment groups reported amputation following a thromboembolic event, aneurysm rupture, stroke and renal failure requiring dialysis in their top-five ranking of most severe complications. These top-five rankings were completed by spinal cord ischaemia, bypass surgery following a thromboembolic event and type-1 endoleak requiring open surgery.

When looking at the top-five rankings of the least severe complications, all three groups reported haematoma, pseudoaneurysm and type-2 endoleak requiring follow-up. These top-five rankings were completed by erectile dysfunction, deep wound infection, superficial wound infection, urine retention, ureteral lesion and postoperative haemorrhage.

Missing Complications

One patient would like to have received information about the risk of developing a delirium following AAA surgery as an additional complication that had not been included in the survey.

Discussion

Using BWS, this study enabled patients to classify potential complications following AAA surgery, based on severity.

As mentioned in the introduction, our research group previously asked vascular surgeons to classify complications into major and minor complications [4]. Using a Delphi method [4], the vascular surgeons were able to reach consensus on 12 major complications. Their survey presented them with the same 33 complications as patients were shown in the present study. When comparing the top ten most severe complications as classified by patients with the 12 major complications acquired from vascular surgeons, nine complications are agreed upon. Table 3 shows these nine complications in alphabetical order. Other complications that vascular surgeons classified as major were allergic reaction, pulmonary embolus and vascular graft difficulties, such as graft infection and technical deployment problems. Vascular surgeons did not classify type-1 endoleak requiring open repair as a major complication. However, patients did classify this complication in their top ten most severe complications.

Table 3

Major complications both patients and vascular surgeons agree upon

	Complication	BWS score
1	Abdominal aortic aneurysm rupture	− 40
2	Below-ankle amputation	− 42
3	Bowel lesion	− 15
4	Heart failure	− 13
5	Myocardial infarction	− 14
6	Peripheral bypass surgery	− 22
7	Renal failure	− 35
8	Spinal cord ischaemia	− 23
9	Stroke	− 39

BWS Best − worst scaling

Vascular surgeons must discuss the most frequently occurring complications and the complications deemed most severe with patients. Thus, in addition to mentioning the most frequently occurring complications, we recommend that vascular surgeons discuss the ten most severe complications according to patients with their patients, albeit as worst-case scenarios, prior to making a decision between undergoing surgery and not (yet) undergoing surgery. This will allow patients to weigh the benefits of undergoing surgery against the harms they themselves deem of significance. Furthermore, vascular surgeons should help patients with this process and explicitly ask their patients about their preferences and concerns. Incorporating these preferences and concerns into the decision-making process makes it possible to reach a shared decision.

If patients cannot undergo open repair due to comorbidities or EVAR due to anatomical features, vascular surgeons obviously do not need to discuss the complications associated with the treatment that their patient cannot undergo. Nevertheless, it is important to inform patients about why they cannot undergo a specific treatment. This makes it easier for patients to accept the remaining options [17]. Vascular surgeons may even use the aforementioned complications to inform patients on why one of the treatment options is not feasible.

The first possible limitation of the study was to include patients both under surveillance and following surgery. The informed consent procedure states that the information provided should be relevant for patients in that particular situation in which a treatment decision needs to be made [3]. Thus, the researchers also included patients under surveillance, as they may face this decision in the future and can already start thinking about the information they would like to receive. Similarly, patients following surgery can reflect on the information they received and contemplate which information they deemed highly important or missed from the discussion with their vascular surgeon. However, as the researchers did not specifically ask patients in the surveillance group whether they were currently considering surgery, it is unclear whether our outcomes are compatible with patients who are facing this decision.

The second limitation concerns the patient sample included in this study. Due to the absence of a sample size calculation, the outcomes of this study were based on a sample size of 50 patients of 140 potential participants during the course of our study. It is unclear if a larger sample of patients would have led to a different list of most severe complications and whether these outcomes are representative of the larger population of patients with an AAA. However, there were no statistical differences in patient demographics between patients who did and did not (correctly) fill out the questionnaire nor were there any apparent differences between treatment groups in how complications were classified.

Third, the researchers used uniform division randomisation to randomise the order in which the complications were presented. Although this randomisation randomly paired all 33 complications in sets of three, some complications may have been paired together more often than with other complications. Thus, some trade-offs may have occurred more often than others.

Fourth, despite the fact that the researchers did perform pilot testing to ensure usability and readability of the newly designed survey, no psychometric testing was performed. It is unclear if patients would have filled out the survey exactly the same if they had to repeat the survey.

The fifth limitation concerns the information provided to patients. Some patients may find it difficult to comment on complications they themselves have not experienced. The researchers tried to assist patients in understanding what these complications would entail for patients by providing them with additional support and information explaining what the complications would mean for the patient in practical terms. Unfortunately, four patients still deemed it too difficult to comprehend the complications presented in the survey.

Finally, the survey presented patients with complications obtained from prior studies concerning AAA treatment. Unfortunately, patients did not partake in designing these studies, which means that outcomes that are relevant to patients may not have been included in these studies. For example, one patient found delirium to be missing from our list of complications. Thus, the authors recommend that researchers involve patients in the design process of new studies to ensure that patient-relevant outcomes are studied as well.

Conclusion

This study enabled patients to classify complications following AAA surgery based on severity. Vascular surgeons should discuss the top-ten most severe complications, as worst-case scenarios, with their patients, in addition to the most frequently occurring complications, while deciding between surgical options and conservative therapy. This will harmonise risk communication and improve shared decision making as it allows patients to effectively weigh the benefits of surgery against the harms patients themselves deem important.

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 39 kb)

Fifty patients ranked 33 potential complications following surgery of abdominal aortic aneurysms based on severity.

Below-ankle amputation following a thromboembolic event was ranked as the most severe complication, followed by aneurysm rupture and stroke.

Vascular surgeons should discuss the complications deemed most severe with their patients while making a decision about undergoing abdominal aortic aneurysm surgery.