Identifying patients who could benefit from palliative care by making use of the general practice information system: the Surprise Question versus the SPICT.

OBJECTIVE: We compared the performance of two tools to help general practitioners (GPs) identify patients in need of palliative care: the Surprise Question (SQ) and the Supportive and Palliative Care Indicators Tool (SPICT).
METHODS: Prospective cohort study in two general practices in the Netherlands with a size of 3640 patients. At the start of the study the GPs selected patients by heart using the SQ. The SPICT was translated into a digital search in electronic patient records. The GPs then selected patients from the list thus created. Afterwards the GPs were interviewed about their experiences. The following year a record was kept of all the patients deceased in both practices. We analysed the characteristics of the patients selected and the deceased. We calculated the performance characteristics concerning predicting 1-year mortality.
RESULTS: The sensitivity of the SQ was 50%, of the SPICT 57%; the specificity 99% and 98%. When analysing the deceased (n = 36), 10 died relatively suddenly and arguably could not be identified. Leaving out these 10, the sensitivity of the SQ became 69%, of the SPICT 81%. The GPs found the performance of the digital search quite time consuming.
CONCLUSION: The SPICT seems to be better in identifying patients in need of palliative care than the SQ. It is also more time consuming than the SQ. However, as the digital search can be performed more easily after it has been done for the first time, initial investments can repay themselves.

The SPICT identifies patients in need of palliative care better than the SQ.

Performing the SPICT was more time consuming than the SQ according to GPs.

The SPICT can be translated into a search in electronic patient records.

When repeated the digital search can become easier to perform.

Introduction

As most people prefer to spend the last phase of their lives at their own home and would want to die there (1), identifying patients in need of palliative care is very relevant for general practitioners (GPs). We know that it can be difficult for caregivers to identify patients who would benefit from palliative care (2–4). Tools to help them to identify these kinds of patients are thus useful. Several instruments were developed the past years. For this article we focus on the usability of two of these tools in general practice: the Surprise Question (SQ) (5) and the Supportive and Palliative Care Indicators Tool (SPICT) (6). If the SQ—‘Would I be surprised if this patient died in the next 12 months?’—is answered with ‘No’, the patient is supposed to benefit from palliative care.

The SPICT consists of a list of general indicators and disease specific indicators, for cancer, dementia and frailty, neurologic, heart and vascular, respiratory, kidney and liver disease. If one or more of these indicators are present a patient could benefit from palliative care.

Both have been investigated before, although the SQ more frequently. Most studies look at prognostication and not at actual identification of patients in need of palliative care. A systematic review about the SQ showed it did not perform very well in predicting 1-year mortality (meta-analytic estimates: sensitivity 67% and specificity 80%) (5). In the two studies with a general population the SQ had a lower sensitivity (7,8), while in studies with a population of patients suffering from cancer (9) or heart failure (10) a lower specificity was found. A study into the SPICT which took place at the geriatric ward of a hospital showed similar results as these last two studies about the SQ with a sensitivity of 84% and a specificity of 58% (11). Identifying patients in a population that is relatively healthy, like in primary care, seems to increase the risk of missing patients, while in in-patient care with a sicker population there is a greater risk to wrongfully identify patients.

The digitalization of patient records creates the possibility to let the computer do part of the work in identifying patients in need of palliative care. The SPICT can be partly translated into a search in data provided by electronic patient records. This was already done in a study by Mason et al. (12). We did this for our study to see if this increased its efficiency.

We choose general practices in the Netherlands as the setting of our study. In the Netherlands palliative care is seen as an integral part of regular health care and relatively more patients die outside of the hospital (13). The goal of the study was to investigate how well GPs using the SQ and SPICT succeed in identifying patients who could benefit from palliative care. We had the following research questions:

(i) What are the characteristics of the patients identified by the GPs when using the SQ and SPICT?

(ii) What are the characteristics of patients who deceased in the year following the performance of the SQ and SPICT and were they identified?

(iii) What are the performance characteristics of the SQ and SPICT when it comes to predicting 1-year mortality?

(iv) What are the experiences of the GPs when using these tools in every day practice?

Methods

Population

This prospective cohort study took place in two general practices in the Netherlands in 2016 and 2017, one with a size of 2160 patients (practice A), the other 1480 patients (practice B), both situated in urbanized areas.

Data collection

The GPs of the practices used the SQ and SPICT to select potential candidates for palliative care. For the SPICT we used the version translated in Dutch (the SPICT-NL) of April 2016. In the Netherlands GPs generally use electronic patient records managed by special software programs, general practice information systems (GPISs). These GPISs are able to serve as a search engine. This feature was used by partly translating the SPICT into symptoms and diseases coded by the International Classification of Primary Care (ICPC) (14), which are routinely used by GPs in the Netherlands when they record contacts with patients (15). A selection of symptoms and diseases was chosen with which the SPICT could possibly be positive. This selection was evaluated by a panel of GPs with a special interest in palliative care and adjusted according to their comments. The list of ICPC-codes created on basis of the SPICT alongside with some extra information on how it was conceived, is provided as Supplementary Material to this article. A search was performed by the GPIS that generated a list of patients by whom the ICPC-codes derived from the SPICT were found in their records.

First the GPs of the participating practices were asked to go over the patients in their practice and select patients for whom the answer to the SQ would be ‘No’. This was done by using their clinical intuition (Fig. 1). Secondly GPs were presented with the list of patients generated by the digital SPICT-search of the GPIS. They then went over this preselection using the SPICT and selected patients where a minimum of one of the indicators was present. To give an example: there could be patients on the list who in the past had suffered from a form of cancer and therefore had the concerned ICPC-code linked to their medical record, but were presently cured. They would not be selected. From both selections of patients data was provided to the researchers about age, sex and diseases they were suffering from.

Figure 1.

Flowchart of the design and population of the study.

Shortly after making the selections, the GPs were interviewed by one of the researchers about their experiences when using the tools: if they had the idea they had missed a lot of patients; how time consuming it was and if this was in proportion; what they thought of the practicability of the tools.

During the year that followed a record was kept of which patients died in the practice, the date and cause of death.

Ethical considerations

Identifying patients who could benefit from palliative care is considered as routine care in the Netherlands (16). The data generated by the identification process that was used for our study was anonymous and could not be traced back to specific patients. The methods fell within the boundaries of the laws about privacy that applied in the Netherlands at the moment the study took place. As there was no direct contact of the researchers with patients and there was no interference in routine care, there was no need for review of this study by an ethical committee.

Statistical analysis

We performed descriptive statistics on the data provided. We looked at differences in background characteristics of the different selections by the tools. We also calculated the performance characteristics of the tools concerning their ability to correctly predict 1-year mortality. In conclusion we generated case reports with available data of all the deceased patients during the 1-year follow-up in both practices.

Results

Identified patients

From both practices in total 67 patients were selected using the SQ (57 from practice A, 2.6% of the total population, and 10 from practice B, 0.7% of the total population). Using the SPICT the GPIS generated a selection of in total 501 patients (299 from practice A, 14% of the total population of the practice, and 202 from practice B, also 14% of the total population). From this preselection the GPs selected 101 patients (88 from practice A, 29% of the preselection, and 13 from practice B, 6% of the preselection) (Fig. 1).

Characteristics of the identified patients

Table 1 shows the descriptive data from the selections made by the GPs with use of the SQ and the SPICT (both the preselection by GPIS and the final selection by GP). In all selections there is a majority of women: 72% for the SQ, 68% for the SPICT by the GP and 59% for the preselection by the GPIS. In both selections made by the GPs a majority of the patients had an age above 80 years (79% for the SQ and 69% for the SPICT) and there were no patients with an age below 60 years. However, the selection made with the SQ was somewhat older as compared with the SPICT, although not statistically significant. In the selection made by the GPIS the age was distributed more equally. The differences concerning age between the digital selection and the selections by the GPs were statistically significant.

Table 1.

Descriptive data from the selections by the two tools from the two general practices at the start of the data collection in 2016

	SQ (n = 67)	SPICT
		Step 1: preselection by the GPIS (n = 501)	Step 2: final selection by the GP (n = 101)
Sex
Male	19 (28%; 19–40%)	206 (41%; 37–45%)	33 (33%; 24–42%)
Female	48 (72%; 60–81%)	295 (59%; 55–63%)	68 (67%; 58–76%)
Age (years)
<60	0 (0%; 0–4%)	105 (21%; 18–25%)	0 (0%; 0–2%)
60–69	4 (6%; 2–14%)	133 (27%; 23–31%)	10 (10%; 5–17%)
70–79	10 (15%; 8–25%)	117 (23%; 20–27%)	21 (21%; 14–29%)
>80	53 (79%; 68–87%)	146 (29%; 25–33%)	70 (69%; 60–78%)
Disease
Cancer	29 (43%; 32–55%)	260 (52%; 48–56%)	42 (42%; 32–51%)
Dementia	20 (30%; 20–41%)	39 (8%; 6–10%)	29 (29%; 21–38%)
Heart failure	16 (24%; 15–35%)	35 (7%; 5–9%)	23 (23%; 15–32%)
CVA	10 (15%; 8–25%)	50 (10%; 8–13%)	17 (17%; 11–25%)
COPD	10 (15%; 8–25%)	43 (9%; 6–11%)	13 (13%; 7–20%)
Renal failure	21 (31%; 21–43%)	125 (25%; 21–29%)	34 (34%; 25–43%)
Liver disease	0 (0%; 0–4%)	7 (1%; 1–3%)	0 (0%; 0–2%)
Neurological disease	5 (7%; 3–16%)	13 (3%; 1–4%)	6 (6%; 3–12%)
Overlap selections
Also selected by:
SQ	67 (100%)	61 (12%)	58 (57%)
SPICT—selection from GPIS	61 (91%)	501 (100%)	101 (100%)
SPICT—selection by GP	58 (87%)	101 (20%)	101 (100%)

Absolute numbers (percentage; 95% confidence interval).

Cancer was the disease most frequently found in all selections (42–52%), followed by renal failure (25–34%). For the two selections made by the GPs those were followed by dementia (29–30%) and heart failure (23–24%), while for the selection by the GPIS cerebrovascular accident (CVA) (10%) and chronic obstructive pulmonary disease (COPD) (9%) were the next most prevalent diseases.

We found no significant differences between the selections made by the GP using the SQ or SPICT when it came to gender, age, or diseases.

Nine patients were selected using the SQ while not when using the SPICT, six of these were also not selected by the GPIS. These six patients of practice A were selected because they lived in a care home. The GP selected all the patients living in the care home without looking at characteristics or medical condition, because they were all considered to be of older age and frail. Forty-three patients were selected by the GPs using the SPICT, but not when using the SQ.

Characteristics of deceased patients

Table 2 shows the characteristics of the deceased in the year following the selection with the tools. Seventeen of the in total 36 deceased patients were selected by the GP when using both the SQ and the SPICT. Four patients were selected when using the SPICT but not when using the SQ. One patient was selected using the SQ, but not when using the SPICT, while this patient was in the preselection of the SPICT by the GPIS. There were eight deceased patients that were in the preselection by the GPIS, but not selected by the GP using the SPICT. There were six deceased patients that were in none of the selections.

Table 2.

Case reports of the deceased in the two practices in the period of 1 year after performing the SQ and the SPICT (2016–17, n = 36)

	Selection by surprise	Selection SPICT by GPIS	Selection SPICT by GP	Age	Disease according to GPIS	Cause of death	Time between SQ/SPICT and death (days)	Practice
In all selections (n = 17)
1	Yes	Yes	Yes	60–69	Cancer	Larynx carcinoma	238	A
2	Yes	Yes	Yes	60–69	Cancer, COPD	Lung carcinoma	199	A
3	Yes	Yes	Yes	70–79	Cancer	Ovarian carcinoma	263	A
4	Yes	Yes	Yes	>80	Cancer, dementia, heart failure	Colon carcinoma	158	B
5	Yes	Yes	Yes	>80	Cancer, CVA	General deterioration	327	A
6	Yes	Yes	Yes	>80	Cancer, heart failure, CVA	General deterioration	323	A
7	Yes	Yes	Yes	>80	Cancer, heart and renal failure, CVA	General deterioration	204	A
8	Yes	Yes	Yes	>80	Dementia, renal failure	General deterioration	180	A
9	Yes	Yes	Yes	>80	COPD, renal failure, CVA	General deterioration	124	A
10	Yes	Yes	Yes	>80	Cancer, dementia, neurological disease	General deterioration	119	A
11	Yes	Yes	Yes	>80	Renal failure	General deterioration	30	A
12	Yes	Yes	Yes	>80	COPD, renal failure, neurological disease	General deterioration, coma	87	A
13	Yes	Yes	Yes	>80	Neurological disease	Old age	99	B
14	Yes	Yes	Yes	>80	General deterioration	Sepsis	61	A
15	Yes	Yes	Yes	>80	Heart failure, COPD, renal failure	Respiratory failure with heart/lung disease	338	A
16	Yes	Yes	Yes	>80	Cancer, heart and renal failure	Heart failure	134	A
17	Yes	Yes	Yes	>80	Cancer, heart and renal failure	Subdural haematoma after a fall	220	A
Selected by GPIS and SPICT (GP), not selected by SQ (n = 4)
18	No	Yes	Yes	60–69	Heart failure	Lung failure	99	B
19	No	Yes	Yes	70–79	CVA	Sudden death, possibly myocardial infarction	344	A
20	No	Yes	Yes	>80	Cancer, COPD	Stomach carcinoma	242	A
21	No	Yes	Yes	>80	Renal failure	Lung fibrosis	196	A
Selected by the SQ and GPIS, not by SPICT (GP) (n = 1)
22	Yes	Yes	No	70–79	Cancer	Mantle cell lymphoma	37	B
Not selected by GP (SQ and SPICT), selected by GPIS (n = 8)
23	No	Yes	No	<60	CVA	Sudden death	219	B
24	No	Yes	No	>80	CVA	Lung cancer, died within 2 weeks after diagnosis	127	B
25	No	Yes	No	>80	Renal failure, CVA	CVA	126	A
26	No	Yes	No	>80	Cancer (basal cell carcinoma), CVA	Endocarditis and multiple CVAs	311	B
27	No	Yes	No	>80	Cancer (lung)	Dyspnoea with lung carcinoma	232	A
28	No	Yes	No	>80	Cancer (non-Hodgkin, basal cell carcinoma)	Non-Hodgkin lymphoma	280	B
29	No	Yes	No	>80	Cancer (breast, squamous cell carcinoma)	Cardiac arrest	37	B
30	No	Yes	No	>80	Heart failure, renal failure	Cardiac asthma, found death	276	A
In none of the selections (n = 6)
31	No	No	No	60–69	Not in selection GPIS	Euthanasia with colon carcinoma	319	A
32	No	No	No	60–69	Not in selection GPIS	Fever and diarrhoea with abdominal focus	272	A
33	No	No	No	60–69	Not in selection GPIS	Ovarian carcinoma	78	A
34	No	No	No	60–69	Not in selection GPIS	Myocardial infarction	14	A
35	No	No	No	70–79	Not in selection GPIS	Oesophageal carcinoma	316	A
36	No	No	No	>80	Not in selection GPIS	Metabolic acidosis with acute renal failure	72	A

Eleven of the 18 deceased patients that were in the selection made by the SQ were living in the care home and for that reason selected by the GP from practice A (data not shown in Table 2).

When looking at the six deceased patients that were not selected by the GP using either the SQ or the SPICT and also were not in the preselection by the GPIS, they all died from a form of cancer that probably was not yet diagnosed at the moment the tools were used (otherwise it would have come up in the selection by the GPIS) or from a cause that develops acutely, like a myocardial infarction or acute renal failure. You could therefore argue that all of these six patients could not have been recognized by the GP as being in need of palliative care at the time that the tools were used. Similar holds for four patients that were in the preselection by the GPIS, but died suddenly, from causes like a CVA, endocarditis or within 2 weeks after lung cancer was diagnosed.

Performance characteristics

Using the deceased in the year after selection with the tools as an outcome measure for identifying patients who would have benefitted from palliative care, the sensitivity of the SQ is 50% and of the SPICT (with a preselection by the GPIS) 58%. The specificity of the SQ is 99% and of the SPICT 98% (see Table 3).

Table 3.

Performance characteristics of both tools of predicting 1-year mortality (both practices combined, n = 3640, 36 deceased in the year after performance of the tools, 2016–17)

	SQ (n = 67)	SPICT (n = 101)
Sensitivity	50% (34–66)	58% (43–73)
Specificity	99% (98–100)	98% (97–98)
Without 10 deceased patients who arguably could not have been identified  a
Sensitivity	69% (50–84)	81% (63–92)
Specificity	99% (98–100)	98% (97–98)

Percentages (95% confidence interval).

aThe 10 patients that deceased relatively suddenly were included in the category ‘true negatives’ for the calculation of these performance characteristics.

When the performance characteristics of the SQ and SPICT are determined leaving out above mentioned 10 patients that probably could not have been recognized at the time the tools were used, the sensitivity for the SQ becomes 69% and for the SPICT 81%.

Experiences of the GPs

Both GPs mentioned that performing the digital search in the GPIS took quite some time to sort out. One GP found making the selection from the list produced by the GPIS quite time consuming, but all in all worthwhile seeing the results. She had the idea that she had missed patients when using the SQ, which she felt was confirmed when she went over the list generated by the GPIS using the SPICT.

The other GP found going over the list created by the GPIS not so time consuming and quite feasible to do, but did not think the SPICT-method added much to the SQ.

Discussion

GPs selected more patients when using the SPICT (with the preselection by the GPIS) (n = 101) than when using the SQ (n = 67). There were no major differences when it came to the diseases the patients included suffered from.

From the patients deceased in the year after selection, 10 seemed to be unexpected and these patients could well not have been in need of palliative care in the period leading up to their death. None of these 10 patients were in either of the selections.

Looking at the ability to predict 1-year mortality, the SPICT had a better sensitivity than the SQ (58% versus 50%) while specificity was more or less the same for both tools (98% versus 99%).

When leaving out the above mentioned 10 patients, the sensitivity of the tools increased (69% for the SQ, 81% for the SPICT).

Strengths and limitations

The most important limitation of this study was that it only comprises two general practices, which makes generalizing the results difficult. Because of the size, some results are less reliable than others, which is shown by the width of the confidence intervals.

The use of the ability to predict 1-year mortality as a surrogate outcome for the ability to identify patients in need of palliative care is not ideal, but it was the best option considering the size and scope of this study.

A strength was the ability to analyse data on the level of individual patients. With a subject that is difficult to measure, like if a patient might benefit from palliative care, this is a great advantage.

Another strength was that the use of computers to try to increase the efficiency and usability of the SPICT was studied.

Characteristics of patients identified and deceased

One of the issues that is raised when it comes to palliative care, are the unmet needs of noncancer patients (2,3). The SPICT specifically focuses on different chronic diseases, while the SQ is a generic selection method. It is interesting to see that there was no difference between the selections of two tools regarding diseases. When comparing both selections from the GPs to the relatively more objective selection by the GPIS there were also no striking differences on this point. While cancer is the most prevalent disease in all selections, other chronic diseases do not seem to be underrepresented in the selection made with the SPICT, but also not with the SQ.

We did find that the selections made for the SPICT (by GPIS as well as GP) were younger as compared with the SQ (only the difference with the GPIS was statistically significant). You could carefully conclude the SPICT identifies patients with a relatively younger age.

We know that 11 of the 18 deceased patients that were identified using the SQ were so because they lived in a care home. Not every GP has the possibility to recognize patients in need of palliative care in this way, because not all GPs have a care home in their area. This possibly distorts the results in favour of the SQ.

Using both instruments, one GP identified relatively more patients than the other, probably because she included all patients that lived in the care home.

Due to the effects of the size of the study on the reliability (shown in the 95% confidence intervals), the results when it comes to the sensitivity of both tools have to be interpreted carefully.

Predicting 1-year mortality is not the same as identifying patients who could benefit from palliative care. We tried to improve this not so ideal outcome by looking at the characteristics of the deceased patients and leaving out the ones with relatively unexpected deaths. It is noteworthy to see that this improved the sensitivity of both tools.

The SPICT (with a preselection by the GPIS) had a better sensitivity than the SQ. Both tools had a high specificity in our study. The setting, general practices with overall a healthier population may be a cause of this. This is confirmed by other studies in a primary care setting that also found relatively higher specificities (7,8).

The setting in which the tools are used, in a general population or an in-patient population, makes quite a difference when it comes to the performance. Trying to predict 1-year mortality in a general population, like in this study and the ones mentioned before (7,8), creates a lower sensitivity and a higher specificity than trying to do this in a sick population (9,10).

When looking at the purpose of these tools, identifying patients in need of palliative care, a lower sensitivity (wrongfully missing more patients) seems to be worse than a lower specificity (wrongfully identifying more patients).

Feasibility

GPs are no exception to other caregivers when it comes to having to deal with a high workload (17,18). Tools to help identify patients who could benefit from palliative care should thus ideally not be too time consuming. The SQ seems to be less time consuming than the SPICT. Both GPs especially mentioned performing the digital search by the GPIS in this respect. When using the SPICT in a general population, making a preselection is essential, because performing the SPICT on the whole population would not be feasible. It is important to note that once created, most GPISs can save searches and repeat them in the future. In this way, the SPICT-search can be performed for instance yearly without taking much extra time. The performance of the SPICT can also be delegated to someone other than the GP, which is something that is not so easy to do when it comes to the SQ.

Conclusion

This study showed that the SPICT, performed with digital help from the GPIS, seems to be better in identifying patients in need of palliative care than the SQ. It is also more time consuming than the SQ. However, as the digital search can be performed more easily after it has been done for the first time, initial investments can repay themselves.

It is recommended that a similar study is done with more heterogenous population, to see if similar results are found.

Declaration

Funding: this study was funded by ZonMW (project number 844001102).

Ethical approval: as there was no direct contact of the researchers with patients and there was no interference in routine care, there was no need for review of this study by an ethical committee.

Conflict of interest: none.

Click here for additional data file.

Click here for additional data file.