Validation and impact of a simplified clinical decision rule for diagnosing pulmonary embolism in primary care: design of the PECAN prospective diagnostic cohort management study.

INTRODUCTION: Combined with patient history and physical examination, a negative D-dimer can safely rule-out pulmonary embolism (PE). However, the D-dimer test is frequently false positive, leading to many (with hindsight) 'unneeded' referrals to secondary care. Recently, the novel YEARS algorithm, incorporating flexible D-dimer thresholds depending on pretest risk, was developed and validated, showing its ability to safely exclude PE in the hospital environment. Importantly, this was accompanied with 14% fewer computed tomographic pulmonary angiography than the standard, fixed D-dimer threshold. Although promising, in primary care this algorithm has not been validated yet. METHODS AND ANALYSIS: The PECAN (Diagnosing Pulmonary Embolism in the context of Common Alternative diagNoses in primary care) study is a prospective diagnostic study performed in Dutch primary care. Included patients with suspected acute PE will be managed by their general practitioner according to the YEARS diagnostic algorithm and followed up in primary care for 3 months to establish the final diagnosis. To study the impact of the use of the YEARS algorithm, the primary endpoints are the safety and efficiency of the YEARS algorithm in primary care. Safety is defined as the proportion of false-negative test results in those not referred. Efficiency denotes the proportion of patients classified in this non-referred category. Additionally, we quantify whether C reactive protein measurement has added diagnostic value to the YEARS algorithm, using multivariable logistic and polytomous regression modelling. Furthermore, we will investigate which factors contribute to the subjective YEARS item 'PE most likely diagnosis'. ETHICS AND DISSEMINATION: The study protocol was approved by the Medical Ethical Committee Utrecht, the Netherlands. Patients eligible for inclusion will be asked for their consent. Results will be disseminated by publication in peer-reviewed journals and presented at (inter)national meetings and congresses. TRIAL REGISTRATION: NTR 7431. © Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

This is the first study that prospectively validates the YEARS algorithm for diagnosing pulmonary embolism (PE) in primary care.

The added diagnostic value of C reactive protein will be quantified by developing a double biomarker approach with the aim to better differentiate between PE and pneumonia.

Because this study does not randomise general practices between the current guidelines and the YEARS algorithm, results are compared with existing literature rather than a direct comparison.

A possible limitation of this study is that different reference tests will be used to determine the presence of PE, which might lead to differential verification bias.

Introduction

Diagnosing acute pulmonary embolism (PE) is challenging, particularly in primary care. Signs and symptoms are often non-specific and may mimic other cardiopulmonary diseases.1–3 D-dimer is used as a biomarker to disentangle PE from such other conditions. Yet, notably in patients with other cardiopulmonary diseases, this is far less efficient, with a chance of a false-positive D-dimer reaching 90% in older patients with cardiopulmonary comorbidity.3 4 Subsequently, many patients suspected of PE are referred for reference testing (CT pulmonary angiography; CTPA) whereas only 10%–15% will have a confirmation of the diagnosis.5 Moreover, CTPA has the inherent risk of contrast nephropathy, that may also occur in up to 10%–15% of all CTPAs performed, depending on pre-existing renal impairment.6 Besides, CTPA is costly and patients are exposed to ionising radiation which may increase their cancer risk later in life.7 While most evidence on use of diagnostic tests for suspected PE has been gained in the hospital setting, patients often visit their general practitioner (GP) first. The few studies that focused on the use of D-dimer testing in primary care observed that both deep venous thrombosis (DVT) and PE could be safely ruled out in almost 50% of suspected patients. However, of the referred patients only 25% and 30%, respectively had a confirmed DVT or PE, a number that is much lower in the elderly.8–10 Importantly, PE is also still one of the most frequently missed diagnoses in primary care, underlining the need for improvement of the diagnostic algorithm.11

Recent studies in secondary care developed and validated a new algorithm with flexible D-dimer thresholds: the YEARS algorithm.12–14 This strategy starts with assessing three patient history and physical examination items: (1) clinical signs of DVT, (2) haemoptysis and (3) PE considered the most likely diagnosis by the physician. At the same time, D-dimer testing is performed in all patients. If none of the three YEARS items are present, a D-dimer threshold of 1000 ng/mL is applied. In contrast, if one or more YEARS items are present, the conventional threshold of 500 ng/mL is used. If the D-dimer is below the relevant threshold, PE is considered ruled out and patients are not referred for CPTA. As compared with applying a fixed D-dimer threshold of 500 ng/mL, this YEARS algorithm increased the proportion of patients in whom CTPA was not required from 34% to 48%. Importantly, refraining from referral for CTPA was safe with a 3 months failure rate in patients with initial normal tests of 0.61% (95% CI 0.36 to 0.96).13 Given the substantially lower prior probability of PE in primary care, we hypothesise that the YEARS algorithm can also be safely used when used in the primary care setting. However, considering differences in case-mix of patients and physician experience, prospective validation and impact assessment of the use of the YEARS algorithm applied by GPs is necessary before its wide-scale use in primary care can be recommended. Therefore, the aim of this study is to prospectively validate the YEARS algorithm for ruling-out PE in primary care, with the use of a point-of-care (POC) or rapid D-dimer assay.

Objectives

The objectives of this study are threefold. The primary objective is to prospectively validate the YEARS algorithm in primary care. We will calculate its calibration (observed vs expected probabilities) and its discriminative ability. Using the previously proposed decision threshold we will also estimate the impact of the use of the YEARS algorithm by estimating its safety (defined as the proportion of missed PE cases in the group of patients not referred for CTPA), and its efficiency (defined as the proportion of patients correctly not referred for CTPA).

A secondary objective is to quantify the added diagnostic value of performing C reactive protein (CRP) measurement in patients with suspected PE in primary care. GPs seldom only consider a single diagnosis in patients presenting with (sub)acute shortness of breath, and pneumonia is an important alternative diagnosis in these patients.1 15 Hence, a combined D-dimer and CRP biomarker approach may lead to a better classification of underlying causes of respiratory and/or chest symptoms, and thus better exclusion of PE.

Another secondary objective is to investigate which determinants contribute to a ‘yes’ answer on the YEARS item ‘PE most likely diagnosis’. We hypothesise that scoring of this item is correlated with several patient-related and physician-related factors. Scoring of this YEARS item may differ in primary and secondary care physicians and may therefore influence the use and interpretation of the YEARS algorithm.

Methods and analysis

Study design

We will perform a multicentre, prospective diagnostic cohort study in the primary care setting including patients with suspected acute PE, defined as (sub)acute onset of unexplained shortness of breath with or without chest symptoms, such as thoracic pain or pain on inspiration. Patients will be managed according to the YEARS algorithm and will be followed up for 3 months, with an uneventful follow-up period being the diagnostic standard for ruling out PE. The inclusion period for recruiting patients into the study is estimated at 2–3 years.

Clinical setting and participants

Our study will be conducted within the Dutch primary care setting. Patients will be recruited by their GP both during working hours and out-of-hours primary care services. Participating GPs will identify eligible patients and obtain informed consent. Consecutive patients with suspected PE are eligible for inclusion if they are aged 18 years or older and provide written informed consent. Exclusion criteria will be current treatment with therapeutic doses of vitamin K antagonists, low-molecular weight heparin or a direct oral anticoagulant, life expectancy less than 1 month estimated by the GP and pregnancy until 6 weeks after delivery.

Study procedures

The study procedures are shown in figure 1. Patients who are visiting their GP because of suspected PE will be asked for consent for participation and data collection. Next, the GP scores the three YEARS items, age, sex, signs, symptoms, co-morbidity items and performs a quantitative POC or rapid D-dimer test. In this study, different D-dimer assays are used, namely the quantitative assay for POC testing and several different D-dimer assays in the laboratories in the participating regions. When a patient has no YEARS items and a D-dimer below the threshold of 1000 ng/mL, or when a patient has one or more YEARS items and a D-dimer below the threshold of 500 ng/mL, PE will be considered as ruled out. However, referral for other reasons, such as a severe pneumonia, remains appropriate and left at the discretion of the attending GP. In patients with no YEARS items and a D-dimer above 1000 ng/mL, or in patients with one or more YEARS items present and a D-dimer above 500 ng/mL, a referral to secondary care for CTPA will follow. Deviation from the YEARS recommendation is allowed for, if judged clinically needed and as such left at the discretion of the physician. Patients will not be followed up in person, but will be instructed to schedule a follow-up appointment in case of worsening or persistent symptoms. If the patient has symptoms of PE during this follow-up appointment, the management decision is made based on the standard guidelines and discretion of the physician.

Figure 1

Flowchart study procedures. CRF, case report form; CRP, C reactive protein; DVT, deep venous thrombosis.

For our secondary objective, additional blood will be drawn to determine CRP during the initial visit with the same venepuncture on the POC or rapid assay. Yet, CRP is only determined for research proposes, that is, one of the secondary objectives, and therefore GPs are instructed to use the YEARS algorithm primarily for further clinical management. Furthermore, several determinants who may have an association with the YEARS item ‘PE most likely diagnosis’ are collected and reported as such by the participating GP, see table 1.

Table 1

Questions and categorical responses associated with the subjective years item

Questions/determinants	Categorical responses
Have you ever missed a diagnosis of a pulmonary embolism in your practice?	Yes/ No
Did you have a ‘gut feeling’ that there was something wrong?	Yes/ No
Does it concern a well-known patient?	Yes/ No
Was there diagnostic delay?	Yes/ No If yes: physician delay, patient delay or both
How was the working load the day of consultation?	Low/medium/high
What is your implicit probability assessment of pulmonary embolism?	Unlikely, likely, very likely

Reference standard

After 3 months of follow-up, the GP receives a follow-up form with questions about the final diagnosis and treatment. This final diagnosis is the reference standard for this study, similarly as done in previous studies performed in the field of diagnosing PE.3 13 16 17 In the follow-up form, it is asked whether PE is proven—that is, by CTPA, VQ scan, ultrasonography showing a DVT, or a combination of these imaging procedures, all as performed and classified by hospital physicians after referral—and (finally) whether there were alternative diagnoses present. Also, the treatment decision (anticoagulation or treatment for alternative diagnosis) is recorded by the GP. Here, it is important to acknowledge that in the Dutch healthcare system GPs are always fully informed of changes in patient status, also from affiliating hospitals, exemplifying the fidelity of this approach which we also found highly reliable in previous diagnostic studies from our group.8 9

A diagnosis of PE presence is considered definitive as (1) CTPA demonstrating a filling defect in a central, segmental or lobular pulmonary artery, or a subsegmental filling defect which requires anticoagulant therapy, or (2) a high probability ventilation/perfusion lungscan, or (3) a pulmonary angiogram demonstrating an intraluminal filling defect, or (4) PE demonstrated at autopsy in case of death, or (5) DVT confirmed with ultrasonography of the leg in patients with suspected PE. Importantly, PE is considered ruled-out in the absence of any PE-defining items as described above during the initial clinical assessment plus 3 months of uneventful follow-up. In case of absence of PE, the GP will fill in the most likely alternative diagnosis on the follow-up form, including pneumonia, heart failure, chronic obstructive pulmonary disease or asthma, cardiac disease and myalgia, based on specialist letters and the clinical judgement and management decisions made by the GP, as explained above. All patients with an unexpected death during follow-up will be adjudicated for the presence of possible PE as the cause of death, following definition from the on-going work on defining PE-related death from the International Society on Thrombosis and Haemostasis (ISTH) which will be available before the study is completed and endpoints are adjudicated.

Sample size calculation

According to previous studies performed in primary care, the prevalence of PE in suspected patients with a low PE probability (indicating no referral for CTPA) based on the used decision rule plus negative D-dimer, was 1.0%–1.5%.9 Although recently the ISTH proposes a variable diagnostic safety threshold with adjustment for the prevalence of PE in the study population,18 3% is internationally deemed as an acceptable safety margin of missed PE in the low probability (ie, non-referred) patients, so-called false negatives, and is widely used in diagnostic studies of PE.8 9 17 19 Hence, we also use 3% as the upper margin of the 95% CI around the point estimate of our false-negative rate. Assuming a conservative false negative rate of 1.5% with the upper margin of the 95% CI not exceeding 3.0% (one-sided, as any proportion lower than 1.5% is preferable), we need to include 300 patients in the low probability group according to the YEARS algorithm. It should be stressed here that this point-estimate of 1.5% is highly conservative as well, given that in the validation study of the YEARS algorithm in secondary care the point estimate was 0.4%. Moreover, the proportion of patients classified in the non-referred category was 48%.13 Yet our study is conducted in primary care, with a lower overall PE prevalence of around 12% (table 2). We therefore anticipate that the proportion of patients in the low probability group will be at least as high. Thus, to arrive at 300 patients with a low YEARS probability (ie, a negative YEARS algorithm), we need to include at least 600 primary care patients suspected of PE. Accounting for 10% of patients with missing follow-up information, we conservatively target to include 750 patients for this study of which at least 300 patients with a negative YEARS algorithm. This full sample of 750 patients would allow us to robustly demonstrate (or reject) the safety of applying the YEARS algorithm in primary care. To arrive at a total study population of 750 patients, we estimated to need approximately 75 full-time working GPs who will include five patients per year in the study period of 2 years.

Table 2

Description of the different prospectively validated clinical decision rules for pulmonary embolism9 13 17

Study characteristics			Results		Diagnostic accuracy
Year	Clinical decision rule	Population	Sample size	Prevalence PE (%)	Safety (%)*	Efficiency (%)†
2012	Wells rule	Primary care	598	12.2	1.5	45.5
2014	Age-adjusted D-dimer threshold	Secondary care	3346	19.0	0.2	34.7
2017	YEARS algorithm	Secondary care	3465	13.2	0.4	47.7

*Proportion of false-negatives among patients not referred at baseline.

†Proportion of patients not referred at baseline among all included patients.

PE, pulmonary embolism.

Data analysis

To quantify the diagnostic accuracy of the YEARS algorithm, we will estimate its discrimination (using the c-statistic) and its calibration (using the calibration plot comparing predicted probability with observed probability). Hereto, the linear predictor of the YEARS algorithm first needs to be estimated for each included patient into our study, using the original regression coefficients of the YEARS items as derived in the original derivation paper, with the intercept refitted to the prospective data as included in our primary care cohort (to best reflect differences between overall PE prevalence across populations).12 Subsequently, by applying the previously proposed YEARS algorithm threshold, we will also estimate the corresponding false negative proportion and efficiency of the strategy. Besides, results will be stratified for each assay specific, where deemed appropriate and necessary. Finally, we will perform an additional analysis with and without including the diagnoses of subsegmental PE because of the clinical unknown significance.20 21

Second, we will quantify the added diagnostic value of CRP beyond the YEARS algorithm. Hereto, we will first construct multivariable logistic regression models with PE being the binary outcome and the YEARS items with D-dimer as dependent variables, which model is then extended by addition of CRP. D-dimer and CRP will be included into the model on a continuous scale, if needed using natural cubic splines function if the association between both laboratory markers and PE is non-linear. We will use the likelihood ratio test (using a p value of 0.15) to quantify the added contribution of CRP. Similarly, as above, we will also quantify the calibration and discrimination of both models using bootstrapping techniques to correct for overfitting.22

Additionally, we will quantify to what extent the YEARS algorithm with D-dimer and CRP can predict the presence of the differential diagnoses simultaneously, using polytomous regression modelling.23 The differential diagnosis is hereto divided in three categories: PE, pneumonia and other. The final polytomous regression model will consist of those two submodels and allows one to estimate the probability of presence of PE, pneumonia and other diagnoses in each patient.

For our third objective, the prevalence of the subjective YEARS item ‘PE most likely diagnosis’ will first be described with a corresponding 95% CI. Then, variables among patients with and without a positive score on this YEARS item will be compared. The variables that will be investigated are described in table 1. These variables are first compared univariably, and then combined in a multivariable logistic regression analysis with the item ‘PE most likely diagnosis’ as a dichotomous outcome. Some of these quantitative data will later be used to complement with qualitative data obtained from another future study entailing interviews with GPs, during a mixed method analysis.

Safety interim analysis

After the first 100 included patients with a negative YEARS algorithm a safety analysis will be performed. Based on previous studies, the expected percentage false-negatives (ie, patients with PE in the low probability category of the YEARS algorithm) should at least not be higher than 1.5%. This will correspond to an expectation of approximately 1–2 missed PE cases in the first 100 non-referred patients. If the proportion of false-negatives in these first 100 patients with a negative YEARS algorithm clearly and beyond reasonable doubt is larger than 1.5%, the study will be put ‘on hold’ pending additional analyses. Although it is difficult to identify when study continuation in such a diagnostic management study is clearly contra-indicated, we arbitrarily use a difference of at least three standard deviations (p value ≈0.002). For this study, that would mean missing no more than arbitrarily 5–6 patients in the first 100 patients with a PE in the low probability category. If this safety analysis is satisfactory, the study will continue as planned with additional safety checks alongside patient accrual into the study where deemed appropriate and necessary.

Handling of missing data

In case of missing data, the researchers will first contact the treating physician to retrieve this information. When this is not possible, or information remains missing, we will use multiple imputation techniques to yield unbiased inferences, if the missing at random assumption is likely.24 We expect to detect missing data which are missing at random, that is, that the missing data for that subject is based on other observed patient characteristics. Multiple imputation could be reliably used even if 40% of the data of one variable is missing (as shown by a simulation study) which however is unlikely to occur in our study but exemplifies that we anticipate multiple imputation to provide robust results. Then the missing values will be multiple imputed with a conditional imputation method to minimise bias and increase precision.22 24

Patient and public involvement statement

There are no patients involved in the development of the study design and protocol. However, a patient representative with a strong network within the field of patient advocates gave insight in the patient experience to our study group.

Ethics and dissemination

This study will be conducted according to the principles of the Declaration of Helsinki and in accordance with the Medical Research Involving Human Subjects Act. The Medical Ethical Committee Utrecht in the Netherlands approved the study protocol. Patients eligible for inclusion will be asked for their written consent by participating GPs before the YEARS algorithm is applied. Results of the PECAN study are expected in 2022/2023 and will be disseminated by publication in peer-reviewed journals and presented at (inter)national meetings and congresses.

Discussion

The PECAN study will evaluate the safety and efficiency of implementing the YEARS algorithm for ruling out PE in primary care. Using the three clinical examination items, plus D-dimer testing and subsequently decide whether referral is necessary, is worldwide routine clinical practice and based on current (inter)national guidelines.25 Previous research showed that the YEARS algorithm is safe in a secondary care setting with 14% fewer referrals for CTPAs and is now standard-of-care in many emergency wards in the Netherlands.13 This decrease of CTPAs could be especially useful for primary care medicine, since GPs are the gatekeepers to secondary care and often need to decide whether the patient could be treated in primary care or has to be referred to secondary care. GPs are constantly balancing between over-referral and under-referral of patients with suspected PE, given the associated harms related to both over-referral and under-referral. The YEARS algorithm may safely reduce the need for referral for CTPA, which may at least partly alleviate the diagnostic uncertainty and dilemma whether or not to refer a patient to secondary care. Yet, good performance and safety in secondary care-based studies is not always a guarantee that the model also performs well in a primary care healthcare setting, due to inherent differences in prevalence, case-mix and physician experience in primary care. This is the primary argument to embark on this prospective diagnostic validation study. Additionally, we will further explore the ability to refine the diagnostic process by incorporating CRP into the diagnostic model, as well as study determinants for the diagnostic item ‘PE most likely diagnosis’.

Limitations

A possible limitation of this study is that, by design, a combined reference standard will be used to determine the presence or absence of PE. The reference standard for the non-referred patients is a clinical follow-up of 3 months, while in the referred patients imaging techniques are used. This combined reference standard may result in differential verification bias.26 This may lead to biassed estimates of the sensitivity and specificity, but gives reliable and clinically interpretable positive and negative predictive values, as the choice of the reference standard is almost—by design—fully dependent on the outcome of the YEARS algorithm. However, we explicitly designed this validation study as a pragmatic study following routine care to evaluate the accuracy and safety of the YEARS algorithm as would be performed in real-world daily practice. Using this combined reference standard is compliable with the practical use when implemented as standard-of-care in primary care centres in the future.

Some GPs participating in our study will perform D-dimer on a specific POC-assay. Yet, not all physicians will have access to this specific POC-assay and a substantial proportion may have to determine D-dimer via the laboratory. This could lead to practical issues when applying the YEARS algorithm, since D-dimer needs to be determined in all patients before it can be decided to refer the patient for CTPA or not. So, when a POC-assay for D-dimer is not available, those patients first have to visit a laboratory. However, including patients from general practices with and without a POC-assay for D-dimer will increase the generalisability of our results and is an advantage when implementing this as standard-of-care when proven safe and efficient.

Also, for our secondary objective, a CRP measurement is done in all patients suspected of PE. Although we instruct GPs to only use the YEARS algorithm without formally interpreting the CRP result, we cannot completely rule-out the possibility that this might influence their management decision. However, this is similar as conducting diagnostic VTE studies in an emergency department where often multiple tests are available and interpreted during the diagnostic work-up of PE.13 14 17

Finally, our study does not include a control group because we do not randomise general practices between the current guidelines and the new YEARS algorithm. Therefore, a direct comparison between the YEARS algorithm and usual care will explicitly not be part of this validation study. Rather, results are compared with existing literature, most notably (although not exclusively) the recent validation of the YEARS algorithm in secondary care.

Comparison with literature findings

Recently, another strategy with the aim to reduce unnecessary CTPAs has been prospectively validated in secondary care: the age-adjusted D-dimer threshold.17 Although this strategy alone would result in a larger proportion of patients in whom PE could be considered ruled out, this is only applicable in patients older than 50 years whereas younger patients benefit most of refraining from CTPA given the long-term radiation effects. Also, a comparison of the YEARS algorithm and the age-adjusted D-dimer threshold showed an absolute reduction of 8.7% of CTPAs in favour of the YEARS algorithm.13 Furthermore, a post-hoc analysis was performed to investigate the added value of the age-adjusted D-dimer threshold when incorporated with the YEARS algorithm. This study showed that in the patients aged above 50 years, the efficiency of the algorithm was increasing with a 4.7% decrease of CTPAs. However, the safety was jeopardised with four additional missed diagnosis of PE resulting in a failure rate of 1.2%.27 A summary of the prospective validation studies of the YEARS algorithm and age-adjusted D-dimer threshold in secondary care, as well as the Wells rule in primary care are shown in table 2.9 13 17

To conclude, the PECAN study will prospectively validate and quantify the safety of the YEARS algorithm in patients with suspected acute PE in primary care. If proven safe, this new clinical decision rule could then be implemented in daily care. In addition, the diagnostic value of performing CRP measurement in patients with suspected PE will be quantified.