Clinical Validation of the Chinese Version of Patient Completed Caprini Risk Assessment Form.

To create and validate patient-completed Caprini risk score (CRS) tools for Chinese people. We revised Chinese patient-completed CRS form according to previously published studies. We prospectively recruited 70 internal medical patients and 70 surgical patients. The average age of these patients was 54.26 ± 15.29 years, 54.29% of them were male and 80% of them had education beyond high school. The study compared: (1) patient-completed CRS and physician-completed CRS; (2) the final value of physician-completed CRS (physician-completed CRS + body mass index) and CRS in the electronic medical record (EMR) system. Patient-completed CRS was 3.71 ± 3.63, patients spent 3.60 ± 1.24 minutes, 57.14% patients were at high-highest risk; physician-completed CRS was 3.84 ± 3.63, physicians spent 2.11 ± 1.13 minutes, 59.28% patients were at high-highest risk; the final value of physician-completed CRS was 4.12 ± 3.62, 63.58% patients were at high-highest risk; CRS value in the EMR system was 4.07 ± 3.58, 65% patients were at high-highest risk. There were strong positive correlations (P < .0001) between patient-completed CRS and physician-completed CRS (r = 0.978, κ = 0.76) and between the final value of physician-completed CRS and CRS in EMR (r = 0.994, κ = 0.97). This study successfully developed and validated a Chinese patient-completed CRS that we found can replace physician-completed CRS. This results in considerable time saving for physicians and this process should increase the percentage of patients having complete risk assessment when they are admitted to the hospital.

Introduction

Venous thromboembolism (VTE) includes both deep vein thrombosis (DVT) and pulmonary embolism (PE) and is a common cause of morbidity and mortality associated with hospitalization. In fact, PE is the leading cause of preventable death in hospitalized patients. When a patient dies often the family questions the physician regarding if proper risk assessment and prophylaxis was used to prevent the fatal event. The use of patient-friendly assessment tools is important to obtain complete family information and satisfy loved ones regarding the risk assessment and prevention process.[1] Up to two-thirds of VTE events are related to a recent hospitalization. However, medical providers and patients lacked awareness of VTE prevention and its impact on patients’ outcomes.[2] Catterick et al showed that hospitals with a VTE risk assessment rate of 90% or more translated not only into a reduction of hospital-acquired VTE but also in a reduction in VTE-related mortality.[3] It’s well-known that VTE risk assessment tools are available including Wells, Geneva, Caprini, Padua, Khorana, Autar, Improve, and so on. We have selected the Caprini Risk Score (CRS) which includes individual VTE risk factors validated in many specialties, including general surgery, vascular surgery, thoracic surgery, abdominal surgery, plastic surgery, gynecological tumor, craniocerebral surgery, and intensive care unit.[4-13]

Yanyan et al conducted a systematic review and meta-analysis of 7 studies to assess the accuracy of the CRS in predicting VTE in surgical patients. In Wang’s study, the CRS score demonstrated its accuracy stratifying patients according to their VTE risk.[14] Moreover, the effect of risk stratification using the CRS coupled to an effective regime of prophylactic anticoagulation for VTE reduction has shown to be effective in multiple studies. Nimeri et al found a significant reduction in VTE incidence in general surgery, vascular surgery, orthopedics, and neurosurgery after mandatory CRS was used coupled with appropriate prophylactic regimes in all patients.[15] Similarly, Babayan et al found that the implementation of mandatory VTE risk assessment coupled to a thromboprophylaxis strategy significantly reduced the incidence of postoperative VTE events.[16] More than 100 000 people died because of PE every year in the United States, and one-third of VTE-related deaths occurred after surgery.[17,18]

At present, our hospital has embedded the CRS into the electronic medical record (EMR) system, using electronic technology to reduce physicians’ workload, increasing the compliance of VTE assessment, and significantly reducing the occurrence of VTE in the hospital. We intended to develop and validate the patient-completed CRS in Chinese patients, so as to further reduce the work burden of physicians. Involving the patient in their medical care. Having them complete the patient-friendly risk assessment substantially reduces the time required to obtain a thorough patient assessment. We hope this will go a long way to reduce the burden of physicians and lead to lower VTE rates in their patients.

Methods

Patients

According to the sensitivity of CRS to identify VTE risk patients in our previous study, 140 samples were calculated. From January 2019 to January 2020, we prospectively recruited 70 internal medical inpatients and 70 surgical inpatients in Beijing Shijitan hospital. The inclusion criteria were as follows: age ≥ 18 years old and length of stay in hospital ≥ 48 hours. The exclusion criteria were as follows: age < 18 years old and length of stay in hospital < 48 hours. We excluded patients with visual disorders, mental disorders, acquired or congenital cognitive impairment.

Methods

The following 4 steps were used for Caprini risk form development and validation:

Step 1: Standardized translation. Three Chinese physicians (X.C., H.D., and X.T.) independently translated the patient-completed CRS to Chinese, and the fourth Chinese physician (Y.W.) completed the unified version. Finally, all translators discussed and revised the final Chinese paper version.

Step 2: Establishing the study. According to the different language versions, including English, Spanish, Arabic, and Polish versions, we identified the body mass index (BMI) as the added value to be calculated by the physician finalizing the form. The BMI was not written into the printed patient-completed CRS and physician-completed CRS. The BMI is calculated by the EMR since patient height and weight are available in the medical record. Patients completed printed patient-completed CRS, and trained physicians completed printed physician-completed CRS and CRS in the EMR system. Medical students used a double-blind method to count the cumulative scores and time of CRS completed by patients and trained physicians, as well as the final value of physician-completed CRS (scores of physician-completed CRS + BMI); the scores of CRS in the EMR system which was finished by the same trained physician.

Step 3: Pilot study. According to the inclusion criteria and exclusion criteria, we selected 10 patients from internal medicine and surgery for a pilot study. By interviewing the patients, we further clarified the Chinese definition of bed rest which was that the patients have been in bed and/or with sustained walking of fewer than 30 feet.[19] There were no other problems with Chinese translation.

Step 4: Formal study. With the consent of the hospital ethics committee, we started this study. The trained physician was trained by a senior Chinese translator. We prospectively recruited 70 internal medical inpatients and 70 surgical inpatients at admission and conducted the study as described above.

Statistical Analysis

We adopted the 2009 CRS revision.[20] The model included about 40 different risk factors of VTE in patients. Caprini risk score includes 41 risk factors with 1, 2, 3, and 5 scoring points: low risk (0-1), moderate risk (2), high risk (3-4), or highest risk (≥5). (1) Spearman’s rank was used to analyze printed patient-completed CRS versus physician-completed CRS, and the final value of physician-completed CRS versus CRS in EMR; (2) Cohen’s Kappa was used to calculate the agreement between the above 2 groups; (3) the Bland Altman method was used to analyze whether the differences between the above 2 groups were within the acceptable range. If the methods were considered to have a good agreement, they could be used interchangeably. SPSS version 25 was used to analyze all data (IBM Corp), and the count data were expressed as a percentage (%); the measurement data conforming to normal distribution was expressed as ± S, P < .05 was statistically significant.

Results

In this study, the average age of the patients was 54.26 ± 15.29 years; 54.29% of the patients were male. The proportions of patients from internal medicine and surgical departments were equal at 50%. Eighty percent of the patients had education beyond high school. The average score of patient-completed CRS was 3.71 ± 3.63, and patients spent 3.60 ± 1.24 minutes, completing the form. There were 57.14% patients at high-highest risk according to the patient completed form. The average score of physician-completed CRS was 3.84 ± 3.63, and physicians spent 2.11 ± 1.13 minutes completing the form. The physicians found that 59.28% patients were at high-highest risk. The average final value of physician-completed CRS revealed was 4.12 ± 3.62, 63.58% patients were at high-highest risk. The average score of CRS in EMR indicated was 4.07±3.58, 65% patients were at high-highest risk. Combining the BMI with the physician completed form increased the percentage of high-risk patients. This final value is placed in the EMR (Table 1).

Table 1.

Patient Characteristics.

Variables	Cohort
Age (years) (X¯ ± S)	54.26 ± 15.29
Male (n, %)	76 (54.29)
Department (n, %)
Internal medicine	70 (50.00)
Surgery	70 (50.00)
Education level (n, %)
Elementary	28 (20.00)
High school	66 (47.14)
College and postgraduate	46 (32.86)
Patient-completed score, mean (X¯ ± S)	3.71 ± 3.63
Patient-completed time (minute) (X¯ ± S)	3.60 ± 1.24
Patient-completed risk classification (n, %)
Low risk	31 (22.14)
Moderate risk	29 (20.71)
High risk	45 (32.14)
Highest risk	35 (25.00)
Physician-completed score, mean (X¯ ± S)	3.84 ± 3.63
Physician-completed time (minute) (X¯ ± S)	2.11 ± 1.13
Physician-completed risk classification (n, %)
Low risk	26 (18.57)
Moderate risk	31 (22.14)
High risk	45 (32.14)
Highest risk	38 (27.14)
Final value of physician-completed CRS (X¯ ± S)	4.12 ± 3.62
Final value physician-completed risk classification (n, %)
Low risk	26 (18.57)
Moderate risk	25 (17.86)
High risk	41 (29.29)
Highest risk	48 (34.29)
CRS in EMR (X¯ ± S)	4.07 ± 3.58
CRS in EMR risk classification (n, %)
Low risk	24 (17.14)
Moderate risk	26 (18.57)
High risk	45 (32.14)
Highest risk	46 (32.86)

Abbreviations: CRS, Caprini risk score; EMR, electronic medical record.

In this study, Spearman correlation analysis was used. A correlation coefficient between 0 and 0.3 indicates a weak correlation, a correlation coefficient between 0.3 and 0.5 indicates a low correlation, a correlation coefficient between 0.5 and 0.8 indicates a moderate correlation, and a correlation coefficient between 0.8 and 1 indicates a strong correlation. We found that there were strong positive correlations between patient-completed CRS and physician-completed CRS (r = 0.978) and between the final CRS calculated by adding the BMI to the physician-completed CRS and CRS in EMR (r = 0.994; P < .0001; Table 2).

Table 2.

Spearman Correlation Analysis of Different Evaluations.

Group	r	P value
Patient-completed versus physician-completed	0.978	<.0001
Final CRS value of physician-completed versus CRS in EMR	0.994	<.0001

Abbreviations: CRS, Caprini risk score; EMR, electronic medical record.

In this study, Cohen kappa was used to calculate the agreement between the 2 groups. A κ value ≤ 0.4 indicates a poor agreement, a κ value from 0.4 to 0.75 indicates a medium agreement and a κ value ≥ 0.75 indicates an excellent agreement. Through analysis, we found that the κ value of patient-completed CRS versus physician-completed CRS was 0.76 (P < .0001); the κ value of the final value of physician-completed CRS versus CRS in EMR was 0.97 (P < .0001). The agreement between the 2 groups was excellent (Table 3, Table 4).

Table 3.

Risk Classification Between Patient-Completed CRS Versus Physician-Completed CRS.

	Physician-completed (n, %)
Patient-completed	Low risk	Moderate risk	High risk	Highest risk	Total
Low risk	24 (17.14)	4 (2.86)	3 (2.14)	0 (0.00)	31 (22.14)
Moderate risk	3 (2.14)	21 (15.00)	4 (2.86)	1 (0.71)	29 (20.71)
High risk	0 (0.00)	5 (3.57)	36 (25.71)	4 (2.86)	45 (32.14)
Highest risk	0 (0.00)	0 (0.00)	2 (1.43)	33 (23.57)	35 (25.00)
Total	27 (19.29)	30 (21.43)	45 (32.14)	38 (27.14)	140 (100.00)

Abbreviation: CRS, Caprini risk score.

Table 4.

Risk Classification Between Final Value of Physician-Completed Versus CRS in EMR.

	CRS in EMR (n, %)
Final value	Low risk	Moderate risk	High risk	Highest risk	Total
Low risk	25 (17.86)	0 (0.00)	0 (0.00)	0 (0.00)	25 (17.86)
Moderate risk	0 (0.00)	25 (17.86)	0 (0.00)	0 (0.00)	25 (17.86)
High risk	0 (0.00)	2 (1.43)	37 (26.43)	0 (0.00)	39 (27.86)
Highest risk	0 (0.00)	0 (0.00)	1 (0.71)	50 (35.71)	51 (36.43)
Total	25 (17.86)	27 (19.29)	38 (27.14)	50 (35.71)	140 (100.00)

Abbreviations: CRS, Caprini risk score; EMT, electronic medical record.

In this study, we made a scatter diagram with the mean value as the horizontal axis and the difference value as the vertical axis. If 95% of the scatters were in 95% confidence interval, the 2 methods were in agreement and interchangeable. Figure 1 showed that the disagreement rate was only 3.57% (< 5%), indicating that patient-completed CRS and physician-completed CRS had a small probability to be different. Figure 2 showed that the disagreement rate was only 1.43% (<5%), indicating that the final value of physician-completed CRS and CRS in EMR had a small probability to be different. We thus conclude that patient-completed CRS and physician-completed CRS can be replaced by each other. The same is true for the final value of physician-completed CRS and CRS in EMR.

Figure 1.

Bland Altman patient-completed versus physician-completed.

Figure 2.

Bland Altman final value of physician-completed versus CRS in EMR. CRS indicates Caprini risk score; EMR, electronic medical record.

Discussion

In 2017, Fuentes et al first made the English version of patient-completed CRS and found its good agreement with physician-completed CRS. Physicians only need to calculate BMI. The average time for patients to complete the form was 5 minutes, and for physicians was 6 minutes.[21] In 2018, Paz Rios et al translated the English version of patient-completed CRS into Spanish, Arabic, and Polish, and also found a strong correlation and an excellent agreement between patient-completed CRS and trained physician-completed CRS. Similarly, physicians only need to calculate BMI value, and the time for patients to fill the form was also about 6 minutes.[22]

Caprini risk score was widely used in China as the VTE risk assessment score recommended by The American College of Chest Physicians (ACCP) guidelines.[23] From 2017, our hospital embedded the CRS into the EMR system and compulsorily completes the VTE risk assessment for all admitted patients. To reduce the physicians’ workload and improve the VTE risk assessment rate of hospitalized patients, we first created the Chinese version of patient-completed CRS and compared it with trained physician-completed CRS. At the same time, we compared the final value of physician-completed CRS with the CRS in EMR to explore whether patient-completed CRS can replace the current CRS in EMR.

Pop TR et al found that CRS can identify 62% of high-risk DVT patients, while Padua can identify 54.7% of high-risk DVT patients.[24] Liu et al found that the sensitivity, positive predictive value, and negative predictive value of CRS were higher than those of Padua (P < .05).[25] A Chinese study confirmed the high sensitivity of the CRS in identifying high-risk patients (82.3% of high-highest risk VTE patients can be identified by CRS, compared with 30.1% of Padua). Caprini risk score can effectively identify the risk of VTE in Chinese hospitalized patients.[26] The prediction ability of the CRS has been also tested in patients with cancer. First, Philip et al demonstrated a high incidence of postoperative VTE (12%) in patients with cancer despite the use of inpatient thromboprophylaxis. Thrombotic complications were the most common cause of 30 days death after surgery,[27-29] and led to prolong hospital stay and increased hospital costs.[30] In the United States, an analysis of the prophylaxis in medical patients with enoxaparin (MEDENOX) trial documented that patients who could walk 30 feet at one time but were not on low molecular weight heparins (LMWH) had a 50% risk reduction in thrombosis incidence. A further significant risk greater than 50% reduction in those ambulatory patients was seen in the group receiving enoxaparin 40 mg.[31] Zhou documented that most VTE patients were considered at low-risk and did not recommend prophylaxis by using the Padua score.[32] Wang and his colleagues compared the 3 evaluation scales including Khorana, Caprini, and Autar and found that the sensitivity of CRS was higher than Khorana and Autar.[33] Stroud et al also verified that CRS may accurately predict the risk of VTE in patients undergoing gynecological cancer surgery. After evaluation, the proportion of patients receiving VTE prevention increased from 12% in 2004 to 63% in 2010, and the incidence of postoperative VTE decreased significantly.[34]

Our previous study compared the effectiveness of CRS and Padua in hospitalized patients, which proved that CRS was effective and had higher sensitivity and better predictability than Padua.[35] Junli and other researchers compared Wells score, Geneva score, and CRS in neurology, neurosurgery, and internal medicine and found that CRS had a higher predictive value in neurology, neurosurgery, and internal medicine inpatients. The advantage of CRS was that the model can recommend corresponding prophylaxis according to the VTE risk level of patients, which was convenient and practical for physicians.[36] The American College of Chest Physicians guidelines showed that CRS can identify and prevent 84.3% of cases with VTE risk. The risk level of CRS was closely related to mortality during hospitalization and 6 months of discharge. However, some researchers believed that the problems of CRS were complex and that it took time for medical practitioners to question patients. To facilitate VTE risk assessment, it is necessary to explore how to improve the scoring method, such as developing a questionnaire convenient for patients based on CRS, patients’ self-evaluation, or embedding CRS into the EMR system to carry out computer-based automatic evaluation.[37] Our study verified that there was no significant difference between the patient-completed CRS and trained physician-completed CRS. The time taken by the patients was about 3 minutes, which was shorter than the English, Spanish, Arabic, and Polish versions mentioned above. There were a strong positive correlation and an excellent agreement between the patient-completed CRS and trained physician-completed CRS, as well as the final value of physician-completed CRS and CRS in EMR (r > 0.95; κ > 0.76). The probability of inconsistency between patient-completed CRS and trained physician-completed CRS, and the probability of inconsistency between the final value of physician-completed CRS and CRS in EMR were low (< 5%). Thus, they can be replaced with each other. This was consistent with the above results of Fuentes and Paz-Rios. It showed that the Chinese patient-completed CRS was effective and feasible, which can replace the CRS completed by physicians in the current EMR system, and the risk assessment of VTE can be completed by patients themselves when they are admitted to the hospital, to reduce the work burden of physicians and improve the public awareness of VTE prevention. We advocate that the physician check the patient form, add the BMI.

We tried to do external validation of our findings, but a limited number of hospitals have integrated CRS into EMR at present. However, we are collecting data and will do external validation in the future. All patients included in our study had a normal cognitive function. However, many elderly patients in China, especially disabled elderly patients in bed, have impaired or lose cognitive function. They are the population with high-highest risk of VTE. In the next step, we will design a VTE assessment score for caregivers or family members who care for these elderly patients with cognitive impairment and evaluate the agreement and correlation between caregiver-completed CRS and physician-completed CRS. It has also been suggested that everyone perform a baseline score with their family now before illness occurs. The form should be checked and finalized by their personal physician. That final score should be kept in a safe place, assessable by family members. When an emergency occurs, the base score can be presented on admission to be adjusted according to the illness. Great time saving is achieved and the data immediately available for unconscious patients or stroke victims. We aim to further reduce the physicians’ work burden, improve the assessment rate of VTE in hospitalized patients, prevent the VTE events, protect the medical safety of hospitalized patients, and reduce the occurrence of adverse safety events in hospitals.