Development and validation of CAVE score in predicting presence of pressure ulcer in intensive care patients.

BACKGROUND: Pressure ulcers (PUs) are one of the quality care indicators in nursing care. They are considered to primarily be preventable. Early identification of the patients most at risk particular for critically ill patients is crucial for providing prompt care. Several tools have been developed to support healthcare providers, but their validities are limited in Thailand. Development of tools with better performance is essential. AIMS: To develop and validate a PU risk assessment tool with good diagnostic properties in intensive care units (ICUs).
METHODS: A prospective study was conducted in ICUs of a tertiary care hospital, Thailand from January 2019 to April 2020. Baseline data were collected at admission to the ICUs. Skin assessment was evaluated every 24 h. Data were divided into two sets: model development and model validation. Creating a risk score which was derived from multivariate methods were performed. Youden index were used to determine the optimal cut-off point. Then, the other dataset was used to validate the risk score. Receiver Operating Characteristic (ROC) curves was used to demonstrate the performance of the test.
RESULTS: The study included 288 and 270 patients for development and validation models. The risk score consisted 4 clinical factors; presence of Cardiovascular disease, low serum Albumin, having Ventilated, and Edema (CAVE score). The area under the ROC curve (AUC) was 0.8 and a score at 2.5 was the best cut-off point. The AUC in the validation group was 0.6, age<60 years was 0.78, and age≥60 years was 0.57.
CONCLUSION: The predictive validity of the CAVE score is limited but comparable to the existing tools in Thailand. However, it has a good diagnostic property in young patients. The CAVE score could be considered as an alternate screening tool in critical care setting particularly for young patients.

Introduction

Pressure ulcers (PUs) are defined according to the Revised National Pressure Ulcer Advisory Panel Pressure Injury Staging System as a localized damage to the skin and underlying soft tissue usually over a bony prominence or related to a medical or other device. The injury can present as intact skin or an open ulcer and may be painful. The injury occurs as a result of intense and/or prolonged pressure or pressure in combination with shear. The tolerance of soft tissue for pressure and shear may also be affected by microclimate, nutrition, perfusion, comorbid conditions, and condition of the soft tissue [1]. It is a frequent complication of inpatients especially in intensive care units (ICUs). Because these patients have more complicated comorbid diseases, unstable hemodynamic, increased tissue pressure, or failure to respond to the tissue pressure properly as a consequence of sedation, analgesia, having mechanical ventilator, bedridden for long times, and/or the use of muscle relaxants [2, 3]. The incidence varied from 3.3% to 59.4% depending on study design [3, 4, 5, 6, 7, 8, 9, 10]. For example, the incidence of patients who developed PUs in an adult ICU of a general third-level hospital, Spain in 2015 was 8.1% (grade I and II pressure ulcer was 40.6% and 59.4%, respectively) [3]. An incidence of 31% was found from a study in Italy (2000–2010) [4] and 25.6% from a study in Iran (2011–2012) [5]. One study in surgical ICU in the US (2005–2008) reported the incidence of PUs of 23.9% [6] and one study in a 1200-bed university hospital in China (2012) was 31.4% [8]. Patients with PUs are at greater risk to have several adverse health outcomes which are increased morbidity, mortality and poorer quality of life [3, 11]. Although not all PUs are preventable [12, 13], active prevention remains crucial in lessening the occurrence of the PUs. In addition, it is a nursing care indicator of quality of care which requires a process of individualized care targeted at declining or controlling risk factors of PUs [7]. Therefore, early detection of patients who are at risk is a challenging issue. Existing pressure ulcer risk assessment tools are available. Currently, there is no particular tool that can be used satisfactory in all clinical settings because it depends on types of care, skills of healthcare workers and possible risk factors in diverse settings [8, 14].

The components of each tool are primarily based on the generally factors associated with pressure ulcers including 1) mobility/activity, 2) perfusion (including diabetes), and 3) skin/pressure ulcer status [15]. Existing tools that have been specifically studied in critical care including the Cubbin and Jackson scale, COMHON Index, Doulas scale, and CALCULATE. For the Braden scale, it is a globally used tool and was originally developed in general wards; however, it has also been validated in ICUs [3, 15, 16, 17, 18]. The performance of those tools at the optimal cut-off points in the same study showed that the sensitivity of the Cubbin and Jackson scale, Braden scale, Douglas scale was 89%, 97%, and 100%, respectively. The specificity of those tools is 61%, 26%, and 34%, respectively. The area under the Receiver Operating Characteristic (ROC) curve of the Cubbin and Jackson scale was 0.83, the Braden was 0.71, and the Douglas scale was 0.79 [16]. Another study with direct comparison of Braden scale, Braden (ALB) scale (a modified version of the Braden scale, in which the nutritional subscale is based on serum albumin) [19], COMHON index and CALCULATE in ICU setting of Thailand reported that the sensitivity and specificity of the Braden scale was 50% and 80.15%, the Braden (ALB) scale was 65.2% and 73.04%, the COMHON index was 37.5% and 83.98%, and the CALCULATE was 68.75% and 68.75%. This study concluded that the Braden (ALB) and the CALCULATE showed the greatest and comparable performance at their optimal cut-off points with the area under curve of 0.69 [20].

Overall, the available pressure ulcer risk assessment tools have limited validity in critical care setting of Thailand. More tools with greater performance in predicting pressure ulcers should be developed. Therefore, the objective of this study was to develop and validate a pressure ulcer risk assessment tool for prediction of pressure ulcers with good diagnostic properties in intensive care setting. Healthcare providers could potentially use this tool as an alternate tool in order to risk stratify patients early in their care.

Methodology

Patient population and study setting

This was a prospective study that was comprised of 2 populations. The first one was used to develop a model to predict development of pressure ulcers in ICUs (model development) whereas the second one was used to validate the predictive model (model validation). Eligible participants of this study were Thai patients who were 18 years of age or over, were admitted to the ICUs of the Internal Medicine (both groups) or Surgical Department (only the model development groups) with a minimum stay of 24 h, had Acute Physiology and Chronic Health Evaluation (APACHE) II score <35, and had no pressure ulcer when admitted. The exclusion criteria were patients who they and/or their families were not willing to participate in the study or the medical team, the patients, and/or the patients' families decided to terminate active treatment with a do-not-resuscitate (DNR) order. Patients who had new pressure ulcer(s), received palliative care, were discharged to other setting or died were terminated from this study.

The model development was composed of 288 participants. It was the sub-study of the “Pressure ulcers in critically ill patients project” which was a prospective descriptive study conducted in the ICUs of Srinagarind Hospital (a tertiary care hospital in Khon Kaen, Thailand) from January to April 2019. The primary project aimed to examine the incidence and predicting factors of pressure ulcers of critically ill patients and was published elsewhere [21]. This current study developed a predictive model from the primary project [21].

The model validation consisted of 270 participants. It was a diagnostic study which was conducted in the same setting from October 2019 to April 2020.

Instrument

Braden scale

The Braden scale is the most widely used tool in clinical setting. It was developed in the United States [22] and was validated in many countries including Thailand (both non-ICU and ICU setting) [17, 18, 23, 24]. It consists six factors: sensory function, moisture, activity, mobility, nutrition, shearing force, and friction. It employs a three- or four-point scale, and the total score ranges from 6 to 23. Higher risk of pressure ulcer development is related to lessor scores. The cut-off points in critical care used have varied from 12 to 13. At the cut-off point of 12, its sensitivity and specificity on the first day of admission were 66.7% and 55.8%, and they were 77.8% and 73.4% on the second day of admission. At the cut-off point of 13, its sensitivity and specificity were 81% and 66% [17].

Skin assessment tool

A skin assessment tool was used to evaluate skin condition that delineated the bony prominences and required the evaluator to rate the presence or absence of lesions at each site. Any lesion on any skin surface that could be attributed to pressure was staged according to the following criteria: (I) nonblanchable erythema which was present at the same site on 2 consecutive study days (intervals of 48–72 h); (II) break in skin, such as blisters and abrasions; (III) break in skin exposing subcutaneous tissue; and (IV) break in skin exposing and/or extending into muscle or bone [25].

Procedure

Assessment of inter-rater reliability of the 2 trained nurses was done with Kappa of 1.0 prior to collect the data in the main study. The two trained nurses were registered nurses who have had experiences of practice of 16 and 10 years and both of them were specialized in pressure ulcer care. All potential patients were asked to participate in this study. Then, after consent, patients' information was collected. Their caregivers would sign the consents instead in case of the patients could not give consent. For the clinical data, there were patients' demographic data at admission to the ICUs and every 24 h. This included age, sex, reason for ICU admission (at admission), mechanical ventilation, serum albumin, length of stay, APACHE II score, and a presence of pressure ulcer. A skin assessment tool was also administered every 24 h and Braden scale was assessed every 72 h according to the hospital's policy regarding the routine pressure-ulcer assessment. If the patients had developed a pressure ulcer at the time of assessment, they were allocated to the pressure ulcer group, otherwise they were allocated to the non-pressure ulcer group until they met the termination criteria. Patients' worst pressure ulcer risk scores prior to ulcer development were used in the analysis, regardless of group.

Statistical analysis

The inter-rater reliability of the two trained persons who collected the data was analyzed using kappa statistic and it required at least 0.8 before proceeding to the main study. Baseline data variables were summarized using descriptive statistics and presented as percentage, mean, and standard deviation, but if the data distribution was not normal, median and inter-quartile ranges were used instead. For Model development, an area under a Receiving Operating Characteristic (ROC) curve of the final model to predict development of pressure ulcers derived from the result of the primary study which using regression analysis was reported [20]. The coefficient from each significant factor derived from multivariate logistic regression analysis were used to create a risk score for development of pressure ulcers. The lowest coefficient was rounded up to 1.0 and recorded as a risk score of 1. The risk scores of other factors were calculated as their coefficient divided by the lowest coefficient, then rounded up by 0.5 points. The individual risk score formula was created by the summation of all significant factors multiplied by their risk score. The optimal cut-off points of the risk score were determined using Youden's index.

The second study population was used to validate the predictive model for development of pressure ulcers at the optimal cut-off point. The area under the ROC curve, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and likelihood ratio were calculated. Subgroup analysis in patients with pressure ulcers under 60 years of age and 60 years of age or over was performed using the same process. All data analysis was performed using STATA version 10.0 (Stata Corp, College Station, TX, USA).

Ethical considerations

The present study was provided by the Khon Kaen University Faculty of Medicine Ethics Committee as instituted by the Helsinki Declaration (HE611532 for the model development and HE621339 for the validation model). Before participation in this study, a detailed description of the study and process was explained to the patients and/or their caregivers if the patients could not make decision. Patient individually signed an inform consent form and received a copy. Assurances were given that all information would be used for academic purposes only and would remain confidential. The patients were informed that their names and identifying address were kept confidential. The researchers used code numbers to present the data. All data were stored in the researchers’ personal computer.

Results

There were 288 patients for model group and 270 patients for validation group during the study period. Baseline data of studied populations are shown in Table 1. The median age, gender, length of ICU stays, APACHE II score, and Braden scale were comparable between the 2 groups. Presence of comorbid cardiovascular disease (including coronary artery disease, stroke and transient ischemic attack (TIA), peripheral vascular disease, and systemic scleroderma) was much higher in validation group while ventilated patients was lessor than the other group. The incidences of PUs in model and validation groups were 11.1% and 8.9%, respectively. The majority of them was in stage 2 of pressure ulcers. In the validation group, patients who were less than 60 years old (92 patients) had comorbid of cardiovascular disease of 7.6% (7/92 cases), mechanical ventilator of 21.7% (20/92 cases), presence of edema of 31.5% (29/92 cases), and median serum albumin was 3.2 mg/dl (inter-quartile range (IQR) 2.6,3.7).

Table 1

Baseline characteristic of model and validation group of studied populations.

Variables	Model populationN = 288		Validation populationN = 270
Age (years), median (IQR1,3)	63	(51,77)	64.5	(55,75)
Male sex, n (%)	146	(51.4)	149	(55.2)
Reasons for ICUs admission; n (%)
Post-operation	20	(6.9)	9	(3.3)
Respiratory failure	113	(39.2)	100	(37.0)
Other organ failure	152	(52.8)	161	(59.6)
Trauma	3	(1.1)	0	(0)
Comorbid cardiovascular disease, n (%)	11	(3.8)	42	(15.6)
DM, n (%)	87	(30.2)	66	(24.4)
Ventilated, n (%)	118	(41)	80	(29.6)
Edema, n (%)	95	(33)	74	(27.2)
Serum albumin (mg/dl), median (IQR1,3)	2.8	(2.3,3.4)	3.3	(2.8,3.7)
Length of ICU stay (days), median (IQR1,3)	5	(3,8)	4	(3,7)
APACHE II score, median (IQR1,3)	18	(13,22)	19	(13,23)
Presence of PUs, n (%)	32	(11.1)	24	(8.9)
Non medical device-related	26	(81.3)	19	(79.2)
Medical device-related	6	(18.7)	5	(20.8)
Stage of PUs
Stage 1	5	(15.6)	1	(4)
Stage 2	26	(82.3)	23	(96)
Stage 3	1	(3.1)	0	(0)
Braden scale	14	(13,16)	15	(13,16)

Note: IQR; inter-quartile range, CI; confidence interval, comorbid cardiovascular disease included coronary artery disease, stroke and TIA, peripheral vascular disease and systemic scleroderma.

Development of risk score in predicting an occurrence of pressure ulcers in ICU patients in model population

According to the primary study entitled of “Factors predicting development of pressure injury in critically ill patients” [20], there were 4 clinical risk factors associated with development of PUs after using logistic regression model which included 1) Patients with mechanical ventilation, 2) Presence of cardiovascular disease, 3) serum albumin <3.3 mg/dl, and 4) Presence of edema. Table 2 shows the results of logistic regression analysis with their coefficients and risk scores. The predictive model was simplified to the clinical risk score as Cardiovascular-low Albumin-Ventilator-Edema (CAVE) score = 2 (presence of cardiovascular disease) + 2 (presence of serum albumin <3.3 mg/dl) + 1.5 (presence of mechanical ventilator) + (presence of edema). The presence and absence of those risk factors was defined as 1 and 0, respectively. The area under the ROC curve in the final model was 0.80 (95% CI 0.73–0.87), as shown in Figure 1. The performance of CAVE score at different cut-off points in predicting development of PUs is shown in Table 3. At the cut-off point of 2.5 demonstrated the best performance according to the are under the ROC curve and the Youden index.

Table 2

Factors associated with development of pressure ulcers in ICU patients according to logistic regression analysis with their coefficients and risk scores.

Factors	Adjusted OR	95%CI	Coefficients	Risk scores
Presence of cardiovascular disease	5.29	(1.37,20.29)	1.67	2
Serum albumin <3.3 mg/dl	5.19	(1.17,23.05)	1.65	2
Ventilated	3.37	(1.39,8.16)	1.22	1.5
Edema	2.74	(1.21,6.2)	1.01	1

Note: the risk factors in the final model development were derived from the primary study [20], OR; odds ratio, CI; confidence interval, comorbid cardiovascular disease included coronary artery disease, stroke and TIA, peripheral vascular disease and systemic scleroderma.

Figure 1

Receiver-operating characteristics (ROC) curve of the CAVE score.

Table 3

Performance of the CAVE score in predicting pressure ulcer risk in ICU patients.

Cut-off point	Sensitivity	Specificity	AUC	PPV	NPV	LR+	LR-	Youden index
≥2	96.9	33.6	0.65	15.4	98.9	1.46	0.09	0.31
≥2.5	87.5	61.3	0.74	22.0	97.5	2.26	0.20	0.49
≥3	84.4	63.3	0.74	22.3	97.0	2.3	0.25	0.48
≥3.5	71.9	71.1	0.72	23.7	95.3	2.49	0.40	0.43
≥4	53.1	86.7	0.70	33.3	93.7	4	0.54	0.40
≥4.5	53.1	87.5	0.54	34.7	93.7	4.25	0.54	0.42
≥5	15.6	98.4	0.57	55.6	90.3	10	0.86	0.15

Note: AUC: area under the Receiver operating characteristic curve, PPV: positive predictive value, NPV: negative predictive value, LR+: likelihood ratio positive, LR-: likelihood ratio negative.

The performance of the CAVE score in validated populations at the cut-off point of≥ 2.5

The overall performance of the CAVE score in predicting development of PUs in validation group was poor (AUC 0.67) and very poor for older patients (age ≥60 years) (AUC 0.57), but it was good for the younger patients (age <60 years) with the AUC of ROC curve of 0.78 as shown in Table 4.

Table 4

Characteristics of CAVE score of model validation in predicting development of pressure ulcers in ICU patients categorized by age group.

Test performances	All patients	Age <60 years	Age ≥60 years
No. of patients	270	92	178
AUC	0.67	0.78	0.57
Sensitivity	58.3	83.3	50.0
Specificity	67.1	72.1	64.4
PPV	14.7	17.2	13.6
NPV	94.3	98.4	92.0
LR+	1.77	1.82	1.4
LR-	0.62	0.04	0.78

Note: AUC: area under the Receiver operating characteristic curve, PPV: positive predictive value, NPV: negative predictive value, LR+: likelihood ratio positive, LR-: likelihood ratio negative.

Discussion

This study developed the risk score predictor of developing PUs in critically ill patients using clinical parameters. The basis of the risk score predictor was derived from the comparable population in general ICU setting where the late adulthood to young-aged adults was the majority of patients and organ failure was the leading reason for ICU admission [[2], [7], [9], [26]]. The incidences of PUs in the development model and the validation model in this study were 11.1% and 8.9%, respectively. Existing reports show diverse incidences of PUs in ICU ranged from 3.3-59.4% as mentioned the details in the introduction part [[2], [3], [4], [5], [6], [7], [8], [9], [27]]. The difference of study methodology is possibly explained the wide range of the incidence such as this study excluded patients with the APACHE II scores ≥35 since they were at a high risk of death whereas one study included all hospitalized patients who were admitted to the ICU [3, 4].

A simple risk score predictor was developed, the CAVE score, there were 4 clinical factors including 1) Presence of cardiovascular disease, 2) Patients with mechanical ventilation, 3) Low serum albumin, and 4) Presence of edema. Those clinical factors had been studied in several studies. Presence of cardiovascular disease might produce a mismatch in blood flow supply and demand. In addition, these patients prone to receive analgesics and might not sense increased tissue pressure [[4], [28]]. Use of mechanical ventilation might change blood flow and lessen the skin's resistance to development of PUs [1, 17]. Low serum albumin which is an indicator of malnutrition, inflammatory response, and a decline of oncotic pressure. Total body fluid then shifts to peripheral tissue and causing an alteration in tissue tolerance [11, 18]. The last factor is presence of edema, it causes a compromised tissue circulation, tissue oxygenation and increases risk of ulceration due to an increase in pressure over the blood vessels and a decline of pressure difference between the capillary blood and tissue fluid. Hypoalbuminemia had a minor role in causing edema. The major mechanisms are primary salt-retaining types of renal disease and homeostatic responses causing salt retention by normal kidneys which is found in patients with cardiac and liver disease [29]. The total score of the CAVE score ranges from 0 to 6.5. Higher scores indicate greater risk of PUs development. The cut-off point of 2.5 showed the best performance with the AUC of 0.74 with sensitivity of 87.5% and specificity of 61.3%.

The validation model was conducted in comparable setting of the development model in different periods. The overall performance of the CAVE score at the cut-off point of 2.5 in this group showed lower than the development model in particular for its sensitivity but the specificity was acceptable. However, among younger patients (less than 60 years old), the CAVE score performed good in predicting development of PUs. This implies that the CAVE score is suitable for using in patients less than 60 years old. The possible explanation is the patients in this group had proportion of cardiovascular disease closer to the development model (7.6% and 3.8%) whereas all patients in validation group had much greater numbers of the patients (15.2%). Other risk score predictor of the CAVE score between the 2 groups (low serum albumin, mechanical ventilation, edema) were comparable. Given the presence of cardiovascular disease was the important clinical risk for predicting development of PUs in the CAVE score, the performance of the CAVE score in younger patients which the characteristics of population was closer than all patients in the validation group appear more satisfied than the older group.

The Braden scale is a broadly and routinely used tools for screening the risk of having PUs. It was actually developed in non-ICU setting though it has been studied in several settings. Its sensitivity is rather low and consumes times to evaluate [2, 14, 15, 16, 17]. Therefore, it might not appropriate to use in critical care. Though the CAVE score in validated group had lower diagnostic properties than the development group, it had acceptable specificity. Patients with a positive CAVE score (a score of 2.5 or over) indicated at high risk in developing of PUs. In comparison with existing risk score predictor in ICU setting in recently published report in Thailand, at the optimal cut-off points, the AUC of ROC curves of the Braden scale, Braden (ALB), COMHON index and CALCULATE were 0.65, 0.69, 0.61 and 0.69, respectively [20] whereas the areas under the ROC curve of the CAVE score at the optimal cut-off points in the model development and the model validation were 0.74 and 0.67, respectively.

As the CAVE score is required only 4 clinical variables, easy to administer, short-time consuming, provide acceptable validity especially in patients younger than 60 years old compared to other studied risk score predictors in Thailand (the Braden scale, Braden (ALB), COMHON index and CALCULATE), it can be used as an alternate risk assessment tool in the ICU setting. Given its high predictive validity in young patients (<60 years old) with its AUC of 0.78 and greater than the highest performance studied tools in the same setting (AUC of 0.69), it is recommended to use the CAVE score for risk stratification in this population.

There were some limitations. First, other factors such as age, diabetes mellitus, APACHE II score were known predictors but could not be included in this model due to the statistic methodology form the primary study [20]. These might cause the lessen diagnostic properties in validation group. Second, the sample size of the patients younger than 60 years old in validated model is slightly low, greater population might be better to examine the validity and generalizability of the test. Last, the two raters might not representative of general ICU nurses because they were trained particularly for this study.

Conclusion

The incidence of pressure ulcers in intensive care setting in this study from 2 populations was about 9–11%. The clinical predictive model (CAVE score) for predicting patients who were at high risk of having pressure ulcers was developed, consisting of presence of cardiovascular disease, low serum albumin, patients with mechanical ventilation and presence of edema. The CAVE score appears to be suitable to use in critically ill patients who are less than 60 years old. Further studies with higher sample size are recommended. For general use, its predictive validity is limited; however, it provides acceptable specificity and had comparable diagnostic performance to the existing risk assessment tools. Thus, the CAVE score can be an option for nursing staff to use in clinical practice in critical care setting.

Declarations

Author contribution statement

S. Ninbanphot and P. Narawong: Conceived and designed the experiments; Performed the experiments; Contributed reagents, materials, analysis tools or data.

A. Theeranut: Conceived and designed the experiments; Wrote the paper.

K. Sawanyawisuth: Analyzed and interpreted the data; Contributed reagents, materials, analysis tools or data; Wrote the paper.

P. Limpawattana: Conceived and designed the experiments; Analyzed and interpreted the data; Contributed reagents, materials, analysis tools or data; Wrote the paper.

Funding statement

This work was supported by the Research and Training center for Enhancing Quality of life of Working Age People and the Neuroscience Research and Development Group, Khon Kaen University, Khon Kaen, Thailand.

Competing interest statement

The authors declare no conflict of interest.

Additional information

No additional information is available for this paper.