Identification and evaluation of observational measures for the assessment and/or monitoring of level of consciousness in adult palliative care patients: A systematic review for I-CAN-CARE.

BACKGROUND: The use of observational measures to assess palliative care patients' level of consciousness may improve patient care and comfort. However, there is limited knowledge regarding the validity and reliability of these measures in palliative care settings. AIM: To identify and evaluate the psychometric performance of observational level of consciousness measures used in palliative care.
DESIGN: Systematic review; PROSPERO registration: CRD42017073080. DATA SOURCES: We searched six databases until November 2018, using search terms combining subject headings and free-text terms. Psychometric performance for each identified tool was appraised independently by two reviewers following established criteria for developing and evaluating health outcome measures.
RESULTS: We found 35 different levels of consciousness tools used in 65 studies. Only seven studies reported information about psychometric performance of just eight tools. All other studies used either ad hoc measures for which no formal validation had been undertaken (n = 21) or established tools mainly developed and validated in non-palliative care settings (n = 37). The Consciousness Scale for Palliative Care and a modified version of the Richmond Agitation-Sedation Scale received the highest ratings in our appraisal, but, since psychometric evidence was limited, no tool could be assessed for all psychometric properties.
CONCLUSION: An increasing number of studies in palliative care are using observational measures of level of consciousness. However, only a few of these tools have been tested for their psychometric performance in that context. Future research in this area should validate and/or refine the existing measures, rather than developing new tools.

What is already known about the topic?

The European Association for Palliative Care (EAPC) framework for sedative use recommends that patients’ level of consciousness should be evaluated as part of their periodical assessments during and after administering sedative medication.

Observational measures are frequently employed for monitoring consciousness levels in settings where sedatives and analgesics are commonly used.

The use of observational measures to assess palliative care patients’ level of consciousness may improve patient care and comfort; however, little is known about which measures are the most appropriate, valid and reliable to use in the palliative care setting.

What this paper adds?

An increasing number of studies are using observational tools for the assessment of palliative care patients’ level of consciousness.

Only eight of these tools have been tested for their psychometric performance with palliative care patients in single validation studies, and none have been tested for all measurement properties.

Most measures of level of consciousness used in primary studies are ad hoc tools for which no formal validation has been undertaken or tools developed and validated in non-palliative care settings.

Implications for practice, theory or policy

Clinicians and researchers should be mindful of the limited evidence supporting the psychometric quality of existing level of consciousness measures, especially in terms of responsiveness, when using such scales in the palliative care setting.

Future research should focus on validating and refining the existing measures for use in palliative care, rather than developing new tools.

Background

Palliative care patients may experience alterations in their level of consciousness, either as a result of disease and symptom progression or as an effect of different pharmacological treatments.[1] Clinicians may intentionally reduce the consciousness of some patients, especially towards the end of life when symptom burden tends to increase, by administering sedative and/or analgesic medication. This practice aims to relieve patients’ intractable distress resulting from one or more treatment-resistant symptoms.[2]

National and international palliative care organisations recommend using sedative medication for the alleviation of refractory symptoms at the end of life.[3] However, the prevalence and practice of sedative use vary considerably according to setting and country.[4-6] Nevertheless, the majority of clinical practice guidelines on the use of sedatives in palliative care agree that sedative medication should be used proportionately, to the extent that distressing symptoms for each individual patient are adequately addressed.[2,7,8]

Inappropriate use of sedative and analgesic medication may have considerable consequences for the care and experience of patients and family members. A survey among palliative care nurses found that sedative use was considered insufficiently effective by approximately 40% of the respondents,[9] while another study reported suboptimal use of palliative sedation performed by general practitioners in 11 of the 27 described cases.[10] Inadequate symptom palliation can be traumatic for patients and a significant source of emotional distress for their families.[10,11] Conversely, the use of disproportionately high doses of sedatives may be equally distressing for relatives due to the impaired ability of the patient to interact with family members and the possible risk of hastening death.[12,13]

The European Association for Palliative Care (EAPC) framework for sedative use recommends that patients’ level of consciousness should be evaluated as part of their periodical assessments during and after administering sedative medication. This is in order to avoid the effects of over- or under-sedation and fulfil the requirements of proportionality.[2] In settings where sedatives and analgesics are commonly used, observer-rated measures are frequently employed for monitoring consciousness levels.[14-16] A review of sedation instruments in intensive care units identified 25 studies describing relevant tools.[14] Similarly, another review found that numerous tools measuring sedation depth had been used in clinical research on procedural sedation.[16] Although the authors of these reviews concluded that further research into the psychometric performance of the identified measures is needed, a number of measures achieved high ratings for validity and reliability in the settings/populations in which they were tested. Most of the instruments in these studies comprise a single item with a categorical grading representing decreasing levels of consciousness, usually assessed by patients’ response to stimulation of increasing intensity. This type of scale structure may create overlaps between different consciousness levels which are not necessarily mutually exclusive, but provides benefits in terms of simplicity and ease of use, so allowing for repeated administrations to be quickly performed and, consequently, enabling the close monitoring of responses to sedative and analgesic use.[17] Other advantages of using valid and reliable observational measures for the assessment of level of consciousness include improved consistency in medication administration, better communication among healthcare professionals, enabling the development of sedation guidelines and protocols, and facilitating comparison between research data and findings.[18-20] Occasionally, level of consciousness scores may also provide an indication of disease progression and expected survival.[21-23]

Despite these benefits being highly applicable and relevant to the palliative care context, little is known about which measures are the most appropriate, valid and reliable to use with palliative care patients. The aim of the present systematic review, therefore, was to (1) identify all relevant observational levels of consciousness tools used in primary research studies, (2) describe their content and (3) critically appraise their psychometric performance. This review was undertaken as part of the sedation work package of I-CAN-CARE (Improving care, assessment, communication and training at the end of life), a Marie Curie-funded research programme on prognosis and sedative use in palliative care.

Methods

This review was reported in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) Statement[24] and the review protocol published in the International Prospective Register of Systematic Reviews (PROSPERO; registration number: CRD42017073080).

Search strategy

A four-step search strategy was employed (Table 1). An initial broad search was performed to identify primary research studies reporting the use of observational level of consciousness measures and produce a list of search terms. Six databases were then systematically searched using a combination of subject headings and free-text terms for palliative care, measurement instruments and sedative use, adjusted for each database. Subsequently, the reference lists of all included papers were hand-searched for relevant publications. When eligible articles were identified, the process of backward reference searching was repeated until no more relevant publications could be located. The same method was applied for finding newer studies citing the included papers. Finally, the authors of conference abstracts meeting inclusion criteria were contacted for full-text publications. Where relevant data were missing from included papers, authors were also contacted.

Table 1.

Search strategy and eligibility criteria.

Search strategy	Step 1: broad search of relevant literature	1. Identification of relevant publications in PsycINFO and MEDLINE2. Compilation of text words contained in title, abstract and authors’ keywords, and database index terms, to produce a list of search terms
	Step 2: Systematic database search	Databases:(1) CENTRAL, (2) CINAHL, (3) Embase, (4) MEDLINE, (5) PsycINFO and (6) WoS
		Dates:Database inception – 14 November 2018
		Restrictions:No language or other restrictions applied
		Search terms (used in MEDLINE and modified for other databases):1. Self Report/2. Checklist/3. (tool* or assess* or survey* or question* or measur* or method* or scale* or checklist* or rating* or test* or instru* or inventor* or technique* or monitor* or observ* or rate* or function* or scoring system* or outcome*).mp.4. 1 or 2 or 35. Palliative Care/6. exp Terminal Care/7. Hospices/8. (palliat* or terminal* or endstage or hospice*).mp.9. (end adj3 life).mp.10. (care adj3 dying).mp.11. ((advanced or late or last or end or final) adj3 (stage* or phase*)).mp.12. 5 or 6 or 7 or 8 or 9 or 10 or 1113. ((continuous or deep or intermittent or intermediate or respite or mild) adj3 (sedat* or an?esthe*)).mp.14. Deep Sedation/15. Conscious Sedation/16. sedat*.mp.17. 13 or 14 or 15 or 1618. 4 and 12 and 17
	Step 3: Citation searching	(1) Backward citation searching (hand-searching reference lists of included publications) and (2) forward citation searching (hand-searching studies citing included publications through Google Scholar) were repeated until no more relevant publications could be located
	Step 4: Contacting authors	1. Authors of conference abstracts meeting inclusion criteria contacted for full-text publications2. Authors of included papers contacted where relevant data were missing from publications
Eligibility criteria	Inclusion criteria	1. Primary research studies2. Full-text research articles3. English language publications4. Studies reporting the use of observer-rated measures5. Studies conducted with adult (>18) palliative care patients6. Scales assessing and/or monitoring depth of sedation/consciousness level
	Exclusion criteria	1. Non-primary studies, including systematic reviews2. Opinion articles, editorials, book chapters3. Case report studies and studies providing no information about sample size4. Non-English language publications5. Studies with non-adult (<18) palliative care patients6. Studies reporting the use of patient/self-reported measures7. Scales measuring drowsiness or somnolence8. Studies reporting on the use of binary-response measures

CENTRAL: Cochrane Central Register of Controlled Trials; CINAHL: Cumulative Index to Nursing and Allied Health Literature; WoS: Web of Science.

Eligibility criteria

Full-text publications of primary studies (prospective or retrospective, patient-based or clinician-based) describing the use of observational measures (validated or ad hoc) for the assessment and/or monitoring of level of consciousness/sedation depth in adult palliative care patients were included.

We excluded non-primary studies, such as systematic reviews, and studies providing no information about sample size. Due to resource constraints, non-English language publications were also excluded.

Study selection

After removing duplicates 11,938 titles and abstracts were screened against eligibility criteria (A.M.K.). A second reviewer (E.M.) independently screened a random 10% selection. The inter-reviewer agreement for the initial title and abstract screening was κ = 0.71. Full-text publications which potentially met inclusion criteria after first screening were each independently assessed for eligibility by two reviewers from a group of six (A.M.K., J.S., E.M., S.M., B.V. and P.S.). Discrepancies at each stage of study selection were resolved through discussion.

Data extraction

We extracted the following information for each included study into a standardised form: first author, date of publication, country of origin, study aim(s), setting, sample size and participant characteristics. For each measure identified, tool name, measurement aim/purpose, number of subscales and items and response options were extracted. Data on the psychometric performance of instruments, where available, were also extracted.

Psychometric performance of included measures

We used a checklist (Table 2) to evaluate the psychometric performance of included measures. This checklist drew on that developed by Zwakhalen et al.[25] with some modifications, following discussion between A.M.K. and B.V., based on the established criteria for developing and evaluating health outcome measures.[26-28]

Table 2.

Quality criteria for measure appraisal.

Domain	Property	Score	Description
	Number of participants	2	N ⩾ 100 and the number of palliative care patients included was relative to the number of items/variables or 50 < N < 100 and corrected for multiple testing
		1	50 < N < 100 and the number of palliative care patients included was relative to the number of items/variables or N < 50 and corrected for multiple testing
		0	N < 50 or number of palliative care patients included not relative to the number of items/variables or N < 50 and not corrected for multiple testing
Validity	Content validity	2	A description of the construct that is being measured is provided and target population is involved in item selection
		1	A description of the construct that is being measured is provided or target population is involved in item selection
		0	The construct that is being measured is not described and limited/no involvement of target population in item selection
	Criterion validity	2	Correlates acceptable to high (r > 0.60) according to the ‘gold standard’ or according to a ‘silver standard’
		1	Correlates moderate–acceptable (0.40 < r < 0.60) according to the ‘gold standard’ or according to a ‘silver standard’
		0	Correlates low (r < 0.40)
	Structural validity	2	Appropriate method of factor analysis performed and factors account for ⩾50% of the total variance
		1	Factor analysis performed but another method would have been more appropriate
		0	Factors account for <50% of the total variance
	Construct validity	2	Correlates with other level of consciousness measures acceptable to high (r > 0.60)
		1	Correlates with other level of consciousness measures are moderate (0.40 < r < 0.60)
		0	Correlates with other level of consciousness measures are low (r < 0.40)
Reliability	Homogeneity (internal consistency)	2	0.70 < alpha < 0.90
		1	Alpha > 0.90 or 0.60 < alpha < 0.70
		0	Alpha < 0.60
	Inter-rater reliability	2	Reliability coefficient > 0.80
		1	0.60 < reliability coefficient < 0.80
		0	Reliability coefficient < 0.60
	Intra-rater and/or test–retest reliability	2	Reliability coefficient > 0.80
		1	0.60 < reliability coefficient < 0.80
		0	Reliability coefficient < 0.60
	Responsiveness	2	Appropriate method of detecting clinically meaningful change over time described and clinically meaningful change over time detected and 15% or less of respondents achieved the lowest or highest possible score, respectively
		1	Appropriate method of detecting clinically meaningful change over time described and clinically meaningful change over time detected or 15% or less of respondents achieved the lowest or highest possible score, respectively
		0	Appropriate method of detecting clinically meaningful change over time not followed or clinically meaningful change over time not detected or more than 15% of the respondents achieved the lowest or highest possible score, respectively
	Origin of items	2	Items specifically developed for use with palliative care patients
		1	Items were modified for use with palliative care patients
		0	Items originated from a scale developed for another population
	Feasibility	2	Scale is short, manageable with instructions, scoring interpretation
		1	Scale is manageable (one format)
		0	Scale is more complex

The psychometric properties appraised include the reported validity, reliability and responsiveness of measures. In addition, the feasibility and origin (source) of tool items were also evaluated.

Validity of an instrument was defined as an assessment of the extent to which it measures what it purports to measure.[26] It is generally understood that there are four types of validity; we assessed three of these: (1) content validity: the degree to which the construct of interest is comprehensively represented by the measure items, assessed through the extent of involvement of the target population in item selection and the provision of a clear description of the concept that the instrument is intended to measure;[28] (2) construct validity: correlation of the level of consciousness scale with other instruments that are known to measure the same construct. Pearson’s or Spearman’s correlation coefficient of 0.6 or above was considered acceptable in this review;[25] (3) structural validity: assessed through the degree of variance explained by factor analysis. There is no agreed ‘gold standard’ for measuring level of consciousness in palliative care, so we did not assess the fourth type of validity, (4) criterion validity: the extent to which a proposed new measure correlates with another instrument generally accepted to accurately measure the construct of interest (‘gold standard’).[26]

Reliability refers to the overall consistency and reproducibility of a measure.[26] Four types of reliability estimates were included in our assessment criteria: (1) homogeneity (internal consistency), assessed through Cronbach’s alpha coefficient; (2) inter-rater reliability; (3) intra-rater reliability; and (4) test–retest reliability. The common statistical methods for evaluating the latter three properties are intraclass correlation coefficient (ICC) for continuous measures and Cohen’s kappa for nominal/ordinal measures.[28] We took values of less than 0.6, between 0.6 and 0.8, and greater than 0.8 as indicative of low, adequate and high reliability, respectively.

Responsiveness is the ability of an instrument to detect clinically meaningful changes over time in the construct measured. The most common approaches to assessing responsiveness are the correlations of change scores for an instrument over time with changes in other available variables, and the area under the receiver operator characteristic (ROC) curve (AUC).[26,28]

Feasibility is described as the user-friendliness of a measure in terms of administration and processing.[26] The burden on staff of collecting and processing data is an important parameter to consider when selecting a tool for use in clinical practice or for research purposes.[26]

Origin of items refers to whether the measure items were specifically developed for use with the target population, modified, or taken from a scale developed for another population.[25]

Evidence of psychometric performance was categorised according to the aforementioned criteria. For each property, measures were scored according to the following scheme: 2 if the property was evaluated and fully met criteria; 1 if criteria were partially met; and 0 when criteria were not met. If a property was not evaluated/not reported or the information provided was unclear, a rating was not given. Psychometric properties were independently evaluated by two raters (A.M.K. and E.M.), achieving a high initial agreement (κ = 0.91). Raters conferred over discrepancies until full consensus on ratings was reached.

Results

The database search yielded 13,827 results. After removing duplicates and initial screening of titles and abstracts, 491 potentially eligible articles remained, which were examined in full. Of these, 55 met criteria for inclusion. Further 10 eligible studies were identified through forward and backward citation searching, resulting in 65 included studies (see Figure 1). Only seven studies provided data on the psychometric performance of level of consciousness tools in the palliative population; 21 studies presented information on ad hoc measures (i.e. those developed specifically for the purposes of individual studies); and 37 reported using established scales, the majority of which had been validated in non-palliative care settings. Table 3 presents a summary of study and measure characteristics.

Figure 1.

PRISMA flow diagram of study selection process.[24]

CENTRAL: Cochrane Central Register of Controlled Trials; CINAHL: Cumulative Index to Nursing and Allied Health Literature; WoS: Web of Science.

Table 3.

Description of identified studies and measures.

AuthorYearCountry	Study aim	Study setting	Study population	Measure name/acronym	Purpose of measure	Subscales/number of items	Response options
Studies reporting ad hoc measures
Abernethy et al.[29] 2003Australia	To determine the efficacy of oral morphine for the management of refractory dyspnoea	Palliative, general, respiratory, cardiac medicine clinics	48 outpatients with refractory dyspnoea	–	To measure sedation depth as a side effect of morphine use	S: –I: 1	4-level scale (‘No’, ‘Mild’, ‘Moderate’, ‘Severe’ sedation)
Aretha et al.[30] 2009Greece	Evaluation of patient/family-controlled sedation with midazolam for intractable symptom control	Tertiary care university hospital	8 terminal cancer inpatients	–	Monitoring of patients after terminal sedation initiation	S: –I: 1	4-point scale (1 = ‘Awake’, 2 = ‘Arousable with voice’, 3 = ‘Arousable with light pain’, 4 = ‘Unarousable’)
Arevalo et al.[31] 2013Netherlands	To describe nurses’ experiences with the decision-making and performance of CPS	Home care organisations, palliative care units (based in nursing homes or inpatient hospices), hospitals	199 nurses reporting on their last patient receiving CPS	–(Scale included in the study questionnaire)	Monitoring of CPS	S: –I: 1	6-level scale (‘Drowsiness’, ‘Eyes closed, reaction to verbal stimuli’, ‘Eyes closed, reaction to physical stimuli’, ‘Eyes closed, no reaction to physical stimuli’, ‘Other’, ‘I don’t know’)
Barbato[32] 2001Australia	Exploration of the clinical application of BIS monitoring in palliative care	Hospice	12 unconscious palliative care inpatients	Consciousness Scale (modified GCS)[33]	Monitoring of consciousness level from the onset of unconsciousness and until death	S: 6 (breathing, movement, pulse volume, eyelash reflex, peripheries and response to name call)I: 1/subscale	4-point scale (1–4) for each subscale. Scores can be calculated per subscale and as a total score.
Baumann et al.[34] 1986USA	Evaluation of the safety and efficacy of patient-controlled analgesia in patients with unsuccessfully treated chronic pain secondary to cancer	Not specified	8 terminally ill cancer patient	–	To evaluate sedation for the assessment of individual analgesic response	S: –I: 1	5-point scale (1 = ‘Wide awake’, 2 = ‘Drowsy’, 3 = ‘Dozing intermittently’, 4 = ‘Mostly sleeping’, 5 = ‘Only awakens when aroused’)
Dean et al.[35] 2014UK	Description of PS decision-making practices in a UK hospice over the course of five years	Hospice	234 patient charts	Sedation scale (modified RASS)[19]	Accessing level of sedation to guide PS clinical decision-making and documentation	S: –I: 1	6-point scale (+2 = ‘Agitated/Distressed’, +1 = ‘Anxious/Restless’, 0 = ‘Alert, orientated, calm’, –1 = ‘Drowsy: Opening eyes and establishing eye contact for periods of 10 seconds or more, responds to commands’, –2 = ‘Moderate sedation: Rousable to voice or physical stimulation. Unable to communicate’, –3 = ‘Deep sedation: Unrousable’)
Fainsinger et al.[36] 2000South Africa, Israel, Spain	To provide a better understanding of the use of sedation for the management of uncontrolled symptoms in terminally ill patients	Hospices and hospital-based palliative care unit	387 palliative care patient	–	To assess level of consciousness after initiation of sedation for uncontrolled symptoms	S: –I: 1	3-level scale (‘Alert’, ‘Drowsy’, ‘Unresponsive’)
Hendriks et al.[37] 2014Netherlands	To investigate symptoms, treatment and quality of life in patients with end-stage dementia	Nursing homes	330 end-stage dementia patients (213 recruited on admission, 117 retrospectively)	–	To assess the level of consciousness that most frequently occurred during the last week of life	S: –I: 1	6-level scale (‘Awake and alert’, ‘Awake’, ‘Awake but drowsy looking’, ‘Falling asleep’, ‘Light sleep’, ‘Deep looking sleep’)
Jaspers et al.[38] 2012Germany	Description of the practice of PS in Germany	Palliative care units, inpatient hospices	1944 electronic patient records	–(Depth of PS item included in the standardised documentation system for palliative care patients)	To assess depth of PS	S: –I: 1	3-level scale (‘Somnolence’, ‘Stupor’, ‘Coma’)
Morita et al.[23] 1998Japan	To investigate the change in physical signs and medical interventions in the dying process	Palliative care unit	100 terminally ill cancer patient	Categorical scale (modified Riker Sedation–Agitation Scale[39])	To examine changes in the level of consciousness in the last four weeks of life	S: –I: 1	4-level scale (‘Awake: arousable, follows commands’, ‘Drowsy: difficult to arouse or unable to attend to conversation or commands’, ‘Very drowsy: awakens to noxious stimuli only’, ‘Coma: does not awaken to any stimuli’)
Morita et al.[40] 2000Japan	Identification of risk factors for the development and persistency of death rattle	Palliative care unit	245 terminally ill cancer patients (of whom 107 developed death rattle)	Categorical scale (modified Riker Sedation–Agitation Scale)[39]	To assess conscious level as a risk factor for the development/persistency of death rattle	Same as above	Same as above
Morita et al.[41] 2003Japan	To investigate the effects of partial opioid substitution and hydration on the occurrence of agitated delirium in the final stage of cancer	Palliative care unit	284 terminally ill cancer inpatient charts	Fainsinger’s consciousness scale (ad hoc scale described in Fainsinger et al.[36])	Evaluation of consciousness level as part of the assessment of the degree of cognitive impairment	S: –I: 1	3-level scale (‘Alert’, ‘Drowsy’, ‘Unresponsive’)
Morita et al.[42] 2003Japan	To establish the communication capacity level and identify factors contributing to communication capacity impairment and agitated delirium in cancer patients in their final week of life	Palliative care unit	284 terminally ill cancer inpatient charts	Fainsinger’s consciousness scale (ad hoc scale described in Fainsinger et al.[36])	Evaluation of consciousness level in the last week of life	Same as above	Same as above
Papavasiliou et al.[43] 2014Belgium	To compare physician-reported practices on CDSUD between general practitioner and medical specialists	Not specified	561 cases of CDSUD reported by physicians	–(Level of unconsciousness item included in questionnaire on end-of-life practices)	Level of unconsciousness (comatose) used to assess the degree of patients’ awareness during the practice of CDSUD	S: –I: 1	11-point scale(0 = ‘Symptom not present’ to 10 = ‘Worst possible symptom’)
Pasman et al.[44] 2005Netherlands	To study the level and course of discomfort, and factors that are associated with discomfort in patients with dementia for whom artificial nutrition and hydration are forgone	Nursing homes	178 patients with severe dementia	–	To assess the level of consciousness as a determinant of discomfort	S: –I: 1	6-point scale (response options not described)
Portenoy et al.[45] 2006USA	Exploration of the relationship between opioid use and survival at the end of life	Hospices	725 palliative care inpatients	–	Level of consciousness at the time of last opioid dose change assessed for its association with length of survival	S: –I: 1	4-level scale (‘Full level of consciousness’, ‘Drowsy’, ‘Confused’, ‘Unable to respond’)
Rys et al.[46] 2014Belgium	Investigation of the practice of CSD in nursing homes	Nursing homes	249 nurse reports of their most recent patient treated with CSD	–(Depth of sedation scale included in the study questionnaire)	To assess depth of sedation reached after the administration of CSD	S: –I: 1	5-level scale(‘Drowsy’, ‘Eyes closed, response to voice’, ‘Eyes closed, response to painful stimuli’, ‘Eyes closed, no reaction to any stimulus’, ‘Other’)
Swart et al.[47] 2012Netherlands	Description of the practice of CPS until death after the introduction of a national palliative guideline	Not specified	370 physicians providing information about their last patient who received CPS until death	–(Depth of continuous sedation item included in the study questionnaire)	To assess depth of continuous sedation reached after the administration of CPS until death	S: –I: 1	5-point scale (‘Drowsy’, ‘Eyes closed, responding promptly to verbal command’, ‘Eyes closed, arousable only by physical stimuli’, ‘Eyes closed, not arousable by physical stimuli’, ‘Other’)
Van Deijck et al.[48] 2010Netherlands	Investigation of the practice of CPS in elderly patients	Nursing homes	316 nursing home physicians reporting on their last case of CPS	–(Level of consciousness item included in the study questionnaire)	Evaluation of level of consciousness at adequate symptom relief after the administration of CPS	S: –I: 1	6-level scale(‘Alert and orientated’, ‘Drowsy’, ‘Eyes closed, following directives’, ‘Eyes closed, responding to physical stimuli’, ‘Eyes closed, not responding to physical stimuli’, ‘Disturbed brainstem function’)
Van Deijck et al.[49] 2015Netherlands	To explore the characteristics of patients with existential suffering treated with CPS and the degree to which preconditions for administering CPS are fulfilled	Nursing homes	314 cases of patients who received CPS described by nursing home physicians	–(Level of consciousness item included in the study questionnaire)	Evaluation of level of consciousness at adequate symptom relief after the administration of CPS	Same as above	Same as above
Van Der Steen et al.[50] 2009Netherlands	To compare discomfort in dementia patients dying from pneumonia with patients dying after intake problems, and to assess associations with treatment	Nursing homes	725 end-stage dementia patients	–	To explore the association between level of consciousness and discomfort	S: –I: 1	6-level scale (‘Awake and alert’, ‘Awake’, ‘Awake but drowsy looking’, ‘Falling asleep’, ‘Light sleep’, ‘Deep looking sleep’)
Studies reporting established measures
Agar et al.[51] 2017Australia	To determine the efficacy of risperidone or haloperidol relative to placebo for delirium symptoms among palliative care patients	Hospice and hospital palliative care inpatient services	247 palliative care inpatients with various diagnoses; predominantly cancer	RASS[19]	To measure sedation as an adverse effect of risperidone/haloperidol use	S: –I: 1	10-point scale (+4 = ‘Combative’, +3 = ‘Very agitation’, +2 = ‘Agitated’, +1 = ‘Restless’, 0 = ‘Alert and Calm’, –1 = ‘Drowsy’, –2 = ‘Light sedation’, –3 = ‘Moderate sedation’, –4 = ‘Deep sedation’, –5 = ‘Unarousable’)
Alonso-Babarro et al.[4] 2010Spain	Assessment of the incidence and efficacy of PS for patients who died at home	Home	245 terminally ill cancer patient records	RSS[52]	To monitor level of sedation after administration of PS	S: –I: 1	6-point scale (1 = ‘Anxious and agitated or restless or both’, 2 = ‘Co-operative, orientated and tranquil, 3 = ‘Responds to commands only’, 4 = ‘Brisk response to a light glabellar tap or loud auditory stimulus, 5 = ‘Sluggish response’, 6 = ‘No response’)
Barbato et al.[53] 2017Australia	To determine the validity of the BIS monitor and two observational scales	Palliative care unit	40 unresponsive palliative care inpatients	RASS[19]	To assess level of sedation for the exploration of the association with BIS values	Same as above	Same as above
Barbato et al.[54] 2018Australia	To examine the effectiveness of breakthrough medication in unresponsive patients and the perception of patient comfort made by nurses and family	Palliative care unit	40 unresponsive palliative care inpatients	RASS[19]	To measure level of sedation for the assessment of the effect of breakthrough opioid/benzodiazepine use	Same as above	Same as above
Benitez-Rosario et al.[55] 2012Spain	To assess the feasibility of a quality care project in PS	Hospital-based palliative care service	204 patient charts	RASS[19]	To assess the level of deep continuous sedation with the aim to reach a predetermined level (–5 RASS for patients with continuous dyspnoea at rest; –4 RASS for delirium or other reasons)	Same as above	Same as above
Boyd and Kelly[56] 1997UK	Evaluation of the effectiveness of oral morphine for the symptomatic treatment of dyspnoea in patients with advanced cancer	Home, hospice	15 advanced cancer patients with dyspnoea	VAS	To measure sedation as a side effect of oral morphine	S: –I: 1	100 mm line anchored by two verbal descriptors; 0: ‘Fully awake’, 100: ‘Asleep’
Campbell et al.[57] 2009USA	To investigate the self-reporting of dyspnoea at the very end of life	Palliative care unit	89 palliative care inpatients at the risk of experiencing dyspnoea	RLS85[58]	To assess consciousness as patient characteristic for the exploration of the association with the ability to self-report dyspnoea symptoms	S: –I: 1	8-point scale (1 = ‘Alert; No delay in response’, 2 = ‘Drowsy or confused; Responsive to light stimulation’, 3 = ‘Very drowsy or confused; Responsive to strong stimulation’ 4 = ‘Unconscious; Localizes but does not ward off pain’, 5 = ‘Unconscious; Withdrawing movement on pain stimulation’, ‘6 = Unconscious; Stereotype flexion movements on pain stimulation’, 7 = ‘Unconscious; Stereotype extension movements on pain stimulation’, 8 = ‘Unconscious; No response to pain stimulation’)
Campbell et al.[59] 2010USA	To establish the reliability and construct validity of a revised RDOS	Palliative care unit	89 palliative care inpatients at the risk of experiencing dyspnoea	RLS85[58]	To assess consciousness for ascertaining the construct validity of RDOS	Same as above	Same as above
Campbell et al.[60] 2013USA	To determine the effect of oxygen administration at the very end of life	Hospice, hospital-based palliative care service	32 hospice and hospital inpatients at the very end of life	RLS85[58]	To measure consciousness for the correlation with respiratory distress and nearness to death	Same as above	Same as above
Campbell et al.[61] 2018USA	Determination of the trajectory of dyspnoea and respiratory distress	Hospice	91 home-based palliative care patients	RLS85[58]	To measure consciousness for the correlation with respiratory distress and nearness to death	Same as above	Same as above
Caraceni et al.[62] 2018Italy	Comparison of PS practices in home care and hospice settings	Home-based palliative care services, hospices	531 terminal cancer patients receiving PS	MWSS[63]	Level of consciousness assessed as part of the PS monitoring process	S: –I: 1	5-point scale (1 = ‘Fully awake and oriented’, 2 = ‘Drowsy but rousable’, 3 = ‘Eyes closed but rousable to command’, 4 = ‘Eyes closed but rousable to mild physical stimulation (earlobe tug)’, 5 = ‘Eyes closed but unrousable to mild physical stimulation’)
De la Cruz et al.[64] 2015USA	To describe the prevalence and severity of symptoms, including delirium, in the final week of life and evaluate the usefulness of the Nursing Delirium Screening Scale	Hospice	78 terminally ill cancer patients	RASS[19]	To measure sedation or agitation as the predominant features of delirium	Same as above	Same as above
Franken et al.[65] 2018Netherlands	To evaluate the variability in response to midazolam and to find clinically significant covariates that predict pharmacodynamic response	Palliative care centre	43 terminally ill inpatients receiving midazolam	RSS[52]	To measure the effect of midazolam on patients’ sedation level	Same as above	Same as above
Goncalves et al.[66] 2012Portugal	Description of the sedation practice of Portuguese palliative care teams	Palliative care inpatient, home care, hospital support care services	181 palliative care patients (of whom 27 received sedation)	CSPC[18]	To assess the deepest consciousness level reached after the administration of sedation	S: –I: 1	6-point scale (1 = ‘Awake’, 2 = ‘Awakens when called by name and stays awake during discussion’, 3 = ‘Awakens but falls asleep during discussion’, 4 = ‘Reacts with movement/brief eye opening, but without eye contact, when called by name’, 5 = ‘Reacts to trapezius muscle pinching’, 6 =’Does not react’)
Goncalves et al.[67] 2013Portugal	To examine the activity of Portuguese palliative care teams	Inpatient, home care and hospital palliative care support care services	164 palliative care patients	CSPC[18]	Evaluation of consciousness level as a patient characteristic	Same as above	Same as above
Ferraz Goncalves et al.[68] 2016Portugal	Comparison of haloperidol alone and in combination with midazolam for the treatment of acute agitation in palliative care	Palliative care unit	79 palliative care inpatients	CSPC[18]	To assess level of consciousness when control of agitation is reached	Same as above	Same as above
Hsu et al.[69] 2013Taiwan	To investigate the characteristics and outcomes of non-cancer palliative care patients in an acute general care setting	Acute general medicine ward	258 inpatients (of whom 193 did not meet criteria for cancer palliative care)	GCS[33]	To measure GCS score as a clinical characteristic for the comparison between cancer and non-cancer patients	S: 3 (motor response, verbal response, eye opening)I: 1/subscale	Eye opening: 4-point scale (1–4), Motor response: 6-point scale (1–6), Verbal response: 5-point scale (1–5)
Hui et al.[21] 2014USA, Brazil	To examine the frequency and onset of bedside physical signs and their diagnostic performance for impending death	Acute palliative care units	357 advanced cancer inpatients	RASS[19]	Decreased level of consciousness (RASS ⩽ –2) assessed as a clinical sign of impending death	Same as above	Same as above
Hui et al.[70] 2017USA	To compare the effect of lorazepam versus placebo as adjuvant to haloperidol for persistent agitation	Acute palliative care unit	93 advanced cancer inpatients with agitated delirium	RASS[19]	To measure sedation and agitation for the evaluation of the effect of pharmacological interventions for the treatment of agitation	Same as above	Same as above
Hwang et al.[22] 2013South Korea	To determine the events that herald the onset of dying process and evaluate their predictive value for death within 48 hours	Palliative care unit	181 terminal cancer inpatients	AVPU[71]	To measure conscious level as clinical sign of impending death	S: –I: 1	4-level scale (A = ‘Eyes opened spontaneously, orientated speech, obeys commands, V = ‘Any verbal, motor, or eye response to verbal stimulus’, P = ‘Any verbal, motor, or eye response to painful stimulus’, U = ‘Unresponsive to any stimulus’)
Imai et al.[72] 2018Japan	To investigate the effect of two types of PS therapy: proportional and deep sedation	Palliative care unit	50 cancer inpatients	Modified RASS[73]	To define deep sedation (RASS ⩾ –4) and the absence of agitation (RASS ⩽ 0)	S: –I: 1	10-point scale (+4 = ‘Combative’ to −5 = ‘Unarousable’)Modifications to RASS:[19] Removal of reference to assisted ventilation from definition of agitation levelScore ‘+1’ can be present in patients who are not fully alert
Klepstad et al.[74] 2002Norway	Investigation of the relationship between patient self-reports of CF and sedation with objective assessments of CF and sedation	Hospital-based palliative care unit	29 cancer inpatients	OAA/S[75]	To objectively assess sedation and compare scores with patient self-reports	S: 4 (responsiveness, speech, facial expression, eyes)I: 1/subscale	Responsiveness: 5-point scale (1–5), Speech: 4-point scale (2–5), Facial expression: 3-point scale (3–5), Eyes: 3-point scale (3–5)
Kohara et al.[5] 2005Japan	Investigation of the influence of sedative drugs on consciousness	Hospital-based palliative care unit	124 terminally ill cancer inpatients (of whom 63 received sedation)	Communication Capacity Scale–Item 1 (Conscious level)[76]	To compare level of consciousness between sedated and unsedated patients	S: –I: 1(for item 1)	6-point scale (0 = ‘Awake with no drowsiness’ to 5 = ‘Cannot remain awake and cannot be awakened by physical stimuli’)
Maltoni et al.[77] 2012Italy	Evaluation of the practice of PS in two Italian hospices	Hospice	327 inpatients (of whom 72 received PS)	RASS[19]	RASS scores used for monitoring PS (negativisation of scores proxy indicator of the efficacy of PS)	Same as above	Same as above
Masman et al.[78] 2016Netherlands	To determine the feasibility and validity of BIS monitoring in terminally ill patients	Palliative care centre	58 terminally ill inpatients	RSS[52]	To assess level of sedation and evaluate the correlation between Ramsay scores and BIS values	Same as above	Same as above
Matsunuma et al.[79] 2016Japan	Evaluation of the signs, symptoms and treatments of patients with ILD before death	Community hospital	82 end-stage ILD and lung cancer inpatient records	JCS[80]	To determine the frequency of loss of consciousness (defined as more than 1 point on JCS) before death and examine its causes	S: –I: 1	10-point scale (One level (0) for ‘fully conscious’, 3 levels (1–3) for the patient who is ‘awake without any stimuli’, 3 levels (10–30) for the patient who ‘can be aroused after stimulation’, 3 levels (100–300) for the patient who ‘cannot be aroused with any forceful mechanical stimuli’)
McMillan and Tittle[81] 1995USA	To describe cancer and palliative care patients’ pain, pain-related side effects and the nurses’ assessment and responses to these	Cancer centre, hospice home care service	44 patients treated for pain	Sedation Item of the Pain Flow Sheet[82]	To evaluate level of sedation as a opioid-induced side effect	S: –I: 1(for sedation item)	5-point scale (0 = ‘Fully alert’, 1 = ‘Relaxed, awake’, 2 = ‘Drowsy, dozing’, 3 = ‘Arousable sleep’, 4 = ‘Comatose’)
Mercadante et al.[83] 2009Italy	Assessment of the need and the effectiveness of sedation for intractable symptoms, and the thoughts of relatives regarding sedation	Acute pain relief and palliative care unit	77 terminally ill cancer patient (of whom 42 received sedation)	Communication Capacity Scale–Item 1 (Conscious level)[76]	To assess patients’ level of sedation after the initiation of PS	Same as above	Same as above
Mercadante et al.[84] 2017Italy	To assess the attitudes of palliative care clinicians regarding PS at home	Home	150 physicians involved in end of life care decisions	RASS[19] RSS[52] Rudkin Sedation Scale[85]	Monitoring of PS	RASS:[19] Same as aboveRSS:[52] Same as aboveRudkin Sedation Scale:[85] S: –I: 1	RASS:[19] Same as aboveRSS:[52] Same as aboveRudkin Sedation Scale:[85] 5-point scale (1 = ‘Fully awake’, 2 = ‘Drowsy’, 3 = ‘Eyes closed but rousable to command’, 4 = ‘Eyes closed but rousable to mild physical stimulation’, 5 = ‘Eyes closed and unrousable to mild physical stimulation’)
Mercadante et al.[86] 2018Italy	To assess the efficacy of hyoscine butylbromide for the management of death rattle	Hospices	132 cancer inpatients with reduced level of consciousness	RASS-PAL[87]	Identification of patients with reduced level of consciousness (RASS-PAL ⩽ –3)	S: –I: 1	10-point scale (+4 = ‘Combative’ to −5 = ‘Unarousable’)Modifications to RASS:[19] Descriptors related to ‘pulling tubes’/‘fighting the ventilator’ modified‘Any movement’ refers to eye and body‘Physical stimulation’ changed to ‘gentle physical stimulation’Clarification on how to score a patient with a mixed-type delirium
Monreal-Carrillo et al.[88] 2017Mexico	Characterisation of the level of consciousness of patients undergoing PS using BIS monitoring	Palliative care unit	20 advanced cancer inpatients receiving PS	RSS[52]	Assessment of sedation level after initiation of PS	Same as above	Same as above
Morita et al.[76] 2001Japan	Development and validation of the Communication Capacity Scale and the Agitation Distress scale	Palliative care unit based in a cancer institute	30 terminally ill cancer inpatients with delirium	Communication Capacity Scale–Item 1 (Conscious level)[76] Sedation Scale (modified Riker Sedation–Agitation Scale)[39]	To test the association between Communication Capacity scores and Sedation Scale scores	-Communication Capacity Scale–Item 1 (Conscious level):[76] Same as above-Sedation Scale:S: –I: 1	Communication Capacity Scale–Item 1 (Conscious level):[76] Same as aboveSedation Scale: 4-point scale (0 = ‘Calm and cooperative’, 1 = ‘Over-sedated’, 2 = ‘Very sedated’, 3 = ‘Unrousable’)
Palacio et al.[89] 2018Colombia	Description of the practice of PS	Specialised palliative care unit based in a cancer institute	66 advanced cancer inpatients undergoing PS	RSS[52]	Assessment of sedation level after initiation of PS	Same as above	Same as above
Porzio et al.[90] 2010Italy	Evaluation of the feasibility and efficacy of PS at home	Home care service	16 terminally ill cancer home patient charts	RSS[52]	To monitor the level of sedation after the administration of PS with the aim to reach deep, continuous sedation (RSS ⩾ 5)	Same as above	Same as above
Pype et al.[10] 2018Belgium	To explore the practice of suboptimal PS in primary care	Home	Seven palliative care home teams and 7 general practitioners reporting on 27 cases of PS	RASS[19]	To measure depth of sedation throughout the procedure of PS	Same as above	Same as above
Schmitz et al.[91] 2016Germany	To investigate the effectiveness of intravenous opioid PCT in reducing breathlessness in patients with advanced malignant disease	Palliative care centre	18 patients with moderate or severe breathlessness	RASS[19]	To monitor changes in sedation and agitation levels after PCT onset	Same as above	Same as above
Van Deijck et al.[92] 2016Netherlands	To explore which patient-related factors at admission are associated with receiving CPS in the terminal phase of life	Hospices, nursing home-based palliative care units	467 palliative care inpatients (of whom 130 received CPS)	GCS[33]	To evaluate the level of consciousness on admission as a patient-related characteristic and examine its association with CPS	Same as above	Same as above
Studies reporting measure psychometric properties
Arevalo et al.[93] 2012Netherlands	To study the reliability and validity of observer-based sedation scales in PS	Hospices, nursing home	54 inpatients receiving PS	MSAT[94] (Dutch version)RASS[19] (Dutch version)VICS[95] (Dutch version)KNMG[96]	To assess the level of consciousness before and during the course of PS	MSAT:[94] S: 3 (motor activity, arousal, quality of sedation therapy)I: 1/subscaleRASS:[19] Same as aboveVICS:[95] S: 2 (interaction, calmness)I: 5/subscaleKNMG:[96] S: –I: 1	MSAT:[94] Motor activity: 4 levels (1–4), Arousal: 6 levels (1–6), Quality of sedation therapy: 3 levels (‘Adequate’, ‘Oversedated’, ‘Undersedated’)RASS:[19] Same as aboveVICS:[95] Interaction: 6-point Likert-type scale per item (1 = ‘Strongly disagree’ to 6 = ‘Strongly agree’; reverse scoring for last item)Calmness: 6-point Likert-type scale per item (1 = ‘Strongly disagree’ to 6 = ‘Strongly agree’; reverse scoring for first item)KNMG:[96] 6-point scale (Level 1: (1) ’Awake and oriented’, (2) ’Drowsy’, (3) ‘Eyes closed, responds promptly to verbal commands’, (4) ‘Eyes closed, arousable only by physical stimuli’, Level 2: ‘Eyes closed, not arousable by physical stimuli’, Level 3: ‘Basic brain functions affected’)
Benitez-Rosario et al.[73] 2013Spain	To test the appropriateness and reliability of the RASS in Spanish patients with advanced cancer	Palliative care unit	156 advanced cancer inpatients	Modified RASS[73]	To monitor sedation and agitation	Same as above	Same as above
Bush et al.[87] 2014Canada	Exploration of the validity and feasibility of a version of the RASS modified for palliative care populations	Acute palliative care unit	10 inpatients with agitated delirium or receiving PS	RASS-PAL[87]	To assess the level of sedation and agitation	Same as above	Same as above
Claessens et al.[97] 2011Belgium	Description of the characteristics of palliative care patients receiving sedation for the management of refractory symptoms	Palliative care units	266 terminally ill cancer inpatients (of whom 20 received PS)	GCS[33] (Dutch version)	Evaluation of level of consciousness at the start and during PS	S: 3 (motor response, verbal response, eye opening)I: 1/subscale	Eye opening: 4-point scale (1–4), Motor response: 6-point scale (1–6), Verbal response: 5-point scale (1–5)
Claessens et al.[1] 2012Belgium	To examine the impact of PS on the level of consciousness of terminally ill patients	Palliative care units	266 terminally ill cancer inpatients (of whom 20 received PS)	GCS[33] (Dutch version)	Evaluation of level of consciousness with the aim to assess the effect of PS	Same as above	Same as above
Claessens et al.[98] 2014Belgium	Description of the effect of PS on oral and/or artificial food and fluid intake in terminally ill patients	Palliative care units	266 terminally ill cancer inpatients (of whom 20 received PS)	GCS[33] (Dutch version)	To evaluate patients’ level of consciousness at admission	Same as above	Same as above
Goncalves et al.[18] 2008Portugal	Validation of a consciousness scale for palliative care	Palliative care unit	38 advanced cancer inpatients	CSPC[18]	To assess level of consciousness	S: –I: 1	6-point scale (1 = ‘Awake’, 2 = ‘Awakens when called by name and stays awake during discussion’, 3 = ‘Awakens but falls asleep during discussion’, 4 = ‘Reacts with movement/brief eye opening, but without eye contact, when called by name’, 5 = ‘Reacts to trapezius muscle pinching’, 6 =’Does not react’)

CPS: continuous palliative sedation; BIS: bispectral index; GCS: Glasgow Coma Scale; PS: palliative sedation; RASS: Richmond Agitation–Sedation Scale; CDSUD: continuous deep sedation until death; CSD: continuous sedation until death; RSS: Ramsay Sedation Scale; VAS: visual analogue scale; RLS85: Reaction Level Scale 85; RDOS: Respiratory Distress Observation Scale; MWSS: Modified Wilson Sedation Scale; CSPC: Consciousness Scale for Palliative Care; AVPU: Alert/Verbal/Painful/Unresponsive Scale; CF: cognitive function; OAA/S: Observer’s Assessment of Alertness/Sedation; ILD: interstitial lung disease; JCS: Japan Coma Scale; RASS-PAL: Richmond Agitation–Sedation Scale–Palliative version; PCT: patient-controlled therapy; MSAT: Minnesota Sedation Assessment Tool; VICS: Vancouver Interaction and Calmness Scale; KNMG: Sedation score proposed in the Guideline for Palliative Sedation of the Royal Dutch Medical Association.

Description of included studies

Morita et al.[41,42] published two articles in which separate analyses of data collected from a single study were performed. Similarly, Barbato et al.,[53,54] Campbell et al.,[57,59] Claessens et al.[1,97,98] and Van Deijck et al.[48,49] reported distinct findings from one study in two or more papers. Each of these papers described discrete study aims and outcomes, so we defined them as separate studies. A large number of studies reporting on level of consciousness measures have been published recently, with 26 of the 65 (40%) included studies published after 2013.

Most included studies were patient-based (n = 58), with recruitment and data collection conducted prospectively (n = 49). In eight studies some or all relevant data were obtained retrospectively from patients’ medical records,[4,35,38,41,42,55,79,90] while in one study patients were recruited both prospectively (on admission) and retrospectively (after death).[37] Another study reported mixed methods for data collection, a prospective quantitative survey and semi-structured interviews with general practitioners involved in the practice of palliative sedation.[10] Six studies used questionnaires as a means of data collection.[31,43,46-49] In these, researchers asked clinicians (physicians (n = 4)[43,47-49] or nurses (n = 2)[31,46]), to provide information about patients under their care who had received sedative medication.

Studies were mainly conducted in a single setting (n = 36); principally hospices, palliative care units or hospitals. Nine studies involved home care patients,[4,10,31,56,62,67,81,84,90] and an equal number included nursing home participants.[31,37,44,46,48-50,92,93] One study included patients recruited from a cancer centre.[81]

Sample size varied considerably (median: 132 participants, interquartile range (IQR): 44–266). The most prevalent diagnosis among study participants was cancer (n = 29). Other reported diagnoses included dementia (n = 3)[37,44,50] and interstitial lung disease (n = 1).[79] A total of 32 studies reported mixed diagnoses or did not provide this information. Patients in almost all studies were at an advanced or an end stage of disease.

Reflecting the wide diversity of study aims, level of consciousness tools in each study were employed to serve a number of distinct purposes. The most frequently reported were: to assess/monitor sedation depth after palliative sedation initiation (n = 29), to assess effects or side effects of opioid use (n = 7),[29,34,45,54,56,81,91] to evaluate signs/symptoms of impending death (n = 8)[21-23,37,42,60,61,79] and to examine associations between level of consciousness and discomfort or other symptoms (n = 6).[40,44,50,60,61,64] It is noteworthy that only four studies sought to validate level of consciousness instruments in the palliative care setting.[18,73,87,93] Of these, only one aimed to develop a new tool.[18]

Description of identified measures

A total of 35 different measures assessing level of consciousness were described in the articles included in this review. Only eight were measures for which evidence of psychometric quality in the palliative setting was available. Fifteen were established instruments or single items taken from compound scales validated as a whole, and 17 were tools constructed for individual study purposes (ad hoc measures). Information on psychometric performance in palliative care was provided for five of the 15 established measures, therefore, there is an overlap between the first 2 described categories (see Figure 2). Across all categories, the tool most frequently employed was the original Richmond Agitation–Sedation Scale (RASS) or its modified versions (n = 17).[10,21,35,51,53-55,64,70,72,73,77,84,86,87,91,93]

Figure 2.

Number of identified studies and measures by instrument category.

Three of the ad hoc measures were modified versions of the existing tools: the Glasgow Coma Scale (GCS),[32,33] RASS[19,35] and Riker Sedation–Agitation Scale.[23,39] All other ad hoc measures comprised unique tools. None of the reported ad hoc measures had been formally validated before use.

The established measures most commonly used were the RASS[19] (n = 11)[10,21,51,53-55,64,70,77,84,91] and Ramsay Sedation Scale (RSS;[52] n = 7).[4,65,78,84,88-90] Most established measures had been developed and validated for use in settings other than palliative care; mainly the intensive care unit. The studies with palliative care patients in which these measures were used provided no information on their validity or reliability.

Two of the existing measures used for the evaluation of level of consciousness consisted of items extracted from multi-item tools developed to assess constructs other than level of consciousness (i.e. the conscious level item of the Communication Capacity Scale (CCS)[5,76,83] and the sedation item of the Pain Flow Sheet[81,82]). These tools had been evaluated psychometrically in palliative care settings, but validity and reliability have only ever been established for each measure as a whole, not for the individual items measuring levels of consciousness.

Almost all of the described measures consisted of one item with a range of mutually exclusive scoring options (n = 27), usually involving observation of spontaneous activities, such as eye opening, or responses to auditory and/or tactile stimuli performed in a logical progression. The majority of these tools (n = 23) evaluated a single construct: consciousness in terms of arousal, while the remaining measures (n = 4) incorporated the assessment of agitation into single scales for consciousness/sedation.

Evidence of psychometric performance was provided for: the Minnesota Sedation Assessment Tool (MSAT),[93,94] RASS,[19,93] Vancouver Interaction and Calmness Scale (VICS),[93,95] Sedation score proposed in the Guideline for Palliative Sedation of the Royal Dutch Medical Association (KNMG),[93,96] Modified RASS,[19,73] Richmond Agitation–Sedation Scale–Palliative version (RASS-PAL),[19,87] GCS[1,33,97,98] and Consciousness Scale for Palliative Care (CSPC).[18]

Dutch versions of original English language measures were created by researchers for the MSAT,[93,94] RASS,[19,93] VICS[93,95] and GCS.[1,33,97,98] The RASS modified by Benitez-Rosario et al.[73] was translated and further adjusted for use with Spanish palliative care patients. Modifications to the original RASS[19] included the removal of descriptors relating to the mechanical ventilation of patients and a clarification to the scoring instructions addressing the possibility that restless behaviour may be present in patients who are not fully alert. Similarly, Bush et al.[87] reported performing minor changes to the RASS[19] when testing its psychometric performance in the palliative care setting. The CSPC was validated in its source language (Portuguese) and, subsequently, translated by its authors into English.[18]

Appraisal of psychometric performance

Evidence regarding structural validity, test–retest and intra-rater reliability was not provided for any of the evaluated measures, so we do not present findings relating to these properties. The CSPC[18] and a modified version of the RASS[73] achieved the highest ratings in our quality appraisal, but our evaluation was based on evidence obtained from just one study for each measure. Table 4 provides a summary of the quality appraisal process for each instrument.

Table 4.

Appraisal of psychometric performance of observational level of consciousness measures.

Measure and studies	Number of participants	Content validity	Criterion validity	Structural validity	Construct validity	Homogeneity (internal consistency)	Inter-rater reliability	Intra-rater and/or test–retest reliability	Responsiveness	Origin of items	Feasibility
MSAT (Dutch version)Arevalo et al.[93]	N = 54No correction for multiple testing	Description of construct provided. No involvement of target population in item selection	Gold standard not available	NE/NR	Assessed per subscaleMSATa: Spearman’s correlation coefficient ranged from 0.48 to 0.83 (mostly above 0.60)MSATm: Spearman’s correlation coefficient ranged from 0.42 to 0.61	NE/NR	Assessed per subscaleMSATa: ICC ranged from 0.59 (95% CI: 0.45 to 0.70) to 0.64 (95% CI: 0.46 to 0.77) depending on time difference between paired assessmentsMSATm: ICC ranged from 0.01 (95% CI: –0.25 to 0.25) to 0.11 (95% CI: –0.09 to 0.29) depending on time difference between paired assessmentsMSATq: Cohen’s kappa ranged from 0.436 to 0.545 depending on time difference between paired assessments	NE/NR	NE/NR	Items originated from a scale developed for another population	Evaluated as clear and easy to use (when compared with the Dutch versions of RASS and VICS)
Rating	1	1	–	–	MSATa: 2MSATm: 1	–	MSATa: 1MSATm: 0MSATq: 0	–	–	0	2
VICS (Dutch version)Arevalo et al.[93]	N = 54No correction for multiple testing	Description of construct provided. No involvement of target population in item selection	Gold standard not available	NE/NR	Assessed per subscaleVICSi: Spearman’s correlation coefficient ranged from 0.31 to 0.72 (mostly above 0.40)VICSc: Spearman’s correlation coefficient ranged from 0.31 to 0.57 (mostly above 0.40)	NE/NR	Assessed per subscaleVICSi: ICC ranged from 0.77 (95% CI: 0.64 to 0.86) to 0.85 (95% CI: 0.73 to 0.92) depending on time difference between paired assessmentsVICSc: ICC ranged from 0.12 (95% CI: –0.18 to 0.40) to 0.34 (95% CI: 0.1 to 0.52) depending on time difference between paired assessments	NE/NR	NE/NR	Items originated from a scale developed for another population	Evaluated as the least clear and easy to use (when compared with the Dutch versions of RASS and MSAT)
Rating	1	1	–	–	VICSi: 1VICSc: 1	–	VICSi: 2VICSc: 0	–	–	0	1
RASS (Dutch version)Arevalo et al.[93]	N = 54No correction for multiple testing	Description of construct provided. No involvement of target population in item selection	Gold standard not available	NE/NR	Spearman’s correlation coefficient ranged from 0.57 to 0.84	NE/NR	ICC ranged from 0.71 (95% CI: 0.60 to 0.79) to 0.73 (95% CI: 0.58 to 0.83) depending on time difference between paired assessments	NE/NR	NE/NR	Items originated from a scale developed for another population	Evaluated as the least time-consuming, clearest and easiest to use (when compared with Dutch MSAT and VICS)
Rating	1	1	–	–	2	–	1	–	–	0	2
KNMGArevalo et al.[93]	N = 54No correction for multiple testing	Description of construct provided. No involvement of target population in item selection	Gold standard not available	NE/NR	Spearman’s correlation coefficient ranged from 0.44 to 0.84	NE/NR	ICC ranged from 0.66 (95% CI: 0.54 to 0.76) to 0.71 (95% CI: 0.55 to 0.82) depending on time difference between paired assessments	NE/NR	NE/NR	Measure specifically developed for use with palliative care patients	NE/NR
Rating	1	1	–	–	2	–	1	–	–	2	–
Modified RASSBenitez-Rosario et al.[73]	N = 156	Description of construct provided. Target population involved in item modification	Gold standard not available	NE/NR	Spearman’s correlation coefficient ranged from 0.81 to 0.89 (p < 0.001)	NE/NR	Weighted Cohen’s kappa ranged from 0.85 (95% CI: 0.85 to 0.92) to 0.95 (95% CI: 0.91 to 0.98)	NE/NR	Not adequate information provided	Items modified for use with palliative care patients	Reported as a very useful, manageable tool that could facilitate fluid communication among the palliative care team
Rating	2	2	–	–	2	–	2	–	–	1	2
RASS-PALBush et al.[87]	N = 10No correction for multiple testing	Description of construct provided. Target population involved in item modification	Gold standard not available	NE/NR	NE/NR	NE/NR	ICC ranged from 0.84 (95% CI: 0.56 to 0.95) to 0.98 (95% CI: 0.95 to 1.00)	NE/NR	NE/NR	Items modified for use with palliative care patients	Evaluated as easy to use, simple and brief
Rating	0	2	–	–	–	–	2	–	–	1	2
GCS (Dutch version)Claessens et al.[1,97,98]	N = 266	Description of construct provided. No involvement of target population in item selection	Gold standard not available	NE/NR	NE/NR	NE/NR	ICC = 0.807 (CI = 0.671–0.891; p = 0.000)	NE/NR	NE/NR	Items originated from a scale developed for another population	NE/NR
Rating	2	1	–	–	–	–	2	–	–	0	–
CSPCGoncalves et al.[18]	N = 38No correction for multiple testing	Description of construct provided. Target population involved in item selection	Gold standard not available	NE/NR	Spearman’s correlation coefficient ranged from 0.82 to 0.95 (p < 0.001)	Cronbach’s α = 0.99	ICC = 0.99 (p < 0.001)	NE/NR	NE/NR	Scale specifically developed for use with palliative care patients	Evaluated as easy to use and useful in clinical practice
Rating	0	2	–	–	2	2	2	–	–	2	2

MSAT: Minnesota Sedation Assessment Tool; NE: not evaluated; NR: not reported; MSATa: Minnesota Sedation Assessment Tool arousal subscale; ICC: intraclass correlation coefficient; CI: confidence interval; MSATm: Minnesota Sedation Assessment Tool motor activity subscale; MSATq: Minnesota Sedation Assessment Tool quality of sedation subscale; RASS: Richmond Agitation–Sedation Scale; VICS: Vancouver Interaction and Calmness Scale; VICSi: Vancouver Interaction and Calmness Scale interaction subscale; VICSc: Vancouver Interaction and Calmness Scale calmness subscale; KNMG: Sedation score proposed in the Guideline for Palliative Sedation of the Royal Dutch Medical Association; RASS-PAL: Richmond Agitation–Sedation Scale–Palliative version; GCS: Glasgow Coma Scale; CSPC: Consciousness Scale for Palliative Care.

Content validity

All studies provided a clear description of the construct measured by the reported instruments. However, the involvement of the target population in selecting or modifying scale items was described only for three of the eight evaluated measures: the CSPC,[18] RASS-PAL[87] and Modified RASS.[73] One study[18] reported receiving feedback on the content of the CSPC from seven palliative care doctors and nurses at the construction stage on the scale. Likewise, the input of palliative care professionals guided the modification of scale items for the RASS-PAL[87] and RASS modified by Benitez-Rosario et al.[73]

Construct validity

Information on construct validity was available for six of the eight included measures: the MSAT,[93,94] VICS,[93,95] RASS,[19,93] KNMG,[93,96] CSPC[18] and Modified RASS.[73] For these, construct validity was evaluated through the correlation of the tested instrument with others that were assumed to measure the same construct (convergent validity). Discriminant validity was not assessed for any tool.

Correlations were reported per subscale for the MSAT and VICS.[93-95] The MSAT arousal subscale performed better than the motor activity subscale with Spearman’s correlation coefficient ranging from 0.48 to 0.83, depending on the measure with which it was correlated (RASS, KNMG and VICS). Low to moderate correlations were reported for the motor activity subscale of the MSAT (ρ = 0.42–0.61). Mostly moderate correlations were found between both subscales of the VICS with other tools measuring level of consciousness (interaction subscale: ρ = 0.31–0.72, calmness subscale: ρ = 0.31–0.57).[93-95]

Construct validity of the RASS and KNMG was supported by moderate-strong associations when compared with corresponding instruments.[19,93,96] Strong correlations with other tools measuring level of consciousness were reported for the Modified RASS and CSPC.[18,73] Spearman’s correlation coefficient for the Modified RASS to the GCS[33] ranged from 0.81 to 0.85 and 0.82–0.89 when compared with the RSS,[52] depending on the group of professionals scoring the scales (palliative care physicians or medical residents).[73] Likewise, the CSPC correlated highly with a 100 mm visual analogue scale (VAS) anchored in the terms ‘awake’ and ‘unarousable’ (ρ = 0.94–0.95) and with the GCS (ρ = 0.82–0.85).[18,33]

Homogeneity (internal consistency)

As the aim of some of the studies was not to address unique measure characteristics, homogeneity was evaluated for only one of the appraised measures, the CSPC.[18] For this instrument, the reported Cronbach’s alpha coefficient was very high (α = 0.99).[18]

Inter-rater reliability

ICC or weighted Cohen’s kappa was used for the assessment of inter-rated reliability in all of the included studies. From the tested measures, inter-rater reliability was found to be high for the CSPC (ICC = 0.99),[18] GCS (ICC = 0.807),[1,33,97,98] RASS-PAL (ICC = 0.84–0.98)[87] and Modified RASS (κ = 0.85–0.95).[73] Moderate correlations within paired observational assessments were reported for the RASS (ICC = 0.71–0.73)[19,93] and KNMG (ICC = 0.66–0.71).[93,96] Of the MSAT and VICS subscales, the VICS interaction scale performed best with ICC ranging from 0.77 to 0.85, followed by the MSAT arousal scale (ICC = 0.59–0.64).[93-95] Depending on the time interval between paired assessments, Cohen’s kappa coefficient ranged from 0.44 to 0.54 for the MSAT overall quality of sedation subscale, suggesting low agreement between scale assessors. No correlations were found for the MSAT motor activity and VICS calmness subscales.[93-95]

Responsiveness

Change scores indicating clinically meaningful change over time in consciousness/sedation levels were not described for any of the appraised measures. Bush et al.[87] provided some information on the floor and ceiling effects for the RASS-PAL but it is not adequate for the assessment of responsiveness.

Origin of items

Items for half of the measures for which evidence of psychometric performance was available originated from scales developed for non-palliative care patients. Specifically, aspects of the measurement properties of the Dutch versions of the MSAT,[93,94] VICS,[93,95] RASS[19,93] and GCS[1,33,97,98] were appraised by study authors adopting the original items of these scales without assessing their appropriateness for the palliative care setting.

For the other half of the scales, items were either modified (RASS-PAL[87] and Modified RASS[73]) or particularly developed (KNMG[96] and CSPC[18]) for monitoring palliative care patients’ level of consciousness.

Feasibility

In a comparison for user-friendliness between the Dutch versions of the RASS,[19] MSAT[94] and VICS,[95] Arevalo et al.[93] reported that most palliative care professionals found RASS the least time-consuming, clearest and easiest to use. Acceptable ratings were achieved for the MSAT, while the VICS was evaluated as the least clear and easy to use among the three tools. The RASS-PAL,[87] CSPC[18] and Modified RASS[73] were also regarded as feasible and useful tools by healthcare professionals.

Discussion

Main findings

This systematic review aimed to identify, describe and appraise the psychometric performance of observational level of consciousness measures used in palliative care. We found 35 different levels of consciousness tools used in 65 studies. Evidence of psychometric performance, however, was available for only eight of these instruments. Two of these eight tools were specifically developed for palliative care populations (CSPC[18] and KNMG[96]), two were versions of an existing tool (i.e. the RASS[19]) modified for use in palliative care (Modified RASS[73] and RASS-PAL[87]) and four were measures developed for different populations, tested for aspects of validity and/or reliability in the palliative setting (GCS,[1,33,97,98] MSAT,[93,94] RASS[19,93] and VICS[93,95]). None of these tools had been evaluated across all relevant psychometric properties; hence no measures appraised had been fully validated.

The majority of measures identified were either ad hoc tools for which no formal validation had been undertaken (n = 17) or tools developed and validated mainly in non-palliative care settings (n = 15). This widespread use of non-validated measures raises questions regarding the methodological robustness of studies and the quality of reported evidence,[99] not least because, although tools’ psychometric performance may have been investigated in specific contexts, this does not transfer to other settings.[100] It is therefore essential, as with any measures to be used in palliative care, that tools assessing level of consciousness should be thoroughly validated with palliative care patients in order to be certain that they are reliable for this population.

Most measures identified sought to measure consciousness in terms of wakefulness and, therefore, mostly (n = 23) comprised one item with a range of levels describing patients’ responses to verbal and/or physical stimulation. Apart from consciousness, a small number of tools (n = 4) included the assessment of agitation, as a domain related to sedative and analgesic use, in a single scale. These tools have been criticised for various reasons, including the lack of clarity in the definition of different consciousness levels, and the poor standardisation of employed stimuli.[16,18] Moreover, the assessment of patients presenting decreased consciousness and restlessness at the same time may be compromised when both conditions are evaluated on the same scale.[14,16] Nevertheless, the most commonly employed measure was the RASS[19] (a tool assessing sedation and agitation on a single-item scale) or modified versions of it (n = 17). An explanation for this may be that the RASS requires minimal training and can be quickly and easily administered at the bedside.[19] These are particularly desirable features for a scale intending to measure level of consciousness, an often unstable characteristic, in clinical environments where patients are cared for by professionals of different backgrounds, as in palliative care.[18]

Limited information was available on the measurement properties of tools, thus making it difficult to draw definitive conclusions about their psychometric performance. Our evaluation was based on evidence obtained from a single study, rather than a group of studies, for each measure. Some studies did not aim to specifically develop and/or validate level of consciousness measures.[1,93,97,98] As a result, these studies assessed only certain psychometric properties on each occasion, and no tools were tested across all measurement properties. Our quality assessment outcomes should be treated with caution, therefore, until further evidence on the psychometric performance of the appraised measures becomes available.

Information on inter-rater reliability and internal consistency was provided by all studies, with most tools performing adequately on both properties. Due to the lack of a ‘gold standard’ level of consciousness measure in palliative care, criterion validity could not be assessed. Instead, in three studies the tested tools were compared with other instruments known to measure level of consciousness.[18,73,93] However, although the reported correlations between the assessed measures and other comparable tools were acceptable to high, the reference measures were not themselves tested for their psychometric performance in a palliative care context.

No publications provided any information regarding test–retest or intra-rater reliability, although all studies described collecting data at more than one time point. This might be explained by the lack of stability of the construct measured, that is, palliative care patients’ fluctuating level of consciousness. Thus, the assessment of these psychometric properties may not be feasible for level of consciousness measures in this population.

The measures with the highest ratings in our appraisal were the CSPC,[18] a tool specifically developed to measure level of consciousness in palliative care, and a version of the RASS modified for use with palliative care patients.[73] However, the only information available about the psychometric performance of either was restricted to that of initial validation studies and insufficient for assessing all appraised measurement properties. Palliative care clinicians and researchers should be mindful of these restrictions when using level of consciousness measures, therefore.

Our findings agree with those of previously published reviews. In their review of level of sedation instruments, De Jonghe et al.[14] reported that responsiveness had not been tested for any of the scales identified. They commented that responsiveness is an important measurement property because it can inform the titration, initiation and withdrawal of sedative drugs.[14] Apart from these benefits, a measure that can reliably detect changes in patients’ level of consciousness over time may enable the longitudinal evaluation of patients and provide a useful outcome measure for palliative care research. Nevertheless, like De Jonghe et al.,[14] we did not find adequate evidence to appraise responsiveness in our review. When seeking to determine clinically important changes in patients’ status or evaluate the effects of medical interventions it may be problematic to use measures that do not demonstrate satisfactory responsiveness, since changes in scores may result from measurement error rather than true changes in patients’ consciousness levels. Thus, it is important that clinicians and researchers are aware of the limited evidence regarding responsiveness when choosing measures to evaluate treatment/intervention outcomes or interpreting level of consciousness scale scores. In order to enable clinical assessment and decision-making, and support the testing of new interventions, future studies that seek to develop new level of consciousness tools or validate existing ones should aim to provide strong evidence on the responsiveness of these measures.

Brinkkemper et al.[101] identified seven scales measuring level of awareness reported in primary studies. Of these, similar to our findings, a significant proportion were ad hoc measures, while the RASS[19] was the most commonly used of the established scales. Brinkkemper et al.[101] found only one tool, the CCS,[76] for which information on psychometric performance was available. Although the authors presented this information, they did not formally evaluate the psychometric quality of the CCS[76] because this was outside the scope of their review. Our search identified the CCS,[5,76,83] but it was excluded from our quality appraisal because the scale used for the assessment of consciousness level constitutes an individual item extracted from a compound measure for assessing the ability of terminally ill patients to communicate that was developed and tested as a whole. Hence, the psychometric evidence provided pertain to the CCS[76] measure as a whole, not its individual items.

Brinkkemper et al. identified a substantially smaller number of tools than we did, because their review focused specifically on the effects of palliative sedation. Our inclusion criteria were broader, allowing the inclusion of studies reporting the use of observational measures regardless of the purpose for which these were employed. Moreover, an increasing number of studies using level of consciousness tools have been published since the publication of their review in 2013. Of the 65 included studies in our review, 26 (40%) have been published since 2013. A possible explanation for this upwards trend may be the recent publication of high impact guidelines recommending the use of observational scales for the monitoring of level of consciousness of palliative care patients receiving sedative medication.[2,102]

Strengths and limitations

A strength of this systematic review is the comprehensive yet broad search strategy followed, including six databases without applying date restrictions. We also performed a thorough backward and forward citation search for all included articles and contacted abstract authors in order to ensure that all relevant publications were identified. A limitation is that we included only English language publications. It is possible that studies providing evidence on measurement properties of translated versions of tools were missed. We are aware of at least one validation study, which was excluded from this review due to language restrictions.[103]

Two reviewers (A.M.K. and E.M.) independently performed the appraisal of the psychometric performance of the identified measures against well-defined quality criteria. Nevertheless, comparability of evidence was hindered by the heterogeneity of studies reporting data on psychometric properties in terms of setting, sample size, participant population, study design and objectives, and of the purposes for which tools were employed on each occasion. Our evaluation, therefore, was based on the limited published evidence from individual studies for each appraised measure.

Conclusion

This systematic review demonstrates that although an increasing number of studies are using observational level of consciousness measures, only a few of these tools have been tested for their psychometric performance in the palliative care setting, and none across all relevant measurement properties. The CSPC and a modified version of the RASS achieved the highest ratings in our appraisal, but further evidence on their measurement properties is needed before either can be recommended as valid and reliable measures for use in palliative care practice and research. Future research in this area should use, and seek to further validate and refine existing level of consciousness measures, rather than developing new tools or using ad hoc instruments.