Validation of the Korean Version of the Delirium Motor Subtype Scale.

OBJECTIVE: The Delirium Motor Subtype Scale (DMSS) is a validated and reliable instrument developed from various methods previously used to assess delirium motor subtypes. It focuses on pure motor disturbances with a relative specificity for delirium. The aim of this study was to investigate the validity and reliability of a Korean version of the DMSS (K-DMSS).
METHODS: We recruited 145 patients who were older than 60 years and had been referred for psychiatric consultation for delirium. These patients were evaluated using the K-DMSS, Liptzin & Levkoff criteria, and the Korean version of the Delirium Rating Scale-Revised-98 (K-DRS-R98) to compare delirium motor subtypes.
RESULTS: The internal consistency of the K-DMSS in assessing delirium motor subtypes was acceptable (Cronbach's alpha=0.79). Delirium motor subtypes identified with the K-DMSS and K-DRS-R98 showed almost perfect agreement (Cohen's Kappa=0.81), while those identified with the K-DMSS and Liptzin & Levkoff criteria showed substantial agreement (Cohen's Kappa=0.78).
CONCLUSION: Our results suggest that the K-DMSS is a valid and reliable tool for identifying delirium subtypes.

INTRODUCTION

The assessment and identification of delirium motor subtypes is clinically important because these subtypes may differ in terms of the severity of symptoms,1 etiology,2 pathophysiology,3 and outcome.45 In addition, delirium is misdiagnosed or detected late in half to two-thirds of cases, which leads to delayed treatment.6 Delirium is likely to be misdiagnosed as psychosis, mania, or akathisia in patients with the hyperactive or mixed subtypes, and as depression or dementia in patients with the hypoactive subtype.5

Altered motor behavior in patients with cognitive disturbance has long been recognized, with the ancient Greeks describing “agitated phrenitis” and “lethargicus”.7 Since the introduction of terms such as “hyperactive”, “hypoactive”, and “mixed” in the contemporary literature,89 there have been new approaches to define motor subtypes using different methods.4101112 However, attempts to categorize the subtype have produced heterogeneous results due to differences in the populations assessed and the methods used to define motor subtypes. In a more recent study, commonly used methods9101113 were compared and the authors reported just 34% concordance between the methods within the same population.14

To develop a valid instrument to assess motor subtypes of delirium, the Delirium Motoric Checklist (DMC)14 was developed by combining 30 features from three existing psychomotor check lists.91011 Among these 30 features, 11 were identified as specific to patients with delirium and correlated with the Delirium Rating Scale-Revised-98 (DRS-R98),13 which resulted in a new motor subtype scale called the Delirium Motor Subtype Scale (DMSS).15 The DMSS focuses on “pure” motor symptoms rather than psychomotoric symptoms. Other studies have shown that the DMSS has good concurrent and predictive validity compared with objective methods for measuring motor activity, including electronic methods such as accelerometry.16 In addition, the scale is simple enough for use by both medical and non-medical staff.

In Korea, there has been no study of the DMSS, although some authors have investigated the associations between the delirium motor subtype and the severity of delirium and clinical characteristics.17

Therefore, the aim of this study was to develop a Korean version of the DMSS (K-DMSS) and investigate the validity and reliability of this scale among elderly patients with delirium.

METHODS

Subjects

In this study, we recruited 145 elderly patients with delirium diagnosed according to DSM-5 criteria.18 All patients had been admitted to a university hospital in South Korea, and subsequently referred to the psychiatric consultation liaison service from any medical or surgical ward. Patients with history of dementia, schizophrenia, bipolar disorder and other severe psychiatric disorder were excluded. We excluded patients suspected to have dementia on the basis of the information provided by caregivers according to DSM-5 criteria.18 We also excluded patients who were admitted to the intensive care unit or emergency room and who refused to participate.

The procedures and rationale for the study, as well as the right to withdraw, were explained to all the patients and their caregivers. Due to the nature of the study, informed consent was obtained from patients where possible, otherwise proxy consent was obtained for caregivers. The study protocol was approved by the Institutional Review Board (IRB No. 2015-04-009).

Assessment tools

K-DMSS

The DMSS is an 11-item scale used to classify delirium into several motoric subtypes.15 Each item is rated as “absent (score=0)” or “present (score=1)” on the basis of the patient's status over the previous 24 h. The DMSS hyperactive subtype is assigned if the patient shows at least two of the following symptoms: 1) increased quantity of motor activity; 2) loss of control of motor activity; 3) restlessness; and 4) wandering. The hypoactive subtype is assigned if the patient shows at least two of the following symptoms: 1) decreased amount of activity; 2) decreased speed of actions; 3) reduced awareness of surroundings; 4) decreased amount of speech; 5) decreased speed of speech; 6) listlessness; and 7) reduced alertness/withdrawal. The mixed subtype is assigned if the patient shows both hyperactive and hypoactive symptoms. No motor subtype is assigned if there is no evidence of either the hyperactive or hypoactive subtype.

To develop the K-DMSS, the DMSS was first translated into Korean by two psychiatrists after obtaining approval by David Meagher, one of the original developers of the DMSS. A bilingual individual conducted back translation and the translation was subsequently reviewed by other psychiatrists in our study team, as well as an English speaker.

K-DRS-R98

The K-DRS-R98 is a 16-item scale comprising 13 items for severity and 3 items for diagnosis. It was validated and standardized in Korean by Lim et al.19 Each item is rated from 0 (absent) to 2 or 3 (severe), with a maximum severity score of 39 and total scale score of 46. Among the severity items, items #7 and #8 rate motor agitation and motor retardation, respectively. For classification of the delirium motor subtype, the hyperactive subtype is assigned if the score for item #7 ranges from 1 to 3 with a score for item #8 of 0. Meanwhile, the hypoactive subtype is assigned if the score for item #8 ranges from 1 to 3 with a score for item #7 of 0. The mixed subtype is assigned if the scores both these items range from 1 to 3, and no motor subtype is assigned if both scores are 0.

Liptzin & Levkoff criteria

Liptzin & Levkoff criteria10 are adapted from the Delirium Symptom Interview (DSI)20 and have been applied to categorize the delirium motor subtype and to compare between methods. The hyperactive subtype is assigned if a patient has three or more of the following symptoms: hypervigilance, restlessness, fast or loud speech, irritability, combativeness, impatience, swearing, singing, laughing, uncooperativeness, euphoria, anger, wondering, easy startling, fast motor responses, distractibility, tangentiality, nightmares, and persistent thoughts. The hypoactive subtype is assigned if a patient has four or more of the following symptoms: unawareness, decreased alertness, sparse or slow speech, lethargy, slowed movements, staring, and apathy. If they meet criteria for both motor subtypes, the mixed subtype is assigned.

Study procedures

This was a prospective study in which medical information was obtained from patients, physicians, nurses, caregivers, and medical records. A trained psychiatrist assessed patients with the K-DMSS, while another trained psychiatrist independently assessed patients with the K-DRS-R98 and Liptzin & Levkoff criteria. In a subset of patients, the K-DMSS was conducted a third psychiatrist to investigate inter-rater reliability.

Statistical analysis

We used Cronbach's alpha to assess the internal consistency of the K-DMSS. We used Cohen's kappa statistic, sensitivity, specificity, and correspondence analysis to compare each motor subtype as defined by the K-DMSS with each motor subtype as defined by the K-DRS-R98 and Liptzin & Levkoff criteria. To investigate inter-rater reliability, we used Cohen's kappa statistic. Cohen's kappa coefficients were interpreted according to the guidelines proposed by Landis and Koch: ‘almost perfect’ (0.81–1); ‘substantial’ (0.61–0.80); ‘moderate’ (0.41–0.60); ‘fair’ (0.21–0.40); ‘slight’ (0.00–0.20); or ‘poor’ (<0.00) agreement.21 All statistical analyses were conducted using SPSS version 22.0 (IBM Corp., Armonk, NY, USA) and statistical significance was accepted at p<0.05.

RESULTS

Patient demographic and clinical characteristics

We included 145 patients (mean age, 75.97±7.09 years) with delirium in the present study. Of these, 87 were male (mean age, 74.54±7.30 years) and 58 were female (mean age, 78.12±6.24 years). Participants had mean K-DRS-R98 total and severity scores of 28.10±5.89 and 21.95±5.59, respectively. The most common primary etiology of the delirium was infection (systemic), followed by other, neoplasm (systemic), and organic insufficiency (Table 1), according to the Delirium Etiology Checklist (DEC) classification.22

Table 1

Primary etiologies underlying delirium in patients (N=145)

Delirium motor subtype distribution

Each patient was classified into the hyperactive subtype, hypoactive subtype, mixed subtype, or no motor subtype according to the K-DMSS, K-DRS-R98 and Liptzin & Levkoff criteria (Figure 1). According to the K-DMSS, the most common motor subtype was the hyperactive subtype (n=72, 50%), followed by the mixed subtype (n=60, 41%), hypoactive subtype (n=9, 6%), and no motor subtype (n=4, 3%). According to the K-DRS-R98, the most common subtype was the mixed subtype (n=70, 48%), followed by the hyperactive subtype (n=64, 44%), hypoactive subtype (n=9, 6%), and no motor subtype (n=2, 1%). According to the Liptzin & Levkoff criteria, the hyperactive subtype (n=80, 55%) is most common, followed by the mixed subtype (n=47, 32%), hypoactive subtype (n=10, 7%), and no motor subtype (n=8, 6%).

Figure 1

Number of delirious patients within each motor subtype.

Reliability

The internal consistency of the K-DMSS was assessed with Cronbach's alpha coefficient (Table 2), with a coefficient for the total items of 0.79. As each item was removed, the internal consistency persisted, with Cronbach's alpha ranged from 0.74 to 0.82.

Table 2

Internal consistency of K-DMSS

For 50 patients, the K-DMSS was assessed by two psychiatrists independently during the same interview to investigate the inter-rater reliability. There was almost perfect agreement, with Cohen's kappa coefficient of 0.86 (p<0.001).

Concurrent validity

The agreement of subtype classification between the K-DMSS and K-DRS-R98 (Tables 3 and 4) was almost perfect, with a Cohen's kappa coefficient of 0.81 (p<0.001). Table 4 shows the additional analyses for the agreement, sensitivity, and specificity for each subtype. When the patients were classified with the K-DMSS and Liptzin & Levkoff criteria (Tables 5 and 6), Cohen's kappa was 0.78 (p<0.001), and Table 6 shows the further analyses for each subtype.

Table 3

Number of the patients within each motor subtype based on K-DMSS and K-DRS-R98 (N=145)

Table 4

Agreement, sensitivity and specificity between K-DMSS and K-DRS-R98 (N=145)

Overall Cohen's κ=0.81. CI: confidence interval

Table 5

Number of the patients within each motor subtype based on K-DMSS and Liptzin & Levkoff criteria (N=145)

Table 6

Agreement, sensitivity and specificity between K-DMSS and Liptzin & Levkoff criteria (N=145)

Overall Cohen's κ=0.78. CI: confidence interval

Correspondence analysis, which graphically represents the relationships between the same subtypes categorized by each scale, showed that the subtypes did not differ from each other (Figures 2 and 3).

Figure 2

Correspondence analysis-relationship between K-DMSS and K-DRS-R98.

Figure 3

Correspondence analysis-relationship between K-DMSS and Liptzin & Levkoff criteria.

DISCUSSION

The aim of the present study was to validate the K-DMSS in a sample of elderly patients with delirium admitted to a general hospital.

There have been numerous attempts to assess the motor symptoms of delirium with psychomotor checklists.91011 Previously, authors have used several delirium severity scales, including the Delirium Rating Scale (DRS),23 DRS-R98,24 Memorial Delirium Assessment Scale (MDAS),225 and Richmond Agitation/Sedation Scale.26 In some cases, clinical observation has been used to identify the delirium subtype.27 However, reported findings have been heterogeneous because of the lack of a standard method with a clear cutoff value and the use of instruments assessing non-motor symptoms. Meagher et al. compared the identification of delirium subtypes with various common methods91011 and the DRS-R98,13 finding just 34% concordance.14 Subsequently, they used items from these various assessments and combined them to create the DMC. From 30 items, they selected 11, that were specific to patients with delirium, and investigated correlation between them and motor items of the DRS-R98. This process created a new assessment, the DMSS.15 The latter allows more precise detection of motor subtypes than other methods by purely focusing on motor symptoms. In addition, it is a simple and easy scale that can be used by non-medical staff, and has been demonstrated to have concurrent and predictive validity.151628

More recently, the amended DMSS was developed by adding “increased speed of actions” to hyperactive items, and “decreased volume of speech” to hypoactive items, following investigation of hospitalized patients referred to consultation-liaison psychiatry.29 Later, an abbreviated version of the DMSS (DMSS-4)30 was presented for brief use in clinical settings where time constraints were present, and this was highly concordant with the original DMSS.

In terms of reliability, our results show that Cronbach's alpha coefficient for items included in the K-DMSS was 0.79, indicating good internal inconsistency. Furthermore, Cohen's kappa coefficient was 0.86, indicating good inter-rater reliability. Therefore, we demonstrated that the K-DMSS is reliable. Indeed, our results suggest slightly higher reliability than those reported in a validation study by Slor et al.31 in which Cronbach's alpha was 0.72.

To evaluate validity, we classified delirium motor subtypes with the K-DRS-R98 and Liptzin & Levkoff criteria, because both have been commonly used in previous studies. Cohen's kappa coefficient for the motor subtypes was 0.81 between the K-DMSS and K-DRS-R98, and 0.78 between the K-DMSS and Liptzin and Levkoff criteria. Therefore, our results demonstrate concurrent validity between the K-DMSS and previously used measure. Validity was also higher than that reported in by Slor et al.31 in which Cohen's kappa coefficient was 0.73 between the DMSS and the DRS-R98. Regarding sensitivity and specificity, except for the sensitivity value (0.5) of the no motor subtype between K-DMSS and Liptzin & Levkoff criteria, the values ranged from approximately 0.80 to 1 (Tables 4 and 6), thereby yielding high accuracy of K-DMSS in classifying each motor subtype and comparing with previously used measures.

In addition, correspondence analysis was used to illustrate the correlations of each subtype with one another, as classified by each scale. In the graphical representation (Figure 2), the distance between each point represents the accordance of each subtype. Each subtype was visualized in the adjacent area. Therefore, the motor subtypes classified with the K-DMSS were concordant with those classified with the other methods.

According to the K-DMSS, the hyperactive subtype was the most common in the present study, affecting 50% of the sample. This is consistent with the results of Grover et al.32 who reported that the hyperactive subtype was most common at 50.15%, but not with the findings of Meagher et al.15 or Uchida et al.33 who reported that the hypoactive subtype was most common, affecting 35% and 58% of their samples, respectively. Grover et al.32 studied hospitalized patients who had been admitted for various diseases and subsequently referred to consultant-liaison psychiatry as a result of delirium. In contrast, Meagher et al.15 and Uchida et al.33 investigated patients who received palliative care or those with advanced cancer, who subsequently presented with symptoms of delirium. Such inconsistent findings may suggest that motor subtype classification may depend on the population characteristics.

In the present study, the hypoactive subtype was relatively less common than the hyperactive and mixed subtypes using all the three assessment methods. This subtype will less likely attract the attention of a caregiver or nurse because there will be less or no overt behavioral disturbance in patients with the hypoactive subtype.34 In contrast, behavioral disturbances in the hyperactive subtype are obvious. As such, some patients with the hypoactive subtype may not have been referred for psychiatric care.

There are several limitations to the present study, which should be borne in mind when considering our results. First, we assessed patients with altered mental state, referred to the psychiatric consultation liaison service. As we did not objectively evaluate the previous mental state of patients and excluded patients with dementia on the basis of the information provided by caregivers, the association between delirium and dementia was not considered. Furthermore, as mentioned above, hyperactive symptoms may have been more detectable; however, hypoactive symptoms may not have been recognized as characteristic of delirium by physicians. This may have resulted in a selection bias. Further studies assessing patients from the time of admission may overcome these limitations. Second, the potential influence of prescribed medications, such as benzodiazepine, antipsychotics, anticholinergics, and opiates, during the assessment was not considered. Third, the scale required symptoms to be rated on the basis of behavior 24 h before the assessment. Nonetheless, it is likely that information provided by patients or caregivers may have referred to times before the previous 24 h. Finally, the sample size was small, and inter-rater reliability was only assessed in a subset of patients.

Nonetheless, to our knowledge, this is the first validation study of the DMSS in elderly Korean patients. Further work in various populations to determine the efficacy of the scale will contribute to our understanding of delirium and its detection, treatment, and prognosis. Furthermore, additional research is required using the various versions of the DMSS, which are continually revised.