Validation of a Paper and Pencil Test Battery for the Diagnosis of Minimal Hepatic Encephalopathy in Korea.

The aim of this study was to validate a new paper and pencil test battery to diagnose minimal hepatic encephalopathy (MHE) in Korea. A new paper and pencil test battery was composed of number connection test-A (NCT-A), number connection test-B (NCT-B), digit span test (DST), and symbol digit modality test (SDMT). The norm of the new test was based on 315 healthy individuals between the ages of 20 and 70 years old. Another 63 healthy subjects (n = 31) and cirrhosis patients (n = 32) were included as a validation cohort. All participants completed the new paper and pencil test, a critical flicker frequency (CFF) test and computerized cognitive function test (visual continuous performance test [CPT]). The scores on the NCT-A and NCT-B increased but those of DST and SDMT decreased according to age. Twelve of the cirrhotic patients (37.5%) were diagnosed with MHE based on the new paper and pencil test battery. The total score of the paper and pencil test battery showed good positive correlation with the CFF (r = 0.551, P < 0.001) and computerized cognitive function test. Also, this score was lower in patients with MHE compared to those without MHE (P < 0.001). Scores on the CFF (32.0 vs. 28.7 Hz, P = 0.028) and the computer base cognitive test decreased significantly in patients with MHE compared to those without MHE. Test-retest reliability was comparable. In conclusion, the new paper and pencil test battery including NCT-A, NCT-B, DST, and SDMT showed good correlation with neuropsychological tests. This new paper and pencil test battery could help to discriminate patients with impaired cognitive function in cirrhosis (registered at Clinical Research Information Service [CRIS], https://cris.nih.go.kr/cris, KCT0000955).

INTRODUCTION

Hepatic encephalopathy (HE) is one of the most serious complications of portal hypertension and liver cirrhosis, occurring in 50% to 70% of patients (1). Minimal hepatic encephalopathy (MHE) is defined as the presence of abnormal results on sensitive psychometric or neurophysiological tests with normal clinical examination in patients in liver disease or portosystemic shunt (2,3). MHE is associated with progression of overt hepatic encephalopathy (OHE), impaired motor vehicle driving and poor health-related quality of life (HRQOL) (4-6). A recent study showed that MHE predicted the development of OHE after insertion of a transjugular intrahepatic portosystemic shunt (TIPS) and psychometric evaluation before TIPS can minimalize OHE (7). In another study, a combination of the model for end-stage liver disease (MELD) and MHE improved survival prediction for patients with liver cirrhosis. These results could help prioritize liver transplantation for waiting patients (8).

Despite all the efforts to find a gold standard method to diagnose MHE, there is no standard diagnostic test for MHE (2,9). In diagnosis of MHE the most widely used method has been psychometric tests, such as the Psychometric Hepatic Encephalopathy Score (PHES) test (10). But the cut-off value of these tests can be influenced by age, education, and sociocultural background (11). Therefore, standardization is needed before applying tests to diagnose MHE in each country (11). Because of these issues, a variety of methods to diagnose MHE have been examined by researchers in different countries. In Europe, PHES test composed of the number connection test-A (NCT-A), number connection test-B (NCT-B), line tracing test, serial dotting test, and digit symbol test has been widely used. In the US, the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS) and other psychometric test batteries are used to diagnose MHE (12). A hepatic encephalopathy study by the working party recommended that MHE diagnosis requires at least 2 of the following tests: NCT-A, NCT-B, block design test, and digit symbol test. Psychometric tests to detect MHE have been individualized according to the country and particular center. In Korea, the Korean version of PHES was validated in 2012 (11). However, there are still copyright issues with the Korean version of this test.

Therefore, the aim of this study was to create and validate a new paper and pencil test battery as a ‘copyleft’ test to evaluate MHE in the Korean population.

MATERIALS AND METHODS

Study design

We recruited voluntary participants from healthy adult populations by advertising in the community to establish a norm for the psychometric test. Volunteers were interviewed in a semi-structured format by a licensed clinical psychologist for screening. Participants were excluded based on present or past history of cognitive dysfunction. Four clinical psychologists with a license from the Ministry of Health and Welfare after completing nationally approved training courses participated in this study. The participating psychologists trained on the test materials until the interrater coefficient was above 0.80. They determined the norms for the new paper and pencil test that was adjusted to normal and abnormal values with respect to age. Finally, 315 participants from a healthy adult population were enrolled to collect normative data.

Another 63 healthy subjects and cirrhosis patients were included as a validation cohort from a tertiary single center. Thirty-one healthy subjects were people who visited the health promotion center at the institution. All participants completed the new paper and pencil test battery, critical flicker frequency (CFF) test and computerized cognitive function test (visual continuous performance test [CPT]).

Enrollment criteria

Healthy control participants did not have a present or past history of substance abuse or addiction (except for nicotine abuse/addiction), psychiatric disease, neurological disease, medical conditions that could alter cerebral functioning (i.e., cardiovascular, endocrinological, oncological, and autoimmune diseases), and/or head trauma with loss of consciousness for 30 minutes or more.

Cirrhotic patients were diagnosed either by 1) biopsy, 2) radiologic findings such as surface nodularity or splenomegaly, 3) endoscopic findings including esophageal or gastric varices, or 4) a present or past history of decompensation (ascites, variceal bleeding, or hepatic encephalopathy).

Exclusion criteria

Healthy subjects for the validating cohort were recruited for the control group. With respect to the control group, subjects with 1) a present or past history of chronic liver diseases, neurological diseases, or psychiatric diseases, 2) a present or past history of substance abuse or addiction, 3) alcohol abuse within the past 3 months (ingestion of > 210 g/week in men and > 140 g/week in women), or 4) taking medications affecting consciousness were excluded.

With respect to the cirrhosis group, patients with 1) OHE, which was defined according to West-Haven criteria (13), 2) presence of neurological diseases or psychiatric diseases, 3) presence of malignancy including hepatocellular carcinoma, 4) presence of substance abuse or addiction, 5) alcohol abuse within the past 3 months (ingestion of > 210 g/week in men and > 140 g/week in women), or 6) taking medications affecting consciousness were excluded.

New paper and pencil test battery

The new paper and pencil test battery was composed of the NCT-A, NCT-B, digit span test (DST), and symbol digit modality test (SDMT), and takes a total 15 minutes. The NCT was given in 2 parts, A and B. In this study, we adopted the scoring system introduced by Reitan based on completion time and number of errors. The DST used the same format as the Wechsler batteries. The DST consisted of a forward (DST-F) and backward (DST-B) test and was measured by the number of complete digits. The results of the SDMT were measured as scores. Each test was scored according to age adjusted norms.

Definition of MHE

MHE was defined as an impaired performance under −1.5 standard deviations (SDs) from the mean on more than 2 tests. The score of the new paper and pencil test battery was the sum of the score of each test computed from the adjusted Z values; −3, −2, −1, 0, and 1 were scored Z ≤ −3, −3 < Z ≤ −2, −2 < Z ≤ −1, −1 < Z ≤ 1, and 1 > Z.

Computerized tests

CFF test (Lafayette Life Science, Lafayette, IN, USA) was used. This test measures visual discrimination ability and general arousal. Luminous pulsating light emitting diodes are presented to a subject using a stepwise decrease in the light frequency from 60 to 25 Hz and the subject presses a button as quickly as possible at the critical fusion threshold. After simple training, each subject performed the CFF test 3 times and the mean value of the 3 tests was calculated as the CFF value.

CPT (Maxmedica, Seoul, Korea) was performed by a computerized cognitive function test. It used a computer based system to evaluate the efficiency of visual attention. Subjects were presented with the digits ‘0 to 9’ at a scheduled time and were asked to press a button as quickly as possible after the digit ‘3’ was presented visually. Omission errors, commission errors, and reaction time were recorded automatically by the computer. The testing time was a 9-minute period.

Participant test-retest reliability

For test-retest reliability, 28 nonalcoholic fatty liver disease patients whose clinical course remained stable completed the new paper and pencil test battery twice within 3 months.

Clinical parameters

Blood samples were taken from each of the liver cirrhosis patients on the day the neuropsychological tests were performed. Blood tests were conducted and included serum albumin, total bilirubin, aspartate aminotransferase (AST), alanine aminotransferase (ALT), prothrombin time (PT; international normalized ratio [INR]), serum creatinine, and serum ammonia.

Statistical analysis

All statistical analyses were performed using SPSS 19.0 for Windows (SPSS Inc., Chicago, IL, USA). P values < 0.05 in 2-sided tests were considered statistically significant.

Categorical variables were presented as frequencies and percentages and continuous variables were presented as mean values ± SDs. Categorical variables were compared using the χ2 test or Fisher's exact test and continuous variables were compared using the Student's t-test or Mann-Whitney test. Spearman's coefficient was used to test the correlation between the new paper and pencil test battery score and each computerized neurocognitive test. Test-retest reliability for each test in the new paper and pencil test was evaluated using a paired t-test or Wilcoxon signed rank test.

Ethics statement

Our study was approved by the Institutional Review Board of the Hanyang University Hospital (IRB No. 2013-08-017) and was registered at Clinical Research Information Service (CRIS), Korea (https://cris.nih.go.kr/cris, KCT0000955). Informed consent was obtained from the patients at enrollment.

RESULTS

Characteristics of the ‘norm’ in healthy subjects

Three hundred fifteen participants were recruited to obtain normative data. Their mean age was 44.0 ± 12.8 years, education period was 14.0 ± 2.9 years and they were predominantly female. The distribution of participants according to age was as follows: age 20–29, 55 participants (17.5%); age 30–39, 53 participants (16.8%); age 40–49, 65 participants (20.6%); age 50–59, 120 participants (38.1%); and age ≥ 60, 22 participants (6.9%) (Table 1). Time for the NCT-A and NCT-B, and errors on the NCT-A and NCT-B tended to increase with age. The time for DST-F, DST-B, and SDMT tended to decrease with age (Table 2, Fig. 1). Education level also showed a negative association with psychometric results.

Table 1

Baseline characteristics of normative participants

Characteristics	Normative participants (n = 315)
Age, yr	44.0 ± 12.8
20–29	55 (17.5)
30–39	53 (16.8)
40–49	65 (20.6)
50–59	120 (38.1)
60–69	22 (6.9)
Gender, male	141 (44.8)
Education, yr	14.0 ± 2.9

Variables are expressed as number (%) or mean ± standard deviation.

Table 2

Standard data for the diagnosis of MHE

Age, yr	NCT-At	NCT-Ae	NCT-Bt	NCT-Be	DST forward	DST backward	SDMT
20–29
Mean, sec	24.71	0.11	42.89	0.40	8.05	6.15	73.09
SD	7.91	0.31	15.73	0.78	1.10	1.45	16.50
30–29
Mean, sec	25.32	0.13	49.15	0.53	7.94	5.79	64.74
SD	8.32	0.34	20.05	0.85	1.05	1.29	16.52
40–49
Mean, sec	31.33	0.20	54.75	0.72	7.17	5.09	58.03
SD	14.30	0.81	16.79	1.35	1.24	1.48	13.50
50–59
Mean, sec	40.42	0.39	71.10	0.75	6.67	4.61	49.38
SD	15.05	1.46	29.98	1.13	1.20	1.23	14.05
60–69
Mean, sec	61.07	0.45	100.53	1.77	5.91	3.91	36.36
SD	31.67	0.60	45.33	1.85	1.06	1.19	10.57

MHE = minimal hepatic encephalopathy, NCT = number connection test, NCT-At = NCT-A times, NCT-Ae = NCT-A errors, NCT-Bt = NCT-B times, NCT-Be = NCT-B errors, DST = digit span test, SDMT = symbol digit modality test, SD = standard deviation.

Fig. 1

Changes in the new paper and pencil test battery according to age. (A) NCT-A and NCT-B. (B) DST. (C) SDMT.

NCT-A = number connection test-A, NCT-B = number connection test-B, DST = digit span test, SDMT = symbol digit modality test.

Clinical characteristics of the validating cohort

Thirty-one healthy subjects and 32 patients with liver cirrhosis were enrolled as the validation cohort. Age (55.3 ± 5.9 vs. 56.5 ± 7.2, P = 0.447) and education period (12.4 ± 4.1 vs. 10.9 ± 3.7, P = 0.169) were not significantly different between the 2 groups. Hepatitis B virus (HBV; n = 18, 56.3%) was the most common cause of cirrhosis in our patients, followed by alcohol (n = 10, 31.3%) and HCV (n = 2, 6.3%). The mean MELD score of the cirrhosis patients was 10.1 ± 4.1 and mean venous ammonia was 76.1 ± 79.0 mg/dL. Only 1 patient (3.1%) had a previous OHE event.

Neuropsychological characteristics according to the presence of MHE in cirrhosis

The total score of the paper and pencil test battery was higher in the control group than the cirrhotic group (0.58 ± 2.00 vs. −1.78 ± 3.40, respectively, P = 0.001). The mean value for CFF (32.6 ± 3.2 vs. 30.8 ± 4.4 Hz, P = 0.048) was lower and the reaction time for the computerized cognitive function test (41.0 ± 8.5 vs. 36.0 ± 7.1, P = 0.015) was higher in the cirrhotic group than the healthy control group.

MHE was defined as an impaired performance under 1.5 SDs from the mean on more than 2 tests. Twelve (37.5%) of the cirrhotic patients were diagnosed with MHE. Age, gender, venous ammonia, Child-Pugh score, and MELD score were not different between the cirrhotic patients with and without MHE. The total score of the new paper and pencil test battery was lower in patients with MHE compared to those without MHE (0.40 ± 1.67 vs. −5.42 ± 2.15, P < 0.001). Time for NCT-A (P < 0.001), NCT-B (P < 0.001), DST-F (P = 0.007), and DST-B (P = 0.002) were significantly increased in the MHE group (Table 3).

Table 3

Comparison of variables between controls to cirrhosis patients with and without MHE

Variables	Control (n = 31)	Cirrhosis (n = 32)		P*	P†
		Without MHE (n = 20)	With MHE (n = 12)
Age, yr	55.3 ± 5.9	56.1 ± 6.2	57.3 ± 9.0	0.627	0.700
Gender, male	12 (38.7)	11 (55.0)	10 (83.3)	0.254	0.139
Education, yr	12.4 ± 4.1	12.2 ± 3.5	8.9 ± 3.1	0.879	0.013
Compensated/decompensated	-	16/4	8/4	-	0.433
Venous ammonia, mg/dL	-	81.2 ± 92.3	69.3 ± 60.0	-	0.700
Platelet, ×103/mm2	-	129.5 ± 101.3	105.9 ± 80.4	-	0.278
Albumin, g/dL	-		4.0 ± 0.8	-	0.882
Total bilirubin, mg/dL	-		1.9 ± 1.8	-	0.179
PT-INR	-	1.2 ± 0.2	1.3 ± 0.3	-	0.344
Child-Pugh score	-	5.6 ± 1.4	6.6 ± 2.4	-	0.150
MELD score	-	9.35 ± 3.80	11.30 ± 4.50	-	0.193
Paper and pencil test score	0.58 ± 2.00	0.40 ± 1.67	−5.42 ± 2.15	0.738	< 0.001
NCT-At, sec	35.57 ± 15.30	39.43 ± 12.47	70.64 ± 26.67	0.350	< 0.001
NCT-Ae, No.	0.19 ± 0.40	0.15 ± 0.37	0.17 ± 0.39	0.697	0.904
NCT-Bt, sec	64.60 ± 30.50	78.89 ± 36.49	176.65 ± 56.02	0.137	< 0.001
NCT-Be, No.	0.97 ± 1.25	0.70 ± 0.98	2.92 ± 1.78	0.422	0.001
DST forward, No.	7.03 ± 1.45	7.25 ± 1.65	5.67 ± 1.23	0.622	0.007
DST backward, No.	4.84 ± 1.42	4.80 ± 1.51	3.25 ± 0.75	0.926	0.002
SDMT, scores	52.00 ± 13.70	47.05 ± 13.35	26.33 ± 9.54	0.208	< 0.001
CFF, Hz	32.67 ± 3.23	32.05 ± 3.76	28.79 ± 3.79	0.541	0.028
CPT
Commission error, scores	59.50 ± 14.80	61.60 ± 15.80	48.18 ± 20.18	0.624	0.049
Commission error, %	69.24 ± 33.10	71.67 ± 33.86	39.09 ± 43.90	0.801	0.028
Omission error, scores	59.45 ± 14.80	57.75 ± 20.52	48.18 ± 20.18	0.732	0.222
Omission error, %	69.24 ± 33.10	71.67 ± 33.86	39.09 ± 43.90	0.801	0.028
Reaction time, scores	41.00 ± 8.50	38.05 ± 7.30	32.36 ± 5.18	0.206	0.030
Reaction time, %	23.99 ± 23.21	16.73 ± 17.31	5.44 ± 6.01	0.236	0.014

Variables are expressed as number (%) or mean ± SD.

MHE = minimal hepatic encephalopathy, PT = prothrombin time, INR = international normalized ratio, MELD = model for end-stage liver disease, NCT = number connection test, NCT-At = NCT-A times, NCT-Ae = NCT-A errors, NCT-Bt = NCT-B times, NCT-Be = NCT-B errors, DST = digit span test, SDMT = symbol digit modality test, CFF = critical flicker frequency, CPT = continuous performance test, SD = standard deviation.

*P < 0.050 between healthy controls and cirrhosis patients without MHE; †P < 0.050 between cirrhosis patients with and without MHE.

The mean value of CFF did not differ between healthy controls and patients without MHE (32.6 ± 3.2 vs. 32.0 ± 3.7 Hz, P = 0.541). However, CFF decreased significantly in patients with MHE compared to those without MHE (32.05 ± 3.76 vs. 28.79 ± 3.79 Hz, P = 0.028). Other computerized test results showed that cognitive function decreased only in the MHE group, but not in all cirrhotic groups or in patients without MHE. Commission error (score: P = 0.049; percent: P = 0.028), omission error (percent: P = 0.028), and reaction time (score: P = 0.030; percent: P = 0.014) were statistically different between patients with MHE and those without MHE (Table 3).

Correlation between the total score of the paper and pencil test battery and other computerized neurocognitive tests

Table 4 shows correlations between the total score of the paper and pencil test battery and other neuropsychological tests. There was a significant positive correlation between the total score of the paper and pencil test battery and education period (r = 0.551, P < 0.001), but not with age. There was also a good positive correlation between the total score of the paper and pencil test battery and CFF (r = 0.551, P < 0.001), commission error (score: r = 0.343, P = 0.006), omission error (score: r = 0.325, P = 0.010) and reaction time (score: r = 0.432, P < 0.001).

Table 4

Correlation between the new paper and pencil test score and other tests

New paper and pencil test score	r	P
Age, yr	−0.126	0.325
Education, yr	0.551	< 0.001
CFF, sec	0.342	0.006
CPT
Commission error, scores	0.343	0.006
Commission error, %	0.375	0.003
Omission error, scores	0.325	0.010
Omission error, %	0.375	0.003
Reaction time, scores	0.432	< 0.001
Reaction time, %	0.384	0.002

CFF = critical flicker frequency, CPT = continuous performance test.

Test-retest reliability

Twenty-eight subjects completed the new paper and pencil test twice within a 12-week period. The mean age of these subjects was 50.9 ± 12.6 years and 18 (64.3%) of the subjects were men. Most parameters, except for NCT-B, showed good test-retest reliability (Table 5).

Table 5

Test-retest reliability

Variables	Baseline	Repeat testing	P
NCT-At, sec	40.79 ± 17.57	40.61 ± 19.84	0.936
NCT-Ae, No.	0.18 ± 0.39	0.07 ± 0.38	0.317
NCT-Bt, sec	113.89 ± 76.83	90.50 ± 44.43	0.021
NCT-Be, No.	2.43 ± 2.39	1.75 ± 1.86	0.024
DST forward, No.	7.39 ± 1.23	7.54 ± 1.10	0.499
DST backward, No.	4.61 ± 1.62	4.50 ± 1.64	0.683
SDMT, scores	42.61 ± 15.66	44.39 ± 15.24	0.099

Variables are expressed as number (%) or mean ± standard deviation.

NCT = number connection test, NCT-At = NCT-A times, NCT-Ae = NCT-A errors, NCT-Bt = NCT-B times, NCT-Be = NCT-B errors, DST = digit span test, SDMT = symbol digit modality test.

DISCUSSION

The paper and pencil test battery including NCT-A, NCT-B, DST, and SDMT showed good correlation with neuropsychological tests. This new paper and pencil test battery could discriminate cirrhotic patients with impairments in cognitive function from healthy subjects.

The prevalence of MHE or covert HE in liver cirrhosis is 20%–80% (3,14,15). MHE adversely affects neurocognitive functions, primarily attention, visuospatial abilities, motor ability, and coordination. MHE has clinical significance as it impairs HRQOL such as daily functioning or driving skill, and is associated with the prediction of OHE development and poor prognosis (4-6). For these reasons MHE brings a host of social and economic burdens to patients, their families, and public health (10). As detected MHE, prompt treatment for it is needed. Currently, the proposed treatment for MHE is reduction of the production and absorption of gut-derived toxins such as lactulose or rifaximin (16). But it is not known how long it should be treated (17).

The PHES test has been the most widely used method for the diagnosis of MHE. In Korea, Seo et al. (11) applied the PHES to a Korean population and validated this test in 2012. However, there are several issues with using the PHES in Korea. First, use of the PHES required approval of the copyright holder (3). Second, ISHEN recommends that measurement on at least 5 out of 8 domains of cognitive function (processing speed, working memory, verbal memory, visuospatial ability, visual memory, language reaction time, and motor function) is needed to assess proper cognitive function (12). However, the PHES only includes measurements of 2 domains (processing speed and visuospatial function) (12). Third, the serial dotting test seemed to have drastic deviations in difficulty level and a low degree of difficulty. Finally, the line tracing test has the complexity and subjectivity of decision-making during interpretation. The new paper and pencil test battery developed in this study included 3 domains of cognitive function (processing speed, visuospatial function, and working memory). Also, we provided 4 alternate forms of the new paper and pencil test battery to decrease the learning effect and to investigate the degree of improvement over time.

We defined the diagnostic criteria for MHE as performance impaired if scores were under 1.5 SD more than 2 tests in this study, while the working party of the 11th World Congress of Gastroenterology in Vienna recommends that MHE is defined as performance impaired if scores are under 2 SD for 2 of the following tests: NCT-A, NCT-B, block design test, and digit symbol test (13). Generally, a meaningful difference between normal and abnormal is about 1.5 SD for a neuropsychological test (18). A cut-off of 2 SD is more rigorous and generally only 1% of all cases have 2 or more impaired test scores, whereas a cut-off of 1 SD is not rigorous (19). Therefore, our study applied a cut-off of “under 1.5 SD on more than 2 tests” and twelve of the cirrhotic patients (37.5%) were diagnosed with MHE based on this criterion.

This study has several limitations. First, we developed a new norm for the paper and pencil test battery using 315 healthy participants. But, a relatively small number of participants in validation cohort were included. Second, we did not compare the new paper and pencil test with the PHES which is widely accepted as the standard method for measuring cognitive function. Instead, we compared the test with other cognitive function tests such as the CFF and CPT, which had good correlation with the new paper and pencil test battery. Third, although the new paper and pencil test battery measured 3 domains (processing speed, visuospatial function, and working memory), ISHEN recommends measuring 5 or more domains (12). However, the conventional PHES only measures 2 domains (processing speed and visuospatial function). Fourth, we did not perform the adjustment of education period in the diagnosis of MHE. There was a significant positive correlation between the new paper and pencil test battery score and education period. Also, education period was significantly different between the cirrhotic patients with and without MHE. We thought that education period affected the diagnosis of MHE to some extent. Finally, the NCT-B did not show test-retest reliability. We thought that this result showed ‘learning effects’ of NCT-B in test-retest reliability, so we developed 4 types of equivalent test sets to minimize learning effects.

In conclusion, we validated a new paper and pencil test battery to screen for MHE in Korea. The new paper and pencil test battery for the diagnosis of MHE was comparable to other neurocognitive function tests. Further studies evaluating its validity in a large cohort are needed.