Development of a risk assessment scale predicting incident functional disability among older people: Japan Gerontological Evaluation Study.

AIM: The aim of the present study was to develop a risk assessment scale for predicting incident functional disability among older adults.
METHODS: We used prospective cohort data from the Japan Gerontological Evaluation Study, a nationwide survey of 90 889 functionally independent older people collected from 23 municipalities. The incidence of functional disability was determined from long-term care information obtained from municipal insurance databases. We constructed a Cox proportional hazards model with forward stepwise selection that used sex, age, and 12 of the essential items of the Public Survey of Long-Term Care Prevention and Needs in Spheres of Daily Life (the Needs Survey). We assigned a score based on the obtained non-standardized regression coefficients for each item and summed the scores to establish the risk assessment scale. The predictive validity was examined.
RESULTS: The cumulative incidence of functional disability during the 3-year follow-up period was 9.7%. A risk assessment scale of 0-48 that used sex, age and the Needs Survey's 10 essential items was established. The area under the receiver operating characteristic curve was 0.804, and the sensitivity and specificity were both 0.733 (cut-off 16/17). There was no significant intermunicipality difference in the associations between the total scores calculated by using the scale and the risk of new incidence (P = 0.135).
CONCLUSIONS: We developed a risk assessment scale predicting incident functional disability composed of 10 essential items of the Needs Survey, sex and age. The scale had superior predictive validity, regardless of the level of urbanness. Geriatr Gerontol Int 2018; 18: 1433-1438.

Introduction

Comprehensive geriatric assessments, which evaluate medical, mental and functional problems of older adults, are useful for identifying health problems that can be treated to improve quality of life.1 For instance, items that have superior predictive validity for the incidence of dementia have been extracted from multidimensional multidisciplinary variables, and then risk assessment scales scored according to their predictive power were developed.2, 3 Furthermore, because the need for such an assessment scale to predict future functional disability has increased in both administrative policy‐making and clinical settings, several attempts have been made to develop such scales.4, 5

We have previously proposed a risk assessment scale for the prediction of incident functional disability, which was included among the essential items of the Public Survey of Long‐Term Care Prevention and Needs in Spheres of Daily Life (the Needs Survey)6 based on data of an ordinance‐designated city.4 However, there are many differences in social background or lifestyles between urban and rural communities, which appears to be the cause of intermunicipality disparities in the proportion of those who are at risk of requiring long‐term care.7 Older people who live in rural areas are reported to have a higher risk of requiring long‐term care than their urban counterparts.8 Furthermore, it has been suggested that the relationship between social participation and mental health among older individuals might differ between urban and rural areas, as differences in levels of urbanness might be causing the differences in factors that trigger functional disability.9 Despite this, because the presence or absence of these differences has rarely been investigated, a scale that is created based on data from a single ordinance‐designated city cannot be applied to all municipalities nationwide. From the perspective of public health and administrative policies, a novel risk assessment scale developed on the basis of data collected on a nationwide scale is required, and it is necessary to compare the predictive validity of this scale across different municipalities with various levels of urbanness.

Therefore, the purpose of the present study was to develop a risk assessment scale for the prediction of incident functional disability with superior predictive validity, regardless of the level of urbanness, in Japan.

Methods

Study design and participants

We used longitudinal cohort data from the Japan Gerontological Evaluation Study (JAGES). The JAGES is an ongoing cohort study investigating social and behavioral factors related to the loss of health with respect to functional decline or cognitive impairment among individuals aged ≥65 years.7 The baseline survey was carried out between October and December 2013, at which time self‐reported questionnaires were distributed by mail to 149 324 people aged ≥65 years who were selected from 23 municipalities in nine prefectures in Japan, and were physically and cognitively independent, and living independently in the community. A random sample was obtained from the official residence registers in 11 large municipalities, as well as a complete census of older residents residing in the remaining 12 smaller municipalities. Among 105 751 respondents (response rate 70.8%), 97 870 (92.5%) participants were successfully linked to the incident records of long‐term care insurance certification. The analytical sample for this study comprised 90 889 participants (42 659 men and 48 230 women) after excluding accidentally included participants who reported limitations in activities of daily living, defined as being unable to walk, take a bath or use the toilet without assistance, to ensure that the sample was actually physically and cognitively independent.

JAGES participants were informed that participation in the present study was voluntary, and that completing and returning the self‐administered questionnaire by mail indicated their consent to participate in the study.10 The Human Subjects Committees of Nihon Fukushi University (No. 13–14) and Chiba University Faculty of Medicine (No. 2493) approved the parent JAGES protocol.

Ascertainment of the incidence of functional disability

The incidence of functional disability was defined according to the new incidence definition of the Needed Support/Needed Long‐Term Care certification under the Japanese long‐term care insurance (LTCI) system.11, 12, 13 Since 2001, the Japanese government has operated a national insurance scheme in which eligibility for long‐term care is based on a standardized multistep assessment of functional and cognitive impairments based on a physician examination.14 The care levels are mainly based on the estimated hours of home care required each week to meet the individual's instrumental and basic activities of daily living.15 Ascertainment of the certification status during the mean follow‐up period of 3 years, from 2013 to 2016 (minimum 2.4 years, maximum 3.3 years), was carried out by linking the cohort participants to the records of the national long‐term care insurance database.

Public survey of long‐term care prevention and needs in spheres of daily life (the Needs Survey)

The Needs Survey's questionnaire formulated by the Ministry of Health, Labor and Welfare was sent from municipalities to older individuals for the purpose of grasping regional issues and needs, and carrying out community diagnosis to establish the Long‐term Care Insurance Service Plan (the 7th phase).6 A total of 12 of the essential items of the Needs Survey included in the JAGES survey, which contains items similar to the Kihon Checklist (KCL),16 were analyzed in the present study (specific items are as presented in Table 1).6 These 12 items were extracted from the total 25 items of the KCL after the review by experts, municipal officials and other individuals concerned according to the following criteria: precision, the representativeness of contents, acceptability by society, academic importance, modifiability and the usability of data.6, 17 As a side note, the KCL worded items, such as “go out by bus or train,” “go shopping to buy daily necessities,” and “manage your own deposits and savings at the bank,” were worded as questions starting with “Do you …?,” whereas we worded these questions with “Can you…?” in the present study to conform with the TMIG Index of Competence.18 Questions were to be answered with a “Yes” or a “No” for all items except body mass index, and answers to unwell or dysfunctional conditions were considered to be “Yes.” Body mass index was calculated as self‐reported weight (kg) divided by height squared (m2), and values of <18.5 were considered to be “Yes” answers.

Table 1

Distribution of participants and cumulative incidences of functional disability during the follow‐up period

Variable	n	Proportion	Cumulative incidence	Variable	n	Proportion	Cumulative incidence
Total	90 889	100.0%	9.7%	Age (years)
Sex				83	1880	2.1%	25.4%
Male	42 659	46.9%	9.2%	84	1605	1.8%	29.7%
Female	48 230	53.1%	10.2%	85	1292	1.4%	31.0%
Age (years)				86	988	1.1%	33.8%
65	3443	3.8%	2.0%	87	747	0.8%	36.3%
66	7005	7.7%	2.1%	88	746	0.8%	42.4%
67	4894	5.4%	2.5%	89	447	0.5%	46.5%
68	4863	5.4%	2.4%	90≤	941	1.0%	48.9%
69	6025	6.6%	3.0%	Essential items of the Public Survey of Long‐Term Care Prevention and Needs in Spheres of Daily Life
70	6427	7.1%	3.5%	Can you go out by bus or train by yourself? (No)	6791	7.5%	27.4%
71	5811	6.4%	4.3%	Can you go shopping to buy daily necessities by yourself? (No)	2054	2.3%	41.3%
72	6144	6.8%	4.7%	Can you manage your own deposits and savings at the bank? (No)	5855	6.4%	22.2%
73	5487	6.0%	6.1%	Do you normally climb stairs without using handrail or wall for support? (No)	35 300	38.8%	14.6%
74	4436	4.9%	7.6%	Do you normally stand up from a chair without any aids? (No)	14 285	15.7%	19.7%
75	4697	5.2%	8.9%	Do you normally walk continuously for 15 min? (No)	12 478	13.7%	15.7%
76	4354	4.8%	9.8%	Have you experienced a fall in the past year? (Yes)	20 498	22.6%	14.8%
77	4078	4.5%	10.8%	Are you very worried about falls? (Yes)	31 971	35.2%	15.2%
78	3772	4.2%	12.6%	Body mass index <18.5 kg/m2	6330	7.0%	16.4%
79	3032	3.3%	14.8%	Do you have any difficulties eating tough foods compared to 6 months ago? (Yes)	21 660	23.8%	13.6%
80	3117	3.4%	17.9%	How often do you go out? (<1 day/week)	2809	3.1%	26.2%
81	2452	2.7%	21.6%	Do you go out less frequently compared to last year? (Yes)	15 706	17.3%	19.1%
82	2206	2.4%	24.9%

Basic characteristics

Sex and age data were obtained from the national long‐term care insurance database. Age was treated as a categorical variable for every 1 year of age from 65 to 89 years, and was aggregated for ≥90 years.

Statistical analysis

The scores were determined by using the following method in accordance with previous research, so that the scores would be whole numbers, taking into consideration the fact that older individuals might also use the scale.2, 3, 4 At first, we used a Cox's proportional hazards model with the new incidence of functional disability as the outcome. Sex and age were forcibly entered into the model, and the 12 items of the Needs Survey were entered with forward stepwise selection. The P‐value of the likelihood ratio test was set at 0.001 for forward selection and 0.01 for backward elimination. If participants did not respond to the items, corresponding observations were assigned to “missing” categories. To check multicollinearity, Spearman's rank correlation coefficient was calculated, and if the result was ρ < 0.5 and a prominent 95% confidence interval (CI) could not be found for each explanatory variable, we determined that there was non‐multicollinearity. After obtaining the final model, we calculated the score as follows. The minimum non‐standardized partial regression coefficient (B) obtained from the items of the Needs Survey was corrected to 1.0, and we then multiplied the correction rate for B for all items. Decimals were rounded off, and the integers thus obtained were established as the score for each item. If participants corresponded to the reference category of the model, the score was as assigned as zero. Then, the scores were summed for each individual according to the responses of the questionnaire. The “missing” category of each item was not included in this scale. To confirm the validity of the scale, the cumulative incidence of functional disability in each total score was calculated among participants who responded to all items (n = 79 536). We further calculated the area under the curve (AUC) on the basis of the receiver operating characteristic line to establish the cut‐off point at which the sum of sensitivity and specificity reached the greatest value. Because the influence of the basic characteristics (particularly age) was expected to be strong, we calculated the AUC, sensitivity and specificity of the score calculated from sex and age alone, and of the total score calculated from all items, and then compared the obtained values. Furthermore, the increased discriminatory value of the Needs Survey items was examined by the integrated discrimination improvement (IDI) and net reclassification improvement (NRI).19 We categorized levels of urbanness by using methods described in previous studies, in which a population density (persons per km2 of inhabitable area) ≥4000 was classified as metropolitan (3 municipalities), 1000–3999 was classified as urban/semi‐urban (11 municipalities) and <1000 was classified as rural (9 municipalities).20, 21 We examined whether predictive validity was maintained in the scale for all levels of urbanness.

There was a possibility that the association between the certification and total scores differed among municipalities. To examine this possibility, we applied multilevel survival analysis (random intercept and random slope model) with the individuals (level 1) nested in the municipalities (level 2). A P‐value >0.05 of the random effect of the slope showed there was no difference in intermunicipality associations between the total scores and new incidence of functional disability.

All statistical analyses, with exception of the IDI, NRI and multilevel survival analysis, were carried out with IBM SPSS Statistics 22 (IBM Corporation, Armonk, NY, USA). For the IDI, NRI and multilevel survival analysis, Stata MP 14.2 (StataCorp, College Station, TX, USA) was used.

Results

Cohort profiles and characteristics of participants

Table S1 shows the follow‐up outcomes of 90 889 participants for a maximum duration of 1201 days and for a mean duration of 1081 days for a total of 269 287 person‐years by the total participants and by level of urbanness. The total sample included 8855 (9.7%) individuals who comprised new incidences of functional disability, which was the primary end‐point of the present study, and the incidence per 1000 person‐years was found to be 32.9 people. Table 1 and Table S2 show the number of applicable people for each item, and the applicable ratios as well as the cumulative incidence of functional disability for the whole group and by the level of urbanness, respectively.

Cox proportional hazards models for developing risk assessment scale

We confirmed that the correlation coefficients, ρ, were <0.5 among all explanatory variables. There were no items with a markedly large 95% CI obtained from the Cox proportional hazard model for scale creation, which indicated an absence of multicollinearity. Table 2 shows the Cox proportional hazard model results and scores for each item. A maximum 48‐point (higher scores indicate higher risk) risk assessment scale predicting incident functional disability was created. Figure 1 shows the total score distribution and cumulative incidence of functional disability by total score for the whole sample and by the level of urbanness. We confirmed that the distribution showed a positive skewness, and the cumulative incidence tended to increase as the total score increased for the whole sample and each level of urbanness. The proportion of new incidence (line graph) becomes unstable toward the right edge, because the number of individuals within the score is smaller.

Table 2

Hazard ratios, 95% confidence intervals, and unstandardized partial regression coefficients of adopted items in the scale and the calculated scores

Variable	HR	95% CI	B	Score	Variable	HR	95% CI	B	Score
Sex					Age (years) cont.
Male	1.12	(1.07–1.17)	0.11	1	83	9.66	(7.51–12.43)	2.27	19
Age (years)					84	11.08	(8.62–14.26)	2.41	21
65	1.00			0	85	11.38	(8.81–14.68)	2.43	21
66	1.02	(0.77–1.36)	0.02	0	86	12.81	(9.89–16.59)	2.55	22
67	1.19	(0.89–1.60)	0.17	1	87	13.28	(10.19–17.29)	2.59	22
68	1.14	(0.85–1.53)	0.13	1	88	15.00	(11.56–19.47)	2.71	23
69	1.43	(1.08–1.88)	0.36	3	89	15.31	(11.65–20.13)	2.73	23
70	1.68	(1.28–2.19)	0.52	4	90≤	15.66	(12.14–20.20)	2.75	24
71	2.02	(1.55–2.64)	0.70	6	Essential items of the Public Survey of Long‐Term Care Prevention and Needs in Spheres of Daily Life
72	2.14	(1.65–2.78)	0.76	7	Can you go out by bus or train by yourself? (No)	1.28	(1.20–1.37)	0.25	2
73	2.71	(2.09–3.50)	1.00	9	Can you go shopping to buy daily necessities by yourself? (No)	1.40	(1.28–1.53)	0.33	3
74	3.38	(2.61–4.37)	1.22	10	Can you manage your own deposits and savings at the bank? (No)	1.22	(1.14–1.31)	0.20	2
75	3.87	(3.00–4.98)	1.35	12	Do you normally climb stairs without using handrail or wall for support? (No)	1.34	(1.28–1.40)	0.29	3
76	4.26	(3.31–5.49)	1.45	12	Do you normally stand up from a chair without any aids? (No)	1.27	(1.21–1.34)	0.24	2
77	4.51	(3.50–5.80)	1.51	13	Do you normally walk continuously for 15 min? (No)	1.12	(1.06–1.19)	0.12	1
78	5.17	(4.02–6.65)	1.64	14	Have you experienced a fall in the past year? (Yes)	1.22	(1.17–1.28)	0.20	2
79	5.89	(4.58–7.59)	1.77	15	Are you very worried about falls? (Yes)	1.32	(1.26–1.39)	0.28	2
80	6.93	(5.40–8.89)	1.94	17	Body mass index <18.5 kg/m2	1.49	(1.39–1.59)	0.40	3
81	8.43	(6.56–10.82)	2.13	18	Do you go out less frequently compared to last year? (Yes)	1.47	(1.40–1.55)	0.39	3
82	9.57	(7.45–12.28)	2.26	19
					Range of total score				0–48

CI, confidence interval; HR, hazard ratio.

Figure 1

Distribution of the total score of the risk assessment scale (bar graph/left axis) and cumulative incidence of functional disability in each total score (line graph/right axis).

Receiver operating characteristic analysis

Table 3 shows the AUC (95% CI) obtained from the receiver operating characteristic curve of the scale for the new incidence of functional disability, and cut‐off points for the maximum sum of the sensitivity and specificity. According to the total score calculated from all items, the AUC when the scale was applied to all participants and to participants from various levels of urbanness ranged from 0.796 to 0.814. These values were larger than those calculated from sex and age alone, without any overlap in the 95% confidence intervals. In addition, the IDI and NRI showed significant improvement in discriminatory value when the Needs Survey's 10 essential items were added. The cut‐off value for determining new incidence was set at 16 out of 17, sensitivity at 0.719–0.759 and specificity at 0.697–0.743.

Table 3

Area under the curve, sensitivity, specificity, integrated discrimination improvement and net reclassification improvement for the risk assessment scale in predicting incident functional disability

	AUC	95% CI	Sensitivity	Specificity	IDI	P	NRI	P
Score calculated by using the scale (sex and age)†
All	0.770	(0.765–0.776)	0.714	0.693
Metropolitan	0.762	(0.753–0.772)	0.690	0.697
Urban/semi‐urban	0.778	(0.771–0.786)	0.717	0.704
Rural	0.765	(0.753–0.776)	0.742	0.659
Total score calculated by using the scale (sex, age and the Needs Survey's 10 essential items)‡
All	0.804	(0.799–0.810)	0.733	0.733	0.041	<0.001	0.130	<0.001
Metropolitan	0.796	(0.786–0.805)	0.719	0.739	0.042	<0.001	0.130	<0.001
Urban/semi‐urban	0.814	(0.806–0.822)	0.732	0.743	0.043	<0.001	0.159	<0.001
Rural	0.797	(0.786–0.809)	0.759	0.697	0.039	<0.001	0.117	<0.001

Cut‐off value for determining new incidence of functional disability is 12 out of 13.

Cut‐off value for determining new incidence of functional disability is 16 out of 17.

AUC, area under the curve; CI, confidence interval; IDI, integrated discrimination improvement; NRI, net reclassification improvement.

Multilevel survival analysis

Table 4 shows the results of the multilevel survival analysis. Each point increase in the total score calculated by using the scale showed an increased risk of a new incidence of functional disability by 1.128‐fold (95% CI 1.125–1.132), and there was no significant intermunicipality difference in this association (P = 0.135).

Table 4

Results of the multilevel survival analysis

			P
Fixed effects
Total score (per 1 point): HR (95% CI)	1.128	(1.125–1.132)	<0.001
Intercept (standard error)	<0.001	(<0.001)	<0.001
Random effects
Variance, total score (standard error)	0.000021	(0.000014)	0.135
Variance, intercept (standard error)	0.033	(0.019)	0.080

CI, confidence interval; HR, hazard ratio.

Discussion

Data from a 3‐year follow up of 90 899 older individuals living in 23 Japanese municipalities were used to develop a risk assessment scale predicting incident functional disability that was composed of 10 essential items of the Needs Survey, sex and age, with a maximum of 48 points. We showed that this scale has superior predictive validity regardless of the level of urbanness of the municipalities.

Previous studies have reported regional differences in the proportion of risk factors for the incidence of functional disability, such as experiencing falls,7 and have also found that living in rural areas increased the risk of functional disability compared with that of life in urban areas.8 Therefore, the present study focused on whether there were regional differences in the factors related to the incidence of functional disability. As a result, the predictive validity of the developed scale did not show any noticeable difference between levels of urbanness. Furthermore, no statistical significance was observed in a multilevel survival analysis, which examined whether the correlation between total scores of the scale and the incidence of functional disability differed among municipalities, thereby showing no intermunicipality difference in these correlations. Certain social background factors, such as the insurer's financial status22 or type of service provider,23 and local environments that are more prone to cause motor disorders in the residents24 are reported to affect the risk of requiring long‐term care, and the incidence of certification for long‐term care required. However, there have been no reports on the factors that cause regional differences in correlations. In evaluating the risk of the incidence of functional disability, differences between regions or levels of urbanness in risk factors do not need to be taken into consideration, and the scale appeared to have a high level of versatility.

The 10 items selected for the present risk assessment scale matched items in the scale created on the basis of data from an ordinance‐designated city.4 Therefore, these 10 items might be especially important for the risk assessment of functional disability regardless of urbanness levels. The AUC of 0.796–0.814, sensitivity of 71.9–75.9% and specificity of 69.7–74.3% obtained from the scale in the present study were not inferior to the AUC (0.783) and sensitivity/specificity (70.5%/73.1%) of scales obtained in past studies on the basis of a sample selected from a single municipality,4 or AUC (0.62–0.83) and sensitivity/specificity (78.1%/63.4%) with KCL.25

The strengths of the present study include the large‐scale sample of older individuals living in 23 municipalities from a wide variety of levels of urbanness nationwide and the extensive longitudinal data with a high follow‐up rate for 3 years obtained from the certification data from the respective municipalities. However, it also includes the following limitations. First, as this was not a nationally representative sample, generalizability requires caution. Furthermore, although the recovery rate for the baseline survey was relatively high at 70.8%, we cannot completely eliminate the possibility of selection bias. The respondents might have had higher function than that of the non‐respondents. Second, the questions and selection options of the KCL, the Needs Survey and the present study were similar, but not completely identical. Because there are partial inconsistencies resulting from system revisions and other causes, these are limitations because some answers required replacement for convenience. Future investigation is warranted to assess compatibility among these items and to determine whether the predictive validity is maintained. Third, the incident functional disability was defined by certification for Needed Support/Needed Long‐term Care under the Japanese LTCI system in accordance with the previous reports. Older individuals with minor disability, however, are not necessarily certified under the LTCI system, and those certified as light grade (i.e. Needed Support) might not necessarily present with disability in activities of daily living.26 Using data with a little discrepancy due to such issues in the LTCI system might have caused systematic errors and the underestimation of predictive validity.

In conclusion, uniform application of the scale in the present study was possible to municipalities nationwide in Japan, and can be expected to show superior predictive validity. The present findings suggested that it is possible to identify older individuals at high risk of incident functional disability or the areas with high rates of high‐risk individuals.

Disclosure statement

The authors declare not conflict of interest.

Table S1 Cohort profiles of the study from all participants and each level of urbanness.

Click here for additional data file.

Table S2 Distribution of participants and cumulative incidence of functional disability during the follow‐up period in each level of urbanness.

Click here for additional data file.