Risk profiles of frequent outpatients among public assistance recipients in Japan: a retrospective cohort study using a classification and regression trees algorithm.

OBJECTIVES: Although several individual risk factors of frequent outpatient attendance (FOA) have previously been reported, identifying a specific risk profile is needed to provide effective intervention for impoverished citizens with complex biopsychosocial needs. We aimed to identify potential risk profiles of FOA among public assistance recipients in Japan by using classification and regression trees (CART) and discussed the possibilities of applying the CART to policypractice as compared with the results of conventional regression analyses.
DESIGN: We conducted a retrospective cohort study.
SETTING: We used secondary data from the public assistance databases of six municipalities in Japan. PARTICIPANTS: The study population included all adults on public assistance in April 2016, observed until March 2017. We obtained the data of 15 739 people on public assistance. During the observational period, 435 recipients (2.7%) experienced FOA. OUTCOME MEASURE: We dichotomised a cumulative incidence of FOA during the study period into a binary variable of exhibiting FOA or not. We adopted the definition of FOA by the Ministry of Health, Labour, and Welfare: visiting the same medical institution more than 15 days a month.
RESULTS: The results of the CART showed that an employed subpopulation with mental disabilities exhibited the highest risk of FOA (incidence proportion: 16.7%). Meanwhile, multiple Poisson regression showed that the adjusted incidence ratio of being unemployed (vs employed) was 1.71 (95% CI 1.13 to 2.59).
CONCLUSIONS: Using the CART model, we could identify specific risk profiles that could have been overlooked when considering only the risk factors obtained from regression analysis. Public health activities can be provided effectively by focusing on risk factors and the risk profiles. © Author(s) (or their employer(s)) 2022. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Using public assistance databases, we could identify the risk factors and risk profiles of frequent outpatient attendance among the impoverished people who were usually difficult to reach in social surveys.

By using a classification and regression trees analysis, the specific high-risk and low-risk profiles and the number of people included in each profile could be statistically identified; such data are not captured by conventional regression analyses.

Focusing on risk factors and profiles may help researchers and policy-makers design population approaches and targeted preventive and supportive care in health management support programmes.

We could not evaluate whether their frequent outpatient attendance was medically necessary or not.

Our study may not represent the characteristics of recipients living in larger urban cities in Japan.

Introduction

Unnecessary frequent outpatient attendance (FOA) affects individuals’ health outcomes and the sustainability of healthcare systems. Frequent outpatients are likely to experience fragmented, uncoordinated, and ineffective healthcare, resulting in poor health outcomes.1–3 Additionally, such FOA is linked to increased economic costs of the country’s health systems, hospital overcrowding, and a decrease in the quality of healthcare.2 3 There is no agreed consensus on the definitions of FOA, and its definition is heterogeneous across the countries and clinical settings4; however, recent studies reported a similar association that patients’ complex biopsychosocial needs drive their FOA.5 6 For example, chronic pain and chronic respiratory problems are known biological risk factors linked to FOA among older people.1 6–14 In terms of psychosocial factors, substance abuse, mental illness, and a state of loneliness or social isolation lead to people’s FOA.6 11 12 15–18

In Japan, FOA, defined as visiting the same medical institution more than 15 days in a month among public assistance recipients, has been discussed for a long time as a policy issue. There is no clear definition of FOA by public assistance recipients; the Ministry of Health (former organisation of the Ministry of Health, Labour, and Welfare (MHLW)) conventionally made the instrumental definition not to include FOA, which may be a necessary consultation in processes such as hemodialysis. The recipients, who are fully exempted from out-of-pocket medical expenses, have been considered heavy users of medical services.19 20 For example, a higher proportion of recipients exhibited FOA when compared with people on other health insurance (1.1% in public assistance recipients and 0.1%–0.7% in people on other types of public health insurance) in 2016.21 Our previous study, which redefined recipients’ FOA as visiting the same medical institution more than ten days in a month, showed that this alternative definition included more recipients who were older and had physical disabilities.22 The findings supported the governmental definition that FOA (15 and more days in a month) can, to some extent, exclude recipients exhibiting necessary FOA. Given that the government has concerns about increasing medical costs for this population as their health status require more attention than ordinary people not receiving the assistance,19 the health management support programme for public assistance recipients was launched in 2021 to maintain the health and sustainability of the healthcare system.20 Addressing recipients’ FOA is one of the critical issues addressed in this programme.

Our previous study revealed that factors related to social isolation, such as living alone, unemployment, and being an immigrant, were associated with FOA among public assistance recipientsin Japan using the regression model.22 Although the conventional regression analysis can capture a single risk factor assuming that all other characteristics are the same across the investigated population, adjusting for covariates and potential confounders, the impoverished population such as public assistance recipients face multidimensional complex social needs.23 The problem of the previous regression approach was the unrealistic assumption and limited identification of the actual high-risk persons or subpopulation for an outcome.24 To provide adequate care for recipients of public assistance, identifying risk profiles of individuals with risk for unideal health or behavioural conditions, including FOA, would be helpful. This will help policy-makers and care providers prioritise subpopulations for preventive and supportive care, with tailormade approaches in designing their care services, considering their demographical and behavioural characteristics. However, to our knowledge, no study has tried to identify the multidimensional profiles of subpopulations like frequent outpatients.

Hence, this study identified the potential risk profiles, including complex interactions between predictive risk factors of FOA among public assistance recipients in Japan using the established machine learning approach, classification and regression trees (CART).24 25 To consider the advantages and disadvantages of applying CART for health and welfare care, we compared the results of our CART analysis to those of conventional regression analyses.

Material and methods

Study participants

We used a retrospective cohort study including adults (20 years and older) receiving public assistance in six municipalities (four suburban and two rural areas) in April 2016. We observed them for a year until March 2017. We excluded the recipients who stopped receiving public assistance during the observational period, mainly due to an increase in income or their death. Public assistance (‘seikatsu-hogo’) is the Japanese governmental welfare programme for people living below the minimum cost of living without any assets.26 Although some residents in small municipalities are evaluated by prefectural welfare offices, the municipal welfare office generally conducts rigorous tests and decides whether the household is eligible for receiving benefits. Approximately 1.7% of the population received public assistance in Japan in 2016. Households on public assistance receive monthly minimum income benefits and are fully exempted from payment on their medical care utilisation.26

Data sources and measurements

Outcome variable

We used FOA as the outcome and obtained the FOA information from the administrative medical assistance data between April 2016-March 2017. The data included information on which month the recipients have used medical services, the total cost of medical claims, the total number of visits each month, and their diagnosis. We dichotomised a cumulative incidence of FOA during the study period into a binary variable of exhibiting FOA or not. We adopted the definition of FOA by the MHLW: visiting the same medical institution more than 15 days a month.20

Explanatory variables

We used the baseline data of the study participants as explanatory variables. The baseline data were obtained from the public assistance recipients’ database of the municipal welfare offices, which included information on age, sex, number of family members, household composition, nationality, working status, and income, including working income, pension, and disability pension. The data were collected by the municipal welfare office staff to determine the receipt of public assistance and the amount of monthly minimum income protection, which resulted in no missing data.

Based on a previous study,22 we extracted information on the age (continuous), sex (women or men), the number of household members (continuous), days which recipients worked in the baseline month (continuous), nationality (Japanese or others), types of households (categorised into four groups according to the MHLW27; households with older people, disabled people, sick people, single parent, and others). The public assistance programme is certified by a household unit by a representative person (head of the household; a person who primarily works and earns the household’s income). Municipal offices categorise the recipients’ households from the characteristics of the head. Households with older people refer to households with only older people but may include children under 18 years of age without adults aged 18–64. Households with disabled people refer to families where the head of the household cannot work due to their disability. Households with sick people indicate that the head of the household cannot work due to their severe illness. Single parent households indicate that the head of the household is a single parent. Other households included households that were not applicable to any of the above definitions. These categories were used because older, disabled, or single-parent households may receive benefits from other welfare services. Health status was classified into six categories using the information on diseases and disabilities; mental disability, intellectual disability, physical disability, alcoholism, mental illness (without disability certification), and other health statuses. Only one status was typed by the staff of municipal welfare offices for each recipient. Recipients with disability certificates received an extra subsidy and were certified by municipal offices according to the diagnosis of designated physicians. Recipients with disability certificates and single-parent households receive additional allowances.27

Each municipality individually linked the administrative medical assistance data and the public assistance recipients’ database using unique identification codes. The welfare offices of the six municipalities agreed to provide anonymised data to the authors via a system company that had provided the management software of the public assistance database for municipalities.

Statistical analysis

We described the characteristics of the study participants and the recipients with FOA during the observational period. We performed a CART analysis designed to explore the interaction effect between multiple factors with maximum statistical power and provide a simple interpretation for a graphical representation of the hierarchy of predictor variables.24 25 We entered all explanatory variables into a partition modelling analysis in JMP V.15 (SAS Institute). The splitting procedure was performed for each resultant node. This was an interactive process; we stopped splitting when an increase in R-squared value became small to avoid overfitting, and we stopped splitting when all the items of the tree diagram reached the fourth level of depth to ensure interpretability of the subpopulations. Additionally, we performed univariable and multivariable logistic regression analyses and calculated the crude and adjusted ORs and 95% CIs of each explanatory variable for FOA. Based on the estimated OR of the logistic regression analysis, we calculated a risk score for FOA (online supplemental file 1). As a sensitivity analysis, we performed the analyses with stratified data by age (over 65 years old or not) of participants. Subsequently, we compared the risk profiles and the risk factors obtained from each analysis. The logistic regression analyses were performed using STATA MP V.17 (Stata Corp.).

Patient and public involvement

Patients and the public were not involved in the present study.

Results

We obtained the data of 15 739 people on public assistance. During the observational period, 435 recipients (2.7%) experienced FOA (table 1). Among the study participants, 8662 (55.0%) were women, 9695 (61.6%) lived alone, 2090 (13.3%) had a job and 484 (3.1%) were immigrant recipients. Regarding recipients’ health status, 1433 (9.1%) had certified mental disabilities, 262 (1.7%) had intellectual disability certificates, and 1344 had physical disability certificates (8.5%). As for household types, 7332 (46.6%) lived in older households, 4537 (28.8%) lived in households with disabled people, and 1012 (6.4%) lived in single-parent households.

Table 1

Characteristics of public assistance recipients and frequent attendees included in this study

Character	Category	Overall(N=15 739)N (%)	Frequent attenders(N=435)n	% for N
					Age	Mean (SD)	62.0 (16.7)	66.8 (14.0)
						Younger (20–64 year)	7585 (48.2)	158	2.1
	Older (65 year-)	8154 (51.8)	277	3.4
Sex
	Female	8662 (55.0)	238	2.8
	Male	7077 (45.0)	197	2.8
Household number
	Living alone	9695 (61.6)	319	3.3
	Two	3832 (24.3)	95	2.5
	Three	1078 (6.8)	14	1.3
	Four or more	699 (4.4)	7	1.0
Working days per month
	Zero	13 649 (86.7)	408	3.0
	1–10 days	748 (4.8)	11	1.5
	11–20 days	1030 (6.5)	12	1.2
	21 days or more	312 (2.0)	4	1.3
Nationality
	Japanese	15 255 (96.9)	423	2.8
	Others	484 (3.1)	12	2.5
Individual health statuses
	Mental disability	1433 (9.1)	34	2.4
	Intellectual disability	262 (1.7)	2	0.8
	Physical disability	1344 (8.5)	62	4.6
	Alcoholism	138 (0.9)	7	5.1
	Mental diseases	1392 (8.8)	28	2.0
	Others	11 170 (71.0)	302	2.7
Types of households
	Older	7332 (46.6)	252	3.4
	Disabled	1843 (11.7)	57	3.1
	Sick	2694 (17.1)	59	2.2
	Single parent	1012 (6.4)	9	0.9
	Others	2858 (18.2)	58	2.0

We used a CART model to classify the recipients till all the items of the tree diagram reached the fourth level of depth, which resulted in producing 15 groups (figure 1) (table 2). The R-square value was 0.034. Of the 15 groups, a group of employed people certified as having a mental disability showed the highest incidence proportion of FOA (16.7%) (Profile no. 1). Profile no. 2 and 3, which showed the second and the third largest incidence proportions, included unemployed recipients with alcoholism and physical disability. Meanwhile, four groups (no. 12–15) showed no risks of FOA (table 2).

Figure 1

Classification and regression trees demonstrating profiles for frequent outpatient attendance.

Table 2

The characteristics of segmented profiles and incidence of frequent outpatient attendance: the results of the classification and regression trees analysis

No	Demographic status			Working status	Health statuses						Household types					Number of recipients in the profile	Frequent attenders in the profile	% of FOA
	Age	Sex	Household number	Working days	1	2	3	4	5	6	A	B	C	D	E	N=15 739	N=435
1	58-			1-	+											30	5	16.7
2	43-			0				+								102	7	6.9
3	43-			0			+									1242	58	4.7
4	28–42	M										+				176	8	4.5
5	66-		5-	0	+	+			+	+						6656	228	3.4
6	−42			−10									+	+	+	117	3	2.6
7	43–65			0	+	+			+	+						4026	99	2.5
8	43–58			1-	+											83	2	2.4
9	−42	F	1									+				129	2	1.6
10	43-		−2	1-		+	+	+	+	+						1093	15	1.4
11	−42		−4	−10									+	+	+	1409	8	0.6
12	43-		3-	1-		+	+	+	+	+						201	0	0
13	−27	M										+				36	0	0
14	−42	F	2-									+				59	0	0
15	−42			11-									+	+	+	380	0	0

Working days indicate the days that recipients worked for in the baseline month. Health statuses: (1) mental disability, (2) intellectual disability, (3) physical disability, (4) alcoholism, (5) mental illness without certification, and (6) other health statuses.

A, with older people; B, with disabled people; C, with sick people; D, single parents; E, others; FAO, frequent outpatient attendance.

Multiple logistic regression analysis showed that the adjusted OR of living alone (vs living with someone) was 1.36 (95% CI 1.06 to 1.75), and the adjusted OR of being unemployed (vs employed) was 1.73 (95% CI 1.14 to 2.63) (table 3). The adjusted ORs of certified physical disability and alcoholism were 1.40 (95% CI 1.02 to 1.91) and 2.02 (95% CI 0.92 to 4.43), respectively, when compared with other health statuses, whereas the OR of certified mental disability was 0.72 (95% CI 0.45 to 1.15) (table 3). The risk score method identified several high-risk and low-risk profiles that CART did not capture; however, this method could only identify a few recipients due to unrealistic variables(online Supplemental File 2).

Table 3

OR and 95% CIs for frequent outpatient attendance by individual characteristics: results of crude and multiple logistic regression

	Crude		Adjusted
	OR	95% CI	OR	95% CI
Age
Under 65	Ref		Ref
65 and over	1.65	1.36 to 2.01	1.46	0.91 to 2.34
Sex
Female	Ref		Ref
Male	0.99	0.81 to 1.19	1.08	0.89 to 1.31
Living alone
No	Ref		Ref
Yes	1.59	1.28 to 1.97	1.36	1.06 to 1.75
Working status
Yes	Ref		Ref
No	2.35	1.59 to 3.49	1.73	1.14 to 2.63
Nationality
Japan	Ref		Ref
Other	0.89	0.50 to 1.59	0.90	0.50 to 1.61
Individual health statuses
Other health statuses	Ref		Ref
Mental disability	0.87	0.61 to 1.25	0.72	0.45 to 1.15
Intellectual disability	0.28	0.07 to 1.12	0.29	0.07 to 1.21
Physical disability	1.74	1.32 to 2.30	1.40	1.02 to 1.91
Alcoholism	1.92	0.89 to 4.15	2.02	0.92 to 4.43
Mental diseases	0.74	0.50 to 1.09	0.82	0.53 to 1.27
Types of households
Others	Ref	R	Ref
Older	1.72	1.29 to 2.29	0.96	0.56 to 1.64
Disabled	1.54	1.06 to 2.23	1.49	0.95 to 2.35
Sick	1.08	0.75 to 1.56	0.98	0.66 to 1.46
Single parent	0.43	0.21 to 0.88	0.56	0.27 to 1.15

The CART analyses captured unique risk profiles which could not be identified by the regression analyses. The sensitivity analyses stratified by age also captured unique risk profiles (online supplemental file 3 and 4).

Discussion

In multiple regression analyses, among the recipients of public assistance in six municipalities in Japan, FOA was significantly prevalent among recipients who were living alone, unemployed, and certified to have a physical disability. Contrarily, the CART analysis extracted different risk profiles. The recipients were classified into 15 profiles through the CART analysis. Incidence proportions of FOA among each profile were calculated, and four groups were not attributed with FOA. The incidence proportion of FOA was the most significant(16.7%) among the subpopulation of the recipients who were aged≥58, working more than a day in a month, and certified to have a mental disability.

The regression analyses indicated that unemployment and living alone were identified as risk factors for FOA, which was consistent with our previous study using the data from two other municipalities.22 A recent qualitative study has reported that an older recipient who exhibited unnecessary FOA received the community-based intervention, which prevents social isolation and reduces the recipient’s FOA.28 Therefore, risk factors such as unemployment and living alone associated with social isolation might lead to unnecessary FOA. The CART analysis indicated that employed recipients certified to have mental disabilities in profile no. 1 were identified as the specific subpopulation exhibiting risk for FOA. This unique risk profile was not detectable as a risk in the conventional multivariable regression analyses. The qualitative research above also reported that recipients with a mental disability sometimes feel anxious about a mild change in their health, resulting in FOA.28 It would be essential to understand the reasons for recipients’ FOA and address their concerns. However, there are few types of research on FOA among public assistance recipients in Japan; further studies on risk profiles are warranted. Additionally, living alone was not captured in any of the high-risk profiles from the present CART analysis; that is, by using a CART analysis, we could statistically identify the specific risk population to care for that we were not able to capture by estimating individual variables. On the other hand, because risk profiles obtained by CART analysis depend on the dataset used and are sometimes overfit, it is challenging to transport the findings to recipients living in other municipalities.

The findings of this study have important policy implications. The CART model in our study could determine multidimensional high-risk and low-risk profiles and the number of people included in each profile. In the health management support programme for public assistance recipients in Japan, each municipality is required to analyse municipal public assistance data and design the intervention for the frequent attenders.20 Identifying the specific population at risk of FOA by using CART analysis can make opportunities to conduct further research to understand the phenomena of FOA by recipients more precisely using qualitative and quantitative methods. Based on the quantitative and qualitative evidence, policy-makers may be able to strategise effective interventions proportionately according to the degree of risks, the clinical importance of FOA, the number of people to be cared for, municipal cultural background, and available resources depending on the risk of FOA and the number of people in the profiles.29 By designing tailormade approaches according to the characteristics of the risk profiles not captured in risk factors identified by conventional regression analyses, policy-makers might prevent unintended consequences of treating FOA impartially through the recipients.

This study has three strengths. First, we identified essential profiles of recipients whose characteristics and behaviours were associated with the risk of FOA using the CART analysis. Second, the data collected did not contain any missing data, decreasing the risk of selection bias. Third, using public assistance databases, we could identify the risk factors and risk profiles of FOA among the socially vulnerable population, who were usually difficult to reach in traditional social surveys.

Alternatively, there are some limitations to this study. First, we did not evaluate some crucial factors, including the severity of diseases and treatments statuses for recipients, owing to which we could not evaluate whether their FOA was medically necessary or not. However, the purpose of our study to identify the risk profiles of FOA is achieved; municipal welfare offices can strengthen targeted intervention of formal care for recipients in need. Collecting further medical and socioeconomic data such as educational attainment and social relationships may improve the accuracy of our analysis and strengthen the effectiveness of the CART analysis. Second, the generalisability of our results was limited to suburban and rural municipalities in Japan. Our study may not represent the characteristics of recipients living in larger urban cities. Third, misclassification of explanatory variables may bias our findings. For example, if some recipients with severe mental health disorders were not certified readily to have a mental disability, the risk profiles of FOA may not have been adequately captured.

Conclusion

Using the CART model, which enables us to identify subpopulations consisting of multiple sociodemographic factors, we could determine specific risk profiles of FOA which may be overlooked in the conventional regression analyses among public assistance recipients. Additional research can identify other mechanisms of FOA by recipients in the high-risk profile. Policy-makers should focus on both the risk factors and the risk profiles to design population approaches and targeted preventive and supportive care in the health management support programme, not to induce unintended consequences of the policy. Further investigations, using more detailed information on medical and socioeconomic characteristics, may improve the accuracy of our findings and the effectiveness of the CART analysis.