Development and Validation of an Attitudinal-Profiling Tool for Patients With Asthma.

PURPOSE: To develop a profiling tool which accurately assigns a patient to the appropriate attitudinal cluster for the management of asthma.
METHODS: Attitudinal data from an online survey of 2,467 patients with asthma from 8 Asian countries/region, aged 18-50 years, having had ≥2 prescriptions in the previous 2 years and access to social media was used in a discriminant function analysis to identify a minimal set of questions for the Profiling Tool. A split-sample procedure based on 100 sets of randomly selected estimation and validation sub-samples from the original sample was used to cross-validate the Tool and assess the robustness of its predictive accuracy.
RESULTS: Our Profiling Tool contained 10 attitudinal questions for the patient and 1 GINA-based level of asthma control question for the physician. It achieved a predictive accuracy of 76.2%. The estimation and validation sub-sample accuracies of 76.7% and 75.3%, respectively, were consistent with the tool's predictive accuracy at 95% confidence level; and their 1.4 percentage-points difference set upper-bound estimate for the degree of over-fitting.
CONCLUSIONS: The Profiling Tool is highly predictive (>75%) of the attitudinal clusters that best describe patients with asthma in the Asian population. By identifying the attitudinal profile of the patient, the physician can make the appropriate asthma management decisions in practice. The challenge is to integrate its use into the consultation workflow and apply to areas where Internet resources are not available or patients who are not comfortable with the use of such technology.

INTRODUCTION

Asthma is a common chronic respiratory disease. As of 2014, it was estimated to have afflicted 334 million people worldwide.1 This was an increase of 100 million new sufferers within a decade from 2004 when there were 234.9 million sufferers. In that same year, the South-east Asia and Western Pacific regions accounted for 45.5% (45.7 and 61.2 million, respectively) of all sufferers.2 The chronic nature of the disease and the large number of sufferers can present a major economic and social burden for countries in these regions.3 Asian studies of patients with asthma, which were conducted through face-to-face or phone interviews, addressed the burden of disease on the patient and their reactions to exacerbations caused by the disease.4567 The 2014 REcognise Asthma and LInk to Symptoms and Experience (REALISE) Asia8 survey of patients with asthma, instead, focused on their perceptions of asthma control, and attitudes toward the disease and its treatment in an accessible, real-life adult Asian population. It showed that many patients perceived their asthma to be controlled and not serious despite the presence of symptoms and exacerbations. A second REALISE Asia survey of physicians who treated patients with asthma showed that physicians also over-estimated the level of control among their patients.910 Physician-patient follow-up consultations were short (average <10 minutes) and usually verbal, with limited use of printed material. Real-life studies also showed that asthma control, as defined by the Global Initiative for Asthma (GINA)11 criteria, was poor and highlighted a complex interaction of disease, doctor, patient and environment-related factors.8121314 As a result, the optimal levels of asthma control achieved in clinical studies could not be replicated in real-life studies.15

Some tools are available to assist the physician in assessing the current level of asthma control and quality of life of the patient include the Asthma Control Test (ACT)16 and Asthma Control Questionnaire (ACQ).17 However, the ability of a patient to cope with a chronic disease, such as asthma, is influenced not only by its symptoms but also his personality profile and behavioral characteristics.181920 Thus, physicians should consider the patient's psychological well-being and attitudes toward his/her disease besides treating the clinical symptoms of asthma as these can have an important bearing on the eventual treatment outcome.18192122 Indeed, Axelsson et al.23 have shown that personality traits can influence how asthma patients adhere to asthma treatment and report their asthma control and quality of life. Unfortunately, tools that can help physicians identify the attitudinal profile of their patients and the treatment approach appropriate to this profile are not readily available. Jones et al.24 proposed a tool that can be used by physicians to compute the risk of having uncontrolled asthma based on patient attitudes toward medical professionals and the treatment for their asthma. However, to our knowledge, the development of a tool that can segment the population into distinct attitudinal profiles and assist in personalizing and tailoring management approaches has not been previously attempted for asthma patients.

In our REALISE Asia patient study (the results of which have been published8), an online survey was conducted among eligible respondents from validated25 consumer panels in China, Hong Kong Special Administrative Region, Indonesia, South Korea, Malaysia, Philippines, Singapore, and Taiwan who used social media. The survey focused on patients' perceptions of asthma control, and their attitudes toward the disease and its treatment in an accessible, real-life adult Asian population. The attitudinal data, collected from 2,467 respondents aged 18-50 years who suffered from the disease, were analyzed and used to identify attitudinal control groupings or 'clusters' for patients with asthma. A factor analysis (also published26) was used to summarise responses to 27 attitude-related question items from the survey questionnaire into a set of 9 attitudinal factors, and established the linkage between these 9 factors and the extent to which they are correlated. Then, through a cluster analysis, these respondents were grouped into clusters based on their ratings against these 9 factors. Each cluster described patients who were similar in their attitudes toward asthma but distinct from those in the other clusters. The result was 5 attitudinal clusters: Cluster 1 (Well-adjusted and at least partly controlled), Cluster 2 (In denial about symptoms), Cluster 3 (Adrift and poorly controlled), Cluster 4 (Tolerating with poor control), and Cluster 5 (Worried with multiple symptoms) which encompassed the whole spectrum of patient attitudes toward their asthma condition (Fig. 1).26

Fig. 1

Patient attitudinal clusters.

This manuscript describes the development and validation of a profiling tool that will identify which 1 of the 5 attitudinal clusters identified above, a patient with asthma belongs. The knowledge of the cluster that a patient belongs to is expected to help healthcare professionals tailor the treatment approach to improve its outcome. At the time of this manuscript, there is an on-going study by an international panel of clinicians and allied health professionals working from Europe and Asia to generate consensus asthma management recommendations for each of these attitudinal clusters.27

MATERIALS AND METHODS

We adopted a 2-step methodology: (1) to develop a Profiling Tool using a minimal list of question items from the REALISE Asia survey questionnaire that will accurately assign a patient to the correct attitudinal cluster, and (2) to validate this Profiling Tool using a split-sample procedure to confirm that its predictive accuracy can be achieved in actual practice.

Development phase

Subjects

The REALISE Asia patient survey was done through email invites to a random sample of individuals from validated consumer panels between December 2013 and March 2014 asking them to complete a 30-minute online questionnaire.25 The survey included individuals aged 18-50 years who were diagnosed with asthma, given ≥2 asthma-related prescriptions in the past 2 years (to ensure that individuals have active asthma) and had used social media. The following respondents were excluded from the survey: (1) those who were employed or had relatives employed in the healthcare, pharmaceutical, advertising or market research industries; and (2) those who had taken part in healthcare-related market research in the past 3 months. The survey flowchart is reproduced and shown in Fig. 2.8

Fig. 2

REALISE Asia patient survey flowchart.

Item generation

The initial question set for the Profiling Tool consisted of the 27 attitude-related questions from the survey questionnaire which were used for the factor analysis to determine the 5 patient attitudinal clusters26 (Supplementary Material, highlighted in blue, green, and yellow). In the survey, respondents were asked on their degree of agreement, i.e., 'strongly disagree,' 'disagree,' 'tend to agree,' and 'strongly agree,' to various statements about their asthma. However, we decided to dichotomize (agree/disagree) their responses to take into account the possible variation in response styles between countries due to their cultural differences.

Item reduction

We set a range of 4-12 items as a viable number of questions that patients would be willing to answer in the Profiling Tool, which would also accurately identify a patient's cluster membership. As the number of combinations of 4 to 12 items that can be selected from the 27 attitude-related questions was very large (>120 million), we used a multinomial logistic regression technique to exclude question items that were deemed to be poorer predictors of cluster membership on the basis of their Wald statistics (<35) to reduce this number. Starting from question items with the highest Wald statistics, the smallest subset was drawn such that it contained at least 1 item from each of the 9 attitudinal factors which was used to obtain the attitudinal clusters.26 The selected subset consisted of 14 question items (namely, Q2: a3, a6, a9; Q34a2; Q39: a2, a6; Q46: a1, a2; and Q48: a2, a3, a5, a6, a7, a11; Supplementary Material, highlighted in green and yellow).

Discriminant function analysis (DFA) was used to determine whether this selected subset of question items (predictors) was effective in predicting the cluster membership of a given case.28 The output from the DFA was a set of equations; 1 for each cluster. Each equation contained the predictors and had a weight associated with each of them. The equation with the highest score for a case's set of predictor values was that with the highest probability of being the cluster that the case belonged to. Besides the 14 question items which were to be answered by the patient, the level of asthma control (i.e., well-controlled, partly controlled, and uncontrolled) using the GINA-based Helping Asthma in Real-life Patients initiative (iHARP)29 criteria was added to provide an input from the physician. Three combinations using the question items and the iHARP classification for the level of asthma control were considered: (1) including the iHARP classification as a single co-variate; (2) using the individual iHARP items as co-variates; and (3) excluding the iHARP classification altogether. We ran the DFA to test several formulations for each of these combinations, each time choosing 12 or fewer question items from the 14 items available to satisfy our desired range of 4-12 items. The objective was to find the formulation with the lowest misclassification rate or conversely, the highest accuracy rate i.e., the predicted cluster compared to the attitudinal cluster of the case which was known at the onset. We applied a rule-of-thumb of 75%3031 as the minimum accuracy to find the best formulations.

Validation phase

We used a split-sample procedure to validate the performance of our Profiling Tool. The original sample (2,400 cases) was divided into 2 sub-samples: one for estimating the predictive model and the other validating the estimated model. This type of cross-validation technique is widely accepted as one that usefully assesses how the results of a predictive model can be generalised to an independent data set.32 We used a repeated 2-fold random sub-sampling method to create independent sub-samples where each case had a 0.50 probability of being in the 'estimation' or 'validation' sub-samples. This meant that the 'estimation' sub-sample would be approximately the same size (between 1,146 and 1,259 cases) as the 'validation' sub-sample.

We repeated the sub-sampling process 100 times. This was a compromise between (1) identifying the discriminant functions for each of the 100 estimation sub-samples and profiling the validation sub-sample using that set of discriminant functions which is time-consuming; and (2) having as many independent estimates of the discriminant functions as possible to increase the overall accuracy of the predictive model. With 100 estimates, we believed that we would have a sufficiently robust estimate of the 95% confidence intervals around the prediction.

We then ran the DFA on each of the 100 estimation sub-samples. We used the resultant set of discriminant functions to assign the cases in the corresponding validation sample to their clusters. The outcome measure is the accuracy rate of the discriminant functions in assigning the estimation and validation sub-samples to their attitudinal cluster, which was known at the onset.

RESULTS

Most (62.3%) of the respondents were in the 26-40 years age groups. Both sexes were almost equally represented, with males forming 53.7% of the population. At the country level, the percentage of males ranged from a low of 41.1% for Malaysia to a high of 60.0% for Korea. They also accessed social media frequently with 85.0% of them using it 1 or more times daily. About 1 in 5 (17.8%) of the population had controlled asthma using the GINA-defined control status.11 Fig. 3 shows the demographic, socio-economic, and disease profiles of the respondents by country and for the population as a whole. These profiles were similar across the countries except for the reported household occupation in China where there was a higher proportion of respondents in the intermediate managerial category. For the distribution of the attitudinal clusters, Clusters 1 and 4 had the highest (28.9% and 29.0%, respectively) representation, while cluster 5 the least (11.3%). Fig. 4 shows the cluster distribution pattern for respondents in each of the surveyed countries and the total population. These cluster profiles were similar to each other.

Fig. 3

Demographic, socio-economic, and disease profiles by country.

Fig. 4

Attitudinal cluster profile by country.

Fig. 5 shows the accuracy rates for the DFA model which used the full sample of 2,400 cases from the REALISE Asia survey and had complete data for the question items of the Profiling Tool. The set of formulations using the iHARP classification as a single co-variate produced solutions with the best accuracy rates ranging from 70.1% to 77.5% depending on the number of question items used. The 11-predictor variables model (Fig. 5, circled brown); consisting of 10 question items (Supplementary Material, highlighted in yellow), and iHARP classification as a single variable was selected as the formulation that had the highest accuracy (77.2%, benchmark: 75.0%) while using a minimal number of question items. The question items in this formulation were applied to the Profiling Tool in the same sequence as in the original survey questionnaire so as to retain the order effect among the questions.

Fig. 5

Predictive accuracy rates corresponding to the number of predictor variables for iHARP variable combinations.

Final adjustments

We re-arranged the order of the questions by bringing forward the 'stressful life' question item (Q46a2) as it is related to the respondent's general situation rather than concerning their asthma specifically or its treatment and by grouping the inhaler-related questions together (Q39a6, Q48a2-3). We excluded Q39a2 (My doctor doesn't understand my asthma) as we felt that it was a leading question on the patient-doctor relationship and replaced it with Q2a3 (I have no time to think about my health as other things are more important). The final list of questions was marginally less efficient (76.2% vs. 77.2%) (Table 1).

Table 1

List of questions in the Profiling Tool

S/No.	Question Number*	Question	Response options
1	Q2a3	I have no time to think about my health as other things are more important	Agree/Disagree
2	Q2a6	I see myself as healthy and fit	Agree/Disagree
3	Q46a2	If someone asked me, I would say I had a stressful life	Agree/Disagree
4	Q2a9	I worry about what my asthma will be like in 10 years	Agree/Disagree
5	Q34a2	My asthma symptoms are not serious	Agree/Disagree
6	Q39a6	I find my inhaler difficult to use	Agree/Disagree
7	Q48a2	I feel embarrassed carrying my asthma inhaler around with me	Agree/Disagree
8	Q48a3	I feel embarrassed using my asthma inhaler in front of others	Agree/Disagree
9	Q48a6	My asthma stops me living life to the full	Agree/Disagree
10	Q48a7	My asthma affects my sex life	Agree/Disagree
11	-	Please enter patient iHARP classification (for doctor use only)	Controlled/Partially controlled/Uncontrolled

*Question number as in patient survey questionnaire. See also Supplementary Material.

Fig. 6 shows the predictive accuracies of the 100 validation runs, ordered independently by the estimation and the validation sub-samples, from the most to the least accurate. The blue line shows the accuracy rates of the estimation sub-samples, each with an error bar (also in blue) denoting the 95% confidence intervals for that accuracy rate. The accuracy rates ranged from 74.8% to 78.7% (average 76.7%) with reference to the original attitudinal cluster of each case. The ordered accuracy rates for the validation sub-samples, which ranged from 72.3% to 78.3% (average 75.3%), were about 1.4 percentage points lower compared to the ordered rates for estimation sub-samples (Fig. 6, red line). The 76.2% accuracy line (Fig. 6, green line) of the Profiling Tool lies within the 95% confidence intervals of all accuracy rates except for 1 estimation and 7 validation sub-samples.

Fig. 6

Ranked accuracy rates of estimation and validation sub-samples.

The scatter plot of the accuracy rates of the estimation and validation sub-samples for the 100 validation runs shows an inverse relationship whereby the more accurate the estimation model gets, the less accurate the validation sub-sample estimates becomes (Fig. 7). The point where the accuracy rates of the validation and estimation sub-samples are approximately equal was 76.0% (Fig. 7, dashed lines). Table 2 shows the accuracy rates by cluster for the Profiling Tool and the estimation and validation sub-samples. Cluster 1 had the highest accuracy rates (90.7%-91.1%). The accuracy rates for the other clusters ranged from 66.3% to 73.4%.

Fig. 7

Scatter plot of accuracy rates of validation sub-samples to estimation sub-samples.

Table 2

Accuracy rates of the Profiling Tool, validation sub-sample, and estimation sub-sample by cluster

Cluster	Estimation sub-samples		Validation sub-samples		Profiling Tool
	No.	Accuracy rate (%)	No.	Accuracy rate (%)	No.	Accuracy rate (%)
Cluster 1	35,157	91.1	35,043	90.7	702	90.7
Cluster 3	15,916	73.4	15,884	70.9	318	72.0
Cluster 4	34,745	70.5	34,555	68.9	693	70.6
Cluster 2	20,739	70.6	20,961	69.3	417	69.5
Cluster 5	13,476	68.4	13,524	66.3	270	67.8
All clusters	120,033	76.7	119,967	75.3	2,400	76.2

DISCUSSION

This study was the first attempt to develop a patient attitudinal-profiling tool that can help physicians tailor the asthma management plan for their patients at the point-of-care. Besides the physician assessment of the disease condition, the patient's attitude toward the disease was concurrently evaluated as it has an important influence on his/her adherence to the management plan and treatment outcome.33 The Profiling Tool contained 10 simple 'agree/disagree' questions for self-completion by the patient, and a question on the level of asthma control for completion by the attending physician. We believed that the small number of questions and simple response structure (agree/disagree) would make the tool easy for patients to use.

We recognized that there could be differences in the socio-economic profiles of respondents from the countries in the survey that may limit the effectiveness of the tool. Such differences can be expected of a survey spanning a number of countries. However, the profile plots for a number of socio-economic and disease factors showed that they were, by and large, similar across the surveyed countries and the total population. There were no indications of bias for these factors in the respondent population of any country. The cluster representation of the respondents in each of the surveyed country matched the total population. We believe that respondents who had access to social media may be similar across the surveyed countries, especially those from the 26-40 years age groups and who used such media frequently, and contributed to the uniformity seen in these profiles. This may place a limitation on the use of the tool beyond the tech-savvy population of the urban centers.

The validation results showed that the discriminant functions for the Profiling Tool were robust and accurate in assigning a case to one of the 5 attitudinal clusters. Although the predictive model from each of the estimation sub-samples used a smaller number of cases (about 1,200 cases) compared to the Profiling Tool (2,400 cases), its accuracy rate in assigning the cases in the validation sub-sample was as good as the tool. These validation sub-samples were sufficiently large and could optimally detect differences in the predictive accuracy of the estimation models. However, there were variations in the accuracy rates (67.8%-90.7%) at the cluster level, Cluster 5 (worried with multiple symptoms) being the lowest and Cluster 1 (well-adjusted and at least partly controlled) the highest. The well-adjusted patients were apparently more consistent in their responses. While the tool was most effective in identifying the well-adjusted patients, it remained sufficiently effective for the other clusters. The lowest accuracy rate of 67.8% exceeded the 20.0% level (i.e., 1 in 5 clusters) that would be expected by chance. Importantly, the tool could identify patients in the most urgent need of attention i.e., those who were tolerating (Cluster 4, 70.6%) or were in denial (Cluster 2, 69.5%) of their poor asthma condition, in 7 cases out of 10.

An inherent limitation to our method was how to ensure that the sample from which we used for the Profiling Tool, which came from a population aged 18 to 50 years with access to social media, was not peculiar or different from future samples. In particular, the management of asthma in the elderly presents its own challenges.34 The parameters derived by the DFA maximized the distinctions between the clusters (i.e., 'patient attitudinal types'). The key factor for the stability of these parameters was that they were not specific to our sample but were also applicable to other samples. In our survey, the cross-validation technique used to address this issue by randomly and repeatedly drawing the sample cases from patient populations in all the 8 countries/area into the 2 sub-samples proved that the derived parameters were not susceptible to the patient composition i.e., country, age, gender, or disease status in the sample. Had some data points of a sub-sample been peculiar, they would affect the tool's predictability and lower its accuracy rate. Low accuracy rates would have meant that our Profiling Tool could not be generalized to other samples. Our results showed that this was not the case. The accuracy rates of the estimation and validation sub-samples (76.7% and 75.3% respectively) were comparable to the accuracy of the discriminant functions operating in the Profiling Tool (76.2%) in assigning the cases to the correct clusters. On average, the tool correctly fitted 3 in every 4 cases: an accuracy rate exceeded the expected 20.0% level and met our 75.0% benchmark target.3031 This reinforced our view that the discriminant functions used in the Profiling Tool were generalizable beyond our population of 2,400 cases.

The relationship between the accuracy rates of the estimation and validation sub-samples exhibited an inverse relationship whereby the more accurate the estimation model, the less accurate the validation sub-sample model estimate. The 76.0% point where the 2 accuracy rates were equal was similar to the accuracy rate of 76.2% of the tool, which suggested that the discriminant functions for the Profiling Tool have not been over-fitted. Furthermore, the difference between the accuracy rates of estimation and validation sub-samples of 1.4 percentage-points provided an upper-bound estimate for the degree of over-fitting that we can expect using the original 2,400 cases to estimate the discriminant functions as the larger sample (2,400 vs. 1,200) would have generated a smaller confidence interval.

Recommendation

The patient can complete the questions in this Profiling Tool at the waiting room prior to the consultation. The attending physician can input his assessment of the level of asthma control of the patient during the consultation. The tool then computes the weighted scores using the responses from the patient and physician to the question items and our discriminant functions, and displays the attitudinal cluster which describes the patient best. The physician can tailor the asthma management plan for the patient using this knowledge of the attitudinal cluster.

While such a profiling tool can be made readily available to physicians via the Internet, the challenge would be to integrate its use in clinical practice, especially in areas where Internet resources are not available or patients are not comfortable with the use of such technology. To use this tool outside the surveyed countries, we suggest that a baseline survey be conducted to validate the Profiling Tool.

CONCLUSION

Our results showed that our Profiling Tool could identify which attitudinal cluster a patient with asthma belongs to and enable the physician to make appropriate asthma management decisions in practice.