Adherence to treatment for diabetes mellitus: validation of instruments for oral antidiabetics and insulin.

OBJECTIVES: to verify the face validity, criterion-related validity and the reliability of two distinct forms of presentation of the instrument Measurement of Adherence to Treatment, one being for ascertaining the adherence to the use of oral antidiabetics and the other for adherence to the use of insulin, as well as to assess differences in adherence between these two modes of drug therapy.
METHOD: a methodological study undertaken with 90 adults with Type 2 Diabetes Mellitus. The criterion-related validity was verified using the Receiver Operating Characteristic curves; and for the reliability, the researchers calculated the Cronbach alpha coefficient, the item-total correlation, and the Pearson correlation coefficient.
RESULTS: the oral antidiabetics and the other showed sensitivity of 0.84, specificity of 0.35 and a Cronbach correlation coefficient of 0.84. For the adherence to the use of insulin, the values found were, respectively, 0.60, 0.21 and 0.68. A statistically significant difference was found between the final scores of the two forms of the instrument, indicating greater adherence to the use of insulin than to oral antidiabetics.
CONCLUSION: it is concluded that the two forms of the Measurement of Adherence to Treatment instrument are reliable and should be used to evaluate adherence to drug treatment among people with diabetes mellitus.

Introduction

Adherence to treatment does not refer simply to the act of taking the medications; rather, it refers to how the person manages their treatment in relation to doses, times, frequency and duration(. In the context of chronic illnesses, in which the people and their relatives are responsible for the greater part of the care, adherence must be seen as a joint activity in which the person not only follows medical advice, but understands, agrees with and adopts the regimen described(.

Among the chronic non-transmissable diseases which are responsible for chronic health conditions, Diabetes Mellitus (DM) stands out, because of its epidemic proportions on the national( and global scale(, and the concept of adherence, for this disease, includes, as well as drug therapy, an individualized nutritional plan, regular physical exercise, and general care(. Non-adherence to the therapeutic regimen contributes to poor metabolic control, resulting in acute and long-term complications(.

Drug treatment for DM is complex and can involve different drugs with multiple dosages, as well as daily administration of exogenous insulin(. The rates of adherence, in general, vary from 31% to 98%(, and there is evidence that the more complex the therapeutic regimen, the lesser the adherence(. In this regard, the literature has shown that adherence to the use of insulin is lower than adherence to the use of oral antidiabetics (OADs), with rates between 36% and 80% and from 46.4% to 86%, respectively(. This also suggests that adherence behaviors may not be related to each other, that is, people can adhere to one aspect of the treatment, but not to the other(.

Another question which is relevant to adherence to treatment is how to measure it. It is difficult to determine an acceptable measurement of adherence due to this being a multi-dimensional concept, which can, moreover, be calculated by different methods, each with its advantages and limitations(. The method used most for assessing adherence to treatment has been structured questionnaires, due to their simplicity and low cost, the fact that one obtains results immediately, and the possibility of detecting approximately 50% of people who do not adhere to the treatment regimen(.

In Brazil, studies aiming to evaluate adherence to drug treatment for DM( and other chronic illnesses(17) have used the Measurement of Adherence to Treatments (MAT) questionnaire(, an instrument which is accessible and easy to administer. Because this questionnaire was not developed exclusively for people with DM, however, it does not specifically cover adherence to OADs and insulin. Furthermore, instruments were not identified in the literature specifically evaluating these two aspects of the treatment.

Therefore, the present study aimed to verify the face validity and criterion-related validity, and the reliability of the MAT instrument, presented in two forms referent to the evaluation of adherence to the use of OADs and insulin, distinctly, and to assess whether there are differences in the scores for adherence between these two modes of drug therapy.

Method

This is a cross-sectional and methodological study, undertaken in the outpatient department of a teaching hospital in the state of São Paulo (SP), in the period November 2010 - April 2011.

The study's sample was by convenience, selected through the weekly review of the medical records of the people scheduled for attendance with the health team, in line with the inclusion/exclusion criteria. The following were included: adults with Type 2 diabetes, of both sexes, being treated with insulin and OADs (monotherapy and/or in association with other therapies), who were able to maintain dialogs and who had an absence of chronic complications at advanced stages. People were excluded if they were receiving hemodialytic treatment, with amaurosis, had sequelae from cerebrovascular accidents/cardiac insufficiency, or were amputees or in wheelchairs. In addition to this, people unable to understand and/or verbalize responses to the interview questions were also withdrawn from the research.

During the period in which the study was carried out, 492 people were attended in the above-mentioned outpatient department. Of these, 136 initially met the inclusion criteria. 17 people refused to participate, and five were excluded for the following reasons: dependence on oxygen, inability to communicate verbally, and amputation. A further 24 people did not respond to the request to participate. As a result, the sample was made up of 90 adults with DM2, a number which made it possible to carry out an initial evaluation of the questionnaires' psychometric properties referring to the state of health, according to a systematic review study of quality criteria for evaluating this type of instrument's properties(.

People were invited to participate in the study verbally, in the waiting room, while they waited to be seen. After the presentation of the study's objectives, and clarifications on the anonymity of participation, the people were asked to indicate whether they agreed or not to participate in the study. Those who agreed were taken to a room set aside for data collection. The first item to be handed over was the Terms of Free and Informed Consent, which was read out loud by one of the researchers. Once it had been read, the participant was requested to sign it.

For the collection of socio-demographic and clinical data, a semi-structured instrument was used, developed based on a previous study(. The data referent to adherence to drug treatment was collected using the questionnaire Measurement of Adherence to Treatment (MAT)(. This instrument had been adapted and validated in Portugal, with good internal consistency (Cronbach alpha coefficient of 0.74)(, and was adapted for Brazilian Portuguese(. The MAT is a scale made up of seven items and is for evaluating adherence to prescribed treatment with medications. All the items present a response pattern which goes from "always" to "never", with scores varying, respectively, from one to six. The adherence is determined by the instrument's global mean, that is, the scores for each item are summed and divided by the number of items (seven). Higher means indicate greater adherence to the treatment(.

A previous study( using the MAT, in a sample of 162 people with DM2 under outpatient treatment, showed that the instrument has acceptable reliability (Cronbach alpha of 0.66), according to a review study on the analysis of the psychometric properties of instruments for evaluating subjective phenomena, which considered values over 0.50 as reasonable(.

Following the authors' agreement(, for the present study, this instrument's questions relating to the modes of treatment were reviewed, giving rise to two ways of presenting the MAT; that is, one for evaluating the adherence to the drug therapy with OADs, and another, with insulin, respectively titled "Measurement of Adherence to Drug Therapy in Diabetes Mellitus - Oral Antidiabetics" (MAT OADs) and "Measurement of Adherence to Drug Therapy in Diabetes Mellitus - Insulin Therapy" (MAT insulin).

In the MAT OADs, the word "medications " was substituted with "tablets" in all the items, apart from item 5 ("Have you taken one or more tablets for diabetes, on your own initiative, because of feeling worse?"), which did not need changing, remaining identical to how it was in the original version. It is believed that the use of the word "tablets", in all the items, may reflect a better standardization of the same.

Similarly, in the MAT insulin, the expression "take the medications" in items 1, 3, 4 and 7 was substituted with the expression "administer the insulin"; in item 2, the expression "time for taking the medications" was substituted with "time to administer the insulin"; in item 5, "taken one or more tablets" was substituted with "administered one or more units of insulin"; and in item 6, the expression "have let your medications run out" was substituted with "have let your insulin run out".

It should be emphasized that no other term of the items was altered, so as to maintain the similarity with the original version, and not to de-characterise the instrument, given that it had been validated for the Portuguese language.

For the face validity(, the two forms of presentation of the MAT were submitted for approval by three health professionals who work in the area of care for people with DM, and the evaluation of each item's relevance and clarity was requested. Following this, the researchers proceeded to the semantic analysis, the aim of which is to check if all the items are understandable for the target population(. It is recommended that the understanding of the items should be verified with few subjects. The items are presented one-by-one and the subjects are requested to re-phrase them. If this is done without doubts, the item is correctly understood. Otherwise, the researcher must explain what is meant by the item in question, after which the respondents are requested to make suggestions such that the item may be reformulated(. In this way, the two forms of the presentation of the MAT were administered to five people with this illness.

For the criterion-related validity, the researchers proceeded to the ROC (Receiver Operating Characteristics) Curves, considering the original of the MAT as the gold standard criteria.

The variables were submitted to the Komolgorov-Smirnov and Levene tests for verification, respectively, of the variables' normal distribution and homogeneity of the variances.

These instruments' reliability was evaluated through three methods: calculation of the Cronbach alpha coefficient for analysis of internal consistency; the item-total correlation and the Pearson correlation co-efficient. The strength of the correlations was ascertained in accordance with the following classification: weak (r<0.3), moderate (0.30.6)(.

The evaluation of the adherence was undertaken by measurements of central tendency (mean and median) and of the variability (standard deviation) of the final scores and of the items of each one of the two forms, and their comparison was undertaken through the test for comparison of means (paired Student t test). The level of significance adopted was 0.05. The data collected was stored in the MS-Excel program, with double-keying, and validation of the same. Later, the database was exported to the Statistical Package for the Social Sciences (SPSS) program, version 17.0, for uni- and bivariate exploratory analysis.

The present study was approved by the Research Ethics Committee of the Teaching Hospital of the Ribeirão Preto Faculty of Medicine, of the University of São Paulo, under process nº 9510/2010, and the mean length of the interviews was 61 minutes.

Results

Socio-demographic and clinical characterization, and treatment of the sample

In the socio-demographic characterization of the sample, the mean age and respective standard deviation (SD) was 60 years (SD=8.05). Of the 90 participants, 61 (67.8%) were female; 57 (63.3%) were married/cohabiting and 74 (82.2%) were from Ribeirão Preto and/or the surrounding region. Regarding occupation, there was a greater frequency of retired persons/pensioners (40%) followed by those who mentioned working in the home without payment (31.1%). The mean length of schooling was five years (SD=3.74).

Regarding the clinical and treatment-related variables, the time since diagnosis was, on average, 17.3 years (SD=8.0); the mean length of use of OADs was 10.9 years (SD=7.73), the daily frequency of use was, on average, 2.5 times per day (SD=0.64), and the most frequent class of medication (86.7%), irrespective of use in association with other OADs, was the biguanides.

Regarding the use of insulin, the mean length of use was 8.6 years (SD=6.61), and the daily frequency of administration was, on average, 2.1 times per day (SD=0.57). The most frequently prescribed insulin schemes were human insulin NPH (43.3%) and the association of this last with regular insulin (53.3%).

Face and criterion-related validity and reliability of the MAT OADs and the MAT insulin

In the face validity, the specialists judged the two ways of presenting the MAT to be relevant to the purpose of the study. Because the original version of the instrument had been previously adapted to Brazilian Portuguese, further changes were not made. In relation to the semantic analysis, there were no difficulties in the re-phrasing of the items and, therefore, no suggestions for changes were made.

Regarding the criterion-related validity, the MAT OADs presented an area under the curve of 0.83, sensitivity of 0.84, and specificity of 0.35. For the MAT insulin, the values were 0.77, 0.60, and 0.21 respectively. (Figure 1).

Figure 1

ROC Curves of the MAT OADs and the MAT insulin in the sample studied. Ribeirão Preto, SP, Brazil, 2011

In the reliability analysis, evaluated using the Cronbach alpha coefficient (α), values were obtained of 0.84 for the MAT OADs, and of 0.68 for the MAT insulin. In the MAT OADs, if item 2 is removed, the Cronbach alpha rises to 0.85, and in the MAT insulin, if this item is removed, it drops from 0.68 to 0.65. Remaining with the MAT insulin, the Cronbach alpha coefficient only increases (from 0.68 to 0.69) when item 6 is removed. In relation to the item-total correlations, values were obtained ranging from 0.38 to 0.77 for the MAT OADs, and from 0.20 to 0.53 for the MAT insulin (Table 1).

Table 1

Cronbach alpha coefficient (α) and item-total correlation of the Measurement of Adherence to treatment. oral antidiabetics (MAT OADs) and of the Measurement of Adherence to Treatment - insulin (MAT insulin). Ribeirao Preto, SP, Brazil, 2011

Items of the forms of Measurement of Adherence to Treatment (N = 90)	MAT OADs (α = 0.84)			MAT insulin (α = 0.68)
	Item-total correlation	Cronbach alpha if the item is removed		Item-total correlation	Cronbach alpha if the item is removed
1-Have you ever forgotten to take the tablets/ administer the insulin for the diabetes?	0.67	0.81		0.45	0.63
2-Have you ever been careless with the time for taking the tablets/administering the insulin for the diabetes?	0.53	0.83		0.41	0.65
3-Have you ever not taken the tablets/administered the insulin for the diabetes because of feeling better?	0.65	0.81		0.46	0.63
4-Have you ever not taken the tablets/administered the insulin for the diabetes, on your own initiative, because of feeling worse?	0.49	0.84		0.53	0.60
5-Have you ever taken one or more tablets/administered one or more units of insulin for the diabetes, on your own initiative, because of feeling worse?	0.38	0.85		0.38	0.65
6-Have you ever interrupted the treatment for diabetes because you had let the tablets/insulin run out?	0.76	0.80		0.20	0.69
7-Have you ever not taken the tablets/administered the insulin for the diabetes, for any reason apart from being so instructed by the doctor?	0.77	0.79		0.40	0.66

In analyzing the reliability by means of the Pearson correlation coefficient, a statistically significant correlation with a strong magnitude was found between the final scores of the two forms of the instrument. There were statistically significant correlations, of weak and moderate magnitude, between the items of the MAT OADs and the MAT insulin. The results referent to the reliability analysis, undertaken through the three calculations, suggest that the forms proposed are reliable (Table 2).

Table 2

Correlations between the items and the total of the Measurement of Adherence to the Treatment - oral antidiabetics (MAT OADs) and Measurement of Adherence to the Treatment - insulin (MAT insulin) scales, in the sample studied. Ribeirão Preto, SP, Brazil, 2011

Items from the forms of Measurement of Adherence to Treatment	r* (p-value)
Items 1	0.43 (0.000)†
Items 2	0.25 (0.002)†
Items 3	0.44 (0.000)†
Items 4	0.53 (0.000)†
Items 5	0.25 (0.020)†
Items 6	0.28 (0.000)†
Items 7	0.56 (0.000)†
Total of the Scales	0.77 (0.000)†

Pearson correlation

Statistical significance (p-value < 0.05)

In the analysis of the adherence, the item with the lowest mean in both forms of the instrument was item 2. The highest means were for 5 (MAT OADs) and 7 (MAT insulin) (Table 3).

Table 3

Description of the final scores and scores for each item in the Measurement of Adherence to the Treatment - oral antidiabetics (MAT OADs) and Measurement of Adherence to the Treatment - insulin (MAT insulin), in the sample studied. Ribeirão Preto, SP, Brazil, 2011

Items	Forms of the Instrument
	MAT OADs (N=90)			MAT insulin (N=90)
	Mean (SD)	Median (interval)		Mean (SD)	Median (interval)
1	5.06 (1.27)	6.00 (1.00-6.00)		5.50 (0.77)	6.00 (4.00-6.00)
2	4.87 (1.37)	5.00 (1.00-6.00)		5.04 (1.04)	5.00 (2.00-6.00)
3	5.62 (1.02)	6.00 (1.00-6.00)		5.71 (0.71)	6.00 (2.00-6.00)
4	5.27 (1.44)	6.00 (1.00-6.00)		5.71 (0.77)	6.00 (1.00-6.00)
5	5.71 (0.92)	6.00 (1.00-6.00)		5.57 (0.89)	6.00 (1.00-6.00)
6	5.56 (1.03)	6.00 (1.00-6.00)		5.80 (0.66)	6.00 (1.00-6.00)
7	5.61 (1.08)	6.00 (1.00-6.00)		5.89 (0.41)	6.00 (4.00-6.00)
Total of the scale	5.39 (0.84)	5.71 (1.71-6.00)		5.60 (0.45)	5.71 (4.00-6.00)

In undertaking the paired Student t-test, a statistically significant difference was found between the final scores of the two forms of the instrument, indicating greater adherence to the use of the insulin than to the OADs. In comparing the means of the items of the MAT OADs with the MAT insulin, there were also statistically-significant differences in items 1, 4, 6 and 7 (Table 4).

Table 4

Comparison of the means of the final scores and of each item in the Measurement of Adherence to the Treatment - oral antidiabetics (MAT OADs) and of the Measurement of Adherence to the Treatment - insulin (MAT insulin), in the sample studied. Ribeirão Preto, SP, 2011

Items	Forms of the Instrument			Statistical value*	p-value
	MAT OADs (N=90)		MAT insulin (N=90)
	Mean (SD)		Mean (SD)
1	5.06 (1.27)		5.50 (0.77)	3.628	0.000†
2	4.87 (1.37)		5.04 (1.04)	1.127	0.263
3	5.62 (1.02)		5.71 (0.71)	0.882	0.380
4	5.27 (1.44)		5.71 (0.77)	3.459	0.000†
5	5.71 (0.92)		5.57 (0.89)	-1.238	0.219
6	5.56 (1.03)		5.80 (0.66)	2.203	0.030†
7	5.61 (1.08)		5.89 (0.41)	2.889	0.005†
Total of the scale	5.39 (0.84)		5.60 (0.45)	3.578	0.001†

Statistical value: paired Student t-test

Statistical significance (p-value < 0.05)

Discussion

In relation to the socio-demographic characterization, the sample was made up of adults of low schooling and low incomes, retired, with a mean age of 60 years old, there being a majority of women (67.8%). These results are similar both to the original study in which the instrument was developed(, and to those in which this instrument was adapted for Brazil(.

In the criterion-related validity, the area under the curve was 0.83 and 0.77 for the MAT OADs and the MAT insulin, respectively. The area under the ROC curve is a measurement of the performance of a test, that is, it determines its exactitude. A test which is totally incapable of distinguishing adherent people and non-adherent people would have an area under the curve of 0.5, which corresponds to the null hypothesis. Areas over 0.70 are considered satisfactory(.

Regarding internal consistency, the Cronbach alpha values found were 0.84 and 0.68 for the MAT OADs and MAT insulin, respectively. Internal consistency is one of the ways of evaluating an instrument's reliability(, as it indicates the extent to which an instrument's items are homogeneous, that is, are inter-correlated, thus measuring the same concept. The Cronbach alpha is considered a good measurement of internal consistency. Its value should vary from 0.70 to 0.95. This coefficient is sensitive to the instrument's number of items, and for this reason higher values are commonly found in instruments with a greater number of items(.

It may be observed that although the MAT instrument has only seven items, the value of the Cronbach alpha coefficient is found within an interval considered satisfactory by the literature(. For the MAT insulin, this coefficient had a value below 0.70; however, according to the review study on analysis of the psychometric properties of instruments for assessing subjective phenomena, values over 0.50 are considered acceptable(. It may also be observed that the two forms proposed in the present study present Cronbach alpha coefficients which are superior to those of the study of the adaptation of the MAT for the use of oral anticoagulants, where the value found was 0.60(.

When the alpha value was evaluated with the removal of each one of the items, a variation from 0.79 to 0.85 was obtained for the MAT OADs, and from 0.60 to 0.69 for the MAT insulin. In the original study in which the instrument (MAT original) was developed, the variation was from 0.69 to 0.73(. The variation in the alpha value of the MAT OADs and MAT insulin was mildly greater than that of the MAT original. This data suggests that the items of these two forms of presentation of the instrument are as homogeneous as those of the original instrument. However, it is worth noting that the removal of the items did not significantly influence the Cronbach alpha values of the forms of presentation proposed in this study. In addition to this, the values obtained for the total of each scale are considered satisfactory and, for this reason, the instrument's seven items have been maintained.

The item-total correlation is another measurement of an instrument's reliability(. In the present study, the values of the item-total correlations varied from 0.38 to 0.77 for the MAT OADs and from 0.20 to 0.53 for the MAT insulin. It is desirable for the item-total correlation to be superior to 0.20. On the other hand, a high item-total correlation (> 0.80) can be indicative of redundancy(. It may be observed that the item-total correlations were equal to or superior to 0.20, in the forms of presentation, suggesting that the administration of this instrument, separately, is reliable.

In the evaluation of the reliability using the Pearson correlation, the final scores of the two forms presented correlations of strong magnitude and which were statistically significant between each other, and each form's items presented correlations which were statistically significant between each other, of weak and moderate magnitude. These findings suggest that the MAT OADs and MAT insulin are as reliable for evaluating the behavior of adherence as the MAT original.

Regarding adherence, the item with the lowest mean in both the forms of the instrument was item 2. This result suggests that in regard to the use of both OADs and insulin, the majority of the participants show the least adherence in regard to the time of day at which they use these medications. The items with the highest means were 5 (MAT OADs) and 7 (MAT insulin). This data shows that, for the OADs, the majority of the participants present adherence to the dosage prescribed, and that in relation to insulin therapy, the majority do not interrupt the use of insulin unless told to do so by their doctor.

The final scores, on average, were 5.39 (SD=0.84) and 5.60 (SD=0.45) for the MAT OADs and MAT insulin, respectively. A statistically significant difference was observed between the means of the two forms' final scores, with a higher mean for the MAT insulin, indicating greater adherence for this mode of treatment. This result differs from that of the systematic review study, which found a lower percentage of adherence among people under insulin therapy, when compared with those using OADs (73%; 86% respectively)(. The greater adherence to insulin than to OADs in the sample studied may be attributed to the long time that the people involved had DM, as well as to the presence of its complications. In addition to this, the risk of hypoglycemia, associated with the use of insulin(, can make the person administer the correct dose of this hormone.

When the means of the items between the two forms were compared, there were statistically significant differences in items 1, 4, 6 and 7, which shows greater adherence, in these items, for the MAT insulin.

These last results suggest that in dealing with the use of OADs associated with insulin, instruments should be used which evaluate the adherence to these modes of treatment separately, because a single instrument cannot capture these differences. It is possible that in the sample studied, had a single instrument been used, adherence to the OADs could have been overestimated, due to the fact of the adherence to insulin being higher.

Conclusion

The present study's results allow one to consider that the two instruments, derived from the original version of the MAT, present psychometric properties which characterize them as reliable.

The two forms of the presentation of the MAT made it possible to detect differences in adherence between the modes of drug treatment, showing there to be greater adherence to the insulin therapy than to the OADs - which might not be detected, were a single instrument to be used. It is therefore emphasized that for people with DM, separate instruments should be used to evaluate the adherence to treatment with OADs and to treatment with insulin.

Some weak points, however, deserve to be considered. The small sample size may possibly not permit clinical validation of the instruments proposed, and may have limited the evaluation of their psychometric properties. The non-existence of instruments for assessing adherence to the two modes of drug therapy for DM separately both required the adaptation of an already-existing questionnaire, and made it difficult to compare the results.

As a result of this, the replication of studies of this nature is recommended, with better methodological design, and in larger populational samples, so as to extend the psychometric analyses.