Development and validation of a questionnaire to assess carbohydrate and insulin-dosing knowledge in youth with type 1 diabetes.

OBJECTIVE The American Diabetes Association advocates insulin regimens for youth with type 1 diabetes that involve adjusting insulin dose based on carbohydrate intake and blood glucose level. Implementing these regimens requires knowledge about carbohydrate content of foods and subsequent calculations of insulin dose, skills that may be difficult to gauge in practice. Therefore, we sought to develop and validate a questionnaire, the PedCarbQuiz (PCQ), to assess carbohydrate and insulin-dosing knowledge in youth with type 1 diabetes. RESEARCH DESIGN AND METHODS After development by an expert panel, the PCQ was administered to 75 youth with type 1 diabetes or their parents. Reliability was assessed by Cronbach alpha and split-half testing. To assess validity, scores were correlated with A1C, expert assessments, parent educational level, and complexity of insulin regimen. RESULTS PCQ mean score was 87 +/- 9.7% (range 42-98%). Cronbach alpha was 0.88, and correlation of split halves was 0.59 (P < 0.0001). Higher PCQ scores correlated significantly with lower A1C (r = -0.29, P = 0.01) and expert assessments (r = 0.56, P < 0.001). Scores were significantly higher in parents with college degrees than in those without (P = 0.01) and in participants with more complex insulin regimens (P = 0.003). CONCLUSIONS The PCQ is a novel, easily administered instrument to assess knowledge about carbohydrates and insulin dosing calculations. Initial analyses support the reliability and validity of the PCQ.

The Diabetes Control and Complications Trial established that tight glycemic control through intensive treatment regimens reduces the risk of long-term microvascular complications of type 1 diabetes (1). To help optimize glycemic control in youth with type 1 diabetes, the American Diabetes Association therefore advocates intensive insulin regimens in which insulin doses are calculated based on carbohydrate intake and blood glucose level (2,3). Such regimens can provide greater flexibility and improve metabolic control but require an understanding of carbohydrate counting and multiple calculations, which are potentially subject to error (4). Successful implementation of flexible regimens requires the ability to count carbohydrates and calculate insulin dose correctly before optimal adherence and tight control can be achieved.

There are currently no standardized measures to assess capacity to count carbohydrates and calculate insulin dose for youth with type 1 diabetes. Measures to assess diabetes regimen adherence have been developed (5–8) but do not assess knowledge and capacity to implement a regimen, which are prerequisites to adherence. Instruments are available to assess general diabetes knowledge (9–11) but are designed for adults and focus on topics including diabetes complications, smoking, and footcare, which are not directly related to implementing flexible insulin regimens. They briefly address general diet and insulin use but, because they were developed before the emergence of flexible regimens, do not address carbohydrate counting and insulin dose calculation skills/knowledge needed to enact basal-bolus regimens.

The Diabetes Numeracy Test is a recently developed instrument that measures numeracy skills for diabetes, including food label interpretation and calculation of insulin dose based on blood glucose and carbohydrate corrections (12). However, the Diabetes Numeracy Test does not address carbohydrate food recognition, carbohydrate food counting, or the incorporation of carbohydrate counting in calculating insulin dose. In addition, it was validated in adults but not children.

Our objective in this study was to develop and validate a questionnaire, the PedCarbQuiz (PCQ), to assess carbohydrate and insulin-dosing knowledge for youth with type 1 diabetes.

RESEARCH DESIGN AND METHODS

Development of the PCQ

To develop the PCQ, content analysis was conducted by a panel of 14 experts including six faculty pediatric endocrinologists, three pediatric endocrinology fellows, three certified diabetes educators, one pediatric nutritionist, and one adult diabetologist. Seven domains (four carbohydrate knowledge domains and three insulin-dosing domains) were identified as being necessary for successful implementation of flexible basal-bolus regimens. The content domains were 1) recognition of carbohydrates, 2) carbohydrate counting in individual food items, 3) carbohydrate counting in whole meals, 4) nutrition label reading, 5) use of insulin dose correction based on blood glucose level, 6) use of insulin-to-carbohydrate ratio in insulin dosing, and 7) calculation of whole meal insulin dose. Food items were drawn from logs of food intake for 1 full day from 21 patients to ensure that realistic dietary choices were represented. Items for each domain were reviewed and revised by the expert panel. Domains contained 3–4 items each, except for the recognition of carbohydrates domain, which contained 18 items. Readability of the test questions, measured by Flesh-Kincaid grade level, is 6th grade. The PCQ was pretested in 13 youth with type 1 diabetes or their parents for clarity and ease of administration. The final PCQ is a 78-item, self-administered, multiple-choice, paper-based questionnaire requiring 20–30 min to complete. The 78-item PCQ was constructed to contain two similar 39-item sets. Higher scores indicate greater degree of knowledge about carbohydrates and insulin-dosing ability. The PCQ is found in an online appendix (available at http://care.diabetesjournals.org/cgi/content/full/dc09-0390/DC1).

Participants

This study was approved by the institutional review board of the University Hospitals of Cleveland. Subjects were recruited from a university-based, multisite pediatric endocrinology practice. Families of children of any age with type 1 diabetes received a letter describing the study and were approached sequentially by telephone before the visit or in person during the regularly scheduled clinic visit. When the patient was aged <12 years, parents were invited to participate; for patients aged ≥12 years, either the parent or adolescent was invited to participate, based on the family's report of which individual had primary responsibility for insulin dosing. Patients with type 1 diabetes of duration exceeding 1 year and parent participants living with the identified child for at least 1 year met the criteria for inclusion. The exclusion criterion was lack of English fluency.

Procedures and measures

After consent/assent, data were obtained at a single clinic visit and through chart review. The visit included a semistructured interview, PCQ administration, and expert assessment. Data collected included patient's age, duration of diabetes, sex, insulin regimen, A1C, parent's educational level, and participant's sex and race/ethnicity.

PCQ administration and scoring

The PCQ was self-administered. Each correctly answered item contributed 1 point to the total raw score; partial credit (½ point) was awarded for answers close to the correct answer. The scoring technique was agreed on by experts. Scoring instructions are included in an online appendix. Percentage scores were calculated as the percentage of items answered correctly. Weighted percentage score for the carbohydrate domains was calculated by averaging percentage scores for the four carbohydrate domains, and weighted percentage score for the insulin-dosing domains was calculated by averaging percentage scores for the three insulin-dosing domains.

Expert assessment

Expert assessments were based on interview and review of a 1-day log of food intake, blood glucose level, and insulin dose. Logs were completed before PCQ administration. Expert assessments were obtained after administration of the PCQ by raters blinded to PCQ score. The single nutritionist rater was instructed to rate each participant on each of the four carbohydrate knowledge domains based on interview and review of food/insulin log. One item was rated for each domain. A question paralleling each domain was asked. For example, for the recognition of carbohydrates domain, the nutritionist rater was asked, “How well does he/she know which foods contain carbohydrates?” The nutritionist was instructed to rate the answer on a 7-point scale ranging from “Not very well” to “Very well.” The physician rater was similarly instructed to rate each participant on each of the three insulin-dosing domains.

The expert ratings in the four carbohydrate domains were summed and correlated with the sum of the scores on the four carbohydrate domains of the PCQ. The expert ratings in the three insulin-dosing domains were summed and correlated with the sum of the scores on the three insulin-dosing domains of the PCQ. The expert ratings for all domains were summed and correlated with the total PCQ score. Expert assessments by nutritionist and physician raters were obtained in participants who were able to see both the nutritionist and physician raters on the day of the visit (38 of 75 participants).

A1C

A1C values were collected from chart review. Of the A1C values, 89% were obtained in a central laboratory using high-pressure liquid chromatography (HPLC) with a reference range of 3.6–6.8%. Of the A1C values not from the central laboratory, three were obtained at laboratories using the same methodology (HPLC), certified by the National Glycohemoglobin Standardization Program as traceable to the Diabetes Control and Complications Trial reference method, and two were obtained via DCA 2000 immunoassay, which has high correlation with the HPLC method (13). These five A1C values were included in the analysis. The analysis was repeated after exclusion of A1C values for three other participants: one whose most recent A1C value was not obtained within 3 months of the study visit, one whose A1C value was obtained via a home kit, and one who had a glycohemoglobin level obtained. Dropping these three values from the analysis did not alter the results substantially.

Analysis plan

Analyses of the psychometric properties of the PCQ were performed to assess reliability and validity (14). Reliability was examined by measuring internal consistency and split-half reliability. Cronbach α was calculated to measure internal consistency. To assess split-half reliability, Spearman correlations between two theoretically equivalent halves (the 39-item odd-numbered half and the 39-item even-numbered half) of the PCQ were calculated. Criterion-related validity was assessed by calculating Spearman correlations between PCQ scores and A1C and between PCQ scores and expert assessments. Construct validity was evaluated by Wilcoxon rank-sum tests comparing PCQ scores between parents with and without college degrees and between groups of participants with differing levels of insulin regimen complexity. Analyses were performed using SAS (version 9.3; SAS Institute, Cary, NC).

RESULTS

Sample

Participation was offered to 109 individuals; 1 was excluded for lack of English fluency, and 33 declined with most stating lack of time as the reason. Nonparticipants were similar to participants in age, diabetes duration, and A1C. Seventy-four families were included; 1 family had 2 adolescents with type 1 diabetes, for a total of 75 participants (41 children and 34 parents).

Characteristics of the sample are summarized in Table 1. As might be expected, in the parent participant group, the child with diabetes was significantly younger and diabetes duration was significantly shorter than in the child participant group. Parent and child participants were similar in race/ethnicity, insulin regimen, and child sex. Parent participants and parents of child participants had similar educational levels. Of the participants, 85% were Caucasian, which is reflective of the practice as a whole.

Table 1

Characteristics of participants

	All participants	Parent participants	Child participants
n	75	34	41
Age of child (years)	13.2 ± 3.9	9.9 ± 2.9	15.9 ± 2.3
Participant sex
Male	29	18	39
Female	71	82	61
Age at diagnosis (years)	7.6 ± 3.8	5.7 ± 2.9	9.2 ± 3.9
Diabetes duration (years)	5.6 ± 3.5	4.3 ± 2.0	6.7 ± 4.2
Race/ethnicity
Caucasian	85	88	83
African American	8	6	10
Other	7	6	7
Parents' education
College degree	52	53	51
No college degree	48	47	49
A1C	8.9 ± 1.6	9.0 ± 1.6	8.9 ± 1.6
Insulin regimen
Basal-bolus MDI	68	71	66
CSII	21	21	22
Traditional	11	9	12

Data are means ± SD or %. CSII, continuous subcutaneous insulin infusion; MDI, multiple daily injections.

PCQ scores

The PCQ score histogram is shown in Fig. 1. Of a total possible score of 78, mean ± SD score was 67.8 ± 7.6 or 87 ± 9.7% (range 42–98%). Two participants (3%) correctly answered ≥97% of the items; both of these participants were parents. Of a total possible score of 58 in the carbohydrate domains, mean score was 50.6 ± 5.8 or 87 ± 10% (range 41–98%). Of a total possible score of 20 in the insulin-dosing domains, mean score was 17.3 ± 2.8 or 87 ± 14% (range 43–100%). Of the participants, 23% scored the maximum value on the combined insulin-dosing domains. The percentages of parents and youth who scored the maximum value on each of the domains were as follows: in the carbohydrate recognition domain, 26% of parents and 10% of youth; in the carbohydrate counting in individual food items domain, 6% of parents and 2% of youth; in the carbohydrate counting in whole meals domain, 3% of parents and no youth; in the nutrition label reading domain, 94% of parents and 85% of youth; in the use of insulin-to-carbohydrate ratio domain, 85% of parents and 85% of youth; in the use of insulin dose correction based on blood glucose level domain, 88% of parents and 73% of youth; and in the calculation of whole meal insulin dose domain, 29% of parents and 20% of youth.

Figure 1

PCQ score histogram.

Mean ± SD score was 69.7 ± 5.3 (89 ± 6.8%) for parent participants and 66.4 ± 8.8 (85 ± 11%) for child participants; this difference trended toward significance (P = 0.09, Wilcoxon rank-sum test). Within each subsample (parent participants and child participants), there was no correlation between PCQ scores and child's current age, age at diagnosis, or duration of diabetes. PCQ score did not differ by participant sex (P = 0.78, Wilcoxon rank-sum test).

Scores on the recognition of carbohydrates, carbohydrate counting in individual food items, carbohydrate counting in whole meals, and nutrition label reading domains were 91.5 ± 11, 73.7 ± 19, 67.2 ± 18, and 97.7 ± 8.1%, respectively; these scores were averaged to obtain a weighted percentage score for the carbohydrate domains (82.5 ± 10%). Scores on the use of insulin dose correction based on blood glucose level, use of insulin-to-carbohydrate ratio in insulin dosing, and calculation of whole meal insulin dose domains were 94.2 ± 17, 93.8 ± 17, and 75.1 ± 23%, respectively; these scores were averaged to obtain a weighted percentage score for the insulin-dosing domains (87.7 ± 13%).

Reliability

Internal consistency.

The Cronbach α was 0.88 for the whole test and ranged from 0.38 to 0.86 for individual domains: recognition of carbohydrates, 0.86; carbohydrate counting in individual food items, 0.38; carbohydrate counting in whole meals, 0.49; nutrition label reading, 0.66; use of insulin dose correction based on blood glucose level, 0.82; use of insulin-to-carbohydrate ratio in insulin dosing, 0.78; and calculation of whole meal insulin dose domains, 0.77.

Split-half reliability testing.

As described, the 78-item PCQ included two theoretically equivalent 39-item sets. The 78-item questionnaire was administered to each participant, and scores on the two halves were correlated. Spearman correlations between total scores on the two halves were 0.59 for the whole sample, 0.66 for parent participants, and 0.60 for child participants (P < 0.0001 for all).

Respective Spearman correlations in the recognition of carbohydrates, carbohydrate counting in individual food items, carbohydrate counting in whole meals, nutrition label reading, use of insulin dose correction based on blood glucose level, use of insulin-to-carbohydrate ratio in insulin dosing, and calculation of whole meal insulin dose domains were 0.52 (P < 0.0001), 0.19 (P = 0.10), 0.36 (P = 0.002), 0.53 (P < 0.0001), 0.58 (P < 0.0001), 0.38 (P = 0.0007), and 0.33 (P = 0.004), respectively, in the whole sample. Supplementary Table A1 (available in an online appendix) contains Spearman correlations in individual domains for parent and child subsamples.

Validity

Correlation of PCQ score with A1C.

Spearman correlation was −0.29 (P = 0.01) for the whole sample, −0.23 (P = 0.18) for parent participants, and −0.32 (P = 0.04) for child participants. Excluding three patients whose A1C was obtained by different methodology or >3 months from the visit yielded Spearman correlations of −0.25 (P = 0.04) for the whole sample, −0.17 (P = 0.34) for parent participants, and −0.27 (P = 0.09) for child participants.

Correlation of PCQ scores with expert assessment.

Spearman correlations of total PCQ scores with total expert assessment scores were 0.56 (P < 0.001) for the whole sample, 0.57 (P = 0.03) for parent participants, and 0.52 (P = 0.01) for child participants. Spearman correlations of PCQ scores with expert assessment scores for the carbohydrate domains were 0.72 (P < 0.0001) for the whole sample, 0.67 (P = 0.003) for parent participants, and 0.70 (P < 0.0001) for child participants. Spearman correlations of PCQ scores with expert assessment scores for the insulin-dosing domains were 0.23 (P = 0.12) for the whole sample, 0.25 (P = 0.28) for parent participants, and 0.25 (P = 0.21) for child participants.

Comparison of PCQ scores by parent educational level.

PCQ scores were compared between parents with and without college degrees. College-educated parents scored significantly higher: 71.9 ± 3.3 vs. 66.8 ± 6.2 (P = 0.01).

Comparison of PCQ scores by complexity of insulin regimen.

Participants were divided into groups based on insulin regimen complexity. The higher complexity group included participants using flexible regimens (injection or pump) and the lower complexity group included participants using traditional regimens not requiring dose adjustment for carbohydrate intake. The higher complexity group scored significantly higher: 68.7 ± 6.6 vs. 60.9 ± 11.7 (P = 0.003).

CONCLUSIONS

Given the importance of flexible insulin regimens to the management of type 1 diabetes in children, an objective measure of knowledge regarding carbohydrate counting and insulin dosing is needed. The PCQ is a newly developed, brief, self-administered questionnaire to assess carbohydrate and insulin-dosing knowledge. Overall, this study provides evidence of the reliability and validity of the PCQ in a population of parents of children with type 1 diabetes and youth aged ≥12 years with type 1 diabetes.

The high Cronbach α for the overall test and moderate to high Cronbach α in individual domains support the reliability of the PCQ. Because reliability is a function of the number of items, it is not surprising that reliability coefficients were lower in individual domains than in the overall test. The inter-item agreement in the individual PCQ domain scales is consistent with measures of internal consistency found in subscales of other established measures (5,6).

Split-half testing revealed positive and significant correlations between total scores on the two halves, indicating good reliability. For the whole sample, all correlations were statistically significant, with the exception of the carbohydrate counting in the individual food items domain, in which the correlation trended toward significance (P = 0.10). Correlations of split halves in the individual domains were expected to be lower, given the smaller number of items.

Higher PCQ scores correlated significantly with lower A1C levels in the whole sample and child subsample, suggesting that greater knowledge is associated with better glycemic control, and supporting validity of the PCQ. The correlation of PCQ scores with A1C in this study is notable because previously developed general diabetes knowledge tests failed to show a relationship between knowledge and A1C, a phenomenon known as the “knowledge-behavior gap” (10,15).

Correlations between PCQ scores and expert assessment scores were statistically significant for total and carbohydrate domains scores, supporting validity of the PCQ. The lack of statistically significant correlations in insulin-dosing domains may reflect the fact that expert assessments obtained through review of a food/insulin log may not be pure assessments of insulin-dosing knowledge. Food/insulin logs reflect behavior and habits, so assessments based on these records are confounded by adherence and may not solely reflect knowledge.

College-educated parents scored significantly higher than parents without college degrees. This result supports construct validity of the PCQ, as individuals with higher levels of education would be predicted to perform better on knowledge-based tests. Participants using more complex insulin regimens scored significantly higher than those using less complex regimens. This result provides additional support of construct validity of the PCQ, as individuals using more complex regimens are expected to have greater mastery of the knowledge required to enact flexible regimens.

Optimal glycemic control remains elusive for many youth with type 1 diabetes. Although many factors contribute to suboptimal diabetes control, poor adherence to treatment regimen, particularly poor dietary adherence, is predictive of poor control (16–18). Just as glycemic control is multifactorial, there are also multiple determinants of diabetes regimen adherence (17,19). Knowledge is an important, but not the only, determinant of behavior. Although knowledge has not been shown to be a good predictor of patient behavior in past studies (9,10), knowledge in the forms of health literacy and numeracy in carbohydrate counting and performing insulin dose calculations are unequivocal prerequisites to adherence to flexible regimens (20).

To the best of our knowledge, the PCQ is the first tool to be developed to assess the specific knowledge required to implement flexible insulin regimens. Additional strengths of the PCQ are brevity and ease of administration. As shown in this study, the PCQ has favorable psychometric properties, demonstrating reliability and validity. One limitation of this study is that subjects were drawn from a single practice with little racial/ethnic diversity. PCQ scores may differ in other populations. Applicability to other racial/ethnic populations also may be limited by the relative lack of racial/ethnic diversity among the expert panel who developed the PCQ (11 Caucasians and 3 Asians). Similarly the food logs (from 18 Caucasians and 3 African Americans) used to generate food items for the PCQ were somewhat limited in representing a broad range of ethnicities. Another limitation of the PCQ is a possible ceiling effect, especially in the relatively high-scoring insulin-dosing domains. However, PCQ development included extensive review and revision by an expert panel who felt that the PCQ adequately assesses the finite amount of knowledge required to implement flexible regimens.

An additional potential limitation of the PCQ is the use of a correction scale in the domain to assess insulin dose correction based on blood glucose level. Some patients using flexible regimens use glucose target and correction factors rather than correction scales. For broader applicability, future versions of the PCQ can include items using glucose target and correction factors. It is interesting to note, however, that Huizinga et al. (12) found that patients had an easier time calculating insulin dose using a correction/sliding scale than using a glucose target and correction factor, suggesting that the practice of determining insulin dosage with a correction scale may be preferable.

The findings of this study have implications for both clinical practice and research. There was a trend toward higher PCQ scores in parent participants than in child participants. This finding suggests that diabetes caregivers should ensure that adolescents are well educated in carbohydrate counting and insulin dosing before being allowed to assume responsibility for their own care. PCQ scores on the carbohydrate domains were lower than scores on the insulin-dosing domains. Although further evaluation is needed, this observation suggests that lack of carbohydrate knowledge, rather than lack of knowledge about performing insulin dose calculations, is a greater hindrance to successful regimen adherence. If so, educational efforts should focus particularly on providing resources for improved carbohydrate counting.

In summary, the PCQ addresses an important need in the clinical care of pediatric patients with type 1 diabetes. Initial analyses support reliability and validity of the PCQ. Further studies are planned to refine the PCQ and assess the relationship between behavior/adherence and knowledge as indicated by PCQ scores and assess even shorter forms of the PCQ. Future work also can include validation of the PCQ in other populations, such as adults or other racial/ethnic groups; such work would involve modifying the PCQ to incorporate food items representative of those populations. It is worth noting that the applicability of the PCQ for pediatric patients with type 2 diabetes has not been established, and it should not be used for clinical application in individuals with type 2 diabetes until it has been validated in this population.

The PCQ has strong potential to aid in improving diabetes care. It can be used to identify readiness to intensify insulin regimens and to determine specific areas of weakness requiring reeducation. It can be used to assess educational interventions. The PCQ also can be used in research studies that require objective assessment of patients' knowledge about carbohydrates and insulin dosing. As such, the PCQ is a promising new instrument that addresses a gap in current diabetes care.