A visual analytics approach for pattern-recognition in patient-generated data.

Objective: To develop and test a visual analytics tool to help clinicians identify systematic and clinically meaningful patterns in patient-generated data (PGD) while decreasing perceived information overload.
Methods: Participatory design was used to develop Glucolyzer, an interactive tool featuring hierarchical clustering and a heatmap visualization to help registered dietitians (RDs) identify associative patterns between blood glucose levels and per-meal macronutrient composition for individuals with type 2 diabetes (T2DM). Ten RDs participated in a within-subjects experiment to compare Glucolyzer to a static logbook format. For each representation, participants had 25 minutes to examine 1 month of diabetes self-monitoring data captured by an individual with T2DM and identify clinically meaningful patterns. We compared the quality and accuracy of the observations generated using each representation.
Results: Participants generated 50% more observations when using Glucolyzer (98) than when using the logbook format (64) without any loss in accuracy (69% accuracy vs 62%, respectively, p = .17). Participants identified more observations that included ingredients other than carbohydrates using Glucolyzer (36% vs 16%, p = .027). Fewer RDs reported feelings of information overload using Glucolyzer compared to the logbook format. Study participants displayed variable acceptance of hierarchical clustering. Conclusions: Visual analytics have the potential to mitigate provider concerns about the volume of self-monitoring data. Glucolyzer helped dietitians identify meaningful patterns in self-monitoring data without incurring perceived information overload. Future studies should assess whether similar tools can support clinicians in personalizing behavioral interventions that improve patient outcomes.

Introduction

More than 2 billion people worldwide use smartphones and wearable activity trackers to collect data related to daily activities., These inexpensive mobile technologies are increasingly used by persons with chronic conditions in addition to more traditional disease self-monitoring technologies like pulse oximeters, blood pressure cuffs, and glucometers.,, The widespread adoption of mobile health technology (mHealth) has generated exponential growth in the volume of patient-generated data (PGD)., In response to these trends, personal informatics solutions have focused on helping individuals gain insights from PGD to gain self-awareness, learn from past experiences, and improve their future choices.

To reflect the potential value of these data for clinical care, Meaningful Use Stage 3 requires providers to integrate PGD into electronic health records (EHRs).Precision health informatics is an emerging discipline which investigates new approaches to using PGD to improve clinical decision-making and behavioral interventions such as smoking cessation, increased physical activity, and adherence to medical treatment. However, an overwhelming amount of data can result in information overload, neglect of critical information, and incorrect interpretation of data., As a result, there exist considerable concerns regarding successful integration of PGD into clinical workflows.,

Interactive visualizations that leverage data science methods are increasingly recognized as potential solutions for information overload in healthcare. Visual analytics integrates concepts from machine learning, human factors engineering, and cognitive psychology to aid interpretation of complex data., Advanced visual analytics solutions can unlock the value of high-dimensional data and support clinical decision-making.,, Interactive visualizations using EHR data have demonstrated effectiveness for clinical tasks including the analysis of disease risk factors, prediction of health outcomes, and review of longitudinal patient records. However, visual analytics has not been extensively studied in the context of voluminous and heterogeneous self-monitoring data, and, specifically, data related to health behaviors such as nutritional intake.,

We hypothesize that visual analytics applied to self-monitoring data can support clinical decision-making in the context of chronic disease management. Our specific focus is on nutritional therapy for individuals with type 2 diabetes (T2DM) – a chronic condition that affects a large segment of the US population. High individual variability of glycemic response to nutrition necessitates development of tailored strategies for nutrition management. However, identifying patterns in an individuals’ history using self-monitoring data may be challenging. We used participatory design to develop Glucolyzer, an interactive tool that uses hierarchical clustering and heatmap visualizations to reveal systematic associations between nutritional content of meals and glycemic response. We evaluated Glucolyzer with registered dietitians (RDs) on its impact on pattern recognition and time-burden associated with high-dimensional PGD.

Methods

The study included 5 phases: (1) the collection of diabetes self-monitoring data, (2) exploratory interviews and iterative participatory design with Certified Diabetes Educators (CDEs), (3) development of Glucolyzer, and (4) a controlled experimental study of Glucolyzer with 10 registered dietitians (RDs), and (5) interviews with 3 participating RDs regarding the integration of Glucolyzer into clinical practice.

Collection of Diabetes Self-Monitoring Data

The datasets used in this study were generated during a self-monitoring study that included participants with T2DM conducted in 2014. Participants photographed their food using smart phones and captured pre- and post-meal blood glucose measurements. An expert RD reviewed all meals and used the USDA nutritional database to estimate nutrition in each meal (grams of protein, fat, carbohydrate, fiber, and calories). More details on the study are available elsewhere.

Exploratory Interviews and Participatory Design with CDEs

Two academic Certified Diabetes Educators (CDEs), one with a background in nursing and another in clinical nutrition, took part in participatory design sessions. These sessions were audio recorded and transcribed verbatim for analysis. In these sessions, CDEs discussed different approaches to personalizing nutritional therapy for T2DM and provided feedback on Glucolyzer mockups. These sessions continued until the design of the tool was finalized.

Development of the Visual Analytics Tool, Glucolyzer

The tool was informed by the design requirements identified by the CDEs and guidelines for interactive visualizations identified by Heer and Schneiderman. This taxonomy describes 12 elements grouped into 3 high-level categories; data specification (visualization, filter, order, and derive), view manipulation (select, navigate, coordinate, and organize) and analytic process (record, annotate, share, and guide). We used d3heatmap, plotly, and shiny packages in R Version 3.0.7 to develop Glucolyzer, and photographic tooltips were added using JavaScript.

Controlled Experimental Evaluation

Participants

Ten Registered dietitians (RDs) evaluated Glucolyzer in the context of a simulated clinical visit. All participants were recruited via the professional network of the study team. The inclusion criteria for study participants were certification as an RD and experience counseling patients with T2DM. Participating RDs received compensation of $40 for the 2 hours required to complete the study.

Study design

We used a within-subjects study design: each participant was asked to evaluate 2 different datasets, one using Glucolyzer and another using a static logbook format. The static logbook simulated a typical paper-based log of meals, including pictures, descriptions, and BG levels and required substantial scrolling to review all data. We randomized both the order in which the datasets were presented and whether participants began with the logbook representation or Glucolyzer. The study was approved by the Institutional Review Board of the Columbia University Medical Center and all participants provided verbal consent.

Procedures

All participants received a 1-hour training session a day prior to the main study trial to reduce potential fatigue. Participants watched a 30-minute instructional video and spent 30 minutes interacting with Glucolyzer using training data while study staff provided instruction.

During the within-subjects study trials, participants had 25 minutes to examine 1-month of PGD using each display. Participating RDs were encouraged to verbalize their thoughts using a “think aloud” protocol to characterize analytical reasoning. All spoken statements were recorded and transcribed. Participants were instructed to generate written observations in reference to 4 standardized questions listed in Appendix A and email them to study staff at the end of each trial.

We administered a survey containing 9 questions on a 5-point Likert scale to participants immediately following the study. The questions examined participants’ sentiment about the analysis methods, perceived utility of the visualization, and preferences for the Glucolyzer user interface (Appendix B).

Analysis

We evaluated the impact of the visual analytics tool on the a) the number of observations generated using PGD, b) the accuracy of those observations, and c) perceived information overload experienced by RDs.

Participant observations were independently characterized by 2 researchers (DJF and LM) with any disagreements reviewed and negotiated until 100% agreement was achieved. Unique observations were classified as focusing on either nutritional content or glycemic impact; the latter were further classified as focusing either on carbohydrates or other macronutrients. This distinction is important because the positive association between carbohydrates and glycemic impact is well-established and is common knowledge among RDs., In contrast, the glycemic impact of protein, fat, and fiber is less understood and requires careful analysis of PGD. In addition, all observations were inductively classified based on their focus (“macronutrients,” “type of meal,” and “ingredients”).

The number of observations generated using Glucolyzer and the logbook was computed by counting the number of observations reported by RDs. We identified discrete and non-redundant observations, aggregated observations by study condition and stratified them across an array of statement characteristics.

Statement accuracy was defined as the correspondence of an observation to the data. Because statements reflected trends, we expected variability in the degree of this correspondence and thus statements were considered “accurate” if the number of meals supporting the statement was greater or equal to those contradicting it. Each observation was assessed by translating words into an expression in the R statistical programming language and evaluating it against the data. For example, the observation “meals with the highest protein have low glycemic impact” was translated into a conditional statement that assessed whether meals with higher than average protein had a lower than average glycemic impact:

Accuracy of 31 observations (19%) that mentioned specific ingredients could not be translated into executable expressions and was evaluated via manual review. For example, the statement “when he drinks coffee with the meal it seems like [it] stabilizes blood glucose” was evaluated by calculating the proportion of meals including coffee that had lower than average glycemic impact. Thirteen observations were too vague to be evaluated and were excluded.

Perception of information overload was assessed through qualitative analysis of study transcripts. Inductive thematic analysis was used to analyze study transcripts and identify themes in participants statements. Study staff was blinded to metadata associated with each statement.

Hypothesis testing was performed to examine whether the study conditions (Glucolyzer and logbook format) elicited significant differences in the characteristics and accuracy of observations. Chi-square goodness of fit tests were used to compare the number of observations generated across study conditions. We conducted significance testing for statement accuracy using McNemar’s test for paired categorical data with a significance level of 0.05.

Finally, we conducted semi-structured interviews with 3 study participants (2 outpatient RDs and 1 inpatient RD) regarding the perceived utility of Glucolyzer in their clinical practice. These interviews were conducted one year after the formal evaluations of Glucolyzer; the RDs were given time to reacquaint themselves with the visual analytics tool before answering questions

Results

Collection of Diabetes Self-monitoring Data

Two individuals with T2DM collected 304 and 211 blood glucose readings, and 105 and 72 meals, respectively, over 1-month. The methods through which this data was collected and processed are described in detail in Section 2.1.

Early conversations with academic CDEs resulted in the following design requirements for an interactive tool that would allow them to analyze associations between macronutrient composition and glycemic impact:

Visualize all available PGD while highlighting important characteristics including glycemic impact and the absolute and relative proportion of individual macronutrients

Facilitate exploration and analysis of PGD using analytic methods that reflect heuristics commonly used by clinicians to identify patterns in PGD:

Differentiate between mealtimes (eg. breakfast)

Examine similarity between meals based on various criteria (eg, proportion of macronutrient, absolute amounts of macronutrient, glycemic impact, etc.)

Facilitate inspection of nutritional profile and images of meals

Multiple approaches to examine the data through sorting, filtering, and more complex analytical mechanisms

As a result of these sessions, we identified data analysis and visualization approaches most consistent with academic CDEs’ requirements and recommendations.

Architecture of Glucolyzer

In response to these requirements, the user interface (UI) of Glucolyzer is comprised of 3 distinct pages; the Analytics tab, the Explore tab, and the Clustering tab.

The Analytics tab supports pattern recognition by illustrating a collection of meals using a heatmap (Figure 1). Each row in the heatmap represents 1 meal and each column represents a variable, including macronutrient content and glycemic impact. Values above the mean are colored red and those below are colored blue. The intensity of the color hue is proportional to the magnitude of the deviation from the column mean. Users can manipulate the visual organization of a collection of meals. For example, using ranking allows clinicians to sort meals in order of descending blood glucose change. Alternatively, the users can apply hierarchical clustering analysis to identify groups of meals with similar nutritional characteristics and glycemic impacts and use a dendrogram to distinguish individual clusters. Toolbars on the right of Glucolyzer (Figure 1) enabled RDs to manipulate the data presented in the heatmap. Users can select meals from a specific time of day (ex. lunches), modify the units of macronutrient values (eg. % calories or absolute grams), and vary the number of clusters plotted (up to 25). Macronutrient levels and blood glucose readings were normalized within each dataset before clustering. Finally, an interactive tooltip (displayed in Figure 2) is generated by mouse-hover and presents RDs with an image and nutritional profile of each meal.

Figure 1.

Analytics tab - heatmap with hierarchical clustering.

Figure 2.

Tooltip within Glucolyzer.

The Explore tab (Figure 3) permits clinicians to visualize atemporal trends in nutritional content and blood glucose changes using 3 probability density plots that display (1) caloric content, (2) macronutrient composition, and (3) glycemic impact of all meals. In these plots, the shape of the curve indicates the likelihood of observing different variables plotted on the X-axis. For example, in Figure 3, one can see that approximately 75% of an individual’s meals had between 400 and 800 calories, even though individual values had a low probability of occurrence. Users can select a macronutrient and examine its distribution separately from other macronutrients and isolate a specific mealtime, to focus their examination. During training, participating RDs learned the meaning of the curves as well as strategies to interpret their shapes to characterize an individual’s aggregate nutritional profile.

Figure 3.

Explore tab - density plot of calories, macronutrients, and blood glucose.

The Clustering tab helps users identify an optimal number of clusters. The x axis of the plot represents the numbers of clusters and the y axis represents cluster quality, defined as the ratio between the sum of squared Euclidean distances within clusters and the sum of squared Euclidean distances across all elements (labeled as Purity in the interface, Figure 1). This measure illustrates how clusters become more narrowly defined as the number of clusters increases.

Participant characteristics

Ten RDs were recruited to participate in the formal evaluation of Glucolyzer. All participants were female, between the ages of 25 and 40, had graduate degrees in nutrition or a related discipline and had professional experience counseling patients with T2DM.

Characteristics of study observations

There were 162 statements generated during the 10 trials. Participants generated 98 observations using the interactive tool and 64 observations using the logbook representation (Table 1).

Table 1.

Accuracy of statements generated using each study condition

	Interactive Tool – “Glucolyzer”		Static Logbook Format
	Accurate	Total	Accurate	Total
Glycemic Impact	46	65	25	33
Non-Carbohydrates	25	35	6	10
Nutritional Trend	22	33	14	31
Total	68	98	39	64

Glucolyzer users more frequently remarked on glycemic impact (66.3% of all observations) compared to the logbook format (50.8%). Conversely, RDs using the logbook representation more often remarked on themes in nutritional content (48.2% vs 33.7%).

Macronutrients were mentioned almost twice as frequently using the Glucolyzer (58.1% of all observations) compared to the logbook (29.7%, p = .0004). Individual ingredients and food groups such as vegetables and whole grains were more often discussed using the logbook representation (21.8% vs 4.1%, p = .0004). Across study conditions, carbohydrates, protein, and fat were discussed with similar relative frequencies, though fiber was mentioned more often using Glucolyzer (41.8% of macronutrient observations) compared to the logbook (9.4%, p = .01).

RDs using Glucolyzer frequently reasoned over patterns with macronutrients other than carbohydrates, as thirty-five (36%) observations generated by Glucolyzer focused on fiber, fat, and protein compared to only 10 (16%, p = .027) observations using the logbook. Moreover, 88% of such observations generated using Glucolyzer reasoned over multiple macronutrients compared to 70% of those generated using the logbook (p = .15).

Assessment of statement accuracy

Statements generated using Glucolyzer had accuracy comparable to those generated using the logbook (69.4% vs 60.9%, p = .17). This was also the case for observations focused on glycemic impact (70.8% for Glucolyzer and 78.1% for the logbook, p = .44). However, there was a trend towards higher accuracy for observations focused on nutritional content using Glucolyzer (66.6% vs 45.2% for the logbook, p = .08).

Most (71.4%) observations related to fat, protein, and fiber generated using Glucolyzer were accurate. This ratio was lower for similar observations generated using the logbook (60%); however, this difference was not significant (p = .23). Further, 80% of the observations focusing on macronutrients other than carbohydrates generated using Glucolyzer considered the impact of multiple macronutrients compared to 66% identified using the logbook format, suggesting that Glucolyzer improved the RDs’ ability to effectively reason using a larger number of variables (p = .22).

Qualitative insights: thematic content identified via open-coding

We identified 4 general themes across 212 coded statements collected using the “think-aloud” protocol during trials. These themes are presented in Table 2.

Table 2.

Qualitative comparison of glucolyzer and static HTML-based representation

Theme	Interactive	Static	Comparison
Reasoning about glycemic impact	Individual macronutrients support reasoning	Specific ingredients & food groups support reasoning	Patterns more generalizable
Breadth of information analyzed	Likely to examine sizeable clusters or ranges sorted by glycemic impact	Likely to focus on a small sample of meals	Reduced selection bias
Pattern Recognition Heuristics	Likely to identify outliers & unexpected associations	Likely to focus on meals confirming pre-existing knowledge	Reduced confirmation bias
Assessment of Outcome	Only able to consider change in pre- & post-prandial blood glucose level	Likely to consider discrete blood glucose concentrations	Modest information loss

Reasoning with specific foods vs reasoning with macronutrients

Qualitative analysis corroborated the quantitative finding that participants using the logbook were more likely to consider ingredients & food groups when reasoning about glycemic impact.;

“… the food with more [calories], you can see the dressing and chicken chunks, and those sauces. I think those can impact your blood glucose level…”. (P2, logbook format)

In contrast, participants using Glucolyzer were more likely to reason with macronutrients:

“the blood glucose could be more stable if maybe when he doesn’t eat more carbohydrate with fiber rich diet.” (P1, Glucolyzer)

Breadth of information analyzed

Glucolyzer allowed RDs to examine a large number of meals quickly, for example:

“I [understood] the macronutrient conventions because I looked at all pictures for this person.” (P9, Glucolyzer)

In contrast, several participants commented on the challenges of reviewing a large number of meals using the logbook:

“as far as to determine a trend, it is hard [because] the meals that I saw, they are not outrageous values, but I just saw a small percent of the meals. If I [had] more time it would be different.” (P2, logbook)

Discoveries

Participants using Glucolyzer often remarked on unexpected associations when applying different types of analysis within the tool:

“we have protein which must be lower [accompanied by] lower glucose changes for dinner which is interesting” (P5, Glucolyzer).

Opportunities for improvement

Participants also identified several limitations of our tool. Participating RDs using the logbook document found it helpful to consider not only the difference between pre- and post- blood glucose levels, but also those levels in themselves:

“I'm trying to see [blood glucose] during lunch and dinner … begins at 85 [for lunch] and then 115 for dinner.” (P4, Logbook)

Some RDs with limited exposure to math and statistics found it difficult to distinguish individual clusters and understand the results of clustering;

“I don’t think it’s easy to understand, and [I’ll need] practice to understand it and play with it, but I do agree very strongly that it’s useful.” (P6, Glucolyzer)

Survey results

Survey results are presented in Table 3 and suggest that Glucolyzer was well received by participating RDs; the majority (70%) of participants found the tool to be useful and easy to understand. While 9 participants found clustering to be useful, only 5 felt they had enough information about how clustering works. Eight participants perceived the “ranking” analysis method most useful, followed by “clustering.”

Table 3.

Response to survey questions

Question	Strongly Agree	Agree	Neutral	Disagree	Strongly Disagree
Overall, I found the tool to be useful and easy to understand	0	7	2	1	0
The “Explore” tab assisted my understanding of the data	2	4	3	1	0
This tool would make me more likely to encourage my diabetic patients to self-monitor	5	3	1	1	0
I would likely find this tool useful for each of my patients with diabetes	4	3	2	1	0
I had enough information about how the clustering technique works	3	2	3	2	0
I found the clustering technique to be useful	5	4	1	0	0
What analysis type was useful?
Ranking - 8	Clustering - 3			Easy Clustering - 7
What macronutrient scale do you prefer?
Grams - 4	% Grams - 4			% Carbs - 3
Which heatmap color scale do you prefer?
	Red/blue gradient - 9		Blue gradient - 1

Inductive Thematic Analysis of Workflow Interviews

Inductive analysis of interview transcripts resulted in 3 main themes including RDs perceptions regarding integrating Glucolyzer into their regular practice, its perceived utility for visit preparation, and its potential as a tool for patient education.

Integration into clinical practice

RDs perceived Glucolyzer as being most useful in outpatient settings. Participating RDs who specialized in outpatient settings suggested that they could envision themselves orienting their patients to mobile self-monitoring during their initial clinical visit and using Glucolyzer to review patient data in subsequent visits:

“I would more say maybe on the initial visit we would set them up with [an application for self-monitoring] and then at the follow up after a month we could look at the data” (P2)

Moreover, because RDs often use computers during clinical consultations with patients, they felt well-prepared to use a visual analytics system during visits.

Perceived utility for visit preparation

All 3 RDs viewed Glucolyzer as providing the most value in helping them to review PGD before patient visits, thereby allowing providers to spend more time delivering counseling;

“[Glucolyzer] would be super helpful. Right now, I go into visits blind and spend 20 minutes trying to figure out what [patients] are doing and then I have 10 minutes to counsel them. So, it’s a huge time saver for me to know exactly where are they struggling, so I can actually sit there and do motivational interviewing.” (P2)

Glucolyzer as an educational tool

Participants also felt that Glucolyzer could be used to communicate to their patients’ specific instances of meals with high or low glycemic impact;

“we could talk together about what exactly was going on when they had – like one of these really red zones and look at the meals and talk about what to do differently. I definitely could see it as being an educational tool.” (P1)

Discussion

In this study, we examined a visual analytics tool for helping clinicians identify patterns in PGD and opportunities to introduce such tools into clinical practice. Overall, the study showed that using the tools clinicians generate more observations without significant decrease in accuracy, focus on more generalizable macronutrients rather than specific ingredients, and consider the impact of not only carbohydrates but also other, less well understood macronutrients and combined effect of multiple macronutrients. Moreover, RDs responded favorably to Glucolyzer and reported that the tool would enable them to spend more time counseling patients during clinical consultations and less time analyzing PGD.

Participating RDs generated 50% more observations from PGD using the visual analytics application compared to the logbook format with comparable accuracy. Many studies have asserted that analyzing large volumes of clinical data can overwhelm clinicians and result in diagnostic errors., Using the logbook format, RDs experienced difficulty analyzing 1-month of PGD and reported drawing conclusions from a limited number of meals that confirmed expected patterns of glycemic response. Such behavior may suggest anchoring (tendency to rely on the first piece of information offered) and confirmation bias (tendency to search for information that confirms preexisting beliefs) both of which could result in the omission of important trends in PGD. In contrast, using Glucolyzer, RDs could quickly overview a large number of meals, thereby reducing the possibility of bias. By generating 50% more observations compared to the logbook, RDs using Glucolyzer were equipped with a larger amount of evidence that could serve in the development of personalized nutrition therapy.

In addition, observations generated using Glucolyzer more often considered macronutrients rather than specific ingredients. The heatmap in Glucolyzer was designed to facilitate consideration of macronutrients in analyzing glycemic response. Macronutrients are situated in the center of the heatmap and thus occupy the majority of the visual field of participants. In contrast, the logbook featured images of meals as the primary source of information and thus it is unsurprising that RDs using the logbook more often reasoned about the impact of specific ingredients. There were 58.1% of observations generated using Glucolyzer that focused on macronutrient content compared to 29.7% of those generated using the logbook format. Previous research suggested that reasoning with macronutrients can help clinicians and patients generalize between different meals and foods similar in their macronutrient composition.

Glucolyzer enabled RDs to identify a larger number of trends related to the impact of protein, fat, and fiber on blood glucose compared to the logbook format. It is common knowledge that carbohydrates disproportionately affect glycemic response, and; therefore, nutritional therapy is typically focused on managing carbohydrates. However, emerging evidence suggests that other macronutrients impact blood glucose levels by mediating the impact of carbohydrates., Identifying trends that involve protein, fiber, or fat requires clinicians to transcend common assumptions about glycemic response and recognize multivariate patterns in complex data, a time-consuming and cognitively burdensome task., Also, 36% of observations generated using Glucolyzer included these macronutrients compared to 16% using the logbook. Visual analytics tools could enable RDs to provide truly personalized nutritional recommendations for improving glycemic control.,,

Glucolyzer was informed by guidelines for the design of visual analytics tools. We found 4 of Heer and Schneiderman’s 12 principles of successful analytic dialogues to be particularly relevant as our participants navigated a large amount of PGD. Specifically, visualization helped participating RDs leverage their perceptual skills to detect patterns, corroborating research which demonstrated that visualization can facilitate analysis of PGD.,Filtering enabled users to reduce the potential of information overload by temporarily excluding irrelevant data. Glucolyzer also allowed users to select an individual macronutrient and examine its distribution using the Explore tab. Filtering and selection within Glucolyzer are similar to the heuristics used by RDs to analyze PGD and have been featured in other visual analytic systems for EHR data and PGD.,,Sorting helped RDs explore the underlying structure of 1-month of PGD. Glucolyzer’s ranking method enabled them to rapidly assess whether meals with extreme glycemic impacts displayed a conspicuous association with a specific macronutrient. Clustering was used to reveal more complex patterns manifest over multiple macronutrients. These 4 principles may serve as the cornerstones of interactive visualizations that facilitate the integration of PGD into clinical practice.

Our findings also suggest that advanced visual analytics methods require mathematical literacy. Because Glucolyzer was developed in participatory design with expert CDEs, using it effectively required a certain level of expertise in both nutrition and data analysis. Most participants grew increasingly comfortable with hierarchical clustering over time and perceived the technique as useful. This finding corroborates previous studies that have demonstrated that hierarchical clustering can pattern recognition in complex clinical data., However, for 2 RDs, one hour of training was not sufficient to understand the concept of hierarchical clustering and these participants were consistently confused by the technique. This suggests that some RDs may require extensive training before gaining proficiency in visual analytic tools and that such tools should be designed for users with varying levels of expertise.

Additional limitations of our study should be considered. The small sample size of our study may have not been sufficient to demonstrate statistical significance. Second, we included 1-month of PGD in trials, but the actual volume of PGD may vary given unique patient needs. Third, colors within Glucolyzer’s heatmap represented deviations from column means instead of recommended values. Meals with the most extreme BG changes were colored with high intensity regardless of whether they were unacceptable for a given patient. Fourth, participant statements were judged accurate if the number of meals supporting the statement were equal to those contradicting it, which could have inflated the number of statements deemed accurate; however, this approach was used across conditions. Fifth, future studies should consider how to integrate additional behavioral data such as physical activity, sleep, and stress levels into analyses of glycemic impact. Sixth, our visual analytics tool was a prototype, and we expect that usability would improve with further refinement. There were several aspects of the tool that reflected native features of R packages and required explanation during training sessions and assessment of patient data. For example, the Y-axis label within the plots of Figure 3 may have inadvertently drawn attention to numbers contained within the Y-axis; however, we instructed participating RDs to focus on the shape of the curve in relation to the X-axis. Removal of the Y-axis may help avoid this confusion. In addition, relying on the default kernel density estimator in the Ploty package created the misleading appearance of negative values; future versions should impose a positivity constraint to avoid confusion.

Conclusion

The volume of PGD produced by mHealth solutions is rapidly increasing, raising concerns among clinicians about information overload. Using a novel visual analytics system, RDs generated a large number of accurate patient-specific observations from 1-month of diabetes self-monitoring data. These findings suggest that visual analytics may transform the challenge of analyzing voluminous PGD into an opportunity to develop tailored behavioral strategies for chronic disease management. Future work should identify opportunities to leverage visual analytics in other areas of disease management and experiment with novel presentations of hierarchical clustering.

Funding

This work was supported by the National Library of Medicine grant number T15 LM007079, the Robert Wood Johnson Foundation grant number 73070, and the National Institute of Diabetes and Digestive and Kidney Disease grant number 1R01DK090372- 01A1.

Competing interests

None.

Contributors

All the authors designed the study. DJF and LM served as the main research investigators, and conducted all the described research activities. DJF developed the software and wrote the first draft of the manuscript. A.M.S. and P.D. served as the domain experts and were responsible for providing domain expertise during the interpretation sessions. All the authors participated in the interpretation of the study findings, formulation of the study conclusions, and the preparation of the manuscript. There are no other collaborators apart from the authors.