Mobile Access to Clinical Information at the Point of Care.

OBJECTIVES: Using library subscriptions and accessible on handheld devices, this study sought to promote authoritative health information apps, and evidence-based point-of-care resources.
METHODS: Three cohorts of internal medicine residents were issued iPads at the beginning of their second year, and were trained to skillfully access resources from the digital library. Pre- and post-intervention surveys were respectively administered at the beginning of the second year and end of the third year of training. The residents' computer experience and computer knowledge was assessed. Additionally, before and after formal introduction to iPads, perceptions on the use of computers to access clinical information were assessed. Survey responses were compared using two sample methods and summarized through descriptive statistics.
RESULTS: Sixty-eight residents completed the pre-survey questionnaires and 45 completed the post-surveys. There were significant improvements in the residents' level of computer experience, and familiarity with medical apps. Furthermore, there was increased knowledge obtained in accessing clinical information through electronic medical records. Residents positively perceived the potential effects of computers and electronic medical records in medicine.
CONCLUSION: Study findings suggested that health science libraries can be instrumental in providing search skills to health professionals, especially residents in training. Participants showed appreciation of iPads and library support that facilitated successful completion of their related tasks. Replicating this study with a larger sample derived from multiple sites is recommended for future studies. Participation of mid-level healthcare professionals, such as Physician Assistants and Nurse Practitioners is suggested.

Introduction

With the tremendous ongoing improvements in both portable hardware and information connectivity, mobile technology has become an essential and ubiquitous component of modern healthcare [1]. Mobile devices coupled with advanced health information technology, either by direct or wireless download, can make vital medical information accessible to medical practitioners at the point of care for medical education and delivery [2]. This makes mobile devices powerful in the clinical settings.

Mobile access to medical information facilitates "just-in-time" learning at the point of care, where learning "...is time- and place-independent and results in the functional use of information." [3] Mobile access to medical information allows for a novel method of experiential and self-directed learning. Furthermore, knowing precisely where specific information is located digitally is likely to save lives at the point-of-care through well-informed decision-making.

Mobile devices have shown great promise not only for medical students and residents, but also they have the potential to empower medical educators [4-6]. By supplementing the medical information base of students, mobile devices can allow educators to more effectively train and "..assess [student] performance and competence at the highest levels of Miller's Pyramid of Clinical Competence, thereby reflecting real-world practice." [3] The four sequential stages of Miller's Pyramid, from base to apex, include: "Information"; "knows how"; "Shows how"; and "Does."

Although mobile access to clinical information is a relatively new field, there exists a wealth of professional medical applications (apps) for mobile devices of varying content, purposes, and delivery, accessible across all mobile device platforms, both freely and commercially available. It can be expected that the medical apps ecosystem will continue to grow and develop substantially [1]. Furthermore, the future of healthcare computing is expected to become rooted in mobile technologies [7-9].

The objective of this study was to better understand how the introduction of mobile access to clinical information at the point of care is beneficial to medical resident physicians.

Methods

Medical librarians and the Internal Medicine residency coordinators conducted this study for three successive years (2012-2014). To raise awareness of point of care resources, free iPads were issued as an incentive to promote the use of mobile devices in accessing specific authoritative resources pertinent to medical education. The residents' computer experience and computer knowledge was assessed, along with computer-related attitudes regarding mobile access to clinical information, before and after formal introduction to iPads and point-of-care resources for clinical use. Three cohorts of second year internal medicine residents volunteered to participate and signed a participation contract. They were surveyed prior to intervention and were surveyed again at the end of the third year of training. The intervention comprised of equipping the residents with iPads and also providing them training and continued user support in the basic utilization of the iPad and point-of-care apps and resources. The Institutional Review Board approved the study.

To formally introduce mobile access to clinical information, study coordinators gave each participant an iPad with point-of-care resources and apps without any cost to the participants. The medical librarians demonstrated the iPad's clinical utility for accessing clinical information in the form of authoritative point-of-care resources available through the digital library and medical information apps on the iPad. Specifically, medical information apps introduced consisted of point-of-care resources from institutional subscriptions, authoritative public health apps from the National Library of Medicine, and general healthcare apps that were evaluated and approved by physicians for clinicians through imedicalapps.com. Comprehensive technical and resource support was provided to participants throughout the 18 months of the study.

Manning and Gadd's(2001) survey for evaluating handheld computing in a residency program was modified and customized for this study [10]. The survey questions were created with REDCap Survey™ software (Research Electronic Data Capture), (http://project-redcap.org) [11]. The REDCap software provided an intuitive interface using validated data entry; collecting, manipulation and export procedures; automated export procedures for seamless downloads to common statistical packages. Using the Likert-scale responses, residents were assessed on their familiarity with current computer technologies, opinions regarding the introduction of mobile technology within a clinical setting, and familiarity with medical information apps.

Composite scores were computed by summing up individual Likert-scale item scores in each of the following domains: "Computer experience", "Computer knowledge", "Perceived necessary capabilities of computer systems in medicine", "Familiarity with medical information apps", "Potential effects of computers", and "Appreciation for electronic medical record (EMR) on medicine." To assess the effectiveness of the intervention, changes in composite scores were calculated as the post-intervention composite score minus the baseline composite score. A high composite score indicates high information for a domain while a large positive pre-post difference is suggestive of improved information attributable to the intervention. Tabulated summary data were stratified by phase, pre-intervention, and post-intervention. For continuous variables the mean difference of composite scores between the two phases was computed. Data was summarized using descriptive statistics (i.e., mean and standard deviation for continuous variables and percentages for categorical variables). Histograms were also constructed comparing pre- and post-intervention categorical responses. Percentage change in composite scores was calculated according to this formula: [(post/pre) -1]*100%. Since there were no unique ID numbers linking pre- and post-intervention responses, paired statistical methods could not be used to assess changes in scores pre- and post-intervention. Instead two-sample t-test and Chi-squared test were used to compare continuous and categorical variables, respectively. Statistical Applications Software (SAS version 9.4) was used to analyze the data. A P-value of 0.10 was considered statistically significant.

Results

Sixty-eight participants, (38% women, mean age = 35.1, SD = 5.1 years) completed the baseline survey. Forty-five completed the post-intervention survey. Post-intervention, there were statistically significant positive increases in composite scores for participant computer experience (percentage change = 9.52%; P = .09) and familiarity with medical information apps for mobile devices (percentage change=50.59%; P < .0001) (Table 1).

Participant appreciation for Electronic Medical Record (EMR), (mean percentage change = 4.10%; P = .10) (Table 1) increased after intervention. Specifically, there were significant increases in the belief that an electronic medical record would be beneficial for time required for documentation (percentage change = 14.29%; P = .04), time required to enter orders (percentage change = 12.90%; P = .03), and patient privacy (percentage change = 25.00%; P = .02).

Furthermore, in the assessment of the perceived necessary capabilities of computers in medicine, it appeared that the residents value mobility (Figure 1-2) and access (Figure 3,4,5) associated with mobile access to clinical information at the point-of-care both before and after intervention.

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Discussion

Using the intervention, we observed varying magnitudes of increases of pre-versus post-intervention composite scores (Table 1). Composite score differences for the domains of computer experience, familiarity with medical information apps, and appreciation for EMR attained statistical significance. Such increases suggest that formal institutional intervention involving access to mobile technology and its utilization in the clinical setting could benefit residents providing patient care.

The value of formal institutional intervention became apparent with a closer examination of the large increase of 51% in familiarity with medical information apps among residents. In addition to demonstrating that residents were unfamiliar with medical information apps, it also suggested that valuable digital resources for mobile technology in the clinical setting were possibly underutilized. This study provided evidence that an intervention such as ours would potentially benefit residents in facilitating the process of becoming familiar with digital resources provided by their library. Additionally, residents should be made aware of evidence-based subscriptions with mobile apps that require log-ins. Familiarity and pre-registration to these resources would allow them quick access to point-of-care information at the bedside.

It is unclear if the improvement observed in the domains of resident computer experience, familiarity with medical information apps, and appreciation for EMR would translate into enhanced graduate medical education or improved quality of care for patients. Such questions would be best addressed in a prospective design and a control group combined with meticulous assessment of patient outcomes before and after implementation.

Further analysis of Table 1 revealed the relatively high baseline perception for some composite scores both before and after formal intervention, implied that residents had already deemed, prior to intervention, that a computer system must be highly capable for clinical use, and positively perceived both the potential effects of computers and EMR on medicine. With such relatively high baseline perceptions, no room for additional information improvement could be derived from this study. This apparent "ceiling effect" meant that no further intervention effect on information level was possible as a result of a prevalent high baseline information. This finding, in addition to the positive effects of intervention, suggested that the residents generally saw great value and potential in integrating reliable computer systems and EMR into the clinical setting. As has been previously reported, such a ceiling effect was often the result of constraints on data-gathering instruments such as the one used in this study. When a ceiling effect occurs in data-gathering, there is a bunching of scores at the upper level reported by an instrument [12].

Further analysis of the perceived necessary capabilities of computers in medicine, revealed that residents valued mobility and access afforded by smart phones and iPads/Tablets in a clinical setting. For mobility, residents appeared to value, both before and after intervention, the ability to access the computer system at any place (Figure 1). This apparent consensus seemingly conflicted with the varied opinions toward interacting with a computer without the need of a keyboard as reported (Figure 2). Assessing both sets of data simultaneously, suggested that mobility did not necessarily coincide with "keyboardless" interaction with computers in a clinical setting. For access, residents greatly seemed to favor, both before and after intervention, a system that always responded to queries in less than five seconds (Figure 3), always displayed X-rays and other images in less than 30 seconds (Figure 4), and always functioned without any "down time" (Figure 5). Such data underscored the idea that speed and reliability were important to mobile access to clinical information.

Limitations

One significant limitation, despite a modest sample size which caused the authors to use a liberal significance level of p<0.10 instead of p<0.05, is that the two-sample statistical analysis used was less optimal for pre- and post-intervention studies. A paired-sample analysis in which each participant serves as own control between the two waves would have been more ideal. However, due to the anonymous nature in which the study was administered, a unique identifier could not be used to link participant responses in the two waves. With no ID variable to link the wave responses for each participant, the characteristics of completers versus the non-completers could not be compared. Another limitation was the smaller number of participants who completed both phases. There was no means to determine whether completers represented a high level of enthusiasm relative to non-completers. Larger multi-institutional studies are recommended in which participant responses are linked in each wave to allow a paired analysis. Furthermore, our small sample size did not allow for multivariable adjustment for potential confounding due to age and gender. It is therefore possible that findings for this study could be attributed to residual confounding that could not be adjusted for multivariable regression analysis.

Conclusion

Regardless of the above weaknesses, this 18-month pilot study offered an encouraging glimpse into merits of formal introduction of mobile access to clinical information and how it impacted internal medicine residents' computer-related attitudes and selected preferences regarding accessing clinical information using iPads. It was apparent that the introduction of mobile access to clinical information was well received by internal medicine residents at our institution because it facilitated medical education and direct healthcare delivery. Even though participants were technology-literate, our findings suggest that institutions play a critical role by providing technology training to their resident physicians, especially with regard to accessing authoritative clinical information from library subscriptions. Depending on the size of institutions and associated allied health programs, health science libraries pay millions of dollars in subscriptions for resources. Skillfully searching and mastering different aspects of these resources can be challenging. It is recommended that future studies be conducted at multiple sites and participants should include mid-level healthcare professionals such as Physician Assistants and Advanced Nurse Practitioners.

Table 1: Composite Scores
Composite Scores	Pre	Post	Percentage Change	P-value*
	N=68	N=45
	mean(SD)	mean(SD)
Computer Experience	21 (6.3)	23 (6.0)	9.52%	.09
Computer Knowledge	17.3 (5.7)	17.9 (5.7)	3.47%	.58
Perceived Necessary Capabilities of Computer Systems in Medicine	64.6 (15.8)	67.6 (8.8)	4.64%	.25
Familiarity with Medical Information Apps	8.5 (4.5)	12.8 (3.5)	50.59%	<.0001
Potential Effects of Computers	51.9 (9.8)	51.2 (13.9)	-1.35%	.77
Appreciation for EMR	70.8 (12.3)	73.7 (15.4)	4.10%	.10
*P-value obtained from paired t-test; **Percentage Change = (mean-post / mean-pre - 1)*100

Computer Experience was derived by summing up 17 items (possible range of values = 0 - 40)

Computer Knowledge was derived by summing up 14 items (possible range of values = 0 - 28)

Perceived Necessary Capabilities of Computer Systems in Medicine was derived by summing up 17 items (possible range of values = 0 - 68)

Familiarity with Medical Information apps was derived by summing up 13 items (possible range of values = 0 - 26)

Potential Effects of Computers was derived by summing up 17 items (possible range of values = 0 - 68)

Appreciation for EMR was derived by summing up 22 items (possible range of values = 0 - 88)