ReadyVax: A new mobile vaccine information app.

Vaccine information of varying quality is available through many different sources. We describe the creation, release and utilization of ReadyVax, a new mobile smartphone app providing access to trustworthy, evidence-based vaccine information for a target audience of healthcare providers, pharmacists, and patients (including parents of children). We describe the information content and technical development of ReadyVax. Between the hard launch of the app on February 12, 2015 and October 8, 2016, the app has been downloaded by 5,142 unique users, with 6,841 total app sessions initiated, comprising a total of 15,491 screen views (2.3 screens/session on average). ReadyVax has been downloaded by users in 102 different countries; most users (52%) are from the United States. We are continuing outreach efforts to increase app use, and planning for development of an Android-compatible version of ReadyVax, to increase the available market for the app.

Introduction

Given recent experiences with pandemic influenza, seasonal influenza epidemics, and pertussis outbreaks, there is an ever-present need to relay important vaccine-related information to populations particularly vulnerable to the effects of infectious diseases. Depending on the nature of the threat and populations at risk, messaging may need to take different forms. Routine vaccine promotion activities are typically educational in nature, informing the target population about vaccines, vaccine-preventable diseases, and populations indicated for vaccination. However, changing recommendations in response to new evidence necessitates widespread and targeted dissemination of new information. For example, in the context of pertussis and influenza outbreaks, messaging to inform pregnant women of the importance of routine tetanus, diphtheria, and acellular pertussis (Tdap) and seasonal influenza vaccination during every pregnancy takes on heightened importance.

Smartphone ownership is high in the United States – 64% among adults in the general population and 80% among physicians. Similarly, healthcare information seeking is common. In 2012, more than half (52%) of smartphone users reported accessing health information on their phones, with 19% having a health-related app on their mobile device. Additionally, many physicians report using smartphone during medical rounds. Having a readily available reference on a smartphone or other mobile device without having to consult a book or website on a computer may add efficiency to the clinical encounter with regard to providing vaccine information to patients. With an ever-increasing number of preventive services recommended and less time available to spend on patient-level counseling, methods such as mobile app usage for checking key references may offer advantages to providing timely and appropriate preventive care.

Information needs related to vaccines and vaccine-preventable diseases are varied. Our previous research has highlighted the information needs and preferred delivery mechanisms for vaccine-related information. These findings, including provision of information before clinical encounters and providing access to information of increasing levels of detail to be used by the user to meet their needs (e.g. per the Elaboration Likelihood Model), were used to develop a multi-level intervention to improve vaccine uptake among pregnant women that included a tablet-based educational model. The use of mobile technologies for vaccine information provision during clinical encounters is a logical step toward improving vaccination recommendations.

Developing a mobile smartphone application (“app”) for use by traditional vaccine providers, pharmacists, and patients or parents of pediatric patients to readily obtain vaccine-related information is a valuable public health systems tool. In addition to providing access to standard information about vaccines, this tool can be used in the event of an emergency to empower public health authorities to convey messages to target populations as quickly as possible.

To assess the use of a smartphone app to provide access to information about vaccines and the diseases they prevent, we created ReadyVax. ReadyVax is a new mobile smartphone app designed to provide access to trustworthy, evidence-based vaccine information. We have made ReadyVax available for free download in the Apple iTunes app store with regard to routine vaccination recommendations, to determine the potential for this app to be used. Here we describe the creation, release and to-date utilization of ReadyVax.

App development and evaluation

Information content was structured around four primary categories: Vaccines, Diseases, Answers to Common Questions, and Resources. For vaccines, information was obtained from relevant CDC publications, including the Morbidity and Mortality Weekly Report-based publication of the Advisory Committee on Immunization Practice (ACIP) recommendations for each vaccine as well as the book “Epidemiology and Prevention of Vaccine-Preventable Diseases,” also known as the CDC “Pink Book.” Disease specific information and answers to common questions were synthesized from MMWR publications of ACIP recommendations, the “Pink Book,” and peer-reviewed published manuscripts as needed. The list of web-based resources was collated from the project team's knowledge of available organizations and information repositories.

Specifications for app functionality were defined by the project team, and are presented in Table 1. Technical development of the app was conducted by AnyPresence, LLC (Reston VA), which specializes in mobile backend as a service (MBaaS) app development and optimization. The project team developed a variety of data flow models to address user accessibility to the app content. Proposed data flow models were presented to the technical developer and optimized through iterative discussions between the project team and developer. Following finalization of data flow, the vendor developed wireframes to guide app development. Routine technical and vaccine-related discussions occurred between the project team and the technical developer to refine the wireframes and ultimately the app.

Table 1.

ReadyVax app technical specifications.

Native app that can be downloaded and installed directly on smartphone or tablet

Information content will be available even with no network connection.

Updated information will be downloaded automatically when app is used on a network connection

Information will be browsable and searchable

Information content can be tailored to specific user groups

User type specifications: Healthcare provider, Pharmacist, Patient/Parent

Information content can be updated through a web-based dashboard interface

Messages can be pushed out to app users, with Alert Notifications appearing on the users' device

Links to multimedia (e.g., YouTube videos) can be embedded

The project team received the initial version of the ReadyVax app for pilot testing on November 12, 2014, and it was initially housed for download on the Emory University Information Technology internal app store. Members of the project team and other colleagues were invited to download the ReadyVax app and conduct user testing. Feedback from this user testing was provided to the technical vendor to make the requested updates. The production version of the app was received by the project team in early January 2015 and submitted to the Apple iTunes app store. ReadyVax was available for public download on January 23, 2015. Following project team-led downloading and use of the app to ensure that the publicly available version worked properly, we began promotion of the app (i.e. our “hard launch”) to public health partners on February 12, 2015. ReadyVax is available through the Apple iTunes store, at https://itunes.apple.com/us/app/readyvax/id957851259?mt=8.

We worked with public health partners in Georgia, Oregon, Washington and New York State to publicize the availability of the ReadyVax app. Additionally, press releases were provided by the Emory University Media Relations office to media outlets, and project team members worked with vaccine advocacy groups to promote the availability of the app on their websites and through their email newsletters.

The project team monitored app acquisition and usage through Google Analytics. We routinely monitored the number of unique acquisitions of the app, total user sessions initiated, engagement time, and number of screen views within the app. For ReadyVax a screen view consists of viewing information in one area. There are ten relevant screens available in ReadyVax: Home (general navigation home screen; see Fig. 1, upper left panel), VaccineDiseases (general home screen from which users can select to view either vaccines or diseases; see Fig. 1, upper right panel), Vaccine (information on individual vaccines; see Fig. 1, lower left panel), Disease (information on individual diseases), Questions (frequently asked questions), Resources (list of external vaccine information resources; see Fig. 1, lower right panel), Alerts (messages provided through push message notifications), Role (screen to set user role), About (general information about the app), and Menu (general menu to view User Role and About information). Within a given screen, multiple pieces of information can be accessed. For example, one screen view of Questions could consist of the user viewing multiple questions and their answers. Likewise, one screen view of Vaccine could involve viewing information on more than one vaccine.

Figure 1.

Representative screenshots of ReadyVax: A. Home screen; B. Vaccines tab of the Vaccines & Diseases screen; C. Representative vaccine entry; D. Resource Information list.

The ReadyVax app project was approved as Exempt research by the Emory University Institutional Review Board.

Results

Representative screenshots of the ReadyVax app are shown in Figure 1.

Between the ReadyVax “hard launch” on February 12, 2015 and October 8, 2016, the app has been downloaded by 5,142 unique users, with 6,841 total app sessions initiated (average 1.3 sessions/user), comprising a total of 15,491 screen views (average 2.3 screens/session). On average, a session lasted approximately 55 seconds. There is a subset of users (N = 427 (8.3%), as of October 8, 2016) who are repeat users of ReadyVax. These users are responsible for 1,648 unique app sessions (average of 3.9 sessions per user), with an average of 5.6 screen views per session and an average session duration of one minute 53 seconds.

ReadyVax has been downloaded by users in 107 countries (Fig. 2); of the 4,106 unique users for whom we have location information 2,183 (52%) were from the United States; US users accounted for 3,550 of the 5,770 sessions (62%) and 13,089 of the 15,471 screen views (85%) for which we have location information. Of the 453 users who have repeatedly accessed ReadyVax for whom we have location information, 383 (85%) were from the United States; US-based users accounted for 1,367 of 1,645 sessions (84%) and 7,773 of 9,208 screen views (84%) by repeat users. There were a total of 37 countries from which returning users originated.

Figure 2.

World map showing countries from which ReadyVax has been accessed, February 12, 2015 to August 4, 2016.

View counts and engagement of individual screens are summarized in Table 2. The most common content-focused screen accessed was Vaccine, which details information about specific immunizations (2,590 views overall, with 1,519 views among the 427 returning users).

Table 2.

User engagement with specific ReadyVax screens, February 12, 2015 to October 8, 2016.

		All Users (N = 5,142)				Returning users (N = 427)
		Screen views (N)	Screen views per session	Average time on screen (mm:ss)	% exiting app	Screen views (N)	Screen views per session	Average time on screen (mm:ss)	% exiting app
Content screens
	VaccinesDiseases	3,676	0.54	0:12	6.1	2,119	1.29	0:12	7.2
	Vaccine	2,590	0.38	0:55	30.5	1,519	0.92	1:03	34.2
	Disease	451	0.07	0:23	9.3	282	0.17	0:25	9.2
	Alerts	1,277	0.19	0:25	35.1	841	0.51	0:22	41.7
	Questions	519	0.08	0:56	31.0	274	0.17	0:52	32.1
	Resources	314	0.05	0:21	32.5	177	0.11	0:25	36.2
Administrative screens
	Home	4,878	0.71	0:16	9.4	2,906	1.76	0:15	9.3
	Menu	1,281	0.19	0:08	2.4	778	0.47	0:10	2.7
	Role	369	0.05	0:14	5.1	225	0.14	0:18	6.7
	About	136	0.02	0:51	19.9	95	0.06	0:42	21.1
	Total	15,491	2.26	N/A	N/A	9,216	5.59	N/A	N/A

The most common vaccine-related screen events (i.e., data elements accessed on a particular screen) were for diphtheria, tetanus, and pertussis vaccine (18% of all vaccine data screen events); general vaccine information (17% of all vaccine data screen events); measles, mumps, rubella vaccine (11% of all vaccine data screen events); and pneumococcal vaccines (11% of all vaccine data screen events) (Table 3).

Table 3.

Vaccine- and disease-specific screen events captured within the Vaccine, Disease, or VaccineDisease screens, ReadyVax, February 12, 2015 to October 8, 2016.

	Vaccine data screens					Disease data screens
	All users (N = 350 events)		Returning users (N = 203 events)			All users (N = 132 events)		Returning users (N = 92 events)
Vaccine name	N	%	N	%	Disease name	N	%	N	%
Diphtheria, tetanus, and pertussis (DTaP and Tdap)	64	18.3%	36	17.7%	Diphtheria, tetanus, and pertussis (DTaP and Tdap)	24	18.2%	23	25.0%
Haemophilus influenzae type b (Hib)	8	2.3%	4	2.0%	Haemophilus influenzae type b (Hib)	9	6.8%	7	7.6%
Hepatitis A	18	5.1%	13	6.4%	Hepatitis A	18	13.6%	11	12.0%
Hepatitis B	3	0.9%	1	0.5%	Hepatitis B	7	5.3%	6	6.5%
Human papillomavirus (HPV)	17	4.9%	11	5.4%	Human papillomavirus (HPV)	13	9.8%	10	10.9%
Influenza (flu)	35	10.0%	25	12.3%	Influenza (flu)	9	6.8%	5	5.4%
Measles, Mumps, Rubella	39	11.1%	13	6.4%	Measles	6	4.5%	2	2.2%
(MMR)					Mumps	3	2.3%	2	2.2%
					Rubella	6	4.5%	5	5.4%
Meningococcal (Meningitis) Vaccines*	33	9.4%	24	11.8%	Meningococcal disease	16	12.1%	10	10.9%
Meningococcal Conjugate Vaccine (MCV4)	10	2.9%	8	3.9%
Pneumococcal vaccines	38	10.9%	18	8.9%	Pneumococcal disease	7	5.3%	4	4.3%
Polio	6	1.7%	4	2.0%	Polio	2	1.5%	1	1.1%
Rotavirus	7	2.0%	2	1.0%	Rotavirus	4	3.0%	2	2.2%
Varicella (chickenpox)	4	1.1%	4	2.0%	Varicella	1	0.8%	1	1.1%
Zoster (shingles)	9	2.6%	5	2.5%	Zoster (shingles)	7	5.3%	3	3.3%
General vaccine information	59	16.9%	35	17.2%

Includes general information on meningitis vaccines, including information on the Serogroup B vaccine.

Discussion and next steps

With over 5,100 app downloads in one year for a mobile app focused on a very specific medical and public health topic area that is available for only one major smartphone operating system, we believe that ReadyVax is capable of establishing itself as a primary source of vaccine information. ReadyVax provides a comprehensive suite of information on the vaccines and the diseases they prevent. While there are other mobile apps related to vaccines, most exist solely as a way to either record immunization history or view immunization schedules (e.g., CDC Vaccine Schedules; My Immunizations). We are aware of only one other comprehensive vaccine information app – Vaccines on the Go, published by Children's Hospital of Philadelphia. Because of funding constraints during the initial development of ReadyVax, we were only able to provide an Apple iOS-compatible version. As of March 2015, Android smartphones accounted for 52% of the smartphone market share in the use, followed by Apple iOS (43%), so expansion to the Android platform could easily double the availability of the app.

With slightly less than 10% of ReadyVax users repeatedly accessing the app, we recognize the need to query ReadyVax users regarding their reasons for use or non-use of the app. However, this does not appear to be a situation unique to ReadyVax. A recent Nielsen consumer survey found that even though app downloads are common – the average smartphone user has 42 apps downloaded – most go unused, as more than half of smartphone users report using only between one and four of the apps on their device daily. In light of these Nielsen findings, ReadyVax is not unique as an app that may be downloaded but not routinely accessed. However, given the degree of engagement observed among our repeat ReadyVax users, there is evidence that value is seen in this type of app.

We are currently planning for expansion of ReadyVax to the Android platform, as well as making an HTML5 compatible version. By increasing the market reach of ReadyVax, and continuing development and refinement of app content and messaging, we anticipate increasing the ability of healthcare providers and the public to be better informed about vaccines. Through this type of routine usage, we believe that ReadyVax can prove itself as a trusted source of vaccine information. Additionally, while we have used the push notification function of the app in limited capacity (e.g., to highlight newly recommended vaccines), routine utilization of this messaging function can have the potential to reach users rapidly in the event of a public health emergency. This is a focus area of the ReadyVax project team moving forward. As part of enhanced dissemination efforts, we are planning more in-depth data collection on users' perceptions of the app and reasons for and against repeat use. Included in these data collection methods, we plan to use the push notification function within ReadyVax to alert users to an online survey, with a direct link to the survey in the notification message. These evaluations will assess perceptions of content, layout, user interface, and perceived utility of the ReadyVax app. As part of the development of the new version of ReadyVax, we are also designing outreach efforts to healthcare providers for pre-release testing and feedback on usability and design, in addition to content. Additionally, we will conduct pre-release usability assessments with parents. As part of both of these outreach efforts, we will evaluate user's perspectives on presentation or functionality that will increase repeat use.

In a short period of time, we have developed and disseminated a new mobile smartphone app to provide easy access to evidence-based information about vaccines and vaccination. Initial uptake and use has shown that this type of app can be accepted by smartphone users. We plan to continue updating and monitoring the use of ReadyVax to improve the app, to allow us to serve the vaccine information needs of healthcare providers and the general public. This includes routine information needs using the standard content in ReadyVax as well as rapidly updated information through the enhanced use of push notifications.