Acceptability, Utility, and Cost of a Mobile Health Cancer Screening Education Application for Training Primary Care Physicians in India.

PURPOSE: Mobile health (mHealth)-based oncology education can be a powerful tool for providing cancer screening knowledge to physicians, as mobile technology is widely available and inexpensive. We developed a mobile application (M-OncoED) to educate physicians on cancer screening and tested the acceptability, utility, and cost of two different approaches to recruit physicians.
METHODS: M-OncoED was designed to perform pre- and postlearning assessments through the in-built quizzes; present case studies and educational materials for cervical, breast, and oral cancer screening; collect responses to interactive queries; document module completion; send reminders and alerts; and track user metrics, including number of sessions to complete each module and time spent per session. We tested two recruitment approaches: a broad-scale recruitment group, for which we relied on e-mails, messaging apps (e.g., WhatsApp), and phone calls, and the targeted recruitment group, for which we conducted a face-to-face meeting for the initial invitation.
RESULTS: Overall, about 35% of those invited in the targeted group completed the course compared with about 3% in the broad-based recruitment group. The targeted recruitment approach was more cost-efficient ($55.33 vs. $109.43 per person). Cervical cancer screening knowledge increased by about 30 percentage points, and breast cancer screening knowledge increased by 10 percentage points. There was no change in knowledge for oral cancer scorings.
CONCLUSION: This study has demonstrated the feasibility and utility of using an mHealth app to educate physicians. A more intensive hands-on recruitment approach is likely required to engage physicians to download and complete the app. Future studies should assess the impact of mHealth tools on physician behavior and patient outcomes. IMPLICATIONS FOR PRACTICE: Mobile health (mHealth)-based oncology education can be a powerful tool for providing cancer screening knowledge to physicians, as mobile technology is widely available and inexpensive. This study has demonstrated the feasibility and utility of using an mHealth app to educate physicians and illustrates the type of recruitment approach (face-to-face) that is likely required to incentivize physicians to download the app and complete the training.

Introduction

Over the past several decades, cancer cases in India have increased substantially [1, 2]. India's cancer burden was estimated at 1.15 million cases in 2018 and is predicted to double by 2040 [3]. Furthermore, a large proportion of cancers are detected at a late stage in India, which leads to substantial suffering and economic burden at both the individual and societal levels [4, 5]. Although India does have published cancer screening guidelines, the uptake of population‐based screening has been slow. Active engagement of the health system, specifically primary care providers, can shape the trajectory of cancer screening and thereby improve cancer outcomes [6, 7, 8]. However, among primary care providers, the awareness of cancer risk factors, symptoms, effective early detection screening approaches, and appropriate triage for diagnosis and treatment is suboptimal [9, 10, 11, 12, 13].

Oncology is embedded within the curriculum for medical students in India and is not a comprehensive, stand‐alone topic, which may limit cancer‐specific knowledge or importance of the issue. Moreover, there is no committed in‐service orientation of cancer screening and early diagnosis for primary care doctors, which limits provider‐initiated early detection of cancer. Additionally, even when physicians who are currently practicing in primary care have some knowledge and awareness of cancer screening, there is no consistent mechanism of continuing education to update their training and ensure that they have current and updated guidance on offering cancer screenings. Studies in other settings have shown that physician recommendation is a key factor in increasing compliance with prevention and screening interventions [14, 15, 16, 17].

Mobile health (mHealth)–based oncology education can be a powerful tool for providing cancer screening knowledge to physicians, as mobile technology is widely available and inexpensive [18, 19, 20]. We developed a mobile application (M‐OncoED) to educate physicians on cancer risk factors, symptoms, and screening approaches. In this article, we report findings from a comprehensive assessment of the acceptability and utility of the M‐OncoED in increasing physician knowledge of cancer screening and early detection. Additionally, we present the effectiveness and cost of the two different approaches we used to recruit providers to participate in mHealth education on cancer screening.

Materials and Methods

App Content Development

We brought together a multidisciplinary team to develop the M‐OncoED application. The team included physicians, cancer experts, mobile technology consultants, and patient advocates. The package consists of learning modules to educate primary care physicians on cancer prevention strategies, risk factors, patient symptoms, screening approaches, early detection benefits, referral pathways, and treatment options. Priority for the educational modules was placed on cancers with a high regional incidence with evidence‐based prevention and early detection guidelines. Using these criteria, we selected cervical, breast, and oral cancers. We identified an expert for each type of cancer and tasked them with developing the app content through an iterative process to incorporate feedback from the multidisciplinary team members who reviewed all content materials. In all, the education course consisted of multiple teaching modules, case studies, reference links, and pre‐ and postquizzes, and each session was tailored to India's epidemiological and cultural context. The breast cancer education module consisted of 118 screens, followed by 75 screens for the cervical cancer module and 66 screens for the oral cancer screening module. Figure 1 presents a screenshot of one of the screens of the M‐OncoED app. The quizzes consisted of 10 multiple choice responses to assess knowledge before and after completion of the education modules for each type of cancer. These quizzes were tested with 10 to 12 medical students to ensure that the pre‐ and postquizzes were at a similar level of difficulty. On the basis of the proportion of correct responses received, we reframed and reallocated the questions before and after sessions of the course to ensure comparability.

Figure 1

Example from the M‐OncoED app.

Technical Specifications, Features, and Functionality

The mobile application was developed for the Android operating system, which encompasses more than 90% of the smartphone market in India. The application has a server component through which we created training modules, uploaded content, and retrieved performance data for analysis. Users downloaded the app via a Web site and then entered their demographic information, including age, gender, practice setting, and practice details related to cancer screening. After submitting this information, they could begin viewing content. The mobile learning tool was designed to conduct pre‐ and postlearning assessments through the in‐built quizzes; present case studies and educational materials; display reference materials and Web site links; accommodate multiple learning modules; collect responses to interactive queries; document module completion; send reminders and alerts; and track user metrics, including number of sessions to complete each module and time spent per session. All respondents had to complete the demographics information and the pre‐ and postquizzes. Viewing of all other content could be personalized, and individuals could skip over screens or spend additional time viewing specific screens.

Pretesting and Finalization of the App

First, we recruited physicians for a series of focus groups and interviews throughout the design and launch of the application to ensure a high‐quality user interface. We conducted three focus groups—each consisting of 8 to 10 physicians and cancer screening experts—to review various features of the application and provide feedback on the content. Second, we conducted 10 in‐depth interviews with a variety of physicians to obtain feedback during the development of the app on usability and functions. Last, before finalization, eight volunteers reviewed the app's user‐friendliness and display of the content. Reviewers were cancer screening experts, medical students, primary care physicians, and cancer treatment specialists.

Recruitment

We collaborated with the Directorate of Health Services and the Indian Medical Association in Kerala, a southern state in India, to recruit primary care physicians for the study. We obtained e‐mail and telephone numbers from the database maintained by the collaborating organizations. As displayed in Table 1, we used two different approaches to recruit provider groups: a broad‐scale recruitment group using e‐mails, messaging apps (e.g., WhatsApp), and phone calls to ensure we could quickly reach a large number of physicians and a targeted recruitment group using face‐to‐face personal interactions.

Table 1

Recruitment groups and overview of recruitment approach

Characteristic	Broad‐scale recruitment group	Targeted recruitment group
Physician type	Government and private primary care physicians from Kerala state.	Government and private primary care physicians from the Trivandrum district, Kerala.
Recruitment approach
Initial invitation	E‐mail/WhatsApp messages with follow‐up telephone calls to provide additional information.	Face‐to‐face meetings with PowerPoint overview presentation and assistance to download the app.
Follow‐up reminders	Reminders to initiate and complete the app were sent by e‐mail, WhatsApp, and text messages. Phone calls were made to those who started the course on the app.	Reminders to complete app were sent by e‐mail, WhatsApp, and text messages. These were followed by telephone calls.
Type of smartphone	Anticipate that majority of the targeted group would have Android phones, but not all.	All those recruited had Android phones.

The broad‐scale recruitment group consisted of government and private primary care physicians from Kerala state, whereas the targeted recruitment group consisted of primary care physicians from the Trivandrum district in Kerala, across government and private sectors. The recruitment began in March 2019, and all follow‐up activities were completed by December 2019.

Data Collection and Analysis

The study team tracked all individuals invited to participate, reminder follow‐up messages and telephone calls, and the requests made for technical assistance. The demographic information captured at course initiation was summarized to identify age, gender, practice setting, and cancer screening practice behavior of the physician. We also analyzed the demographic characteristics of those who completed the course and those who did not and conducted F‐tests and chi‐square tests as appropriate to identify statistically significant differences at the 5% level. User interaction that was automatically tracked within the app was analyzed to identify number of sessions by cancer screening module and average time spent per cancer screening module. Additionally, we requested qualitative and quantitative feedback on user experience. We asked participants to indicate whether they would recommend the app to others and to identify whether the app was a “very effective,” “effective,” or “not effective” approach to educate physicians. Furthermore, pre‐ and post‐test results captured in the app were used to compare the percentage of correct responses before and after completing the cancer screening module. Pre‐ and postscores were calculated for respondents who completed both the pre‐ and post‐test for a given module. We report the percentage of correct responses separately for each cancer screening module and conducted testing to determine statistically significant based on the differences in the pre‐and postscores reported for each screening module. We conducted both paired t tests and Wilcoxon signed rank tests (to account for instances when normality of the mean values cannot be assumed) for each cancer module and report significance at the 5% level. In all comparisons, the p value from the t tests and Wilcoxon tests were very similar. We developed a detailed list of activities involved in planning (compiling list of physicians with contact details), inviting participants, conducting follow‐up (including sending reminders), and providing technical assistance to each group to assess the overall cost of implementing the two different recruitment approaches. We tracked the hours required for each activity and maintained records of nonlabor and other support costs, such as technical support to send bulk text messages. Labor hours were converted to cost per activity by using standard Indian wage per hour for the staff involved in each activity. Costs are reported in both Indian rupees and in U.S. dollar values using a conversion rate of 75 Indian rupees to one U.S. dollar. We present the cost per person invited, per person who initiated the course, and per person who completed the course.

Results

Of the 3,917 providers who were invited to the broad‐scale recruitment group, 525 (13.4%) created an account, 317 started the course (8.1%), 111 (2.8%) completed the course, and 109 (2.78%) received a certificate of completion. In contrast, of the 46 providers who were invited to the targeted recruitment group, 46 (100.0%) created an account, 35 started the course (76.1%), 19 (41.3%) completed the course, and 16 (34.8%) received a certificate of completion. Figure 2 displays the flow of providers along the path to course completion.

Figure 2

Response rate by recruitment group.

Table 2 summarizes the demographic characteristics of the 316 respondents in the broad‐scale recruitment group and 35 respondents in the targeted recruitment group who started the course. For both groups, the majority of the participants who initiated the app were younger than 40 years and were predominantly women: 57.3% in the broad‐scale group versus 68.6% in the targeted group. The principal practice setting for both broad‐scale (65.2%) and targeted recruitment (57.1%) groups was a government facility. A majority of the physicians in both groups indicated that they provide advice on cancer screening, with the largest proportion indicating advice for breast cancer screening (88.0% and 82.9% in broad‐scale and targeted groups, respectively). A smaller proportion of physicians reported that they conducted cancer screenings, with the highest percentages reported for oral cancer screening (56.3% and 42.9% in broad‐scale and targeted groups, respectively). The table also presents the demographic characteristics of those who completed the course in both groups. In the targeted recruitment group, participants younger than 40 years were more likely to complete the education course than those in the older age category, but this was not a statistically significant finding in the broad‐scale recruitment group. Overall, female physicians appeared to be more likely to complete the course than their male counterparts. Additionally, those who provided advice on screening or conducted screenings were generally more likely to complete the course in the app than those who did not.

Table 2

Demographics and practice details of physician participants

Characteristic	Broad‐scale recruitment group, n (%)		Targeted recruitment group, n (%)
	Initiated course in app	Completed course in app	Initiated course in app	Completed course in app
Sample size	316	111	35	19
Age
Younger than 40 years	250 (79.1)	90 (81.1)	20 (57.1)	14 (73.7) a
40 years or older	62 (20.9)	21 (18.9)	15 (42.9)	5 (26.3) a
Gender
Male	135 (42.7)	31 (27.9) a	11 (31.4)	5 (26.3)
Female	181 (57.3)	80 (72.1) a	24(68.6)	14 (73.7)
Practice setting
Government facility	206 (65.2)	66 (59.5) a	20 (57.1)	10 (52.6)
Private facility	42 (13.3)	9 (8.1) a	8 (22.9)	6 (31.6)
Not reported	68 (21.5)	36 (32.4) a	7 (20.0)	3 (15.8)
Provide advice on cancer screening
Breast	278 (88.0)	106 (95.5) a	29 (82.9)	17 (89.5)
Cervical	227 (71.8)	93 (83.8) a	23 (65.7)	13 (68.4)
Oral	231 (73.1)	92 (82.9) a	24 (68.6)	14 (73.7)
Conduct cancer screening
Breast	157 (49.7)	65 (58.6) a	13 (37.1)	7 (36.8)
Cervical	89 (28.2)	44 (39.6) a	6 (17.1)	3 (15.8)
Oral	178 (56.3)	66 (59.5)	15 (42.9)	9 (47.4)

Indicates statistically significant difference at the 5% level between those who completed versus those who did not complete the course in the app.

Table 3 summarizes key metrics by recruitment group for the respondents that completed the course across three cancer screening modules covered by the application: breast, cervical, and oral cancer. The average total minutes for course completion (introduction and screening modules) was 136.52 for the broad‐scale recruitment group and 125.97 for the targeted recruitment group, with average time per cancer screening module ranging from 26.25 to 37.73 minutes. The average number of sessions to complete each module ranged from 3.25 sessions for the oral cancer module to 6.31 sessions for breast cancer module in the broad‐scale recruitment group and similarly 3.21 and 8.28 sessions for oral and breast cancer screening modules, respectively, for the targeted recruitment group. Table 3 also presents the feedback received from respondents, which was overwhelmingly positive. Overall, 95.3% indicated that they would recommend the app to others, and 97.6% found the app to be a “very effective” or “effective” approach to educate physicians. Suggestions for improvement included the following: (a) better functionality of the app to reduce delays in loading content; (b) more videos, pictorial content, and supplemental reference documents; (c) inclusion of other cancer modules; and (d) clearer instructions on assessing additional reading materials.

Table 3

Completion of learning modules and user feedback

Learning Module Completion	Broad‐scale recruitment group (n = 111)	Targeted recruitment group (n = 19)
Time to complete modules
Total duration, average minutes	136.52	125.97
Average time per module, minutes
Breast cancer	37.73	30.84
Cervical cancer	33.66	26.25
Oral cancer	36.25	30.16
Average number of sessions per module
Breast cancer	6.31	8.28
Cervical cancer	4.31	4.79
Oral cancer	3.25	3.21
Feedback on M‐OncoED app, across broad‐scale and targeted recruitment groups (n = 128)
Participants who would recommend app to others, %	95.3
Participants who found the app to be
Very effective, %	55.4
Effective, %	42.2
Participant qualitative feedback
Positive comments on app	“Good initiative to help medical professionals to update their knowledge” “Excellent Continuing Medical Education” “I'm very satisfied with the app” “Very interesting method for studying” “Very useful knowledge on protocols to follow” “Found it very informative”
Areas for improvement	“Loading of pages is too slow causing delays” “Include videos in modules” “More pictorial representation could improve content delivery” “Better instructions to guide user to additional readings” “Add additional cancer modules” “Additional information on follow up reading on screening topics”

The results from the pre‐ and postassessments are presented in Figure 3. The proportion of pretest correct responses was lowest for the cervical cancer module (around 40%) and highest for the oral cancer module (around 77%). The postscores increased across all combinations of recruitment group and cancer screening module, with the exception of broad‐scale recruitment group for the oral cancer module, which did not change (prescore: 76.71%; postscore: 76.24%). Statistically significant increases were observed in the broad‐scale and targeted recruitment groups for the breast cancer and cervical cancer modules. The changes in pre‐ and postscores were the largest for cervical cancer screening, about 30 percentage points. Breast cancer screening scores increased by about 10 percentage points, and, overall, there was no change in the oral cancer screening scores (Fig. 3).

Figure 3

Pre‐ and post‐test scores for each screening module.

Cost Assessment

Table 4 presents the estimate from the cost analysis. The cost data for implementing the targeted recruited group and broad‐scale recruitment group are shown separately in both Indian rupees and U.S. dollars. Overall, the broad‐scale recruitment group implementation incurred a total cost of $12,146.67 compared with $1,013.33 for the targeted recruitment group. On a per capita basis, the broad‐scale recruitment approach was the much less expensive option per person invited than the targeted recruitment ($3.10 vs. $20.68 per person invited). Nevertheless, based on the completion rate, the targeted recruitment approach had a lower cost per person completing the course than the broad‐scale recruitment ($53.33 vs. $109.43).

Table 4

Cost analysis of methods of recruitment used to target physician participants

Participation and Cost Categories	Targeted recruitment group	Broad‐scale recruitment group
M‐OncoED app participation
Invited	49	3,917
Course initiated	35	316
Completed	19	111
Cost in Indian rupee
Total cost	₹76,000.00	₹911,000.00
Cost per person invited	₹1,551.02	₹232.58
Cost per person who initiated course	₹2,171.43	₹2,882.91
Cost per person who completed course	₹4,000.00	₹8,207.21
Cost in U.S. dollars
Total cost	$1,013.33	$12,146.67
Cost per person invited	$20.68	$3.10
Cost per person who initiated course	$28.95	$38.44
Cost per person who completed course	$53.33	$109.43

Discussion

We compared two different recruitment approaches to invite primary care physicians to complete mHealth‐based cancer screening education using the M‐OncoED app. The targeted approach, which required more hands‐on involvement with participants, including face‐to‐face interaction, was more effective than the broad‐scale recruitment approach, which was designed to reach a wider audience using the less intensive options of e‐mail, WhatsApp, and text messaging. Overall, about 35% of those invited completed the course in the targeted group compared with about 3% in the broad‐based recruitment group. Only about 13% of those invited in the broad‐scale group even downloaded the app, compared with everyone who was invited in the targeted group. Nevertheless, even among those who downloaded the app in the broad‐scale recruitment group, only about 60% (316/525) moved to the next step of completing the demographics information to initiate the course compared with about 76% (35/46) in the targeted recruitment group. So overall, at each step, the targeted recruitment effort performed better than the broad‐scale recruitment approach. Thus, even though the broad‐scale recruitment was quite inexpensive, only $3.10 per person invited compared with $20.68 per person invited in the targeted group, the overall cost per person who completed the course ($109.43 vs. $53.33) revealed that the targeted recruitment approach was more cost‐efficient.

In general, physicians who completed the course exhibited improvement in knowledge related to cervical and breast cancer screening. Cervical cancer knowledge increased by about 30 percentage points and breast cancer by 10 percentage points. We did not see a change in the knowledge for the oral cancer modules, which could reflect the overall high level of awareness among physicians, as almost 80% were able to correctly answer the questions posed in the pretest quiz of the oral cancer screening module. Kerala is in the forefront of oral cancer screening studies [21, 22], and thus physicians in Kerala likely have a high level of awareness of oral cancers and screening approaches. There are a limited number of studies that have evaluated the effectiveness of eHealth tools, such as online sessions, to deliver education to improve cancer care [23, 24, 25, 26]. These studies have reported increases in knowledge that range from 8% to 24%, which is similar to the changes reported in the present study for cervical and breast cancer screening. This growing body of eHealth and mHealth assessment studies is building the evidence base on the utility of these tools to improve cancer knowledge among physicians.

This study has demonstrated the feasibility of using an mHealth app to educate physicians. Each individual cancer screening module was completed in approximately 30 minutes, and participants completed the education materials over multiple sessions. Thus, physicians were able to tailor the amount of time they spent on the app and could devote short durations of time. Participants provided overwhelmingly positive feedback on their experience learning via the M‐OncoED app. The M‐OncoED platform could be an ideal tool to delivery education content to busy physicians who do not have long periods of uninterrupted time. There is a growing call to implement cost‐effective approaches to reduce the burden from cancers [27, 28], and mHealth training can offer a powerful tool to support these efforts. As shown in this study, mHealth apps will only be effective if participants use them, and, therefore, recruitment and retention efforts require a multimodal approach; face‐to‐face interaction is likely required along with remote app‐based learning. Additionally, there is growing evidence in the literature that multicomponent interventions are required to ensure sustainable increase in guideline‐recommended cancer screenings [29]. Physician education via the M‐OncoED app can be supplemented with multilevel interventions, such as health system capacity building and community awareness campaigns. Furthermore, approaches may have to be tailored to specific physician groups, as there is some evidence from this study that there are differences in course completion based on gender and other characteristics. User feedback received in this feasibility assessment will be used to further improve the functionality and content of the M‐OncoED app, and we plan to conduct a large‐scale study to assess multimodel recruitment approaches.

This study has a few limitations that need to be considered while interpreting the findings. First, change in knowledge is measured based on baseline level of awareness of risk factors and screening options. When baseline knowledge is high, we will not see large differences in knowledge gained, as in the M‐OncoED oral cancer module. Conversely, when knowledge is low, we are likely to see larger changes. Second, we only report change in knowledge before and after learning, and longer follow‐up is required to assess whether the knowledge gained is sustained over time. Third, we did not control for potential differences between those in the broad‐scale recruitment and the targeted recruitment groups, and this could have impacted the differences seen across the two groups. Fourth, for this initial study we focused on knowledge gain, but a key metric of importance is whether this knowledge gain can be translated into positive behavior change in clinical practice. Fifth, we only created an Android‐compatible app, as the majority of targeted users had an Android‐based phone. In the future, to allow equal participation by all eligible physicians, the app should be available for iPhone users as well.

Conclusion

The M‐OncoED application provides a template that can be used to guide the design and launch of further oncology‐based mHealth platforms such as colorectal cancer screening, clinical signs for urgent referral, patient navigation across health services, and facilitation of palliative care, that can be used synergistically alongside other initiatives to train physicians. This study has taken an important step in generating the evidence base for using mobile applications to improve overall oncology care, but large‐scale studies should be initiated to further assess impact on physician behavior and patient outcomes. If scaled, M‐OncoED and other applications of this type have the potential to decrease the cancer burden in low‐ and middle‐income countries by providing training for new physicians and offering continuing medical education to physicians already in clinical practice.

Author Contributions

Conception/design: Sujha Subramanian, Regi Jose, Anoop Lal, Shinu Krishnan Swayamvaran, Rengaswamy Sankaranarayanan

Provision of study material or patients: Regi Jose, Anoop Lal, Paul Augustine, Bipin K. Gopal, Shinu Krishnan Swayamvaran, Veena Saroji, Resmi Samadarsi, Rengaswamy Sankaranarayanan

Collection and/or assembly of data: Regi Jose, Anoop Lal, Paul Augustine, Bipin K. Gopal, Shinu Krishnan Swayamvaran, Veena Saroji, Resmi Samadarsi, Rengaswamy Sankaranarayanan

Data analysis and interpretation: Sujha Subramanian, Regi Jose, Anoop Lal, Paul Augustine, Madeleine Jones, Bipin K. Gopal, Shinu Krishnan Swayamvaran, Veena Saroji, Resmi Samadarsi, Rengaswamy Sankaranarayanan

Manuscript writing: Sujha Subramanian, Madeleine Jones

Final approval of manuscript: Sujha Subramanian, Regi Jose, Anoop Lal, Paul Augustine, Madeleine Jones, Bipin K. Gopal, Shinu Krishnan Swayamvaran, Veena Saroji, Resmi Samadarsi, Rengaswamy Sankaranarayanan

Disclosures

The authors indicated no financial relationships.