Codeveloping a multibehavioural mobile phone app to enhance social and emotional well-being and reduce health risks among Aboriginal and Torres Strait Islander women during preconception and pregnancy: a three-phased mixed-methods study.

OBJECTIVE: Describe the development and pretest of a prototype multibehavioural change app MAMA-EMPOWER.
DESIGN: Mixed-methods study reporting three phases: (1) contextual enquiry included stakeholder engagement and qualitative interviews with Aboriginal women, (2) value specification included user-workshop with an Aboriginal researcher, community members and experts, (3) codesign with Aboriginal researchers and community members, followed by a pretest of the app with Aboriginal women, and feedback from qualitative interviews and the user-Mobile Application Rating Scale (U-MARS) survey tool. SETTINGS: Aboriginal women and communities in urban and regional New South Wales, Australia. PARTICIPANTS: Phase 1: interviews, 8 Aboriginal women. Phase 2: workshop, 6 Aboriginal women. Phase 3: app trial, 16 Aboriginal women. U-MARS, 5 Aboriginal women.
RESULTS: Phase 1 interviews revealed three themes: current app use, desired app characteristics and implementation. Phase 2 workshop provided guidance for the user experience. Phase 3 app trial assessed all content areas. The highest ratings were for information (mean score of 3.80 out of 5, SD=0.77) and aesthetics (mean score of 3.87 with SD of 0.74), while functionality, engagement and subjective quality had lower scores. Qualitative interviews revealed the acceptability of the app, however, functionality was problematic.
CONCLUSIONS: Developing a mobile phone app, particularly in an Aboriginal community setting, requires extensive consultation, negotiation and design work. Using a strong theoretical foundation of behavioural change technique's coupled with the consultative approach has added rigour to this process. Using phone apps to implement behavioural interventions in Aboriginal community settings remains a new area for investigation. In the next iteration of the app, we aim to find better ways to personalise the content to women's needs, then ensure full functionality before conducting a larger trial. We predict the process of development will be of interest to other health researchers and practitioners. © Author(s) (or their employer(s)) 2021. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

The study has active user engagement at every step of the app development.

Aboriginal led project, with ongoing engagement, developing relationships and increased commitment within the communities.

A limitation was the small sample of women pretesting the app, and that most of the participants were lost to follow-up.

Introduction

Improving the health and well-being of Aboriginal and Torres Strait Islander (hereafter ‘Aboriginal’ will be used with recognition of the distinct sovereign nations) mothers and children are core foci of National calls to action to close the gap in life expectancy between Aboriginal and non-Aboriginal Australians. These areas vitally include improving pregnancy and birthing outcomes, as well as preventing chronic disease later in life.1 Behavioural risk factors during pregnancy including smoking,2 cannabis and alcohol,3 nutrition4 and physical activity (PA),5 afford necessary health education and support to improve the health and well-being of Aboriginal mothers and children. Addressing Aboriginal health and well-being must also address the impacts on social and emotional well-being (SEWB) that are a direct result of historical oppression, dispossession and ongoing racism since colonisation of Australia.6–8 About one-third (22%–34%) of Aboriginal women in the reproductive age group (14–49 years) experience compromised SEWB9 including mental, cultural, spiritual and social well-being.10 Chronic stress in mothers can also adversely affect the mental and physical health of offspring.11 In the general population the most well known risk factors for chronic disease and cancer such as tobacco and alcohol addiction, inadequate PA and poor nutrition also predict poor mental health.12–15 Addressing SEWB directly coupled with these behavioural risk factors among Aboriginal women would have a high potential to improve SEWB and physical health, and in turn improve the health and well-being of their children.

Tackling more than one risk factor at a time (multihealth behaviour interventions) is an emerging area of public health research.16 Multibehaviour interventions are effective, however, only four trials have been reported in pregnancy.17 With over one-third of Aboriginal patients willing to tackle more than one health behaviour at a time,16 tackling multiple health behaviours has promise. Mulitbehavoural interventions have potential to better align with Aboriginal people’s holistic definition of health and to address limited access to self-help resources to improve health behaviours in this setting. A comprehensive smartphone mobile application (mhealth) focused on providing healthy lifestyle support (nutrition, smoking cessation, alcohol and PA), mental health advice and resources for pregnant or preconceptual Aboriginal women is currently not available in Australia and has potential to reach and engage Aboriginal women who are reported to be avid users of digital platforms.18

While mhealth has been trialled in Aboriginal communities and appears to be a favourable method for intervention,19 this has predominately been within the context of mental health and suicide interventions.20 An mhealth intervention for Aboriginal women, tailored for prepregnancy, pregnancy and postbirth, has not been tested. This research builds on ongoing collaborative research between Aboriginal women, communities and researchers exploring culturally responsive approaches to support and empower Aboriginal mothers to be smoke-free.21–24 Aboriginal young people are adopters of new technologies and avid users of social media.18 Our prior research conducted with groups of Aboriginal women about resources for smoking cessation in pregnancy across New South Wales (NSW), Queensland and South Australia found a desire for information and videos about smoking cessation available on their smartphone.22 In our recent study, Aboriginal women in NSW reported having a smartphone (94%) they could download an app to, and had phone or data credit most of the time (~90%).25 When women were asked to nominate their top three topics for an app from a list, 82% included SEWB, 55% nutrition, 31% bush tucker, 22% to stop smoking, 10% pregnancy and 8% to stop alcohol or drugs.25

In 2019, we developed a minimum viable product (MVP) as a prototype of an innovative mobile phone app called MAMA-EMPOWER using a postcolonial approach26 which brought together tailored health messages, a self-paced health and well-being programme, and a shared platform for peer support, based on the behaviour change wheel (BCW).27 It addressed SEWB of Aboriginal women during preconception or pregnancy, and provided tailored support to women to tackle three out of the seven top modifiable risk factors impacting the health gap between Aboriginal and non-Aboriginal Australians including: smoking (17%), alcohol (7%), and low fruit and vegetable intake (5%).28

Developing mhealth interventions requires targeting end-users, cocreation, continuous evaluation, consideration for conditions of implementation, persuasive design techniques and methods to measure impact.29 The aims of this paper are to describe the process undertaken to codesign the MAMA-EMPOWER mobile phone app for Aboriginal women, to support SEWB before, during pregnancy. In this paper, we discuss the development and findings of the preliminary user trial in line with the Centre for eHealth Research and Disease Management (CeHRes) Roadmap for designing eHealth technology three phases: of the contextual inquiry, values specification and design.29 By articulating the ways in which this project was designed we hope to engage in more dialogue regarding practice models of care utilising modern technology, particularly in the area of Aboriginal health which has limited mhealth intervention research.

Methods

The formative research and pretest study used a mixed-methods study design. Each phase of the app design incorporated qualitative and quantitative explorations and evaluations.

Phase 1 (contextual inquiry) consisted of convening stakeholders and conducting qualitative interviews to explore how Aboriginal women engage with apps in general.

Phase 2 (values specification) consisted of an active user workshop in partnership with an app development company, researchers and Aboriginal women.

Phase 3 (design) consisted of the content design and pretesting of the app with Aboriginal women over a period of a month. During this phase, we gathered data from participants using a baseline demographic questionnaire and after the study, a user Mobile Application Rating Scale (u-MARS) survey,30 user metrics and qualitative interviews. The methods and results for each phase will be described below, followed by a combined discussion section (see figure 1).29

Figure 1

Study phase.

Phase 1: contextual inquiry

Undertaking a contextual inquiry incorporated three parts: convening stakeholders, attending a Commonwealth Scientific and Industrial Research Organisation ON Prime preaccelerator programme and undertaking qualitative interviews.

Stakeholders: Convening community stakeholders and the research team. Aboriginal and non-Aboriginal academics with expertise in: mental health, smoking cessation, nutrition and drug and alcohol worked collaboratively with Aboriginal women in the Hunter Region of NSW. Consultations were held with colleagues and a range of organisations who had prior experience of developing apps for Indigenous peoples. Specific expertise was obtained from KB and FS, (mental health experts and developers of iBobbly App for suicide prevention in Aboriginal youth);31 KD (community dietician with expertise in Aboriginal nutrition and use of native foods); KL, expert in Aboriginal alcohol use. Within the core team authors GSG, MK and RK provided considerable expertise in Aboriginal tobacco use and smoking cessation. Stakeholders explored options for the most suitable on-line platform/s to engage Aboriginal pregnant women in health behavioural changes. This involved consulting with digital specialists, Aboriginal digital specialists, researchers who have engaged with the Aboriginal community previously to develop online, social media and app platforms, and Aboriginal women and other community members. Potential for use of closed Facebook sites were explored. The research team used the BCW to analyse factors related to Aboriginal smoking in pregnancy, to develop strategies for community-based resources.

ON Prime preaccelerator programme: A programme run by Australia’s national science research agency to empower researchers to take forward their ideas into innovation. The 6-month programme was attended by GSG and MG, as well as an Aboriginal research assistant and a PhD student, during which the team was guided by an industry mentor, developed a business case for the app, considered the market segmentation and conducted over a hundred informal market interviews to establish community need and ascertain whether working hypotheses for the build would have interest. The working hypotheses were: ‘An App to improve SEWB will give Indigenous women a better experience of pregnancy and a healthier baby’ and ‘An App to improve access to health services will increase enagagement with health providers’. All team members conducted interviews which involved cold-calling contacts from their own networks. Contacts included Aboriginal community members and end-users, non-Indigenous end-users, health professionals, policy and service informants. Interviews were recorded on an excel spreadsheet, and up to three key learnings from each interview were noted. During this process, the objective of developing a multibehavioural app that could holistically support pregnant Aboriginal women’s SEWB was confirmed.

Public involvement

As outlined in this manuscript, end users and consumers were involved in each stage of the research including workshops and design. The model followed a user-centred and human-centred design that engaged the those who the app is intended for, placing planned users at the centre of the design process.32

Recruitment

Qualitative interviews

We recruited a convenience sample of interested urban Aboriginal women through word of mouth, researchers’ personal networks and posters displayed in a local Aboriginal Community Controlled Health Service in Newcastle. Informed consent was provided by the participants prior to the commencement of data collection. Individual interviews with Aboriginal mothers were conducted by MK (a female Aboriginal (Wiradjuri) researcher with a PhD in Aboriginal health) in the women’s homes and/or public spaces of their choosing. The researcher was known to participants, or their extended kinship. Using a semistructured topic guide, the interviews explored the participant’s current use of apps, their expectations from an app and the desired content for a targeted app about pregnancy and well-being.

Analysis

Interviews were audiorecorded, transcribed and analysed using NVivo V.11. Thematic analysis was conducted by Aboriginal researcher MK with 30% of the interviews independently coded by GLHF, and themes agreed by consensus.

Results

Interviews were conducted with 7 Aboriginal and 1 Torres Strait Islander mothers with children 5 years and under, in the urban Newcastle area of NSW during January and February 2019.

Three major themes emerged from the interviews: current app use, desired app characteristics and implementation as outlined in table 1.

Table 1

Themes and indicative quotations

Theme	Results	Indicative quotations
Current app usage	Social media apps such as Facebook, Instagram, Snapchat were the most commonly used by the participants. These were used for inspiration, interpersonal connection and seeking in-the-moment health information for their children. Other commonly used apps were those which are required for routine life activities in Australia such as myGov, online banking, Centrelink, maps and clock. Online shopping and the television app for Australian Broadcasting Company were used by some women as were baby information apps such as The Wonder Weeks. Motivation to continue to engage in an app included those that were visually engaging.	“I go on my Instagram and things like that and I look for inspiring ideas of food, different things like that” (W1)“Some of the pages that I follow are mums getting back into the fitness world. When I see them doing it, it makes me feel like I can”. (W3)“Communication with family, friends a radar of what’s going on” (W2).“It tells me what’s going on. If they’re unsettled, it can tell me that’s why, because they’re going through a leap.” (W5)“Mainly ones with great visuals that grab my eye I tend to click on and then I’m just hooked because I just love my iPhone. I just live on it”. (W1)
Desired app characteristics	Women spoke of a desire for an app made specifically for Aboriginal women that incorporated culture, history and art. Incorporation of history was desired to inspire younger women through a connection to culture and tradition. Use of photos and imagery of Aboriginal women was also seen as an element to enhance trust in the app and suggested as a way to inspire women. Videos were recommended by all women over written forms of delivery to support ease of information sharing and build confidence.Women spoke of their use of social media when discussing apps currently used and desired for app characteristics similar to social media apps, where connectivity is one of the main purposes.The inclusion of a social platform or forum type element was requested to strengthen community connection, while connectivity to community services was an alternative to the forum platform.The ability to share content between the app and social media was highlighted as being an avenue for continued engagement with the app over time.	“So it’s, like, breastfeeding is the longest-lasting tradition that still applies in today’s society” (W1).The cultural things are added into the app. That’s what will make me come back to it only because I know myself, I can relate to it” (W3).“Then you can put up leaders and people that might inspire us from our own community” (W1).“For Aboriginal people they want comfort and a bit of security, so they know that they can trust other Aboriginal mums and Torres Strait Islander mums. Yes, you’d feel more comfortable” (W2).“In the videos, for example. I guess it makes us feel more comfortable when we see other people like ourselves. That’s what draws us to things that we choose to watch and listen to or read about” (W3).“Everyone would have their own profile and that’s where you could put your updates, your baby updates or your health or whatever” (W1).“If there was some sort of community connection throughout. Like, when you first start, you introduce yourself.” (W2)“Getting support within the app maybe or knowing where you can get support” (W3).
Implementation	Aboriginal women reported a need for Aboriginal women and services to promote the use of the app to provide interest and trust in the app. Women also explained that the app could not take up too much storage space on their phones or require too many updates.	“If I see it advertised on Indigenous pages and stuff, yes, I would. See other woman using it, I would, yes, give it a go”. (W3)“Maybe video filming mums confirming that it is safe” (W2).“That’s the only reason I ever delete, is when you update, and you’ve got to use more space” (W4).

Phase 2: value specification

Active-user workshop

Purposive sampling was used to invite participants who had been recruited for the qualitative interviews, as well as their family or friends to further participate in an active user group workshop. Informed consent was provided by the participants prior to the commencement of data collection. The workshop was held at a research institute in Newcastle on the 7 May 2019 and ran for 2 hours. Members of the research team attending the workshop included chief investigator GSG, Aboriginal researcher MK, community dietician KD, postdoctoral researcher RK, led by the app developer. The app developer followed a research-based user experience model to undertake the design session. The workshop consisted of four key exercises (see online supplemental appendix 1). The aim of this process was to provide guidance for the user experience.

Notes were taken during the active-user workshop by the app developer and workshop audio recordings were used for reflective practice. All notes and recordings were analysed collectively from all four exercises by app developer with the research team and reported using narrative synthesis.

Several key areas for user experience of an app were raised by Aboriginal and Torres Strait Islander women including; app qualities, content and caution with wording.

Women advised the app needs to include similar qualities to apps that the women already use; Apps most frequently used had characteristics that insisted participants return to the app on a daily/intradaily basis. Community and social connection were the number one instigator for increased engagement in a potential app with other features to prompt women to return to the app daily such as daily questionnaire and notifications also being of relevance. Women wanted the app to incorporate social and community interaction features, however, discussions on practical implications of managing a community forum were raised, it was agreed that this could only be incorporated if moderated by the project team to ensure that the advice provided through forums is relevant and accurate.

Women suggested the user experience is guided based on demographic and health related questions so information provided to women is relevant, for example, non-smokers should not be shown smoking cessation related information. App content should be brief and to the point with video content strongly advised. Wording was highlighted as requiring attention to be concise and appropriate. Participants discussed how different Aboriginal communities use different words and this would require careful consideration if an app was designed.

A native app that can be run both on the android and iOS operating system should be made as women possessed both types of smartphones. Women made recommendations on the visual look and feel of the app based on examples provided.

Phase 3: design

A young Aboriginal mother (RL) (Wiradjuri), who was one of the attendees at the previous active user workshop, and an Aboriginal research assistant with a background in neonatal nursing (JB) (Gomeroi), were employed to codevelop the app content with the researchers. In an iterative process, Aboriginal and non-Aboriginal academics from expertise areas (mental health, smoking cessation, nutrition and drug and alcohol) worked collaboratively with Aboriginal women in the Hunter Region of NSW to develop the MAMA-EMPOWER mobile phone app. Some of the included features were also resourced from a previously codeveloped patient booklet from the research team’s ICAN QUIT in Pregnancy project.22 33–35 These features included five colouring-in templates by Aboriginal artist Saretta Fielding, videos and text about smoking cessation techniques (such as use of nicotine replacement therapy) and quit smoking education and resources. Key features of the app were characterised by the research team against known behavioural change techniques (BCT).27 While the BCTs were not identified in phase 1 and 2, this retrofitting activity was intended to safeguard the inclusion of important BCTs considered to be effective for smoking in pregnancy and ones that have been used before in a smoking cessation apps for pregnant women.36 37 The BCT framework is commonly used for the purpose of determining gaps and being able to analyse interventions post hoc to advance the understanding of which features may need improvement.38 The features of the app and their related BCTs are detailed in table 2, using the taxonomy of BCTs of Michie and West.27

Table 2

Outline of key features of APP content

Area nb	MAMA-EMPOWER app feature	Behaviour change techniques applied
1	Self-assessment and feedback tools for SEWB and psychological distress	Feedback on behaviour, self-monitoring of behaviour, self-monitoring of outcomes of behaviour
2	Tailored health messages, education and feedback to improve health depending on user interest	Information about health consequences, feedback on behaviour, self-monitoring of behaviour, information about social and environmental consequences, Pros and cons
3	Mindfulness and strength-based positive psychology elements for example, gratitude, art-based activities (colouring-in of Aboriginal art templates) and relaxation exercises	Reduce negative emotions, Identity associated with changed behaviour, comparative imagining of future outcomes, Mental rehearsal of successful performance, Identification of self as role model
4	Smoking cessation messages, practical advice about quitting, goal setting and making a quit plan, cost calculator, education on secondhand smoke and making a home smoke-free.Drug and alcohol messages, education and feedback	Instruction on how to perform the behaviour, Goal setting, Information about health consequences, action planning, problem solving, feedback on behaviour, self-monitoring of behaviour, information about social and environmental consequences, Pros and cons, behaviour substitution, verbal persuasion about capability
5	Dietary feedback to increase intake of fruit, vegetables, micronutrients and native and bush foods, easy to prepare recipes	Instruction on how to perform the behaviour, goal setting, problem solving, action planning, demonstration of behaviour, information about health consequences, adding objects to the environment, pros and cons, behaviour substitution, habit formation, comparative imagining of future outcomes
6	Physical activity messages and education to encourage a safe level of exercise	Instruction on how to perform the behaviour, demonstration of the behaviour, social support, action planning, information about health consequences, self-monitoring of Behaviour, prompts/cues, pros and cons, verbal persuasion about capability, Information about emotional consequences, goal setting (outcome).
7	Pregnancy features: information on stage of pregnancy and reminders antenatal checks	Instruction on how to perform the behaviour, information about health consequences, credible source, prompts/cues.
8	Education on spiritual and health importance of the placenta	Information about health consequences, information about social and environmental consequences.
9	Information on helplines	Instruction on how to perform the behaviour, credible source
10	Community forum: participant chat room to support and empower each other during pregnancy, preconception and the health of their children. (Forum was moderated and anonymous).	Social support, feedback on behaviour, social comparison, social reward, information about other’s approval, restructuring the social environment.

SEWB, social and emotional well-being.

RL and JB led the review and feedback process to develop culturally appropriate content. An artwork was purchased from Saretta Fielding to complete the overall look for the app (see figure 2 for overall look).

Figure 2

Images demonstrating app features.

Key features of the app as informed by phase 1 and 2

Video content

Video content was developed to sit within content areas and videos topics included; increasing fruit and vegetable intake, not drinking alcohol during pregnancy and keeping healthy through pregnancy, videos were 15–30 s long (see online supplemental appendix).

Quit calculator

Participants accessing smoking content were presented with a smoking calculator that would calculate the cost of smoking depending on the number of cigarettes a participant smoked daily.

Community forum

A community forum was created for participants to discuss topics relevant to their health, and the health of their child(ren). The community forum was accessible from an icon visible on every page of the app and would lead to a moderator driven forum. During baseline, each participant was asked to provide an anonymous name to use in chat. Each participant had app permissions to create, edit, delete their own conversation thread. Each participant also had permission to reply to other participants. Moderators controls were created for conversation threads, including to edit or remove participant threads.

Pre-test study

The MVP of the app was pretested for 1 month among Aboriginal women of childbearing age. Two surveys were administered, a demographic questionnaire before the test of the app and the user-MARS survey after the 1 month test of the app. Brief qualitative interviews were held with participants at the end of the study to evaluate the design and use of the application.

Recruitment

A convenience sample of 16 Aboriginal women were recruited from Newcastle and Tamworth, NSW to pretest the app. The study was advertised through partnerships with the Aboriginal Community Controlled Health Services, the local Aboriginal Land Council and community networks. Inclusion criteria were women of reproductive age (16–49 years old), who could give informed consent and were an owner of a smartphone. Meetings were arranged with individual or multiples of women by JB or RL to introduce women to the app and help them download it on their smartphone. Participants completed an eight-item demographic survey and were shown the features of the app.

After testing the app, women were invited to complete the u-MARS survey39 and short qualitative interview. The u-MARS survey comprises 26 questions and 5 point Likert scales (where 1 represents poor response/low quality and 5 high quality) about engagement, functionality, aesthetics, information, subjective quality, perceived impact on the user’s knowledge, attitudes, intentions, awareness, seeking further help and behaviour change (where 1 represents strongly disagree and 5 strongly agree).30

Analysis

u-MARS Survey: were analysed using Stata/IC V.15.1. Descriptive statistics were calculated reporting number of cases and percentages for the categorical variables. In the case of continuous variables, the range was estimated, and for the u-MARS scores means and SDs were calculated for the whole sample.

Qualitative interviews were uploaded to NVivo V.12. A framework analysis was conducted using predetermined categories used in the u-MARS by GLHF (a medical doctor and statistician) and checked by NR (a postdoctoral researcher). Emergent themes were classified where relevant. No additional data outside of publication is available.

Results

Sixteen women trialled the app on their smart phone for 1 month. Demographic characteristics can be found in the table 3.

Table 3

Demographic characteristics of women that tested the app

Characteristics		N/mean	Percentage/range
Aboriginal	Aboriginal	15	94
	Torres Strait Islander	1	6
Education	Year 10–11	5	31
	Year 12	3	19
	Undergraduate	1	6
	Student	6	38
	Postgraduate	1	6
Current smoker	Yes	3	19
	No	13	81
Alcohol consumption	Yes	9	56
	No	7	44
Have a partner	Yes	12	75
	No	4	25
Smoking partner	Yes	6	50
	No	6	50
Children	None	7	44
	2	7	44
	3 or more	2	12
Pregnant	Yes	6	38
	No	10	62
Age		26	22–40

Four missing values.

u-MARS results

Five out of 16 participants answered the u-MARS survey. Lost to follow-up was due to women having other commitments: one woman gave birth during follow-up time and several women travelled for work or family commitments. Women were contacted once by the research team for follow-up and respected women’s rights to opt into final interview. On average the highest quality ratings on the u-MARS were for information (mean score of 3.80 out of 5, SD=0.77) and aesthetics (mean score of 3.87 with SD of 0.74), while functionality, engagement and subjective quality had lower scores. Table 4 shows in detail the u-MARS subcategory scores for the sample.

Table 4

Descriptive statistics for uMARS from five participants poststudy

uMARS	Mean	SD	Summary	Interview quotes
Engagement	3.2	1.08	Women expressed that while the app content was very relatable, there were some barriers to their engagement. Barriers included technical problems which were frustrating, and the fact that they did not fit into the main target group such as, not being a smoker or not pregnant at the time.The navigation problems may also have limited how much content and what features women were able to view, and leave women with the sense that there was not much to explore.This category overlaps with functionality below, but the emphasis here is how that influenced women’s ability to engage.	Yes, it’s very relatable. It relates to not only Indigenous women but all women, I believe, especially with Indigenous women coming up with different backgrounds and different language barriers. Everyone can relate to it a lot and it’s not too scientific it just speaks the facts. It’s like just talking to another person.I felt like once I’d explored all of the options, it sort of didn’t go anywhere.I guess maybe it might have been different if I was a smoker and I could have logged that kind of stuff it was probably just technical and probably that I couldn’t interact, I guess, because I’m not pregnant at the moment, and not smoking.I was deterred a little bit—like, I'd have a second to get into it and have a look, and then I'd lose that second and it’d be gone, and I wouldn’t interact with it as much as I probably could have.
Functionality	3	0.73	The main functionality issues were length of time the app content or pages took to load, required a considerable amount of patience, complex navigatability between pages and not being able to easily go back to areas that were previously seen.The layout and menu structure was seen to need improvement, and simple things like having a back button on pages could have facilitated functionality.	It looks great, but to actually find stuff, I felt was a bit hard.It would take too long to load so I’d just get out of it. Which is probably not the best way to sort of work something out, but, yes, if it was taking too long, I’d just get out of it.Because I tried to keep going to here, to get, like, your dropdowns. Like, you’ve got your alcohol, but then if it has another section, so you could go into the specific ones, rather than having to go right through everything to find it.So sort of get to here and then I’m like, “No, that’s not where I want,” so I had to go right back to the start again, rather than be in that little sublevel you used to go through.I think I would have used the recipes if I could have got into them a bit quicker, like been a bit more accessible, because that’s something I’m always looking for at home is stuff to cook, so if I could have done it a bit quicker.I think for me it was more the fact that it was just taking too long, so I got out of it, rather than anything else.Probably just the layout, like, the way that – with the menu structure. Just to have those little submenus to be able to pick it straightaway rather than having to go through everything to get to the end point that you want to be at.RecommendationsFind it very appropriate and useful but needs fine tuning tech wise.App needs back button on most pages.
Aesthetics	3.87	0.74	The visual appeal of the colouring in activity was appreciated and suggestions for more indigenous artists or choice may add to this appeal. The women liked the colourful aspects overall.	The colouring was cool, I reckon if you had spare time you could do that, which was interesting.The nutrition section was really good—colourful. Like, it caught your eye.The design is really great, but there’s just a lot of room for improvement.I feel like a lot of people need that visual. I need that visual. And I think from certain demographics that you’d be going for, they’d need that visual too.RecommendationsThe colouring in, it needs black, more choices, different artists, and a different range of artists,Just more options of different pictures.
Information	3.8	0.77	While women thought the information was very useful, there was clear scope to improve or extend the information content. The overall tone and the way the information was conveyed had a familiar ring to it. Some thought the information was too wordy or suggested more video content, including some real stories. There were some suggestions to cover more on mental health, drug use, fitness and allow a photo feature as a visual way to take notes. Having the artist acknowledged on every colouring in page was an important suggestion.	The information being very useful and talking about really good things that need to be said and spoken about, especially during pregnancy.The wording made me feel like I was talking to my sister or aunty, I loved how simple it was to understand.Smoking content was good it’s good to have the information there, but it’s just, yes, too much to read.Some of the sections, too, had videos of people talking, hey?RecommendationsEven with the fitness as well, being pregnant or not pregnant, what kind of exercises can you be doing to get your body strong for pregnancy, or even just get your body healthy and strong in general.Maybe even add some mental health information in there too. There’s single mums who may be going through financial issues, that have no support from family or dad of child, maybe going through any domestic violence or anything like that. Help them with ways to cope with their mental health, especially during pregnancy so the baby can come into this world with positive vibes and a positive atmosphere and they can be able to support their baby the best that they can without having the negative impacts behind the scenes, and knowing how to cope with the negative impacts.Information about drug use in pregnancy would be useful.I think more videos would be really good, though, and, like, people speaking from experience. Because I think it’s all well and good to provide the information, but if you can relate it to someone, it makes it more real, and it might deter them from doing the things that they shouldn’t be doing for the sake of their family and for themselves.I know through my pregnancy, I would always take photos every week to write down little things, so if it’s already in somewhere, so you’re not having to physically do that yourself, that would probably be great.Acknowledging the artist too, not just have their work but at the bottom have who the artist is.Acknowledge the artist on every picture.
App overall quality	3.47	0.83
Subjective quality	3.2	0.95
Perceived impact	3.55	1.06

u-MARS, user Mobile Application Rating Scale.

Post-test qualitative interviews

Five interviews were performed, key emerging areas of quality were identified relating to the u-MARS categories. Participants reported positive feedback about information and aesthetics, in terms of visual appeal, quality, comprehensive and reliable information. The qualitative data confirmed and further explained the ratings of the u-MARS. In terms of functionality, all the participants reported problems with the app with performance and navigation being the most frequently reported problems. Table 4 summarises the u-MARS scores and the qualitative quotes.

Discussion

Principal results

This paper outlined the three-phase study to develop the MAMA-EMPOWER app, a multi-behavioural app to support Aboriginal women’s SEWB. Using the CeHRes Roadmap for designing eHealth technology, this paper outlined the; phase 1 (contextual inquiry) phase 2 (values specification) and phase 3 design which included pretesting the app by 16 Aboriginal women in urban and regional NSW, Australia. Taken as a whole, the paper outlines a methodology which has been useful to describe the process engaged with for the MAMA-EMPOWER development. The phased approach enabled the team to determine the benefits and challenges for using this technology to elicit healthy behaviour change.

This is the first app that has been designed to support the SEWB and health behaviours of Aboriginal women in the reproductive age group, as far as we are aware. It addresses SEWB of Aboriginal women during preconception or pregnancy, and provides tailored support to women to tackle three out of the seven top modifiable risk factors impacting the Indigenous health.28 This research builds on the reported need to increase evidence based interventions with an improved understanding of end user engagement, which is essential to develop successful applications.40 Early extensive consultations with Aboriginal women and stakeholders through the ON Prime programme revealed that Aboriginal women and providers would be interested in an app that targeted health behaviours in the context of a holistic SEWB approach, but not specific to smoking cessation only.

Early qualitative interviews identified the need to have the developed app connect and engage with apps already used by Aboriginal women such as Instagram and to promote social connections which was also been identified in a recent smoking cessation app trial with Aboriginal smokers.41 This highlights the importance of sociability consideration as well as the usability of the app as suggested in previous research.42 There was also a desire to celebrate Aboriginal culture throughout both the visual elements of the app as well as the written educational content. User-workshops assisted the research team to refine visual design, considerations for wording and the importance of photo and video imagery for content using Aboriginal women.

The pretest reported the acceptability of the colouring and visual design of the app, and that the written information was useful and credible. Aboriginal women reported that the information provided was clear and easy to understand with a familiar and empathetic conversational style. This design pretest did not target Aboriginal mothers with specific characteristics to test the behavioural change functions of the app, including smoking and pregnancy status. This influenced the use of particular content areas of the app such as quit smoking plans, as women who were not smoking did not engage with them. Participants reported problems of functionality of the app, therefore, it would be possible that the low score on engagement has been affected by the functionality deficits, which would need to be addressed before further testing.

Comparison with prior work

#Thisismymob describe the diversity of beliefs and sociocultural practices among Aboriginal communities that present challenges and considerations when developing technology that is appropriate and meaningful for Aboriginal peoples across Australia.43 Similarly, Soro describes a cross-cultural design working with remote Aboriginal peoples which undertook a participatory design approach to extend the development of cultural and technology probes.44 The work of Soro challenges what counts as knowledge and how knowledge is made and shared.44 This process of codesign and reporting considerations is crucial to the exploration of meaningful technologies for Aboriginal peoples given the impact of colonisation and diversity of Aboriginal groups across Australia.

There are very few maternal health apps developed with health providers that use evidence-based approaches, and none previously for this Aboriginal target population.45 The features of our MAMA-EMPOWER app are rooted in the well-known BCTs described in table 1, which have established mechanisms of action.27 46 Smoking cessation apps Kick it47 and Smoke-free baby have also been informed by BCTs, but there is no comparable data on u-MARS scores.37

A systematic review of iPhone apps providing nutritional information to pregnant clients found that the 51 included apps used only 11 BCTs out of a possible 40.48 Overall the apps in this review were of low quality. For the 51 iPhone apps assessed, the mean overall u-MARS quality rating score was 3.05 out of 5 with the functionality subscale scoring highest (mean=3.32), and aesthetics subscale scoring the lowest (mean=2.87) highlighting a need for well-designed apps focused on pregnancy health behaviours.48 Interestingly, our MAMA-EMPOWER MVP had an overall u-MARS quality rating score of 3.47 out of a possible five with the aesthetics subscale scoring the highest (mean=3.87), and the functionality subscale scoring lowest at a mean of 3.0. Similar to this study, qualitative research has been used in the past to gauge the interest, usability and acceptability of smartphone interventions among preconception and pregnant women.49–52 The qualitative research from our study was valuable to guide the next iteration of app development.

While systematic reviews have reported no firm conclusions on the effectiveness of mobile phone apps during pregnancy for behavioural change53 recent interventions are showing positive results. The Smoke-free baby app explored the usability, feasibility and acceptability of app design and reported that an app during pregnancy is acceptable and that content that is motivational, educational and personalised was most valuable.54 However, a randomised factorial experiment of the Smoke-free baby app found engagement to be low and no effect to reduce smoking rates.55 A recent mood tracking and alert app was trialled with pregnant women in the USA and found that women who received a call from a health provider following the app alert were significantly more likely to receive a mental health specialist referral when compared with controls.56 App group participants also rated their ability to manage their health better than the control group.56 Similarly, the Pregnancy Exercise And nutrition Research Study carried out in Ireland randomised 565 pregnant women to a nutrition and PA intervention with additional app support and usual care (without app and dietary advice) found participants in the group with addition of a smartphone application demonstrated greater improvement in diet and PA compared with the usual care control group.57

The use of mobile apps is a growing field in Aboriginal populations. However, there is limited reporting on the characteristics of mHealth interventions evaluated with Aboriginal people. In a recent systematic review only 13 studies (9 from Australia) were identified.19 The most common health challenge addressed was mental health and suicide (5 out of 13), with two of these being mobile apps (AIMhi Stay Strong and iBobbly) and tested for acceptability in Aboriginal Australians.20 Both these apps were developed by Australian research teams in collaboration with Aboriginal people and communities. Cultural expertise were used to ensure that appropriate language, imagery, and literacy from their health interventions were used. Visual representations, voiceovers, action-based content and goal setting to increase mental health and reduce risk of suicide were all employed.58–60

While greater evidence is needed to measure the effectiveness of mobile apps for smoking cessation,61 currently trials are exploring acceptability among Aboriginal communities. A recent mobile app Can’t even quit, designed to assist among Aboriginal people to quit smoking, reported a need to integrate apps with commonly used functions of mobile phones and draw on social networks to support their use.41

We believe it is still early in the progress of research for targeted apps for the Aboriginal and Torres Strait Islander population and pregnant populations, considering the volume of research on apps for the general population. Although clinical outcomes may have not been demonstrated to date, there could be other intermediate benefits such as feeling one’s needs are understood and community connectivity. These aspects warrant further research.

Strengths and limitations

The main strength of this research is active user engagement at every step of the app development. Aboriginal leadership, ongoing engagement, building trust, developed relationships and increased commitment within the communities, this project builds on other research with Aboriginal communities such as the; #ThisMyMob, iBobbly, Can’t even quit and AIMhi Stay Strong.43 Opinions of the target audience of Aboriginal women were taken into consideration via qualitative interviews, workshops and by employing Aboriginal women as research assistants to design the app. Aboriginal research team members were at the forefront of leading this research and helped guide the major decisions regarding the design and testing of the app and developed much of the text-based content using a friendly engaging voice. The Aboriginal researchers were also invaluable in recruiting the participants from their extended networks, which made sure that feedback from the target audience was received to inform app features in phases 1 and 2.

A limitation from the ON prime interviews were that as this was an informal process, the results cannot be detailed. A further limitation was the small sample of women pretesting the app, and that most of the participants were lost to follow-up. However, Aboriginal smokers have been identified in other research as having an elevated likelihood for lost to follow-up.62 It is therefore uncertain whether the loss to follow-up could signal other design features that may need to change in the next iteration. The women who did respond to the invite to comment on the app pre-test may have a higher level of social desirability bias and given more favourable accounts of their experience than those who did not respond. The fact that women were recruited from extended networks may have added to this bias. The app development and pretest were conducted with urban and regional Aboriginal women across the Hunter New England region in NSW and their experiences should not be generalised to all Aboriginal women. Thus, the app would need to be tested further afield to understand how diverse Aboriginal women may respond to it across other states and territories.

A serious limitation of the app was its lack of speed and functionality including the community forum feature which did not function as planned and as such engagement could not be measured. Within the next iteration of the app community consultation and consideration will be made regarding online safety, upholding privacy, anonymity, potential stigma and security to decide whether this feature is feasible.

The speed and functionality might have made women frustrated with the app features leading to low usage over the trial period. The limitation of speed was noted by the researchers prior to the app going live, and the app developers tried to remediate the issue. Women in regional Australia also may not have access to fast internet speeds or effective cellular networks (3G is common in rural areas), web-based apps have been known to have this limitation of speed.63 The content that the team developed may be more suited to a native app and have more functionality for the purpose. A native app will be considered for the next iteration of the MAMA-EMPOWER app. Other limitations on the app development and research were budgetary constraints due to the level of pilot funding.

Conclusions

Developing a mobile phone app, particularly in an Aboriginal community setting, requires extensive consultation, negotiation and design work. Using a strong theoretical foundation of BCTs coupled with the consultative approach has added rigour to this process. The use of mobile phone apps to implement behavioural interventions in Aboriginal community settings is still a new area for investigation and has immense potential, not yet being fully realised. A key learning is that a multibehavioural and holistic approach is warranted for this target population of Aboriginal women of reproductive age and is still the best option for addressing multiple challenged that women may face. In the next iteration of the app, we aim to find better ways to personalise the content to women’s needs, address further design considerations with end-users, then ensure full functionality before conducting a larger trial. We predict the process of this project development will be of interest to other health researchers and practitioners.