Novel electronic adherence monitoring devices in children with asthma: a mixed-methods study.

INTRODUCTION: Adherence monitoring to inhaled corticosteroids is an essential component of asthma management. Electronic monitoring devices (EMD) provide objective data on date, time and number of actuations. However, most give no information on inhalation. Novel EMD (NEMD) platforms have the potential to monitor both activation and inhalation. AIM: To assess the feasibility of NEMDs, in terms of usability, acceptability to patients and healthcare professionals and accuracy.
METHODS: This was an open-label, prospective, mixed-methods, pragmatic randomised study. Children with asthma attending specialist tertiary care were randomised to one of four NEMD: Remote Directly Observed Therapy (R-DOT), Hailie Smartinhaler, INhaler Compliance Assessment device (INCA) and the Rafi-tone App. Following monitoring, participants were invited to focus groups or one-to-one interviews. Usability and acceptability were evaluated using themes identified from the focus groups and interviews. Adherence accuracy was determined using adherence data from each NEMD.
RESULTS: Thirty-five children were recruited; 18 (51%), (11 males, median age 13.5 (7-16) years) completed monitoring, 14 (78%) provided feedback. Participants identified various features such as ease of use and minimal effort as desirable criteria for an NEMD. The Hailie and INCA fulfilled these criteria and were able to record both actuation and inhalation. Negative themes included a 'Big Brother' effect and costs.
CONCLUSION: There was no 'one size fits all', as participants identified advantages and disadvantages for each NEMD. Devices that can easily calculate adherence to activation and inhalation have the potential to have greatest utility in clinical practice. Each NEMD has different functionality and therefore choice of platform should be determined by the needs of the patient and healthcare professional. © Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Which novel electronic adherence monitoring devices (NEMDs), that can measure both inhalation and activation is the most feasible to use as a tool for monitoring adherence in children aged 6-16 years with asthma in terms of accuracy, usability and acceptability?

NEMDs that have the ability to monitor actuation, inhalation and technique provide more accurate estimates of adherence to inhaled corticosteroids in children with asthma. Participants identified ease of use and lack of disruption to daily routine as desirable attributes of an NEMD.

EMDs are increasingly used to measure adherence to asthma medications; however, there are limitations. NEMDs address some of these limitations by assessing both inhalation and actuation. The parents and young people who participated in this study offer unique insights into the utility of NEMDs in clinical practice.

The four NEMDs tested were found to have different functionalities and therefore choice of platform should be determined by the needs of the patient and healthcare professional. The ability to monitor adherence remotely in real time is an added bonus in the current digital era. These devices could be utilised in virtual clinics to monitor adherence remotely.

Introduction

It is estimated that 5.4 million people in the UK are diagnosed with asthma, of which 1.1 million are children.1 The UK has the highest rate of asthma deaths in Europe2 and young people in the UK are at higher risk of dying from asthma or having poorer quality of life than their counterparts in other high-income countries.3

Asthma accounts for a high economic burden with the National Health Service spending approximately £1 billion a year on asthma.3 The majority (90%) of these costs are attributed to asthma medications.4 Inappropriate use of medicines and suboptimal adherence results in higher healthcare costs and poorer health outcomes, including an increased risk of asthma attacks and hospital admissions.5

A key aspect of improving adherence is being able to measure medication adherence accurately.3 4 6–9 However, this is challenging as clinicians’ ability to estimate adherence is poor10 11 and parents and children often overestimate adherence in self-reports12 13 to please clinicians. Objective adherence measures such as dispensing data or electronic adherence monitoring using electronic monitoring devices (EMDs), are more accurate. EMDs attach onto inhalers and capture data on inhaler activation, timing of the activation and the number of doses activated. Several EMDs are available for asthma and are generally considered the ‘gold standard’ for adherence monitoring,8 9 14 15

EMDs have two key roles: (1) enabling healthcare professionals to interpret asthma control in the context of adherence and inform better decision making; and (2) as an intervention and tool for the patient to aid self-management and improve adherence. However, current EMDs only measure activation (eg, the depression of the inhaler canister) and provide no information on whether the inhaler was used correctly or if inhalation occurred.10 16 17

Several novel EMD (NEMD) platforms are currently in development which enable clinicians and patients to monitor both activation and inhalation, thus ensuring more accurate estimates of adherence and to better identify and address practical and perceptual barriers to adherence. However, to date only three studies on EMDs have assessed platforms which monitor activation and inhalation in asthma: two were on the inhaler compliance assessment device (INCA)18 19 studying accuracy, adherence rates and inhaler technique and one on mobile directly observed therapy (m-DOT),20 investigating connectivity and usability as a primary outcome; only the latter study included children. The feasibility of these platforms in terms of usability, acceptability,21 and accuracy of the EMDs to children, their families and healthcare professionals has not been explored. Understanding these three parameters, is key to ensure patient and healthcare professional engagement with these devices and ensure accurate decision making.

The aim of this study was to evaluate four NEMDs which have the potential to monitor both inhaler device actuation and inhalation in terms of usability, acceptability and monitoring accuracy.

Methods

This was a prospective, single-centre, open-label, pragmatic randomised, mixed-methods study to explore the feasibility of using NEMDs in children with asthma at a specialist tertiary centre. Feasibility refers to the capability of the device to monitor adherence in clinical practice specifically investigating usability, acceptability and accuracy, using a qualitative and quantitative approach. Usability and acceptability of the NEMDs for patient and healthcare providers were assessed qualitatively using focus group discussions or one to one interview with children and their carers, and separately with paediatric respiratory Clinical Nurse Specialists (CNS). Device accuracy and impact on asthma outcomes were assessed by quantifying usable data from the devices and comparing adherence according to activation and correct inhalation and measuring asthma outcomes in the children. Ethical approval was obtained from the London Fulham Research committee and the Health Research Authority and the study was registered with clinicaltrials.gov. General Data Protection Regulations were followed.

Study population

Children aged between 6 and 16 years with a diagnosis of asthma, prescribed inhaled corticosteroids (ICS) and attending the paediatric asthma clinic at the Royal Brompton Hospital (RBH), London, UK were recruited from October 2018 to May 2019.

Children’s CNS working with the asthma team at RBH were recruited for the CNS focus group.

Written, informed consent was obtained from the parent/carer, and CNS’, and consent or age appropriate assent from the child.

Children were assigned to use one of the following four NEMDs for up to 12 weeks duration. All devices were CE marked and tested for safety (figure 1):

Figure 1

Novel Electronic Monitoring Devices used in the studyINCA, inhaler compliance assessment; R-DOT, remote directly observed therapy.

Hailie sensor (Adherium, New Zealand)

The Hailie is a battery powered electronic data logger which is suitable for attachment to a Turbohaler (eg, Symbicort). The sensor has an inbuilt electronic clock and calendar which logs the date and time of inhaler actuation. The device also contains a microphone that detects inhalation of the drug and a sensor for detecting the positioning of the Turbohaler.

Rafi-tone acoustic enabled smartphone APP (clin-e-cal, UK) with Flo-Tone (Clement Clarke, UK)

The Flo-Tone attaches to a metered-dose inhaler (MDI) and produces a sound when the inhaler is used correctly with a spacer. This sound activates a game which the child plays as they are inhaling the dose. The game only functioned when the correct flow was generated through the spacer for the Flo-tone to make a noise, which is detected by the App and drives the game. A record of the date and time the game was played is logged on a tracker. The game is aimed at younger children (6–11 years). It was designed as an incentive tool to improve inhaler technique for younger children and adapted for this study to record inhaler use.

INhaler Compliance Assessment (INCA_device (INCA, Ireland)

The INCA is an electronic data logger that attaches to a Diskus/Diskhaler dry powder inhaler (eg, Seretide Accuhaler). This device measures adherence using an audio recording. An inbuilt analysis of the digital audio recordings enables objective assessment of inhaler use and technique.

Remote Directly Observed Therapy (R-DOT)

The R-DOT platform monitors adherence by filming inhaler use, using a Smartphone. The clip is then uploaded via a secure website (Continga, UK). The clips were reviewed by a Paediatric asthma CNS to assess inhaler technique using the seven step inhaler checklist from the UK inhaler group.22 These were classified into effective if all seven steps were followed correctly, poor if none were followed and partially effective if at least one to six steps were followed correctly.

A pragmatic approach was used for recruitment, based on the child’s age and inhaler type as each device fitted different inhaler types. Blinding was not possible as the devices were easily identifiable. A sample size of 30 was initially selected, 10 in each arm (Hailie and R-DOT) and 5 each in Rafi-tone and INCA, as the Rafi-tone is aimed at younger children (6–12 years)and older children were recruited for the INCA arm as good inspiratory flow is needed for the Accuhaler.

Patient and public involvement

The lay summary was formally reviewed externally by young people with asthma and their parents. Their insightful and useful suggestions were incorporated into the study design and design of information sheets and the proposed study improved as a result.

Additionally, this is largely a qualitative study and therefore the views of children and their parents/carers has been the focus of this study. The results of the study were discussed in the focus groups/interviews conducted.

Usability and acceptability: a qualitative study

Separate topic guides were used to obtain qualitative feedback on the acceptability and usability of the devices for children/parents and CNS’. A second researcher (CP) provided support in both focus group discussions to minimise bias. All discussions were audio recorded and transcribed as ‘intelligent verbatim’. The qualitative data were analysed using Braun and Clarke’s six step process for conducting thematic analysis consisting of an initial familiarisation step of reading transcripts while listening to audio recordings.23 An inductive approach was then used to code the data. Initially complete coding was conducted using participant transcripts, followed by second and third coding. Themes and sub themes were identified until thematic saturation was reached, then categorised and reviewed before writing up. Twenty five percent of the transcripts were randomly chosen and double-coded by an independent researcher (AC) experienced in qualitative research in line with other qualitative studies which recommend ~ 20%24 25 as a method of analyst triangulation.

Accuracy of NEMDs : a quantitative study

Accuracy of the NEMD was determined by the adherence data collected from the devices as percentage of doses activated, inhaled and where appropriate correct technique. Asthma control was measured by using the asthma control test (ACT)26 for children ≥12 years or childhood ACT (cACT)27 for children <12 years; quality of life was assessed using the mini Paediatric Asthma Quality of Life questionnaire (mini PAQLQ)28 29; fractional exhaled nitric oxide (FENO)30 was measured using a NIOX VERO: Aerocrine, Stockholm, Sweden; spirometry for forced expiratory volume in 1 second (FEV1), forced vital capacity (FVC)30 31; and bronchodilator reversibility (BDR)32 33 were measured using a Vitalograph, Buckingham UK spirometer. These were conducted as part of routine clinical care assessments at the beginning and end of the monitoring period.

Results and analysis

A total of 52 patients were identified as eligible for inclusion, of whom, 35 (66% male, median age 12 years) consented to participate, and were randomised. Of these, 18 (51%) completed monitoring, 14 (78%) provided feedback, 7 (20%) were lost to follow-up, 4 (11%) experienced device failure, 4 (11%) lost their device or it was thrown away and 2 (7%) withdrew. The Consolidated Standards of Reporting Trials diagram is shown in figure 2.

Figure 2

CONSORT flow diagram for NEMD study. ADHD, attention-deficit hyperactivity disorder; ASD, autistic spectrum disorder; Safeguarding concern: child in need plans were in place or about to be placed. CONSORT, Consolidated Standards of Reporting Trials; DOT, directly observed therapy; INCA, inhaler compliance assessment; NEMD, novel electronic monitoring device; R-DOT, remote DOT.

The duration of monitoring for each device varied: the median number of days of Hailie monitoring was 88.5 days (range 6–120), Rafi- tone 25.5 days (range 25–26) and R-DOT 54 days (range of 8–145). The length of monitoring was longer in the R-DOT and Hailie arm as recruitment occurred earlier in the study.

Usability and acceptability: qualitative data results

Of the participants completing monitoring, 14/18 (78%) participated in the focus groups or individual interviews. Of these, four participants and their parents attended the focus group meeting, eight participants were interviewed individually face to face and two participant were interviewed over the telephone.

The duration of the patient focus group was 63 min; individual interviews were a mean 15.3 (SD ±8) min.

For the healthcare professional focus group, all of the three invited CNS attended the focus group meeting. The focus group lasted 56 min.

Four key themes influencing patient and healthcare professional engagement were identified from these focus groups and interviews:

Functionality of the device.

Usability of the device.

Perceptions and emotions.

Enhancements, improvements and preferences.

These are described below—full tables of quotes have been included in the online supplemental 1.

Functionality of device

Accuracy and reliability

The INCA and Hailie were considered by both patients and CNS, the most accurate and reliable followed by R-DOT as it only records adherence to video uploads.

An adolescent commented ‘I think with the Hailie, it’s quite black and white. You've either done it properly or you haven't’ (NEMD1).

Another adolescent using R-DOT stated: ‘I took the inhaler just not the video’(NEMD31).

Rafi-tone was considered to be less accurate, as observed by the CNS’ as the App records the sound of the Flo-tone and did not differentiate the inhaler type, ‘with the Rafi tone it’s just attached to the spacer, the inhaler that they’re using could be their preventer or it could be their reliever’ (CNS1).

Assessment of inhaler technique

Both R-DOT and Rafi-tone were perceived to help with improving technique. Although Hailie and INCA assessed technique, interventions could not be implemented until the patient returned to clinic and the data downloaded.

An adolescent reflected: ‘one good thing about the R-DOT was the technique thing, which for me was important’(NEMD1).

A child described how the Rafi-tone ‘makes me feel like I know how to do my pump more, and I can actually use it to help me out with my breathing’ (NEMD 34). A CNS commented that the Rafi-tone is ‘Good if we’ve got concerns about someone that has dysfunctional breathing’(CNS 1).

Notifications, reminder and display functions

Notifications, reminders and visual data display were reported as positive attributes of the Hailie.

An adolescent commented: ‘So, when I was using the Hailie I thought it was pretty good because when…on my phone there would pop up a notification. And then I'd take it’ (NEMD 18).

An adolescent felt that with the Hailie: ‘I just think it’s having my data there, would just help me to remember to take it more’ (NEMD 4).

Usability of device

Ease of use

Children, parents/carers and CNS’ considered the Hailie and INCA easier to use as they attached directly to the inhaler, did not affect routine and required minimal effort.21

With the Hailie an adolescent felt that ‘it’s just the easiest thing, as I said, fast mornings, just take it normally. So, yes, if you want to do the bare minimum, the Smartinhaler/Hailie is good for that’(NEMD 4).

The R-DOT often needed two people and was considered to require effort, affected routine and some adolescents felt it diminished independence and autonomy.

An adolescent commented on R-DOT ‘you can't film yourself, that’s the issue. So, mum had to film me’(NEMD1). The participant’s mum commented ‘It wasn't the fact that it took time, it was the fact that we had to coordinate’. The same adolescent commented: ’for me, when I have a set routine that I get up and straight away I have my inhaler, it doesn't work as well because then you've got to sort out a time that works for everyone’ (NEMD 1).

However, R-DOT provided physical evidence to parents/carers/healthcare providers that the medication was actually taken correctly as commented by one of the participant’s parent: ‘For me, recording, it’s better because I can see how [the participant’] doing. Sometimes [the participant] doesn't do it properly when [the participant] doesn't have time. But if I record [the participant] I can see how [the participant’s] doing it properly’ (Parent of NEMD2).

The cost of the device and reusability was considered by participants to be important. One CNS commented ‘The problem is that if you have a very expensive device in a disposable form, [i.e. INCA] you throw away the inhaler afterwards. You've got to be able to take it off and attach it to your next one’ (CNS1).

Independence and autonomy were considered by children to be valuable with one adolescent in the INCA arm stating: ‘my mum and dad can be there to say to take it but at the end of the day they can't force me to take it. I have to take that step. So, I think by getting rid of them telling me to take it and me just remembering myself or be reminded would really help me to take the step forward and actually benefitting from the medicine’ (NEMD5).

This was reflected also in the CNS’ perspectives: ‘obviously if you’re like 14, 15, 16 years they don’t want their mum or dad kind of looking over them making sure they’re going to do it’(CNS1)

One parent in R-DOT arm felt that monitoring was to keep an eye on them ‘To fault parents to see if they are doing it wrong because I personally think that’s what it is, to say, yeah, you are not doing this right’ (parent of NEMD 32).

Emotions such as embarrassment due to personal or domestic appearance were expressed by some participants using R-DOT. A child said ‘I don’t feel I look nice, and I don’t want to be videoed. Sometimes, oh my hair is not right. And then I think in my head, oh, it doesn’t matter they know that sometimes in the morning having like crazy hair’ (NEMD 32).

A parent using R-DOT felt ‘so you're sitting there thinking I've got to get a headshot because [the participant’s] not dressed. It’s things like that. And [the participant’s] room’s a tip and you're thinking oh crikey’ (NEMD 1).

Some participants reported that monitoring per se led to behaviour change as described by this adolescent using Hailie: ’When I'm taking my inhaler sometimes, I realize that I’m being monitored. So that gives me a little boost and just try and do it properly. Because I remember oh, they're monitoring me, so I want to get good results. I think that’s why I do it’ (NEMD 10).

Improvement and personal preferences

Expression of personal preferences

In the focus group, most had previously been monitored using an EMD (Smartinhaler). The Hailie was felt to have many positive attributes as it had the ability to show adherence data in visual format such as bar charts, required little additional effort and time, provided instant feedback on percentage adherence in clinic and was able to monitor activation, inhalation and correct orientation of the device.

‘I think the Halie is a better choice because if it’s programmed and you know you’ve got to take it, and if you’ve forgotten it, it will remind you’ (NEMD 18).

‘You can get a lot of data from a single device, you can see kind of patterns of when they’ve done it with the Hailie’(CNS1).

Recommendations on improvements

In terms of suggestions for improvements, participants wanted to see their own adherence data on their phone to act as a reminder. Others wanted instant feedback on their technique especially when they are unwell.

‘Like seeing what I’ve visually taken, would just help me to remember to take more, because then when you’re feeling bad at the end of the week you look back and you think actually, I forgot’ (NEMD18).

One adolescent compared the Hailie with a previous EMD ‘So that’s very useful. Because previously with a Smartinhaler we had no idea what the data was you were getting, or this data and we didn't know what was happening day-to-day. So, to get that feedback instantly… is useful’(NEMD1).

Concept of personalised adherence monitoring

Being able to personalise the NEMD according to the needs of the patient was discussed as important, as reflected in the quote from one of the CNS:

‘I think probably depending on the patient…for parents who kind of want to distract them into taking their inhaler they would probably like the Rafi-tone, then for those who are independent, are getting a bit older, the Hailie is the way forward depending on age, then those with inhaler technique issues R-DOT, biggest thing is what would work’ (CNS1).

Accuracy and impact on asthma outcomes: quantitative data results

The median (range) adherence for R-DOT was 72% (51%–94%) for Hailie 65% (50%–100%) and for Rafi-tone 87% (82%–92%) Insufficient INCA devices were returned to calculate adherence.

Comparison of type of adherence data collected from the Hailie can be seen below (table 1).

Table 1

Comparison of adherence data with Hailie

Patient ID	Activation (%)	Activation and inhalation(%)	Activation, inhalation and correct orientation(%)
NEMD4	63	58	24
NEMD8	73	52	5
NEMD18	50	50	25
NEMD11	52	50	22
NEMD12	100	72	25
NEMD10	53	30	28
Mean (SD)	65 (19)	52 (13)	22 (8)

NEMD, novel electronic monitoring device.

The median (range) adherence to video uploads with the R-DOT was 73% (64%–94%) based on duration of R-DOT usage (table 2). There was no intervention during the monitoring period.

Table 2

Comparison of adherence to video uploads

Patient ID	Duration of monitoring (weeks)	No of days R-DOT usage	No of videos uploaded	No of days missed at least one video upload	% adherence to video uploads based on duration of usage
NEMD1	11 weeks	76	106	40	70
NEMD2	10 weeks	70	105	32	75
NEMD11	36 weeks	84	158	10	94
NEMD14	21 weeks	84	70	16	83
NEMD15	1 weeks	8	6	2	75
NEMD16	2 weeks	13	18	7	64
NEMD31	33 weeks	84	55	46	66
NEMD32	5 weeks	35	46	19	66

NEMD, novel electronic monitoring device; R-DOT, remote directly observed therapy.

Inhaler technique was assessed to be effective (>80%) in 3/8 children and partially effective in 5/8 with only one occurrence of poor technique (table 3).

Table 3

Assessment of inhaler technique by CNS using the UKIG checklist

Patient ID	Week1	Week2	Week3	Week4	Week5	Week6	Week7	Week8	Week9	Week10	Week11	Week12
NEMD1	E	E	E	PaE	E	PaE	E	E	E	PaE	PaE	PaE
NEMD2	PaE	PaE	PaE	PaE	PaE	PaE	PaE	PaE	PaE	PaE	NR	NR
NEMD11	E	PaE	E	PaE	E	E	E	E	E	E	E	E
NEMD14	PaE	PaE	P	PaE	PaE	PaE	PaE	PaE	PaE	PaE	PaE	PaE
NEMD15	E	E	NR	NR	NR	NR	NR	NR	NR	NR	NR	NR
NEMD16	E	E	NR	NR	NR	NR	NR	NR	NR	NR	NR	NR
NEMD31	PaE	PaE	PaE	PaE	PaE	PaE	PaE	PaE	PaE	PaE	PaE	PaE
NEMD32	PaE	PaE	PaE	PaE	PaE	NR	NR	NR	NR	NR	NR	NR

CNS, clinical nurse specialist; E, effective; NEMD, novel electronic monitoring device; NR, no recording; P, poor; PaE, partially effective; UKIG, UK Inhaler Group.

Impact on asthma control

Baseline and follow-up scores for ACT/cACT, m-PAQLQ, FENO, spirometry and BDR for the different groups are reported in supplementary material (online supplemental 2 and 3).

Acceptability criteria

The characteristics of each of the NEMD in terms of usability, acceptability and accuracy were amalgamated and used to devise a list of acceptability criteria by the research team and an expert panel (consisting of paediatric respiratory consultants, CNS and pharmacist) who reviewed the study results box 1.

Accuracy

Activation of dose recorded.

Inhalation of dose recorded.

Technique assessed.

Minimal effort

Minimum amount of time.

Fits in with daily routine.

Ability to set notifications and reminders.

Allows independent monitoring.

Ability to display visual data to children/carers.

Ability to display data to healthcare professionals.

Easy data analysis.

Ability to easily upload data in clinic.

Ability to correct technique.

Reliable adherence monitoring.

Cannot be thrown away/lost.

Minimal cost of device.

Flexibility to fit all inhaler types.

Technology issues.

Discussion

This is the first study to explore the feasibility of four NEMDs for use in children with asthma using both qualitative and quantitative methods.

The key finding was that devices that use objective measures of activation, inhalation and technique which are accurate, require least effort, are easy to use and fit into existing routines of children and carers were preferred by children, their carers and healthcare professionals. Other ideal characteristics include cost-effectiveness as expressed by CNS, flexibility to set up reminders with notifications as suggested by children, and ability to provide visual display of adherence data in real time which was desired by both children and healthcare professionals.

The four platforms were chosen for this study as they use different methodologies to assess inhaler technique. Advantages and disadvantages were identified by participants for each. The Hailie and the INCA were popular with both children and CNS’ due to their ease of use and minimal additional effort required on the part of the child. The Hailie has the ability to detect adherence in real time calculating percentage adherence easily, though this function was not enabled in our study. Furthermore, it provided useful data for healthcare professionals on individual components of inhaler technique. However, both the Hailie and the INCA devices are specific to a single type of inhaler device: the Turbohaler and Accuhaler, respectively. These are both breath-actuated inhalers and require adequate inspiratory flow to use, which younger children may find difficult to generate. Of note, recruitment in the block for the INCA device was low as most children who are prescribed Seretide in our clinic use the MDI version not the Accuhaler. Unfortunately for the INCA block, no usable adherence data were obtained as the devices were thrown away or not fitted correctly. This is in contrast to the study by Sulaiman et al in adults which demonstrated the utility of the device for monitoring inhaler use and technique.19 Many of the advantages identified with the Hailie, such as ease of use, requiring no extra steps are also relevant to the INCA. At the study site, an MDI with a spacer was the most frequently used inhaler type. Unfortunately, there were no devices available at the time of the study which had the usability and functionality of the Haile and INCA, and could also be used with an MDI and spacer.

The Rafi-tone App was designed to help young children improve inhaler technique and to incentivise inhaler use via a game. This was adapted by the developer of the Clin-e-cal App for this study to record adherence. The App is aimed at younger children and therefore we limited recruitment to this block to children aged 6–11 years. Feedback from parents was positive in terms of the impact on technique. However, the current functionality of the App is limited to collecting data when the correct pitch is heard from the Flo-tone. It is not possible to know which inhaler was used, the number of puffs used or if the Flo-tone was even attached to an inhaler. This limits the utility of this platform for monitoring ICS adherence (for which it was not originally designed). Additional effort was required on the part of the parent/caregiver to ensure that the App was open at the time the inhaler was given. Nonetheless based on our study findings, an App for children which incentivises inhaler use, particularly one that rewards good technique, has the potential to be an effective intervention that merits further study and consideration.

The R-DOT platform was the only platform capable of monitoring use for all types of inhaler devices, which is particularly advantageous given the number of different inhaler types now available. It also enables technique to be assessed by a trained professional and ensures that all aspects of inhaler technique are checked. Furthermore, feedback can be given rapidly to correct technique. In our study, it was not possible to review inhaler technique in real time, due to logistical constraints and there was no intervention during the monitoring period. In a pilot study of this device, positive user feedback was reported with improvement in technique over successive weeks of monitoring and intervention.20 In contrast, some of the comments in our study from children reflected concerns about filming themselves. This was particularly the case among adolescents who were concerned about their appearance. Although this platform is excellent for information on technique it provides little definitive information on adherence. Failure to record a video does not necessarily mean that the inhaler has not been used. Parents highlighted the difficulties of fitting this in to a busy routine. It is possible that if this change in routine could be achieved in the short term, it might lead to longer-term behavioural change where correct inhaler use is embedded in the daily routine.2 CNS' felt that this was a good way to assess technique; however, reviewing the videos took time and they do not currently have the capacity to do this outside of a clinical study. Advantages with the R-DOT included the physical evidence that the dose was taken correctly enabling remote offsite monitoring of children with provision of instant feedback.

The main strength of this study is that it is the first time that the INCA, Rafi-tone and Hailie have been used as adherence monitoring tools in a real-world setting in children, and compared using both quantitative and qualitative methods.

The main limitation of this study is the small sample size which did not enable robust quantitative statistical analysis of the data. This study took place within a specialist tertiary asthma clinic and the findings may not be generalisable to the general population of children with asthma, particularly in primary care. Further evaluation of these NEMDs in a wider population is warranted. However, many of the findings are pertinent across care settings, particularly those relating to cost and impact on the workload of healthcare professionals. The acceptability criteria developed from our findings could potentially be used for testing future NEMDs.

Conclusion

There is no ‘one size fits all’ NEMD and there were advantages and disadvantages identified for all the devices tested. The choice of device needs to be tailored to the needs and preferences of the patient based on improving technique or monitoring adherence, and there is a place for personalised adherence monitoring. The data from this study can inform the development of an ideal device which combines the key functionalities of each of the platforms. Device selection needs to be versatile and compatible with all inhaler types, provide accurate adherence monitoring, require minimum additional effort on the part of the child and their carer, and be able to provide real-time visual adherence data to patients and healthcare professional, including accuracy and feedback on technique. Our study identified the factors that influence device usability and acceptability and provided early data on the accuracy of these devices. Further studies are now warranted to evaluate how these findings can be applied to interventions to improve adherence, use in virtual clinics and how using these devices influence adherence and asthma outcomes