Delivering digital health and well-being at scale: lessons learned during the implementation of the dallas program in the United Kingdom.

OBJECTIVE: To identify implementation lessons from the United Kingdom Delivering Assisted Living Lifestyles at Scale (dallas) program-a large-scale, national technology program that aims to deliver a broad range of digital services and products to the public to promote health and well-being.
MATERIALS AND METHODS: Prospective, longitudinal qualitative research study investigating implementation processes. Qualitative data collected includes semi-structured e-Health Implementation Toolkit-led interviews at baseline/mid-point (n = 38), quarterly evaluation, quarterly technical and barrier and solutions reports, observational logs, quarterly evaluation alignment interviews with project leads, observational data collected during meetings, and ethnographic data from dallas events (n > 200 distinct pieces of qualitative data). Data analysis was guided by Normalization Process Theory, a sociological theory that aids conceptualization of implementation issues in complex healthcare settings.
RESULTS: Five key challenges were identified: 1) The challenge of establishing and maintaining large heterogeneous, multi-agency partnerships to deliver new models of healthcare; 2) The need for resilience in the face of barriers and set-backs including the backdrop of continually changing external environments; 3) The inherent tension between embracing innovative co-design and achieving delivery at pace and at scale; 4) The effects of branding and marketing issues in consumer healthcare settings; and 5) The challenge of interoperability and information governance, when commercial proprietary models are dominant.
CONCLUSIONS: The magnitude and ambition of the dallas program provides a unique opportunity to investigate the macro level implementation challenges faced when designing and delivering digital health and wellness services at scale. Flexibility, adaptability, and resilience are key implementation facilitators when shifting to new digitally enabled models of care.

BACKGROUND

Healthcare systems globally recognize the need to adapt in order to accommodate unprecedented changes in population demographics and related increases in incidence of chronic disease. Interactive, person-centered digital tools and services offer a vehicle to promote a more citizen-led, self-care and preventative health and well-being agenda.

Previous studies in the evolving interdisciplinary field of health informatics have highlighted the complex nature of implementing digital health and well-being tools in practice. This is due in part to the breadth and complexity of the systems, processes, and stakeholders involved in implementing e-health interventions. A recent study by Cresswell et al. highlighted 10 key considerations for implementing e-health interventions at scale, including: clarification of the problem being addressed, building consensus, planning, addressing infrastructure, and evaluation. However, to date, most of the evaluation literature focuses on single digital tools or systems at a time, such as the implementation of electronic health records, computerized decision support systems, or the implementation of telemedicine services. , In contrast, the Delivering Assisted Living Lifestyles at Scale (dallas) program aims to deploy a broad portfolio of digital tools and services and represents the next stage toward deployment of such technologies for health and well-being at scale in the United Kingdom.

The dallas program is a pan-UK program that was funded by Innovate UK (formerly the Technology Strategy Board) ( https://www.gov.uk/government/organisations/innovate-uk ), the National Institute for Health Research, The Scottish Government, Scottish Enterprise, and Highlands and Islands Enterprise. The total investment of £37 million (over the period 2012–2015) reflects the current emphasis being placed on developing digitally enabled healthcare and wellness globally. The funding was delivered via Small Business Research Initiative to 4 lead contractors and the projects developed in conjunction with a highly innovative group of sub-contracted organizations, mostly private sector business and small companies (referred to as SMEs). The dallas program is highly ambitious and aims to deliver health and well-being services (to 169 000 individuals) using a wide range of technologies including interactive, person-centered digital portals; telecare; electronic personal health records; and Mobile applications (Apps) at scale and across remote, rural, and urban areas of the United Kingdom. It consists of 4 multi-agency consortia or “communities”: More Independent ; i-Focus ; Living it Up , and Year Zero working in new collaborative partnerships and distributed across the United Kingdom ( Figure 1 ). Each community involves health and care services, industry (including small-, medium-, and large-size companies), nongovernmental, third-sector and voluntary organizations, as well as academia and government bodies (see table in online Appendix). As such, dallas aims to harness new knowledge across traditional boundaries and disparate systems to introduce interoperable, person-centered digital tools and enable more adaptive systems to provide a new “space” for interactive, person-centered, digital health and wellbeing products and services.

Figure 1:

The 4 multi-agency dallas consortia.

OBJECTIVE

The objective of the present study was to report on the qualitative evaluation conducted, which aimed to identify the barriers and facilitators in the dallas implementation journey and to share implementation lessons learned within and across the unique dallas program.

MATERIALS AND METHODS

We have a multi-disciplinary team with expertise in General Practice and Primary Care, Computing Science & Human Computer Interaction (HCI), Health Informatics, Nursing, Health Economics, Statistics, and Social Sciences. The team is working closely with the dallas communities to conduct an independent evaluation.

Data Collection

Our evaluation adopts a socio-technical approach, using a mixed methods evaluation framework consistent with evaluations of complex interventions. Qualitative data has been collected longitudinally from the four communities. Table 1 outlines the breadth and extent of the data collected.

Table 1:

Summary of Qualitative datasets collected (as of 23 January, 2015).

Qualitative data collected	Number of items	Number of pages
e-Health Implementation Toolkit (e-HIT) baseline research interviews	17	257
e-HIT midpoint research interviews	21	454
User stories	9	12
Evaluation alignment interviews	5	14
Semi-structured research interviews	9	111
Barriers/facilitators/ lessons learned reports	6	18
Product/service development planning documents	18	245
Contract/bids and appendices	13	74
Observation research logs	10	34
Reach recruitment and membership documents	14	59
Quarterly technical reports	38	262
Quarterly evaluation reports	25	190
Focus group/workshop reports	3	36
Local evaluation reports	4	207
Other: (Initiation report/Dissemination report/Eval planning doc/Outline brief)	9	125
Total (as of 23 January 2015)	201	2098

The present study draws on the evaluation alignment interviews, the barriers and facilitators reports, and the in-depth e-Health Implementation Toolkit (e-HIT) led semi-structured research interviews held at baseline ( n = 17) and approximately 12–14 months later (mid-point; n = 21) of the implementation process. The e-HIT is a tool to aid consideration of implementation issues in e-health, underpinned by Normalization Process Theory (NPT). , Stakeholders sampled represent a cross-section of those involved in the dallas digital service re-design and delivery and include professionals from public sector health (NHS) and social care, business and industry leads, technical ICT personnel, voluntary and third-sector organizations, academia, and other government bodies.

Research Governance and Ethics

University of Glasgow ethical approval was granted for this study. All respondents provided consent for participation. Identities are protected and are assigned a confidential generic descriptor to ensure anonymity.

Theoretical Framing of Qualitative Data Analysis

In order to conduct a meaningful evaluation of the complex socio-technical processes involved in the implementation of digital tools and services within dallas, we have drawn on NPT, which has been used in e-health implementation projects. ,,, The judicious choice of a robust underpinning theoretical framework is known to aid with conceptualization of analysis in complex adaptive systems such as healthcare settings. NPT has 4 constituent constructs ( Figure 2 ).

Figure 2:

Representation of the 4 constituent NPT constructs which attend to the 4 key aspects in e-health implementation. (From May and Finch, 2009).

Qualitative Data Analysis Procedure

All baseline and midpoint research interviews were transcribed verbatim and transcripts checked for accuracy. Transcripts were coded and analyzed in an inductive manner. Codes and themes were then mapped to NPT, as a conceptual framework and system of organizing the data ( Table 2 ).

Table 2:

Normalization Process Theory coding framework used for qualitative data analysis.

Coherence (sense-making work)	Cognitive participation (engagement/buy in work)	Collective action (enacting work)	Reflexive monitoring (appraisal work)
Differentiation	Enrollment	Skill-Set Workability	Reconfiguration
Is there a clear understanding of how the dallas technology products, tools, and e-health services differ from existing, current practice and services?	Do implementers, service providers, service users, and other partners “buy into” the dallas technology developments, tools, and e-health services?	How does the implementation of the dallas services and products affect division of labor of work practices, roles and responsibilities, or training needs?	Do participants (service user/service provider/other individuals) try to develop a “work around” or somehow alter a dallas service, technology, or product?
Communal Specification	Activation	Contextual Integration	Communal Appraisal
Do the dallas implementers, stakeholders, service users, service providers, business leads, third sector, voluntary, and other partners have a shared understanding of the aims, objectives, and expected benefits of the dallas e-health products and service(s)?	Can implementers, service users, service providers, and other partners who participate in the dallas communities/program sustain its implementation?	Is there organizational support in terms of resource allocation to enable the service users and service providers to enact a new set of practices to implement the new dallas products or services?	How do service user groups/service provider groups/service leaders/other groups judge and determine the value of the dallas technology products and other services?
Individual Specification	Initiation	Interactional Workability	Individual Appraisal
Do all dallas stakeholders (in each community) have a clear understanding of their own specific tasks and responsibilities in achieving the implementation of the dallas product or services?	Are key individuals willing to drive the implementation of the dallas products, tools, and services forward? Who are they?	Do the dallas e-health service(s) and products make routines of practice easier or make people’s work easier?	How do individual participants/individual service users/other individuals appraise the effects of the implementation of the dallas service, technologies or products on them and their (work/home, as in context of tool resource, etc.) environment?
Internalization	Legitimation	Relational Integration	Systematization
Do all dallas stakeholders understand the value, benefits, significance, and importance of the dallas products or services and their future value?	Do implementers and participants believe it is right for them to be involved in implementation of dallas services and products? Do they feel they can make a valid contribution to the implementation of the dallas products and services?	Do service users/service providers/other participants have confidence in using the dallastechnologies, products, and services?	How do participants and implementers determine the effectiveness (benefits and limitations) or usefulness of the dallas tool, service, or product? How can this be measured?

Data coding clinics were conducted at regular intervals among the team using samples of coded transcripts at baseline and mid-point to ensure accuracy and consistency of coding. We then mapped the results from each community in order to capture 5 of the significant challenges and navigation processes implemented across dallas as follows:

Challenges related to working as part of a large multi-agency, heterogeneous consortium.

Challenges related to the wider socio-political and economic environment.

Challenge of co-design at scale.

Challenge of branding and marketing.

Challenges related to interoperability and information governance (IG).

Emergent findings were shared with key leads and related stakeholders who concurred with the findings. The mapping of the 5 main challenges and navigation processes to the fine-grained NPT codes is presented in Figure 3 . Results are presented and organized according to overarching themes as identified within dallas, and data presented drawing from the cross-section of stakeholders involved in order to provide depth and breadth to the findings.

Figure 3:

Diagrammatic representation of the 5 overarching dallas themes and the underlying mapping to the Normalization Process Theory constructs.

RESULTS

Here we provide details of 5 key implementation themes identified from the early phases of the dallas program.

1. Challenges and Learning to Work within New, Multi-agency, Heterogeneous Partnership Models

One of the strategic aims of the dallas program was to support new partnerships to foster innovation, drawing on a diverse range of organizations including the NHS, local authorities, SMEs, voluntary and community organizations, as well as academia. However, challenges emerged related to forming and sustaining such heterogeneous partnerships with little shared history of working together. Reservations were expressed across sector boundaries, with perceptions of inertia and resistance to change in the NHS compared to the speed of change in the business world ( Table 3 , Q1). There were also cultural differences between NHS organizations and the retail or technology/ business partners. Examples included the way in which each viewed dallas as a scaled-up and live project, adherence to governance, and focus on developing finished digital products ( Table 3 , Q2).

Table 3:

Illustrative data excerpts related to Partnership Working in Multi-agency, Heterogeneous Consortia.

Working across boundaries	Q1 “ … in the health service, there’s a big inertia to bringing in a change and… the intervention in the consumer space, it’s, you know, it’s much more receptive to that.” (C4(b) Operations – Business).
	Q2 “ … there are NHS organisations and they’re very keen on making sure governance is adhered to. I’m not saying that the retail or the manufacturing partners aren’t, but we’ve got a very keen eye for that whereas they’ve got a very keen eye on finished products and getting things there. But that doesn’t cause any issues, I don’t think, I think it probably complements each other and it’s a new way of working as well. ” (C2(b) Manager, Informatics).
	Q3 “ … we are comfortable with—as community engagement partners—that they be strong for the people that are involved. The industry and technology partners are comfortable that a tangible outcome means they can get on and do something and build something. ” (C1(b) Representative, Third Sector organization).
Differences in the local digital health economy	Q4 “ … we’ve gone from having four [name of product] deploying partners down to two and the contrast between [NHS organisation 1] and [NHS organisation 2] in some levels is quite striking. So [NHS organisation 1] seem to be much clearer on their process maps and their interactions and the benefits of the product. [NHS organisation 2] don’t seem to understand the internal structural process … [NHS organisation 1], as I say, they’re much further developed in terms of their own Digital Strategy as an organisation so their staff are… they do Mobile working, they have tablets and, you know, they’re digitally enabled.” (C3(m) Manager 1 – Business).
Lack of shared understanding between partners	Q5 “ So … various things that took longer than expected and I think the contract, getting it one week and then expecting us to sort of sign it and start the, start within a couple of weeks, that was never going to happen.” (C2(b) Manager 1 – NHS).
	Q6 “ … and on the NHS side, thinking… about six months in… people started to talk about pilots…and we were going, it’s not a pilot. It says that it’s not a pilot. This isn’t a pilot. It’s not going to help you if you think, it’s not going to help any of us to think of it as a pilot. We’re supposed to be deploying these things into use, not talking about pilots, not inventing… you know, and… but that only occurred later. And they’d already started.” (C3(m) Manager 2 – Business).
	Q7 “ … in hindsight I think what should have been done is… each of those partners should have articulated those things much more clearly beforehand and been selected on that basis. You know… a clear position on where they’re at within their own digital strategies, organisationally.” (C3(m) Manager 1 – Business).
Partners in the right spaces	Q8 “It’s all about partners working together, making sure we all understand what we’re doing, who’s doing what so we can then feedback to our teams to give people that kind of general understanding. But also I need to make sure that [Voluntary organization name], we’re delivering on the champion’s front, which is recruiting 150 volunteers to go and talk about health and wellbeing, but around the assisted technology as well. So, I manage that and underneath me you have a project manager and you have eight staff who are all working on it … ” (C2(b) Representative, Voluntary organization).
	Q9 “I think [Name’s] point about being—the disconnect—perhaps between the visionaries and the resource that’s got, or the Management that’s got to implement is an important lesson. It’s about making sure that those people who are sitting at the table saying, my organisation can do X, Y and Z are actually connected with the people who’ve got to do the X, Y and Z and we can see that within the program in that all the partners so [Third sector organisation name], [Charity organisation 1 name], [Charity organisation 2 name], the people who are sitting around the table are the people who have it in their authority to go, will this work, yes, it will, we’ll sign it off. Whereas within the NHS and the local authority the visionaries weren’t necessarily directly connected or influential… To the operations bit of the organisation. So that’s an interesting lesson.” (C2(m) Manager 2 – NHS).
Leadership and project management skills	Q10 “ We are not frightened of making decisions, there are clearly risks roundabout that and we’ve taken them on our shoulders and made sure that the right people are briefed but yes I think that’s actually been quite a significant benefit to the project as well.” (C1(m) Manager 1 – NHS).
New collaborative working	Q11 “ … the thing that’s more difficult to describe is the activity I think [Name] referred to earlier on, the activity that’s starting to happen between partners so it’s more about the relationship which we’re starting to get here where people see mutual benefit in doing things differently together … ” (C2(m) Manager 2 – NHS)

Some communities struggled initially to communicate across the diverse array of partners and had to work toward understanding new processes and ways of working. For example, the voluntary sector is comfortable with “grass-roots” community engagement whilst technical/digital technology partners feel more comfortable with progressing directly to build an actual product ( Table 3 , Q3). Other partnerships involved different NHS organizations, which varied in terms of their digital readiness ( Table 3 , Q4). This lack of strategic knowledge was communicated by stakeholders as being, at least partially, due to the unusual initial contractual procedures with tension related to the speed with which the lead and sub-contractors had to sign off the contract ( Table 3 , Q5). Stakeholders reported it took some time for the contracts to be fully understood and embedded and understanding of roles and commitments to crystallize ( Table 3 , Q5, Q6). This experience led some stakeholders to reflect on what would help across multi-agency partnerships if this type of contracting arrangement was implemented in the future including, e.g., a much clearer articulation of roles at the beginning ( Table 3 , Q7).

Each community overcame barriers differently, but the issue of “choosing the right partner” arose across all communities ( Table 3 , Q8). Suitable partners should be able to implement action at the level of the operations within their own respective organizations as well as balance the “visionaries” with the “pragmatists” to prevent aspirations outrunning ability and capacity of the consortium as a whole ( Table 3 , Q8, Q9).

In spite of initial challenges, the multi-agency partnerships made significant progress and can now share their learning on what helps to facilitate new collaborative partnerships across traditional silos between different communities of practice. Most of the facilitators are typical of good project management and include keeping in constant dialogue across the partnerships, clear communication, negotiation, and active problem-solving skills. The importance of team work and understanding exactly what roles entail at an individual and collective level are of key importance as are astute, strategic leadership, and strong project management skills in ensuring that a shared vision or coherence emerges and stakeholders “buy into” the direction of travel ( Table 3 , Q10, Q11).

2. Need for Resilience in the Face of Challenging Socio-political and Economic Factors in the External Environment

Digital and technology based health interventions are not implemented in a vacuum, but are intrinsically related to the complex socio-technical features within organizations, as well as the wider political and economic factors in the external environment. Some dallas consortia had to work on digital innovation against the backdrop of NHS England undergoing a radical restructuring process. This resulted in uncertainty and disruption along with a fear of role redundancy ( Table 4 , Q1, Q2), which affected engagement and the operationalization of services (mapping onto coherence, cognitive participation, and collective action constructs of NPT). This was particularly challenging for one of the business-led dallas consortia in the initial stages when they were trying to engage with several NHS partners, each of which were facing structural changes within their own localities ( Table 4 , Q3).

Table 4:

Illustrative Data Excerpts Relating to the Challenging Wider External Environment.

Restructuring of NHS England	Q1 “ … the health services are going through this tremendous upheaval. It’s beyond anything that any of us have experienced. I’ve been in the health service now nearly 40 years. I mean, it has… I think it’s well evidenced that nothing like this has happened since the health service was incepted. So people are trying to deal with the here and now, and understand what’s happening., in their own sphere of work in health service-land. I think the local authority are doing the same, against a background of massive efficiencies and so, they can be very distracting, understandably.… So I think people, in their hearts, understand them and want it promoted and be sponsors for it, but I think because there’s so much going on, they’re just distracted from that … ” (C2(b) Clinicial – NHS).
Fear of role redundancy	Q2 Interviewee “…..but I think there is some negative… negativity among different staff groups thinking technology will be replacing people.” IV “What staff groups are you sensing that among?” Interviewee “Carers.” IV “The social Carers, that they’ll be made redundant by this… these technologies?”Interviewee “Well, yes. I think that’s quite a big initial thought of a lot of different care groups, that they’d be made redundant by the introduction of technology.” (Interview with C2(b) Representative – Charity organization).
	Q3 “I think there’s been, particularly the first year, there was almost no focus, it was very hard to get attention from the NHS, where everybody whom you thought who could have become a Champion was… was worried about what their next job was. You know, the whole thing, you it’s hard to imagine, […], how disruptive that was.” (C3(m) Manager 2 – Business).
Aligning with new organizational structures	Q4 “ … Because the clinical commissioning groups are now bought into dallas and they will drive this forward. I think the only concern is that there is a lot of change going on at the moment, but a lot of the change that is going on with the clinical commissioning groups involved are things that were suggested in dallas… ” (C2(b) Manager, Informatics).
Impact of wider economic environment	Q5 “ … and unfortunately they went bust so that was quite a setback for us. The alternative we came up with which was for online and telephone sales; that hasn’t gone down brilliantly … and what we’ve done since then is we’ve re-grouped and reconsidered; well, should we be selling them?[…]. We’ve just sort of moved into that because we’ve lost our Retail so we’ve regrouped now.We are back where we wanted to be, I think, which is we will vet products, make sure that the products are reliable, etc., before we actually feature them … and so on. But we would look to proper Retailers to actually sell them rather than trying to sell them ourselves. ” (C2(m) Manager 1 – NHS).

The wider political environment of austerity and economic recession was thought to be an external driver with some suspicion voiced that the real motive for introducing digital tools and services was as a cost-cutting measure as opposed to improving person-centered healthcare and well-being ( Table 4 , Q2). However, in at least one of the consortia, work was invested in ensuring that the dallas program was in alignment with the ongoing strategic planning of the new NHS structures. Indeed there was recognition that many of the dallas aims and objectives were similar to those of the new organizations, with regards to technologies as an enabler of more integrated, personalized health and well-being systems ( Table 4 , Q4).

In addition to the challenge of navigating the restructuring of NHS England, there were some key retail partners who went out of business and into receivership against the backdrop of economic austerity. A large commercial partner also withdrew from a consortium, due to wider company-related issues. This was recognized as an inherent risk from the outset, but consortia had worked with such new partners in order to try and build consumer-based business models in the existing health economy. However, the reality faced by more than one of the multiple partnerships was the collapse of their “route to market” through well branded, trusted commercial partners going bankrupt or withdrawing. As a result, the dallas communities had to recover and actively find new solutions in order to overcome these significant set-backs and move forward ( Table 4 , Q5).

3. Inherent Tension between Embracing Co-design and Achieving Delivery at Pace and Scale

One of the major strategic aims of dallas was to innovate through the co-design of more person-centered, interactive digital tools and services and to do this at scale. The emphasis on more personalized tools and services was viewed as part of the solution to the challenges in current healthcare and well-being provision (see Table 5 , Q1). The dallas communities used a spectrum of “co-design” methodologies ranging from 1) “grass-roots” community engagement using creative, participatory co-design methodologies to enable end users to directly shape services ( Table 5 , Q3); 2) HCI technical co-design methodologies that are iterative and contribute to product or tool development via prototyping and refining; and 3) a wider, broad-based, community asset design methodology which involved creative modification of a range of digital tools and services and linking in with pre-existing, large networks.

Table 5:

Illustrative Quotes Relating to Challenge of Co-design at Scale.

Integrated care enabled by techs is welcome	Q1 “ … the new difference is that we will be doing things with people, and in some instances patients will be saying no, that’s not what I want and I think technology can assist in that process, and it’s to be welcomed. NHS is public service … It’s about serving the public. And sometimes organisations… you know, services are wrapped round organisations and not round patients. [….] So I think there is a fundamental shift … and the reforms … the NHS reforms and local authority support that shift, and it’s to be welcomed. It’s long overdue, in my humble opinion .” (C2(b) Clinical – NHS).
Participatory design	Q2 “I think initially […] the industry and technology partners couldn’t really understand why they were engaging with people locally. Why they were engaging with real people, they already had the answers; they already had the product, why are they just not serving it to them. A very traditional if you like industry model of we’ve found a solution and let’s just punt it out there .” (C1(b) Representative, Third Sector organization).
	Q3 “… We are delivering community engagement and co-design so we are going out to talk to people who we hope will benefit from [community name] in… different regions. So we've gone out… to start conversations, in shopping centres, in hospitals, and we've really chatted to people about what they value about their community and themselves, and what they want to do more of… to, kind of, understand what [community name] can do to connect people to the resources that already exist … it’s focusing on the opportunities that are there and people can see that designing around their lifestyles and around their needs, and people-centered services… so designing with them, rather than for them.” (C1(m) Researcher, Academia).
Ambitious recruitment numbers	Q4 “I think they know that the overall, sort of, sign up target for dallas was hugely ambitious. I think also there’s, kind of, what we realized and all the partners have realized is we’ve set… we have set a really high benchmark for ourselves … ” (C3(m) Manager 1 – Business).
	Q5 “So I feel like I’ve been through it, stuck with it through all of that time where we had no idea what it was and kind of been up here in selling it to people without even being able to tell them what it actually is … A key lesson I’ve learned is… wait until you know what it is before you start to engage with people. It was really, really hard last year trying to talk to people and that’s why our e-health department is only just now properly getting engaged because despite lots of conversations I wasn’t able to tell them what we were doing.” C1(m) Manager 2 – NHS).
Co-design and learning	Q6 “So we can make decisions a lot quicker, we can sign off funding a lot quicker than the traditional NHS projects so we have that flexibility … and the speed of decisions and getting things started and the other, big change I see is that the companies we are involved with they tend to be smaller companies … so the NHS very often have very big companies. You know multi-nationals so we have fewer of those here, we have more SME type of companies in this project .” (C1(m) Information Technology, NHS).
Collaboration	Q7 “ … I am the lead contact and […] we are mainly interested in Telehealth deployment but eh, we are also interested in how the dallas projects help us understand how to deploy Assistive Living Technologies in a broader context. So the WSD… was strictly an RCT so eh… So it was very fixed and clear what had to be done, three million lives (3ML) was very commercially driven so the beauty of the dallas project is its collaborative aspect which allows us to be more innovative.” (C2(m) Researcher – Industry).
Community asset based approach	Q8 “But… I think that [community name] approach has been very practical so it’s been don’t reinvent the wheel, if there’s someone who’s already doing it… then get in touch with the person who’s already doing it. So we try to use existing resources and processes and, well, anything that we can rather than start from scratch and say, well, we’re going to develop this big machine and it’s going to do everything for you. We’ve looked to integrate with what’s good out there in [city name] already.” (C2(m) Manager 1 – NHS).
	Q9 “Yes, that’s what [Charity organization name] are leading on. We’ve actually got our own mobile smart house, which we take and set up at events, it’s got four rooms. Now that’s always been, sort of, directed at learning disabled events… and things like that. But, now we’re creating a more generic model in the [name of retail store] in [name of city] which is a big hardware store in the middle of [name of city]. So that’s going to be a similar sort of model but with technologies that are not just aimed at people with learning disabilities. ” (C2(b) Representative – Charity organization).
Traditional user testing	Q10 “One of the biggest lessons for the [community name] project was… understanding the User Acceptance Testing that [name of Company 1]and [name of Company 2] do isn’t sufficient on its own. It needs to come to health for us to test as well because we are testing it as a health professional would use it … or as a (person) would use it, they are testing it from the technical, and so that was a really important lesson to learn. That step has been put into the process now … sometimes it does delay products being released but it prevents any products being released that aren’t fit for purpose.” (C3(m) Manager 1 – NHS).

Such collaborative digital design methods were at first foreign to the technology partners who raised concerns about the time commitment required. In one community, extensive input from end users via face-to-face workshops and “pop-up” events was undertaken to shape all aspects of the service, foster ownership, and ensure the development of a digital health and well-being product that was “fit-for-purpose” ( Table 5 , Q2). However, the nature of iterative, agile, co-design caused a challenge because contractual arrangements with the communities required them to recruit large numbers of users simultaneously, which took time. Target recruitment numbers were perceived as overly ambitious and unrealistic to attain within a fixed 3-year timeframe ( Table 5 , Q4). There were also difficulties in engaging end users with a product undergoing iterative development. This conflicting tension of innovation and recruitment was a concern of all of the communities and seen as a real challenge ( Table 5 , Q4, Q5).

However, there were advantages and learning associated with working in new partnership models involving smaller business partners. These included more flexibility and the opportunity for active collaboration as compared to working with large multi-national companies ( Table 5 , Q6, Q7).

One consortium adopted a community asset based approach to co-design as their means of innovating, through drawing on pre-existing networks and resources ( Table 5 , Q8, Q9). This also allowed the consortium to build on some assisted living technologies which already existed. Some adopted a federated membership model or approach in order to address target recruitment numbers by partnering with pre-existing networks with significant reach in their local community.

Yet another approach to co-design involved more traditional HCI iterative methods with the overall aim of designing fit-for-purpose digital health tools. This partnership involved workshops with end users and service providers with the learning and feedback obtained from prototypes being fed back into the design of the digital health product. This also provided an important learning opportunity about person-centered design with the emergent learning being written in to form the basis of new e-health tool and service design processes ( Table 5 , Q10).

4. Branding and Marketing Challenges in Consumer Healthcare Settings

One of the strategic aims of dallas was to stimulate consumer and retail business models in order to drive innovation and economic growth in the United Kingdom. However, culturally, health is still not usually perceived as a commercial venture in the United Kingdom. All 4 consortia have carried out significant work in building person-centered brands and corporate identities, aligned to more personalized branding. However, challenges emerged—for example, one community discovered their brand was already in use by a pre-existing organization and they had to undergo a very expensive and time-consuming re-branding exercise.

In other communities, challenges existed since the grass-roots, participatory co-design process was time and labor intensive. A significant investment was made in this iterative co-design methodology which involved the end users in all aspects including choosing the colors and visual representation of the brand, resulting in a tailored, unique digital product ( Table 6(A) , Q1).

Table 6:

Illustrative Quotes Related to (A) Branding and Marketing and (B) Interoperability and Information Governance.

(A)	Branding and Marketing
Branding challenges	Q1 “ … we've done the branding work for [community name] and all the different services, so we've been doing that with the communities as well, and the aim of that is to make it feel that it's owned by the community … so that it could be made by the community, and I think the colours that we've used, as well, I think that demonstrated that the brand works, because people were curious about what it was, because … it doesn’t say Health, and I think the fact that it wasn't selling anything was just, that’s just weird. So, let's go in.” (C1(m) Researcher, Academia).
	Q2 “Yes. What’s actually happened is we’ve been dragged down an NHS, you know, service route which is basically it needs to comply with information governance, you know, and we’ve just gone down a vortex of bureaucracy.” (C3(m) Manager 1 – Business).
Digital health brand recognition	Q3 “We’ve got a desire to engage our Creative and Digital sector in the city so that’s small and medium enterprises that is thriving in the city, very much focused on technology and particularly the Creative Arts so Media, Music, Digital Content. They will start to become a Centre of Excellence for the Region, hopefully the UK, possibly the world… and I think the work that we’ve done […] what it’s done is it’s placed this agenda, e-health, assisted living, whatever we’ll call it; it’s really placed it in the eyeline of the Local Enterprise Partnership who now see this as being one of the planks of city region growth. Em it’s taken us a while to get here but we’re here now and they will begin to major in this area.” (C2(m) Manager 2 – NHS).
	Q4 “ We have started to take our experiences from [community name] into our European dimension so… because we have very good links now within the commission and with a range of European projects European partners… industry players and indeed commissioners in some of our partner organisations very interested in what we are doing with [community name] and it aligns very well to some other approaches that are going on in different countries… ” (C1(m) Manager 1 – NHS).
	Q5 “ And I think, if I’m truthful, there’s virtually nobody you speak to at Clinical Commissioning Group now that doesn’t know about the [community name] program and whereas before I think when the [community name] program was first started and even when we were at the dallas bid stage it was like, oh, they didn’t—you know, it’ll never happen, it’ll never happen. And now those same sceptics are now saying, but that’s really good, that, I think we need to .” (C2(m) Manager 3 – NHS).

Another community invested significant time and resource in working toward the launch of a digital consumer version of a traditional health product only to face it not being endorsed by a key regulator ( Table 6(A) , Q2). Despite these challenges and set-backs the communities “ stayed the course ,” and through agility and adaptive learning, have made significant progress toward achieving digital health brand recognition ( Table 6(A) , Q3, Q4). There is now growing European interest and wider recognition of the innovative, digital dallas services and products which provide exemplars of new models of collaborative, partnership working and perseverance in the face of seemingly intractable problems ( Table 6 A, Q5).

5. Facing the Challenges of Interoperability and Information Governance

The dallas program aims to facilitate person-centered, seamless digital healthcare and well-being; a key feature of this is the role of information sharing between services and the user and the need to open up proprietary/statutory IT systems in order to become more interoperable and flexible. One consortium in particular has been working on interoperability in order to open up the market and enable more customized technologies to be introduced that are tailored more closely to local needs. The technology companies believe that the current limited success of digital technologies may be partially related to a lack of customized products that people actually want and which take into account the organizations’ and/ or the end user’s needs, choices, and requirements ( Table 6(B) , Q1).

To achieve this, there is a need to design systems and products that are interoperable, which some traditional suppliers see as a threat since increased competition may result in them losing their market share ( Table 6(B) , Q2). In order to progress this interoperability agenda new guidelines and open architectures are being developed ( Table 6(B) , Q3), as well as the launch of the “Digital Health and Care Alliance” in the United Kingdom. Their aim is to try and reshape the current healthcare landscape to move the field forward from locked down proprietary systems to one of open sharing with digital products working across systems.

The information governance (IG) rules and regulations surrounding patient records, which are required to ensure patient confidentiality and security, also presented challenges. New person-centered, health and well-being digital tools that enable citizens to access and own parts of their personal digital health records also require new IG approaches. Within dallas, one consortium has been working to launch a broad range of digital health and care planning and management tools but are finding a lack of IG that would accommodate such tools ( Table 6(B) , Q4, Q5). There exists fear and a lack of understanding and clarity about security and associated issues of trust surrounding such new interventions ( Table 6(B) , Q6). Thus, IG represents a significant part of the process of trusted implementation that has yet to be addressed and represents a barrier toward implementation at the present time ( Table 6(B) , Q6). Initially, business partners did not fully understand the deeply embedded nature of IG rules in the NHS and its status on sharing information. However, the consortia have contributed to policy discussions and, although not a tangible operationalized product, this work is making new pathways and “in roads” as an important part of the wider dallas implementation processes ( Table 6(B) Q7).

DISCUSSION

This article communicates key challenges and lessons learned across dallas, a large-scale, national, multi-agency, and multi-site deployment of a wide range of digital technologies for the promotion of health and well-being in the United Kingdom. Importantly, we report on the implementation challenges faced when rolling out a broad portfolio of digital tools and services nationally at scale and at pace (see table in Supplementary Appendix ) as opposed to previous studies which describe implementation lessons arising from individual interventions (e.g., telecare or telehealth or electronic medical record implementations). We have highlighted 5 key challenges: 1) establishing and maintaining large heterogeneous, multi-agency partnerships in the consortia; 2) the need for resilience in the face of barriers and set-backs including continually changing external environments; 3) the inherent tension between embracing innovative co-design and achieving delivery at pace and scale; 4) the effects of branding and marketing issues in healthcare settings; and 5) the challenge surrounding interoperability and IG, when commercial proprietary models dominate. These challenges generate a valuable evidence base about issues for consideration when embarking on any large scale digital health or well-being deployment. Key lessons for consideration include:

Successful, multi-agency partnership working requires robust management, excellent continual communication, and time to achieve coherence in order to influence health and care models.

The importance of resilience when embracing real risk in order to support and enable healthcare innovation.

The ability to navigate complex socio-technical change against a backdrop of challenging wider uncertainty.

The benefits of capitalizing on the opportunity to innovate locally with communities and implement person-centered design at scale.

How to build consumer-facing life enhancing health technologies and enhance digital health brand recognition.

The benefits, but difficulties in practice, of advancing interoperability and IG agendas.

The importance of brand trust and confidence as well as intervening and promoting at the right time and place—and with the right people—to increase meaningful uptake of digital healthcare services.

Mechanisms for innovating can be important for generating a sense of coherence across heterogeneous stakeholders, to facilitate traction in this emergent field.

Some of the lessons learned across the dallas program align closely with work reported by Cresswell et al., especially the importance of building consensus, which relates to issues raised in working in large, multi-agency partnerships; some aspects of infrastructure, particularly interoperability; the importance of maintenance, which was a constant feature across the dallas program and noteworthy in the work undertaken to maintain consortia; and finally, the importance of “ staying the course ” which has been a clear and successful feature of the dallas program. Furthermore, additional insights involve the need for agility in service re-design and adaptive learning to overcome seemingly intractable problems related to the wider socio-economic and political environment. The management of organizational change literature emphasizes the agency of people as a key factor influencing the implementation of change. Technologies can be enablers of organizational change but only if the surrounding socio-technical factors are taken into account and the dallas program has faced significant challenges posed by organizational restructuring and economic austerity. Our findings also resonate with current studies which recognize that a lack of integration and interoperability across traditional services is not conducive to flexible, joined up healthcare provision. The importance of flexibility and adaptability and an iterative, agile approach to both development of digital systems and the implementation strategy highlighted here resonates with previous reports concerning national deployments of electronic medical record systems.

Since the locus of healthcare is shifting to the home and community setting, there is an increasing need to adopt a broader approach across the traditional boundaries of health and social care in order to operationalize a more integrated and personalized healthcare service provision. Indeed, May et al. have called for the need to form new partnerships across a diverse range of healthcare communities and to include nongovernmental, third sector and voluntary organizations in order to harness multiple skillsets and localized knowledge to deliver more appropriate digital models of healthcare and well-being. Here we have demonstrated the importance of building coherence and cognitive participation feedback loops across consortia in order to sustain engagement. Other lessons learned relate to the nuanced, yet crucial, shifts in shared understanding (coherence) between public and private sectors with one of the shared goals being the need to enhance interoperability.

One of the major challenges reported here was the need to innovate and recruit at the same time. Nonetheless, the number of users, as reported by the communities (in January 2015) was 24 588. However, importantly, the dallas program has resulted in the development and deployment of a wide range of digital tools and services across the United Kingdom (see table in Supplementary Appendix ) with associated wider impact. These include a national digital health and well-being portal, which represents a new suite of interactive, web-based tools that can be personalized to each individual user; an electronic personal health record which has been endorsed by a key Regulatory body in the United Kingdom; and a consortium whose region is now recognized as a European Reference Site for innovation in digital healthcare.

Developing digital health and well-being products cognizant of users’ needs that also had trusted brand recognition highlighted crucial differences in approaches to product design between the two principal communities of healthcare and digital technology. The dallas consortia aimed to innovate in the area of consumer-facing healthcare and well-being digital tools, resources, and services, which is a lucrative area of market growth. However, the program has shown how risks can manifest into reality and how difficult it can be in identifying the best “routes to market.” This illustrates the perils of forging new routes to facilitate change within complex ecosystems when people and systems are not necessarily ready to change at equal pace.

Interoperability is needed to facilitate data and information sharing in alignment with more integrated, personalized healthcare and well-being provision but there is resistance from statutory suppliers who have dominated the market and which can be a barrier to innovation. Person-centered, digital healthcare, and wellness records requires not only interoperable systems but also “real time” access to records. In the United Kingdom, the IG legislation is historical and deeply embedded in a culture of high security and confidentiality, with the concept of sharing still foreign. Experience from dallas suggests that the healthcare and well-being community would welcome better integration of health records but with some caution, perhaps due to the lack of legislation and system readiness for such change.

Strengths and Limitations

This study has a number of strengths and limitations. We have addressed the implementation processes and systems within dallas by drawing on NPT, ,, which served as a socio-technical analytical lens to help us analyze the implementation processes and emergent learning across the dallas program, and which is considered good practice by those examining implementation issues in the sphere of digital health. We have also used “data coding clinics” to ensure the validity and robustness of our coding framework and we have drawn on data from multiple different sources to enhance confidence in our findings. However, the dallas program is large and diffuse and the evaluation data presented here has focused primarily on macro and meso-level implementation issues and the perspectives of key implementers, with less information gained from professionals “at the coalface.” In addition, we provide no data on the views of users of dallas services or products. Our use of theory to inform our coding framework may raise concerns that we “shoehorned” data to fit the framework or were unnecessarily constrained by the theory. However, we explicitly looked for data that fell outside the framework and did not exclude such data in order to conduct a rigorous and meaningful analysis of the implementation processes. Finally, while we describe here a national deployment, the work was undertaken across only 2 countries, Scotland and England, which both operate a system of free healthcare at the point of delivery. While this may be viewed as an additional limitation, we would contend that the issues we have raised and the resultant generic learning have widespread, international applicability.

Considerations for Future Studies

The present study suggests three key areas that should be addressed for future large-scale implementation of digital healthcare tools and services: 1) For a program of this scale, there should be a longer timeline between signing the contract to program initiation and a minimum 5-year timeline (5–10 year plan ideally) for the overall program of innovation at scale; 2) There should be significant time invested in assessing the digital readiness of the local health economies and a greater degree of intelligence gathering across partners before embarking on innovation at scale, and; 3) There needs to be greater attention paid to the current status of IG (and lack of interoperability) which still represents a barrier to the meaningful deployment of innovative digital healthcare services at scale.

Conclusions

As challenges have been overcome and alternative “routes” or pathways forged, dallas has gained momentum within each community and across the overall program, representing a significant network of expertise that is building capacity in this new interdisciplinary field. As far as we are aware, it is also one of the first programs in the world to undertake such large-scale digital health interventions and implementation, providing new evidence about creative partnership models, integrating new digital services, innovating, co-designing and delivering at scale , and navigating socio-technical change. Therefore, in conclusion, the identification of the key challenges in this unique program—and the mapping of the resultant solutions—provides rich learning that will benefit both future evaluation capacity and real world implementation of digital health and well-being at scale.

Click here for additional data file.