Using stakeholder perspectives to develop an ePrescribing toolkit for NHS Hospitals: a questionnaire study.

OBJECTIVE: To evaluate how an online toolkit may support ePrescribing deployments in National Health Service hospitals, by assessing the type of knowledge-based resources currently sought by key stakeholders.
DESIGN: Questionnaire-based survey of attendees at a national ePrescribing symposium.
SETTING: 2013 National ePrescribing Symposium in London, UK. PARTICIPANTS: Eighty-four delegates were eligible for inclusion in the survey, of whom 70 completed and returned the questionnaire. MAIN OUTCOME MEASURES: Estimate of the usefulness and type of content to be included in an ePrescribing toolkit.
RESULTS: Interest in a toolkit designed to support the implementation and use of ePrescribing systems was high (n = 64; 91.4%). As could be expected given the current dearth of such a resource, few respondents (n = 2; 2.9%) had access or used an ePrescribing toolkit at the time of the survey. Anticipated users for the toolkit included implementation (n = 62; 88.6%) and information technology (n = 61; 87.1%) teams, pharmacists (n = 61; 87.1%), doctors (n = 58; 82.9%) and nurses (n = 56; 80.0%). Summary guidance for every stage of the implementation (n = 48; 68.6%), planning and monitoring tools (n = 47; 67.1%) and case studies of hospitals' experiences (n = 45; 64.3%) were considered the most useful types of content.
CONCLUSIONS: There is a clear need for reliable and up-to-date knowledge to support ePrescribing system deployments and longer term use. The findings highlight how a toolkit may become a useful instrument for the management of knowledge in the field, not least by allowing the exchange of ideas and shared learning.

Introduction

Despite growing interest in and financial support for the deployment of ePrescribing systems[1-4] that offer varying degrees of functionality in the supply, administration, recording and prescribing of medication,[4] there is relatively little experience of implementing and using these systems in National Health Service (NHS) hospitals in the UK.[4-6] The recently announced NHS ‘Safer Wards, Safer Hospitals Technology Fund’[7] marks the beginning of a £1 billion investment in Information Technology (IT) over the next three years by the UK government and NHS organisations and provides strong evidence of the policy drive to support the establishment of health IT systems across NHS hospitals. This push for the digitisation of NHS hospitals has been driven by expectations that the introduction of new IT systems, or eHealth systems, may increase safety,[8-10] organisational efficiency in the delivery of care and medication,[11] as well as improved communication,[2,12] both within the hospital and between primary and secondary care settings. Efforts to establish paper-light environments across the NHS risk, however, being hampered by the lack of experience and knowledge of implementing eHealth systems in NHS hospitals.[6,13] Appropriate knowledge-based support may go some way to addressing the immaturity of the UK eHealth landscape. The NHS Connecting for Health Programme has until recently provided an authoritative online resource to help promote and deliver eHealth services across the NHS,[14] including ePrescribing. Many of these resources have now migrated to the Health and Social Care Information system portal,[15] a gateway to a wider range of health informatics-related data and material. The availability and use of such knowledge is essential to support an organisation’s ability to change and innovate.[16,17] It also facilitates the transfer and exchange of knowledge between research and practice and is a key underpinning of successful eHealth deployments.[18,19]

In line with such understandings, the National Institute for Health Research (NIHR) has commissioned us to undertake a programme of research evaluating the implementation and adoption of ePrescribing systems in English hospitals. Part of this has included the development of a toolkit[13] that reflects the various stages and associated needs of users to successfully deploy, use and integrate a hospital ePrescribing system. Building on the preliminary recommendations made in earlier work,[13] a study was designed to guide the subsequent toolkit prototype development. Since it is now well established that user involvement and participatory design are absolute requisites in the planning and design of user-friendly learning technologies and resources,[20,21] the study sought to map participants’ information and knowledge needs in relation to ePrescribing implementation and adoption, in order to support the toolkit’s usability. We report here on findings from this study.

Methods

Study design

The National Symposium for the Health Service on deriving value from ePrescribing in hospitals held in June 2013 at the Royal Society in London, UK, was selected as the setting for the study, as it brought together key stakeholders, including delegates from 46 different Hospital Trusts, 12 commercial organisations and eight policy and government bodies as well as academic researchers. A questionnaire-based survey was chosen as a relatively efficient data collection method within the limited time available in a conference setting.

Developing the questionnaire

The content and focus of the questionnaire (see Appendix 1) were informed by the needs of users as defined by the literature[1-13] as well as the research team’s own experience as researchers and clinicians and contained nine numbered questions (closed- and open-ended) designed to investigate respondents’ needs at different stages of the deployment of an ePrescribing system. Some questions contained multiple sub-questions to concentrate on areas of interest, providing up to 41 data items per respondent and covering the following areas: These areas were targeted: first, to provide basic background information on respondents, thus allowing us to compare the sample to the wider ePrescribing stakeholder base; second, to evaluate areas of need identified in research elsewhere;[13] and third, to explore pathways to better utilisation of knowledge.[16-19] At the end of the questionnaire, respondents were also encouraged to provide free text comments.

Respondents’ role/place of work.

Current access to a toolkit to support ePrescribing implementation and use, and interest in having access to an ePrescribing toolkit.

Audience, type of content to include and user support required for an ePrescribing toolkit.

Knowledge, perceived usefulness and usability of other toolkits in existence in the NHS.

The questionnaire was developed and piloted with seven members of the NIHR ePrescribing Programme team, which included doctors, pharmacists, social scientists and a Patient Public Involvement (PPI) group. This process allowed for increasing refinement of the questionnaire through feedback and testing.

Sampling and data collection

The questionnaire was distributed to all the delegates (n = 99) who attended the National ePrescribing Symposium by placing one copy of the questionnaire in each delegate pack. The questionnaires were numbered serially with a unique reference number to track response rate and facilitate the exclusion of individuals associated with organising the event. Only delegates external to the ePrescribing Research team were eligible for inclusion in the survey, providing a sample size of n = 84. This exclusion was applied to ensure that new insights could be collated which were not guided by the Research team’s existing knowledge and to minimise the risks of introducing biases in the findings given that the toolkit is a major output from the Research Programme. No other exclusion criteria were applied.

The questionnaire was self-administered to allow for a greater rate of participation within the limited one-day timeframe of the symposium and to maintain as much confidentiality and anonymity as possible within a public setting. Time was provided at the end of the opening keynote session of the conference for delegates to complete the questionnaire, and the majority was handed in to the research team at that point. Other opportunities throughout the course of the day were also available (i.e. during refreshment intervals). Reminders to complete and return the questionnaire were made in person by the research team and via posters at the conference. In two cases, the questionnaire was returned by post after the event.

Ethical considerations

All records of individual names or contact details were removed from the dataset prior to analysis to ensure that all responses remained anonymous. This involved removing identifiable information including place of work, name, email address or other contact details. Participation in the survey was completely voluntary and no incentives were used. The Programme of Work has been reviewed by and received ethical approval from the Centre for Population Health Sciences Ethics Review Group at the University of Edinburgh.

Data handling and analysis

The data collected were entered into a Microsoft Access database (Microsoft Corporation, Washington, USA) and identifiers removed. Fully anonymised data were then exported for analysis in Minitab 16 Statistical Software (Minitab Inc., Pennsylvania, USA), and descriptive statistics were obtained. The questionnaire contained only a small number of open-ended questions. Answers to these were either quantified through frequency counts (i.e. when respondents were asked for details of other toolkits they used c.f. Q.3 and Q.8) or coded manually and organised by type (e.g. for questions asking about ‘other categories’ not listed, c.f. Q.5 and Q.6). Additional comments obtained through free text were coded manually and organised by theme.

Results

Sample

Seventy completed valid questionnaires were returned, providing a response rate of 83.3%. An initial analysis of the total sample revealed that pharmacists represented the majority of respondents (n = 38; 54.3%). Other groups of participants included NHS Managers, Policy, Research and Development (n = 9; 12.9%), IT staff (n = 8; 11.4%), doctors (n = 6; 8.6%) and suppliers (n = 5; 7.1%).

Access, use and interest in an ePrescribing toolkit

Only a very small number of delegates reported having access to a toolkit specifically designed to support the implementation and adoption of ePrescribing systems (n = 2; 2.9%). The only two toolkits reported were a ‘home-grown’ product (n = 1), developed within the hospital where it is currently used, and the NHS Connecting for Health website (n = 1). It is worth pointing out that NHS Connecting for Health was also cited in responses obtained elsewhere in the questionnaire in relation to use and access to NHS relevant toolkits that are not solely dedicated to ePrescribing (see ‘Experience of existing toolkits used in the NHS’ section). Almost all the participants (n = 64; 91.4%) indicated their interest in having access to an ePrescribing toolkit, with a small proportion being undecided (n = 5; 7.1%).

Most respondents agreed that the ePrescribing toolkit should be aimed at the implementation (n = 62; 88.6%) and IT teams (n = 61; 87.1%), pharmacists (n = 61; 87.1%), doctors (n = 58; 82.9%) and nurses (n = 56; 80.0%). Far fewer respondents felt that it would be useful for those in overarching managerial roles such as the Trust Board (n = 47; 67.1%) or NHS managers more generally (n = 45; 64.3%) to have access to the toolkit. Other potential users, who were cited via free text, included allied health professionals (n = 9), finance and administration staff (n = 5) and patients and patient groups (n = 4).

Almost two-thirds of respondents stated that they would value hands-on expert support in using the toolkit to support local needs and practices (n = 45; 64.3%), with almost a third (n = 22; 31.4%) undecided.

Content of the ePrescribing toolkit

Participants were asked to rate a list of content types on a scale ranging from ‘very useful’ to ‘not useful at all’ (see Table 1). The results indicated that a range of practical tools were needed to support an ePrescribing implementation. There was also considerable interest among the participants in discussing issues and sharing ideas and knowledge on ePrescribing through case studies and via a discussion forum, rather than a social media platform. A small number of delegates provided details in free text of other specific areas of interests for inclusion in the toolkit, such as the integration of ePrescribing systems into electronic health records (EHRs), standards, benefits and impact, conferences and meetings, how to conduct site visits, as well as guidance aimed specifically at the hospital board on ‘how not to implement’.

Table 1.

Proposed content type and perceived usefulness.

Type of material	Tools and checklists	Discussion forum	FAQs	Templates	Reference materials and publications	Social media platform	Overview of deployment of systems	Case studies	‘How to’ guides
Very useful	47 (67.1%)	29 (41.4%)	37 (52.9%)	43 (61.4%)	29 (41.4%)	9 (12.9%)	35 (50.0%)	45 (64.3%)	48 (68.6%)
Useful	21 (30.0%)	33 (47.1%)	28 (40.0%)	24 (34.3%)	32 (45.7%)	38 (54.3%)	28 (40.0%)	24 (34.3%)	20 (28.6%)
Neither useful or not useful	0	6 (8.6%)	4 (5.7%)	2 (2.9%)	2 (2.9%)	14 (20.0%)	6 (8.6%)	1 (1.4%)	0
Not useful	0	0	0	0	0	7 (10.0%)	0	0	0
Not useful at all	1 (1.4%)	0	1 (1.4%)	1 (1.4%)	1 (1.4%)	1 (1.4%)	1 (1.4%)	0	0
No response	1 (1.4%)	2 (2.9%)	0	0	6 (8.6%)	1 (1.4%)	0	0	2 (2.9%)
Total (N)	70	70	70	70	70	70	70	70	70

Experience of existing toolkits used in the NHS

Delegates’ awareness of the availability of tools and toolkits covering any area relevant to NHS staff was markedly lower than levels of interest in an ePrescribing specific toolkit. Twenty-eight participants (40.0%) acknowledged the existence of NHS relevant toolkits, while the majority (n = 41; 58.6%) either did not know (n = 23; 32.9%) or was unsure (n = 18; 25.7%) if such tools or toolkits were in existence (see Table 2).

Table 2.

NHS relevant toolkit awareness, interest and current access to an ePrescribing toolkit among participants.

	Awareness of NHS relevant toolkits	Interest in ePrescribing toolkit	Current access to a toolkit to support ePrescribing
Yes	28 (40.0%)	64 (91.4%)	2 (2.9%)
No	23 (32.9%)	1 (1.4%)	54 (77.1%)
Not sure	18 (25.7%)	5 (7.1%)	13 (18.6%)
No answer	1 (1.4%)	0	1 (1.4%)
Total (N)	70	70	70

The tools and toolkits in existence and/or used by the respondents covered eHealth, medicines management, hospital safety and efficiency, as well as patient engagement or interaction (see Table 3). The most frequently cited toolkit was NHS Connecting for Health (n = 7), followed by the Productive Ward Toolkit (n = 3) and Homecare (n = 3). As well as having the highest frequency of citation, the NHS Connecting for Health toolkit was viewed favourably by delegates who had used it, with ease of use (8/10), quality of content (7.8/10) and breadth and depth of its coverage (7/10) all obtaining high scores.

Table 3.

Toolkits cited and frequency (counts).

Toolkit Name/Area of specialisation	Frequency of citation
Agency for Healthcare  Research & Quality (AHRQ)	1
Connecting For Health (CfH)	7
Chemocare	1
Choose & Book	1
Dashboard Toolkit	1
Define	1
ePACT	1
Homecare	3
Infection Control Risk Assessment	1
Information Governance Toolkit	2
Keele	1
Medicines Best Practice	1
Mental Health Medicines Toolkit	1
Method M (Cerner)	1
National Institute for Clinical  Excellence (NICE)	1
Non-Medical Prescribing Toolkit	1
Nursing Toolkit	1
Patient Information	1
Patient Safety Assessment (PSA)	1
Pharmacy Automation Toolkit	2
Productive Ward	3

Knowledge management and knowledge transfer

The space provided for additional comments offered interesting insights into the issues relating to ePrescribing knowledge management and knowledge transfer. First, it was noted that ‘learning from others’ was key to avoiding ‘re-inventing the wheel’ or ‘repeating the same mistakes’ and could be formalised through a mentoring programme to allow ‘buddying’ of less experienced hospitals. Second, it was suggested that supporting local needs through the use of a toolkit may be complicated by an aspiration to provide generic support to all hospitals in England, aiming to implement an ePrescribing system. This tension between a designed-for-all toolkit versus one addressing local needs was further echoed in requests for content to be tailored to specific areas, sectors and specialities. Third, there was demand for a toolkit that is up-to-date in its content and application. Novel uses, including interactive and visual capabilities, such as in situ simulation training or videos to provide visual support, were cited. Constantly updated material relating to technical and infrastructure guidance, as well as evaluation and audit requirements, were also highlighted as particularly important areas.

Discussion

The findings from this study suggest a high degree of interest in developing a toolkit to support the roll-out of ePrescribing systems in NHS hospitals. This should be of little surprise given the relatively nascent state of ePrescribing deployments in NHS hospitals and the patchy knowledge and unproven track-record of many hospitals implementing and adopting ePrescribing systems. The complexity and lengthy timescales needed to implement EHRs and associated challenges[6,22] clearly align stakeholders’ needs to those of the aims of the toolkit. We should also be mindful when interpreting these findings of the study’s limitations, notably the use of a convenience sample that had a relative over-representation of pharmacists (even if not atypical for such a study given this group’s early involvement in ePrescribing projects)[4,23] as well as the low levels of use of existing ePrescribing support websites reported by the respondents. That said, we believe representatives from the majority of hospitals who are either planning to implement or have implemented ePrescribing systems attended this national event. Furthermore, the limited range of participants is a reminder of the need and opportunity for the toolkit to act as a vehicle for wider engagement with all those involved in the use of ePrescribing from patients right through to the Trust Board.

The study’s findings underline the demand for a toolkit that provides highly practical resources to help plan ePrescribing deployments as well as support their continued use, to a broad base of users. Such findings mirror the calls made by MacFarlane et al.[19] who conclude in their work that there is indeed an immediate need to address ‘the translational gap by developing tools … to promote implementation … in the field of eHealth’. The considerable interest in ensuring the toolkit can become both a practical resource centre, as well as a repository for ePrescribing knowledge and research in the UK and internationally, may go some way to help achieve this, by supporting improved knowledge exchange[16-18] in the field. Notwithstanding such aims, the study demonstrates how content is only one aspect of this exchange of knowledge: how ideas and expertise are shared and used matters too. The real value of the toolkit will thus come from its actual use, and further research is needed to evaluate how valuable this toolkit will become once in use, and whether support mechanisms such as mentoring could further promote applying usefully expert knowledge.

Conclusions

The toolkit’s potential role in helping address the knowledge gap facing hospitals in their deployment of ePrescribing systems is significant. This study has arguably provided a better understanding of users’ needs and aspirations for the toolkit which will enable it to offer a suitable range of resources to facilitate the introduction and longer term integration of ePrescribing systems. While helpful for the further elaboration of the tools and content of the toolkit itself, this is only a first step. Indeed the online and publicly accessible prototype of the ePrescribing toolkit[24] that this work has supported will now need to be further evaluated and critiqued by its users. It also remains essential for future work in this area to involve a broader base of ePrescribing knowledge users than those surveyed in this study. More attention will need to be paid in particular to groups, such as patients, whose knowledge needs have not been clearly addressed so far, and to establishing the best ways to reach out and engage with those who are at present not closely involved in the implementation and adoption of ePrescribing systems.

IT Team	□
Implementation Team	□
Trust Board	□
Doctors	□
Nurses	□
Pharmacists	□
Managers	□
Others – Please specify  …  …  …  …  …  …  …  …  …  …  …  …   …  …  …  …  …  …  …  …  …  …  …  …   …  …  …  …  …  …  …  …  …  …  …  …	□

‘How to’ guides that explain key considerations at every stage of the implementation, whether it is when writing a business case, going live, upgrading or maintaining a system	Very useful	Useful	Neither useful or not useful	Not useful	Not useful at all
Case studies highlighting how other hospitals have implemented their systems, what issues they have dealt with, and the lessons they have learnt	Very useful	Useful	Neither useful or not useful	Not useful	Not useful at all
Tools and checklists that allow to estimate the costs of implementing a system, facilitate workflow mapping, support staff training plans, evaluate safety and other benefits, outline potential pitfalls and monitor how users feel about the system	Very useful	Useful	Neither useful or not useful	Not useful	Not useful at all
An overview of the ePrescribing and Medicines Administration systems available in the UK and where they have been deployed	Very useful	Useful	Neither useful or not useful	Not useful	Not useful at all
A social media platform where suppliers can present their products and hospitals can learn about products on offer and ask questions directly to suppliers	Very useful	Useful	Neither useful or not useful	Not useful	Not useful at all
A discussion forum where individuals can share ideas and exchange knowledge relating to ePrescribing and Medicines Administration implementation within and across hospitals	Very useful	Useful	Neither useful or not useful	Not useful	Not useful at all
FAQs on system types, implementation strategies and adoption processes	Very useful	Useful	Neither useful or not useful	Not useful	Not useful at all
Templates that can be used when drafting key documents and to support the planning and management of the project	Very useful	Useful	Neither useful or not useful	Not useful	Not useful at all
Reference materials and publications	Very useful	Useful	Neither useful or not useful	Not useful	Not useful at all
Other - Please specify	Very useful	Useful	Neither useful or not useful	Not useful	Not useful at all
…  …  …  …  …  …  …  …  …  …    …  …  …  …  …  …  …  …  …  …    …  …  …  …  …  …  …  …  …  …

Tool/Toolkit name or purpose	Ease of use (e.g. easy to locate relevant information, page layout, overall design)	Quality of content (e.g. reliable, relevant, useful, informative)	Content Coverage (e.g width and depth of coverage)
1.
2.
3.
4.
5.