Matthew T Neame1, Gerri Sefton2, Matthew Roberts3, David Harkness3, Ian P Sinha3, Daniel B Hawcutt4. 1. Department of Information Technology, Alder Hey Children's Hospital, Liverpool, UK; Department of Women's and Children's Health, Institute of Translational Medicine, University of Liverpool, UK. 2. Department of Women's and Children's Health, Institute of Translational Medicine, University of Liverpool, UK. 3. Department of Respiratory Medicine, Alder Hey Children's Hospital, Liverpool, UK. 4. Department of Women's and Children's Health, Institute of Translational Medicine, University of Liverpool, UK; NIHR Alder Hey Clinical Research Facility, Liverpool, UK. Electronic address: Dhawcutt@liverpool.ac.uk.
Abstract
OBJECTIVE: Evaluating Health Information Technologies (HITs) can be challenging, but studies are necessary so that the most beneficial interventions can be identified. Our objective was to systematically review the available recommendations for improving the methods used in HIT evaluations. METHODS: HIT evaluation frameworks were identified from database (MEDLINE, EMBASE, CINAHL) and grey literature searches. Outcome measures included framework recommendations and characteristics. Recommendations were coded and organised using thematic analysis methods. A scoring instrument was used to measure framework quality. RESULTS: The search identified 23 frameworks and 272 recommendations. These were organised into five evaluation domains and 42 themes. The themes included recommendations for improving the evaluation of technical aspects of HITs (e.g. describing aspects of HIT functionality) and suggestions for improving the evaluation of complex factors that may influence the overall effects of HITs (e.g. careful reporting of whether the HIT became integrated into existing working patterns). The frameworks were not generally developed in association with healthcare professionals, or with input from patients. The frameworks tended not to have been developed using systematic methods designed to reduce the risk of bias. DISCUSSION: HIT evaluations are important but they are challenging to conduct and appraise. This review was conducted using systematic methods enabling the organisation of framework recommendations into key themes. These findings may help investigators to successfully plan, conduct and appraise HIT evaluations. The quality appraisal demonstrated that HIT evaluation research may be improved by using more systematic methods and the involvement of participants from a range of differing backgrounds.
OBJECTIVE: Evaluating Health Information Technologies (HITs) can be challenging, but studies are necessary so that the most beneficial interventions can be identified. Our objective was to systematically review the available recommendations for improving the methods used in HIT evaluations. METHODS:HIT evaluation frameworks were identified from database (MEDLINE, EMBASE, CINAHL) and grey literature searches. Outcome measures included framework recommendations and characteristics. Recommendations were coded and organised using thematic analysis methods. A scoring instrument was used to measure framework quality. RESULTS: The search identified 23 frameworks and 272 recommendations. These were organised into five evaluation domains and 42 themes. The themes included recommendations for improving the evaluation of technical aspects of HITs (e.g. describing aspects of HIT functionality) and suggestions for improving the evaluation of complex factors that may influence the overall effects of HITs (e.g. careful reporting of whether the HIT became integrated into existing working patterns). The frameworks were not generally developed in association with healthcare professionals, or with input from patients. The frameworks tended not to have been developed using systematic methods designed to reduce the risk of bias. DISCUSSION: HIT evaluations are important but they are challenging to conduct and appraise. This review was conducted using systematic methods enabling the organisation of framework recommendations into key themes. These findings may help investigators to successfully plan, conduct and appraise HIT evaluations. The quality appraisal demonstrated that HIT evaluation research may be improved by using more systematic methods and the involvement of participants from a range of differing backgrounds.
Authors: Polina V Kukhareva; Charlene Weir; Guilherme Del Fiol; Gregory A Aarons; Teresa Y Taft; Chelsey R Schlechter; Thomas J Reese; Rebecca L Curran; Claude Nanjo; Damian Borbolla; Catherine J Staes; Keaton L Morgan; Heidi S Kramer; Carole H Stipelman; Julie H Shakib; Michael C Flynn; Kensaku Kawamoto Journal: J Biomed Inform Date: 2022-02-12 Impact factor: 6.317
Authors: Matthew T Neame; David Reilly; Ajmal Puthiyaveetil; Liza McCann; Kamran Mahmood; Beverley Almeida; Clare E Pain; Victoria Furfie; Andrew G Cleary Journal: Rheumatol Adv Pract Date: 2022-08-17