Matthew P Horning1, Charles B Delahunt2,3, Christine M Bachman2, Jennifer Luchavez4, Christian Luna4, Liming Hu2, Mayoore S Jaiswal5, Clay M Thompson6, Sourabh Kulhare2, Samantha Janko7, Benjamin K Wilson2, Travis Ostbye2, Martha Mehanian2, Roman Gebrehiwot5, Grace Yun5, David Bell8, Stephane Proux9, Jane Y Carter10, Wellington Oyibo11, Dionicia Gamboa12, Mehul Dhorda13, Ranitha Vongpromek14, Peter L Chiodini15, Bernhards Ogutu16, Earl G Long17, Kyaw Tun18, Thomas R Burkot19, Ken Lilley20, Courosh Mehanian2. 1. Global Health Labs (formerly at Intellectual Ventures Laboratory/Global Good), 14360 SE Eastgate Way, Bellevue, WA, 98007, USA. matthew.horning@ghlabs.org. 2. Global Health Labs (formerly at Intellectual Ventures Laboratory/Global Good), 14360 SE Eastgate Way, Bellevue, WA, 98007, USA. 3. Applied Math Department, University of Washington, Seattle, WA, 98195, USA. 4. Research Institute for Tropical Medicine, Muntinlupa, Philippines. 5. formerly Intellectual Ventures Laboratory, 3150 139th AVE SE, Bellevue, WA, 98005, USA. 6. Creative Creek LLC, Camano Island, WA, USA. 7. Arizona State University, Tempe, AZ, USA. 8. Independent Consultant, Issaquah, WA, USA. 9. Shoklo Malaria Research Unit, Mahidol Oxford Tropical Medicine Research Unit, Faculty of Tropical Medicine, Mahidol University, Mae Sot, Thailand. 10. Amref Health Africa, Nairobi, Kenya. 11. University of Lagos, Lagos, Nigeria. 12. Laboratorios de Investigacion y Desarrollo, Facultad de Ciencias y Filosofia, Universidad Peruana Cayetano Heredia, Lima, Peru. 13. World Wide Antimalarial Resistance Network and Mahidol-Oxford Tropical Medicine Research Unit, Bangkok, Thailand. 14. Infectious Diseases Data Observatory and World Wide Antimalarial Resistance Network, Asia- Pacific Regional Centre, Bangkok, Thailand. 15. Hospital for Tropical Diseases and the London School of Hygiene and Tropical Medicine, London, UK. 16. Kenya Medical Research Institute, Nairobi, Kenya. 17. Centers for Disease Control and Prevention, Atlanta, GA, USA. 18. Defence Services Medical Academy, Mingaladon, Myanmar. 19. Australian Institute of Tropical Health and Medicine, James Cook University, Cairns, Australia. 20. Australian Defence Force Malaria and Infectious Disease Institute, Enoggera, Australia.
Abstract
BACKGROUND: Manual microscopy remains a widely-used tool for malaria diagnosis and clinical studies, but it has inconsistent quality in the field due to variability in training and field practices. Automated diagnostic systems based on machine learning hold promise to improve quality and reproducibility of field microscopy. The World Health Organization (WHO) has designed a 55-slide set (WHO 55) for their External Competence Assessment of Malaria Microscopists (ECAMM) programme, which can also serve as a valuable benchmark for automated systems. The performance of a fully-automated malaria diagnostic system, EasyScan GO, on a WHO 55 slide set was evaluated. METHODS: The WHO 55 slide set is designed to evaluate microscopist competence in three areas of malaria diagnosis using Giemsa-stained blood films, focused on crucial field needs: malaria parasite detection, malaria parasite species identification (ID), and malaria parasite quantitation. The EasyScan GO is a fully-automated system that combines scanning of Giemsa-stained blood films with assessment algorithms to deliver malaria diagnoses. This system was tested on a WHO 55 slide set. RESULTS: The EasyScan GO achieved 94.3 % detection accuracy, 82.9 % species ID accuracy, and 50 % quantitation accuracy, corresponding to WHO microscopy competence Levels 1, 2, and 1, respectively. This is, to our knowledge, the best performance of a fully-automated system on a WHO 55 set. CONCLUSIONS: EasyScan GO's expert ratings in detection and quantitation on the WHO 55 slide set point towards its potential value in drug efficacy use-cases, as well as in some case management situations with less stringent species ID needs. Improved runtime may enable use in general case management settings.
BACKGROUND: Manual microscopy remains a widely-used tool for malaria diagnosis and clinical studies, but it has inconsistent quality in the field due to variability in training and field practices. Automated diagnostic systems based on machine learning hold promise to improve quality and reproducibility of field microscopy. The World Health Organization (WHO) has designed a 55-slide set (WHO 55) for their External Competence Assessment of Malaria Microscopists (ECAMM) programme, which can also serve as a valuable benchmark for automated systems. The performance of a fully-automated malaria diagnostic system, EasyScan GO, on a WHO 55 slide set was evaluated. METHODS: The WHO 55 slide set is designed to evaluate microscopist competence in three areas of malaria diagnosis using Giemsa-stained blood films, focused on crucial field needs: malaria parasite detection, malaria parasite species identification (ID), and malaria parasite quantitation. The EasyScan GO is a fully-automated system that combines scanning of Giemsa-stained blood films with assessment algorithms to deliver malaria diagnoses. This system was tested on a WHO 55 slide set. RESULTS: The EasyScan GO achieved 94.3 % detection accuracy, 82.9 % species ID accuracy, and 50 % quantitation accuracy, corresponding to WHO microscopy competence Levels 1, 2, and 1, respectively. This is, to our knowledge, the best performance of a fully-automated system on a WHO 55 set. CONCLUSIONS: EasyScan GO's expert ratings in detection and quantitation on the WHO 55 slide set point towards its potential value in drug efficacy use-cases, as well as in some case management situations with less stringent species ID needs. Improved runtime may enable use in general case management settings.
Entities:
Keywords:
Automated diagnosis; Machine learning; Malaria; Microscopy; WHO
Authors: Debashish Das; Ranitha Vongpromek; Thanawat Assawariyathipat; Ketsanee Srinamon; Kalynn Kennon; Kasia Stepniewska; Aniruddha Ghose; Abdullah Abu Sayeed; M Abul Faiz; Rebeca Linhares Abreu Netto; Andre Siqueira; Serge R Yerbanga; Jean Bosco Ouédraogo; James J Callery; Thomas J Peto; Rupam Tripura; Felix Koukouikila-Koussounda; Francine Ntoumi; John Michael Ong'echa; Bernhards Ogutu; Prakash Ghimire; Jutta Marfurt; Benedikt Ley; Amadou Seck; Magatte Ndiaye; Bhavani Moodley; Lisa Ming Sun; Laypaw Archasuksan; Stephane Proux; Sam L Nsobya; Philip J Rosenthal; Matthew P Horning; Shawn K McGuire; Courosh Mehanian; Stephen Burkot; Charles B Delahunt; Christine Bachman; Ric N Price; Arjen M Dondorp; François Chappuis; Philippe J Guérin; Mehul Dhorda Journal: Malar J Date: 2022-04-12 Impact factor: 2.979