Paola Velardi1, Giovanni Stilo2, Alberto E Tozzi3, Francesco Gesualdo3. 1. Department of Computer Science, University of Rome "Sapienza", via Salaria 113, 00198 Rome, Italy. Electronic address: velardi@di.uniroma1.it. 2. Department of Computer Science, University of Rome "Sapienza", via Salaria 113, 00198 Rome, Italy. 3. Bambino Gesù Children Hospital, Piazza S. Onofrio 4, 00165 Rome, Italy.
Abstract
BACKGROUND: Digital traces left on the Internet by web users, if properly aggregated and analyzed, can represent a huge information dataset able to inform syndromic surveillance systems in real time with data collected directly from individuals. Since people use everyday language rather than medical jargon (e.g. runny nose vs. respiratory distress), knowledge of patients' terminology is essential for the mining of health related conversations on social networks. OBJECTIVES: In this paper we present a methodology for early detection and analysis of epidemics based on mining Twitter messages. In order to reliably trace messages of patients that actually complain of a disease, first, we learn a model of naïve medical language, second, we adopt a symptom-driven, rather than disease-driven, keyword analysis. This approach represents a major innovation compared to previous published work in the field. METHOD: We first developed an algorithm to automatically learn a variety of expressions that people use to describe their health conditions, thus improving our ability to detect health-related "concepts" expressed in non-medical terms and, in the end, producing a larger body of evidence. We then implemented a Twitter monitoring instrument to finely analyze the presence and combinations of symptoms in tweets. RESULTS: We first evaluate the algorithm's performance on an available dataset of diverse medical condition synonyms, then, we assess its utility in a case study of five common syndromes for surveillance purposes. We show that, by exploiting physicians' knowledge on symptoms positively or negatively related to a given disease, as well as the correspondence between patients' "naïve" terminology and medical jargon, not only can we analyze large volumes of Twitter messages related to that disease, but we can also mine micro-blogs with complex queries, performing fine-grained tweets classification (e.g. those reporting influenza-like illness (ILI) symptoms vs. common cold or allergy). CONCLUSIONS: Our approach yields a very high level of correlation with flu trends derived from traditional surveillance systems. Compared with Google Flu, another popular tool based on query search volumes, our method is more flexible and less sensitive to changes in web search behaviors.
BACKGROUND: Digital traces left on the Internet by web users, if properly aggregated and analyzed, can represent a huge information dataset able to inform syndromic surveillance systems in real time with data collected directly from individuals. Since people use everyday language rather than medical jargon (e.g. runny nose vs. respiratory distress), knowledge of patients' terminology is essential for the mining of health related conversations on social networks. OBJECTIVES: In this paper we present a methodology for early detection and analysis of epidemics based on mining Twitter messages. In order to reliably trace messages of patients that actually complain of a disease, first, we learn a model of naïve medical language, second, we adopt a symptom-driven, rather than disease-driven, keyword analysis. This approach represents a major innovation compared to previous published work in the field. METHOD: We first developed an algorithm to automatically learn a variety of expressions that people use to describe their health conditions, thus improving our ability to detect health-related "concepts" expressed in non-medical terms and, in the end, producing a larger body of evidence. We then implemented a Twitter monitoring instrument to finely analyze the presence and combinations of symptoms in tweets. RESULTS: We first evaluate the algorithm's performance on an available dataset of diverse medical condition synonyms, then, we assess its utility in a case study of five common syndromes for surveillance purposes. We show that, by exploiting physicians' knowledge on symptoms positively or negatively related to a given disease, as well as the correspondence between patients' "naïve" terminology and medical jargon, not only can we analyze large volumes of Twitter messages related to that disease, but we can also mine micro-blogs with complex queries, performing fine-grained tweets classification (e.g. those reporting influenza-like illness (ILI) symptoms vs. common cold or allergy). CONCLUSIONS: Our approach yields a very high level of correlation with flu trends derived from traditional surveillance systems. Compared with Google Flu, another popular tool based on query search volumes, our method is more flexible and less sensitive to changes in web search behaviors.
Authors: Richard S Hopkins; Catherine C Tong; Howard S Burkom; Judy E Akkina; John Berezowski; Mika Shigematsu; Patrick D Finley; Ian Painter; Roland Gamache; Victor J Del Rio Vilas; Laura C Streichert Journal: Public Health Rep Date: 2017 Jul/Aug Impact factor: 2.792
Authors: James Lester; Sarah Paige; Colin A Chapman; Mhairi Gibson; James Holland Jones; William M Switzer; Nelson Ting; Tony L Goldberg; Simon D W Frost Journal: PLoS One Date: 2016-06-09 Impact factor: 3.240