Examining the Correlation of Google Influenza Trend with Hospital Data: Retrospective Study.

INTRODUCTION: Many studies have explored social media and users search activities such as Google Trends to predict and detect influenza activities. Studies that examined Google Trends correlation with the actual hospital influenza cases were conducted in non-tropical regions that have clearly defined seasons. Tropical areas are known for having less-defined seasonality and the extent of Google Trends concordance with actual influenza cases is unknown for these areas. The goal of this study is to compare Google Trends with hospital cases in tropical regions.
METHODS: We analyzed 48,263 influenza cases in the time period of 2010 to 2019. The cases were retrieved from central hospital medical records in tropical regions using the corresponding codes for influenza ICD-10 AM. Cases from the medical records were compared with Google Trends to determine trends, seasonality, and correlation.
RESULTS: Graphically, there were some similar areas of the trend, but cross-correlation analysis did not show any significant correlation between hospital and Google Trends with a maximum correlation rate of 0.300. Seasonality analysis showed a clear pattern that peaked around November in Google Trends while hospital data showed less defined seasonality with a smaller peak occurring at the end of December and beginning of January.
CONCLUSION: Based on the results, there is a weak correlation between Google Trends and hospital data. More innovative methods are emerging to predict influenza activity using social media and user search data and further study is needed to examine the concurrent trends derived using these methods across regions that have different humidity levels and temperatures.

Background

Seasonal influenza is an acute respiratory infection caused by the influenza virus. There are various forms of seasonal influenza and it spreads easily and rapidly in communities throughout the world. The virus is able to assume epidemic proportion in a short time frame.1 In temperate climates, seasonal epidemics occur primarily during the winter, but in tropical regions, influenza may occur throughout the year, something that causes irregular and hard to predict outbreaks.2,3

Surveillance systems for influenza often rely on hospital-based clinical data to monitor the virus’ activities and allow public health officials to take necessary actions.4 Different surveillance systems may have different criteria and reporting eligibilities. The determination of data included in this report is determined by the surveillance system. The assessment of case eligibility for reporting requires data from multiple sources including physical examination, collection of signs and symptoms, and laboratory results.

The lifecycle of an influenza outbreak has several phases and the detection in each phase requires different information. The lifecycle begins with initial exposure and then progresses to incubation, symptoms, a visit to a healthcare facility, and confirmation of diagnosis. There are various surveillance systems that can be used at different phases of the disease lifecycle. Each system has its own strengths and weaknesses that can be identified based on the data being collected.5 Generally, systems can be in one or two categories: data collected before visiting a healthcare facility and data collected after visiting a healthcare facility. Syndromic surveillance methods involve “relying on detection of clinical case features that are discernible before confirmed diagnoses are made”.5 This type of system often uses data prior to confirmation of diagnosis. It is an early stage of detection that uses information to estimate the probability of a case or outbreak and then may ask for a further public health response. This information occurs prior to diagnosis, something which often requires laboratory results to be obtained.

Some systems require laboratory confirmation of diagnosis. Although this type of system can improve the accuracy of the reported data by allowing physicians to assess the case and confirm the diagnosis, it has the disadvantage of longer reporting time as well as the possibility of missing false-negative cases.6–9 This system could result in longer outbreak detection time since there are additional steps needed prior to the data being entered into the reporting system. The time taken for analysis at the central registry and the time needed for the public health registry to communicate the findings with the applicable hospital(s) authority and treating physician(s) and the issuing of a warning signal can be somewhat substantial.10–12

Another limitation of the system is that some physicians may miss cases. This issue is more probably with Influenza Like Illness (ILI) as it shares similar signs and symptoms with other diseases, especially during the early stages of the infection. Confirmation is typically obtained through laboratory testing.9 Physician awareness of an outbreak in an area will increase the likelihood of ordering laboratory resting and engaging in a more thorough investigation. Early warning signs of unusual influenza activity in an area can direct physician attention toward a potential outbreak and greater investigation. This added attention can ultimately lead to a proper diagnosis of the epidemic and the triggering of necessary public health actions.

With the advancement of business intelligence and the availability of “big data” sources that provide information collected from multiple sources, researchers have started to explore the value of using non-hospital data for outbreak detection. One of the common sources of data is Google search terms, also known as Google Trends.10 Google Trends can be used to analyze search keywords, in this case searches for specific diseases. Unlike hospital data, Google Trends can provide early warning signals as it collects and analyses online activity prior to patients visiting healthcare facilities. Another advantage of this system is that Google Trend collects additional information that may not be within the data set collected by hospitals or reported to public health agencies.

Despite the popularity of the Google influenza trend, studies in different countries have provided conflicting results of how Google Trend data reflect the actual trends of influenza based on hospital cases.13–17 These studies have indicated that more research is needed if Google Trends will be used for forecasting purposes. Studies about how Google Trends correlates with actual cases during off-peak periods and smaller waves are also needed.13 While many studies indicated the value of Google Trends for complementing the traditional methods.15,18,19 Others pointed out some limitations related to the reliability and accuracy of Google Trends when compared with seasonal or pandemic influenza.14,20,21

Prior studies were conducted in climates with clearly defined seasonality,13–17 but areas with more tropical climates are known for having smaller peaks and less-defined influenza seasonality.2,3 The extent to which Google Trends reflects actual influenza activity in a tropical region or hospital level is unknown.

Given the documented impact of climate on the trend of influenza cases, little is known if Google Trends will be impacted by climate condition to the same extent. More studies about the correlation between of influenza cases and Google Trends at different climate will improve our understanding of the limitation and strength of Google Trends.

Determining the accuracy of Google Trend data in different regions can unlock a wide range of public health and epidemiological applications. In this study, we will compare influenza-related online activity based on Google Trend with actual influenza cases in a hospital.

Methods

In this study, we compared influenza activity trends on Google Trend with the actual trend based on hospital influenza cases. We created time-series and seasonal graphs. This study was approved by the institutional review board (IRB) at the Saudi Ministry of Health, Jazan Health Affairs, protocol number H-10-Z-073. The study conducted in compliance with the Declaration of Helsinki and the ethical guidelines of ensuring that every precaution has been taken to protect the privacy of research subjects and the confidentiality of their personal information.22 All data employed in the study were de-identified secondary data and no patients identifying information were included. Therefore, explicit patients consent to review their medical records at this case was not required by the Saudi Ministry of Health.

Google Trend Data

We carried out a search in Google Trend () using the keywords “influenza”, “corona”, and “MERS” for a 10-year period starting in January 2010 and ending in December 2019. The search trends pertained to the entire geographical territory of the Kingdom of Saudi Arabia. Google Trends uses the proportion of cases that peaked during a specified period and the data was graphed along with data from hospital record systems. The collection method and use of Google Trends data complied with the terms and conditions of the website ().

Hospital Data

The influenza incidence data used in this analysis were retrieved from the Electronic Health Records (EHRs) of King Fahad Central Hospital which is tertiary-level care hospital. It is a government hospital that operate under the Saudi Ministry of Health. Therefore, the IRB for the study was obtained from the Saudi Ministry of Health. The hospital is in Jazan, by the Red Sea, in the southwestern part of Saudi Arabia. This region has a somewhat tropical climate with an average temperature range of 78°F to 91°F and an average humidity of over 64% (more about the weather can be found in Table 1).23

Table 1

The Historical Weather in Jazan Region Including Temperature, Humidity, and Rain

Avg. Temperature °F	78.3 °F	79.2 °F	82 °F	85.9 °F	89.2 °F	90.8 °F	91.2 °F	90.5 °F	90.7 °F	87.7 °F	83.6 °F	79.9 °F
Min. Temperature °F	73.7 °F	74.5 °F	77.1 °F	80.8 °F	84.2 °F	85.5 °F	86.1 °F	86.2 °F	86.4 °F	83.1 °F	79.1 °F	75.4 °F
Max. Temperature °F	82.7 °F	83.9 °F	86.8 °F	90.9 °F	94.4 °F	95.9 °F	96.3 °F	95 °F	95.5 °F	92.6 °F	88 °F	84.2 °F
Precipitation/Rainfall mm (in)	17 (0.7)	7 (0.3)	10 (0.4)	13 (0.5)	2 (0.1)	1 (0)	12 (0.5)	20 (0.8)	6 (0.2)	25 (1)	21 (0.8)	18 (0.7)
Humidity(%)	75%	74%	71%	69%	67%	66%	64%	69%	69%	68%	70%	74%
Rainy days (d)	3	1	1	1	0	0	1	2	1	4	4	4

Note: Reproduced from Climate-Data.Org. Weather by month/weather averages Jizan. Available from .23

We used the ICD-10AM codes for Influenza and Influenza Like Illness (ILI) spanning from 2010 to 2019. All flu cases retrieved for the study were identified based on the direct influenza codes (J09, J10, J10.1, J10.8, J10.81, J10.89, J11, J11.1, J11.8) and the ILI codes identified in prior studies.24,25 We also mapped to the earlier version of ICD for the period before the ICD-10AM implementation.

We did not use the criteria of fever and cough as indicators for the viruses under study since they are not specific to ILI like the other codes used in this study area.24 All data retrieved were anonymized before being accessed by the research team and no personal identifying information was collected.

Analysis

All data are entered in the hospital EHR included a unique medical records number and visit ID. Because this study is interested in the number of events, the codes were aggregated based on the number of visits. Patients who may have multiple diagnoses were counted as a single data point and anonymized before being shared for research. The annual number of hospital incidents retrieved is displayed in Table 2.

Table 2

The Number of Annual Cases Retrieved from the Hospital Database from 2020 to 2019

Year	2010	2011	2012	2013	2014	2015	2016	2017	2018	2019	Total
Number of cases	2025	4430	6780	5994	5183	5981	5695	5054	4389	2732	48,263

For comparison, both Google Trends and hospital data were converted based on the two variables of time and intensity. The intensity was calculated as the monthly proportion of incidents from its peak within the timeframe of 2010 through 2019. Data were graphed together to compare the alignment and patterns of change.

The pattern of the influenza season and peaks were examined by plotting the seasonal graph for each data source. The annual trends for each year were recorded separately to compare changes in the pattern of influenza over time and at each data source. The Statistical Package for the Social Sciences (SPSS) v 21 (IBM Corp) was used for seasonality analysis and visualization. We also conducted a cross-correlation analysis to examine the association between the trends generated by Google and hospital cases. In addition, cross correlation enables us to examine if an association exist with the lag (delay) between the two time series trends (Google and hospital cases). We used the same time frame used for the seasonal and trend analysis (2010–2019) and the same unit of time interval (months). The cross-correlation results were interpreted as follow; weak (<0.4) Moderate (0.4–0.6), strong (>0.6).26

Results

Using hospital medical records, 48,263 influenza cases spanning from 2010 to 2019 were retrieved. These cases were compared with influenza activities found by Google Trends for the same time period. The results of our study will include a time-series trend comparison, seasonality analysis, and cross-correlation analysis.

Trends and Time-Series Analysis

The comparison of Google Trend results (Figure 1) with hospital influenza incidence patterns also included the Google search for the terms “MERS”, and “corona”. Of the three keywords, only “influenza” showed a noticeable level of online activity that might correlate with the hospital trend. The only exception is a high online activity for “MERS” during 2014. This activity corresponds with the onset of this infection in the country.27

Figure 1

Comparing the patterns of the monthly influenza activities from 2010 to 2019 using hospital data, and Google Trends with the terms “influenza”, “MERS”, and “corona”.

Google Trends showed a much lower volume in the initial periods compared to the second half of the study period. These results could reflect the lower use of internet searches in the period prior to 2015. For the search of “influenza”, a noticeable overlap can be observed between Google Trend results and hospital data starting from 2015 and ending in December 2019. Through the seasonal peaks of the hospital data trend, there has been a corresponding rise in the result of the keyword “influenza” based on Google Trend data. The hospital data also showed remarkable differences in the number of peaks from year to year.

From 2010 to 2014, Google Trends and hospital cases showed somewhat similar pattern with different magnitudes. Both trends and hospital cases peaked during December 2010, December 2011, and November 2012. They had slightly similar moves during 2013 and 2014. The difference in magnitude can be explained based on how the proportion of cases was calculated in relation to the peak for the entire study period.

Figure 1 also shows that there is a lag in peaking time between Google Trends and hospital data from 2015 to 2017. We analyzed the overlap of disease occurred during the two seasonal peaks between the last quarter of 2015 through the first quarter of 2017 and compared the peaks to the corresponding results for “influenza” in Google Trend. Figure 1 shows that Google Trends showed increased levels of online searches for “influenza” before the disease peaks of May 2015, February 2016, and January 2017. There is also an overlapping peak in November 2017.

Seasonality Analysis

The Google Trend data for the term “influenza” show clear seasonal disease trends over the 10-year period (Figure 2). The onset of disease peak starts in the last quarter of the year and subsides in the first quarter of the following year. There are variations in disease incidence intensity and the exact time of onset each year. The hospital data showed much less defined seasonality (Figure 3) during the last 3 years of the examined period (2017 to 2019). From 2010 to 2016, the peak season starts in October and peaks by the end of the year. It begins to gradually decline around the middle of the year (June).

Figure 2

The monthly proportion of influenza activities at Google Trends for Saudi Arabia from 2010−2019 using the term “influenza”.

Figure 3

The monthly proportion of influenza activities from the hospital data 2010−2019.

Cross-Correlation Analysis

To examine the correlation between Google Trends and hospital data, we conducted a cross-correlation analysis (Figure 4). In this figure, the cross-correlation between the two trends was generally weak. The highest correlation was observed at lag 0 and lag 5 with 0.299 and 0.300, respectively.

Figure 4

A cross-correlation analysis between Google Trends and hospital data for influenza cases.

Discussion

Assessing how Google Trend represents actual influenza cases opens a wide range of practical epidemiological and public health applications. Studies documented the impact of influenza on hospitalization and how surveillance can inform decision making.28,29 Surveillance from search data like Google Trends has the potential of being an early indicator and enable the timely preparedness and intervention.

Several studies have examined the relationship between Google Trends and hospital influenza cases, but the hospitals used in these studies were in non-tropical areas and had clearly-defined seasonality.13–17 While previous studies reported conflicting results about Google Trend data reliability,13–17 no studies were found to examine how Google Trend data can be used to confirm actual influenza cases in different weather conditions. This study compares hospital data with Google Trends data of 10 years to explore the potential usefulness of Google Trends in surveillance and prediction of influenza in tropical regions.

Our analysis shows a weak relationship between hospital influenza cases and Google Trends. There was some similarity between the two during peak influenza season from 2010 to 2015. There was approximately a month lag between the two during the peak influenza season for 3 years. The time-series analysis shows some similarities in certain years and the cross-correlation analysis shows weak correlation between the two trends.

The application of internet data for health research is an evolving field and the domain needs further refinement to generate actionable intelligence for disease control efforts.17,30 A study of Google Flu Trends in the United States provided timely and accurate estimates of the influenza activity, especially during epidemic peaks, however, the study also reported that more research is needed to explain the Google Trends accuracy during smaller waves and off-peak periods.13 Our analysis shows a similar experience with smaller disease peaks. In a study that assessed the sensitivity of search data on Google, Twitter, and Wikipedia, there was low to medium sensitivity in influenza event detection. The study concluded that the search data had a limited ability to contribute to surveillance systems designed to detect influenza.16 Similarly, cross-correlation analysis in our study showed a weak correlation.13–17 Influenza patterns in tropical regions do not always follow seasonal patterns and there is high background influenza activity throughout the year.31

Despite its benefits, Google Trends is prone to bias and can provide inaccurate signals.10,14,32 For example, peaks could be overestimated when users search of post-influenza-related keywords based on media coverage. In these cases, individuals who do not suffer from influenza-related symptoms would impact the results.20 According to Cervellin et al., Google's Trends is more influenced by media coverage than the actual epidemiological burden.33 The spike in the use of the search term “MERS” during 2014 (Figure 1) was the result of high public anxiety due to media coverage and was not an accurate reflection of the cases during this time.34

Limitations

One of the limitations was the difference in the level of geographical coverage of Google Trends and hospital data. In our study, the number of cases generated by Google Trends at a city or regional level was low and made inferences difficult. We used the country-level Google Trend for this reason. The extent to which Google Trend search terms represent a particular geographic region should be taken into account. This is especially important when a country or state has areas with different environmental factors, in particular humidity and temperature.3 The data collection area could have caused some cities and areas with higher populations to be overrepresented. Google Trends data for the study’s hospital is available in smaller quantities, a factor that makes it somewhat difficult to use.

Conclusions

Our study showed a weak correlation between Google Trends and hospital influenza data. Although the two trends had some similarities during the peak influenza season, the overall correlation was statistically weak. In general, influenza cases in hot and humid areas have less defined seasonality compared to non-tropical areas. Given the strong association between weather and influenza. We found it very important to examine the accuracy of Google Trends of influenza under the different regions with different climates and the degree to which regional climates can impact the accuracy of Google Trends. Further studies could also examine the alternative sources of influenza data derived from search terms and social media. The author also recommends examining ILI compared to pandemic influenza as the search term for “pandemic influenza” may be more a product of media coverage than the actual virus.