Electronic data collection and management system for global adult tobacco survey.

INTRODUCTION: Portable handheld computers and electronic data management systems have been used for national surveys in many high-income countries, however their use in developing countries has been challenging due to varying geographical, economic, climatic, political and cultural environments. In order to monitor and measure global adult tobacco use, the World Health Organization and the US Centers for Disease Control and Prevention initiated the Global Adult Tobacco Survey, a nationally representative household survey of adults, 15 years of age or older, using a standard core questionnaire, sample design, and data collection and management procedures. The Survey has been conducted in 14 low- and middle-income countries, using an electronic data collection and management system. This paper describes implementation of the electronic data collection system and associated findings.
METHODS: The Survey was based on a comprehensive data management protocol, to enable standardized, globally comparable high quality data collection and management. It included adaptation to specific country needs, selection of appropriate handheld hardware devices, use of open source software, and building country capacity and provide technical support.
RESULTS: In its first phase, the Global Adult Tobacco Survey was successfully conducted between 2008 and 2010, using an electronic data collection and management system for interviews in 302,800 households in 14 countries. More than 2,644 handheld computers were fielded and over 2,634 fieldworkers, supervisors and monitors were trained to use them. Questionnaires were developed and programmed in 38 languages and scripts. The global hardware failure rate was < 1% and data loss was almost 0%.
CONCLUSION: Electronic data collection and management systems can be used effectively for conducting nationally representative surveys, particularly in low- and middle-income countries, irrespective of geographical, climatic, political and cultural environments, and capacity-building at the country level is an important vehicle for Health System Strengthening.

Introduction

Portable handheld computers and electronic data management systems have been used for surveys in many high-income countries [1-3]. However, information on the use of mobile hand-held computers for national surveys in developing countries is limited [4-7] and implementation can be a challenge both administratively and technically because of the varying geographical, economic, climatic, political and cultural environments [1].

Most surveys in developing countries use paper questionnaires with manual input into a computer database for collation and statistical analysis [Figure 1.1]. This method can be time-consuming, error-prone and expensive, and a barrier to increasing the volume of data and speed of collection and analysis.

Figure 1.1:

Paper questionnaire data management model

As part of a plan to develop methods to monitor and measure global adult tobacco use, the World Health Organization (WHO) and the US Centers for Disease Control and Prevention (CDC) initiated planning and implementation of the Global Adult Tobacco Survey (GATS) [8], as a component of the Global Tobacco Surveillance System (GTSS). GATS is a nationally representative household survey of adults, 15 years of age or older, using a standard core questionnaire, sample design, and data collection and management procedures that have been reviewed and approved by a committee of international experts.

Tobacco use is a major preventable cause of premature death and disease worldwide. It kills nearly six million people each year and causes hundreds of billions of dollars of economic damage worldwide. Should current trends continue, this figure is expected to increase to more than eight million a year by 2030 [9]. If efforts to mitigate this epidemic are to succeed, there must be an efficient and systematic surveillance mechanism to monitor and manage the epidemic [10].

GATS was designed to produce high quality, globally comparable national and sub-national estimates on tobacco use, exposure to secondhand smoke, and quit attempts among adults in countries surveyed and to enhance the capacity of these countries to design, implement, and evaluate tobacco control and prevention programs [8]. GATS has been designed to produce national and regional estimates on tobacco use and tobacco control measures among adults. GATS is a face-to-face interview survey of civilian, non-institutionalized men and women aged 15 years or older who consider the surveying country to be their primary place of residence. GATS uses a stratified multistage cluster sampling approach in which probability-proportional-to-size random selection is used to successively select the sample of households in one or more stages to ensure adequate coverage of the entire target population while simultaneously minimizing the costs of data collection. After each sample household is selected, one eligible resident from each selected household is chosen electronically for the full survey interview. The selection is made by having the handheld computer generate a random number that corresponds to one member of the household [8].

During 2008–2010, GATS was conducted in 14 low- and middle-income countries [11-24] (Bangladesh, Brazil, China, Egypt, India, Mexico, Philippines, Poland, Russian Federation, Thailand, Turkey, Ukraine, Uruguay and Viet Nam) representing about 3.6 billion people [25].

Portable handheld computers were employed for data collection and the General Survey System (GSS) was the electronic data management system used [Figure 1.2]. This paper describes the implementation of electronic data collection during GATS and discusses the challenges and future directions for the use of handheld computers for data collection and electronic data management.

Figure 1.2:

Electronic data management model used in GATS

Methods

GATS data collection was conducted using portable handheld computers. To create and adapt this data collection and management system for GATS [Figure 2.1], a comprehensive data management protocol [8,26-29] was developed, which included the following components:

Figure 2.1:

Process diagram: GATS data collection activities

Developing and adapting the data management implementation plan to specific country needs

Selection of appropriate portable handheld computers

Software customization, using the Case Management System (CMS) and the General Survey System (GSS), for the portable handheld computers

Training, capacity building, and technical support.

The process of programming the questionnaire, planning and testing the data collection started after the questionnaire instrument and the sample had been adapted and approved to suit country-specific needs.

Developing the data management implementation plan:

The following three data transmission models were developed for countries to adapt to their specific needs:

: This model (Figure 2.2) was designed for countries with experience in electronic data collection systems and a wide reach of access to wireless Internet, which was considered a relatively high level of infrastructure and technical capacity. The sample was loaded onto the handheld computers from the national center to the field via wireless Internet using a Web interface and the data from the field were transmitted back to the national data center using the handheld devices and wireless Internet connections. Of the 14 countries, only one (Poland) implemented GATS using the Web-based data transmission model [Table 1].

Figure 2.2:

Model A – Web-based transmission

Table 1:

Summary of Participating GATS Countries

Countries	Year	Data management model implemented	Number of handhelds	Number of country specific languages	Number of field Interviewers trained	Number of households contacted
Bangladesh	2009	B	87	1	72	11,200
Brazil*	2008	B	N/A	1	N/A	51,011
China	2010	B+C	245	1	245	15,000
Egypt	2009	B+C	160	1	100	23,760
India	2009–10	B	500	19	500	79,690
Mexico	2009	B	175	1	177	18,540
Philippines	2009	B	205	6	189	12,086
Poland	2009	A	140	1	187	14,000
Russian Federation	2009	B	260	1	447	12,000
Thailand	2009	B	147	1	109	22,780
Turkey	2008	B	275	1	275	11,200
Ukraine	2010	B	130	2	94	13,833
Uruguay	2009	B	135	1	135	6,558
Viet Nam	2009	B	185	1	104	11,142
Total			2,644†	38	2,634†	302,800

In Brazil GATS was conducted on a sub sample as a part of National Household Sample Survey (PNAD)

N/A – Not Available

Excluding Brazil

Model B - Combination of Web and card based: This model (Figure 2.3) was designed for countries that did not fit the Model A requirements but had field Internet capabilities similar to most countries, which was considered an intermediate level of infrastructure and technical capacity. The data were transmitted using Secure Digital (SD) cards for loading the sample from the national level to the field level and over the Internet from the field to the national level, either via email or secure File Transfer Protocol (FTP) sites. All countries other than Poland used Model B, either fully or partially, for data management of GATS [Table 1].

Figure 2.3:

Model B – Card –based with field Internet capabilities

Model C - Card-based data transmission: This model (Figure 2.4) was designed for countries with no Internet access, considered the weakest level of infrastructure and technical capacity. The data were transmitted manually in both directions using SD cards, from the national center to the field interviewer’s portable handheld computers and back, or in some cases via FTP file transfers. None of the 14 countries used this model exclusively; China and Egypt used a combination of Models B and C for some areas with very poor infrastructure and no Internet access [Table 1].

Figure 2.4:

Model C: Card-based data transmission

Selection of appropriate portable handheld computers

For the first phase, two main models of HP IPAQ® portable handheld computers were used for data collection in most countries. The HP IPAQ hx2490c was used initially and then after it went out of production, the HP IPAQ 210/211 series was used.

The hardware was procured centrally to avoid country-specific procurement delays. Each portable handheld computer was accompanied with two SD cards and one extra battery as backup. Brazil used already existing handhelds with the Windows 5 operating system, which had been used for previous surveys [12]. A 10% hardware contingency was provided. After the completion of data collection, the handheld computers and other equipment were donated and remained in each country, so that both hardware and software could be reused to conduct other research or surveys as needed.

Software customization using General Survey System (GSS) for the handheld computers programming and data management [26]

The General Survey System (GSS) used in GATS is open-source software, which meant it could be used for data collection in any national or sub-national surveys. The database was stored on the handheld computers in an encrypted Structured Query Language (SQL) Server, compact edition database file (SDF) format. GSS had two main components, as described below:

GSS Integrated Development Environment (IDE) Suite (Desktop computer Interface)

The desktop application had a variety of tools to adapt the handheld computer questionnaire to country-specific settings, as well to aggregate, monitor, and report data during and after field data collection. Some of the key tools were:

The questionnaire designer: This user-friendly tool enabled users to develop questionnaires in multiple languages and build the files needed for loading the handheld computers. It also had tools for version control.

Sample management (Case File Tools): In GATS the household sample was preloaded on to the handheld computers with all household identification details. Case file tools in GSS program are then used for creating and managing the case files.

Data aggregation tool: Data collected from field were aggregated for analysis and fieldwork monitoring using this tool.

Data viewing tool: Every input was tracked by the software to provide the highest data quality standards. The data viewer allowed the central data hub to view data entry values without aggregating or converting the database to another format.

Reports tool: Daily progress and status reports along with basic analysis could be generated by just reading the database and selecting the method of analysis. Fast and easy reporting of progress and/or issues enabled smoother and more efficient data collection, while any potential issues throughout the fieldwork could be traced and addressed immediately instead of having to wait for completion of data collection and entry, as is the case in most paper-based surveys.

This comprehensive software component evolved over the two years of GATS implementation and proved to be a very effective tool for capacity building and helping countries use the technology for other survey/research needs.

GSS Engine and a Case Management System (CMS) (Handheld computers component)

The GSS handheld computers component had various tools, described below, the main one being :

The GSS Engine: This engine included a folder with system files that enabled a questionnaire designed using the GSS Desktop application to be read and displayed on the handheld computer. The GSS engine was also responsible for data security, encryption and backup tasks on the handheld computers.

The data were stored in real time in the handheld computer’s memory. In addition, a copy of the dataset was backed up onto the SD card in the handheld computer at the end of each data collection session, thus minimizing data loss.

Training, capacity building, and technical support

A standardized training mechanism was used for training in-country IT/fieldwork personnel. The data management team was extensively trained on each component of the survey system so they could train fieldwork personnel and maintain GATS specific data collection and quality control procedures. This also insured that countries would be able to conduct future surveys/research using the software and hardware. The training package consisted of three main components:

Training for pre-test and full survey implementation: Each country conducted a pretest prior to the actual fieldwork to test the data management model, software and hardware system, and questionnaire used in GATS, in order to extract lessons for effectively and systematically implementing the full survey. The training, ranging from three to five days, was provided to the country implementation team, IT and data management teams, and fieldwork personnel, using standardized training procedures with a specific focus on data collection and management with handheld devices [8]. In some countries (e.g., Bangladesh), refresher training was conducted in addition to the full survey training to meet country-specific needs and address changes in the data collection implementation mechanism. Other countries (e.g., Russia) were able to conduct the full survey with remote support after the in-country pre-test training.

Manuals and guides: Specific manuals for data management were developed and adapted for each country as needed. These followed the standard protocol and were an important instrument for maintaining systematic and standard procedures, as well as sustaining capacity. They included:

Programmer’s Guide to the General Survey System [26]

Core Questionnaire Programming Specifications [27]

Data Management Implementation Plan [28]

Technical support: An ongoing support mechanism via an international team of technical experts was in place to manage any challenges in the large-scale field implementation and provide remote and in-country technical knowledge for all countries.

Findings

GATS was successfully conducted in all 14 Phase I countries, representing over 3.6 billion people or over half of the world’s population, using handheld computers for data collection, between 2008 and 2010. Data were collected on approximately 303,000 households, representing more than 600 million smokers. In total, approximately 4300 field interviewers, supervisors and monitors were trained to use portable handheld computers, out of which, excluding Brazil, almost 2700 field interviewers alone trained in using these devices for data collection.

Approximately 3000 portable handheld computers used in the survey were:

Programmed to work with nearly 40 languages and dialects including English.

Programmed to work in various scripts, both Western and non-Western characters, including Arabic, Hindi, Latin, Mandarin, and Cyrillic.

Implemented in extreme environmental conditions, including

High altitude areas in China

Freezing winter in Ukraine and Poland

Hot and dry summer in Egypt

High humidity in coastal India

Monsoon season in Bangladesh.

Data quality, data availability, data security, data usability, cost-effectiveness and sustainability were observed to be the major advantages of using the electronic data collection and management system, as explained in detail below.

Data quality

Paper-based systems have been used for a long time in developing countries, but with many quality issues, such as transcription, data entry, and editing errors, skip logic errors, out-of-range values, and recording of ambiguous values. This has been especially true where the interviewer had to refer to answers to previous questions (sometimes in previous sections) in order to determine what question to ask next, based on the logical flow of the questionnaire. These errors were usually found during data entry from the paper forms and edited out with a good data entry and cleaning program, however it was difficult to rectify them at the entry level. The use of portable handheld computers allowed quality control at the time of data capture while the respondent was still present. The use of skip programming, range checks, validation and other data checks for valid data entry on portable handheld computers has proven to be highly instrumental in enabling high data quality and standardized entry at the collection stage. In-built skip and logic checks increased efficiency and reduced interview time.

In GATS, the data were stored digitally on the handheld computer and backed up on an external memory card, thus no data entry was required at the end of the survey. The data were transmitted during fieldwork, using the Web or emails on a daily or weekly basis, allowing early monitoring and quality control of the fieldwork at the regional and/or national level, which was not easily possible in paper-based surveys.

Overall the electronic data collection mechanism enabled better data quality control, as the data were verified at the entry level from the original data sources. In addition, rapid availability of the electronic files made early monitoring and review possible.

Data Availability

Use of portable handheld computers in GATS proved to be both time- and cost-effective. It also reduced error during risk-prone processes, such as administration of a huge number of paper questionnaires, shipping the equipment, and data entry and data cleaning.

Based on feedback from the countries, the fieldwork was usually slower than average in the first two weeks, as fieldworkers learned to use the technology. However over time the pace picked up and overall timing actually improved by the end of the survey. Most countries completed their fieldwork on or before the scheduled end date.

The process was reported to be more efficient and timely overall as compared to paper and pencil methods. One reason was the much reduced manual work required during interviews when using fully programmed electronic questionnaires. The automated data checks also saved interviewing time and costs.

Decreasing the time between data collection and data reporting had many advantages. It reduced costs, allowed better tracking of survey completions, helped estimate time to project completion, promoted early analysis, and allowed issues with questionnaires to be reported and corrected more swiftly. Electronic data capture reduced the time between data collection and reporting by putting the data in a clean, electronic format as soon as they were collected. GATS showed that electronic data collection could be optimally efficient for large-scale activities, given the rapid availability of data for reporting, analysis and prompt action.

Usability

The portable handheld computers were easy to use and carry. The electronic data collection system using portable handheld computers proved to be very manageable administratively over the entire span of the data collection, especially for large-scale surveys.

A 10% hardware contingency was planned, however, the actual hardware failure rate GATS was less than 1%. Battery life, which was an early concern, proved not to be a problem in the field as the instructions to charge the equipment every night were followed and monitored, and there was proper advance planning, especially for fieldwork in remote and rural areas.

Data Security

Handheld computers provided a more secure environment for data because they were validated and stored electronically at the entry stage, with backup on the device as well on an SD card in a secure encrypted format. This ensured safety, security and anonymity of the data.

The electronic data collection system used in GATS resulted in 0% data loss globally. It provided an accurate and stringent audit trail of response recording, which facilitated early monitoring of data integrity issues at the field level, highlighting any unconventional changes in data values.

Cost effectiveness

The initial cost to purchase equipment was a prominent budget item. It seemed likely that the use of handheld computers would be cost effective only if they were used for multiple surveys or other data collection initiatives.

Although the initial cost of equipment was higher, there were other major areas of cost savings, including paper, printing, and dispatch and handling of paper questionnaires, in addition to savings associated with data entry software purchases, data entry forms programming and development, and labor costs for data entry and editing. The entry-level checks with enhanced data quality may have provided greater precision with smaller sample sizes, compared to paper-based systems where the data cleaning removes invalid data thus requiring additional data to achieve similar precision. Overall this methodology allowed for faster data availability, enhanced data quality, and reduced administrative and logistical efforts and costs. The higher up-front costs of the electronic systems were recovered or mitigated by reducing back-end costs (e.g., keying and editing) and speeding up access to the data for analysis. In addition, it improved data quality.

It was expected that a significant issue in using handheld computers for GATS would be the initial training and technical support. This was addressed by adding additional one or two days of structured training for interviewers in the use of handheld computers for data collection. Although this was an extra initial cost, one day of extra training time was gained because less training was needed on questionnaire skips and quality checks, as these were built into the system. Any extra time and cost was easily compensated by the faster fieldwork using electronic data collection.

Sustainability

The GATS partners facilitated significant technical capacity building at the country level in the implementation of nationally representative surveys. Sustainable technical skills and the electronic data collection and management system together have proven to simplify and shorten the data collection process, enhance data quality and facilitate Health System Strengthening at the country level. This has been demonstrated by GATS countries having subsequently successfully conducted other national or health surveys using the new system (e.g., Egypt used the system for an economic survey, Bangladesh used the portable handheld computers for a non-communicable disease risk factor survey, China used a similar system for its Behavioral Risk Factor Survey, and India has conducted a national feasibility review for implementing this system across all health surveys in that country).

Conclusions

GATS was one of the largest global public health surveys. The use of an electronic data collection and management system provided data at a fast pace to meet programmatic needs with the highest quality of data. Successful use of this technology in GATS has proven that the use of handheld computers for national surveys in developing countries can significantly enhance the quality of data collection and management processes.

This system is could be used in most nationally representative surveys. It has been shown to expand the technical capacity of the host country and strengthen the overall health systems by providing effective tools for in-country staff, both in tobacco control and in other areas of survey research, thus acting as a strong vehicle for Health System Strengthening.

What was already known on the topic:

What this study added to our knowledge:

➢ Use of electronic data collection and management is effective and provides better data quality and faster data availability for policy making and action. ➢ Paper based surveys have been used for long time to collect data for nationally representative health surveys in middle and low income countries. ➢ Portable handheld computers and electronic data management systems have been used for national surveys in many high-income countries. ➢ The challenge, in the implementation of a standardized global survey across multiple middle and low income countries is the heterogeneity of geographical, economic, climatic, political and cultural environment of countries.

➢ The electronic data collection and management system used for GATS illustrates that a standardized electronic data collection and management systems can be used effectively in a number and variety of Middle and low income countries for conducting nationally representative health surveys. ➢ For an electronic data management system to be efficient in multiple countries, a standard comprehensive protocol is a very critical instrument. ➢ The capacity built in-country staff for electronic data collection and management system acts as a strong vehicle for Health System Strengthening both in tobacco control and in other areas of survey research

National Agencies and Collaborators

Country Name	Agencies	Collaborators
Bangladesh	Ministry of Health and Family Welfare (MoHFW), National Institute of Preventive & Social Medicine (NIPSOM), Bangladesh Bureau of Statistics (BBS) National Institute of Population, Research & Training (NIPORT)	Md. Amirul Hasan (NIPSOM)
Brazil	Ministry of Health-Secretariat of Health Surveillance (SVS), Brazilian Institute of Geography & Statistics (IBGE), National Cancer Institute (INCA), The National Health Surveillance Agency (ANVISA)	Deborah Carvalho Malta (MoH-SVS), Eduardo Pereira Nunes, Marcia Quintslr, Cimar Azeredo (IBGE), Liz Maria de Almeida (INCA), Humberto Martins (ANVISA)
China	Ministry of Health (MoH), Chinese Centers for Disease Control (China CDC)	Yang Gonghuan, Yang Yan, Xiao Lin, Li Qiang (China CDC)
Egypt	Ministry of Health (MoH) Central Agency for Public Mobilization & Statistics (CAPMAS)	Sahar Latif Labib (MoH), Awatef Hussein (CAPMAS)
India	Ministry of Health & Family Welfare (MoHFW) – Government of India, International Institute for Population Sciences (IIPS)	Anuradha Vemuri, Jagdish Kaur (MoHFW), F. Ram, Sulabha Parasuraman (IIPS)
Mexico	Ministry of Health (MoH) National Institute of Public Health (INSP)	Mauricio Hernandez Avila (MoH), Luz Miriam Reynales-Shigematsu (INSP)
Philippines	Department of Health (DoH), National Statistics Office (NSO)	Agnes Segarra (DoH), Glenn Barcenas, Benedicta Yabut (NSO)
Poland	Ministry of Health (MoH), Maria Skłodowska-Curie Cancer Center Institute of Oncology, Medical University of Warsaw, Pentor Research International	Tadeusz Parchimowicz (MoH), Witold Zatonski, Krzysztof Przewozniak (CCI), Filip Raciborski (WMU), Krzysztof Siekierski (Pentor)
Russian Federation	Ministry of Health & Social Development (MoHSD), Federal State Statistics of Russia (RosStat), Pulmonary Research Institute (PRI)	Maria Shevireva, Natalya Kostenko, (MoHSD), Vadim Nesterov, Tamara Chernisheva, Tatiana Konik (RosStat), Galina Sakharova (PRI)
Thailand	Ministry of Public Health (MoPH), National Statistical Office (NSO), Tobacco Control Research & Knowledge Management Center (TRC) at Mahidol University	Sarunya Benjakul (MoPH), Lakkhana Termsirikulchai, Mondha Kengganpanich (TRC), Areerat Lohtongmongkol, Hataichanok Puckcharern, Chitrlada Touchchai (NSO)
Turkey	Ministry of Health (MoH), Turkish Statistical Institute (TurkStat), Hacetteppe University (HU)	Hüseyin İlter (MoH), Enver Tasti, Ramazan Celikkaya, Guzin Erdogan (Turkstat), Nazmi Bilir, Hilal Özcebe (HU)
Ukraine	Ministry of Health (MoH), Kiev International Institute of Sociology (KIIS), School of Public Health, National University of Kyiv-Mohyla Academy (SPH)	Alla Grygorenko, Konstantin Krasovsky (MoH), Natalia Kharchenko, Volodymyr Paniotto (KIIS), Tatiana Andreeva (SPH)
Uruguay	Ministry of Health (MoH), National Statistics Institute (INE)	Winston Abascal, Ana Lorenzo (MoH), Franco González Mora (INE)
Viet Nam	Ministry of Health (MoH), Viet Nam Standing Office on Smoking and Health (VINACOSH), General Statistics Office (GSO), Hanoi Medical University (HMU)	Phan Thi Hai (MoH), Nguyen The Quan (GSO), Hoang Van Minh (HMU), Kim Bao Giang (HMU)