An innovative web based system for reporting rare diseases in paediatrics.

BACKGROUND: Surveillance of rare diseases in children is an important aspect of public health. Rare diseases affect thousands of children worldwide. The Canadian Paediatric Surveillance Program (CPSP) has been in existence since 1996, and provides an innovative means to undertake paediatric surveillance and increase awareness of childhood disorders that are high in disability, morbidity, mortality, and economic costs to society, despite their low frequency. Traditionally, CPSP used manual paper-based reporting on a monthly basis, which although had an impressive response rate, it had inherent longer processing times and costs associated with it.
OBJECTIVES: To provide an overview and evaluate an innovative web-based system that enables seamless reporting from participants across the country providing a quick, reliable and simple mechanism for the participants to submit data while yielding better data quality, timeliness and increased efficiencies.
METHODS: In 2011, a proprietary electronic CPSP (eCPSP) system was developed to provide a simple, quick and reliable reporting environment for participants. It supports both the electronic and hardcopy reporting. The analysis presented in this paper was conducted based on usage data of this system.
RESULTS: The response rates of the new eCPSP were found to be very favorable with adjusted rate of 80%, which equals the baseline. Approximately 50% of online participants report the first day they receive the notification e-mail. The response time was also reduced considerably. Furthermore, there has been significant reduction in data handling related activities (by almost 70%) from estimated 690 hours per year. Finally, the number of cases reported that do not fit the study case criteria has fallen, likely because participants can now immediately access the case definition and protocol via the online system. This has reduced both staff and investigator time for case processing.
CONCLUSION: The eCPSP has modernized the CPSP program from paper-based reporting to efficient online technology while maintaining the core principles of the program. This simple and intuitive approach has proven to be an efficient approach cutting response times significantly while maintaining the desired response rates.

Introduction

The importance of surveillance to the practice of medicine cannot be overstated. Through ongoing, systematic collection of data, the burden of disease can be determined, interventions to prevent the occurrence of a disorder can be assessed, and the information collected can be used to develop health policy. Surveillance takes research data into action.

According to Statistics Canada, the Canadian population on July 1, 2014, was an estimated 35,540,400. There were close to 8 million children and youth under the age of 20, representing approximately 22% of the population [1]. Although individually uncommon, rare diseases affect hundreds of thousands of Canadian children and youth and typically have lifelong impacts. The incidence of many rare diseases is unknown, and yet is essential for improved clinical care, advocacy and health service planning.

This article describes the process followed by the Canadian Paediatric Surveillance Program (CPSP) and the Canadian Network for Public Health Intelligence (CNPHI) to design and implement an electronic reporting system for rare disease surveillance. It describes the application, discusses preliminary outcomes of implementation, and presents key lessons learned.

Canadian Paediatric Surveillance Program (CPSP)

The Canadian Paediatric Surveillance Program (CPSP) is a joint program between the Canadian Paediatric Society (CPS) and the Public Health Agency of Canada (PHAC), established in 1996. The program has since become an innovative epidemiological real-time mechanism [2] to undertake paediatric surveillance and increase awareness of rare childhood disorders that are high in disability, morbidity, mortality, and economic costs to society, despite their low frequency. Studies that are executed through the CPSP are each led by a principal investigator(s) and co-investigators across Canada, and must have strong scientific and public health importance or could not be undertaken any other way. All studies must conform to high standards of scientific rigour and practicality, and the CPSP assures the confidentiality of all information collected.

The program has steadily grown from three surveillance studies in the pilot year to over 45 conditions studied since its inception. Topics are varied and span a wide range, from infectious diseases, medical and genetic conditions, preventable injuries, and mental health conditions.

Because not all research questions warrant a full study, the CPSP is also available to investigators as a cost-effective tool to survey participants on a one-time basis to capture a signal, document a change in pre- and post-study knowledge of a particular disease, identify the prevalence of a problem or answer a specific question of clinical or public health relevance.

CPSP surveillance studies have led to important medical and public health actions over the years. For example:

The wheeled baby walker survey results contributed to the total ban on the sale, import and advertisement of these walkers in Canada [3].

The vitamin D deficiency rickets study confirming the importance of reinforcing the current CPS recommendation that exclusively breast-fed infants and children receive vitamin D supplementation [4].

The medium-chain acyl-coenzyme A dehydrogenase deficiency study documented the efficacy of newborn metabolic screening programs in detecting asymptomatic cases that allow for early preventive measures (the two reported deaths during the course of the study did not occur in jurisdictions with screening programs) [5].

The one-time survey results on concussion management by paediatricians demonstrated the use of concussion/mild traumatic brain injury guidelines and criteria used in the initiation of return-to-play and management of return-to-play for brain-injured children and youth [6].

The program also offers the opportunity for international collaboration with 12 other national paediatric units worldwide, through the International Network of Paediatric Surveillance Units (INoPSU). Incredibly, many of the units have been collecting data on rare childhood diseases for 20 years or more. Over 300 rare conditions have been studied to date, including rare infectious and vaccine-preventable diseases, mental health disorders, child injuries and immunological conditions. The network encompasses approximately10,000 health care providers who voluntarily contribute data on these rare diseases every month [7].

Canadian Network for Public Health Intelligence (CNPHI)

The Canadian Network for Public Health Intelligence (CNPHI) is a pre-existing cutting-edge PHAC resource that has been developed and is managed by the National Microbiology Lab (NML), and for which many federal, provincial and territorial epidemiologists and laboratory scientists are familiar and often routine users [8]. CNPHI provides a suite of online tools that enable laboratory and epidemiologic collaboration on a range of public health topics.

CNPHI currently supports over 4000 registered users and has successfully developed a suite of web-based resources including a Pan-Canadian alerting system, collaboration centres in use by various communities of practice, national disease surveillance systems, and laboratory based surveillance systems for human, animal, and environmental health domains.

The CPSP Surveillance System

The CPSP gathers data from over 2,500 clinically active paediatricians and paediatric subspecialists each month, on a voluntary basis. To enhance data capture, the program works collaboratively with other professional groups beyond paediatrics, such as psychiatrists, pathologists/coroners, endocrinologists, and adult infectious disease specialists. Data is only requested from participants in active practice. Participant status is adjusted on an ongoing basis to account for any absences.

The CPSP uses a two-tiered reporting system to ascertain and investigate cases: an initial check-off form, and a detailed questionnaire. The check-off form, listing the conditions currently under surveillance, is distributed monthly to participants, who are asked to indicate the number of new cases seen in the previous month, or if no new cases were seen, which is very important in active surveillance; the CPSP cannot assume that no reply means no cases. The check-off form and detailed questionnaires are provided in either English or French depending on the preferred language of the participant (see Figure 1).

Figure 1

Reporting Process

Development of the e-reporting system: Feasibility study

For the first 15 years of the program, monthly forms were sent by mail. This represented significant overhead to the program as well as long delays to receive surveillance data.

A feasibility survey for implementation of an electronic reporting system was conducted with a response rate of 31% [9]. Responses indicated that more than two-thirds of participants (72%) were willing to report electronically, 13% were undecided or would require additional information and 15% preferred to continue reporting by hardcopy through the mail.

Suggestions from respondents included a simple, quick and reliable system with minimal or no log-in requirements, e-mail reminders to submit their reports with links to Web forms, very clear sender and subject line to enable identification when rapidly scanning and deleting emails from the inbox, and the highest levels of security and confidentiality.

In 2011, the CPSP partnered with CNPHI to develop a proprietary electronic CPSP (eCPSP) application.

eCPSP Application Components

The eCPSP application comprises of various innovative components as discussed below.

Participant Management: The eCPSP application includes the ability to send email notifications to participants for a specific reporting month by interfacing with the CPS membership database to gather the participant lists including "absence" data to exclude specific participants who are away during a given reporting month. Each participant is known to the eCPSP application through an internal participant administrative number; however, the details of each participant are stored in the resident CPS database only for confidentiality purposes. During a fetch cycle, each participant has an indicator as to whether they are participating as an online user or offline user. This indicator is used to differentiate between those participants who receive an email notification versus those who will be sent a paper form.

Studies: Each monthly report may include multiple studies that the program is conducting. Given the dynamic nature of these studies, the eCPSP application includes a tool to manage each study. The monthly report auto configures based on the active studies when the notifications for a specific reporting month are generated. This approach allows the administrators full flexibility of managing the studies as needed.

Monthly Submissions: The application supports two types of monthly submissions as described before, i.e., online and offline (hardcopy). For online submissions, an email notification is sent to each participant and includes a one-time use link to an online reporting form for the current reporting month including any reminders if required while excluding any absences. For offline submissions, the application generates hardcopy report forms with barcodes, which are used by CPSP staff to expedite data entry upon receipt.

Case Management: Participants may report cases in a given reporting month and may also, optionally, include the gender and month/year of birth of the patient to help them recall the case when asked to complete the follow-up detailed questionnaire. Participants can also immediately access the case definition and protocol via the online system, which makes it easier for them to determine if the case meets the study criteria.

Detailed Questionnaires: When participants identify having seen a case on the monthly form, the system generates a letter to the participant with the clinical detailed questionnaire for the study. Detailed questionnaires at present time are mailed out in paper form and the eCPSP system maintains an audit for analysis and follow-up purposes.

Surveys: In certain situations, a one-time survey of participants may be conducted to either identify the prevalence of a problem or to answer a specific question. Much like the full studies, the one-time surveys vary in topic. The eCPSP application includes an integrated capability to design interactive surveys using CNPHI's WebData technology [10]. The ability to send customized surveys through the online system greatly improves speed of response, important in emergency or potentially high-risk situations.

Status Board: Online participants can view the status of their monthly reports and see if they have any outstanding detailed questionnaires to submit. The status board also provides links to communications such as alerts, quarterly statistics and resource articles.

Analysis and Reports: The eCPSP application generates various reports for monitoring response rates and overall program activity including follow-ups.

Preliminary outcomes

A number of goals were identified for the new e-reporting system. For participants, the CPSP wanted a fast, easy system of reporting, reliability, security, flexibility, and reduced waste. The program aimed to see 70% of participants convert to online reporting, an average monthly response rate equal or better to the baseline 80%, increased speed of communications, better data access and control, reduced staff time dedicated to mailing production and manual reminders, and reduced materials and postage costs.

At 3 months following the launch, 44% of participants were reporting online. Help requests were minimal, and feedback about ease of use was very positive. Response rates remained consistent with the previous system. As of July 1, 2014, 68% of CPSP participants were enrolled in electronic reporting, very close to the 70% target.

The average monthly response rate for participants in 2013 was 84% for online participants and 72% for hard-copy participants. With year-end follow-ups, the overall annual adjusted response rate was 80%, equal to baseline.

Median response time for hardcopy participants in 2013 was 23 working days (mean 54 working days) compared to 2 working days for online participants (mean 35 days). Approximately 50% of online participants report the first day they receive the notification e-mail.

Before implementing eCPSP, all participants received a mailed item at least 12 times per year. Including follow up reminders, this amounted to over 67,000 mailed items each year with associated materials, printing and postage costs. Staff hours dedicated to monthly surveillance data handling (mailing, intake, etc.) was estimated at 690 hours per year, or 57 hours per week. This has been reduced by almost 70%.

Another positive outcome of eCPSP has been increased confidentiality. Considering the time delay between reporting a case and receiving the detailed questionnaire in the former mail-based system, approximately 1/3 of participants would hand-write recall notes on the monthly forms The online system does not accommodate these types of entries. Moreover, participants now receive the questionnaires much more quickly, reducing the need for such notes.

The number of cases reported that do not fit the study case criteria has fallen, likely because participants can now immediately access the case definition and protocol via the online system. This has reduced both staff and investigator time for case processing.

CNPHI's ability to provide timely reports and statistics on response data has enabled CPSP administrators to gain a better understanding of participants' reporting behavior and to identify appropriate solutions or follow-up actions.

Limitations

The main limitation at present time of the existing approach is the co-existence of electronic and paper based reporting. The program has not yet been able to abandon the hybrid online/offline system, which will be maintained for the foreseeable future. This has been important to the continued satisfaction of participants and maintenance of the response rate.

Specifically,

Although staff hours dedicated to the processing of hardcopy monthly forms and reminders have been greatly reduced by use of the electronic reporting, personalized follow up with participants continues to be required to maintain response rate;

Reduction in the amount of time needed to launch a new study was among the goals of converting to an online system. However, given the need to maintain a hybrid system with simultaneous launch to both offline and online participants, and the need to prepare the hard copies of the protocols and questionnaires, the program has not been able to achieve this goal.

Conclusion

The eCPSP initiative has modernized the CPSP program from paper-based reporting to efficient online technology while maintaining the core principles of the program. This simple and intuitive approach has proven to be an efficient approach cutting response times significantly while maintaining the desired response rates.

Some of the key lessons learned include: better documentation to support staff turnover; up-front investment to carefully assess the needs both of participants and the program is very important, as is a firm commitment to base system specifications on this data; systematic prioritization of identified needs (i.e. essential versus desired) takes time and negotiation among stakeholders; pilot testing is of extreme importance; and finally, a strong communications and promotions plan was needed to ensure the successful implementation of the online program and recruitment.

Going forward, the program will be looking to further engage study investigators to take greater advantage of the application features. This will enable greater accuracy and timeliness of case updates. Future plans also include the ability to obtain more detailed reporting trends and look into the possibility of implementing detailed questionnaires into the online system.