Development of indicators for assessing rational drug use to treat community-acquired pneumonia in children in hospitals and clinics: A modified Delphi study.

Community-acquired pneumonia (CAP) is a common infectious disease in children. Rational drug use (RDU) is an important approach to reducing the disease burden and mortality rate of CAP in children. There are no monitoring indicators for assessing RDU in children. This study aimed to develop a set of indicators to assess RDU to treat CAP in children in hospitals and clinics using a modified Delphi method.Initial indicators were generated based on a systematic review of guidelines and studies investigating CAP in children. A 3-round modified Delphi process in the form of an email survey combined with round-table discussion was then carried out, and an analytic hierarchy process (AHP) was applied to determine the weight of each indicator.A total of 24 and 8 experts were invited to participate in the email survey and round-table discussion, respectively. A consensus was reached after 3 rounds of the Delphi survey. Three first-rank indicators and 23 second-rank indicators were developed, and each indicator was weighted. The first-rank indicators comprised drug choice (45.5%), drug usage and dosage (36.4%), and the duration of drug therapy (18.2%); the second-rank indicators were indicators related to antibiotics (63.6%), antiviral agents (18.2%), traditional Chinese medicines (4.5%), and adjuvant drugs (13.6%). The weight value of drug selection was the highest, followed by the values of drug usage and dosage and the duration of drug therapy.The developed indicator set constitutes the first set intended to assess RDU to treat CAP in children in hospitals (including community hospitals) and clinics. The indicators were based on drug selection, drug usage and dosage and duration of drug therapy, which are associated with most therapeutic drugs for CAP in children. Monitoring these indicators will guide people towards the promotion of RDU in the absence of drug monitoring indicators for CAP. Furthermore, the indicator set constitutes a methodological reference for the development of other indicator sets.

Introduction

Community-acquired pneumonia (CAP) is a common infectious disease in children and an important cause of hospital admissions.[ More than 2 million children 0 to 5 years of age die from pneumonia each year, accounting for nearly 1 in 5 deaths in children under 5 years worldwide.[ Moreover, due to limitations in health services, more than 150 million episodes of pediatric pneumonia occur in developing countries every year, accounting for more than 95% of all new cases in the world.[ In 2015, a study published in Lancet suggested pneumonia as a leading cause of deaths in children under 5 years in China: 14.8% of the 6.3 million deaths of children under the age of 5 years resulted from pneumonia.[

Rational drug use (RDU) is an important approach reducing the disease burden and mortality rate of CAP in children. In 1986, RDU was first defined by the World Health Assembly (WHA) as “patients receive medications appropriate to their clinical needs, in doses that meet their own individual requirements, for an adequate period of time, and at the lowest cost to them and their community.”[ Although RDU has attracted increasing attention since it was proposed by the World Health Organization (WHO) 30 years ago, ensuring medication safety and effectiveness is still challenging, particularly in children.[ RDU in children is a global issue, and some indicators have been developed to monitor the rationality of drug use.[ In 2014, our research group conducted a systematic literature review to estimate the existing drug-related indicators,[ and showed that only one of the 42 retrieved RDU indicator sets had been developed for children. Moreover, this indicator set was designed for children in primary care and was thus not suitable for the treatment of a specific disease.

The purpose of this study was to develop a set of indicators to assess RDU for the treatment of CAP in children using a modified Delphi method. These indicators can be applied to outpatient or inpatient hospital departments (including community hospitals) and clinics to assess the rationality of drug use for a given period of time.

Materials and methods

Study design

A modified Delphi survey technique, which is an iterative multistage process designed to transform opinion into group consensus, was implemented to develop this indicator set.[The Delphi process was modified in the form of an email survey combined with round-table discussion.

This study was approved by the Institutional Review Board of West China Second University Hospital.

Development of initial indicators

Initial indicators were developed based on guidelines and studies investigating CAP in children. We searched the Guidelines International Network (GIN) library and the National Guideline Clearinghouse (NGC) for treatment guidelines and PubMed, EMbase, the Cochrane Library and the China Biology Medicine disc (CBM) for studies with a search date of October 2015 and an updated search in October 2016. Two independent observers (GG, JS) selected studies. The included guidelines and studies met the following criteria: (1) patients with CAP between 0 and 18 years; (2) interventions related to drug treatment; (3) guidelines were the latest edition; (4) published in English or Chinese; (5) guidelines that the drug treatment recommendations could be developed indicators (Online appendix 1 shows the search strategy).

Drug treatment recommendations were independently extracted and classified based on the included guidelines and studies by 2 reviewers (WRL and LNZ), and a project group then developed the indicators based on treatment recommendations. For example, the IDSA guidelines[ suggested that antibacterial therapy is not necessary for children, either outpatients or inpatients, with a positive test result for influenza virus in the absence of clinical, laboratory, or radiographic findings suggesting bacterial coinfection. Therefore, we developed the indicator “the proportion of antibiotic use,” which was defined as the number of children who received antibiotics as a percentage of all children, and a too-high or too-low proportion of antibiotic use would serve as a warning for clinical drug use.

Identification of an expert panel

A total of 24 experts were invited to participate in the email survey. Expert selection was based on the Group of People with Highest Risk of Drug Exposure of the International Network for the Rational Use of Drugs (INRUD) in China. Eight experts from 4 hospitals were respectively invited from the eastern, central, and western regions according to the geographical distribution of the INRUD member units;[ each hospital provided 2 experts, specifically 1 clinician and 1 clinical pharmacist.

In addition to the 24 experts, another 8 experts (who were not project group members) from the West China Second University Hospital were invited to participate in the round-table discussion.

Experts included in the Delphi survey met the following criteria: (1) more than 5 years of practice in a pediatric pneumology department; (2) possessed at least an intermediate title (attending doctor or pharmacist-in-charge); (3) were interested and willing to participate in our study; and (4) had no direct conflict of interest with this study.

Delphi process and the weight of each indicator

Three rounds of email surveys and 3 round-table discussions were carried out. In each email survey, the 24 experts were instructed to grade the importance, accessibility, degree of familiarity, judgment of evidence, and degree of influence of each indicator in the questionnaires. An “opinions and suggestions” section was placed at the end of the questionnaire such that each expert could provide their own suggestions. After each email survey, the indicators were discussed in a round-table discussion if the average scores of importance and accessibility were less than 7 in the email questionnaires.

Eight experts from the West China Second University Hospital were invited to participate in each round-table discussion to determine whether the indicators should be added, rejected or modified based on the scores and suggestions in the email survey. The Delphi survey was finished if the experts’ active coefficient, authority coefficient (Cr) and agreement coefficient (ω) all correlated.

The experts’ active coefficient, Cr and ω, were used to evaluate the reliability of the developed indicator set. The active coefficient was the degree to which experts were concerned with this study, represented by the recovery rate of the questionnaires, and Cr was the experts’ degree of authority on the evaluated indicators. Cr≥0.7 indicated a high degree of authority among experts; ω represented the degree of harmony of all evaluated variables among all experts in the Delphi method, and this value ranged from 0 1. Based on several large studies that employed the Delphi method, ω ≥0.4 indicated a good degree of harmony when all evaluating variables among all experts.[

The analysis hierarchy process (AHP) was implemented to give a weight to each indicator in this study. In AHP, the relative weight of an indicator is obtained by constructing a paired comparison matrix, and the weight is calculated by normalizing the elements of each column in a consistent paired comparison matrix.[

Results

Study population

In the first email survey, 22 out of 24 experts, comprising 10 (45.5%) physicians and 12 (54.5%) clinical pharmacists, completed the questionnaire; the response rate was 91.7%. In the second and third email surveys, all 22 experts completed the questionnaire. The response rate was 100% (22/22). All questionnaires returned were valid, and the effective rate of the questionnaire was 100%.

The 22 experts were from 12 provinces or municipalities, specifically Beijing, Shanghai, Jiangsu, Guangdong, Heilongjiang, Shanxi, Hubei, Hunan, Sichuan, Guizhou, Yunnan and Shaanxi. The average age (mean±SD) was 41.77 ± 6.87 years, and all had a bachelor's degree or higher: 8 (36.4%) bachelor's, 10 (45.5%) master's, and 4 (18.2%) doctorates degrees. Twenty-one experts had an intermediate title or higher, 5 (22.7%) held a senior title and 7 (31.8%) held a vice-senior title.

Development of indicators

Seven guidelines[ and 73 studies that met the criteria were included. Thirty-seven indicators, consisting of 4 first-rank indicators and 33 second-rank indicators were developed based on these guidelines and studies and limited to the first-round Delphi survey (Online appendix 2 shows the flow chart for screening).

The 3-round Delphi survey was carried out from January 2016 to October 2016. The indicator development process is shown in Figure 1 (Table 1   shows the sources, calculation formula and outcome of each indicator in the Delphi survey).

Figure 1

Flow diagram of quality indicator development. Initial indicators were generated based on a systematic review of guidelines and studies, a 3-round modified Delphi process was then carried out, and some indicators were added, rejected, or modified in each round Delphi survey.

Table 1

Sources, definition, and outcomes for each indicator in the Delphi survey.

Final indicators and their weights

Three first-rank indicators and 27 second-rank indicators were generated after the 3 email surveys and the 3 round-table discussions. “Drug selection,” “drug usage and dosage,” and “duration of drug therapy” were the first-rank indicators, and the second-rank indicators were related to antibiotics, antiviral agents, traditional Chinese medicines and adjuvant drugs. Among the second-rank indicators, 14 (63.6%) indicators were developed to evaluate antibiotic use, whereas 4 (18.2%), 1 (4.5%), and 3 (13.6) were developed to evaluate antiviral agents, traditional Chinese medicines and adjuvant drugs, respectively. 14 (63.6%) indicators were extracted from the guidelines, whereas 4 (18.2%) were from the studies and 4 (18.2%) were derived from experts’ suggestions.

The reliability analysis results for the final indicators were good: for each indicator, the score of importance and accessibility was greater than 6, Cr≥0.7 (except for 1 indicator: proportion of injections used among children who took traditional Chinese medicines) and ω ≥0.4.

Each indicator was weighted by AHP. The final indicators and the weight of each indicator are shown in Table 2. The weight value is positively related to the importance of the indicators. Among the first-rank indicators, the weight value of drug selection was the highest, followed by the values of drug usage and dosage and the duration of drug therapy. For the second-rank indicators, the weight values of the top 10 indicators are shown in Table 3.

Table 1 (Continued)

Sources, definition, and outcomes for each indicator in the Delphi survey.

Table 1 (Continued)

Sources, definition, and outcomes for each indicator in the Delphi survey.

Discussion

Analysis of the indicators

Using a modified Delphi survey, we developed 25 indicators to assess RDU for the treatment of CAP in children in hospitals (including community hospitals) and clinics. The indicators were based on “drug selection,” “drug usage and dosage,” and “duration of drug therapy,” which is consistent with the 3 important dimensions of RDU, intended to monitor and evaluate actual drug use at the hospitals and clinics levels. Antibiotics, antiviral agents, traditional Chinese medicines and adjuvant drugs are the primary drugs used to treat CAP in children. The developed indicators covered all major drugs and comprised a set of comprehensive quality indicators to monitor and evaluate the drug-use process. Because antibiotics are most widely used, they are also incorrectly used most often.[ Thus, more than half of the indicators were developed to evaluate antibiotic use. The developed indicators are administrative indicators at the hospitals or clinics levels, and the majority of the indicators were developed to monitor the proportion of drug use. Each indicator was weighted and the weight reflects the importance of the indicators. The larger the weight, the more important the indicator. By monitoring the indicators, one can evaluate drug use in hospitals or clinics and provide a warning regarding the rationality of drug use.

In follow-up studies, our research team will conduct cross-sectional studies at hospitals and clinics across the country that are intended to monitor the actual values of the indicators. Then compare and analysis the monitoring results of indicators in the same level (hospital or clinic, outpatient department or inpatient department), and a reasonable range will be provided for each indicator according to the results of the cross-sectional studies and the opinions of experts in related fields.

Thus, this study not only established an indicator set to assess RDU for the treatment of CAP in children but also provided a monitoring method for RDU.

Strengths of this study

The developed set of indicators is the first set of quality indicators aimed to assess the RDU of CAP treatment in children. Although intended for children, the development process and method based on Delphi technology are recognized all over the world. The initial indicators were developed from 2 sources: clinical guidelines and studies. Of these, more than half of the indicators were extracted from guidelines, as guidelines direct disease treatment and are thus the best available evidence for the treatment of CAP. Guidelines and studies provide objective support for the developed indicators. Three indicators were developed based on the suggestions of experts, incorporating clinical practice experience. We included 22 experts based on the Group of People with Highest Risk of Drug Exposure of INRUD in China, evenly distributed among the eastern, central and western regions and 12 provinces. These participants comprised a broad and representative sample and included both medical and drug experts. Finally, the addition, acceptance, rejection, and modification of the indicators in the Delphi survey were based not only on the scores obtained from the email survey but also on the suggestions of another 8 experts in 3 round-table discussions, which made the indicators more accurate than those derived from email surveys alone.

Limitations of this study

Although a standardized method was performed to develop the indicators, the study has several limitations. First, it is challenging to develop a set of indicators that apply to all children diagnosed with CAP in different countries. In our study, most of the indicators were based on guidelines and studies published in English or Chinese, and all the experts came from China. Nevertheless, the guidelines were developed by national academic institutions, such as Oxford University Press on behalf of the Infectious Diseases Society of America and the British Thoracic Society Community Acquired Pneumonia in Children Guideline Group, which are widely recognized in the medical field with respectable authority. Second, AHP was used to weigh the indicators. Although AHP is the most well-known and widely employed multicriteria method, it is also a subjective method determined by experts. The combination of subjective and objective methods not only provides a strong theoretical basis but also employs the practical experience of experts. However, in our study, given the lack of existing objective data, it was impractical to weigh indicators using an objective method, which constitutes a limitation. Finally, because the experts were distributed across the country, questionnaires were sent to experts via email rather than administered as a face-to-face survey. Although detailed explanations were offered in the questionnaire, the experts who completed the questionnaires may not have fully understood the content.

Practical implications

Children comprise one of the highest risk populations that require monitoring for rational drug use. Monitoring these indicators will guide people towards the promotion of RDU in the absence of drug monitoring indicators for CAP. The majority of the indicators were developed from guidelines recognized worldwide such that people in other countries or areas outside of China will be able to directly use or modify them based on actual situations.[ In addition, we provide a complete method for the development of quality indicators of RDU for a variety of diseases. Although intended for children, our method also constitutes a methodological reference for developing other types of indicator sets.

Conclusions

The developed indicator set is the first set intended to assess RDU to treat CAP in children in hospitals (including community hospitals) and clinics, and constitutes a methodological reference for the development of other indicator sets. Through a 3-round modified Delphi process, 3 first-rank indicators and 23 second-rank indicators were developed, and each indicator was weighted. The utility of this indicator set will be tested in further clinical practice.

Table 2

Final indicators and weight of each indicator.

Table 3

Weight of the top 10 indicators among the second-rank indicators.