Development of an evaluation indicator system for the rational use of proton pump inhibitors in pediatric intensive care units: An application of Delphi method.

ABSTRACT: Proton pump inhibitors are frequently used to prevent gastrointestinal bleeding in critically ill patients. But there is little information in the instructions about the usages for children. It is important to monitor the appropriate use of proton pump inhibitors, especially in pediatrics. Therefore, we developed an evaluation indicator system for the rational use of proton pump inhibitors in pediatric intensive care units.First, a systematic review was conducted to developed the initial indicators. Then 2 rounds of Delphi surveys were conducted to collecting opinions from a panel of independent experts, and the indicator system was modified to form the final indicators according to the opinions. Finally, the analytic hierarchy procedure was used to determine the weight of each indicator.A total of 6 guidelines and 2 studies met the inclusion and exclusion criteria. Based on literature and discussion among experts, an initial indicator system including 4 first-rank indicators and 12 second-rank indicators was formed. After 2 rounds of Delphi surveys, 2 indicators were added, 5 indicators were deleted, and 1 indicator was revised, so the final indicator system contained a total of 13 indicators including 3 first-rank indicators (drug selection, drug usage and dosage, duration of drug therapy) and 10 second-rank indicators (the proportion of PPIs used in children, children under 1 year old, children who is using glucocorticoids, children with nonsteroidal anti-inflammatory drugs, children with gastroesophageal reflux disease, children with sepsis, children with ventilators in PICU; the strength of PPIs' use, the proportion of omeprazole in children using PPIs during the same period; the average days of PPIs used in children). By analyzing scores, all coefficients met the standard, indicating the indicators were scientific and credible.Through a two-round Delphi survey, 3 first-rank indicators and ten second-rank indicators were developed, which will help drug administrative departments to promote the rational use of PPIs for children in PICUs. What is more, our study can constitute a methodological reference for the development of other indicator systems.

Introduction

Nearly 30 years after their introduction into clinical practice, proton pump inhibitors (PPIs) remain the mainstay of gastric acid inhibitors. The common mechanism of action is to inhibit the H+-K+-ATPase on the secretion tube of parietal cells, which is the most efficacious method of preventing harmful gastric acid secretion.[ PPIs have prophylactical indications for stress ulcers, taking non-steroidal anti-inflammatory drugs, and other gastrointestinal risk factors.[

Critically ill patients may develop stress-related gastrointestinal mucosal damage, increasing the risk of gastrointestinal bleeding.[ International guidelines recommended prevention for critically ill patients at risk of gastrointestinal bleeding.[ Acid suppressants are frequently used to prevent gastrointestinal bleeding in ICU patients,[ such as PPIs.[ PPIs have been commonly prescribed in pediatrics for 30 years,[ but there is little information in the instructions about PPI usages for children. Children's physiology in critical illnesses may be very different from those of adults,[ so children cannot copy adults’ therapeutic regimen. Meanwhile, there is no conclusive evidence on the benefits or harms of PPIs in critically ill children.[ Guidelines warned against the overuse of antacids,[ which was a common phenomenon,[ leading to unnecessary exposure to adverse effects and inflation of healthcare costs. It is important to monitor the appropriate use of PPIs, especially in pediatrics.[

Healthcare quality was defined as “the degree to which health services for individuals and populations increase the likelihood of desired health outcomes and care consistent with current professional knowledge” by the Institute of Medicine.[ To realize this objective, authorities and health care professionals use a wide range of methods and tools to promote quality improvement, such as the development and implementation of quality indicators in different areas to detect problems in the process of health care.[ Therefore, developing a set of indicators may help for drug administrative departments to promote the rational use of PPIs for children in PICU.

Systematic evaluation is an evidence-based methodology that can process mass information in a scientific and reasonable way. Delphi survey is an exercise in group communication that brings together and synthesizes the knowledge of a group of geographically scattered participants who never meet,[ and its authority has been admitted by the world,[ which has been widely used to development indicator in healthcare. Therefore, we combined the advantage of the systematic review and Delphi survey to develop a set of indicators, to help drug administrative departments promoting the rational use of PPIs.

Aim of the study

In order to promote the rational use of PPIs and detect medication problems in time, we developed an evaluate indicator system of the rational use of PPIs in pediatric intensive care units (PICUs) for drug administrative departments.

Ethics approval

This study was approved by the Institutional Review Board of West China Second University Hospital.

Methods

Development of the initial indicator system

We searched the English databases and Chinese databases, including PubMed, EMbase, the Cochrane Library, the China Biology Medicine disc (CBM), China National Knowledge Infrastructure, WANFANG database and the Chinese Scientific Journal Database. And we also searched the international guideline websites, including the Guidelines International Network library, the National Guideline Clearinghouse (NGC), the National Institute for Clinical Excellence (NICE), the Scottish Intercollegiate Guidelines Network. The search terms were “proton pump inhibitor∗,” “PPI∗,” “Omeprazole,” “Esomeprazole,” “Lansoprazole,” “Rabeprazole,” “pantoprazole,” “child∗,” “pediatric∗,” “paediatric∗,” “toddler∗,” “adolescen∗,” “preschool∗,” “infant∗,” ect (See Table 1 for details).

Table 1

Literature retrieval strategy (Take PubMed as an example).

#1	Child[Mesh]
#2	Pediatrics[Mesh]
#3	Infant[Mesh]
#4	child∗[Title/Abstract]
#5	pediatric∗[Title/Abstract]
#6	Paediatric∗[Title/Abstract]
#7	infant∗[Title/Abstract]
#8	infancy[Title/Abstract]
#9	preschool∗[Title/Abstract]
#10	toddler∗[Title/Abstract]
#11	adolescen∗[Title/Abstract]
#12	or/1-12
#13	Proton Pump Inhibitors/
#14	((Proton Pump Inhibitor∗) or PPI or PPIs).tw.
#15	Omeprazole/
#16	(omeprazole or losec or nexium or prilosec or rapinex or zegerid or Omepral or Omez).tw.
#17	Esomeprazole/
#18	(Esomeprazole or Nexium or Esotrex or Alenia or Escz or Esofag or Nexiam).tw.
#19	lansoprazole/
#20	(lansoprazole or lanzoprazole or agopton or bamalite or Inhibitol or Levant or Lupizole or lanzor or monolitum or ogast or ogastro or opiren or prevacid or prezal or takepron or ulpax or zoton).tw.
#21	rabeprazole/
#22	(rabeprazole or aciphex or dexrabeprazole or pariet or Rabecid or Nzole-D or Rabeloc).tw.
#23	(Dexlansoprazole or Kapidex or Dexilant).tw.
#24	“pantoprazole” [Supplementary Concept]
#25	(pantoprazole or protium or protonix or Pantozol or Pantor or Pantoloc or Astropan or Pantecta or Somac or Pantodac or Zurcal).tw.
#26	or/13-25

The included guidelines and studies met the following criteria. Inclusion criteria:

Study type: The latest edition of guidelines, expert consensus, national or provincial official documents, drug instructions or literature published in the last 5 years;

Population: The ideal study population was children (under 18 years of age), and meanwhile, guidelines and studies that didn’t consider the age of the population were also included as references;

Content: The clinical usage of PPIs (including drug selection, dosage, duration, cost, etc.), without limited to diseases.

Exclusion criteria:

The study population was adults only;

Non-Chinese and English materials;

Case report, news and editorials;

Repeated published literature.

Two researchers screened and extracted the literature independently. When the opinion was different after discussions, the third reviewer made the final decision. Subsequently, the researchers organized the information, including indicators, source, calculation formulation, population, evidence level, etc.

Finally, 4 experts (who were not members of our project team) from the West China Second University Hospital were invited to participate in the round-table discussion to determine an initial indicator system based on existing information. The standard for developing the indicator system:

Indicators could provide certain clinical suggestions for the rational use of proton pump inhibitors, including drug selection, usage, duration, cost and so on;

Indicators were broadly applicable to children in PICU, not just those with specific diseases or conditions;

If guidelines or studies provide relevant data, indicators should have good reliability and validity;

Indicators were supported by certain evidence (such as clinical researches, systematic reviews, expert consensus);

Indicators were easy to obtain or calculate through available data.

Delphi method

Twenty four experts were invited to participate in the Delphi surveys, who were the member of the Group of People with Highest Risk of Drug Exposure of the International Network for the Rational Use of Drugs (INRUD) in China. Principles of expert selection:

Experts with more than 3 years of practice in PICU or pediatric department;

Experts with at least an intermediate title;

Experts who were interested and willing to participate in our study;

Experts without direct conflict of interest with our study.

According to the geographical distribution of the INRUD member units,[ 2 experts from each of 12 hospitals in eastern, central, and western China were invited; the 2 experts included 1 clinician and 1 clinical pharmacist.

Two rounds of Delphi surveys were conducted by E-mail. The questionnaire included background, instructions, general information of the experts, and text content. In text content, the experts were instructed to grade the importance, feasibility and familiarity of each indicator (the score was ranged from 1–10). Meanwhile, the experts need to report the influence of different judgement basis (such as theoretical analysis, practical experience, peer communication, personal intuition) in the questionnaires (the score was ranged from 1–5).[ Experts could give their own suggestions at the end of the questionnaire, such as addition and deletion of indicators along with the reasons. The results of surveys were presented anonymously so that the experts could reevaluate the answers without peer pressure.[

Four experts from the West China Second University Hospital were invited to participate in a round-table discussion after each Delphi survey to translate consensus into indicators. And they discussed indicators with an average score of less than 7 in importance and feasibility, to determine whether the indicators should be added, rejected or modified based on the scores and suggestions in the survey.[

Statistic analysis

Microsoft Excel and SPSS22.0 was used to analyze the data. The experts’ active coefficient, variable coefficient (CV), authority coefficient (Cr) and agreement coefficient (ω) were used to evaluate the developed indicator system. The active coefficient was the degree of experts’ concern with this study, represented by the recovery rate of the questionnaires (active coefficient ≥80% indicated a high degree of concern among experts). The importance and feasibility of indicators were expressed as the mean and standard deviation. CV represented the degree of divergence of views (CV < 0.3 was considered acceptable). Cr was the degree of experts’ authority on the evaluated indicators, calculated according to the experts’ familiarity and judgment basis (Cr ≥ 0.7 indicated a high degree of authority among experts). ω was the degree of consistency among all experts’ opinions in the Delphi method (because the number of experts is ≥7, ω counld be tested by Chi-Squared test, and a statistical significance suggested the consistency was good).

The analytic hierarchy procedure (AHP) was implemented to give a weight to each indicator. In AHP, the relative weight of an indicator was obtained by constructing a paired comparison matrix, and the weight was calculated by normalizing the elements of each column in a consistent paired comparison matrix.[

Results

Development of initial indicators

A total of 6 guidelines and 2 studies were obtained through systematic review, which were from the United Kingdom, France, the United States, and China.[ Based on the standard for developing the indicator system, an initial indicator system was formed after discussions with the 4 experts, including 4 first-rank indicators and 12 second-rank indicators (See Table 2 for details).

Table 2

The process of Delphi method.

Indicators	Calculation formulation	Source	Population	First-round Delphi survey	Second-round Delphi survey
The first-rank indicators
Drug selection	–	WHA	All children in PICU	Accepted	Accepted
Drug usage and dosage	–	WHA	All children in PICU	Accepted	Accepted
Duration of drug therapy	–	WHA	All children in PICU	Accepted	Accepted
Drug cost	–	WHA	All children in PICU	Rejected	–
The second-rank indicators
Drug selection
1. The proportion of PPIs used in children in PICU	Number of children with PPIs/Total number of children	Guideline (PPIs Prescription Evaluation Guide for Medical Institutions in Sichuan Province 2016)	All children in PICU	Accepted	Accepted
2. The proportion of PPIs used in children under 1 yr old in PICU	Number of children with PPIs below 1 year old/Total number of children under 1 year old	Guideline and study (Ward 2013)	All children in PICU	Accepted	Accepted
3. The proportion of PPIs used in children with gastroesophageal reflux disease in PICU	Number of children with gastroesophageal reflux disease using PPIs/Total number of children with gastroesophageal reflux disease	Guideline (Nice2015+AHRQ2013)	All children in PICU	Accepted	Accepted
4. The proportion of children in PICU who used two or more PPIs	Number of children with two or more PPIs/Total number of children with PPIs	Self-made	All children in PICU	Rejected	-
5. The proportion of PPIs used in children who is using glucocorticoids in PICU	Number of children using PPIs who is with glucocorticoids/Total number of children with glucocorticoids	Guideline (France 2008)	All children in PICU	Accepted	Accepted
6. The proportion of PPIs used in children with NSAIDs in PICU	Number of children using PPIs who is with NSAIDs/Total number of children with NSAIDs	Guideline (France 2008)	All children in PICU	Accepted	Accepted
7. The proportion of omeprazole used in children using PPIs in PICU	The number of children using omeprazole/Total number of children with PPIs	Guideline (France 2008)	All children in PICU	Modified: the proportion of omeprazole in children using PPIs in PICU during the same period	Accepted
8. The proportion of PPIs used in children with sepsis in PICU	Number of PPIs in the denominator/Number of children with sepsis in PICU during the same period	Suggestions of experts in Delphi survey	All children in PICU	Added	Accepted
9. The proportion of PPIs used in children with ventilators in PICU	Number of PPIs in the denominator/Number of children using ventilators in PICU during the same period	Suggestions of experts in Delphi survey	All children in PICU	Added	Accepted
10. The incidence of pneumonia in children with PPIs	Number of children with pneumonia in the denominator/Number of children with PPIs in PICU during the same period	Suggestions of experts in Delphi survey	All children in PICU	Added	Rejected
11. The incidence of intestinal infection in children with PPIs	Number of children with intestinal infections in the denominator/Number of children with PPIs in PICU during the same period	Suggestions of experts in Delphi survey	All children in PICU	Added	Rejected
Drug usage and dosage
1. In PICU, the strength of PPIs’ use	PPIs consumption in hospitalized children (cumulative DDD number)/days of children admitted in the same period × 100	Study (Joret-Descout 2017)	All children in PICU	Accepted	Accepted
2. The proportion of intravenous medication in Children using PPIs in PICU	Number of children with intravenous administration using PPIs/Number of children with PPIs	Study (Chen Yong 2016)	All children in PICU	Rejected	–
3. The proportion of one type of PPI used in children using PPIs in PICU during the same period	Number of DDDs for one type of PPI/cumulative DDD for all PPIs in children in PICU for the same period	Suggestions of experts in Delphi survey	All children in PICU	Added	Rejected
Duration of drug therapy
1. The average d of PPIs used in children in PICU	Total therapy time on PPIs/Total number of children treated with PPIs	Guideline (PPIs Prescription Evaluation Guide for Medical Institutions in Sichuan Province 2016)	All children in PICU	Accepted	Accepted
Drug cost
1. The average cost of PPIs among children using PPIs	Total cost of PPIs in children/Total number of children treated with PPIs	WHO	All children in PICU	Rejected	–
2. The percentage of PPIs cost in the total cost for children using PPIs	Total cost of PPIs in children/Total cost of children treated with PPIs	WHO	All children in PICU	Rejected	–

A total of 24 experts comprised 12 (50%) clinicians and 12 (50%) clinical pharmacists who were from 11 provinces or municipalities, specifically Beijing, Jiangsu, Fujian, Heilongjiang, Shanxi, Henan, Hubei, Hunan, Guizhou, Xinjiang and Shaanxi. The average age (mean ± SD) was 38.5 ± 8.95 years, and all had a Bachelor's degree or higher: 7 (29.17%) experts with a Bachelor's degree, 12 (50%) experts with a Master's degree, and 5 (20.83%) experts with a Doctor's degree. In the first round, the response rate was 100% (24/24); in the second round, it was 91.67% (22/24), because 1 clinician and 1 clinical pharmacist did not respond. All questionnaires returned were valid, that is to say, the effective rate was 100%. Therefore, the experts’ active coefficient was satisfactory.

The results of the first round of Delphi survey showed, except the statistical values of 2 indicators (the average cost of PPIs among children using PPIs, the percentage of PPIs cost in the total cost for children using PPIs), the analysis of other indicators showed that the average value of importance (range: 7.08–9.38) and feasibility (range: 7.63–9.29) of the indicators were higher than 7; the values of Cv (range: 0.09–0.29) were lower than 0.3; the values of Cr (range: 0.8–0.85) were higher than 0.7 (See Table 3 for details). Meanwhile, some experts suggested removing or adding certain indicators (See Table 4 for details). After the round-table discussion about the result of the first round of Delphi survey, 1 initial first-rank indicator was deleted, and 12 initial second-rank indicators were revised as follows: 5 indicators were added, 4 indicators were deleted, and 1 indicator was revised (See Table 2 for details).

Table 3

The analysis of the importance, feasibility and authority in the first round of Delphi survey.

	Importance		Feasibility
Indicators	mean ± SD	CV	mean ± SD	CV	Cr
The first-rank indicators
1. Drug selection	9.33 ± 0.96	0.10	9.21 ± 1.02	0.11	0.85
2. Drug usage and dosage	9.38 ± 0.88	0.09	9.29 ± 0.91	0.10	0.85
3. Duration of drug therapy	8.88 ± 1.15	0.13	8.75 ± 1.33	0.15	0.85
4. Drug cost	7.08 ± 1.67	0.24	7.63 ± 2.18	0.29	0.8
The second-rank indicators
Drug selection
1.1 The proportion of PPIs used in children in PICU	8.71 ± 1.81	0.21	8.58 ± 1.74	0.20	0.85
1.2 The proportion of PPIs used in children under 1 year old in PICU	8.29 ± 8.17	0.26	8.17 ± 1.61	0.20	0.85
1.3 The proportion of PPIs used in children with gastroesophageal reflux disease in PICU	8.17 ± 1.69	0.21	7.79 ± 1.44	0.19	0.8
1.4 The proportion of children in PICU who used two or more PPIs	8.42 ± 2.28	0.27	8.71 ± 2.01	0.23	0.85
1.5 The proportion of PPIs used in children who is using glucocorticoids in PICU	8.67 ± 1.93	0.22	8.08 ± 1.56	0.19	0.85
1.6 The proportion of PPIs used in children with NSAIDs in PICU	8.75 ± 1.51	0.17	8.04 ± 1.65	0.21	0.85
1.7 The proportion of omeprazole used in children using PPIs in PICU	8.79 ± 1.28	0.15	8.83 ± 1.37	0.16	0.85
Drug usage and dosage
2.1 In PICU, the strength of PPIs’ use	8.63 ± 1.64	0.19	8.71 ± 1.52	0.17	0.85
2.2 The proportion of intravenous medication in Children using PPIs in PICU	8.04 ± 2.16	0.27	8.09 ± 2.23	0.28	0.85
Duration of drug therapy
3.1 The average days of PPIs used in children in PICU	8.54 ± 1.53	0.18	7.96 ± 1.57	0.20	0.85
Drug cost
4.1 The average cost of PPIs among children using PPIs	7.04 ± 2.18	0.31	7.63 ± 2.37	0.31	0.8
4.2 The percentage of PPIs cost in the total cost for children using PPIs	6.54 ± 2.60	0.4	7.42 ± 2.43	0.33	0.8

Table 4

The list of suggestions or comments from experts in the first round Delphi survey.

Indicator	No. of experts
Indicators suggested to be deleted
1. The percentage of PPIs cost in the total cost for children using PPIs	4
2. The proportion of intravenous medication in children using PPIs in PICU	3
3. The proportion of children in PICU who used two or more PPIs	2
4. In PICU, the strength of PPIs’ use	2
5. The average cost of PPIs among children using PPIs	2
6. The proportion of omeprazole used in children using PPIs in PICU	1
Indicators suggested to be added
1. The proportion of children in PICU who used PPIs for preventive purpose	2
2. The proportion of children in PICU who used PPIs after surgery	1
3. The proportion of PPIs used in children with acute severe pancreatitis in PICU	1
4. The proportion of PPIs used in children with sepsis in PICU for preventive purpose	1
Other suggestions and opinions
1. For children in PICU, receiving mechanical ventilation and suffering disorder of blood coagulation are high risk factors for stress ulcer, so the use of PPIs can be considered. In evaluating reasonableness, this group of patients should be taken into consideration.	1
2. PPIs can be used for preventive purpose during high-dose glucocorticoid shock therapy. So the indicator of “the proportion of PPIs used in PICU children receiving glucocorticoids” should be properly considered to limit the purpose of glucocorticoid.	1
3. Add the relevant indicator of PPIs’ adverse reactions.	1
4. The indicator of “the proportion of PPIs used in PICU children receiving glucocorticoids” should be properly considered to limit the types and dosages of glucocorticoid.	1

The results of the second round of Delphi survey showed, except the statistical values of 3 indicators (the incidence of pneumonia in children with PPIs, the incidence of intestinal infection in children with PPIs, the proportion of PPIs used in children under 1 year old in PICU), the statistical values of other indicators were acceptable: the average value of importance ranged from 7.32 to 9.36; the average value of feasibility ranged from 7.77 to 9.27; the values of Cv ranged from 0.07 to 0.22; the values of Cr ranged from 0.76 to 0.88 (See Table 5 for details). Similarly, some experts suggested removing or adding certain indicators (See Table 6 for details). After the second round of Delphi survey and round-table discussion, 3 second-rank indicators were deleted. So the final indicator system contained a total of 13 indicators, including 3 first-rank indicators and 10 second-rank indicators (See Table 2 for details). The values of ω ranged from 0.32 to 0.47 (P < .001), indicating that the responses of experts had a acceptable consistency with the indicator system (See Table 7 for details).

Table 5

The analysis of the importance, feasibility and authority in the second round of Delphi survey.

	Importance		Feasibility
Indicators	mean ± SD	CV	mean ± SD	CV	Cr
The first-rank indicators
1. Drug selection	9.32 ± 0.82	0.09	9.27 ± 0.75	0.08	0.88
2. Drug usage and dosage	9.36 ± 0.64	0.07	9.23 ± 1.08	0.12	0.85
3. Duration of drug therapy	8.77 ± 1.08	0.12	8.50 ± 1.14	0.13	0.82
The second-rank indicators
Drug selection
1.1 The proportion of PPIs used in children in PICU	8.45 ± 1.30	0.15	8.59 ± 1.03	0.12	0.82
1.2 The proportion of PPIs used in children under 1 year old in PICU	7.73 ± 1.71	0.22	6.59 ± 2.42	0.37	0.75
1.3 The proportion of PPIs used in children with gastroesophageal reflux disease in PICU	8.68 ± 0.97	0.11	8.36 ± 1.40	0.17	0.82
1.4 The proportion of PPIs used in children who is using glucocorticoids in PICU	8.50 ± 1.23	0.15	8.32 ± 1.14	0.14	0.84
1.5 The proportion of PPIs used in children with NSAIDs in PICU	8.05 ± 1.26	0.16	7.95 ± 1.33	0.17	0.81
1.6 The proportion of PPIs used in children with sepsis in PICU	7.68 ± 1.58	0.21	8.14 ± 1.25	0.15	0.80
1.7 The proportion of PPIs used in children with ventilators in PICU	7.32 ± 1.58	0.22	7.77 ± 1.47	0.19	0.76
1.8 The incidence of pneumonia in children with PPIs	6.36 ± 1.52	0.24	6.64 ± 1.58	0.24	0.62
1.9 The incidence of intestinal infection in children with PPIs	6.41 ± 1.75	0.27	6.82 ± 1.59	0.23	0.70
Drug usage and dosage
2.1 In PICU, the strength of PPIs’ use	8.23 ± 1.00	0.12	8.50 ± 1.03	0.12	0.83
2.2 The proportion of one type of PPI used in children using PPIs in PICU during the same period	7.64 ± 1.33	0.17	7.86 ± 1.55	0.20	0.78
Duration of drug therapy
3.1 The average days of PPIs used in children in PICU	8.45 ± 1.06	0.13	8.27 ± 1.24	0.15	0.80

Table 6

The list of suggestions or comments from experts in the second round Delphi survey.

Indicator	No. of experts
Indicators suggested to be deleted
1. The proportion of PPIs used in children with sepsis in PICU	1
2. The proportion of PPIs used in children with ventilators in PICU	1
3. The incidence of pneumonia in children with PPIs	4
4. The incidence of intestinal infection in children with PPIs	3
5. The proportion of one type of PPI used in children using PPIs in PICU during the same period	2
Indicators suggested to be added
1. The proportion of PPIs used in children with gastrointestinal hemorrhage in PICU	1
2. The proportion of PPIs used in children with severe pancreatitis in PICU	1
3. The proportion of children in PICU who used PPIs for preventive purpose	1
4. Incidence of adverse events with PPIs	1
5. The proportion of PPIs used in children with cranial hypertension in PICU	1

Table 7

The analysis of the consistency among experts’ responses in Delphi surveys.

	Importance			Feasibility
	ω	Chi Squared value	P value	ω	Chi square value	P value
The first round	0.47	56.73	.000	0.35	31.9	.001
The second round	0.46	171.15	.000	0.32	122.43	.000

Indicators were weighted by AHP, and the weight values were positively related to the importance of the indicators. Among the first-rank indicators, “usage and dosage” had the highest weight (0.3509), followed by “drug selection” (0.3462) and finally “duration of drug therapy” (0.3030). Among the second-rank indicators, the weight values of indicators ranged from 0.3030 for the average days of PPIs used in children in PICU to 0.0242 for the proportion of PPIs used in children under 1 year old in PICU. The final indicators and the weight values are shown in Table 8.

Table 8

Final indicators and weight of each indicator.

First-rank indicators (weight)	Second-rank indicators (weight)
1. Drug selection (0.3462)	1.1 The proportion of PPIs used in children in PICU (0.0246)
	1.2 The proportion of PPIs used in children under 1 year old in PICU (0.0242)
	1.3 The proportion of PPIs used in children who is using glucocorticoids in PICU (0.0678)
	1.4 The proportion of PPIs used in children with NSAIDs in PICU (0.0511)
	1.5 The proportion of PPIs used in children with gastroesophageal reflux disease in PICU (0.0751)
	1.6 The proportion of PPIs used in children with sepsis in PICU (0.0467)
	1.7 The proportion of PPIs used in children with ventilators in PICU (0.0566)
2. Drug usage and dosage (0.3509)	2.1 In PICU, the strength of PPIs’ use (0.1960)
	2.2 The proportion of omeprazole in children using PPIs in PICU during the same period (0.1520)
3. Duration of drug therapy (0.3030)	3.1 The average days of PPIs used in children in PICU (0.3030)

Discussion

Findings and clinical significance of this study

Through 2 rounds of Delphi survey, the indicators were determined from 3 aspects: drug selection (the proportion of PPIs used in children in PICU, the proportion of PPIs used in children under 1 year old in PICU, the proportion of PPIs used in children who is using glucocorticoids in PICU, the proportion of PPIs used in children with non-steroidal anti-inflammatory drugs in PICU, the proportion of PPIs used in children with gastroesophageal reflux disease in PICU, the proportion of PPIs used in children with sepsis in PICU, the proportion of PPIs used in children with sepsis in PICU), usage and dosage (the proportion of PPIs used in children with sepsis in PICU, the proportion of omeprazole in children using PPIs in PICU during the same period), and duration of drug therapy (the average days of PPIs used in children in PICU).

In a clinical sense, this indicators system was provided a reference for the evaluation of the safety, effectiveness and economy of drug use in children, solving and filling the research blank in the rational use of PPIs for critically ill children. Applying this evaluation indicator system to monitor the rational use of PPIs, can prevent complications of digestive system and reduce unnecessary adverse reactions and economic burden for children in pediatric intensive care units at the same time.

Methodologically, we combined the advantages of evidence-based medicine and expert experience, conducting the systematic review and Delphi method to make the indicators more reliable.[ During the research process, we controlled the quality of the research in multiple links. For example, in the process of systematic review, we ensured the quality of the research results through comprehensive retrieval and double review. At the same time, we had considered the influence of the region, number of experts and experts’ authority on the study, which can help improve the representativeness of experts sample and avoid 1 or several experts dominating the consensus process. In addition, through the AHP to determine the weights, it provided the priority for doctors or pharmacists in practice.

Limitations of this study

Although the Delphi method has many advantages, there are still some limitations. First of all, the Delphi method is hard to avoid some subjective factors, which is the main source of bias risk. Although we provided the source and basis of the indicators, we cannot guarantee the objectivity of the experts’ grading; and especially in the AHP, weights were mainly based on experts’ experience. Consequently, we analyzed the degree of experts’ authority to judge the reliability of the results, and the values of Cr were higher than 0.7, which suggested acceptable reliability of the responses of experts. Finally, we only included the English or Chinese literature with the language limitation and all experts were from China, so this indicator system might not be applicable to other countries.

Conclusions

Through a two-round Delphi survey, 3 first-rank indicators and ten second-rank indicators were developed, which will help drug administrative departments to promote the rational use of PPIs for children in PICUs. What is more, our study can constitute a methodological reference for the development of other indicator systems.

Acknowledgments

The authors gratefully acknowledge the Group of People with Highest Risk of Drug Exposure of INRUD in China and the members of the Delphi panel who provided their expertise and experience—Rongsheng Zhao, Shigang Ding (Peking University Third Hospital), Xing Ji, Qin Zhang (Nanjing Children's Hospital), Huan Gui, Wen Zhong (The First Affiliated of Soochow University), Zhiyi Huang, Hui Xie (Xiamen Maternal and Child Health Hospital), Tianzi Li, Dongyuan Zhou (Harbin Children's Hospital), Ruiling Zhao, Chaohai Wang (Children's Hospital of Shanxi Province), Xiaojing Huang, Xinjing Zhao (He’nan Provincial People's Hospital), Hong Zhou, Lei Li (Tongji Medical College of HUST), Ying Wang, Xingfang Li (The Second Xiangya Hospital of Central South University), Zhen Mao, Jia Ni (Maternal and Child Health Hospital of Guiyang Province), Yuan Li, Jianping Chu(Xi’an Children's Hospital), Daiqin Xiong, ABuduhaer·ABulaiti(The First Affiliated Hospital of Xinjiang Medical University). The authors also gratefully acknowledge Program for Yangtze River Scholars and Innovative Research Team in University (No. IRT0935).

Author contributions

Funding acquisition: Lingli Zhang.

Investigation: Xiaofeng Ni, Mao Lin, Jialian Li, Wenrui Li.

Methodology: Linan Zeng.

Project administration: Linan Zeng, Liang Huang, Deyuan Li, Lingli Zhang.

Supervision: Deyuan Li, Lingli Zhang.

Visualization: Xiaofeng Ni, Mao Lin.

Writing – original draft: Xiaofeng Ni, Mao Lin.

Writing – review & editing: Xiaofeng Ni, Mao Lin.