Predictor variables for post-discharge mortality modelling in infants: a protocol development project.

BACKGROUND: Over two-thirds of the five million annual deaths in children under five occur in infants, mostly in developing countries and many after hospital discharge. However, there is a lack of understanding of which children are at higher risk based on early clinical predictors. Early identification of vulnerable infants at high-risk for death post-discharge is important in order to craft interventional programs.
OBJECTIVES: To determine potential predictor variables for post-discharge mortality in infants less than one year of age who are likely to die after discharge from health facilities in the developing world.
METHODS: A two-round modified Delphi process was conducted, wherein a panel of experts evaluated variables selected from a systematic literature review. Variables were evaluated based on (1) predictive value, (2) measurement reliability, (3) availability, and (4) applicability in low-resource settings.
RESULTS: In the first round, 18 experts evaluated 37 candidate variables and suggested 26 additional variables. Twenty-seven variables derived from those suggested in the first round were evaluated by 17 experts during the second round. A final total of 55 candidate variables were retained.
CONCLUSION: A systematic approach yielded 55 candidate predictor variables to use in devising predictive models for post-discharge mortality in infants in a low-resource setting.

Introduction

Two-thirds of the five million children under five years old who die every year are under the age of one.1 Most of these deaths occur in low and middle-income countries resulting from preventable infectious causes.2,3 Thus, reducing the under-five mortality rate to less than 25 per 1000 live births by the year 2030 as targeted by the United Nations Sustainable Development Goals (SDG) relies on addressing this issue.4

Mortality rates in the months following discharge are often equal to, or greater than during hospitalization.5 Despite this burden, few studies have explored health seeking behavior and mortality or evaluated interventions in children following hospital discharge; none have implemented interventions to improve post-discharge outcomes. One proposed solution to improve post-discharge outcomes is through a precision health approach, whereby vulnerable children in resource-poor countries are identified prior to discharge, ideally at time of admission, through the use of prediction models.6 Using this approach, health systems are better able to deploy scarce resources and life-saving interventions to those most likely to benefit.

Optimal predictive models to inform health systems should be based on candidate predictor variables most likely to be associated with the outcome of interest.7,9. The Delphi process is a well-recognized process that involves the solicitation of a panel of experts through two or more rounds of structured questionnaires, combined with review and modification by the research team.8 The goal is to acquire expert opinions from those outside the primary research team to optimize input and identify context appropriate variables for evaluation in predictor models.

A similar modified Delphi approach was used previously to determine candidate predictors for post-discharge mortality in the 6 months to 5 years age group, which led to a model derivation study and the subsequent development of the Smart Discharge intervention.9–11 However, models currently in use may not be applicable nor optimal for infants less than one year of age due to differences in disease etiologies, physiology, vital signs, and presenting signs and symptoms. Using a similar approach to the prior variable selection process, the purpose of this project was to generate a comprehensive list of candidate predictor variables for infants less than one year of age. The variables identified will be used to derive prediction models for post-discharge mortality in newborns and young infants and will ultimately be included in an expanded Smart Discharges program for Uganda.

Methods

Design

A modified, two-round, Delphi process was performed to determine a potential set of candidate predictor variables for post-discharge mortality in infants less than one year of age. The inability to modify individualized responses based on the aggregate response was the modification to the standard Delphi process. Research Ethics Board approval was obtained from the University of British Columbia.

Participants

Participants were selected based on internal discussions by the primary research team. The desired expertise of participants included pediatrics, sepsis/infectious diseases, microbiology/laboratory medicine, global health, epidemiology, social sciences, neonatology and obstetrics. The target sample size was 25 individuals covering all areas of expertise, and to include multiple participants from the proposed research country, Uganda. The Delphi process does not include a required sample size or formal sample size calculation. However, it has been suggested that between 10 and 50 participants may be ideal, although no consensus has been reached between authors and studies utilizing the methodology.12 At the beginning of each round of the survey, experts were asked to self-identify their area of expertise, role, and affiliation (Table 1).

Table 1

Respondent demographics*

Primary affiliations	Number of respondents (18)
Teaching hospital	11
General/community hospital	2
Pediatric hospital	4
Outpatient clinic	1
University	9
Other	2 (research institute)
Roles at the institutions
Physician	12
Nurse	2
Other clinician	0
Clinical scientist	6
Social scientist	0
Hospital administration	1
Epidemiologist	5
Other
Other areas of expertise
Pediatrics	14
Infectious disease	6
Microbiology/laboratory medicine	0
Global health	9
Epidemiology	4
Social sciences	0
Neonatology	7
Obstetrics	2
Other areas identified: public health, pediatric pulmonology, critical care, internal medicine and surgery, informatics, implementation science, tropical oncology, immunology, emergency

Results based on respondent self-identification. Respondents were able to select all categories that applied; therefore, many identified more than one area.

Process

The two-round modified Delphi process was conducted between August and November 2017 through the use of emailed surveys using Research Electronic Data Capture (REDCap).13 The participants were given 14 days to respond for each round of the process. After each round, the primary research team determined whether or not existing or suggested variables should be added, modified or removed, based on survey responses and the research team's knowledge of the study setting, Uganda. A final list of candidate variables was compiled by the research team following the results of the second survey. Since there was no direct interaction between participants, this was considered a modified Delphi process. However, during the second round of the survey, participants reviewed and critiqued the variables proposed in the first round. The first round further included general questions on expert-perceived rates and importance of post-discharge mortality as a public health issue in resource-poor countries (Table 2).

Table 2

Participant perception of post-discharge mortality rates*

	Post-discharge mortality rates
	>10%	5–10%	2–5%	<2%
Children age 0–1 months	39%	50%	11%	0
Children age 1–12 months	17%	44%	39%	0

within the first six months following discharge in children admitted with an infectious illness in resource-limited countries. Responses gathered pre-survey.

Round 1

An initial list of 37 candidate variables was generated by the research team through a systematic review of existing literature as well as the clinical experiences of the co-investigators (Table 3). This list included multi-part variables (e.g. anthropometric variables included mean upper-arm circumference (MUAC), weight and height, and associated z-scores). Experts were requested to evaluate the initial list of candidate variables based on (1) predictive value (2) measurement reliability (3) availability (4) applicability in low-resource settings. Those variables rated during the first round and selected as suitable for predicting risk in the specified population and setting (Uganda) were incorporated into the final list of candidate variables (Table 5).

Table 3

Round 1 surveyed variables (N=37)

Clinical*	Birth	Laboratory*	Social/Demographic
1. Temperature 2. Respiratory rate 3. Oxygen saturation (SpO2) 4. Age 5. Dehydration (using WHO dehydration scale) 6. Central cyanosis 7. Anthropometrics (weight, height, MUAC) 8. Chest indrawing 9. Bulging fontanel 10. Feeding status 11. Jaundice 12. Multiple associated infectious symptoms (e.g. pneumonia + diarrhea/sepsis/UTI) 13. Grunting 14. Diarrhea 15. Convulsions 16. Abdominal distension 17. Hepatomegaly 18. Capillary refill 19. Coma score (i.e. Blantyre Coma Scale) 20. Other comorbidities (e.g. congenital defect, sickle cell anemia, TB etc.)	1. Birth weight 2. Location of birth (home vs facility) 3. Use of maternal antenatal care 4. Number of weeks gestation at birth 5.	1. Blood glucose 2. Blood culture 3. Blood lactate level 4. HIV status	1. Sex 2. Number of siblings 3. Exclusively breastfed for first 5 months 4. Mother's education (# of years) 5. Immunization status 6. Number of previous hospitalizations 7. Distance from child's home to nearest health facility 8. Mother's age

At time of admission

Table 5

Final list of candidate predictor variables (N=55)

Clinical*	Birth	Laboratory*	Social/Demographic
1. Temperature 2. Respiratory rate 3. Oxygen saturation (SpO2) 4. Age 5. Dehydration (using WHO dehydration scale) 6. Anthropometrics (weight, height, MUAC) 7. Chest indrawing 8. Bulging fontanel 9. Feeding status 10. Jaundice 11. Multiple associated infectious symptoms (e.g. pneumonia + diarrhea/sepsis/UTI) 12. Grunting 13. Diarrhea 14. Convulsions 15. Abdominal distension 16. Capillary refill 17. Coma score (i.e. Blantyre Coma Scale) 18. Other comorbidities (e.g. congenital defect, sickle cell anemia, TB etc.) 19. History of weight gain/weight loss 20. Pallor 21. Oral/motor coordination impairment 22. Hypotonia 23. Spasticity 24. Blood in stool (dysentery) 25. History of cough for two or more weeks 26. Duration of present illness	1. Birth weight 2. Location of birth (home vs facility) 3. Use of maternal antenatal care 4. Number of weeks gestation at birth 5. Mother's age at time of birth of first child 6. History of birth asphyxia 7. Mode of delivery (vaginal vs caesarean section) 8. History of resuscitation after delivery 9. Details of umbilical cord care at/after birth (ex: cutting, cleaning practices, cultural practices, etc.)	1. Blood glucose 2. Blood lactate level 3. HIV status 4. Hemoglobin 5. Sickle cell/thalassemia status	1. Sex 2. Number of siblings 3. Exclusively breastfed for first 5 months 4. Mother's education (# of years) 5. Immunization status 6. Number of previous hospitalizations 7. Distance from child's home to nearest health facility 8. Mother's age 9. Mother is acutely ill (at time of admission) 10. Mother is chronically ill (HIV, TB, mental illness, etc.) 11. Mother has died 12. Primary caregiver (at home) other than mother (e.g. father, grandmother, aunt) 13. Primary caregiver (during admission) other than mother (e.g. father, grandmother, aunt) 14. Mode of transport to health facility (e.g. by foot, public transport, private car) 15. Parental substance use (e.g. smoking, alcohol consumption)

In addition to rating the candidate variables, participants were encouraged to comment on and suggest additional variables for inclusion in the second round. The research team considered each proposed variable and eliminated those considered redundant (e.g. malnutrition can be determined from MUAC or weight for age z-score). The revised list of potential candidate variables suggested through the first survey became the basis of the second round.

Round 2

The second round evaluated 27 new variables using the same criteria utilized in the first round. Participants were again encouraged to comment on each of the variables under evaluation. However, the second round disallowed for suggestion of further variables. The primary research team again utilized the results from the second round to retain, modify or eliminate the additional candidate predictor variables, and incorporated the selected variables into the final list of candidate variables.

Analysis

Each candidate variable was scored by participants as having (1) high, (2) moderate, (3) unlikely, or (4) no applicability for each of the four criteria previous described. Responses were tabulated and reported using descriptive statistics (Microsoft Excel, Seattle, WA). The proportion of respondents who scored a variable as highly applicable was of primary interest to the research team. The proportion of respondents indicating unlikely or no applicability was also of note.

Results

This modified Delphi process included 18 participants from low, middle and high-income countries (including Kenya, South Africa, Bangladesh, Malawi, Uganda, Canada, United States) (Table 1).

These participants identified roles as physicians (12), nurses (2), clinical scientists (6), hospital administrators (1), and epidemiologists (5), with expertise in areas including pediatrics, infectious disease, global health, epidemiology, neonatology, and obstetrics. During the survey period, participants evaluated 37 candidate variables during round 1 (Table 3), and resulted in an additional 27 candidate variables during round 2 (Table 4), resulting in a final list of 55 candidate variables to be used on subsequent post-discharge mortality prediction modeling research (Table 5).

Table 4

Proposed additional variables and subsequently surveyed variables in round 2

Category	Proposed Additional Variables (N=26)	Round 2 Surveyed Variable (N=27)
Clinical Variables	Weight gain/history of weight loss	History of weight gain/weight loss
	Pallor/anemia (marker of malnutrition/chronic illness)	Pallor at time of admission
	Oral/motor coordination/impairment (predisposing to malnutrition/dehydration/aspiration)	Oral/motor coordination impairment
	Malnutrition	Not included as a new variable to survey, as it is evaluated under previous variables “anthropometrics, feeding status, and abdominal distention”
	Specific comorbidities	Not included as a new variable to survey, as it is evaluated under previous variable “other comorbidities”
	Hypotonia/spasticity (restrictive lung disease/aspiration/poor nutrition)	Hypotonia at time of admission
		Spasticity at time of admission
	Blood in stool (dysentery)	Blood in stool (dysentery)
	Cough of two or more weeks	History of cough for two or more weeks
	Length of illness prior to admission/long duration of illness	Duration of present illness at time of admission
Birth Variables	Perinatal infection	History of perinatal infection (except HIV)
	Birth asphyxia	History of birth asphyxia
	Mode of delivery and color of baby at birth	Mode of delivery (vaginal vs. cesarean section)
		Skin color at birth to detect hypoxemia, anemia, infection, stress etc.
	Any resuscitation/treatment needed immediately after delivery	History of resuscitation after delivery
	Umbilical cord practices (risk of tetanus and sepsis)	Details of umbilical cord care at/after birth (e.g. cutting, cleaning practices, cultural practices, etc.)
Laboratory Variables	Hemoglobin	Hemoglobin at time of admission
	Platelets	Platelet count at time of admission
	Urea/creatinine	Urea/creatinine at time of admission
	White cell count	White blood cell count at time of admission
	Sickle cell/thalassemia status	Sickle cell/thalassemia status
Social/demographic Variables	Deceased or sick mother	Mother is acutely ill (at time of admission)
		Mother is chronically ill (HIV, TB, mental illness, etc.)
		Mother has died
	Resources/salaries/health insurance	Family wealth index
		Child is covered under a health insurance plan
	Maternal immunization status (such as tetanus)	Already captured by maternal chronic and acute illness, and child's comorbidities
	Exclusive BF vs BF + supplementation vs supplementation as well as type of supplementation (cow milk vs formula)	Not included as a new variable to survey, as it is evaluated under previous variable “exclusively breasted for first 5 months”
	Caregiver other than mother (e.g. father, grandmother, aunt)	Primary caregiver (at home) other than mother (e.g. father, grandmother, aunt)
		Primary caregiver (during admission) other than mother (e.g. father, grandmother, aunt)
	Mode of transport	Mode of transport to health facility (e.g. by foot, public transport, private car)
	Smoking/drinking (behavioral factors)	Parental substance use (e.g. smoking, alcohol consumption)

Eighty-nine percent and 83% of participants rated post-discharge mortality as ‘very important’ for children 0–1 months in age children 1–12 months, respectively (Table 2). Ninety-four percent of experts surveyed thought that, in comparison to other public health issues in resource poor countries, the current allocation of resources for post-discharge care of children under the age of 1 was ‘very inadequate’. Responses varied when asked about what they believed the post-discharge mortality rate was in the first 6 months following discharge for children admitted with an infectious illness in resource-limited countries. For children age 0–1 month of age, respondents generally believed mortality rates to be between 5% and greater than 10%. For children age 1 to 12 months, the majority chose between 2–10%.

A total of 18 participants completed round 1 (Table 1). Each survey question received between 16 and 18 responses, out of a possible total of 18, since not every question received a response by each participant.

Predictive value

Responses varied from “no applicability” to “high predictive value”; however, only two variables (temperature and abdominal distension at admission) received a response indicating no applicability. The majority of responses ranged from “unlikely” to “high predictive value”. Those receiving the most responses (>70% of respondents) for high predictive value were: anthropometrics, coma score at admission, comorbidities (e.g. congenital defect, sickle cell anemia, tuberculosis) birth weight, number of weeks gestation at birth, HIV status (94%), immunization status, and number of previous hospitalizations. Gender and the number of siblings received less than 10% high predictive responses as well as the highest amount of responses for unlikely predictability. Those variables deemed to be the least predictive for mortality (>30% unlikely responses) included temperature at admission, jaundice at admission, and abdominal distention at admission. Comments made by participants indicated that predictive value for many variables may itself vary by the age of the child, with some being more pertinent for the younger infant and vice versa. For example, birth weight was indicated in a comment to potentially be more predictive in the younger neonate than in those approaching one year of age.

Measurement reliability

None of the variables received responses indicating no applicability for measurement reliability (inter- and intra-rater reliability). High measurement reliability was indicated by at least 70% of expert respondents for the variables gender, HIV status, blood lactate level at admission, location of birth (home vs facility), birth weight, age, and oxygen saturation (SpO2) at time of admission. Furthermore, the variables rated as less reliable (i.e. fewer than 20% rated as highly reliable)included dehydration (using WHO dehydration scale), jaundice, multiple associated infectious symptoms and abdominal distension at admission, other comorbidities (e.g. congenital defect, sickle cell anemia, TB etc.), and number of weeks gestation at birth. Most variables received ratings of moderate measurement reliability. The variables receiving more than 30% responses for unlikely reliability included central cyanosis and multiple associated infectious symptoms at time of admission (e.g. pneumonia and diarrhea). Experts repeatedly commented that reliability would diminish when intense staff training is required for variables, or when recall of memory is needed in the face of unreliable or unavailable medical records.

Availability

No respondents rated any variable as being unavailable; responses varied between unlikely to high availability within low-resource settings. Those deemed highly unlikely to be available were blood culture at admission (94%), blood lactate level at admission (88%), and oxygen saturation (SpO2) at time of admission (56%). Most available variables (as indicated by >75% responses as highly available) included signs and symptoms such as bulging fontanel, grunting, diarrhea and convulsions at admission, and location of birth (home vs facility), gender, number of siblings and mothers age. Excluding HIV testing, no respondents rated the proposed laboratory variables as highly available and instead overwhelmingly rated them as unlikely. Availability was considered as something that was mainly dependent upon consistency of supplies and training of personnel. Those variables requiring least supplies and training were regarded as highly available whereas those needing specialized equipment or supplies generally deemed as largely unavailable and increasing staff training were scored lower.

Time and material resources required

Variable ratings varied from unlikely (high amounts of resources required) to highly applicable (fewer resources required); no variables were deemed to be absolutely unavailable in terms of time and material requirements. Those indicators receiving the highest applicability (fewest resources required) included mother's age (94%), age of the child (89%), sex (88%), convulsions at admission (89%), and chest indrawing at admission (89%). Overwhelmingly, laboratory variables (excluding HIV status) were those deemed to require the most time and material resources. Variables were overwhelmingly rated as requiring few resources; however, comments suggested that even when supplies themselves may have been deemed largely available (e.g. weight scale, thermometer), the need for maintenance and calibration precluded them from being rated as highly available in terms of time and material resources.

Proposed new variables

Twenty-six new variables were proposed, which when separated into individual parts, and overlap and similarities removed, yielded the 27 variables included in the second round (Table 4).

Participants comprised 17 of the 18 who completed round one, with the same self-described areas of expertise and roles (Table 1). Response rates for survey questions varied between 13 and 17 responses, out of a possible total of 17.

The 27 variables evaluated during the second round had varying perceived predictive value, ranging from noapplicability to high applicability. Most responses indicated moderate predictability. The variables receiving the most scores of “high applicability” included the child's mother having died (88%), mother being chronically ill (e.g. HIV, TB, mental illness) (69%), and history of birth asphyxia (67%). Those variables receiving the lowest ratings for predictability with high response rates of “unlikely predictability” included mode of delivery, skin color at birth, and platelet count at admission. Health insurance coverage for the child was the only variable to receive modest ratings of no applicability by experts (27%). Expert comments suggested that predictive value for indicators may be influenced in some case by objectivity or subjectivity of the measure (e.g. pallor) as well as the specificity with which the predictor indicates mortality (e.g. the implications of the presence of hypotonia varies from insignificant to grave consequences).

The variables rated as having high measurement reliability by over 75% of respondents included mode of delivery, hemoglobin, and a deceased mother. Blood in stool (dysentery), duration of present illness, mode of delivery, history of resuscitation after delivery, lab values (including hemoglobin, platelet count, urea/creatinine, white blood cell count) at time of admission, mother deceased, acutely ill, or chronically ill (at time of admission), child covered under a health insurance plan, primary caregiver (at home or admission) other than mother, and mode of transport to health facility all had 80% of respondents rate it as having moderate to high measurement reliability. Oral/motor coordination impairment (53%), skin color at birth (67%), and parental substance use (44%) were largely rated as unlikely to be measured reliably. One variable (pallor at admission) was rated as having no predictive value by one expert only. Experts once again continued to comment on the important role of training requirements for staff (e.g. oral/motor coordination impairment) on measurement reliability and the potential impact of memory on variables (e.g. history of resuscitation) in settings where no medical record is available.

More than 75% of respondents rated primary caregiver (during admission) other than mother, deceased mother as highly available. Laboratory values, including platelet count, urea/creatinine, and sickle cell/thalassemia status was not rated as highly available by any expert. Those receiving at least 85% of responses indicating moderate to high availability included the following at time of admission: pallor, hypotonia, spasticity, blood in stool (dysentery), history of cough for two or more weeks, duration of present illness, mode of delivery, mother is acutely or chronically ill, deceased mother, primary caregiver (at home or during admission) other than mother, and mode of transport to health facility. No variables received ratings indicating an absolute lack of availability. However, those consistently rated as unlikely (by greater than 60%) included urea/creatinine at time of admission, sickle cell/thalassemia status, and child is covered under a health insurance plan. Experts further commented that although variables may have predictive value, their availability limits the extent to which they are useful within low-resource settings.

Those variables rated as requiring the fewest resources to acquire (by at least 70% of respondents) included information on a deceased mother and pallor at admission. All clinical and sociodemographic variables (excluding oral/motor coordination impairment, family wealth index and health insurance coverage), history of birth asphyxia, history of resuscitation after delivery, and umbilical care were rated highly (by at least 85% of respondents), indicating low to moderate amounts of time and material requirements. Although some variables were rated as requiring moderately few resources, many (especially laboratory variables) were rated to require large amounts of resources. Experts further emphasized that laboratory tests relied on resources that were not available in most resource-limited settings. Comments also indicated that amounts of resources required depended on diagnostic approaches; for example, confirmation of perinatal infection would be resource-intensive if it relies on culture techniques rather than on clinical signs and symptoms.

Final list of candidate predictor variables

The final list of 55 candidate predictor variables accepted through the modified Delphi process (Table 5) were identified through expert opinion together with real-time considerations of budget and availability at the proposed research site. The variables identified will be included in future research aimed at establishing prediction models in Uganda.

Discussion

A modified two-round Delphi process conducted using experts from a variety of relevant backgrounds yielded a list of 55 candidate predictor variables to be utilized in the development of a predictive model for post-discharge mortality in infants in resource-limited settings. The clinical presentation of infants is likely to differ from older children due to developmental characteristics and physiology, thus assessment of risk requires an age-specific set of predictors. The differences between pertinent potential risk-factors for the infant versus the older child are further solidified based on variables selected during the Delphi process from all categories including clinical, laboratory, birth, and social/demographic, in comparison to those previously identified for the older child.9

A major strength of this study is the inclusion of experts from multiple pertinent fields. The unique and relevant knowledge participants applied to their evaluation of proposed variables provided additional candidate variables not previously considered by the research team. The breadth of experience resulting from the inclusion of participants with a broad range of expertise also helped ensure that the identified indicators were appropriately evaluated within the clinical context in which they would be utilized. The participants' experiences, knowledge, and understanding of policies, practices, procedures, and availability of personnel and resources in the proposed research country ensured that variables are indeed practical for the research and implementable within the proposed prediction models.

The selection of final candidate prediction variables was both objective and subjective, incorporating results of the Delphi survey process as well as the considerations of the primary research team in terms of relevance and feasibility. Availability at the proposed research site and resource requirements (including time, personnel, and monetary constraints) were central to the acceptance or elimination of proposed factors. For example, although 82% of respondents rated blood culture as having a moderate to high strength for predicting mortality, practical considerations at the proposed research site, as well as the unavailability of this variable as most of the targeted sites for future implementation, negated it from being an included variable. Furthermore, although a variable may have had a lower score for predictive value in the Delphi process, its easy accessibility and measurement may have enabled it to be included as a variable to be considered (e.g. mode of delivery).

The predictor variables identified through this modified Delphi process will be utilized in a model derivation study to predict death post-discharge in children less than one year of age, admitted with infectious illness. By doing so, the limited resources available may be channeled to those children at high-risk for mortality. The process outlined within this paper, coupled with the planned future research and derivation of prediction models, has been recognized as a form of precision public health.6 Precision public health has been proposed as an ideal framework to utilize in decreasing post-discharge mortality, as it improves the efficacy of public health interventions through using precise data, focusing on those who would benefit the most. Although precision public health has started to gain momentum high income countries, its effect in low-resource countries has yet to be fully explored, and yet the potential to impact child mortality could be significant. If vulnerable children can be identified during the admission through the use of risk stratification, effective interventions can be developed and implemented to target those children. Ongoing work in Uganda, described as Smart Discharges, has demonstrated that using prediction to identify high-risk children, paired with interventions including comprehensive discharge teaching and referrals for routine follow-up, could potentially reduce mortality in children (6 months to 5 years of age) during the critical post-discharge period.11 As an extension of this work, the predictor variables identified within this process will focus on predictive modeling for children under one year.

There are several limitations of this study. A primary limitation of this process is the lack of participants' ability to modify responses based on the responses of other experts. Although discussion of specific variables among the experts was not available, an opportunity to comment on each variable was provided to facilitate any questions, comments, or further clarification needed by the variable definition proposed. These comments were seen by the primary research team and discussed, thus allowing the research team to carefully weigh decisions related to inclusion and exclusion of variables. Furthermore, the participants included experts from both developing as well as developed countries as a means of eliciting a wide range of expertise and viewpoints; however, not all experts had an in-depth or practical understanding of the clinical context within the proposed country of study. Expert responses, therefore, based upon what their known context, may have resulted in variables being rated as having lower impact than they actually do, and vice versa. While this diversity may have created some heterogeneity in responses, these diverse options, however, strengthened the ability of the research team to make informed decisions regarding the final list of candidate predictor variables.

Conclusion

The modified Delphi process contributed to the evaluation and identification of potentially useful predictor variables for post-discharge mortality among infants. It helped broaden the selection of variables obtained from on a systematic review and brought objectivity and insight to aspects of predictive value, reliability, availability, and applicability in low-resource settings. The identified variables are a valuable starting point for the construction of a predictive model to identify at-risk infants, who may then be able to benefit from specific interventions aimed towards reducing mortality. Low-resource settings demand that the vulnerable be identified and resources allocated accordingly. The variables identified are an important step towards the goal of reduced childhood mortality.