Improving timeliness in the neglected tropical diseases preventive chemotherapy donation supply chain through information sharing: A retrospective empirical analysis.

BACKGROUND: Billions of doses of medicines are donated for mass drug administrations in support of the World Health Organization's "Roadmap to Implementation," which aims to control, eliminate, and eradicate Neglected Tropical Diseases (NTDs). The supply chain to deliver these medicines is complex, with fragmented data systems and limited visibility on performance. This study empirically evaluates the impact of an online supply chain performance measurement system, "NTDeliver," providing understanding of the value of information sharing towards the success of global health programs.
METHODS: Retrospective secondary data were extracted from NTDeliver, which included 1,484 shipments for four critical medicines ordered by over 100 countries between February 28, 2006 and December 31, 2018. We applied statistical regression models to analyze the impact on key performance metrics, comparing data before and after the system was implemented.
FINDINGS: The results suggest information sharing has a positive association with improvement for two key performance indicators: purchase order timeliness (β = 0.941, p = 0.003) and-most importantly-delivery timeliness (β = 0.828, p = 0.027). There is a positive association with improvement for three variables when the data are publicly shared: shipment timeliness (β = 2.57, p = 0.001), arrival timeliness (β = 2.88, p = 0.003), and delivery timeliness (β = 2.82, p = 0.011).
CONCLUSIONS: Our findings suggest that information sharing between the NTD program partners via the NTDeliver system has a positive association with supply chain performance improvements, especially when data are shared publicly. Given the large volume of medicine and the significant number of people requiring these medicines, information sharing has the potential to provide improvements to global health programs affecting the health of tens to hundreds of millions of people.

Introduction

Public-private partnership programs provide medicines for preventive chemotherapy (PC) through mass drug administration (MDA) campaigns to more than one billion people annually. The programs are sustained by large-scale donations from major pharmaceutical companies in support of the World Health Organization’s 2012 “Roadmap to Implementation,” which outlined global strategies and 2020 targets to control, eliminate, and eradicate Neglected Tropical Diseases (NTDs) [1]. While significant progress was made towards these 2020 targets, the WHO has recently released a new NTD roadmap with 2030 targets, which many pharmaceutical manufacturers have committed to continuing to support [2]. As of January 2020, 15 billion doses of medicines were donated towards these PC-NTD programs [3]. The donations from pharmaceutical companies are what makes these the world’s largest and most successful public health programs [4]. MDA campaigns are comprised of once or twice-a-year treatment with one or more medicines at the community level that bring together a number of stakeholders, requiring considerable coordination as they typically involve treating hundreds of thousands to millions of patients in endemic regions within entire countries over the course of days or weeks [5].

The logistics involved to make these medicines available to support MDAs is both critical and complex, due to the many stakeholders and partnerships involved in meeting the targeted treatment date. Fig 1 provides an overview of the processes involved and performance measures associated with each link in the supply chain. With the considerable resources and coordination involved within a narrow timeframe, inefficiencies caused by fragmented data systems and a lack of visibility to supply chain performance have resulted in substandard performance for on-time delivery of medicine to in-country central medical stores. Sometimes delivery targets may lag as much as 40% below the WHO target for 80% of all shipments to be delivered at least one month before the planned MDA date [6]. Delivery delays result in waste, increased program costs, and delayed or even completely missed MDAs, leaving communities susceptible to infections and disease recrudescence [7].

Fig 1

Components of the NTD supply chain from application for donation to in-country delivery to central medical stores.

To improve the efficiency and performance of NTD supply chain, the NTD Supply Chain Forum was established in 2012 [3]. The NTD Supply Chain Forum is comprised of NTD supply chain experts from the WHO, pharmaceutical companies, nongovernmental organizations, donor organizations, ministries of health, and logistics providers [3]. Subsequently to the formation of this forum, “NTDeliver,” a centralized information system, was launched in 2016 by the NTD Supply Chain Forum to share data from various partners along supply process chain as means of facilitating performance information sharing [8]. Through NTDeliver, all stakeholders in the supply chain—and even the general public—may access performance metrics on all shipments of the four medicines that participate in data sharing through NTDeliver.

Information sharing in the context of supply chains—“the extent to which crucial and/or proprietary information is available to members of the supply chain”—is an integral aspect of performance management and the sharing of accurate and timely information has been linked to supply chain performance improvements [9,10]. Many studies have been conducted on the value of information sharing to improve supply chain performance in the private sector. Information sharing has been proven to have a range of benefits, from improved resource utilization to reduced cycle time between order and delivery [11]. These benefits stem from the increased transparency that enable risks to be anticipated and shared among supply chain partners, which strengthens coordination to achieve optimal operational performance [11-13]. Largely, this body of literature with empirical studies investigates the value of information sharing in a commercial supply chain focused on a dyadic relationship, primarily between two partners, a buyer and a supplier [14].

Despite the growing importance of supply chain initiatives in the global NTD agenda, there is limited research dedicated to exploring measures to improve NTD supply chain performance nor the impact of information sharing. This limitation may be an indicator that performance measurement and management systems have not been widely developed and systematically implemented as part of the overall humanitarian supply chain strategy [15]. Only Korpoc et al’s 2015 research on the impact of the NTD “first mile” processes (the segment of the NTD supply chain covering up to delivery to central medical stores) on MDA timeliness acknowledges this area of NTD supply chain performance measurement by identifying the need for performance indicators and outlining suggested metrics [7]. Furthermore, the recent COVID-19 pandemic has raised the profile both of the criticality of publicly sharing timely data in the global health domain and the topic of assuring robust supply chains to meet global health goals [16-21]. Thus, the timing could not be better to study information sharing and supply chain management in the wider global humanitarian health context.

This study seeks to evaluate empirically the impact of information sharing via NTDeliver on supply chain performance—improvements which ultimately contribute to achieving the global NTD targets. We examine the following two research questions: 1) what is the impact of information sharing through NTDeliver on the performance of the NTD PC medicines donation supply chain? and 2) what is the impact on the performance when country-level data are made publicly accessible? We use data obtained from the NTD Supply Chain Forum and implement regression models on these research questions. We find that information sharing has a positive association with improvement to two performance indicators of the NTD supply chain: purchase order timeliness and delivery timeliness. Furthermore, when country-level information sharing is made publicly accessible, a positive association is observed primarily on three downstream indicators: shipment timeliness, arrival timeliness, and delivery timeliness.

Methods

Study design and scope

We used retrospective data from NTDeliver that are routinely collected from and managed by supply chain partners supporting delivery of PC medicines from pharmaceutical manufacturing facilities to central medical stores. Permission was granted to use this data by the NTD Supply Chain Forum. The data are derived from vetted, existing data sources managed by NTD supply chain partners, such as:

WHO Preventive Chemotherapy and Transmission databank

Data provided to WHO country offices by the countries’ Ministry of Health through the joint application process to request donations

Purchase orders raised by the WHO headquarters

Shipping documents generated by logistics service providers, in partnership with pharmaceutical donors

The data represent shipments of four medicines from four manufacturers to treat three different diseases, accounting for almost 11.5 billion doses of PC medicines to 103 recipient countries covering 1,484 total shipments from February 28, 2006, to December 31, 2018. The data are refreshed and uploaded from these various sources daily [22].

While there are numerous medicines for NTDs donated by numerous pharmaceutical manufacturers for the NTDs, this research’s scope focuses on PC medicines donations managed by the WHO through the “joint application package” (JAP) established in 2013, which supports an integrated review and subsequent reporting on medicines usage [23]. The JAP streamlines the application for donation of multiple medicines, especially as medicines are co-administered where diseases are co-endemic [23]. These PC medicines include: diethylcarbamazine citrate, albendazole, mebendazole, and praziquantel [24]. This focus is justified by the considerable volume of medicines, the unique nature of this supply chain that includes WHO involvement, the importance of PC to achieve NTD targets, and accessibility of data through NTDeliver. There are opportunities to improve processes across the supply chain, but the focus of this research will be on the segment of the supply chain that entails delivery from pharmaceutical manufacturing facilities to central medical stores, also referred to by partners as the “first mile” [7]. We chose to focus on the first mile due to the WHO drive to improve on-time delivery to central medical stores, the accessibility of relevant data, and opportunity to leverage information sharing among the many partners involved in this segment. Central medical stores (CMS) (most commonly utilized in Africa, Asia, and Latin America) serve as a warehouse and administrative facility that receives, stores, and manages medical supplies for national health programs and initiatives and are generally leveraged for humanitarian stock [6,25]. Improving the on-time delivery of PC medicine to the CMS is critical to the downstream in-country distribution to program sites where they are needed [6]. While the last mile (which encompasses drugs transport from central medical stores to district-level stores then on to MDA distribution) certainly involves many challenges that contribute to MDA delays, issues with the first mile have shown in practice and research to have a downstream effect on the last mile and ultimately on MDA timeliness [3,7]. Thus, alleviating first mile issues will be a step in the right direction to improve MDA timeliness.

Variables

The variables for this analysis are various key performance indicators (KPIs), which are actively reviewed through the NTD Supply Chain Forum and of interest to the supply chain partners, including the WHO. The most critical KPI is delivery timeliness, by which the WHO evaluates performance of this first mile of the NTD supply chain [6]. Both the independent and dependent variables were created from the data in the system, reflecting the KPIs and benchmarks standards tracked by the NTD Supply Chain Forum partners. Table 1 summarizes the key variables in the analysis.

Table 1

Variable definitions.

Variable type	Label	Characteristics/description
Independent	NTDeliver	Binary (or dichotomous) variable distinguishing whether the donation order has been tracked in NTDeliver
	NTDeliver public access	Binary (or dichotomous) variable distinguishing whether country-level data was publicly accessible in NTDeliver
Dependent	PO timeliness	Continuous variable measuring the timeliness of the Purchase Order (PO) provided by the WHO HQ to initiate the order for the pharmaceutical donation; measured in months
	Go signal timeliness	Continuous variable approval from the WHO for the order to be shipped, once the needed documentation is prepared; calculated by finding the number of days between go signal request date and go signal approval date; measured in days
	Shipment timeliness	Continuous variable measuring the timeliness of the order being shipped from the manufacturing site; calculated by finding the number of months between the shipment date and the MDA; measured in months
	Arrival timeliness	Continuous variable measuring the timeliness of the arrival of the shipment into the country; calculated by finding the number of months between the arrival date and the MDA; measured in months
	Delivery timeliness	Continuous variable measuring the timeliness of the shipment’s arrival to the delivery warehouse; calculated by finding the amount of time between the delivery date and the MDA; measured in months
Control	WHO region	Categorical variable which identifies the WHO region the ordering country is associated with; regions include Regional Office for Africa (AFRO), Regional Office for the Americas (AMRO), Regional Office for the Eastern Mediterranean (EMRO), Regional Office for Europe (EURO), Regional Office for Southeast Asia (SEARO), Regional Office for the Western Pacific (WPRO)
	International Logistics Performance Index (LPI)	Continuous variable which provides the score from the benchmark tool that the World Bank manages to measure performance of the logistics supply chain within a country, providing a qualitative evaluation by trading partners of the country’s logistics performance
	Medicine type	Categorical variable which identifies the specific donated medicine ordered; four medicines included: albendazole (ALB), diethylcarbamazine citrate (DEC), mebendazole (MEB) and praziquantel (PZQ); each specific medicine is associated with one pharmaceutical manufacturer, but may be used to treat more than one disease
	Disease treated	Categorical variable which identifies the disease(s) the medicine is used to treat; three diseases included: lymphatic filariasis (LF), schistosomiasis (SCH), or soil-transmitted helminthiases (STH)
	Order size	Categorical variable which identifies number of tablets ordered; segmented into three groups: ≥10M; <10M and ≥1M; <1M
	Mode of shipment	Categorical variable which identifies the mode of shipment for the given order; one of three modes identified for each order: air, sea, or land

Important cofactors, incorporated via control variables, were included in the analyses to explore whether the relationship of the independent and dependent variables is skewed or invalidated by other factors. The WHO region was also included as a control since performance may vary according to the destination (shipment routes and customs clearance processes vary according to the destination) and to account for any regional-level improvement initiatives that could also explain performance improvements. Controlling for region may be considered more meaningful than controlling for country since regional-level improvements are more likely to impact the results as regional efforts likely impact all countries under their purview and therefore a great scope of the data sample. Yet, there is value to control for country logistics factors. While the NTD Supply Chain Forum has limited visibility of the country-level logistics since the focus is on the first mile, we can incorporate an external assessment of country logistics factors as a control variable. We believe the World Bank international Logistics Performance Index (LPI) benchmark tool is a relevant indicator to help control for such factors. This score measures performance of the logistics supply chain within a country and provides a qualitative evaluation by trading partners of the country’s logistics performance [26].

The controls for medicine and disease were included with consideration for the fact that the medicines are produced by different manufacturers for different disease programs, which may lend itself to some variability in the supply chains. Mode of shipment was also incorporated as a control since the shipment speed varies between air, land, and sea. Lastly, order size was also included even though the supply chain process is the same regardless of order size; larger orders could take more time to prepare and smaller orders are often shipped by air, making this also a necessary factor to control. Details on the coding of the control variables can be found in S1 and S2 Tables.

Connecting back to the full process shared in Fig 1, Fig 2 provides a diagram illustrating the process flow in a summary view and the components in/out of scope of the study, along with reference to any targeted timeline benchmarks as relevant.

Fig 2

Simplified timeline of the NTD supply chain, associated benchmarks, and scope of research.

Statistical analysis

A quasi-experiment design, using a “one-group pretest-posttest design without control group,” was chosen as the research was initiated a few years after the launch of NTDeliver. In addition, NTDeliver was implemented in a real-world application that did not roll out the system in a phased approach, barring any ability to conduct randomization. This design was used to leverage historical data available on performance to understand the impact of this intervention. An ordinary least squares (OLS) regression model was used to review the relationship between implementing NTDeliver and its impact on delivery timeliness and other KPIs. Furthermore, review of the data confirms that the normality of the error distribution assumption for OLS regression is met. Q-Q plots were used to verify that most data are on a distribution lying on approximately on a straight line. While go signal timeliness did not show a straight trend, the central limit theorem enables the normality assumption to be met in the case of a “sufficiently large sample,” for which the literature generally notes a sample of >50 would be “robust to violation of the normality assumption” [27]. This variable had over 200 samples, thus meeting the central limit theorem condition. While both medicine and disease type are included as control variables, these variables are not found to be significantly correlated since some medicines treat more than one disease and therefore is not a 1:1 mapping; hence, multi collinearity is not a concern. Control variables and robustness checks were applied to strengthen the validity of the results. We considered p values less than 0.05 to be statistically significant.

The shipment data extracted from the system cover orders made through December 31, 2018 and was therefore segmented in two groups to address the first research question: 1) shipments with POs raised prior to the implementation of NTDeliver (February 28, 2006-August 31, 2016) for the “pre” NTDeliver group; 2) shipments with POs raised after the implementation of NTDeliver (September 1, 2016-Dec 31, 2018) for the “post” NTDeliver group. Only the data in the “post” group were used to answer the second research question regarding the impact on shipment performance of making country-level data publicly accessible. The data in the “post” group were split into two groups, with consideration to February 1, 2018, as the implementation date of this publicly accessible data: 1) “Post 1” = 0 for shipments with a PO date prior to February 1, 2018 but after August 31, 2016; 2) “Post 2” = 1 for shipments with a PO date equal to or later than February 1, 2018 but earlier than January 1, 2019.

Results

We first study the impact of information sharing through NTDeliver on shipment performance within the NTD PC medicines supply chain. The data collected included 1,484 total shipments, with 1,068 shipments classified in the “pre” NTDeliver group and 416 in the “post” group. As noted, pairwise deletion was used in cases where data were missing, which accounts for the differing number of observations between variables. Table 2 summarizes the regression results illustrating the bivariate associations between the implementation of the NTDeliver system and NTD supply chain KPIs. Complete regression results are included in Table A3 in the S3 Table.

Table 2

Regression results showing the relationship between NTD supply chain key performance indicators and implementation of information sharing through NTDeliver.

Variables	N	Β reg coefficient	p-value	Predicted direction	Actual direction
PO timeliness (in months)	897	0.941	0.003	↑	↑
Go signal timeliness (in days)	216	41.483	< .001	↑	↓
Shipment timeliness (in months)	535	0.033	0.927	↑	↑ NS
Arrival timeliness (in months)	661	0.516	0.129	↑	↑ NS
Delivery timeliness (in months)	528	0.828	0.027	↑	↑

Abbreviations: Reg, regression; NS, non-significant

Incorporated control variables: WHO region, international LPI, medicine type, disease, order size, mode of shipment

Notably, delivery timeliness, the KPI considered most important to measure supply chain performance, shows a positive and significant association with a p-value of 0.027. PO timeliness also demonstrates positive and significant results supporting the hypothesized direction

Conversely, the results for go signal timeliness appear to support the null hypothesis. While go signal timeliness is highly significant (p<0.001), even though the coefficient value is positive, this actually indicates a negative relationship with information sharing due to the calculation method for the variable (calculated as the difference between requested and actual go signal date). The coefficient suggests an increase in the difference between the go signal request and approval of about 40 days. Because there is no perceivable standard for establishing this request date and it is defined per request of the pharmaceutical manufacturer and any supporting partners, additional analysis provides further insight into how the go signal timeliness calculation may have changed after NTDeliver was implemented.

These results provided indicate that the difference between PO date and go signal request date has significantly decreased, since the coefficient is negative and therefore indicate changes to how the request date was defined (Table 3).

Table 3

Bivariate association between regression results showing the relationship between implementation of information sharing through NTDeliver and key differences between go signal dates.

Variables	N	Β reg coefficient	p-value
Difference between PO date and go signal request date (in days)	263	-48.862	<0.001
Difference between go signal approved date and shipment date (in days)	263	-50.899	<0.001

Abbreviations: Reg, regression; NS, non-significant

Next, we study the impact of making country-level data publicly accessible and particularly promoting this data access to country program managers, as training sessions were also provided for country program managers to publicize the release of this data and provide education on effective usage. The hypothesis is that extending information sharing may have a positive impact on the downstream processes that are actively displayed in the public country pages, which provides country managers with shipment, arrival, and delivery statuses. Table 4 provides the analysis results to answer the second research question regarding the impact of making country-level data publicly accessible.

Table 4

Regression results showing the relationship between implementation of publicly accessible country-level data through NTDeliver and NTD supply chain key performance indicators.

Variables	N	Β reg coefficient	p-value	Predicted direction	Actual direction
PO timeliness (in months)	387	0.33	0.49	NS	↑NS
Go signal timeliness (in days)	132	-14.88	0.42	NS	↑NS
Shipment timeliness (in months)	296	2.57	0.001	↑	↑
Arrival timeliness (in months)	262	2.88	0.003	↑	↑
Delivery timeliness (in months)	255	2.82	0.011	↑	↑

Abbreviations: Reg, regression; NS, non-significant

The results show three variables with significant, positive association: shipment timeliness, arrival timeliness, and delivery timeliness—consistent with the hypothesis that these variables would be impacted since they are actively displayed in these publicly accessible pages. In the main regression results, shipment timeliness and arrival timeliness did not show any significance from the information sharing. In these results, shipment timeliness is significant at a p-value of 0.001 and with a substantial coefficient of 2.57. Arrival timeliness is significant with a p-value of 0.003 and a coefficient of 2.88. Lastly, the important delivery timeliness performance indicator is also significant with a p-value of 0.011 and a coefficient of 2.82.

We also conducted further analyses to check the robustness of the results (see S4 Table). First, we re-ran the regression accounting for a lag in impact from information sharing. The analysis revealed that that all dependent variables, except arrival timeliness, remain significant and generally consistent with the main results when accounting for a six-month theoretical “lag time to benefit,” considered as the time between implementing the intervention and observing improved outcomes [28]. Additionally, we conducted a “double pretest” to test the validity of the “pre” group as comparison. This “double pretest” was used as a validity check to ensure that merely “history” and/or “maturation” is not the reason for the differences between the pretest and posttest results, rather than independent variable [29]. The pretest group was divided and compared for significant differences in performance to assure any differences in performance within the pretest group are minimal and/or less than the difference between the pretest and posttest groups. The groups were divided with roughly the same number of shipments in each group, with one group comprised of shipments with POs raised between 2006–2013 and the other with POs raised between 2014–2016 August. The results from this analysis indicated that only PO timeliness had a positive, significant difference between these two pretest groups. No other dependent variables have an observable, significant difference in performance.

Discussion

Lack of coordination and limited transparency are two top issues in humanitarian supply chains [30]. Although existing literature on for-profit supply chains suggests addressing the issues though information sharing, it is unclear whether and how information sharing can improve performance in the non-profit humanitarian context [10-13]. In this paper, we examined the potential impact of information sharing on humanitarian supply chain performance. Our analysis is the first to undertake an empirical study evaluating performance and information sharing both in the context of the NTD supply chain and in the broader humanitarian space. While most existing literature focuses on information sharing in for-profit supply chains, which is typically focused on sharing information between the buyer and supplier, this paper contributes to the existing literature and addresses the gap by investigating the impact of sharing information publicly for non-profit supply chains. The results of our study demonstrate the value of investment in supply chain performance measurement and information sharing towards the success of global health partnerships and such initiatives may be implemented in the broader context of humanitarian programs.

We find that information sharing is positively associated with the timeliness of several key stages in NTD supply chain, i.e., PO timeliness, arrival timeliness, shipment timeliness, and the key success measure for the NTD supply chain—delivery timeliness. Delivery timeliness, arguably the most critical KPI to measure first mile performance, appears to have a positive and significant association with information sharing. The analysis showed that in the post implementation phase, on average, completed POs were submitted one month earlier. In addition, go-signals were issued expeditiously after POs were issued. Improvements in these two metrics possibly contributed to the observed results of earlier delivery of medicines to the CMS, suggesting that information sharing as a result of the NTDeliver may be a factor.

Furthermore, information sharing appears to be possibly more impactful when information is released publicly and particularly promoted to country program managers, compared to when it is shared only with the supply chain partners (i.e., WHO and pharmaceutical donors). A significant positive association was evident for three variables upon this information extension: shipment, arrival, and delivery timeliness Neither shipment nor arrival timeliness met the threshold for significance in the first set of results, but both variables meet this threshold in these additional results. Delivery timeliness continues to meet the threshold for significance in these results, but also with a coefficient substantially larger than in the first set of results. While there is clearly an impact from the implementation of NTDeliver without the country pages, the addition of the country pages appeared to extend the impact to downstream processes after PO creation and the go signal. As previously noted, this result was hypothesized since the country pages specifically provide access to status from the point of shipment. Thus, it is logical that we did not see in the results significant improvements to upstream KPIs such as PO timeliness and go signal timeliness, since they are not featured in these country pages. These results suggest that it may be more effective to extend information sharing to stakeholders responsible for performance across the entire supply chain.

The robustness checks also showed that even if the information sharing effect took time to make an impact, the performance still improved and can be associated more confidently with the implementation of NTDeliver. Furthermore, although there is no “control group” in our research design, we conduct a “double pretest” to investigate if other observed time-varying variables may contribute to the significant results. Comparison with performance before the information sharing was implemented suggests that the supply chain performance does not simply improve over time, with exception to one performance indicator. Only PO timeliness indicates a significant positive change over time during the period before NTDeliver was implemented. The reason for this change in PO timeliness may be attributed to a process change coinciding around the timing of the second pretest group defined in this analysis, which included medicines ordered from 2014 onward: the JAP was implemented by the WHO in 2013 to standardize processes to support an integrated application submission and review for donated medicines [23]. This JAP process most positively impacts the PO process as it promoted more coordination between the various levels of WHO offices to assure timely applications and order fulfillment for donated medicines [23].

This research has some limitations that may naturally inform future research. The quasi-experimental design used lacks a control group and random assignment since NTDeliver was implemented for all PC medicine donations managed through the NTD Supply Chain Forum [31]. Although we used a “double pretest” robustness check to verify our results, future research could study the impact of information sharing in a controlled setting with randomization incorporated in the design. Also, modifications to the design could entail incorporating qualitative research using interviews and/or surveys to investigate whether stakeholder behavior changes may be drivers for observed performances changes. Furthermore, there is a growing desire for financial donors to understand the impact of investments from an outcomes perspective, especially in the interest of funding effective health innovations that offer value for money [32].While our research results certainly helps to validate the positive association between information sharing and supply chain performance, further research on how the supply chain performance improvements in this first mile result in a reduction in delayed and/or missed MDAs would provide more perspectives on linking the delivery timeliness improvements to the number of additional individuals reached. Also, incorporating any data pertaining to the last mile aspect of the NTD supply chain in NTDeliver might help to improve those downstream processes, which remains a priority, given the last mile’s impact on efficient and effective distribution and planning for MDAs [33]. Lastly, with respect to the current global health climate, the COVID-19 pandemic had a significant impact on NTD programs, with the WHO recommending postponing MDAs to respect public health measures that advocate for physical distancing to slow the spread of the virus [32]. Further research is needed to gain insight on new challenges from these disruptions to understand the impact by region and how information sharing may help to mitigate such disruption and support managing uncertainty for global health campaign supply chain planning during a pandemic.

Our results have practical implications for NTD supply chain management practices. As the deadline approaches for achieving the 2030 targets set out in the new WHO roadmap and the relevant NTD goals in target 3.3 of the Sustainable Development Goals, there is a high degree of confidence that these results affirm that investment in supply chain information sharing is a critical to ensuring success. In fact, the new WHO roadmap dedicates an entire section to “Access and logistics,” in which supply chain management priorities for improvement are outlined under the umbrella concept that “effective supply chain management is vital to ensuring access to quality-assured NTD medicines and other products” [34]. Given the relationship between first mile supply chain performance and timeliness of MDAs, investment in supply chain information sharing is worthwhile to support improvements to NTD program management [7].

Furthermore, the findings imply additional benefits of information sharing when extending information sharing to a broader audience, particularly focusing on program managers. Incorporating visibility to upstream data, such as attaching country applications or tracking the regional office approval date, may improve these processes as well. Such upstream processes have been noted as potentially impacting delivery timeliness by the WHO HQ—such as the fairly significant time taken for the regions to review applications—and further incorporation of these processes in NTDeliver may benefit the end-to-end NTD supply chain [7].

Given the significant volume of medicines and the number of people requiring these medicines, the research implications have the potential to impact global health programs affecting the health of tens to hundreds of millions of people. The research supports that, even in absence of financial remuneration, information sharing contributes measurable supply chain improvements and supports investing further in performance measurement in humanitarian supply chains. Beyond the NTD space, data transparency is generally viewed as a challenge with country governments citing national sovereignty and privacy in refusing to release data for public consumption [35]. Positive results from extending the information sharing argue in favor of the value and benefits of information sharing in the global health space. With the profile and importance of the supply chain continues to elevate in humanitarian programs, especially those in the healthcare space, there will be an opportunity to invest further in such performance measurement tools to bring more evidence-based approaches to decision making. This has significant potential to promote accountability and coordination resulting in goals achieved and improved health outcomes. As the global health aid landscape is becoming more focused on driving measurable performance and impact from investments, these findings support investing in supply chain systems and commitment to data transparency.

Dummy coding matrices.

(DOCX)

Click here for additional data file.

International Logistics Performance Index (LPI) Control Variable Assignment Details.

(DOCX)

Click here for additional data file.

Full linear regression results.

(DOCX)

Click here for additional data file.

Robustness check results.

(DOCX)

Click here for additional data file.

30 Aug 2021

Dear Dr. Kasparis,

Thank you very much for submitting your manuscript "Improving Timeliness in the Neglected Tropical Diseases Preventive Chemotherapy Donation Supply Chain through Information Sharing: A Retrospective Empirical Analysis" for consideration at PLOS Neglected Tropical Diseases. As with all papers reviewed by the journal, your manuscript was reviewed by members of the editorial board and by several independent reviewers. In light of the reviews (below this email), we would like to invite the resubmission of a significantly-revised version that takes into account the reviewers' comments.

We cannot make any decision about publication until we have seen the revised manuscript and your response to the reviewers' comments. Your revised manuscript is also likely to be sent to reviewers for further evaluation.

When you are ready to resubmit, please upload the following:

[1] A letter containing a detailed list of your responses to the review comments and a description of the changes you have made in the manuscript. Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.

[2] Two versions of the revised manuscript: one with either highlights or tracked changes denoting where the text has been changed; the other a clean version (uploaded as the manuscript file).

Important additional instructions are given below your reviewer comments.

Please prepare and submit your revised manuscript within 60 days. If you anticipate any delay, please let us know the expected resubmission date by replying to this email. Please note that revised manuscripts received after the 60-day due date may require evaluation and peer review similar to newly submitted manuscripts.

Thank you again for your submission. We hope that our editorial process has been constructive so far, and we welcome your feedback at any time. Please don't hesitate to contact us if you have any questions or comments.

Sincerely,

Gregory Deye

Associate Editor

PLOS Neglected Tropical Diseases

Bruce Lee

Deputy Editor

PLOS Neglected Tropical Diseases

***********************

Reviewer's Responses to Questions

Key Review Criteria Required for Acceptance?

As you describe the new analyses required for acceptance, please consider the following:

Methods

-Are the objectives of the study clearly articulated with a clear testable hypothesis stated?

-Is the study design appropriate to address the stated objectives?

-Is the population clearly described and appropriate for the hypothesis being tested?

-Is the sample size sufficient to ensure adequate power to address the hypothesis being tested?

-Were correct statistical analysis used to support conclusions?

-Are there concerns about ethical or regulatory requirements being met?

Reviewer #1: see below

Reviewer #2: The study objectives are clear and there is a clear set of testable hypotheses.

However, the study design does not take into account several factors:

-Were there any time-varying factors besides the NTDDeliver and NTDPublic which may be coincident with the study time frame. Countries, manufacturers, and WHO are constantly trying new improvements. Not clear how the study design robustly controls for that?

-Presumably there is a higher likelihood of missing data in the pre-intervention sample. So without explicitly showing that missing data would be completely random (and covariance matrix etc.) it is unclear how the study design removes biases that may result from non random missing data.

-An important control variable which is missing is country characteristics e.g. country logistics factors, etc. What if there was a differences in the countries receiving the orders in the pre-and post intervention phase. The authors have controlled for WHO region, but that is too coarse a variable to account for country specific differences

-Did the authors explore the use of a RD type study design?

Reviewer #3: No issues

--------------------

Results

-Does the analysis presented match the analysis plan?

-Are the results clearly and completely presented?

-Are the figures (Tables, Images) of sufficient quality for clarity?

Reviewer #1: see below

Reviewer #2: The analysis matches what the study plan was.

The result are difficult to interpret without a clear exposition of what each of the time variables means ie. when does the clock start for each timeliness variables and then does the clock stop. The authors should consider creating a diagram with the timelines and variable definition on that.

Reviewer #3: No issues

--------------------

Conclusions

-Are the conclusions supported by the data presented?

-Are the limitations of analysis clearly described?

-Do the authors discuss how these data can be helpful to advance our understanding of the topic under study?

-Is public health relevance addressed?

Reviewer #1: see below

Reviewer #2: the conclusions are supported by the data with the caveats in the study design outlined earlier.

The work has significant relevance for public health but needs to be more robust in its analysis

Reviewer #3: -Based on the findings can the authors further clarify which stakeholder behavior changed based on the variable. E.g., Did PO timeliness reflect WHO management? Did shipping times reflect the pharmaceutical companies. Were these results possibly influenced by other stakeholders indirectly such as NGOs supporting governments efforts?

--------------------

Editorial and Data Presentation Modifications?

Use this section for editorial suggestions as well as relatively minor modifications of existing data that would enhance clarity. If the only modifications needed are minor and/or editorial, you may wish to recommend “Minor Revision” or “Accept”.

Reviewer #1: see below

Reviewer #2: (No Response)

Reviewer #3: CLARIFICATION IN THE INTRODUCTION

• “While significant progress was made towards these 2020 targets, the WHO has recently released a new NTD roadmap with 2030 targets, which pharmaceutical manufacturers have committed to continuing to support [2].” Pharmaceutical companies support the road map but not all donation programs have committed to donating pharmaceuticals through 2030.

• “Delivery delays result in waste, increased program costs, and delayed or even completely missed MDAs, leaving individuals susceptible to NTD infections [7].” More accurate to state “…..leaving communities susceptible to disease recrudescence.”

• “Figure 1. Components of the NTD supply chain from application for donation to in-country delivery to central medical stores”.

- Application: performance measure is 8 months prior to MDA (not 6)

- Manufacturing: performance measures are variable by manufacturer

- Shipment: three months prior to MDA is the UNICEF standard of 10 years ago for sea freight. This is not WHO policy.

- In-country processes: Performance measures should say varies by country

Clarify the NTD Supply Chain Forum membership. If stating the present Forum makeup, some members are missing. If wishing to indicate the makeup at the time of establishment, please make this clear.

--------------------

Summary and General Comments

Use this section to provide overall comments, discuss strengths/weaknesses of the study, novelty, significance, general execution and scholarship. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. If requesting major revision, please articulate the new experiments that are needed.

Reviewer #1: Overall, this is an interesting manuscript that attempts to utilize a pre-post design to evaluate the potential impact of the NTDeliver platform on improving supply chain management for PC diseases. The findings are of interest to the NTD funder, policy and program community and fit well within the scope of this journal. There are substantial issues with the way in which the paper is presented and written that need to be addressed. In addition, the authors need to tone down the language implying causality, as this study design can only assess associations, not causality. If these changes are made, I believe the paper will be a useful contribution to the literature.

A critical limitation of this analysis is that it only evaluates the impact on the “first mile”. It is well known that many of the delays in getting MDA drugs to communities happen between the central store and the program. As such, it is difficult to know how much of an impact this platform actually has on drug delivery. In addition, it does not seem logical to evaluate timeliness as a continuous measure – it probably makes no difference if the drugs are in the central warehouse 6 months or 5 months ahead of MDA – what matters is whether they are there and can be distributed in time for MDA. I would think a binary cutoff based on what this sufficient time is would be more practical. Finally, some of the “improvements” may be statistically significant but that does not make them programmatically relevant. I am not sure a two week improvement is meaningful for example.

The paper is written from the standpoint of a data analyst and much of the language is difficult to follow and not clear to the intended audience. The way in which the analysis is presented is not intuitive and does not align with metrics that the audience will want to understand.

The authors seem to erroneously attribute magnitude of significance with the p value in their analysis. First, it is not at all clear why a non-standard p value of .1 was chosen to represent significance, particularly given that the study design is already weak. This seems likely to have been a post-hoc decision made following a look at the results, particularly given that many analyses are right at the 0.9 level. Please revise to use a standard p value of 0.5 for significance. In addition, something is not “more significant” if its p value is smaller. The key metric is the effect size, and then whether or not it meets the threshold for significance. Please revise the text accordingly.

The results and the discussion need to be separated. There is far too much discussion included in the results section. Please just present the results cleanly and then interpret them in the discussion.

The weaknesses of this study design are a major limitation (as noted by the authors). There is no reason why an RCT or a stepped wedge study could not have been designed in coordination with WHO. One key takeaway in the conclusion is the need to plan robust evaluations from the start so that more meaningful impact assessment can be conducted.

I was confused by the language around confounders and controls – these are important cofactors or mediators of effect – not confounders.

The data could be presented graphically or at least in some much more readily digestible format. The raw output of the regression analyses are not the best way to present these data to the intended audience.

Small issue – data are pleural – datum is singular – update text accordingly.

Reviewer #2: This paper addresses and important and very interesting area for policy relevance for NTDs.

The authors utilize an interesting data set and define performance measures with direct operational relevance to NTD programs and NTD supply chains

However, there are multiple deficiencies in the study design which need to be addressed. It is important for this study to be robust in its methods for the findings to be taken seriously for further investment in the interventions analyzed.

Reviewer #3: (No Response)

--------------------

PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

Reviewer #3: No

Figure Files:

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org.

Data Requirements:

Please note that, as a condition of publication, PLOS' data policy requires that you make available all data used to draw the conclusions outlined in your manuscript. Data must be deposited in an appropriate repository, included within the body of the manuscript, or uploaded as supporting information. This includes all numerical values that were used to generate graphs, histograms etc.. For an example see here: http://www.plosbiology.org/article/info%3Adoi%2F10.1371%2Fjournal.pbio.1001908#s5.

Reproducibility:

To enhance the reproducibility of your results, we recommend that you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. Additionally, PLOS ONE offers an option to publish peer-reviewed clinical study protocols. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols

4 Nov 2021

Submitted filename: NTD PLOS Response Letter to Reviewers_Elena_Kasparis_FINAL.docx

Click here for additional data file.

5 Nov 2021

Dear Dr. Kasparis,

We are pleased to inform you that your manuscript 'Improving Timeliness in the Neglected Tropical Diseases Preventive Chemotherapy Donation Supply Chain through Information Sharing: A Retrospective Empirical Analysis' has been provisionally accepted for publication in PLOS Neglected Tropical Diseases.

Before your manuscript can be formally accepted you will need to complete some formatting changes, which you will receive in a follow up email. A member of our team will be in touch with a set of requests.

Please note that your manuscript will not be scheduled for publication until you have made the required changes, so a swift response is appreciated.

IMPORTANT: The editorial review process is now complete. PLOS will only permit corrections to spelling, formatting or significant scientific errors from this point onwards. Requests for major changes, or any which affect the scientific understanding of your work, will cause delays to the publication date of your manuscript.

Should you, your institution's press office or the journal office choose to press release your paper, you will automatically be opted out of early publication. We ask that you notify us now if you or your institution is planning to press release the article. All press must be co-ordinated with PLOS.

Thank you again for supporting Open Access publishing; we are looking forward to publishing your work in PLOS Neglected Tropical Diseases.

Best regards,

Gregory Deye

Associate Editor

PLOS Neglected Tropical Diseases

Bruce Lee

Deputy Editor

PLOS Neglected Tropical Diseases

***********************************************************

19 Nov 2021

Dear Dr. Kasparis,

We are delighted to inform you that your manuscript, "Improving Timeliness in the Neglected Tropical Diseases Preventive Chemotherapy Donation Supply Chain through Information Sharing: A Retrospective Empirical Analysis," has been formally accepted for publication in PLOS Neglected Tropical Diseases.

We have now passed your article onto the PLOS Production Department who will complete the rest of the publication process. All authors will receive a confirmation email upon publication.

The corresponding author will soon be receiving a typeset proof for review, to ensure errors have not been introduced during production. Please review the PDF proof of your manuscript carefully, as this is the last chance to correct any scientific or type-setting errors. Please note that major changes, or those which affect the scientific understanding of the work, will likely cause delays to the publication date of your manuscript. Note: Proofs for Front Matter articles (Editorial, Viewpoint, Symposium, Review, etc...) are generated on a different schedule and may not be made available as quickly.

Soon after your final files are uploaded, the early version of your manuscript will be published online unless you opted out of this process. The date of the early version will be your article's publication date. The final article will be published to the same URL, and all versions of the paper will be accessible to readers.

Thank you again for supporting open-access publishing; we are looking forward to publishing your work in PLOS Neglected Tropical Diseases.

Best regards,

Shaden Kamhawi

co-Editor-in-Chief

PLOS Neglected Tropical Diseases

Paul Brindley

co-Editor-in-Chief

PLOS Neglected Tropical Diseases