Return on investment of the electronic vaccine intelligence network in India.

The electronic vaccine intelligence network (eVIN) was introduced by India's Ministry of Health and Family Welfare in 12 states and was implemented by the United Nations Development Programme through the Gavi health system strengthening support during 2014-17 to replace the traditional paper-based cold-chain management system with an electronic vaccine logistics management system. An economic assessment was conducted as part of the overall assessment of eVIN. The objective of the economic assessment was to conduct a return on investment analysis of eVIN implementation. Return on investment was defined as the ratio of total benefits (savings) from eVIN to total investment in eVIN. All costs were calculated in 2020 prices and reported in Indian rupees (1 US dollar = INR 74.132). A one-rupee investment in eVIN led to a return of INR 0.52 for traditional vaccines. The highest cost savings from eVIN was from better vaccine stock management. When same percentage of savings from the new vaccines were incorporated into the analysis, one-rupee investment in eVIN led to a return of INR 1.41. In the future, when only recurrent costs will exist, the return from eVIN will be even higher: a one-rupee investment in eVIN will yield a return of INR 2.93. The assessment of eVIN showed promising results in streamlining the vaccine flow network and ensuring equity in vaccine stock management along with good return on investment; hence, there was a rapid expansion of eVIN in all 731 districts across 36 states and union territories in the country.

Introduction

India’s immunization program is the largest in the world, with a beneficiary count reaching up to 26 million children and 30 million pregnant women every year.[1] The cold chain is one of the most important components for ensuring the delivery of quality vaccines.[2] The vaccine delivery network in India operates through more than 27,000 functional cold-chain points (CCPs), of which 750 (3%) are located at the district level and above, with the rest located at the sub-district and below levels, at community health centers, primary health centers, and urban health facilities.[1] India’s cold-chain management system has been paper based since the introduction of the universal immunization program in 1978. This system did not allow for real-time stock information, which led to vaccine overstocking and stockouts.

The electronic vaccine intelligence network (eVIN) was designed to replace the traditional paper-based cold-chain management system with an electronic vaccine logistics management system. eVIN was introduced by India’s Ministry of Health and Family Welfare in 12 states (Assam, Bihar, Chhattisgarh, Gujarat, Himachal Pradesh, Jharkhand, Madhya Pradesh, Manipur, Nagaland, Odisha, Rajasthan, and Uttar Pradesh)[3] and was implemented by the United Nations Development Programme (UNDP) through the Gavi health system strengthening support during 2014–17. Since its introduction, eVIN has successfully digitized vaccine stocks and monitored the temperature of the cold chain through a smartphone application. This has enabled monitoring of vaccine life and stock at all administrative levels and provided an opportunity to understand the risks and benefits of the electronic vaccine logistics management system on an unprecedented scale.

Realizing this opportunity and scope, the immunization division of the ministry commissioned an assessment of the eVIN program. An economic assessment was also conducted as part of the overall assessment of eVIN. The objective of the economic assessment was to evaluate the overall economic impact of eVIN implementation and to conduct a return on investment (ROI) analysis of eVIN implementation. The results of the economic assessment are presented in this study.

Materials and methods

A pre-post design was considered to evaluate the economic impact of eVIN implementation in 12 Indian states.[4] Data for the economic assessment were collected from the immunization division of the ministry, from the UNDP, and from 102 randomly selected CCPs of 7 states: Assam (19), Chhattisgarh (12), Gujarat (15), Jharkhand (11), Nagaland (7), Odisha (20), and Rajasthan (18) during the period of April–June 2018.a Although eVIN was implemented initially in 12 states, primary data were collected from 7 states because of time constraints. While data collected from the ministry were used to understand the cost savings/dissaving related to vaccines in the pre- and post-eVIN periods, the data from the CCPs were used to estimate the time spent by the staff on different eVIN-related activities. A structured questionnaire was used to collect the primary data from the CCPs. Data on time spent were collected by interviewing the cold-chain handlers (CCHs) of the respective CCPs. All costs were calculated in 2020 prices and reported in Indian rupees. An average exchange rate of 1 United States Dollar (USD) = Indian Rupee (INR) 74.132 can be used for the conversion.

Costing methodology

The incremental economic and financial costs related to eVIN implementation were calculated.[5] While financial costs focused only on the actual expenses incurred for eVIN-related activities, economic costs represented the opportunity costs associated with the program as compared with the next best alternative and included a valuation of all inputs needed for the program, including a valuation of time. We used an ingredient approach in which all functions related to eVIN were identified and costed.[6] The government perspective was used for this economic assessment, where all costs incurred by the government/its partner (the UNDP) were considered.[7]

Financial costs included expenditures on staff salaries at different levels (UNDP staff), travel at different levels, training, eVIN software development and maintenance, purchase of mobile phones, temperature loggers, accessories, communication materials, printing, and stationery. UNDP staff were hired specifically for eVIN at the state and district levels; hence, their salaries were added in the financial cost calculation. Time spent by different categories of existing staff (e.g., district immunization officers and CCHs) on activities related to eVIN implementation (for example, time spent on training and meetings related to eVIN, transporting and handling vaccines, and entering vaccine stock positions into smartphones) were considered in the economic cost calculation. Hence, economic cost was the sum of financial cost and the time cost for various activities related to eVIN. As the analysis focused on incremental investment related to eVIN implementation, capital costs such as the costs of the existing cold-chain equipment, vehicles, and cold-chain space were not considered.

The training pattern during eVIN implementation was similar in all states, and two CCHs from each CCP were trained for 2 days along with district immunization officers. Hours spent on training by different categories of staff were multiplied by the salary per hour of those staff to obtain the human resources (HR) cost related to training. The average monthly gross salaries of the staff involved in eVIN were collected from the respective pay roll sections.

The number of immunization sessions held during 2017–18 collected from the Health Management Information System (HMIS) was multiplied by minutes spent per session day to obtain the total hours spent in 1 year on eVIN data entry into smartphones.[8] The hours spent were then multiplied by the salary per hour to estimate the HR cost related to eVIN entry. Entering vaccine information into registers continued during primary data collection (April to June 2018); hence, the time spent on paper recording was not considered in this calculation as it remained the same in the pre- and post-eVIN periods. Only the additional time spent entering vaccine information into smartphones was estimated.

Return on investment (ROI)

ROI analysis provides a convenient and comparable measure of the efficiency of one or more investment choices. ROI makes investment decisions easier as it expresses both the costs and the full range of benefits of an intervention in the same units (money).[9] The commonly used metric in ROI analysis is the benefit–cost ratio. If the ratio is greater than one, it implies that the benefit from the investment is higher than the cost, and hence is a sound investment. Studies have demonstrated a high ROI from immunization programs in low- and middle-income countries;[10,11] however, there is no specific ROI analysis for cold chain-related interventions.

In the eVIN ROI analysis, ROI is defined as the ratio of total benefits (savings) from eVIN to total investment in eVIN. Investment was the total amount invested in eVIN during 2014–17. State-wise investment data related to eVIN were collected from the finance division of the UNDP for 12 states from 2014 to 2018. As all output data were gathered until 2017, 2018 expenditure was excluded from the calculation. Further, even though eVIN was introduced in Himachal Pradesh, there was no state-specific expenditure for Himachal Pradesh during 2014–2017. Hence, Himachal Pradesh was excluded from the state-level cost assessment, and the ROI was calculated for 11 out of 12 states. Along with state-wise expenditure data related to eVIN, national-level expenditure for eVIN, including eVIN software development and management, and the development of communication materials, were also collected and distributed equally among 12 states. eVIN activities did not start in Himachal Pradesh until 2017; however, the initial national-level investment was made in the state. Hence, when distributing the national-level expenditure, it was distributed among 12 states, including Himachal Pradesh.

The expected savings from eVIN were (1) cost savings through reduction of overstocking of vaccines, (2) savings from the reduction of the vaccine wastage rate, and (3) savings from health-care costs due to fewer children missing out on immunization because of reduction in the stockout of vaccines in the post-eVIN period.

Cost savings through reduction of overstocking of vaccines

As mentioned earlier, before the implementation of eVIN, vaccine stock management was paper-based, and hence, real-time vaccine stock information was not available at different levels of CCPs. For example, the state CCH had no real-time information about vaccine availability at different district CCPs. Similarly, the district CCH had no information about vaccine availability in sub-district and below CCPs. This resulted in either stockout or overstock of vaccines at different levels of CCPs. With the introduction of eVIN, vaccine stock information was digitized at all levels of CCPs, and hence, vaccine stock information was visible at all levels through the eVIN smartphone application, leading to less overstock and stockouts. Reduced overstock of vaccines leads to a lower number of doses procured, and hence, cost savings as less investment is required for vaccines.

To understand the savings from reduction of overstocking, the utilization of each vaccine during the pre- and post-eVIN periods was collected from the immunization division of the Ministry of Health and Family Welfare, and the difference in utilization was multiplied by the unit price of each vaccine to calculate the savings/dissaving related to vaccines. Saving in any vaccine indicated lower doses utilized in the post-eVIN period. The pre-eVIN period for utilization data for Uttar Pradesh, Madhya Pradesh, and Rajasthan was 2015–16, while for the rest of the states it was 2016–17. The pre- and post-eVIN periods were determined based on eVIN implementation in the respective states. It should be noted that complete data were not available for hepatitis B for Assam and Jharkhand and oral polio vaccine (OPV) for Uttar Pradesh, so they were not considered in the cost savings analysis.

(2) Vaccine wastage rate

The vaccine wastage rate was defined as the number of doses wasted due to freezing, expiry, breaking, or unusable unopened vials.[12] The pre- and post-eVIN wastage of vaccines reported in the vaccine registers was collected. As in the pre-eVIN period, the reasons for vaccine wastage were not always clearly mentioned, all types of waste were considered for both periods.

(3) Missed opportunities

Before the introduction of eVIN, as there was no real-time information on vaccine stock at different levels of CCPs, vaccine stockouts were common, and children used to miss immunization because of this (commonly known as a missed opportunity). The duration of stockouts of all vaccines was calculated in the pre- and post-eVIN periods. To calculate the number of immunization sessions missed because of vaccine stockouts, the assumption was that if the duration of a vaccine stockout was more than 3 days, children missed one immunization session. If the duration was more than 7 days, the children missed two immunization sessions. The total number of children vaccinated, and the number of sessions held during the pre- and post-eVIN periods were gathered from the HMIS.[8] The number of children immunized in one session was calculated by dividing the total number of children immunized and the total number of sessions held in the observation period. The total number of children who missed immunization due to vaccine stockout was calculated based on the duration of the vaccine stockout and the number of children vaccinated per session.

It was expected that when fewer children missed immunization sessions because of fewer stockouts, the risk of getting a disease would be lower. This would lead to lower health-care costs in terms of fewer visits to the hospital and/or a lower rate of hospitalization. Using the missed opportunity data, the number of children at risk of different diseases during the pre- and post-eVIN periods was estimated. The incidence of disease, treatment-seeking behavior, and cost of illness from different vaccine-preventable diseases were sourced from the literature. India’s expanded immunization program was introduced in 1978 and it provided vaccines to protect children against diphtheria, pertussis, tetanus, childhood tuberculosis, poliomyelitis, and typhoid-paratyphoid. The program expanded gradually under the universal immunization program and currently, the program includes bacillus Calmette–Guérin (BCG), hepatitis B, oral polio vaccine (OPV), diphtheria pertussis tetanus (DPT), measles rubella (MR), Haemophilus influenzae type B (Hib) containing pentavalent (DPT+Hepatitis B+ Hib), inactivated polio vaccine (IPV), rotavirus, pneumococcal conjugate vaccine (PCV), Japanese encephalitis (JE in endemic districts) and tetanus toxoid (TT) vaccines. It should be noted in this context that cost of illness information was not available for most of these vaccine-preventable diseases in the Indian context; for example, no cost of illness study was found for diphtheria, tetanus, pertussis, childhood tuberculosis, poliomyelitis, and measles. Only one study reported cost of illness of childhood pneumonia and meningitis that could be averted by using Haemophilus influenzae type b (Hib) vaccination, a component of currently used pentavalent vaccine in Indian context.[13] Hence, savings from averted healthcare costs were calculated only for these two diseases, assuming the incidence rate and cost of illness.

The annual incidence rates of Hib pneumonia and meningitis were assumed to be 1,102 and 22 per 100,000 children, respectively.[13] Among those affected by the disease, 70% were assumed to have access to treatment, and the distribution of access across various levels was calculated from Clark et al. (2013).[13] The incidence of hospital admission was assumed to be 55 per 100,000 children per year. The costs of outpatient visits and hospitalization were also obtained from Clark et al.[13] The study reported data in 2010 USD, and all figures were converted into 2020 Indian rupees using the consumer price index. The amount that could be saved from the cost of illness of these diseases was then estimated.

Ethical approval

The study was approved by the Institutional Ethics Review Board of Sigma Research and Consulting, New Delhi, India.

Results

Saving/dissaving related to vaccines during pre- and post-eVIN implementation at state vaccine stores

One of the objectives of eVIN implementation was to rationalize the stock position of the vaccines as entering vaccine information in the eVIN smartphone application provides real-time information on vaccine stock availability at different levels of the cold chain. Overall, all vaccines showed savings during the post-eVIN period, indicating better stock management after the introduction of eVIN. The utilization of all vaccines in the pre- and post-eVIN periods is presented in Table 1. Considering all vaccines, the total savings from better stock management was INR 967 million (Table 2). The highest savings were achieved with the pentavalent vaccine (INR 554 million), followed by hepatitis B (INR 128 million), measles (INR 114 million), oral polio vaccine (INR 75 million), tetanus toxoid (INR 55 million), Bacillus Calmette Guerin (INR 30 million), and diphtheria tetanus pertussis (INR 13 million) (Table 2). The highest savings were expected from pentavalent, as it was the most expensive vaccine.

Table 1.

Utilization of vaccines during pre- and post-electronic vaccine intelligence network (eVIN) period (doses in million)

	BCG		OPV		Hepatitis B		Pentavalent		Measles		DPT		Tetanus toxoid
States	Pre-eVIN	Post-eVIN	Pre-eVIN	PosteVIN	Pre-eVIN	Post-eVIN	Pre-eVIN	Post-eVIN	Pre-eVIN	Post-eVIN	Pre-eVIN	Post-eVIN	Pre-eVIN	Post-eVIN
Assam	2.17	1.69	5.52	1.76	NA	NA	2.25	1.28	1.81	0.97	1.69	1.47	2.45	1.26
Bihar	7.71	6.08	20.94	15.92	1.77	1.35	9.75	5.46	8.33	5.56	4.30	3.12	8.34	6.76
Chhattisgarh	1.81	1.21	3.90	1.58	0.29	0.38	1.81	1.19	1.31	1.01	1.20	1.19	2.10	2.01
Gujarat	2.26	2.38	5.43	4.49	0.32	0.54	2.73	2.60	2.98	2.18	2.38	1.78	2.76	3.72
Jharkhand	1.51	2.25	4.02	3.34	NA	NA	2.22	1.97	2.64	1.61	2.29	1.07	2.63	2.45
Madhya Pradesh	4.53	3.55	11.17	7.99	0.60	0.65	4.83	3.72	4.40	4.64	2.70	5.06	5.87	6.26
Manipur	0.13	0.10	0.25	0.18	0.06	0.03	0.15	0.12	0.12	0.08	0.12	0.08	0.15	0.09
Nagaland	0.11	0.10	0.16	0.16	0.03	0.04	0.10	0.09	0.09	0.03	0.08	0.07	0.07	0.09
Odisha	1.74	1.80	5.39	3.21	0.15	0.24	1.91	1.68	1.68	1.70	2.23	1.23	2.55	2.12
Rajasthan	3.30	3.14	4.66	9.99	0.14	0.31	4.03	2.69	2.43	3.77	2.31	2.89	4.13	4.20
Uttar Pradesh	19.94	15.04	NA	NA	25.63	2.01	15.37	14.02	18.19	13.15	10.25	9.06	32.59	13.43

(1) NA indicates data not available. (2) Bacillus Calmette Guerin (BCG), Oral Polio Vaccine (OPV), Diphtheria Pertussis Tetanus (DPT).

Table 2.

Saving/dissaving related to vaccine stock at the state vaccine stores (2020 INR million)

States/vaccines	BCG	OPV	Hep-B	Pentavalent	Measles	DPT	TT	Total
Assam	1.83	21.86	–	52.07	10.34	1.11	3.07	90.29
Bihar	6.18	29.20	2.31	229.67	34.00	6.08	4.09	311.53
Chhattisgarh	2.28	13.48	−0.46	33.64	3.70	0.04	0.22	52.90
Gujarat	−0.47	5.48	−1.18	6.75	9.82	3.11	−2.50	21.01
Jharkhand	−2.83	3.97	–	13.29	12.62	6.27	0.45	33.77
Madhya Pradesh	3.70	18.46	−0.27	59.25	−2.92	−12.16	−1.01	65.06
Manipur	0.13	0.45	0.17	1.88	0.50	0.20	0.15	3.48
Nagaland	0.04	−0.03	−0.03	0.91	0.67	0.04	−0.05	1.55
Odisha	−0.23	12.68	−0.50	12.64	−0.21	5.12	1.12	30.61
Rajasthan	0.59	−30.96	−0.93	71.74	−16.50	−3.00	−0.16	20.77
Uttar Pradesh	18.56	–	129.01	72.27	61.88	6.17	49.77	337.66
Total	29.78	74.57	128.11	554.12	113.91	12.96	55.16	968.62

The blanks indicate complete data were not available; 1 USD = INR 74.132.

Bacillus Calmette Guerin (BCG), Oral Polio Vaccine (OPV), Hepatitis B (Hep-B), Diphtheria Pertussis Tetanus (DPT), Tetanus Toxoid (TT).

Savings from missed opportunities

Missed immunization opportunities because of pentavalent vaccine stockouts in the pre-eVIN period led to 138,988 additional children at risk of childhood Hib pneumonia and meningitis. INR 0.77 million will be saved through averted outpatient visits and/or hospitalization for Hib pneumonia and meningitis in the post-eVIN period because of a reduction in vaccine stockouts and hence, fewer missed opportunities. As mentioned earlier, the cost of illness information was not available for most vaccine-preventable diseases in the Indian context. Hence, cost savings were calculated only for Hib pneumonia and meningitis and were underestimated. Savings from the reduction of wastage of all vaccines was estimated at about INR 6 million (not reported in the table).

Investment related to eVIN

The total expenditure related to eVIN in 11 states is presented in Table 3. The highest expenditure on eVIN was in Uttar Pradesh (INR 267 million), followed by Rajasthan (INR 208 million) and Madhya Pradesh (INR 207 million). The lowest investment was in Manipur (INR 113 million), followed by Nagaland (INR 114 million) and Jharkhand (INR 138 million).

Table 3.

Total financial expenditure on electronic vaccine intelligence network (eVIN) during 2014–2017 (2020 INR million)

States/components	Personnel	Travel	Training	eVIN software development and management	Mobile phones	Temperatureloggers	Accessories	Communication	Printing and stationery	Total
Assam	53.22	4.40	4.36	79.62	6.09	5.96	6.37	0.06	1.52	161.59
Bihar	52.21	8.32	4.64	79.62	5.27	7.95	7.02	0.06	5.24	170.34
Chhattisgarh	38.42	5.54	3.59	79.62	4.41	4.53	4.72	0.06	0.72	141.61
Gujarat	47.54	4.97	7.14	79.62	15.22	14.41	15.66	0.06	8.20	192.83
Jharkhand	44.44	4.18	1.76	79.62	2.22	2.29	2.38	0.06	0.68	137.62
Madhya Pradesh	77.77	7.91	8.10	79.62	9.29	10.97	10.73	0.06	2.53	206.99
Manipur	24.68	4.09	0.99	79.62	1.14	0.92	1.09	0.06	0.46	113.05
Nagaland	25.58	4.70	0.99	79.62	1.21	0.84	1.08	0.06	0.03	114.11
Odisha	44.55	3.79	7.05	79.62	8.36	8.79	9.07	0.06	2.38	163.67
Rajasthan	58.75	6.91	9.73	79.62	15.45	15.76	16.50	0.06	5.29	208.09
Uttar Pradesh	115.77	14.81	7.24	79.62	11.11	17.79	15.34	0.06	5.13	266.87
Total	582.94	69.63	55.60	875.80	79.77	90.21	89.96	0.70	32.19	1876.78

Note: 1 USD = INR 74.132

eVIN software development was the major investment, contributing 30% (Uttar Pradesh) to 70% (Manipur and Nagaland) of the total investment in eVIN. Investments in other states ranged from 38% (Madhya Pradesh and Rajasthan) to 58% in Jharkhand. The second major investment was for personnel. Manipur and Nagaland had the lowest share of the total investment for eVIN personnel (about 22%), followed by Gujarat (25%), Chhattisgarh, and Odisha (27%), and Rajasthan (28%). Investment in eVIN personnel in Assam, Bihar, Jharkhand, and Madhya Pradesh ranged from 31% to 38%, with the highest percentage in Uttar Pradesh (43%).

Time spent by different categories of staff related to eVIN

The total time cost related to training was estimated to be INR 64 million. Rajasthan (INR 14 million) had the highest HR cost for training, followed by Gujarat (INR 12 million) and Uttar Pradesh (INR 8 million) (not reported in table).

On average, each CCH spent 10 minutes per session day to calculate the utilization of different vaccines and enter the same in eVIN entry format on their smartphones. The total HR cost related to eVIN entry was INR 168 million. Uttar Pradesh had the highest cost (INR 56 million), followed by INR 26 million in Bihar and INR 22 million in Madhya Pradesh. Uttar Pradesh had the highest HR cost for eVIN data entry, as the number of sessions held was the highest in there among all eVIN states.

ROI related to eVIN

The total investment related to eVIN was INR 1,877 million during the period of 2014 to 2017 (Table 3); however, more than half of the investment was made in the year of implementation in each state, and on average 16% in the previous year of implementation. Start-up costs, which included training, purchase of mobile phones, temperature loggers, and accessories, contributed 51% (in Uttar Pradesh) to 75% (in Manipur) of the total investment in eVIN.

The total benefits from eVIN were estimated at INR 977 million, resulting in a ROI of INR 0.52; that is, a one-rupee investment in eVIN led to a return of INR 0.52 (Table 4). It should be noted that the net total savings of INR 977 million presented in Table 4 include savings from wastage and averted health-care costs along with savings from better vaccine stock management because of eVIN. As savings from averted health-care costs (INR 0.77 million) could not be estimated at the state level, they are not presented in Table 4. Further, state-wise savings from wastage were insignificant (0.03%) compared to savings from better stock management; hence, savings from wastage were only added in the total savings in Table 4. The highest return on eVIN investment was in Bihar, where a one-rupee investment in eVIN returned INR 1.83, and the second highest return was in Uttar Pradesh (a one-rupee investment gave a return of INR 1.27) (Table 4). When time cost was added, the ROI ratio decreased for all states, but Bihar and Uttar Pradesh still had a more-than-one-rupee return over a one-rupee investment (not reported in the table).

Table 4.

Return on investment of electronic vaccine intelligence network (eVIN). 2020 INR

States	Net savings (million)	Investment (million)	Return on investment	Savings from new vaccine (million)	Total saving (million)	Return on investment	Future investment (million)	Return on investment
Assam	90.29	161.59	0.56	71.19	161.48	1.00	68.66	2.35
Bihar	311.53	170.34	1.83	285.89	597.41	3.51	92.72	6.44
Chhattisgarh	52.90	141.61	0.37	62.22	115.13	0.81	66.63	1.73
Gujarat	21.01	192.83	0.11	130.12	151.13	0.78	94.07	1.61
Jharkhand	33.77	137.62	0.25	82.42	116.19	0.84	60.42	1.92
Madhya Pradesh	65.06	206.99	0.31	195.40	260.46	1.26	104.03	2.50
Manipur	3.48	113.05	0.03	4.42	7.89	0.07	52.11	0.15
Nagaland	1.55	114.11	0.01	3.14	4.68	0.04	52.28	0.09
Odisha	30.61	163.67	0.19	81.72	112.33	0.69	74.71	1.50
Rajasthan	20.77	208.09	0.10	179.99	200.76	0.96	101.71	1.97
Uttar Pradesh	337.66	266.87	1.27	570.63	908.29	3.40	137.68	6.60
Total	976.67	1876.78	0.52	1667.14	2649.47	1.41	905.03	2.93

*(1) State wise net savings are from better vaccine stock management only. Total net savings included savings from wastage rate and averted health care costs. As state wise data on averted health care costs were not available and state wise wastage rates were minimal, those were added in total savings only. (2) Return on investment is calculated based on financial expenditure on eVIN, time cost was not added in this calculation. (3) 1 USD = INR 74.132.

In the ROI calculation, an average saving of 11.47% from better stock management was estimated from all seven traditional vaccines (BCG, hepatitis B, pentavalent, OPV, measles, DPT, and TT) in the post-eVIN period. Assuming the same percentage of savings from the new vaccines (inactivated polio vaccine, rotavirus vaccine, pneumococcal conjugate vaccine, and measles rubella vaccine), total savings from better vaccine stock management because of eVIN will increase from INR 977 million to INR 2,649 million (Table 4). Given an investment of INR 1,877 million, a one-rupee investment in eVIN will lead to a return of INR 1.41 if these new vaccines are added in the ROI calculation (Table 4).

The reason for the initial lower ROI of eVIN was that the calculation considered start-up cost as well as recurrent expenses. In the future, start-up costs will not exist. Only recurrent costs, including the maintenance of mobile phones, temperature loggers and accessories, and salaries of the staff hired for eVIN activities and their travel expenses will remain. Assuming that half of the mobile phones, temperature loggers, and accessories need replacement every year, and personnel at the national level will spend half of their time in the coming years, the investment in eVIN will be reduced in the future. The recurrent expenditure on eVIN in 11 states in the coming years will be approximately INR 905 million. Assuming the same level of savings from traditional vaccines, eVIN will be a sound investment at the national level in the future. Adding savings from new vaccines, the return will be even higher: a one-rupee investment in eVIN will yield a return of INR 2.93 (Table 4).

Sensitivity analysis

The highest cost savings was from better stock management, and we assumed that all savings from better stock management were due to eVIN. However, during the pre- and post-eVIN periods, there may have been other factors that influenced the utilization of vaccines. For example, there could have been a delay in the procurement of vaccines that reduced the total utilization of vaccines in the post-eVIN period. As we had no data to incorporate these changes into the calculation, we conducted a sensitivity analysis assuming 75% of the cost savings from better stock management were from eVIN and the rest were from other factors. This reduced the benefit cost ratio from 1.41 to 1.18. Assuming that half of the savings were from eVIN and half from other factors, the ratio reduced further to 0.96.

As it was not possible to separate out vaccine waste by reason in the pre-eVIN period, all types of waste were considered while calculating the cost savings from wastage because of eVIN. However, wastage, such as wastage from breakage of vaccine vials, which was unrelated to eVIN was also added in the calculation. As it was not possible to calculate the proportion of wastage unrelated to eVIN, a sensitivity analysis was conducted excluding total savings from vaccine wastage from the total benefits from eVIN. As the savings from vaccine wastage were only INR 6 million, it did not have any impact on the benefit cost ratio.

Further, it was assumed that the children who missed out on immunization for the whole duration of the vaccine stockout period were not vaccinated in the future. It might be that all missed children or a proportion of the missed children were vaccinated when the vaccine was available again or the pending doses were given in the next immunization schedule of those missed children. As there was no information on the proportion of children vaccinated later when the vaccine was available, we performed a sensitivity analysis assuming that all missed children received their scheduled vaccination later. Therefore, these children will not be at risk of contracting the disease, and the estimated health-care costs averted because of missed opportunities will not be there. As savings from averted health-care costs were less than 1% of the total benefit from eVIN, excluding this benefit did not have any impact on the ROI ratio.

Discussion

In this paper, the ROI of a specific cold-chain intervention in India’s immunization program (eVIN) was presented. Globally, only a few studies have estimated the ROI of the immunization program and the costs of the vaccine supply chain.[10,11,14-17] To the best of the authors’ knowledge, this is the first comprehensive study to present an economic assessment of a cold chain intervention in the Indian context. The study found that the major benefit obtained from eVIN was better vaccine stock management. Of the 11 states for which the ROI results were presented, 2 states (Bihar and Uttar Pradesh) had more than a one-rupee benefit from a one-rupee investment when only traditional vaccines were considered. However, when recurrent expenses will only be required for eVIN and new vaccines will be fully included in the routine immunization system, eVIN will be a sound investment at the national level.

The lower initial return of eVIN can be explained by several factors. First, the ROI calculation did not consider new vaccines such as inactivated polio vaccine, rotavirus, pneumococcal conjugate vaccine, and measles rubella (MR), as these were not included in India’s immunization program in the pre-eVIN period. The ROI calculation considered only seven traditional vaccines: BCG, hepatitis B, OPV, DPT, pentavalent, measles, and TT. Bihar and Uttar Pradesh showed higher returns than investments, even with traditional vaccines. As most of the savings from eVIN came from reduced overstocking of vaccines, better usage of eVIN for real-time availability of vaccine stock data at different levels can yield a good return on the investment. For example, vaccine utilization in Uttar Pradesh was 45% lower in the post-eVIN period than in the pre-eVIN period; in Bihar, it was 28% lower. On the other hand, in Madhya Pradesh (another large Indian state), it was only 6% lower and in Rajasthan, another large Indian state, the average utilization was 28% higher in the post-eVIN period. Because of the better availability of vaccine stock information at various levels of CCPs and management in the post-eVIN period, Bihar and Uttar Pradesh had better returns on investment. When the savings from the new vaccines were incorporated into the analysis, eVIN became a sound investment considering all 11 states. Second, the cost savings from all vaccine-preventable diseases could not be calculated because of the dearth of cost of illness data in the Indian context, which led to an overall lower ROI.

The implementation of eVIN in any new state will require a few start-up expenditures, such as eVIN software development and management, training of trainers, communication materials, and personnel at the national level to supervise all activities. Based on the UNDP expenditure data, this initial investment will be about INR 87 million per state for a period of 3 years. It is expected that in any state, the implementation will be in a phased manner and the expenditure will incur gradually over a period of two to 3 years. Other expenditures such as mobile phones, temperature loggers, printing and stationery, accessories, and training depend on the number of CCPs in the respective states. Based on the expenditure pattern of 11 states, it was estimated that the average amount required per CCP will be INR 37,194 (ranging from INR 28,582 to INR 49,184). This was the estimated financial cost; it did not include the time cost required to implement the program. Human resource costs were not calculated, as the government pay scale will be different from the UNDP pay scale and will vary across states. The time cost related to eVIN estimated in this study (INR 232 million) was less than the financial cost of the program (INR 1,877 million). Additional time was required only for initial training and for entering the vaccine stock information into the eVIN software. As eVIN is not a labor-intensive intervention and the existing staff can do that after training, it would not create much burden on the existing health system. However, as this requires huge investment, to scale up the program, the government needs to ensure this additional funding. The initial implementation of eVIN covered a total of 9,856 CCPs. India has more than 27,000 functional CCPs; therefore, at least 17,144 CCPs need to be covered in the next phases if the government wants to scale it up. Considering the average cost per CCP of INR 37,194 obtained from this study, it was estimated that the government will require a minimum of INR 638 million to purchase mobile phones, temperature loggers, and accessories for the CCPs. In addition, an initial investment will be required per state (estimated average of INR 87 million per state).

Limitations

The present study has several limitations. First, the ROI analysis was conducted from a government perspective, not from a societal perspective; hence, only the cost of illness averted because of missed opportunities was considered, and the amount of savings was not significant. As the government perspective was used, savings from productivity loss and premature mortality were not included. Further, because of the non-availability of data, costs of illness averted by avoiding diphtheria, tetanus, pertussis, poliomyelitis, measles, and childhood tuberculosis were not considered in the ROI calculation. Adding these factors will further increase the benefits of eVIN. Second, as complete data were not available for the hepatitis B vaccine in Assam and Jharkhand and the OPV vaccine in Uttar Pradesh, these were not considered in the calculation, but might have some impact on the ROI calculation. Third, the major benefit of eVIN was better stock management; however, there could be factors unrelated to eVIN that could also influence the changing utilization pattern in pre- and post-eVIN periods. As these factors were not adjusted in this calculation, the ROI ratio presented in this study needs to be interpreted with caution. Fourth, the present analysis was an incremental cost analysis, existing capital costs were not considered. However, there could be some cost savings in terms of lesser cold-chain space used because of better vaccine stock management, lesser number of trips required for vaccine transport. Those were also not considered in this analysis which probably leads to an underestimation of the benefits from eVIN. Finally, after the implementation of eVIN, as there was clear information on vaccine stock availability at different levels of CCPs, OPV was better managed during the trivalent OPV to bivalent OPV switch, several near-to-expiry DPT vaccines were distributed first for better utilization, and vaccines were saved because of temperature alerts through the eVIN system. These could have some cost implications and may lead to some cost savings; however, because of the unavailability of pre-eVIN period data, they were not considered in the ROI analysis.

Conclusion

The assessment of eVIN commissioned by the government of India showed promising results in streamlining the vaccine flow network and ensuring equity in vaccine stock management through the timely supply of safe and potent vaccines[4] along with good ROI, as reported in this study. Hence, there was a rapid expansion of eVIN in all 731 districts across 36 states and union territories in the country.[18] For the initial 12 states, the states were asked to incorporate the recurrent expenses related to eVIN into their routine immunization budget for sustainability.