Global status of essential medicine selection: a systematic comparison of national essential medicine lists with recommendations by WHO.

OBJECTIVES: Examining the availability of essential medicines is a necessary step to monitor country-level progress towards universal health coverage. We compared the 2017 essential medicine lists (EML) of 137 countries to the WHO Model List to assess differences by drug class and country setting.
METHODS: We extracted all medicines prioritised at country level from most recently available national EMLs and compared each national EML with the 2017 WHO Model List of Essential Medicines (MLEM) as the reference standard. We assess EMLs by WHO region and for different types of medicine subgroups (eg, cancer, anti-infectives, cardiac, psychiatric and anaesthesia medicines) using within second-level anatomical therapeutic class (ATC) drug classes of the ATC Index.
RESULTS: We included 406 medicines from WHO's 2017 MLEM to compare to 137 concurrent national EMLs. We found a median of 315 (range from 44 to 983) medicines listed on national EMLs. The global median F1 score was 0.59 (IQR 0.47-0.70, maximum possible score indicating alignment with MLEM is 1). The F1 score was the highest (ie, most similar to MLEM) in the South-East Asia region and the lowest in the European region (ie, most dissimilar to MLEM). The F1 score was highest for stomatological preparations (median: 1.00), gynaecological-anti-infectives and antiseptics (median: 1.00), and medicated dressings (median: 1.00), and lowest for 9 anatomical or pharmacological groups (median: 0.00, eg, treatments for bone diseases, digestive enzymes).
CONCLUSIONS: Most countries are expected to improve their national health coverage by 2030 offering access to essential medicines, but our results revealed substantial gaps in selection of medicines at the national level compared with those recommended by WHO. It is crucial that governments consider investing in those effective medicines that are now neglected and continue monitoring progress towards essential medicine access as part of universal health coverage. © Author(s) (or their employer(s)) 2022. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

We present a novel data science statistical approach using the F1 statistic, not yet extensively used in the health sciences field to assess the proximity of national essential medicine lists (EMLs) to the Model List of Essential Medicines.

We present complex visualisations to support deeper understanding of national EMLs by country, WHO region and drug class.

A limitation of this study including the subjectivity of drug class coding and heterogeneity in year of listing in national EMLs.

Background

Essential medicine lists (EMLs) are critical to prioritising evidence-based interventions that people around the world should have access to, and governments have to work to fund. The adoption of these priority tools into public policy could generate important savings by concentrating competition on a smaller number of medicines and better negotiating medication prices. In some settings, EMLs may drive medicine procurement decisions and in other settings national insurers will use EMLs for reimbursement decisions. Approximately 137 countries out of 194 WHO member states have formal national EMLs (70.6%).1 Since 1977, the WHO has updated the Model List of Essential Medicines (MLEM) every 2 years.2 3 The MLEM, which includes all medicines that are considered necessary for all health systems, provides guidance to governments, health facilities and procurers on which medicines are the best value in terms of benefits for individuals and communities.2 3 Countries, regardless of development and resource level, can base their own national lists on the MLEM.4 Because the adoption of the MLEM offers clues on that availability of effective treatments at country levels and the amount of waste related to medicines with limited value, it is a key tool for achieving universal health coverage (UHC). Focusing on a finite list of essential medicines represents an opportunity to limit the continued increases in country care expenditures. While few items in the MLES are highly priced, listing is the first necessary step to activate virtuous policies targeting drug prices.5 For these reasons, the list is primarily targeted at public policymakers in member states. However, it is of interest to several target audiences, including the general public, healthcare professionals, managers working in health facilities (eg, hospitals) or regional policymakers (eg, at the level of districts).

Examining the availability of essential medicines and associated diseases at the country level is a necessary step to follow country-level progress towards UHC. In a previous study, we measure 2017 baseline of national EMLs, and analyse global attainment as compared with essential medicines recommended by 2017 WHO MLEM.6 In the present study, we take this analysis further to explore attainment of essential medicines listing coverage at the level of individual drug class, presenting trends and substantial deviations by WHO region and drug class. In particular, we explore analysis and visualisations using a single entity, the F1 statistic to assess national EMLs in relation to the MLEM. Our hope is that such stock-taking informs discussion on how countries can improve the selection of categories of medicines for their populations and how WHO could better support member states in identifying medicines that are more effective than others within these categories.

Methods

Using the WHO Global Essential Medicines database,1 we extracted all medicines prioritised at country level from the most recently available EM national lists. This database draws on national EMLs that have been included in the WHO repository and does not directly draw from WHO member states for the purposes of this paper. The database consists of 137 country EMLs and the validation and full methods of extraction are described elsewhere.6 The database contains the absolute majority of listed items in MLEM, with few omissions (eg, condoms, blood and its derivatives), which were excluded as not pertinent for the present study, as they are often outside the remit of medicine selection national authorities. The MLEM includes medicines with a square box indicator, which denotes therapeutic equivalence with other medications in the same class.6 For the purpose of this study we have assumed that for square box MLEM medicines any class therapeutic equivalent alternative listed by national EMLs is a matching entry.

We conducted this analysis comparing national EMLs to the WHO MLEM by applying concepts of test accuracy, where the national EML was the index tests and the MLEM the reference standard. We considered a true positive to be a medicine listed on a national EML that is also listed on the MLEM. In the context of NEMLs, the definition of true negatives is somewhat arbitrary, as it may well include all medicines available in some markets that are not listed on the MLEM. Using a conservative approach, we considered all of the possible medicines that are not listed by the MLEM to be true negatives. The list and number of true negative medicines was derived from all medicines listed on any NEML that are not on the MLEM. A false positive was a medicine listed on a national EML that is not listed on the MLEM and a false negative was a medicine listed on the MLEM but not listed on a national EML. Sensitivity (also called true positive rate (TPR) or recall) was defined as the proportion of medicines on a national EML out of all medicines recommended by the MLEM, and false positive rate (FPR) as the proportion of medicines on a national EML out of all possible medicines listed by any national EML or the MLEM globally.

First, we estimate differences between the reference standard and index tests using the TPR, (sensitivity) and FPR, (1-specificity). We generated a plot of the sensitivity against the FPR for all medicines included in the national EMLs compared with the MLEM. The analysis and relative plot define which countries are associated with optimal medicine selection and which one are at risk of hazardous selection, similarly to a receiver operator characteristic (ROC) curve but without thresholds for test cut-off values.

Second, we conducted an analysis based on a harmonic mean (ie, F1 score), of the sensitivity and precision (also called positive predictive value) as a single measure of performance. The F1 score (described in visual format in figure 1) is a well-established single measure of performance.7 Here we use it as a single measure of performance of the national list for positive list entries, with its best value at 1, and worst value at 0. In this context the mathematical property of the harmonic mean tends to give more weight to countries with shortest lists (which often will have better sensitivity and precision), as opposed to the arithmetic mean, which is more impacted by countries with large listings of medicine. The advantage of the F1 score, is that it does not incorporate true negatives, as is done in the FPR calculation. For this reason, the F1 score has been often used in health data science fields, such as in the study of machine learning models trained on electronic medical records.8

Figure 1

Defining sensitivity and precision in the context of the medicines on national EMLs, visual diagram adapted from Wikipedia.19 EMLs, essential medicine lists.

We performed analyses in Python, V.3.6.5 (Python Software Foundation) and Pandas library. We prepared figures in Seaborn (doi: 10.5281/zenodo.592845) and Tableau (Seattle, Washington, 2019.1). The dataset for this analysis is available in an interactive dashboard at: http://essentialmeds.org/.9 We include graphical representation the ROC plot of sensitivity against 1 – specificity. We also present the box-and-whisker plot of the TPR and F1 statistic for core (ie, ambulatory or community-based medicines) and complementary (ie, specialty or hospital-based medicines) lists, by WHO region, and by drug anatomical therapeutic class (ATC) level 2 categorisation of medicines.10 Finally, we present a heat map of the F1 statistic by ATC level 2 class and country.

Patient and public involvement

Patients and public were not specifically involved in the conduct of this research. However, the MEML is a highly democratic process, in which all requests for change are published, and open for public review and comment.

Results

There were 425 entries on the WHO’s 2017 MLEM, of which we included 406 medicines. We excluded certain MLEM entries that are not per se medicines (eg, devices such as condoms). National EMLs had a median of 315 medications listed (IQR 268–421; range 44–983). Further descriptive analyses on the countries and medicines included are available in Persaud et al.6 There were 2049 medicines in total identified. Differences between the national EMLs and the MLEM varied by drug class and WHO region.

National EMLs had a median TPR (sensitivity) of 54.5% (IQR 47.2%–63.3%) and a median FPR of 5.6% (IQR 3.7%–11.1%). The TPR was the highest in the Pakistan EML (84.5%, 344 true positive medicines) and the lowest in the Cambodia EML (8.6%, 35 true positive medicines). The FPR was the highest for the Slovakia EML (41.8%, 694 false positive medicines) and the lowest in both Somalia and Cambodia EMLs (0.5%, 9 false positive medicines). That means, that in Slovakia 41.8% of medicines on the EML were not on the MEML but only 0.5% in Cambodia were not on the EML.

Results of the TPR (sensitivity) and FPR (1 - specificity) for each national EML in relation to the MLEM are presented in figure 2. National EMLs in the top left of the plot (eg, Pakistan) have the highest sensitivity and lowest FPR. Countries such as Cambodia and Angola, bottom left of the plot, have a low FPR by also a low sensitivity, while countries such as Portugal, Czech Republic, Tunisia and Romania, top right of the plot, have higher sensitivity, but also high FPR. As expected, the visual inspection of the plot shows a general trend to increasing FPR with increasing sensitivity, that is, the more a country lists the more it diverges from the MEML. However, this trend does not apply to several countries, including Pakistan, which stands as an outlier with a high sensitivity and low FPR, and countries such as Algeria, Bulgaria and Poland, with a relatively low TPR and a higher FPR, that is, their lists only marginally overlap with WHO recommended options. Assessing figure 2 for colour denoting WHO regions, there is a trend towards a lower FPR and lower TPR for Africa, South-East Asia and the Western Pacific.

Figure 2

Essential medicine list receiver operator curve (sensitivity vs 1 – specificity). In this figure, we present the sensitivity (true positive rate) plotted against 1 – specificity (false positive rate). Circles represent each national EML and circle size represents the total number of medicines listed. Circle colour represents who region. National EMLs in the top left of the plot have the highest true positive rate and lowest false positive rate. Many outliers exist, however, this plot demonstrates a general trend to increasing false positive rate with increasing true positive rate. EMLs, essential medicine lists.

Our analysis of the TPR for core and complementary MLEM medicines, presented in figure 3, demonstrates substantial variation by WHO region. TPR is higher for the core essential medicines in every WHO region. While Eastern Mediterranean, Europe, and the Americas have a smaller difference between the sensitivity of core and complementary essential medicines, Africa, South East Asia and Western Pacific have large differences indicating that many complementary essential medicines often used in secondary care institutions are not being listed in these regions.

Figure 3

Box and whisker plot of true positive rate for core and complementary EML by WHO region. This figure demonstrates the median, min, max, and IQR, in a box-and-whisker plot for the true positive rates of core and complementary essential medicines by WHO region. True positive rates are higher for the core essential medicines in every WHO region. While eastern Mediterranean, Europe and the Americas have a smaller difference between the true positive rates of core and complementary essential medicines, Africa, South East Asia and Western Pacific have large differences indicating that many complementary essential medicines are not being listed in these regions.

The global median F1 score of the national EMLs in reference to the MLEM was 0.59 (IQR 0.12). The national EML with the highest F1 score, denoting closest alignment to the MLEM, was Pakistan (0.88) and the lowest F1 score, denoting greatest deviation from the MLEM, was Cambodia (0.16). In our analysis by WHO region, as we present in table 1 and in a box-and-whisker plot in figure 4, shows that the F1 score was the highest in the South-East Asia, and the lowest in the Europe region. The variability of the F1 score was the highest in Europe region and the lowest in the Americas, indicating similar selection patterns in the region.

Table 1

F1 statistic median by who region for all medications on model EML compared with national EML

Region	# of National EMLs	f1 median	f1 first quartile	f1 third quartile
African	36	0.62	0.58	0.65
Americas	30	0.60	0.58	0.63
Eastern Mediterranean	16	0.57	0.50	0.63
European	26	0.49	0.41	0.56
South-East Asia	11	0.64	0.60	0.67
Western Pacific	18	0.51	0.46	0.57

EMLs, essential medicine lists.

Figure 4

Box and whisker plot of F1 statistic by who region. This figure demonstrates the median, min, max, and IQR, in a box-and-whisker plot for F1 statistic for each who region. This figure demonstrates the lowest median F1 statistic for Europe (0.49) and the highest for south-east Asia (0.64). As a marker of within region variability, Europe has the largest IQR (0.16), and the Americas demonstrates the lowest (0.05).

We present our analysis of the F1 statistic by ATC second-level class in table 2, box-and-whisker plot in figure 5, and in heat map format by national EML in figure 6. The F1 score was the highest for A01—stomatological preparations (caries prophylactic agents (eg, sodium fluoride), anti-infectives and antiseptics (eg, metronidazole) for local oral treatment and corticosteroids for local oral treatments; median 1.00, IQR 0.52), D09—medicated dressings (eg, chlorhexidine, povidone iodine; median 1.00, IQR 0.00) and G01—gynaecological antiinfectives and antiseptics (antibiotics, imidazole derivatives (eg, nystatin), corticosteroids; median 1.00, IQR 0.33) and 0.00 for a number of categories, including medicines with controversial therapeutic roles such as treatments for bone diseases and digestive enzymes. For certain medicine classes, including D09—medicated dressings there is a high F1 (F1 median 1.00) and low IQR (F1 IQR 0.00). For other classes, including D07—corticosteroids, dermatological preparations there is a moderate median F1 (F1 median 0.67), yet a high IQR (F1 IQR 0.60) denoting significant within class variability of the F1 score.

Table 2

F1 statistic median (in descending F1 median) by ATC medication category for model EML compared with national EML

ATC and medicine class name	f1 median	f1 first quartile	f1 third quartile	IQR
A01 Stomatological preparations	1.00	0.74	1.26	0.52
D09 Medicated dressings	1.00	1.00	1.00	0.00
G01 Gynecological antiinfectives and antiseptics	1.00	0.83	1.17	0.33
A11 Vitamins	0.80	0.73	0.87	0.15
A12 Mineral supplements	0.80	0.68	0.92	0.24
C03 Diuretics	0.80	0.70	0.90	0.19
A03 Drugs for Functional Gastrointestinal Disorders	0.80	0.64	0.96	0.31
N01 Anesthetics	0.77	0.63	0.90	0.27
B03 Antianemic preparations	0.75	0.70	0.80	0.11
C08 Calcium channel blockers	0.75	0.62	0.88	0.26
H02 Corticosteroids for Systemic Use	0.75	0.64	0.86	0.22
N03 Antiepileptics	0.73	0.64	0.82	0.18
C07 Beta blocking agents	0.72	0.65	0.79	0.13
J07 Vaccines	0.69	0.56	0.81	0.25
J01 antibacterials for systemic use	0.67	0.60	0.74	0.14
C01 Cardiac therapy	0.67	0.56	0.77	0.21
S01Ophthalmologicals	0.67	0.55	0.78	0.22
B01 Antithrombotic agents	0.67	0.55	0.78	0.22
L04 Immunosuppressants	0.67	0.52	0.82	0.30
C02 Antihypertensives	0.67	0.49	0.84	0.36
H03 Thyroid therapy	0.67	0.52	0.81	0.29
C05 vasoprotectives	0.67	0.47	0.87	0.40
D07 Corticosteroids, dermatological preparations	0.67	0.37	0.97	0.60
H01 Pituitary and hypothalamic hormones and analogues	0.67	0.56	0.78	0.22
M04 Antigout preparations	0.67	0.50	0.83	0.33
P03 Ectoparasiticides, Including scabicides, insecticides and repellents	0.67	0.41	0.92	0.51
N05 Psycholeptics	0.63	0.52	0.74	0.22
J04 Antimycobacterials	0.62	0.54	0.71	0.17
L01 Antineoplastic agents	0.62	0.43	0.80	0.38
A02 Drugs for acid related disorders	0.60	0.45	0.75	0.31
A07 Antidiarrheals, intestinal antiinflammatory/antiinfective agents	0.60	0.49	0.71	0.21
A10 Drugs used in diabetes	0.60	0.52	0.68	0.17
D01 Antifungals for dermatological use	0.60	0.52	0.68	0.17
D06 Antibiotics and chemotherapeutics for dermatological use	0.60	0.47	0.72	0.25
J06 Immune sera and immunoglobulins	0.60	0.47	0.73	0.27
N02 Analgesics	0.59	0.48	0.69	0.21
A06 Drugs for constipation	0.57	0.49	0.65	0.17
R03 Drugs for obstructive airway diseases	0.57	0.49	0.65	0.17
J02 Antimycotics for systemic use	0.57	0.48	0.66	0.18
N04 Anti-Parkinson drugs	0.57	0.41	0.73	0.31
R05 Cough and Cold Preparations	0.57	0.40	0.75	0.35
B02 Antihemorrhagics	0.54	0.38	0.70	0.33
G03 Sex hormones and modulators of the genital system	0.52	0.44	0.61	0.17
N06 Psychoanaleptics	0.50	0.36	0.64	0.28
P02 Anthelmintics	0.50	0.32	0.68	0.35
M03 Muscle relaxants	0.50	0.33	0.67	0.33
N07 Other nervous system drugs	0.50	0.37	0.63	0.27
A04 Antiemetics and antinauseants	0.50	0.17	0.83	0.67
D10 Anti-acne preparations	0.50	0.17	0.83	0.67
G02 Other gynecologicals	0.50	0.10	0.90	0.80
J05 Antivirals for systemic use	0.49	0.40	0.57	0.17
V03 All other therapeutic products	0.44	0.31	0.58	0.27
C09 Agents acting on the renin-angiotensin system	0.44	0.23	0.66	0.43
M01 Antiinflammatory and antirheumatic products	0.44	0.31	0.58	0.26
P01 Antiprotozoals	0.41	0.27	0.56	0.29
L02 Endocrine therapy	0.40	0.33	0.47	0.15
L03 Immunostimulants	0.40	0.31	0.49	0.17
R01 Nasal preparations	0.40	0.10	0.70	0.60
C10 Lipid modifying agents	0.22	0.00	0.56	0.67
R06 Antihistamines for systemic use	0.17	0.03	0.32	0.29
A09 Digestives, includingenzymes	0.00	0.00	0.25	0.50
B05 Blood substitutes and perfusion solutions	0.00	0.00	0.25	0.50
D02 Emollients and protectives	0.00	0.00	0.25	0.50
D04 Antipruritics, including antihistamines, anesthetics, etc	0.00	0.00	0.50	1.00
D05 Antipsoriatics	0.00	0.00	0.33	0.67
D08 Antiseptics and disinfectants	0.00	0.00	0.00	0.00
D11 Other dermatological preparations	0.00	0.00	0.50	1.00
H04 Pancreatic hormones	0.00	0.00	0.50	1.00
M05 Drugs for treatment of bone diseases	0.00	0.00	0.13	0.26

ATC, anatomical therapeutic class; EMLs, essential medicine lists.

Figure 5

Box and whisker plot of F1 statistic for all national Essential Medicine Lists by ATC level two drug class. This figure demonstrates the median, min, max and IQR, in a box-and-whisker plot for the F1 statistic by ATC level two drug class. The colours present level 1 groupings of drug class. For certain drug classes, including A11—vitamins and B03—antianemic preparations, there is a high median F1 and low IQR. For other classes, including D04—antipruritics, D11—other dermatological preparations, H04—pancreatic hormones, the IQR of the F1 statistic ranges from 0 to 1. ATC, anatomical therapeutic class.

Figure 6

Heat map of F1 statistic by National EML list and ATC drug class (alternative presentations provided). This figure demonstrates a heat map of the F1 statistic by drug class for each national EML, grouped by WHO region. As is demonstrated, there is substantial variation in the F1 statistic by National EML and by drug class. ATC, anatomical therapeutic class; EML, essential medicine lists.

Discussion

In this study, we have found substantial variability in listing between national EML and the MLEM across therapeutic classes and WHO regions. This suggests limited interest in or difficulties in co-ordinating medicine prioritisation and a high risk of waste of health system resources from low value choices. In the context of efforts towards UHC, achieving value in medicine investment, through a focus on essential medicines, is a critical approach.

In 2017, we collected and analysed all national EMLs to measure if they align with those medicines recommended by WHO.6 The number and complexity of national documents supporting listed medicines suggest that countries invested a significant effort in prioritising medicines. However, this amount of energy resulted in a very heterogenous scenario, with countries making inconsistent selection choices, irrespective of their average income. We expanded the analyses to evaluate community and hospital-based medicines and pharmacological class across WHO world regions. Most countries are already selecting primary care and infectious disease medicines privileging those items that ensure best returns in terms of health, whereas selection of specialty or hospital-based medicines are in need of major improvements to broaden coverage of relevant diseases targeted by these medicines.

For many years, the WHO Model List has been viewed by some as including mostly medicines for infectious disease syndromes and off-patent medicines, and as being applicable only to middle-income countries or resource-constrained settings.11 This has never been true as the List always selected medicines relevant to any world region. In recent years, the MLEM has updated and expanded its sections on chronic and non-communicable diseases, including cancer and autoimmune conditions, to reflect shifting global patterns of disease burden and the ageing population.11 Since 2013 the number of patented agents on the MLEM has been stable, oscillating between 5% and 10% of all listed medicines.12 The availability of targeted and biotech-based medicines (eg, biologics such as trastuzumab for breast cancer), that typically have relatively high costs, is reinforcing the global role of the Model List as a guide of a limited number of highly effective medicines. The problem might not be with few high priced, highly effective medicines but with the plethora of highly priced marginally or non-effective items, which seems to be pervasive in several countries as identified in this analysis.

Most countries are expected to improve their national health coverage by 2030 offering access to a higher number of essential medicines, although our results revealed substantial gaps in which medicines are selected at and beyond the national level. It is crucial that governments invest in those effective medicines that are now neglected and continue to monitor progress on the promise of UHC, particularly for therapeutic classes with a low F1 statistic including blood substitutes, antihistamines for systemic use, and medicines for treatment of bone diseases. It is worth noting that in the bone disease group the MLEM makes highly selective recommendations, including injectable zoledronic acid treatment for malignancy-related bone disease. Efforts to examine and explain areas where large range in the F1 statistic exist are important to identify opportunities to better align the MLEM and national EMLs.

There are several potential interpretations of findings related to misalignment of national EMLs and the MLEM. It is possible that the WHO List either does not make the selection at the right time, anticipating or postponing medicine recommendations when countries do not contemplate or have already made their decisions, or that it prioritises medicines that are of less priority or not considered at country level. Another and perhaps more salient explanation for the misalignment is that the rationale for essential medicines selection might not be efficiently disseminated to countries. Relatively little attention has been given by WHO to its role and responsibility related to effective dissemination of its rigorous evaluation of EMs to date. Since 1977, recommendations of the Expert Committee are presented in the Technical Report Series, a report of the EML which summaries the decisions only of those medicines for which an application was presented.2 There is, however, not yet a repository of all decisions made by the Committees over time. We are in the process of developing this repository. This means that member states cannot easily retrieve, appraise and interpret the evidence used for developing the List. Progress in the way that WHO disseminates MLEM to member states, including the use of the electronic list now available in essentialmeds.org, and in how it supports member states in their efforts to adapt and implement MLEM, will require strong leadership.

The second potential interpretation is that the process to develop national EM lists at a country level in certain countries is less restrictive, or more apt to select medicines, than that of WHO and responds more to the pressure of the market to list additional items. Alternatively, when member states adapt global recommendations that take into account local needs, conditions, resources, costs and values, the local adaptation may have far reaching consequences, resulting in listing different medicines. This requires exploring how countries undertake the local list-development processes, ensuring that the process is transparent, and differences between the MLEM and national EM lists are justified.4 However, many countries do not clearly report on how they use the MLEM to inform the development of their own national EMLs. Decisions and methods rely heavily on local EM committees that rarely present in detail reasons beyond listing.

Strengths

This paper presents a novel approach for a single score, the F1 statistic, to assess the proximity of national EMLs to the MLEM. We propose that this statistic, broadly used in the data science field, could be more used in the health sciences field. We have used a large database to explore a previously under researched topic, the listing of medicines on EMLs. Furthermore, we have presented analyses and visualisations to assess a broad range of medicine classes for a large number of countries. This exploratory analysis also presents trends that can be further analysed in subsequent research work.

Limitations

There are limitations to this present work. Limitations of the database used, including heterogeneous years of national EML listing, and subjectivity of ATC coding in the database are discussed elsewhere.6 13 With respect to the years of listing, in extraction of national EMLs for the development of this database, we used the most recently available EML, which for some countries is now quite out of date. For example, the EML for Gambia that was most recently available for update and inclusion in the GEM database was from 2001. As such, there may be limitations in comparing to the 2017 MLEM due to evolution of the included medicines. Finally, our evaluation is limited to the availability of essential medicines in official government documents. Results can or cannot translate in availability at patient level.

Implications for policy

Our analysis provides evidence for improving the transparency around decisions to include medicines on EMLs. Some degree of variability is expected to account for contextualisation based on local epidemiology or resources. However, the vast differences observed between different EMLs, and the significant variability within WHO regions, suggest that further transparency and consistency is necessary. For areas where we have indicated there are significant deviations, reflected by a low F1 statistic, there is a need to explore at a country and medicine level whether these are important and countries may wish to reconsider whether they should be listing missed medicines or reconsidering medicines not listed by the MLEM or many other countries.

Implications for research

Future research should explore the differences observed by groupings of medication class, WHO regions, and core versus complementary medicine listings. Analysis of specific medication differences within these groups will allow increased understanding of the significance and importance of these differences. Analyses over time, which we are currently conducting and will be available on the website (essentialmeds.org) will allow an understanding of how older national EMLs compare to the historical MLEM. Our research also highlights the importance of research into the availability of medicines from EMLs. We use official listings, but our understanding of implementation of these lists to support access on the ground is still limited and further research required.14 15 Future research continue to assess medicine listings on NEMLs by disease groups and for focused disciplinary as has been done for tobacco addiction, diabetes and heart disease among other topics.16–18

Methodologically, we have used innovative methods including the ROC and F1 statistic that should be considered for future research on essential medicines. We propose that the F1 statistic be considered in analyses of EMLs in relation to the WHO MLEM, due to its ability to present a single measure in relation to the MLEM.

Research should also assess divergences from the EML in the context of contextualisation. WHO has always maintained that the EML should be contextualised to country context. An example of appropriate contextualisation would include differing local disease burden. It is not yet known what constitutes appropriate contextualisation of the list, and how this differs for different medication classes. Research assessing divergence by drug class in the context of disease burden would be helpful to explore divergence further. Finally, further research is needed to better understand how listing on an EML translates to access policies and availability of medicines for patients, the ultimate goal. Simply listing medicines is not going to solve the problem of scarce coverage, but it is a necessary first step to enable identification of priority medicines and the subsequent tracking of their availability.

Conclusions

This work highlights divergence in EML listing in countries that are particularly pronounced for certain geographies, medication classes and the MLEM complementary medicine listings. Increased attention is needed to EMLs as countries work towards achieving UHC. Lists of medicines that should be accessible and covered, and that constitute the most essential medicines, are important to this endeavour. This work enhances understanding of medicine listings and highlights the importance of increasing the transparency of decisions to add or remove medicines from national EMLs. We hope increased transparency will translate into better lists, and better access to essential medicines.