Evaluation of Point-of-Care Resources for Dietary Supplement Information.

OBJECTIVE: To evaluate 6 tertiary, point-of-care drug information resources' dietary supplement content.
METHODS: This was a cross-sectional evaluation of Lexicomp Natural Products Database, Micromedex Alternative Medicine, Clinical Pharmacology, Natural Medicines, The Review of Natural Products, and Handbook of Nonprescription Drugs. Each resource was evaluated for scope, completeness, consistency, and ease of use.
RESULTS: For a sample of 66 supplements, scope scores ranged from 69.7% (Micromedex) to 100% (Natural Medicines). Completeness scores were high considering uses, dose, adverse effects, and mechanism (85.7% to 100%). Overall completeness scores ranged from 82.5% ( Handbook of Nonprescription Drugs) to 100% (Clinical Pharmacology, Natural Medicines, The Review of Natural Products). Consistency scores ranged from 0% ( Handbook of Nonprescription Drugs) to 100% (Natural Medicines, The Review of Natural Products). Mean time to locate and gather information was similar among groups.
CONCLUSIONS: Resources were similar for completeness and ease of use. Scope and consistency varied depending on the resource.

Evidence-based medicine is the process of defining a clinical question, retrieving pertinent patient and scientific information, evaluating the validity of that information, categorizing the quality of the evidence, and developing a plan of action.[1] Dietary supplements are defined by the US Food and Drug Administration as products with a “dietary ingredient” that could provide nutritional value beyond that of a balanced diet.[2] Amino acids, botanicals, concentrates, constituents, extracts, herbs, metabolites, minerals, and vitamins are all classified as dietary supplements. It is estimated that 50% of the U.S. population uses dietary supplements, 18% of which involves nonvitamin, nonmineral supplement usage.[3,4] It has been estimated that dietary supplements are responsible for 3% of patient-reported adverse drug reactions and about 23 000 annual emergency department visits. Although quantitative data are scarce regarding drug-supplement interactions, it is well-known that substantial interactions do exist.[5-7]

Applying the paradigm of evidence-based medicine to dietary supplements is a challenging task for health care professionals owing to lack of high-quality evidence and underreporting of adverse events,[5] as well as the lack of requirement for Food and Drug Administration approval.[2] Additionally, clinicians may practice in settings where general access to the Internet and medical library services is limited. Previously published previous studies have identified several high-quality tertiary resources for gathering information on herbal products[8,9]; however, substantial time has elapsed since publication of those evaluations. Additionally, past studies focused specifically on herbal products, as opposed to dietary supplements as a whole. Anecdotal observations have demonstrated variability amongst resources, particularly in reference to supplement efficacy; thus, health care professionals should be aware of the quality and extent of variability of resources that are commercially available. The objective of this study was to evaluate 6 point-of-care, tertiary drug information resources for their dietary supplement content considering scope, completeness, ease of use, and consistency.

Methods

This was a cross-sectional evaluation of six distinct dietary supplement resources that could be used at the point-of-care: Lexicomp Natural Products Database,[10] Micromedex Alternative Medicine,[11] Clinical Pharmacology, Natural Medicines,[12] The Review of Natural Products (8th edition),[13] and Handbook of Nonprescription Drugs—An Interactive Approach to Self-Care [14] (18th edition). Potential resources were identified through review of published studies[8,9] and the American Association of Colleges of Pharmacy (AACP) Basic Resources for Pharmacy Education.[15] Continuously updated electronic resources and books published within three years of data collection were included. The list of resources was also shared with practicing clinicians in order to identify any additional resources that would be relevant to include. Data collection for Micromedex was completed by the investigators using the Complete Micromedex Clinical Evidence Bundle, which includes the Complete Alternative Medicine Module and comes with Royal Pharmaceutical Society herbal medicine content.

In order to analyze each resource, a sample of dietary supplements for investigation was developed. The initial search for dietary supplements was conducted with a goal of developing a clinically relevant list of supplements for investigation. This was accomplished by searching with PubMed using the following terms: “dietary supplements,” “supplements,” “herbal medicines,” “botanicals,” “common,” and “prevalence.” Published studies describing supplement use, primarily focusing on supplement-drug interactions assessments[5-7] and national supplement use data[3,4,16] were used to create the initial list. The final determinations of which dietary supplements to include in the study were based on independent review and input by three subject matter experts in clinical practice: a drug information pharmacist, an ambulatory care pharmacist, and a community pharmacist. The reviewers were instructed to differentiate which supplements they encountered most often in practice or those that were clinically relevant from those that they rarely encountered or were less relevant. Supplements that were noted as clinically relevant by at least two reviewers were included in the sample. To avoid selection bias all participants involved in the formation of the initial and final supplement list were asked to refrain from examining the resources.

Two independent reviewers collected data for the included dietary supplements (if there was an entry) from each of the six resources. The reviewers were given a common form for use during information gathering and trained on data abstraction methods in order to maintain consistency. Data collected for each supplement aligns with a previous similar study,[8] and included proposed use(s), effective dose(s) (ie, recommended dose), adverse events (ie, potential adverse drug reactions or adverse effects), and mechanism of action (ie, means through which the supplement achieves its therapeutic effect). Data collection also included measures of time: the time it took to locate the supplement within the resource and the time it took to gather the information. Any discrepancies were resolved by consensus with the primary investigator. The primary investigator then entered all data into an Excel spreadsheet for analysis. Data were gathered and entered over a 5-month period in Fall 2016.

Based on previous studies, 6 endpoints were developed in order to address the study objectives.[17-19] The first endpoint was scope (ie, does the resource contain an entry for the supplement?), which was calculated as the percentage of supplements that had a listing by resource. The second endpoint was an individual completeness score (ie, does the resource contain unambiguous information addressing the component?) regarding proposed use(s), effective dose(s), potential adverse events, and mechanism of action, that was assessed by calculating the percentage of supplements that addressed each component found in each respective resource. Overall completeness scores were also calculated through summation of each component for a total of 4 possible points for each supplement, with aggregate scores calculated by determining the percentage of supplements scoring 4 out of 4 possible points by resource. Each resource was also evaluated for whether it utilized a systematic rating system to categorize strength of evidence supporting supplement uses. Ease of use was determined for each resource by assessing the mean time taken to locate the supplement and then gather the aforementioned information. Determination of the level of consistency among resources was completed independently by the primary investigator. The primary investigator analyzed the information gathered by the independent reviewers and determined whether the information in each resource was mostly consistent, partially consistent, or mostly inconsistent with the information in the other resources. It was decided to focus on consistency among resources, rather than benchmarking, given the lack of a true “gold standard” resource for dietary supplement information. Definitions of and methods for gathering information to assess endpoints were guided by past similar studies addressing other types of drug information content; namely, general drug information, infectious diseases drug information, and drug interactions questions.[17-19] Results were assessed using descriptive statistics. The scoring results for scope, completeness, and consistency were described using percentages as noted above. Means were used to describe the ease of use of each resource.

Results

The initial search for dietary supplements for investigation generated a total of 146 potential agents. Of these, 66 were recommended for analysis by 2 or more subject matter experts and were ultimately investigated in order to gather study data. A list of the assessed agents can be found in Table 1. The sample of supplements included 36 herbs or botanicals, 11 hormones (eg, dehydroepitestosterone, melatonin) or other biological molecules (eg, amino acids, chondroitin), 10 vitamins or cofactors, 7 minerals, and 2 probiotics. See the Appendix for an overview of specific data extracted for each included supplement.

Table 1.

Final Sample of Assessed Supplements.

5-Hydroxytryptophan	Iron
Aloe vera	Kava
American ginseng	Licorice
Arginine	Magnesium
Asian ginseng	Melatonin
Belladonna	Milk thistle
Bitter orange	N-acetylcysteine
Black cohosh	Niacin
Calcium	Omega-3/fish oil
Capsicum	para-Aminobenzoic acid
Carotenoids/β-carotene	Peppermint
Chamomile	Potassium
Chondroitin	Probiotic (Bifidobacteria)
Chromium	Probiotic (Lactobacillus)
Cinnamon	Psyllium
Coenzyme Q	Red yeast rice
Cranberry	Rose hips
Creatine	Saw palmetto
Dehydroepiandrosterone (DHEA)	Selenium
Digitalis	Senna
Echinacea	Siberian ginseng
Eleuthera	Soy extract
Evening primrose	St. John’s wort
Feverfew	Tryptophan
Fiber	Valerian
Flaxseed	Vitamin A
Folic acid/vitamin B9	Vitamin B12
Garlic	Vitamin B6
Ginger	Vitamin C
Ginkgo	Vitamin D
Glucosamine	Vitamin E
Green tea	Vitamin K
Guarana	Zinc

Results for the principle endpoints of interest are provided in Table 2. Briefly, scope scores ranged from 69.7% (Micromedex[11]) to 100% (Natural Medicines[20]). Completeness scores were generally high for proposed uses, dose, adverse effects, and mechanism of action; however, resources did not consistently provide an assessment of the strength of evidence (only Micromedex[11] and Natural Medicines[20] systematically rated strength of evidence). Overall completeness scores range from 82.5% (Handbook of Nonprescription Drugs [14]) to 100% (Clinical Pharmacology,[12] Natural Medicines,[20] The Review of Natural Products [13]). Mean time to locate (0.05-0.58 minutes) and gather (2.9-4.4 minutes) information on a supplement was similar among groups. When consistency among resources was assessed, the percentage of supplements that had “mostly consistent” information compared to other resources ranged from 0% with Handbook of Nonprescription Drugs [14] to 100% with Natural Medicines and The Review of Natural Products.[13]

Table 2.

Study Results for Scope, Completeness, Time, and Consistency.

	CP	HNPD	LC	MM	NM	RNP
Scope (%)	90.9	95.5	80.3	69.7	100	75.8
Uses (%)	100	95.2	100	100	100	100
Dose (%)	100	85.7	92.5	97.8	100	100
AE (%)	100	88.9	100	97.8	100	100
Mechanism (%)	100	90.5	92.5	97.8	100	100
Overall completeness (%)a	100	82.5	92.5	97.8	100	100
Time to locate (mean minutes)	0.05	0.58	0.05	0.18	0.06	0.55
Time to gather (mean minutes)	3.2	4.4	3.2	3.3	2.9	3.0
Consistencyb	88.3	0	79.2	95.7	100	100

Abbreviations: AE, adverse effects; CP, Clinical Pharmacology; HNPD, Handbook of Nonprescription Drugs; LC, Lexicomp; MM, Micromedex; NM, Natural Medicines; RNP, The Review of Natural Products.

aPercentage of supplements scoring 4 out of 4 for completeness.

bPercentage of supplements rated as “mostly consistent.”

Discussion

The evaluated resources scored highly across all considered components, including scope, completeness, ease of use, and consistency, suggesting that there is good availability of point-of-care references that can help clinicians quickly answer general questions about dietary supplements. Strikingly higher scores were obtained for scope, completeness, and consistency, specifically, compared with prior studies evaluating drug information resources,[17-19] suggesting the evaluated resources contained thorough, high-quality content. This is especially meaningful for dietary supplement resources given the known challenges posed by dietary supplements in evidence-based medicine.[5-7] Alternatively, the observed high scores, particularly for completeness, may simply represent use of less specific and more straightforward endpoints in the present study (eg, dose, mechanism) compared to prior studies that assessed whether resources provided answers to specific drug information questions.

Natural Medicines, anecdotally considered a gold standard for dietary supplements information, performed as expected, as illustrated by its perfect results for scope, completeness, and consistency. Clinical Pharmacology[12] also demonstrated high performance across all evaluated components. Total time to locate and gather data with each resource was less than 5 minutes, suggesting that all the studied resources were reasonably efficient. Furthermore, the information gathered from each resource was generally similar, suggesting an overall consistency among the studied resources. One shortcoming noted across most resources was an apparent absence of a systematic approach for evaluating and ranking the strength of evidence supporting the resource content. The lack of a standard rating system limits the health care professional’s ability to quickly apply the fourth step of evidence-based medicine, where the health care professional categorizes the strength of evidence supporting his or her action. However, Natural Medicines and Micromedex both included rating systems that provided evidence that supported the information in these resources, instilling confidence in the quality of this information. Therefore, these resources may be preferable in answering dietary supplement questions related to efficacy.

The drug information resources evaluated were similar in terms of their completeness of information and ease of use, implying that any of the evaluated resources would be appropriate for accessing basic dietary supplement information. However, considering the variability in scope and consistency found amongst the resources that were examined, it would still be beneficial for libraries to subscribe to multiple specialty resources to ensure comprehensive access to reliable drug information. Correspondingly, the results of this study could assist institutions in making critical decisions regarding which drug information resources would be most beneficial to purchase when financial resources are limited. In an academic setting, the results of this study could help guide drug information faculty when teaching pertinent characteristics of various resources to students. For example, The Review of Natural Products [13] was very strong in terms of its content, however its scope was somewhat limited, as it lacked coverage of vitamin and mineral supplements.

The results of this study lend themselves to future research. A natural extension of this study would be to specifically examine the quality of evidence of the information presented in dietary supplement resources to evaluate the potential efficacy of included dietary supplements. Additional studies resembling the present study could also be conducted to similarly compare resources regarding management of drug toxicities or for information pertaining to pregnancy and lactation, which would be especially beneficial considering the recent implementation of new pregnancy and lactation labeling guidelines. Moreover, from an academic perspective, a future study could evaluate the quality of education regarding dietary supplement information and resources within health professions.

One strength of this study is that it is it analyzed content of point-of-care drug information resources for dietary supplement content as a whole, as opposed to focusing specifically on herbal products. The evaluation also utilized an objective, systematic process for selecting the sample of supplements and studied resources based on evidence from past work,[3-7,16] and incorporated clinician subject matter experts with diverse perspectives to narrow the potential list. This resulted in a highly relevant sample of commonly encountered products. Success of this step was demonstrated in the study by the top resource scoring 100% for scope, and others returning a variety of scores. A limitation of this study was that one of the resources, Micromedex, offers various bundles in which each bundle may contain different content. For this study, the content for inclusion was gathered from the Alternative Medicine database or the Royal Pharmaceutical Society herbal medicine content. The bundles will vary among facilities; however, more inclusive bundles can be added on depending on the needs of the facility. Also, high quality was demonstrated for all resources, particularly for completeness, making it difficult to differentiate between each one as previously described. A future investigation could employ more rigorous endpoints in order to better separate the quality of the researched resources.

In conclusion, the 6 drug information resources evaluated for their dietary supplement content were similar in terms of their ease of use and completeness regarding proposed uses, dose, adverse effects, and mechanism of action. There was, however, more variability among resources in terms of their consistency with each other and scope of included dietary supplements.

Supplement	CPa		HNPDa		LCa		MMa		NMa		RNPa
	Present	Complete	Present	Complete	Present	Complete	Present	Complete	Present	Complete	Present	Complete
5-Hydroxytryptophan	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4
Aloe vera	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4
American ginseng	No	N/A	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4
Arginine	Yes	4	Yes	2	Yes	4	Yes	4	Yes	4	Yes	4
Asian ginseng	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4
Belladonna	No	N/A	No	N/A	No	N/A	Yes	4	Yes	4	No	N/A
Bitter orange	Yes	4	Yes	2	Yes	4	Yes	4	Yes	4	Yes	4
Black cohosh	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4
Calcium	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4
Capsicum	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4
Carotenoids	Yes	4	Yes	4	Yes	4	No	N/A	Yes	4	Yes	4
Chamomile	Yes	4	Yes	1	Yes	4	No	N/A	Yes	4	Yes	4
Chondroitin	No	N/A	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4
Chromium	Yes	4	Yes	4	Yes	4	No	N/A	Yes	4	Yes	4
Cinnamon	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4
Coenzyme Q	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4
Cranberry	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4
Creatine	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4
DHEA	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4
Digitalis	Yes	4	Yes	1	Yes	4	No	N/A	Yes	4	Yes	4
Echinacea	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4
Eleuthera	No	N/A	Yes	4	Yes	4	No	N/A	Yes	4	Yes	4
Evening primrose	No	N/A	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4
Feverfew	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4
Fiber	Yes	4	Yes	4	Yes	4	No	0	Yes	4	No	0
Flaxseed	Yes	4	Yes	2	Yes	4	Yes	4	Yes	4	Yes	4
Folic acid	Yes	4	Yes	4	Yes	0	Yes	4	Yes	4	Yes	0
Garlic	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4
Ginger	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4
Ginkgo	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4
Glucosamine	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4
Green tea	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4
Guarana	Yes	4	Yes	2	Yes	4	Yes	4	Yes	4	Yes	4
Iron	Yes	4	Yes	4	Yes	0	Yes	4	Yes	4	Yes	0
Kava	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4
Licorice	Yes	4	Yes	1	Yes	4	No	N/A	Yes	4	Yes	4
Magnesium	Yes	4	Yes	4	No	N/A	No	N/A	Yes	4	No	N/A
Melatonin	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4
Milk thistle	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4
N-acetylcysteine	Yes	4	No	N/A	No	N/A	No	N/A	Yes	4	No	N/a
Niacin	Yes	4	Yes	4	Yes	0	Yes	0	Yes	4	Yes	0
Omega-3/fish oil	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4
para-Aminobenzoic acid	Yes	4	Yes	4	No	N/A	No	N/A	Yes	4	No	N/A
Peppermint	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4
Potassium	Yes	4	Yes	0	Yes	0	Yes	0	Yes	4	Yes	0
Probiotic (Bifidobacteria)	No	N/A	Yes	3	Yes	2	No	N/A	Yes	4	Yes	4
Probiotic (Lactobacillus)	Yes	4	Yes	3	Yes	4	Yes	4	Yes	4	Yes	4
Psyllium	Yes	4	Yes	4	Yes	4	No	N/A	Yes	4	Yes	4
Red yeast rice	Yes	4	Yes	4	Yes	4	No	N/A	Yes	4	Yes	4
Rose hips	Yes	4	Yes	1	Yes	2	Yes	1	Yes	4	Yes	4
Saw palmetto	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4
Selenium	Yes	4	Yes	4	No	N/A	No	N/A	Yes	4	No	N/A
Senna	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4
Siberian ginseng	Yes	4	Yes	4	Yes	4	No	N/A	Yes	4	Yes	4
Soy extract	Yes	4	Yes	4	Yes	2	No	N/A	Yes	4	Yes	4
St. John’s wort	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4
Tryptophan	Yes	4	Yes	1	Yes	2	No	N/A	Yes	4	Yes	4
Valerian	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4
Vitamin A	Yes	4	Yes	4	No	N/A	Yes	4	Yes	4	No	N/A
Vitamin B6	Yes	4	Yes	4	No	N/A	Yes	4	Yes	4	No	N/A
Vitamin B12	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4
Vitamin C	Yes	4	Yes	4	No	N/A	Yes	4	Yes	4	No	N/A
Vitamin D	Yes	4	Yes	4	Yes	4	No	N/A	Yes	4	Yes	4
Vitamin E	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4
Vitamin K	Yes	4	Yes	4	No	N/A	Yes	4	Yes	4	No	N/A
Zinc	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4	Yes	4

Abbreviations: AE, adverse effects; CP, Clinical Pharmacology; DHEA, dehydroepiandrosterone; HNPD, Handbook of Nonprescription Drugs; LC, Lexicomp; MM, Micromedex; N/A, not applicable; NM, Natural Medicines; RNP, The Review of Natural Products.

aPresent: Yes = entry for supplement, No = no entry for supplement. Complete: 1 point each for use, dose, adverse effects, and mechanism.