Searching biomedical databases on complementary medicine: the use of controlled vocabulary among authors, indexers and investigators.

BACKGROUND: The optimal retrieval of a literature search in biomedicine depends on the appropriate use of Medical Subject Headings (MeSH), descriptors and keywords among authors and indexers. We hypothesized that authors, investigators and indexers in four biomedical databases are not consistent in their use of terminology in Complementary and Alternative Medicine (CAM).
METHODS: Based on a research question addressing the validity of spinal palpation for the diagnosis of neuromuscular dysfunction, we developed four search concepts with their respective controlled vocabulary and key terms. We calculated the frequency of MeSH, descriptors, and keywords used by authors in titles and abstracts in comparison to standard practices in semantic and analytic indexing in MEDLINE, MANTIS, CINAHL, and Web of Science.
RESULTS: Multiple searches resulted in the final selection of 38 relevant studies that were indexed at least in one of the four selected databases. Of the four search concepts, validity showed the greatest inconsistency in terminology among authors, indexers and investigators. The use of spinal terms showed the greatest consistency. Of the 22 neuromuscular dysfunction terms provided by the investigators, 11 were not contained in the controlled vocabulary and six were never used by authors or indexers. Most authors did not seem familiar with the controlled vocabulary for validity in the area of neuromuscular dysfunction. Recently, standard glossaries have been developed to assist in the research development of manual medicine.
CONCLUSIONS: Searching biomedical databases for CAM is challenging due to inconsistent use of controlled vocabulary and indexing procedures in different databases. A standard terminology should be used by investigators in conducting their search strategies and authors when writing titles, abstracts and submitting keywords for publications.

Background

The increasing research in Complementary and Alternative Medicine (CAM) and the importance placed on practicing evidence-based CAM require ready access to the CAM scientific literature. The optimal retrieval of a literature search in biomedicine depends on the appropriate use of Medical Subject Headings (MeSH), descriptors and keywords among authors, indexers, and investigators [1]. It has been recognized that available online databases covering CAM differed in their thesaurus construction and indexing procedures, making effective and efficient searching difficult [2].

The controlled vocabulary for biomedicine has been developed and continuously updated by the National Library of Medicine (NLM). It is referred to as the NLM Medical Subject Headings (MeSH). The purpose of MeSH is to provide uniformity and consistency to the indexing of the biomedical literature [3]. With the recent development of CAM on PubMed [4,5], MeSH descriptors for CAM have been expanded.

As of December 2002, there were a total of 21,973 MeSH descriptors found in the NLM Medical Subject headings – Annotated Alphabetic list, 2002 [6]. While the National Center for Complementary and Alternative Medicine (NCCAM) has identified over 360 healing modalities, MeSH currently included only 83 descriptors for CAM, arranged hierarchically under the sets of terms in Complementary Therapies[7]. In 2001, there were only 41 MeSH descriptors for CAM, arranged under Alternative Medicine[8]. Cross-references have been available to assist searchers in finding the most appropriate MeSH Heading, for example, Alternative Medicine see Complementary Therapies[9]. Therapeutic Cults was the MeSH descriptor between 1963 and 1993. It was only in 1994 that the term Alternative Medicine was implemented in the NLM MeSH thesaurus. Medicine, Tibetan Traditional was the only new MeSH descriptor added under Complementary Therapies in the 2003 NLM Medical Subject Headings [10].

Other biomedical databases that include CAM literature, such as CINAHL – Cumulative Index to Nursing and Allied Health Literature [11] and MANTIS – Manual, Alternative and Natural Therapy Index System [12], also use MeSH as their standard thesaurus with a list of supplement terms in their subject areas. The CINAHL medical subject headings include also a broad range of terms for research methodology including, for example, 12 subject headings for validity [13].

The main purpose of database indexing is to enhance the yield and accuracy of search results. Indexing is done either manually or generated by computer programs. Indexers carry out indexing manually in MEDLINE, CINAHL, and MANTIS according to their own perceptions and understanding of the study contents. In comparison, computer programs generate indexing in the Web of Science database. These programs pool words or phrases and group them by frequency of use [14]. Computer generation of keywords is referred to as semantic indexing while assigning controlled vocabulary by humans is referred to as analytic indexing [15,16].

Web of Science provides Author Keywords and Keywords Plus. Author Keywords are taken from a list of keywords that the authors provided in their papers, Keywords Plus are taken from references cited by authors. Therefore, Keywords Plus often include important terms not listed in the study title, abstract, or list of author keywords [17].

Several studies on indexing practices have focused on areas other than complementary and alternative medicine. A recent study looked at the process of indexing and retrieving medical information for populations of different ethnicity in several major health-related databases. The researchers found that information might not be obtained if the health professionals were not familiar with the indexing policy. They noted that databases were not in agreement about the definition and use of most of their search terms [18].

Another study compared 28 articles that happened to be indexed twice. Comparing the two indexers, the investigators found significant differences in depth of indexing (e.g., the prevalence of major and minor descriptors, check-tags, and subheadings) and choice of subject headings [19]. This study illustrated that indexing practices are often inconsistent and little is known about indexing and retrieving information from databases on CAM.

Searching databases for a CAM topic can be confusing and ineffective if terms are not used correctly. If authors are reporting studies without using standard controlled vocabulary, indexers might not assign the appropriate terminology to represent the studies. Indexers are constrained further by the specifications of the controlled vocabulary [20]. In particular, a relatively small number of descriptors pertaining to CAM are contained in the controlled vocabulary [21], thus limiting indexers further in their choice of assigning specific subject terms.

The objective of this study was to compare indexing practices in four selected databases in relation to key terms used by authors and search terms used by investigators to locate studies on the validity of spinal palpation as a model for CAM searches. We posed four questions.

1. What are the key terms used by authors in the fields of chiropractic, physical therapy, allopathic and osteopathic medicine in defining the validity of spinal palpatory diagnostic tests in patients with spinal neuromuscular dysfunction?

2. What MeSH terms and descriptors are available to indexers in different databases on the subject of validity of spinal palpation?

3. How frequently do indexers assign appropriate MeSH terms or descriptors to represent the terms used by authors in titles and abstracts?

4. What are the key terms used by investigators to conduct a literature search on the validity of spinal palpation?

We investigated the frequency of use by authors, indexers, and investigators, of search terms for validity, spinal palpation and musculoskeletal dysfunction. We evaluated and compared analytic manual indexing to computer program generated semantic indexing.

Methods

Develop Key Concepts and Identify Search Terms

This study was conducted at the Susan Samueli Center for Complementary and Alternative Medicine (University of California, Irvine) between 2001 and 2002. A multidisciplinary team of investigators including researchers, clinicians, and a health sciences librarian undertook this study. Our research question was: "What is the validity of spinal palpatory procedures for diagnosis in patients with spinal neuromuscular dysfunction?" The first step for conducting a literature search was to break down the research question into four key concepts:

1. validity/validity assessment,

2. spine

3. palpation procedures

4. neuromusculoskeletal dysfunction.

The multidisciplinary team identified MeSH terms, descriptors, and potentially related keywords for these four key concepts [see Table 1].

Table 1

Controlled vocabulary and keywords used to retrieve studies on the validity of spinal palpation

#1: Validity Terms	#2: Spinal Terms	#3: Procedure Terms	#4: Neuromuscular Dysfunction Terms
Accuracy, accurate	Cervical verterbrae [mh][sh][de]/cervical	Diagnosis [mh][sh][de]	Apophyseal
Analysis of variance [mh] [sh] [de]	Lumbar vertebrae [mh] [sh] [de] /lumbar	Manual	Asymmetry
Construct validity [sh]	Neck [mh] [sh] [de]	Manual exam*/manual diagnosis	Back pain [mh] [sh] [de]
Content validity [sh]	Para-spine, para-spinal	Manipulat*	Blockage
Convergent	Spine [mh] [sh] [de] / Spinal	Manipulation, chiropractic [mh] [sh] [de]	Fixation/ Tissue Fixation [mh] [de]
Criterion / Criterion related validity [sh]	Thoracic vertebrae [mh] [sh] [de] / thoracic	Manipulation, orthopedic [mh] [sh] [de]	Hypomobility
Discriminant analysis [mh] [de]	Vertebral	Manipulation, osteopathic [mh] [de]	Joint Instability [mh] [sh] [de]
Discriminant validity [sh]		Manipulation, spinal [mh] [de]	Low back pain [mh] [sh] [de]
Face validity [sh]		Palpation [mh] [sh] [de] /palpat*	Manipulable lesion
Gold standard			Mobility
Instrumentation [sh] / instrument*			Motion unit
Judgment [mh] [sh]			Muscle tension / Muscle contraction [mh] [sh] [de]
Likelihood functions [mh] [de] / Likelihood ratio			Myofascial Pain Syndromes [mh] [sh] [de]
Predict* (predictive, predictor, predictability)			Neck pain [mh] [sh] [de]
Predictive value of tests [mh] [sh] [de]			Quality of Motion
Reference standards [mh] [de]			Range of Motion, Articular [mh] [de]/Range of motion [sh]
ROC / ROC curve [mh] [de]			Stiffness
Sensitivity and specificity [mh] [sh] [de]			Somatic dysfunction [de]
Valid, Validation, Validation studies [pt] [sh]			Subluxation [sh] [de]
Validity [sh]			Tender point*
			Tissue texture
#5: Validity Assessment Terms			Trigger points [de]
Galvanic skin response [mh] [de]			Zygapophyseal joint [mh] / Zygapophysial
Measure*, Pain Measurement [mh] [sh] [de]
Radiography [mh] [sh] [de]
Thermography [mh] [sh] [de]
X-rays [mh] [sh], x-ray [de]

[mh] = Medical Subject Headings in MEDLINE; [pt] = publication type in MEDLINE; [sh] = Subject Heading in CINAHL [de] = Descriptor in MANTIS Search strategy: (#1 AND #2 AND #3 AND #4) OR (#2 AND #3 AND #4 AND #5) Note: Reliability and Validity see Reproducibility of Results; Validity of Results see Reproducibility of Results.

We then verified the MeSH terms [mh] using the National Library of Medicine Medical Subject Headings-Annotated Alphabetic List, 2001, which was the current thesaurus at the inception of the study. In addition to MeSH, we used subject headings [sh] in the CINAHL online thesaurus and the descriptors [de] in the online MANTIS thesaurus to prepare searches in other online databases.

Searching and Selecting Studies for Comparison

Using the identified search terms listed in Table 1, we conducted multiple search strategies. The search in general can be stated as follow:

Search queries #1

(accuracy OR accurate OR analysis of variance OR construct validity … see Table 1: column #1: Validity terms) AND (cervical vertebrae OR cervical OR lumbar vertebrae OR lumbar … see Table 1: column #2: Spinal terms) AND (diagnosis OR manual OR manipulat* … see Table 1: column #3: Procedure terms) AND (apophyseal OR asymmetry OR back pain … see Table 1: column #4: Neuromuscular Dysfunction terms.)

Search queries #2

(galvanic skin response OR measure* OR pain measurement … see Table 1: column 1, #5: Validity assessment terms) AND (cervical vertebrae OR cervical OR lumbar vertebrae OR lumbar … see Table 1: column #2: Spinal terms) AND (diagnosis OR manual OR manipulat* … see Table 1: column #3: Procedure terms) AND (apophyseal OR asymmetry OR back pain … see Table 1: column #4: Neuromuscular Dysfunction terms.)

Search strategy

Search queries #1 OR #2

We began the search in PubMed and continued in the other three databases. We proceeded with hand searching and contacting experts and arrived at 4,945 citations including overlapping and non-relevant studies. Inclusion and exclusion criteria were used to identify relevant studies. Inclusion criteria consisted of a document pertaining to manual spinal palpation procedures, measurement of validity or accuracy, primary research study published in a peer reviewed journal in any language, and made available between January 1, 1966 and September 30, 2002. Exclusion criteria consisted of a document pertaining to non-manual procedures, lack of sufficient tests or data, and anecdotal, speculative or editorial reports.

A thorough review of titles and abstracts resulted in the identification of 38 unique studies that were indexed in at least one of the four selected databases (see Appendix I [Additional file: 1] for citations). Five of these 38 studies were indexed in all four databases [22-26].

In order to compare the use of CAM terminology among authors and indexers, we focused the study on the standard practice of using MeSH among various databases. Based on the subject area of the research question, and the availability of two institutions library database subscriptions, we were able to evaluate databases like PubMed MEDLINE, MANTIS, and CINAHL in comparison to a semantic database, Web of Science.

Databases Selection

PubMed MEDLINE is the most widely used and comprehensive scientific literature database in biomedicine and it has a subset focusing on complementary medicine. MANTIS, a specialized database, was selected because it covers subject areas of interest to the research question including osteopathic medicine, chiropractic, and manual medicine. CINAHL was included because the database covers literature related to nursing and allied health, including physical therapy, radiologic technology, occupational therapy and social service/health care. The thesaurus of MANTIS and CINAHL are based on MeSH with supplements for certain subject areas. For comparison, we added one database, Web of Science, which relies on computer generated indexing terms (i.e. Keywords Plus) in addition to key terms provided by authors (i.e. Author Keywords). Similar to the other selected databases, Web of Science covers a wide area of life sciences.

Compilation of Key Terms

We verified how many of the four databases actually indexed each of the 38 studies. We downloaded and imported the citations with title, abstracts, and controlled vocabulary into EndNote. This resulted in 38 titles with 37 abstracts. PubMed indexed 29 studies, MANTIS 28 studies, CINAHL 13 studies, and Web of Science 26 studies. Across databases, we eliminated overlapping abstracts but retained the controlled vocabulary. We then used the EndNote search field to identify the key terms/controlled vocabulary in titles, abstracts, and indexes. We calculated and tabulated the frequency of search terms that were used by authors and indexers. The tabulation for validity and validity assessment terms is shown in Table 2, for spinal terms in Table 3, for procedural terms in Table 4, and for neuromusculoskeletal dysfunction terms in Table 5. The controlled vocabulary of the five studies that were indexed in all four databases is provided in Table 6.

Table 2

Frequency of validity and validity assessment terms in titles, abstracts and four databases

Validity Terms Used in the Search Strategies	Titles n = 38	Abstracts n = 371	MEDLINE (mh) n = 292	MANTIS (de) n = 282	CINAHL (sh) n = 132	Web of Science n = 262,3,4
Accuracy, accurate	3	10				1
Analysis of variance [mh] [sh] [de]			2		3
Construct validity [sh]	2	1
Content validity [sh]
Convergent
Criterion / Criterion related validity [sh]		3			1
Discriminant Analysis [mh] [de]	1	3
Discriminant validity [sh]
Face validity [sh]
Gold standard		1
Instrumentation [sh]/instrument*	1	3	1	2	1
Judgment [mh] [sh]	1	1
Likelihood functions [mh] [de] / Likelihood ratio
Predict* (predictive, predictor, predictability)	3	7				2
Predictive value of tests [mh] [sh] [de]			6	1	1
Reference standards [mh] [de]
ROC / ROC curve [mh] [de]
Sensitivity and specificity [mh] [sh] [de]	2	7	3		2	2
Valid, validation, validation studies [pt] [sh], validity [sh]	7	6			2	2
Validity Assessment Terms
Galvanic skin response [mh] [de]	1	1	1	1
Measure*, Pain Measurement [mh] [sh] [de]	2	7	4	2	4	2
Radiography [mh] [sh] [de]	1	6	2	5	3	1
Thermography [mh] [sh] [de]	1	1	1			1
X-rays [mh]		3
Terms Used by Authors and Indexers but not in Search Strategies.
Machine*	1	2				2
Reference values [mh] [sh] [de]		1	2
Reference based	1	1
Reproducibility of results [mh] [sh] [de]			7	13

[mh] = Medical Subject Headings in MEDLINE; [pt] = publication type in MEDLINE; [sh] = Subject Heading in CINAHL [de] = Descriptor in MANTIS Note: 1One study did not have an abstract online. 2Not all studies had abstracts. 3Web of Science does not have a controlled vocabulary. 4Eleven studies had no keywords generated. * is a truncation symbol for a wild card search in PubMed and Web of Science.

Table 3

Frequency of spinal terms in titles, abstracts, and four databases

Spinal Terms Used in Search Strategies	Titles n = 38	Abstracts n = 371	MEDLINE (mh) n = 292	MANTIS (de) n = 282	CINAHL (sh) n = 132	Web of Science n = 262,3,4
Cervical vertebrae [mh] [sh] [de] / cervical	10	15	8	7	4	2
Lumbar vertebrae [mh] [sh] [de] /lumbar	3	6	1	7	2	4
Neck [mh] [sh] [de]		5	5	2	2	0
Para-spine, para-spinal		1
Spine [mh] [sh] [de] / Spinal	14	21	15	8	4	8
Thoracic vertebrae [mh] [sh] [de] / thoracic	1	2	1	3
Vertebral	2	4		4

[mh] = Medical Subject Headings in MEDLINE; [pt] = publication type in MEDLINE; [sh] = Subject Heading in CINAHL [de] = Descriptor in MANTIS Note: 1One study did not have an abstract online. 2Not all studies had abstracts. 3Web of Science does not have a controlled vocabulary. 4Eleven studies had no keywords generated. * is a truncation symbol for a wild card search in PubMed and Web of Science.

Table 4

Frequency of palpation procedural terms in titles, abstracts and four databases

Palpation Procedural Terms Used in the Search Strategies	Titles n = 38	Abstracts n = 371	MEDLINE (mh) n = 292	MANTIS (de) n = 282	CINAHL (sh) n = 132	Web of Science n = 262,3,4
Diagnosis [mh] [sh] [de]	2	3	17	23	7	2
Manual	3	9				2
Manipulat*	2	7
Manipulation, chiropractic [mh] [sh] [de]				4	1
Manipulation, orthopedic [mh] [sh] [de]			4	6	3
Manipulation, osteopathic [mh] [de]
Manipulation, spinal [mh] [de]	2	7		5		1
Palpation [mh] [sh] [de] /palpat*	7	19	11	16	6	4
Authors' and Indexers' Terms
Physical examination [mh] [sh] [de]	1	3	5	15	5	3

[mh] = Medical Subject Headings in MEDLINE; [pt] = publication type in MEDLINE; [sh] = Subject Heading in CINAHL [de] = Descriptor in MANTIS Note: 1One study did not have an abstract online. 2Not all studies had abstracts. 3Web of Science does not have a controlled vocabulary. 4Eleven studies had no keywords generated. * is a truncation symbol for a wild card search in PubMed and Web of Science.

Table 5

Frequency of neuromuscular dysfunction terms in titles, abstracts and four databases

Neuromusculoskeletal Dysfunction Terms Used in Search Strategies	Title n = 38	Abstracts n = 371	MEDLINE (mh) n = 292	MANTIS (de) n = 282	CINAHL (sh) n = 132	Web of Science n = 262,3,4
Apophyseal
Asymmetry	1	2
Blockage
Fixation / Tissue Fixation [mh] [de]	1	3		4
Hypomobility
Joint Instability [mh] [sh] [de]					1
Low back pain [mh] [sh] [de] / back pain [mh] [sh] [de]	5	8	6	12	4	4
Manipulable lesion
Mobility	1	6			1	2
Motion unit
Muscle tension / Muscle contraction [mh] [sh] [de]		1	2
Myofascial Pain Syndromes [mh] [sh] [de]			2	1		1
Neck pain [mh] [sh] [de]	2	2	2	4	1
Quality of Motion
Range of Motion, Articular [mh] [de]/ Range of motion [sh]	1	5	4	7	2
Stiffness	1	2
Somatic dysfunction [de]	1	3		3		1
Subluxation [sh] [de]		1		9	1
Tender point*	1	1
Tissue texture		2
Trigger points [de]	1	1				1
Zygapophyseal joint [mh] / Zygapophysial	1	1
Authors' Terms5
Dysfunction (cervical, joint, neck, spinal, etc.)	6	6				2
Motion [mh] [sh] [de]		7		3		3
Motion palpation/ Palpation, motion [de]	3	3		9		1
Motion restriction		4
Spinal (cervical) pain	2	4

[mh] = Medical Subject Headings in MEDLINE; [pt] = publication type in MEDLINE; [sh] = Subject Heading in CINAHL [de] = Descriptor in MANTIS Note: 1One study did not have an abstract online. 2Not all studies had abstracts. 3Web of Science does not have a controlled vocabulary. 4Eleven studies had no keywords generated. 5These terms were not used in the search strategies but were used by authors in titles and abstracts. * is a truncation symbol for a wild card search in PubMed and Web of Science.

Table 6

Comparison of five studies indexed in all four databases (subheadings, gender, age group, human/animal, and publication type were removed)

Citations (Authors, Title, Source)	MEDLINE – MeSH	MANTIS – Descriptors	CINAHL Subject Headings	Web of Science – Author Keywords/Keywords Plus
Gracovetsky SA, Newman NM, Richards MP, et al. Evaluation of clinician and machine performance in the assessment of low back pain. Spine 1998; 23:568–75.	Biomechanics Electrodes Evaluation Studies Low Back Pain Medical History Taking Pain Measurement Prospective Studies Single-Blind Method Truth Disclosure Weight Lifting	Clinical competence. Diagnosis. Disability evaluation. Instrumentation. Lifting. Low back pain. Lumbar vertebrae. Physical examination. Reproducibility of results	Data Analysis Software Descriptive Statistics *Diagnosis, Musculoskeletal *Honesty *Lifting *Low Back Pain P-Value *Physical Examination Prospective Studies	Author Keywords– Abnormal, Clinician, Concordance, Diagnosis, Evaluation, Low back pain, Machine, Normal, Receiver Operating characteristic, Spine Keywords Plus– Physical-examination, Lumbar spine, Diagnostic-accuracy, Flexion extension, Iso-machines, strength, Signs
Haas M, Panzer D, Peterson D, et al. Short-Term Responsiveness of Manual Thoracic End-Play Assessment to Spinal Manipulation: A Randomized Controlled Trial of Construct Validity. Journal Of Manipulative And Physiological Therapeutics 1995; 18:582–9.	Analysis of variance Chiropractic Prospective studies Spinal diseases Thoracic vertebrae Treatment outcome	Chiropractic Manipulation, chiropractic. Manipulation, orthopedic. Manipulation, spinal. Palpation, motion. Reproducibility of results. Thoracic vertebrae	Chiropractic manipulation Evaluation research Students, allied health Two-Way Analysis of variance	Author Keywords– Outcomes research, Chiropractic, Clinical trials, randomized
Kristiansson P, Svardsudd K. Discriminatory power of tests applied in back pain during pregnancy. Spine. 1996 Oct 15;21(20):2337–43; discussion 2343–4.	Back pain, Cohort studies Cross-sectional studies Diagnostic tests, routine Pain measurement Pain threshold *Predictive value of tests Pregnancy Prevalence Spine	Back pain. Diagnosis (di). back Epidemiology (ep). Low back pain. Pain measurement. Physical examination. Pregnancy Reproducibility of results	Back Funding source Observational Methods Pain Measurement *Physical Examination Pregnancy Prospective Studies Questionnaires Spearman's Rank Correlation Coefficient Visual Analog Scaling	Author Keywords– Back pain, Epidemiology, Physical examination Pregnancy Keywords Plus– Sacroiliac joint Reliability
Leboeuf-Yde C, Kyvik KO. Is it possible to differentiate people with or without low-back pain on the basis of tests of lumbopelvic dysfunction? J. Manipulative Physiol Ther 2000; 23:160–7.	Chiropractic Denmark Diseases in Twins Low Back Pain Physical Examination Predictive Value of Tests Prevalence Questionnaires Sensitivity and Specificity	Chiropractic. Diagnosis. Extension. Flexion. Lordosis. Low back pain. Methods. Orthopedic. Physical examination. Predictive value of tests. Range of motion, articular. Reproducibility of results	Chi Square Test *Chiropractic Assessment Data Analysis Software *Instrument Validation *Low Back Pain Predictive Value of Tests Questionnaires Self Report Sensitivity and Specificity Twins	Author Keywords– Chiropractic, Sensitivity, Specificity, Predictive value, Prevalence, Low pack pain. Keywords Plus– Lumbar segmental abnormality, 8 evaluative dimensions, Physical-examination, Interexaminer reliability, Spinal manipulation, Sacroiliac joint, History, Rates
Sandmark H, Nisell R. Validity of five common manual neck pain provoking tests. Scand J Rehabil Med 1995; 27:131–6.	*Epidemiologic Methods *Neck Pain Predictive Value of Tests Questionnaires Reproducibility of Results Single-Blind Method	Diagnosis. Neck pain. Pain measurement. Reproducibility of results	*Cervical Vertebrae Data Analysis, Statistical Descriptive Statistics *Diagnosis, Musculoskeletal False Negative Reactions False Positive Reactions *Neck *Pain Measurement Palpation Physical Therapy Questionnaires Random Sample Single-Blind Studies	Author keywords– Epidemiology, Electricians, Manual medicine, Neck dysfunction, Physical examination, Sensitivity, Specificity

For the purposes of this study, we printed in "bold" the controlled vocabulary and capitalized the first character of each term. The free key terms were set in quotation marks. A "wild card" search is represented by a truncation symbol, e.g., palpat* will include palpation, palpated, palpatory, etc.

Results

Results are presented on the frequency of MeSH, descriptors, subject headings, and key terms for each of the four key concepts of the search strategy.

Validity and Validity Assessment Terms

The frequency of use of validity and validity assessment terms is presented in Table 2. The most frequent validity term used by authors was "accuracy", "accurate" (3/38 titles, 10/37 abstracts). Web of Science generated "Diagnostic-accuracy" as Keywords Plus (1/26). "Accuracy" was not a controlled vocabulary in the selected analytic databases. Even though Validation Studies is a publication type in MEDLINE, we did not find any study from our initial search result classified in this category. CINAHL provided a cross-reference to see Validity as a search term for subject heading. We found 3/13 studies indexed either under Criterion-Related Validity or Validation Studies in CINAHL. Authors used various terms containing "valid" in 7/38 titles and 6/37 abstracts, while MEDLINE and MANTIS did not have the actual terms like "valid", "validation", and "validity" as controlled vocabulary. Web of Science indexed 2/26 studies for "validity", i.e. Keywords Plus "Validation, Disorders" and Author Keyword "Predictive validity".

Reproducibility of Results was the most frequently assigned validity term in MANTIS (13/28 studies) and MEDLINE (7/29 studies) even though Reproducibility of Results is a reliability term. Among the 13 studies in CINAHL, we did not find a study indexed as a subject heading under Reproducibility of Results. Instead, CINAHL indexed 10/13 studies using various validity terms from the extensive list of validity indexing terms available in the CINAHL thesaurus.

Predictive Value of Tests was the next most frequently assigned validity term by NLM indexers (6/29 MEDLINE studies). While MANTIS and CINAHL also contain Predictive Value of Tests in their thesaurus, the term was indexed only once in these two databases. Web of Science listed "Predictive value" once as Author Keywords.

Authors used Sensitivity and Specificity in 2/38 titles and 7/37 abstracts, compared to 3/29 studies indexed in MEDLINE, none in MANTIS, 2/13 in CINHAL, and 2/26 in Web of Science, as Author Keywords listed individually.

Several of our validity search terms, such as Content Validity, Convergent Validity, Discriminant Validity, and Face Validity were CINAHL but not MEDLINE or MANTIS Subject Headings. We did not find authors or indexers using these terms. Similarly, Likelihood Functions, Reference Standards, and ROC Curve were Medical Subject Headings for MEDLINE and MANTIS, but none were used by indexers in the analytic databases.

Reference Values, a Medical Subject Heading for all three selected databases, was the only controlled vocabulary used by authors and indexers that was not included as a search term. Instead, we used "Gold Standard" as keyword and Reference Standards as the Medical Subject Headings in our search [See Table 2]. "Gold Standard" was used once in an abstract; neither authors nor indexers ever used Reference Standards.

The most frequent validity assessment terms was used by authors were "measurement" or Pain Measurement as well as Radiography. Indexers also assigned these terms in all four databases [See Table 2].

Spinal Terms

The frequency of spinal terms is presented in Table 3. The spinal term most frequently used by authors and indexers was Spine or "Spinal". Fourteen of 38 titles and 21/37 abstracts contained Spine or "Spinal". MEDLINE indexed Spine (or spinal as an adjective) in 15/29 studies, MANTIS in 8/28, and CINAHL in 4/13 studies. Web of Science generated 8/26 studies as Author Keywords and Keywords Plus. Of these eight studies, two were found as Keywords Plus, i.e. "Spinal Manipulation" and "Lumbar Spine".

Authors and indexers preferred Cervical Vertebrae to Neck. Cervical Vertebrae (or cervical as an adjective) was used in 10/38 titles and in 15/37 abstracts, and indexed in 8/29 studies in MEDLINE, 7/28 in CINAHL, and 2/26 in Web of Science as Author Keywords. In comparison, Neck was never used by authors in titles and in only 5/37 abstracts. It was indexed in 5/29 studies in MEDLINE, 2/28 in MANTIS, 2/13 in CINAHL, and none were generated in Web of Science.Thoracic Vertebrae (or thoracic as an adjective) was used infrequently by authors and indexers.

Palpation Procedural Terms

The frequency of palpation procedural terms is presented in Table 4. The most frequent palpation procedural terms used by authors as well as indexers were Palpation or "palpat*". These term were used frequently (7/38 titles, 19/37 abstracts, 11/29 studies in MEDLINE, 16/28 in MANTIS, 6/13 in CINAHL, and 4/26 in Web of Science as Author Keywords).

Authors frequently used palpation procedural terms that were not in the controlled vocabulary, e.g., "manual exam*", "manual diagnosis", or "manual tests" (3/38 titles and 9/37 abstracts), and "manipulat*" (2/38 titles, 7/37 abstracts). From the controlled vocabulary for manipulation, authors often chose Spinal Manipulation rather than Chiropractic, Orthopedic, or Osteopathic Manipulation. While MEDLINE indexed Manipulation, Orthopedic as the only manipulation term, MANTIS indexed Chiropractic, Orthopedic, and Spinal Manipulation. Neither authors nor indexers used Manipulation, Osteopathic. This was also the only palpation procedural term used in the search strategies that was not used by authors or indexers.

Indexers in MEDLINE, MANTIS and CINAHL selected Diagnosis most frequently as a palpation procedural term (17/29 studies in MEDLINE, 23/28 in MANTIS, and 7/13 in CINAHL). Diagnosis was usually indexed in combination with a neuromuscular dysfunction term as a subheading.

While Physical Examination was not included in our search strategies, authors occasionally used it in the abstracts. MANTIS indexed Physical Examination in 15/28 studies compared to MEDLINE (5/29), CINAHL (5/13) and Web of Science (3/26 as Author Keywords and Keywords Plus).

Neuromusculoskeletal Dysfunction Terms

The frequency of neuromuscular/musculoskeletal dysfunction terms is presented in Table 5. Of the 22 dysfunction terms used in the search strategies, only Low Back Pain and Back Pain were used relatively frequently by authors in titles (5/38) and in abstracts (8/37). Also the four selected databases indexed these terms with highest frequency (6/29 studies in MEDLINE, 12/28 in MANTIS, 4/13 in CINAHL, and 4/26 as Author Keywords and Keyword Plus in Web of Science.)

While authors used "mobility" and "dysfunction" relatively frequently, these terms were not part of the controlled vocabulary in MEDLINE, MANTIS, and CINAHL. Motion, a controlled vocabulary, was used by authors in 7/37 abstracts but was not indexed in MEDLINE and CINAHL. Three of 28 studies in MANTIS indexed Motion as a descriptor. Web of Science indexed Motion twice as Keywords Plus and once as Author Keywords. Subluxation, a widely used term in chiropractic medicine and a controlled vocabulary in MANTIS and CINAHL, was not used by authors in titles and abstracts but frequently indexed in MANTIS (9/28 studies).

Of the 22 neuromusculoskeletal dysfunction terms used in the search strategies, 6 were never used by authors or indexers (apophyseal, blockage, hypomobility, manipulable lesion, motion unit, and quality of motion.) While "motion palpation" was not used as a combined search term, MANTIS indexed Palpation, Motion in 9/28 studies. Authors used "motion palpation" in 3/38 titles and 3/37 abstracts. "Motion palpation" was not a MeSH or subject heading, and therefore it was not indexed in MEDLINE or CINAHL.

Comparison of Five Studies Indexed in all Four Databases

Of the 38 studies, five studies were indexed in all four databases as shown in Table 6. The comparison of validity terms showed that two of these five studies included a validity term in the title [23,26], and two studies had a validity term in the abstract [22,25]. The study by Kristiansson et al. [24] did not provide a validity term in either title or abstract, but PubMed and MANTIS indexed validity terms for this study. PubMed and CINAHL did not pick up the validity term that appeared in the abstract by Gracovetsky et al. [22] MANTIS recognized and indexed the validity terms for all five studies. CINAHL indexed two of the five studies with validity terms [23,25]. In Web of Science, three of the five studies included Author Keywords for validity [22,25,26]. Three studies, published after 1996, generated Keywords Plus from the references cited by the authors, but only one study included a validity term [22].

All five studies included a neuromuscular dysfunction term in the title. Authors included Back Pain in the title and abstract in three studies [22,24,25] and Neck Pain in the title and abstract in one study [26]. One author used "thoracic end-play" in the title and no other neuromuscular dysfunction term in the abstract [23]. All four databases indexed Back Pain, Low Back Pain, and Neck Pain in the four respective studies [22,24-26]. In Web of Science, these terms were provided by the authors as Author Keywords and not generated from Keywords Plus.

Discussion

Of our four search categories, validity was the most difficult search concept, largely because of inconsistent terminology used among authors, indexers, and investigators. Most authors did not seem familiar with the controlled vocabulary in the field of validity. In-depth comparison of five studies indexed in all four databases supported this result. For instance, Kristiansson et al. [24] did not use a validity term in either title or abstract and did not provide Author Keywords pertaining to validity. Yet indexers from MEDLINE and MANTIS recognized this study to address validity and assigned a validity term, e.g., Predictive Value of Tests in MEDLINE and Reproducibility of Results in MANTIS. On the other hand, one of these studies used validity terms three times in the abstract, but indexers from MEDLINE and CINAHL did not recognize the validity component of the study [22]. Thus, authors seem to under-represent the validity aspects of their studies, while database indexing shows inconsistent awareness of validity component.

Authors frequently used "uncontrolled" terminology in titles and abstracts instead of the corresponding terminology that is part of controlled vocabulary in the databases of interest to their studies. For example, authors chose the term "accuracy" to represent validity but "accuracy" is not an indexing term in MEDLINE and MANTIS. One suggestion for MEDLINE and MANTIS database vendors would be to augment their controlled vocabulary to specifically include terms for validity, e.g., validity and accuracy, since these are terms preferred by authors and investigators.

Currently, the controlled vocabulary for validity in MEDLINE and MANTIS is limited and indexers use Reproducibility of Results as both a reliability and a validity indexing term. However, if the term is properly understood by authors and researchers to represent reliability rather than validity, they may not consider using Reproducibility of Results to search for validity studies. Unfortunately, automatic term mapping is not available in MEDLINE for the search term validity. Database vendors might consider applying Reproducibility of Results to reliability only and augmenting their thesaurus with commonly used validity terms.

Presently, Predictive Value of Tests, Reference Standards, Sensitivity and Specificity are some of the controlled vocabulary available in MEDLINE and MANTIS for indexing validity studies. In comparison to these MeSH descriptors, CINAHL subject headings cover a wide range of validity terms, including Validity, Concurrent Validity, Consensual Validity,Construct Validity, Content Validity,Criteria-related Validity,Qualitative Validity,Discriminant Validity, External Validity,Face Validity, Internal Validity,Predictive Validity, and Validation Studies. Further investigation might explain why only the subject headings Validation Studies and Criteria-related Validity were used to index the 13 studies in CINAHL on validity that were examined in this project.

This study shows that CINAHL indexers did not use all the validity terms available in their thesaurus. For example, Haas et al. used Construct Validity in the title, but the CINAHL indexer did not assign Construct Validity as a subject heading for that study [23]. In the study by Sandmark et al., validity appeared in the title, yet CINAHL did not assign a subject heading for validity [26]. In spite of the relatively large array of validity terms in the CINAHL thesaurus, validity indexing is under-utilized in this database.

In the entire database of MANTIS, the controlled vocabulary Sensitivity and Specificity was indexed only 24 times. None of these citations were in our subject area even though the term was used in six titles or abstracts. Thus, similar to CINAHL, indexers in MANTIS did not use their available controlled vocabulary as required for optimal search result. Searching for validity studies becomes complicated further as authors and investigators do not seem to have a common understanding of the types of validity. Consequently, indexers are at a loss which validity terms to use.

Fortunately, using validity assessment terms was productive because authors and indexers used these terms more often than just validity terms. Pain Measurement and Radiography showed relatively consistent use by authors and indexers.

The search category for procedural terms was relatively uncomplicated. Palpation and Manipulation were useful palpation procedural search terms. In addition, the controlled vocabulary term Physical Examination should be included in search strategies on spinal diagnostic procedures in order to maximize search results. Procedural terms in manual medicine seemed quite well represented in the databases that we studied.

Like validity, the search concept of neuromuscular dysfunction presented with challenges. For example, half of the 22 neuromusculoskeletal dysfunction terms provided by the investigators for the search strategies were not contained in the controlled vocabulary. Of the 11 controlled terms, only Back Pain and Low Back Pain were used with consistent frequency by authors and indexers, suggesting a need for standard terminology and definitions in the field of neuromusculoskeletal dysfunction with which to search the literature. Four of the five studies that were indexed in all four databases addressed low back pain or neck pain, and one study examined thoracic end-play. This admittedly small sample of studies would suggest that back and neck pain represent major areas of investigation in neuromuscular dysfunction. End-play was not recognized as a neuromuscular dysfunction by the investigator of this project. Indexing in all four databases did not include any neuromuscular dysfunction term for this study (see Table 6) which would suggest that end-play is not a recognized dysfunction term in manual medicine. Based on this few studies, some neuromuscular dysfunction terms, like low back pain and neck pain, seem clearly defined and utilized equally by authors and indexers. Other dysfunction terms, though, might be understood by experts but not by indexers, leading to an apparent under-representation of some neuromuscular dysfunction terms.

Current Status of Standard Glossaries in Manual Medicine

In 1975 the National Institutes of Health sponsored a multidisciplinary research conference on status of spinal manipulative therapy [27]. The orthopedic, osteopathic, and chiropractic representatives reported on the understanding and use of the terminology in each respective profession. There was no consensus within or amongst professions at that time.

Searching the Internet for orthopedic terminology for professionals, we located many glossaries designed to facilitate patient comprehension of orthopedic terms. For example, the American Academy of Orthopaedic Surgeons provides a glossary of orthopedic diagnostic tests online which seems to be designed for patients [28]. At this web site, we did not identify a glossary of standard terminology for orthopedics.

The osteopathic profession developed a standardized Glossary of Osteopathic Terminology in 1981 that has been updated annually ever since [29]. The online version is available at the AOAnet Yearbook and Directory [30]. The osteopathic literature database OSTMED went online October 1, 2002[31]. For standard indexing practice, the database uses the NLM MeSH terms and about 70 terms that are uniquely osteopathic, called Osteopathic Subject Headings (OSH). To further assist in accessing the osteopathic literature database, Manipulation, Osteopathic became a MeSH term in 2002 [6].

The Chiropractic Library Consortium (CLIBCON) published the first list of chiropractic thesaurus (CHIROSH) in 1979 to improve access to the chiropractic literature. The consortium produced the first version of the Index to Chiropractic Literature in print in 1980 and added the online version in 1985. The standard indexing practice is based on NLM MeSH and CHIROSH [32]. Chiropractic librarians but not the chiropractic practitioners use this CHIROSH thesaurus. Presently there does not seem to be a universally accepted glossary of terms in the chiropractic profession [33].

In recent years, PubMED MEDLINE articles that were submitted by publishers do not contain MeSH vocabulary. This increasingly common situation contributes to the data retrieval problems that we have presented here and furthers the argument for authors to use controlled vocabulary or commonly understood terminology in the titles and abstracts of their publications in order to facilitate retrieval of their studies by key terms.

In the field of CAM, semantic indexing might be more useful than analytic indexing because keyword indexing is generated by computer programs that pool words or phrases used by authors at least twice. For instance, if the author mentioned subluxation at least twice in the title or abstract, the term will be indexed regardless of professional agreement about the definition of the term.

Strengths and Limitations of the Controlled Vocabulary and Keywords

In our study, the limitations of semantic indexing in Web of Science produced the lowest yield of relevant studies. While Web of Science indexed 26/38 studies, only a few of these had been retrieved by the search strategy. Instead, we arrived at the 26 studies by individually verifying the 38 studies that we knew to exist by using author name and title words.

Of the 26 studies indexed in Web of Science, nine had been generated with both Author Keywords and Keywords Plus, and six studies with Author Keywords only. Web of Science did not generate any keywords for 11/26 studies. If investigators relied on searching by Keyword Plus, the search strategy might miss many potentially relevant studies. At the same time, if investigators are not familiar with the terms used by authors, a search strategy might miss relevant studies as well.

In Web of Science, Author Keywords have only been available for search since 1991. This limits users' ability to find relevant literature prior to 1991. A search of earlier years has to rely on title word searches. It is likely because of this limitation, we found relatively few Author Keywords or Keywords Plus for our four search concepts.

For databases that utilize analytic indexing, like MEDLINE, the NLM Medical Subject Headings Section staff continually revises and updates the MeSH vocabulary. While there were major changes in the 2002 NLM MeSH thesaurus for CAM, the current 2003 edition added only one CAM therapeutic modality. It would be helpful if investigators, information specialists and CAM practitioners become more active in contributing to the selection process of CAM descriptors in NLM MeSH and other database descriptors. Individual subscribers and subscribing institutions as well, have significant influence on the modifications and updating processes of databases.

The NLM Web site has a page for suggesting MeSH changes with a one or two-sentence statement, the reason for the change, and one or two authoritative citations [34]. Since the NLM Medical Subject Headings are the standard controlled vocabulary for the allied health and biomedicine, investigators and authors in the fields of osteopathic medicine, chiropractic and physical medicine should take the opportunity to establish consistent terminology in the field of validity of spinal palpation for neuromuscular dysfunction.

We recognize the difficulties of adding new descriptors and updating the existing controlled vocabulary because searching for a new concept requires a dynamic thesaurus while searching retrospectively requires stability and continuity [35]. Fortunately, the National Library of Medicine implements changes to the MeSH thesaurus in MEDLINE retrospectively.

Conclusions

Searching databases on CAM related topics is challenging due to the diversity in the use of controlled vocabulary and indexing procedures in different databases. Collaboration is needed among indexers, authors, investigators and information specialists to develop standard terminology in CAM. Furthermore, database vendors should augment their control vocabulary in CAM, particularly in the fields of manual medicine. A standard terminology then could be used and referred to by investigators in conducting their search strategies, and authors when writing titles, abstracts and submitting keywords for publications. As the CAM literature grows, dissemination and knowledge of the controlled vocabulary will become even more important.

Competing Interests

The authors of this study did not have competing interests with this study. None of the authors received reimbursement fees, funding, or salary from an organization that may in any way gain or lose financially from the publication of this paper in the past five years.

Authors' Contributions

LM conceived of the study, designed and carried out the searches, analyzed the search outcomes and the controlled vocabulary, researched the context of this study, drafted and finalized the manuscript.

SR participated in the design of the study, filtered the search results, participated in researching the context of this study, and assisted in drafting and finalizing the manuscript.

WN coordinated the study, provided search terms, reviewed and filtered search outcomes, and assisted in drafting the manuscript.

VD provided search terms, reviewed and filtered search outcomes, provided essential comments and approved the final manuscript.

MS was the content expert in osteopathic medicine. He provided search terms, reviewed and filtered search outcomes, provided essential comments and approved the manuscript.

AA was the content expert in chiropractic. He provided search terms, reviewed and filtered search outcomes, and approved the final manuscript.

SM provided search terms, reviewed and filtered search outcomes, reviewed and approved the manuscript.

Pre-publication history

The pre-publication history for this paper can be accessed here:

Click here for file