OBJECTIVE: This paper explores alternative approaches for the evaluation of an automatic indexing tool for MEDLINE, complementing the traditional precision and recall method. MATERIALS AND METHODS: The performance of MTI, the Medical Text Indexer used at NLM to produce MeSH recommendations for biomedical journal articles is evaluated on a random set of MEDLINE citations. The evaluation examines semantic similarity at the term level (indexing terms). In addition, the documents retrieved by queries resulting from MTI index terms for a given document are compared to the PubMed related citations for this document. RESULTS: Semantic similarity scores between sets of index terms are higher than the corresponding Dice similarity scores. Overall, 75% of the original documents and 58% of the top ten related citations are retrieved by queries based on the automatic indexing. CONCLUSIONS: The alternative measures studied in this paper confirm previous findings and may be used to select particular documents from the test set for a more thorough analysis.
OBJECTIVE: This paper explores alternative approaches for the evaluation of an automatic indexing tool for MEDLINE, complementing the traditional precision and recall method. MATERIALS AND METHODS: The performance of MTI, the Medical Text Indexer used at NLM to produce MeSH recommendations for biomedical journal articles is evaluated on a random set of MEDLINE citations. The evaluation examines semantic similarity at the term level (indexing terms). In addition, the documents retrieved by queries resulting from MTI index terms for a given document are compared to the PubMed related citations for this document. RESULTS: Semantic similarity scores between sets of index terms are higher than the corresponding Dice similarity scores. Overall, 75% of the original documents and 58% of the top ten related citations are retrieved by queries based on the automatic indexing. CONCLUSIONS: The alternative measures studied in this paper confirm previous findings and may be used to select particular documents from the test set for a more thorough analysis.
Authors: Serguei V S Pakhomov; Ted Pedersen; Bridget McInnes; Genevieve B Melton; Alexander Ruggieri; Christopher G Chute Journal: J Biomed Inform Date: 2010-10-31 Impact factor: 6.317
Authors: Sebastian Köhler; Marcel H Schulz; Peter Krawitz; Sebastian Bauer; Sandra Dölken; Claus E Ott; Christine Mundlos; Denise Horn; Stefan Mundlos; Peter N Robinson Journal: Am J Hum Genet Date: 2009-10 Impact factor: 11.025
Authors: Yuqing Mao; Kimberly Van Auken; Donghui Li; Cecilia N Arighi; Peter McQuilton; G Thomas Hayman; Susan Tweedie; Mary L Schaeffer; Stanley J F Laulederkind; Shur-Jen Wang; Julien Gobeill; Patrick Ruch; Anh Tuan Luu; Jung-Jae Kim; Jung-Hsien Chiang; Yu-De Chen; Chia-Jung Yang; Hongfang Liu; Dongqing Zhu; Yanpeng Li; Hong Yu; Ehsan Emadzadeh; Graciela Gonzalez; Jian-Ming Chen; Hong-Jie Dai; Zhiyong Lu Journal: Database (Oxford) Date: 2014-08-25 Impact factor: 3.451