Evaluating the impact of MEDLINE using the Critical Incident Technique.

An adaptation of the Critical Incident Technique for the evaluation of an online information system is described. 552 users of the National Library of Medicine's MEDLINE database, interviewed by telephone and responding to a highly structured set of open-ended questions, reported 1,158 incidents in which the results of a MEDLINE search was especially helpful (or not helpful) in carrying out professional activities. Systematic analysis of these "critical incidents" produced three comprehensive and detailed views of the purposes and outcomes of MEDLINE searches: (1) why information is sought from MEDLINE; (2) the impact of MEDLINE-derived information on medical decision-making; and (3) the ultimate outcome of having (or not having) the desired information on medical situations prompting a MEDLINE search. Results revealed that MEDLINE is used to satisfy a diversity of medical needs concerning patient care, the progress of biomedical research, the quality of education received by health professionals in training, the safety and effectiveness of health care institutions, the operation of the system of third-party reimbursement, for legal decisions, and for the knowledge of the public.