Romil F Shah1, Stefano Bini1,2, Thomas Vail1,2. 1. Department of Orthopedic Surgery, University of California - San Francisco, San Francisco, California, USA. 2. Department of Orthopedic Surgery, University of Texas at Austin, Austin, Texas, USA.
Abstract
AIMS: Natural Language Processing (NLP) offers an automated method to extract data from unstructured free text fields for arthroplasty registry participation. Our objective was to investigate how accurately NLP can be used to extract structured clinical data from unstructured clinical notes when compared with manual data extraction. METHODS: A group of 1,000 randomly selected clinical and hospital notes from eight different surgeons were collected for patients undergoing primary arthroplasty between 2012 and 2018. In all, 19 preoperative, 17 operative, and two postoperative variables of interest were manually extracted from these notes. A NLP algorithm was created to automatically extract these variables from a training sample of these notes, and the algorithm was tested on a random test sample of notes. Performance of the NLP algorithm was measured in Statistical Analysis System (SAS) by calculating the accuracy of the variables collected, the ability of the algorithm to collect the correct information when it was indeed in the note (sensitivity), and the ability of the algorithm to not collect a certain data element when it was not in the note (specificity). RESULTS: The NLP algorithm performed well at extracting variables from unstructured data in our random test dataset (accuracy = 96.3%, sensitivity = 95.2%, and specificity = 97.4%). It performed better at extracting data that were in a structured, templated format such as range of movement (ROM) (accuracy = 98%) and implant brand (accuracy = 98%) than data that were entered with variation depending on the author of the note such as the presence of deep-vein thrombosis (DVT) (accuracy = 90%). CONCLUSION: The NLP algorithm used in this study was able to identify a subset of variables from randomly selected unstructured notes in arthroplasty with an accuracy above 90%. For some variables, such as objective exam data, the accuracy was very high. Our findings suggest that automated algorithms using NLP can help orthopaedic practices retrospectively collect information for registries and quality improvement (QI) efforts. Cite this article: Bone Joint J 2020;102-B(7 Supple B):99-104.
AIMS: Natural Language Processing (NLP) offers an automated method to extract data from unstructured free text fields for arthroplasty registry participation. Our objective was to investigate how accurately NLP can be used to extract structured clinical data from unstructured clinical notes when compared with manual data extraction. METHODS: A group of 1,000 randomly selected clinical and hospital notes from eight different surgeons were collected for patients undergoing primary arthroplasty between 2012 and 2018. In all, 19 preoperative, 17 operative, and two postoperative variables of interest were manually extracted from these notes. A NLP algorithm was created to automatically extract these variables from a training sample of these notes, and the algorithm was tested on a random test sample of notes. Performance of the NLP algorithm was measured in Statistical Analysis System (SAS) by calculating the accuracy of the variables collected, the ability of the algorithm to collect the correct information when it was indeed in the note (sensitivity), and the ability of the algorithm to not collect a certain data element when it was not in the note (specificity). RESULTS: The NLP algorithm performed well at extracting variables from unstructured data in our random test dataset (accuracy = 96.3%, sensitivity = 95.2%, and specificity = 97.4%). It performed better at extracting data that were in a structured, templated format such as range of movement (ROM) (accuracy = 98%) and implant brand (accuracy = 98%) than data that were entered with variation depending on the author of the note such as the presence of deep-vein thrombosis (DVT) (accuracy = 90%). CONCLUSION: The NLP algorithm used in this study was able to identify a subset of variables from randomly selected unstructured notes in arthroplasty with an accuracy above 90%. For some variables, such as objective exam data, the accuracy was very high. Our findings suggest that automated algorithms using NLP can help orthopaedic practices retrospectively collect information for registries and quality improvement (QI) efforts. Cite this article: Bone Joint J 2020;102-B(7 Supple B):99-104.
Entities:
Keywords:
Data registries; Joint arthroplasty; Machine learning; Natural language processing
Authors: Aditya V Karhade; Jacobien H F Oosterhoff; Olivier Q Groot; Nicole Agaronnik; Jeffrey Ehresman; Michiel E R Bongers; Ruurd L Jaarsma; Santosh I Poonnoose; Daniel M Sciubba; Daniel G Tobert; Job N Doornberg; Joseph H Schwab Journal: Clin Orthop Relat Res Date: 2022-04-12 Impact factor: 4.755
Authors: Nicholas J Giori; John Radin; Alison Callahan; Jason A Fries; Eni Halilaj; Christopher Ré; Scott L Delp; Nigam H Shah; Alex H S Harris Journal: JAMA Netw Open Date: 2021-03-01
Authors: Peijin Han; Sunyang Fu; Julie Kolis; Richard Hughes; Brian R Hallstrom; Martha Carvour; Hilal Maradit-Kremers; Sunghwan Sohn; V G Vinod Vydiswaran Journal: JMIR Med Inform Date: 2022-08-31
Authors: Jacob K Greenberg; Ayodamola Otun; Zoher Ghogawala; Po-Yin Yen; Camilo A Molina; David D Limbrick; Randi E Foraker; Michael P Kelly; Wilson Z Ray Journal: Global Spine J Date: 2021-05-11