Wiard Jorritsma1, Fokie Cnossen2, Rudi A Dierckx3, Matthijs Oudkerk4, Peter M A van Ooijen4. 1. University of Groningen, University Medical Center Groningen, Dept. Radiology, Hanzeplein 1, 9713 GZ Groningen, The Netherlands. Electronic address: w.jorritsma01@umcg.nl. 2. University of Groningen, Institute of Artificial Intelligence and Cognitive Engineering, Nijenborgh 9, 9747 AG Groningen, The Netherlands. 3. University of Groningen, University Medical Center Groningen, Dept. Radiology, Hanzeplein 1, 9713 GZ Groningen, The Netherlands; University of Groningen, University Medical Center Groningen, Dept. Nuclear Medicine and Molecular Imaging, Hanzeplein 1, 9713 GZ Groningen, The Netherlands. 4. University of Groningen, University Medical Center Groningen, Dept. Radiology, Hanzeplein 1, 9713 GZ Groningen, The Netherlands; Center for Medical Imaging North East Netherlands, Hanzeplein 1, 9713 GZ Groningen, The Netherlands.
Abstract
OBJECTIVES: To perform a post-deployment usability evaluation of a radiology Picture Archiving and Communication System (PACS) client based on pattern mining of user interaction log data, and to assess the usefulness of this approach compared to a field study. METHODS: All user actions performed on the PACS client were logged for four months. A data mining technique called closed sequential pattern mining was used to automatically extract frequently occurring interaction patterns from the log data. These patterns were used to identify usability issues with the PACS. The results of this evaluation were compared to the results of a field study based usability evaluation of the same PACS client. RESULTS: The interaction patterns revealed four usability issues: (1) the display protocols do not function properly, (2) the line measurement tool stays active until another tool is selected, rather than being deactivated after one use, (3) the PACS's built-in 3D functionality does not allow users to effectively perform certain 3D-related tasks, (4) users underuse the PACS's customization possibilities. All usability issues identified based on the log data were also found in the field study, which identified 48 issues in total. CONCLUSIONS: Post-deployment usability evaluation based on pattern mining of user interaction log data provides useful insights into the way users interact with the radiology PACS client. However, it reveals few usability issues compared to a field study and should therefore not be used as the sole method of usability evaluation.
OBJECTIVES: To perform a post-deployment usability evaluation of a radiology Picture Archiving and Communication System (PACS) client based on pattern mining of user interaction log data, and to assess the usefulness of this approach compared to a field study. METHODS: All user actions performed on the PACS client were logged for four months. A data mining technique called closed sequential pattern mining was used to automatically extract frequently occurring interaction patterns from the log data. These patterns were used to identify usability issues with the PACS. The results of this evaluation were compared to the results of a field study based usability evaluation of the same PACS client. RESULTS: The interaction patterns revealed four usability issues: (1) the display protocols do not function properly, (2) the line measurement tool stays active until another tool is selected, rather than being deactivated after one use, (3) the PACS's built-in 3D functionality does not allow users to effectively perform certain 3D-related tasks, (4) users underuse the PACS's customization possibilities. All usability issues identified based on the log data were also found in the field study, which identified 48 issues in total. CONCLUSIONS: Post-deployment usability evaluation based on pattern mining of user interaction log data provides useful insights into the way users interact with the radiology PACS client. However, it reveals few usability issues compared to a field study and should therefore not be used as the sole method of usability evaluation.