OBJECTIVE: To explore the applicability of a syndromic surveillance method to the early detection of health information technology (HIT) system failures. METHODS: A syndromic surveillance system was developed to monitor a laboratory information system at a tertiary hospital. Four indices were monitored: (1) total laboratory records being created; (2) total records with missing results; (3) average serum potassium results; and (4) total duplicated tests on a patient. The goal was to detect HIT system failures causing: data loss at the record level; data loss at the field level; erroneous data; and unintended duplication of data. Time-series models of the indices were constructed, and statistical process control charts were used to detect unexpected behaviors. The ability of the models to detect HIT system failures was evaluated using simulated failures, each lasting for 24 h, with error rates ranging from 1% to 35%. RESULTS: In detecting data loss at the record level, the model achieved a sensitivity of 0.26 when the simulated error rate was 1%, while maintaining a specificity of 0.98. Detection performance improved with increasing error rates, achieving a perfect sensitivity when the error rate was 35%. In the detection of missing results, erroneous serum potassium results and unintended repetition of tests, perfect sensitivity was attained when the error rate was as small as 5%. Decreasing the error rate to 1% resulted in a drop in sensitivity to 0.65-0.85. CONCLUSIONS: Syndromic surveillance methods can potentially be applied to monitor HIT systems, to facilitate the early detection of failures.
OBJECTIVE: To explore the applicability of a syndromic surveillance method to the early detection of health information technology (HIT) system failures. METHODS: A syndromic surveillance system was developed to monitor a laboratory information system at a tertiary hospital. Four indices were monitored: (1) total laboratory records being created; (2) total records with missing results; (3) average serum potassium results; and (4) total duplicated tests on a patient. The goal was to detect HIT system failures causing: data loss at the record level; data loss at the field level; erroneous data; and unintended duplication of data. Time-series models of the indices were constructed, and statistical process control charts were used to detect unexpected behaviors. The ability of the models to detect HIT system failures was evaluated using simulated failures, each lasting for 24 h, with error rates ranging from 1% to 35%. RESULTS: In detecting data loss at the record level, the model achieved a sensitivity of 0.26 when the simulated error rate was 1%, while maintaining a specificity of 0.98. Detection performance improved with increasing error rates, achieving a perfect sensitivity when the error rate was 35%. In the detection of missing results, erroneous serum potassium results and unintended repetition of tests, perfect sensitivity was attained when the error rate was as small as 5%. Decreasing the error rate to 1% resulted in a drop in sensitivity to 0.65-0.85. CONCLUSIONS: Syndromic surveillance methods can potentially be applied to monitor HIT systems, to facilitate the early detection of failures.
Authors: Kenneth D Mandl; J Marc Overhage; Michael M Wagner; William B Lober; Paola Sebastiani; Farzad Mostashari; Julie A Pavlin; Per H Gesteland; Tracee Treadwell; Eileen Koski; Lori Hutwagner; David L Buckeridge; Raymond D Aller; Shaun Grannis Journal: J Am Med Inform Assoc Date: 2003-11-21 Impact factor: 4.497
Authors: Marlena Gehret Plagianos; Winfred Y Wu; Colleen McCullough; Marc Paladini; Joseph Lurio; Michael D Buck; Neil Calman; Nicholas Soulakis Journal: Emerg Infect Dis Date: 2011-09 Impact factor: 6.883
Authors: Zach Landis-Lewis; Ronald Manjomo; Oliver J Gadabu; Matthew Kam; Bertha N Simwaka; Susan L Zickmund; Frank Chimbwandira; Gerald P Douglas; Rebecca S Jacobson Journal: Int J Med Inform Date: 2015-07-19 Impact factor: 4.046