Pamela Wronski1, Michel Wensing2, Sucheta Ghosh3, Lukas Gärttner2, Wolfgang Müller3, Jan Koetsenruijter2. 1. Department of General Practice and Health Services Research, Heidelberg University Hospital, Im Neuenheimer Feld 130.3, 69120, Heidelberg, Germany. pamela.wronski@med.uni-heidelberg.de. 2. Department of General Practice and Health Services Research, Heidelberg University Hospital, Im Neuenheimer Feld 130.3, 69120, Heidelberg, Germany. 3. Scientific Databases and Visualization Group (SDBV), Heidelberg Institute for Theoretical Studies - HITS gGmbH, Schloss-Wolfsbrunnenweg 35, 69118, Heidelberg, Germany.
Abstract
BACKGROUND: Quantitative data reports are widely produced to inform health policy decisions. Policymakers are expected to critically assess provided information in order to incorporate the best available evidence into the decision-making process. Many other factors are known to influence this process, but little is known about how quantitative data reports are actually read. We explored the reading behavior of (future) health policy decision-makers, using innovative methods. METHODS: We conducted a computer-assisted laboratory study, involving starting and advanced students in medicine and health sciences, and professionals as participants. They read a quantitative data report to inform a decision on the use of resources for long-term care in dementia in a hypothetical decision scenario. Data were collected through eye-tracking, questionnaires, and a brief interview. Eye-tracking data were used to generate 'heatmaps' and five measures of reading behavior. The questionnaires provided participants' perceptions of understandability and helpfulness as well as individual characteristics. Interviews documented reasons for attention to specific report sections. The quantitative analysis was largely descriptive, complemented by Pearson correlations. Interviews were analyzed by qualitative content analysis. RESULTS: In total, 46 individuals participated [students (85%), professionals (15%)]. Eye-tracking observations showed that the participants spent equal time and attention for most parts of the presented report, but were less focused when reading the methods section. The qualitative content analysis identified 29 reasons for attention to a report section related to four topics. Eye-tracking measures were largely unrelated to participants' perceptions of understandability and helpfulness of the report. CONCLUSIONS: Eye-tracking data added information on reading behaviors that were not captured by questionnaires or interviews with health decision-makers.
BACKGROUND: Quantitative data reports are widely produced to inform health policy decisions. Policymakers are expected to critically assess provided information in order to incorporate the best available evidence into the decision-making process. Many other factors are known to influence this process, but little is known about how quantitative data reports are actually read. We explored the reading behavior of (future) health policy decision-makers, using innovative methods. METHODS: We conducted a computer-assisted laboratory study, involving starting and advanced students in medicine and health sciences, and professionals as participants. They read a quantitative data report to inform a decision on the use of resources for long-term care in dementia in a hypothetical decision scenario. Data were collected through eye-tracking, questionnaires, and a brief interview. Eye-tracking data were used to generate 'heatmaps' and five measures of reading behavior. The questionnaires provided participants' perceptions of understandability and helpfulness as well as individual characteristics. Interviews documented reasons for attention to specific report sections. The quantitative analysis was largely descriptive, complemented by Pearson correlations. Interviews were analyzed by qualitative content analysis. RESULTS: In total, 46 individuals participated [students (85%), professionals (15%)]. Eye-tracking observations showed that the participants spent equal time and attention for most parts of the presented report, but were less focused when reading the methods section. The qualitative content analysis identified 29 reasons for attention to a report section related to four topics. Eye-tracking measures were largely unrelated to participants' perceptions of understandability and helpfulness of the report. CONCLUSIONS: Eye-tracking data added information on reading behaviors that were not captured by questionnaires or interviews with health decision-makers.
Entities:
Keywords:
Data use; Decision-making; Evidence-based health policy; Eye-tracking; Health policy; Laboratory; Reading behavior
Authors: Bradley C Johnston; Pablo Alonso-Coello; Jan O Friedrich; Reem A Mustafa; Kari A O Tikkinen; Ignacio Neumann; Per O Vandvik; Elie A Akl; Bruno R da Costa; Neill K Adhikari; Gemma Mas Dalmau; Elise Kosunen; Jukka Mustonen; Mark W Crawford; Lehana Thabane; Gordon H Guyatt Journal: CMAJ Date: 2015-10-26 Impact factor: 8.262