Literature DB >> 35312626

Ensuring Interrater Reliability When Evaluating Voice Assistants. Comment on "Evaluating Voice Assistants' Responses to COVID-19 Vaccination in Portuguese: Quality Assessment".

Rujittika Mungmunpuntipantip1, Viroj Wiwanitkit2.   

Abstract

Entities:  

Keywords:  COVID-19; Portuguese language; natural user interface; vaccine; voice assistant

Year:  2022        PMID: 35312626      PMCID: PMC8981016          DOI: 10.2196/36610

Source DB:  PubMed          Journal:  JMIR Hum Factors        ISSN: 2292-9495


× No keyword cloud information.
We would like to share our ideas on the paper “Evaluating Voice Assistants’ Responses to COVID-19 Vaccination in Portuguese: Quality Assessment” [1]. Seródio Figueiredo et al [1] concluded, “Under the urgent context of COVID-19 vaccination, this work can help to understand how VAs must be improved to be more useful to the society and how careful people must be when considering VAs as a source of health information.” We agree that voice assistants (VAs) could be useful in managing COVID-19 mass immunization campaigns. Current reports can provide an overview of existing technologies and the mission of various VA suppliers in order to meet current information distribution requirements. However, as previously said, the question of system functioning remains critical. Findings on VAs still have variability and need to be harmonized [2]. In addition, effective governance is required for transparency and usefulness in information partnerships [2]. Seródio Figueiredo et al [1] assessed VAs via 2 evaluators. A set of questions was used. The focus was on agreement between the 2 evaluators. Interevaluator variability should be assessed and presented as well. In general, there should be 3 evaluators to aid the final decision. Additionally, the scoring system of this study might be easily influenced by bias. Essentially, any questionnaire-based study requires a reliability test of the questionnaire. Content validity, face validity, and criterion-related validity tests are required. A standard method, as presented by Bolarinwa [3], should be used. A statistical analysis of the questionnaire’s reliability must be presented in addition to a general description. Without proven measures of questionnaire reliability and evaluator variability, the use of this tool in this study might be questionable.
  3 in total

1.  Principles and methods of validity and reliability testing of questionnaires used in social and health science researches.

Authors:  Oladimeji Akeem Bolarinwa
Journal:  Niger Postgrad Med J       Date:  2015 Oct-Dec

2.  Evaluating the quality of voice assistants' responses to consumer health questions about vaccines: an exploratory comparison of Alexa, Google Assistant and Siri.

Authors:  Emily Couvillon Alagha; Rachel Renee Helbing
Journal:  BMJ Health Care Inform       Date:  2019-11

3.  Evaluating Voice Assistants' Responses to COVID-19 Vaccination in Portuguese: Quality Assessment.

Authors:  Carlos Maurício Seródio Figueiredo; Tiago de Melo; Raphaela Goes
Journal:  JMIR Hum Factors       Date:  2022-03-21
  3 in total

北京卡尤迪生物科技股份有限公司 © 2022-2023.