Faina Linkov1, Mita Lovalekar, Ronald LaPorte. 1. Department of Epidemiology, Graduate School of Public Health, University of Pittsburgh, Pittsburgh, PA 15261, USA. fyL1@pitt.edu
Abstract
AIM: To examine the feasibility of using peer review for the quality control of online materials. METHODS: We analyzed the inter-rater agreement on the quality of epidemiological lectures online, based on the Global Health Network Supercourse lecture library. We examined the agreement among reviewers by looking at kappa statistics and intraclass correlations. Seven expert reviewers examined and rated a random sample of 100 Supercourse lectures. Their reviews were compared with the reviews of the lay Supercourse reviewers. RESULTS: Both expert and non-expert reviewers rated lectures very highly, with a mean overall score of 4 out of 5. Kappa (Kappa) statistic and intraclass correlations indicated that inter-rater agreement for experts and non-experts was surprisingly low (below 0.4). CONCLUSIONS: To our knowledge, this was the first time that poor inter-rater agreement was demonstrated for the Internet lectures. Future research studies need to evaluate the alternatives to the peer review system, especially for online materials.
AIM: To examine the feasibility of using peer review for the quality control of online materials. METHODS: We analyzed the inter-rater agreement on the quality of epidemiological lectures online, based on the Global Health Network Supercourse lecture library. We examined the agreement among reviewers by looking at kappa statistics and intraclass correlations. Seven expert reviewers examined and rated a random sample of 100 Supercourse lectures. Their reviews were compared with the reviews of the lay Supercourse reviewers. RESULTS: Both expert and non-expert reviewers rated lectures very highly, with a mean overall score of 4 out of 5. Kappa (Kappa) statistic and intraclass correlations indicated that inter-rater agreement for experts and non-experts was surprisingly low (below 0.4). CONCLUSIONS: To our knowledge, this was the first time that poor inter-rater agreement was demonstrated for the Internet lectures. Future research studies need to evaluate the alternatives to the peer review system, especially for online materials.
Authors: Ronald E LaPorte; Faina Linkov; Tony Villasenor; Francois Sauer; Carlos Gamboa; Mita Lovalekar; Eugene Shubnikov; Akira Sekikawa; Eun Ryoung Sa Journal: BMJ Date: 2002-12-21
Authors: Nicolas Padilla-Raygoza; Ronald LaPorte; Eugene Shubnikov; Meredith Hennon; Robert P Edwards; Faina Linkov Journal: J Immigr Minor Health Date: 2011-12
Authors: Nicolas Padilla-Raygoza; Faina Linkov; Eugene Shubnikov; Ron E LaPorte; Rosalina Diaz-Guerrero Journal: Cent Asian J Glob Health Date: 2013-02-21