INTRODUCTION: Over the past 10 years, a number of systematic reviews have evaluated the validity of multisource feedback (MSF) to assess and quality-assure medical practice. The purpose of this study is to synthesize the results from existing reviews to provide a holistic overview of the validity evidence. METHODS: This review identified eight systematic reviews evaluating the validity of MSF published between January 2006 and October 2016. Using a standardized data extraction form, two independent reviewers extracted study characteristics. A framework of validation developed by the American Psychological Association was used to appraise the validity evidence within each systematic review. RESULTS: In terms of validity evidence, each of the eight reviews demonstrated evidence across at least one domain of the American Psychological Association's validity framework. Evidence of assessment validity within the domains of "internal structure" and "relationship to other variables" has been well established. However, the domains of content validity (ie, ensuring that MSF tools measure what they are intended to measure); consequential validity (ie, evidence of the intended or unintended consequences MSF assessments may have on participants or wider society), and response process validity (ie, the process of standardization and quality control in the delivery and completion of assessments) remain limited. DISCUSSION: Evidence for the validity of MSF has, across a number of domains, been well established. However, the size and quality of the existing evidence remains variable. To determine the extent to which MSF is considered a valid instrument to assess medical performance, future research is required to determine the following: (1) how best to design and deliver MSF assessments that address the identified limitations of existing tools and (2) how to ensure that involvement within MSF supports positive changes in practice. Such research is integral if MSF is to continue to inform medical performance and subsequent improvements in the quality and safety of patient care.
INTRODUCTION: Over the past 10 years, a number of systematic reviews have evaluated the validity of multisource feedback (MSF) to assess and quality-assure medical practice. The purpose of this study is to synthesize the results from existing reviews to provide a holistic overview of the validity evidence. METHODS: This review identified eight systematic reviews evaluating the validity of MSF published between January 2006 and October 2016. Using a standardized data extraction form, two independent reviewers extracted study characteristics. A framework of validation developed by the American Psychological Association was used to appraise the validity evidence within each systematic review. RESULTS: In terms of validity evidence, each of the eight reviews demonstrated evidence across at least one domain of the American Psychological Association's validity framework. Evidence of assessment validity within the domains of "internal structure" and "relationship to other variables" has been well established. However, the domains of content validity (ie, ensuring that MSF tools measure what they are intended to measure); consequential validity (ie, evidence of the intended or unintended consequences MSF assessments may have on participants or wider society), and response process validity (ie, the process of standardization and quality control in the delivery and completion of assessments) remain limited. DISCUSSION: Evidence for the validity of MSF has, across a number of domains, been well established. However, the size and quality of the existing evidence remains variable. To determine the extent to which MSF is considered a valid instrument to assess medical performance, future research is required to determine the following: (1) how best to design and deliver MSF assessments that address the identified limitations of existing tools and (2) how to ensure that involvement within MSF supports positive changes in practice. Such research is integral if MSF is to continue to inform medical performance and subsequent improvements in the quality and safety of patient care.
Authors: Eva K Hennel; Andrea Trachsel; Ulrike Subotic; Andrea C Lörwald; Sigrid Harendza; Sören Huwendiek Journal: Med Educ Date: 2022-03-16 Impact factor: 7.647
Authors: Eva K Hennel; Ulrike Subotic; Christoph Berendonk; Daniel Stricker; Sigrid Harendza; Sören Huwendiek Journal: BMC Med Educ Date: 2020-10-12 Impact factor: 2.463