Using expert panels to determine the level of cancer knowledge required of junior doctors in australia. Part 2: sources of variability.

This paper describes the sources of variability encountered in the use of an expert panel to review cancer-related knowledge items, necessary for medical students. Variability was observed in the interpretation of written material relating to the definition and rationale for the task to be completed by individual panel members, including the definition of a junior doctor, and levels of understanding and specificity. Panel sessions undertaken in phase II provided facilitated discussion and the ability to clearly define the aims and tasks required of participants, resulting in data of a higher quality. Consensus was achieved in a single session that would have likely taken several iterations of individual data collection to achieve. Eliminating phase I has the potential to remove the majority of variability encountered in this study. Subsequently, the resultant decrease in time demanded of participants would likely result in higher recruitment and participation rates.