U Mansmann1. 1. IBE, Biometry and Bioinformatics, University of Munich, Munich, Germany. mansmann@ibe.med.uni-muenchen.de
Abstract
OBJECTIVES: The current literature on the use of microarrays to generate prognostic profiles is still a methodological wasteland. Many valid questions, such as how profiling studies should be evaluated and what conclusions can be drawn, remain unanswered. Or how can flaws in the collection, analysis and interpretation of data be detected? Undiscovered imperfections can lead to a waste of valuable resources as well as be a latent source of false conclusions. METHODS: Three seminal papers on the prognosis of breast cancer using genomic profiling will be discussed. Six principles of good experimental design will be used for methodological guidance: defining relevant endpoints, avoiding systematic bias, generalizability of results, appropriately sized samples to achieve sufficient power, simple design to improve interpretability, and avoiding artificial assumptions. RESULTS: Severe violations of at least one of the six principles of good experimental design can be found in each of the three papers. A strategy is presented to assess whether a study has achieved a high level of methodological quality. This strategy also helps to establish a suitable protocol for future profiling projects. CONCLUSIONS: Determining the design of a study in a protocol is a first step to avoid impending pitfalls. The protocol should deal with the problem of understanding the complex reality behind genomic profiling. There are basic guiding principles which can help handle the complex task of designing prognostic studies to find genomic signatures.
OBJECTIVES: The current literature on the use of microarrays to generate prognostic profiles is still a methodological wasteland. Many valid questions, such as how profiling studies should be evaluated and what conclusions can be drawn, remain unanswered. Or how can flaws in the collection, analysis and interpretation of data be detected? Undiscovered imperfections can lead to a waste of valuable resources as well as be a latent source of false conclusions. METHODS: Three seminal papers on the prognosis of breast cancer using genomic profiling will be discussed. Six principles of good experimental design will be used for methodological guidance: defining relevant endpoints, avoiding systematic bias, generalizability of results, appropriately sized samples to achieve sufficient power, simple design to improve interpretability, and avoiding artificial assumptions. RESULTS: Severe violations of at least one of the six principles of good experimental design can be found in each of the three papers. A strategy is presented to assess whether a study has achieved a high level of methodological quality. This strategy also helps to establish a suitable protocol for future profiling projects. CONCLUSIONS: Determining the design of a study in a protocol is a first step to avoid impending pitfalls. The protocol should deal with the problem of understanding the complex reality behind genomic profiling. There are basic guiding principles which can help handle the complex task of designing prognostic studies to find genomic signatures.