Hands-on workshops as an effective means of learning advanced technologies including genomics, proteomics and bioinformatics.

Genomics and proteomics have emerged as key technologies in biomedical research, resulting in a surge of interest in training by investigators keen to incorporate these technologies into their research. At least two types of training can be envisioned in order to produce meaningful results, quality publications and successful grant applications: (1) immediate short-term training workshops and (2) long-term graduate education or visiting scientist programs. We aimed to fill the former need by providing a comprehensive hands-on training course in genomics, proteomics and informatics in a coherent, experimentally-based framework. This was accomplished through a National Heart, Lung, and Blood Institute (NHLBI)-sponsored 10-day Genomics and Proteomics Hands-on Workshop held at National Jewish Health (NJH) and the University of Colorado School of Medicine (UCD). The course content included comprehensive lectures and laboratories in mass spectrometry and genomics technologies, extensive hands-on experience with instrumentation and software, video demonstrations, optional workshops, online sessions, invited keynote speakers, and local and national guest faculty. Here we describe the detailed curriculum and present the results of short- and long-term evaluations from course attendees. Our educational program consistently received positive reviews from participants and had a substantial impact on grant writing and review, manuscript submissions and publications.

Introduction

In recent years, genomics and proteomics have emerged as key technologies in biomedical research programs. These types of analyses are now critical to performing effective clinical research aimed at developing a better understanding of disease pathology, as well as designing new approaches for clinical treatment [1]. For example, as genomic technologies have matured, so have a growing number of examples of genomic profiles directly being used to create individualized treatment plans [2]. These include the use of relatively simple single nucleotide polymorphism (SNP) genotyping for determining an appropriate treatment plan or for predicting the chemotherapeutic regimen most likely to benefit a breast cancer patient [3-5]. In addition, mass spectrometry, metabolomics and proteomics are becoming essential for providing sensitive methods of characterizing and analyzing both small molecules and proteins [6-12]. Both genomics and proteomics have proven so successful in these regards that demand has far exceeded the supply of trained personnel equipped to exploit the available technologies. This has resulted in a surge of interest in training by investigators keen to utilize these technologies in their research. The need for researchers trained in omics technologies has been described by several groups, including working groups from the National Institutes of Health (NIH) [13]. In our own experience, the demand for training has been demonstrated at local, national and international levels. For example, in the past 8 years, previously established hands-on workshops in Metabolomics and Proteomics at National Jewish Health (NJH) and the University of Colorado School of Medicine (UCD) have been attended by over 400 individuals from around the world.

To support technology-dependent clinical research studies, at least two types of training in omics technologies can be envisioned: (1) immediate short-term training in the form of workshops and (2) long-term instruction in the form of graduate education and visiting science programs. We aimed to fill the former need by providing an intensive and comprehensive hands-on training course in genomics, proteomics and bioinformatics in a coherent experimentally-based framework. Information gleaned from former participants’ surveys enabled us to determine that there is still a large need for introductory courses which focus on biomedical research such as the one described herein.

Several recent publications have focused on the need for experimental standards in omics research, especially when applied to human disease [14-17] and education is one means of lessening experimental error. To be of maximum benefit, education should ideally occur before experimental planning has begun, in order to decrease the possibility that the wrong platform, technology, experimental design or data analysis is chosen. For example, a first step toward utilizing omics technologies is the development of a well-defined hypothesis which clearly outlines the research question under investigation. If a researcher is interested in global expression patterns and/or biomarkers, he/she will need to determine if genomics, metabolomics or proteomics is more appropriate. Alternatively, an investigator may determine that a combination of approaches is necessary. Even after determining a general approach, the researcher must choose which specific platform and sample preparation strategies to utilize and this requires an understanding of the technologies available and the strengths and weaknesses of each approach. Given the plethora of equipment and protocols currently available, a new investigator could conceivably become overwhelmed by alternatives and subsequently make a poor choice that could lead to downstream difficulties in data analysis and interpretation. After determining an appropriate platform in which to test the hypothesis, careful experimental design and quality control measures are also critical to the success of the project as well as for interpreting the results. Given the large datasets generated by these technologies, knowledge of data analysis workflows and options, biostatistics and bioinformatics are vital and these are areas in which many investigators require additional training, specifically in the context of omics research. Finally, interpretation of omics results is dependent on a broad understanding of biochemistry and molecular biology, often presented within a knowledge-based pathway framework. Our training course exposed participants to each of these areas and, more specifically, enabled them to make informed decisions regarding the critical choice of platform for their particular research question.

To this end, we assembled a group of educators and scientists who understand the need for training and have experience in omics technologies and hands-on workshops. NJH is a premier hospital and research institution with a reputation for high quality clinical and basic research, effective professional education and extensive collaborations. The mass spectrometry facility is well established and supports over 100 national investigators involved in NIH-funded research in lung and immune diseases, diabetes, nutrition/obesity and others [18-25]. Similarly, the Program Directors at Colorado State University (CSU) and UCD have established genomics and proteomics research and support projects in pulmonary diseases, cancer and women’s health, and have programs in nutrition, environmental and microbial science [26-38]. The Directors of the workshop have developed and taught courses and workshops covering the subjects of metabolomics, genomics, proteomics, bioinformatics and biostatistics. In general, we accomplished the following: (1) built a curriculum focusing on outcome-structured learning in genomics, proteomics and bioinformatics technologies, which enabled investigators to immediately apply their knowledge to existing research studies; (2) provided novel, multidisciplinary training experiences using hands-on techniques and an established model by experienced investigators and educators; (3) disseminated educational materials such that knowledge and information could be used to benefit a wide variety of audiences; and (4) provided a portal for collaborative opportunities for both young and established investigators, thereby encouraging formation of interdisciplinary teams.

The intensive course schedule covered 10 consecutive days and was offered annually from 2007 to 2010 in the summer months. The NHLBI provided funding through a T15 award (N. Reisdorph, Principal Investigator) which covered faculty salary, invited keynote speakers, transportation between institutions and hotels, administrative support and laboratory module consumables. One salient feature of the course was the inclusion of over 20 faculty members with an array of experience spanning genomics, proteomics, bioinformatics, biostatistics, mass spectrometry, genetics, molecular biology, biochemistry, lung and heart disease, clinical studies, human research, and mouse and cell models. In addition, a number of vendors (Agilent Technologies, Affymetrix, Biorad and Pierce) provided laboratory module consumables as well as beverages, lunches and keynote speaker dinners for participants. Participants paid for their travel and housing expenses and a moderate $1000 tuition fee was charged to cover items not covered through the T15 award, including catering and keynote receptions. Scholarships were available each year to cover all costs for up to four participants.

Course details

The genomics and proteomics workshop structure combined hands-on training with lecture-based learning. It was carefully designed to allow students to experience an application-based workflow from sample preparation to data analysis. Classes were composed of no more than 22 students, who were split into 2 groups. Following an introductory session on the day of arrival, Group 1 attended proteomics training for days 2–4, then optional workshops for day 5, then genomics training for days 6–8 and finally bioinformatics for days 8–10 (Table 1 showing general schedule). Conversely, Group 2 attended genomics training for days 2–4, then optional workshops for day 5, then proteomics training for days 6–8 and finally bioinformatics for days 8–10. This allowed for small class size and one-on-one contact during instruction, while enabling all participants to interact with each other. Individual curricula for genomics, proteomics, bioinformatics and optional workshops are described below and detailed schedules are included in the online supplementary information. Information on group events is also described.

Table 1

General course layout

Note: Group 1 attended proteomics training for days 2–4, then optional workshops for day 5, then genomics training for days 6–8 and finally bioinformatics for days 8–10. Conversely, Group 2 attended genomics training for days 2–4, then optional workshops for day 5, then proteomics training for days 6–8 and finally bioinformatics for days 8–10.

Proteomics curriculum

A copy of the course curriculum is included in the supplementary information (Supplementary File 1), with proteomics spanning days 2–4. The proteomics curriculum was carefully designed to maximize the retention of information and minimize student fatigue. Training material was presented in both lecture and laboratory session modules. Lectures and laboratory sessions were strategically interspersed throughout the 3 day period. Thus student training time consistently alternated between lecture and hands-on activities, thereby reducing mental fatigue and helping to keep students engaged. Lectures and laboratory sessions were ordered logically to reflect a typical workflow, i.e., sample preparation > data acquisition > data analysis. Lecture topics included introduction to mass spectrometry and database searching, quantitative proteomics, advanced technologies and clinical proteomics. During hands-on lab sections, students were divided into groups of three, making the student-to-instructor ratio 3:1. The composition of these groups was predetermined by experience level and remained the same throughout the entire course. This provided an element of consistency that helped students feel comfortable with the learning environment. It also enabled the laboratory instructor to tailor the session to meet the needs of the individual participants. For example, the instructor may spend additional time describing the individual components of a mass spectrometer to one group, but may discuss specific method parameters with another more experienced group. Laboratory sessions were divided into three stations and organized in a rotation-based format. Each group of three students spent 60–75 min per rotation at the following stations: sample preparation, instrumentation, informatics and data analysis. Over the course of the workshop, students returned several times to specific lab stations to continue and complete various aspects of sample preparation and instruction on mass spectrometry instruments and software. This rotation style format exposes students to a wide variety of proteomics and mass spectrometry concepts, while maintaining the continuity required to effectively experience an application-based workflow from sample preparation to data analysis.

During laboratories, students were taken through typical proteomics work flows from sample preparation to data acquisition and from data analysis to protein identification. Each student was responsible for preparing their own digested protein sample and students contributed to creating mass spectrometry data acquisition methods. Students identified the protein(s) in their sample during the database search laboratory session, which was the final session of the 3-day proteomics course. In the days leading up to this final session, students were reminded that they would discover the identity of the protein(s) in their sample and there was a friendly competition to see who would have the highest search result score. This was designed to build anticipation and help keep students engaged.

Genomics curriculum

A copy of the course curriculum is included in the Supplementary File 1, with genomics spanning days 6–8. The genomics curriculum schedule interweaved hands-on bench work with lectures and demonstration videos. For the hands-on section, the participants in 5 groups of 2 converted total RNA into biotinylated cDNA using the standard protocol for running on an Affymetrix gene expression microarray. Quality control of the intermediate and final products was assessed using the Agilent Bioanalyzer. During incubation times, lectures and demonstration videos were presented. The lecture topics included: principles and applications of DNA microarray technology, a comparison of multiple gene expression platforms, genomic DNA SNP analysis using microarrays, epigenetic applications, application of microarray technology to advancing medicine and translational research, using publicly available datasets in your own research projects, troubleshooting quality control issues and data analysis, and integrating proteomic and gene expression datasets. In the final year’s course offering (2010), lectures in next-generation sequencing methods and applications were expanded. Various hands-on components typically completed by genomic core personnel were presented by video demonstrations, including microarray sample loading, washing, staining and scanning.

Bioinformatics curriculum

A copy of the course curriculum is included in the Supplementary File 1, with bioinformatics spanning days 8–10. All bioinformatics instructions were provided in the computational lab and synchronized with the laboratory sessions for a unique hands-on approach. Lecture topics included experimental design, power analysis, data pre-processing, quality control, exploratory data analysis, class discovery, differential expression and class prediction. The BRB-Array Tools software, based on an Excel interface of R BioConductor packages, was used for demonstrations by the instructors followed by hands-on sessions by the participants. This software was selected since it allowed participants with a diverse computational background to learn the fundamental data analysis steps in a user-friendly environment. Additional bioinformatics instruction was provided on the final days of the workshop and covered additional online resources such as Genome Browsers, Gene Set Enrichment Analysis, Promoter Analysis, Gene Ontology Analysis, Database Resources for Genetic Variations and Disorders and in later years, an introduction to Next-generation Sequencing Data Analysis.

Keynote and special lectures, optional workshops and on-line training

In addition to the genomics, proteomics and informatics curricula, the course consisted of 4 keynote lectures, one half-day of optional workshops, and pre- and post-course online training. Table 2 lists keynote lecturers and their affiliations. Special lectures that were presented by local speakers are also listed in Table 2.

Table 2

Keynote and special lecturers

Speaker	Institution/affiliation	Title	Year(s) presented
Natalie Ahn, PhD	University of Colorado, Boulder	Phosphoproteomics	2010
Keith Baggerly, PhD	MD Anderson Cancer Center	Evolution of forensic bioinformatics	2009, 2010
Pierre Chaurand, PhD	University of Montreal	Mass spectrometry imaging	2009
David Erle, MD	University of California, San Francisco	Microarray pearls and pitfalls	2007–2010
Leroy Hood, MD/PhD	Institute for Systems Biology	Systems biology and systems medicine	2007
Alexy Nesvizhskii, PhD	University of Michigan	Bioinformatics in proteomic applications of heart, lung and blood disorders	2007–2010
Kevin Schey, PhD	Vanderbilt University	Spatially-resolved proteomics: MALDI tissue imaging and beyond	2007, 2008, 2010
David Schwartz, MD	National Jewish Health	Environmental genomics and human health	2008
Michael Edwards, PhD	University of Colorado, AMC	Integrating publicly-available microarray data into your analysis	2007–2010
Mark Geraci, PhD	University of Colorado, AMC	Advancing translational medicine with DNA microarrays	2007–2010
Kirk Hansen, PhD	University of Colorado, AMC	Practical proteomics and spectral interpretation	2007–2010
Sonia Leach, PhD	National Jewish Health	SNP analysis with F-SNP, VarioWatch and SequenceVariantAnalyzer	2010
David Riches, PhD	National Jewish Health	Introduction to model systems	2008–2010
Ivana Yang, PhD	University of Colorado, AMC	Genomic studies of DNA methylation patterns	2009, 2010

Note: Keynote speakers are highlighted in bold.

Optional workshops varied each year depending on participants’ interest. Optional workshops were held on Sunday morning and included the following topics: Additional Hands-on Experience with Mass Spectrometers, Two-dimensional Gel Electrophoresis, Introduction to Grant Writing, Biostatistics, Alternate Microarray Platform, Bioinformatics, MicroRNA Analysis and Literature Reviews.

Online training was held via live and recorded sessions and consisted of introductory material for the proteomics sessions. A schedule with topics is listed in Supplementary File 2. In addition, a 4-session online workshop was held in January 2012 to provide updated information to participants (Supplementary File 3). The fields of genomics, proteomics and bioinformatics are constantly changing and one challenge of omics training is consistently providing updated and relevant information. This was accomplished both through annual review and revision of material, and through the final online session. All sessions were recorded and are available by contacting the corresponding author.

Participant demographics and statistics

The workshop was aimed at junior and senior investigators but was also open to postdoctoral and clinical fellows and senior graduate students. In all, we provided hands-on and web-based training to 87 individuals from 55 institutions, all of whom rated the training they received as outstanding at the end of the course. Over 120 individuals applied for training and, as the course was NIH-NHLBI funded, priority was given to individuals with funding in an area related to heart, lung or blood diseases. Participants were chosen based on recommendations from a selection committee composed of course directors, instructors and advisors. Educational experience of participants from the four annual course offerings included: BS/BA (13%), PhD (33%), MD (48%) and MD/PhD (6%) (Figure 1A). Participants were primarily (48% of total) faculty members including Assistant Professor (26%), Associate Professor (6%), Professor (7%), Research Assistant Professor (3%) and Instructor (3%), although postdocs and fellows made up 30% of the participants (Figure 1B). Just over 10% of participants were graduate students and 2 proteomics core managers attended the course. Generally speaking, only 5% of the participants stated that they had significant training in omics and considered themselves experts, with the vast majority not having any prior training. Information gleaned from these participants’ statistics enabled us to determine that there is still a large need for introductory courses which focus on biomedical research. While advanced course work is certainly necessary, our course filled the significant niche for introductory and mid-level investigators.

Figure 1

Participant demographics Shown are professional positions (A) and educational background (B) of course participants. Participants were mainly PhDs and MDs with faculty positions and included 2 proteomics core facility managers.

Short-term evaluations

Following each section, participants were asked to fill out a web-based evaluation form which was developed in collaboration with Dr. Carol Hodgson, an Associate Dean for Curriculum and Evaluation and Director of the Educational Development and Research Office at the University of Colorado, Denver. The form required rating various aspects of the workshop from poor, fair, good, very good and excellent, with corresponding numerical scores of 1 (poor)–5 (excellent). Median scores were tabulated for each laboratory and lecture and for administrative and overall course ratings. Composite scores for courses are shown in Figure 2. Scores for “overall quality” of the workshops in 2007, 2008, 2009 and 2010 were 4.76, 4.80, 4.84 and 4.63, respectively (far right of bar graph).

Figure 2

Composite scores Scores are shown for courses held in 2007–2010. The numbers indicate average scores for all lectures and laboratories within a category.

Course evaluations were taken seriously and were critical to shaping and improving the following years’ offering. For example, a number of participants suggested video demonstrations to replace hands-on genomics sample preparation, which was implemented in 2008. In addition, bioinformatics sessions originally spanned entire days; however these were split into half-days and interspersed with hands-on laboratory sessions after participant feedback suggested there were too many hours spent in the computer lab. Lectures and labs that received relatively poor ratings (greater than 10% fair ratings or that had less than 40% excellent ratings) were revised where possible in subsequent years.

Long-term evaluations

Approximately 3 months following the 2010 course, a follow-up evaluation was distributed to former NHLBI 10-day omics course attendees. Supplementary File 4 lists the questions that were asked to participants. Of particular note is that of the 49 responders, 98% rated the course very good to excellent and 100% indicated that they would recommend the course to others. Ninety-eight percent of responders rated the quality of the workshop as very good to excellent. Impressively, 44% said that the workshop had had high impact on their research. As shown in Figure 3, responders who were frequently using specific techniques in their own laboratories, or through a core facility, more than quadrupled following the course. For example, only 10% frequently used genomics informatics techniques prior to the course, but over 50% frequently used these techniques following the course. This could indicate that education helped overcome a significant barrier in learning by exposing individuals to the techniques in a hands-on fashion. In addition to increased utilization of techniques, training apparently helped participants in other aspects of their work. Figure 4 shows the results to the question “please indicate if you have used knowledge gained in the course in any of the following scenarios”. Almost all participants indicated that they had used knowledge gained in the workshop in reading journal articles (96%) and attending scientific meetings/seminars (94%). An impressive 76% indicated that the course had benefited them in terms of manuscript review, and almost half indicated the course had helped with non-NIH grant review and thesis review. This indicates that the course not only helped with research programs, but the knowledge gained was applied in a myriad of other ways, all of which have a direct positive impact on NIH-funded omics research.

Figure 3

Usage of techniques The percentage of responders who frequently use a technique in their own laboratory (green bars) or through a core facility (blue bars) is shown. A total of 49 individuals responded to this series of questions. In all cases the number of individuals using a specific technique increased following participation in the course. Percentage of participants utilized the listed techniques in their own lab before the workshop and after the workshop was indicated in light green bars and dark green bars, respectively, while percentage of participants utilized the listed techniques through core facility before the workshop and after the workshop was indicated in light blue bars and dark blue bars, respectively.

Figure 4

Value of workshop beyond research Respondents’ answers to the question “please indicate if you have used knowledge gained in the NHLBI omics course in any of the following scenarios” were analyzed. This chart demonstrates the utility of hands-on training beyond direct usage in research projects.

Participant comments

In addition to ranking specific course elements, participants were asked to provide written feedback regarding the course. This enabled participants to make specific requests regarding the curriculum. For example, in the first year the curriculum included a “mock grant proposal” which entailed working in groups to develop short grant proposals that included genomics and proteomics experiments. Groups were balanced to include junior and senior scientists and were given time for 2 group discussions and 1 presentation. In addition to the discussion periods, participants were also encouraged to discuss the mock proposals during non-workshop hours as a means of facilitating collaborations after the workshop. While over half of the participants rated the mock grant proposal modules as very good to excellent, their written comments demonstrated that the inclusion of this module was too time-consuming and challenging and it was thus eliminated in subsequent workshops.

Participant feedback also resulted in developing videos to complement the genomics modules, including alternate genomics strategies (SNP, next-generation sequencing), including more proteomics informatics sessions, expanding quantitative proteomics sessions and decreasing the amount of material covered in the bioinformatics sessions. A short compilation of written comments submitted by workshop attendees is included in Supplementary File 5.

Technical details and additional information

A search of PubMed and educational journals demonstrated a lack of publications related to omics-based education, in spite of the availability of several courses at educational and industrial institutions. To our knowledge, the described course constituted the only hands-on multi-omics course of its kind. As mentioned, the NHLBI provided significant funding to enable the presentation of the courses from 2007 to 2010 and hence no further courses are currently being held. Therefore, we have included additional information that could be of use to others in the field of omics education and training.

Advertising

The course was advertised through several portals including local institutions, institutions with primarily-underserved members, vendor contact lists, proteomics and genomics core laboratories and websites. Scientists receiving funding from NHLBI for proteomics or genomics research were identified through the NIH Research Portfolio Online Reporting Tools (RePORT) website and contacted through email. Course brochures were also sent to prospective and previous participants and course instructors for distribution.

Course demand

In the first year 43 individuals applied and 22 were admitted. In the second year, there were 48 applicants and 23 were admitted. In the third year, 27 applied and 20 were admitted into the course. In the fourth year, 28 applied and 22 were admitted. Due to the hands-on nature of the course, the course directors determined that 20–22 participants allowed individuals to receive experience on instrumentation while providing substantial one-on-one time with instructors. While lecture-only based courses could conceivably reach a larger audience, by definition they lack hands-on experience. The drop in demand from the last 2 years was attributed to several circumstances including a declining national economy that resulted in a decrease in overall scientific budgets. In addition, the course necessarily recruited investigators whose research focused on heart, lung and blood studies and our applicant pool was therefore limited. Members of industry were also not allowed to attend the course. Based on the number of inquiries between 2007 and 2010 from non-NHLBI investigators, and based on full capacity for current metabolomics and proteomics courses at NJH and UCD, it is the opinion of the authors that there is still a high demand for these types of hands-on courses.

Summary and accessing course material

Course evaluations and participant comments have shown that our intensive hands-on workshop formats are an effective model for presenting essential lecture and laboratory information in a relatively short amount of time. This curriculum could be the framework for long-term training strategies and can be conducted in any number of laboratories. While intensive, 10 days seemed an appropriate length of time for the course and allowed full-time researchers, physician-scientists, post-docs and graduate students to attend. For copies of the videotaped lectures, copies of course evaluations or access to the web-based introductory material, please contact the corresponding author.

Authors’ contributions

NR was responsible for the overall development of the workshop, directing the proteomics modules, managing the grant, compiling evaluations and for the production of the manuscript. CC and RS were responsible for the genomics modules, NR, RR and JP were responsible for proteomics modules including instructing and curriculum development, while KK and TLP were responsible for the bioinformatics modules. DJE, KS, AN and MG assisted with curriculum development, participant selection and presented lectures in the workshop. TLP prepared figures for the manuscript. All authors reviewed the manuscript, read and approved the final manuscript.

Competing interests

None of the authors has any competing interests to declare.