Using a Concept Inventory to Reveal Student Thinking Associated with Common Misconceptions about Antibiotic Resistance.

Misconceptions, also known as alternate conceptions, about key concepts often hinder the ability of students to learn new knowledge. Concept inventories (CIs) are designed to assess students' understanding of key concepts, especially those prone to misconceptions. Two-tiered CIs include prompts that ask students to explain the logic behind their answer choice. Such two-tiered CIs afford an opportunity for faculty to explore the student thinking behind the common misconceptions represented by their choice of a distractor. In this study, we specifically sought to probe the misconceptions that students hold prior to beginning an introductory microbiology course (i.e., preconceptions). Faculty-learning communities at two research-intensive universities used the validated Host-Pathogen Interaction Concept Inventory (HPI-CI) to reveal student preconceptions. Our method of deep analysis involved communal review and discussion of students' explanations for their CI answer choice. This approach provided insight valuable for curriculum development. Here the process is illustrated using one question from the HPI-CI related to the important topic of antibiotic resistance. The frequencies with which students chose particular multiple-choice responses for this question were highly correlated between institutions, implying common underlying misconceptions. Examination of student explanations using our analysis approach, coupled with group discussions within and between institutions, revealed patterns in student thinking to the participating faculty. Similar application of a two-tiered concept inventory by general microbiology instructors, either individually or in groups, at other institutions will allow them to better understand student thinking related to key concepts in their curriculum.

INTRODUCTION

In academic and nonclinical research arenas, the phrase Good Laboratory Practice (GLP) specifically refers to a system of management controls that ensures researchers’ tests and operations provide reliable, consistent, and reproducible results under safe conditions. GLP began in New Zealand and Denmark in 1972. It came to the US in 1978 in response to the debacle at Industrial Bio-Test Labs, where pharmaceutical testing results were falsified due to total operations failure, animal cruelty, and lack of proper oversight (1).

In a reputable academic institution it is inconceivable that faculty, staff, and/or students would falsify data or mistreat animals used in research. Still, in the absence of proper training and oversight, many academic laboratory situations can precipitate deleterious results. These include, but are not limited to, breakage of equipment, poor sanitation in facilities, routing and reporting confusion, and, in the worst case scenario, increased health risk. A tenured chemistry professor at the University of Texas at San Antonio was fired on charges that he had threatened the safety of colleagues and students by improperly storing corrosive chemicals in his laboratory and keeping more than 100 boxes of books in his office (2). At the University of California Santa Barbara, a doctoral student ignited a fire that caused approximately $3.5 million in damage. The student was purifying benzene using a reflux/distillation apparatus. The system became over-pressurized, and the student attempted to physically hold the distillation head on the distillation flask. An unknown ignition source ignited the benzene vapor, resulting in an explosion and subsequent fire. The student was seriously burned (3).

Over the past few decades, GLP has been extensively developed with guidelines set forth by the Food and Drug Administration (FDA) in the Code of Federal Regulations (Title 21, Part 58—Good Laboratory Practice For Nonclinical Laboratory Studies) (4). These practices are accepted worldwide to ensure safety, efficiency, and accuracy in laboratory settings (5, 6). GLP is the nonclinical counterpart of Good Manufacturing Practice (GMP) used in industrial settings. Discussed below are suggestions for adapting GLP to academic institutions.

GLP IN SMALL COLLEGES AND UNIVERSITIES

Some GLP methods go beyond the needs of smaller academic and nonclinical laboratories. However, the fundamental principles of GLP can and should be employed throughout departmental operations. These encompass the “who, what, when, how, and where” of operations and reporting. Support for GLP must come from every tier—students to administrators. For nonclinical laboratories that produce or test foods, cosmetics, and drugs, the components of GLP focus on quality assurance (QA) and quality control (QC), as well as the safety of products and/ or the tests that validate their reliability. Academic settings have a different mandate, namely the education, training, and career preparation of students.

A comparison of QA and QC in industry with modifications for education is shown in Table 1. In universities the “product” is the student rather than a commodity, and the “market” is the professional workplace for graduating students. “Quality” in a teaching lab appears as a continuous track record of excellence in training and performance. Department-wide adoption of an SOP facilitates training and provides means to monitor inventory and shared equipment while ensuring the safety of users. Also, the SOP identifies performance outcomes that are validated by instructors (e.g., sterility, formulation accuracy). Immediate performance outcomes are measured in a manner that equates to proactive QA. For example, if the syllabus for a microbiology laboratory course cites “preparation of culture media” as a learning objective, then detailed methodology regarding how to prepare media can be written as an SOP. A formative assessment is performed to ensure that the media can be used. It is understood that all departmental members will follow the protocol associated with this SOP: who can make the media and how; where the media is prepared and stored; how the media is labeled; how the media is discarded; where media preparation logs are kept; and so on.

TABLE 1

Comparison of quality assurance (QA) and quality control (QC) and applications in educational settings.

	Quality Assurance (QA)	Quality Control (QC)	Educational Applications
Definition	A set of activities for ensuring quality in the processes during which products are developed	A set of activities for ensuring quality in products ready for market. The activities focus on identifying defects in the actual products produced.	The “products” are students and the “market” is the professional workplace upon graduation. QA processes are the formative educational practices that address lab skill sets and course objectives. Conversely, QC would reflect summative review of benchmark achievement for learner outcomes.
Focus	Proactive: Aims to prevent defects with a focus on the process used to make a product.	Reactive: Aims to identify and/or correct defects in the finished product.	Proactive QA activities are encompassed by adhering to accepted SOP and GLP and modifying these as needed during the evolution of education. Reactive QC activities address correcting failures to adhere to protocols and policies or to improving practices as part of the evolution of education.
Goal	To improve development and test processes so that defects do not arise when the product is being developed.	To identify defects after a product is developed and before it’s released.	During formative assessment of learner outcomes, educators can identify what skill sets require improvement. Departmental review of GLP supports revision of procedures and policies during summative assessment.
Developing Plans	Establish a good quality management system and the assessment of its adequacy. Periodic conformance audits of the operations of the system.	Find and eliminate sources of quality problems through tools and equipment so that [customer] requirements are continually met.	Many universities offer shared access to handbooks, policies, and procedures on internal networks. QA as applied to GLP seeks to advise and train everyone involved in teaching lab activities; QC identifies gaps in information distribution and application conformity and guides modification and corrective actions.
Methods	Prevent quality-related problems through planned and systematic activities, including documentation.	Apply activities or techniques to achieve and maintain the product quality, process, and service.	QA in educational settings could be outlined in student and mentor handbooks, SOP manuals, and documents addressing GLP. QC follows through the verification procedures that document how GLP is practiced and reviewed.
Responsibility	Everyone on the team involved in developing the product is responsible.	Usually the responsibility of a specific team that tests the product for defects.	360° Oversight during formative and summative assessment at all levels.

Modified from http://www.diffen.com/difference/Quality_Assurance_vs_Quality_Control.

Ultimately, academic assessment of summative outcomes for “preparation of culture media” is measured using tools such as grading rubrics and practical exams, and by the usability of the media itself. These will illuminate any need for modification to instruction. Longitudinal summative assessment at universities, such as student exit exams and departmental reviews, could be likened to reactive QC in industrial settings.

The key to GLP is providing uniform training through structured processes. By adopting a common SOP an academic department can assess whether or not students, lab technicians, teaching assistants, and faculty members consistently perform operations appropriately. When GLP is implemented, the workflow from task performance to assessment of performance supports efficiency in time and budget allocation, as well as academic excellence. As industrial QC is applied post-process, the equivalent educational assessment would entail the summative review of student, staff, and faculty performance noted above. Benchmark indicators may be encompassed by course review, faculty promotion and tenure documents, staff performance appraisals, grades, and exams.

Mechanisms for constructing a modified GLP appropriate to academic settings are depicted in Table 2. QA and validation of product testing is a standard component of industrial GLP but not included in this academic model. However, budgets are included since equipment and instructional supplies are impacted by policies and guidelines unique to educational settings.

TABLE 2

Six components of good laboratory practice (GLP) in small colleges and universities.

Who Is Responsible	What Actions Are Performed	When Are Actions Performed	How Are Actions Performed	Where Are Actions Performed
1. Organization and Personnel
Student Faculty Support Staff Chairperson Dean Administration	Delineation and segregation of duties and responsibilities, handbooks, policies, SOP, and other guidelines	In the course of Instruction Reporting Monitoring Audits Reviews	Oversight structures are clear and guidelines transparent and easily accessible	Throughout the infrastructure, classroom, laboratories, and online
2. Facilities
Trained staff and end users (instructors, students, lab personnel)	Sanitation Maintenance Calibration Repair and Replacement Needs Assessment	On a regularly scheduled basis	According to best GLP; FDA, and other regulatory agencies; and the University	On-site, with records of actions taken archived in a secure location in hard copy or electronically
3. Budget Operations: University and Grant Expenditures
All designated budget authorities, faculty and PIs	Purchasing Reporting Reconciliation Audits Projections	Semester Yearly As needed	According to University policies	According to routing policies
4. Equipment, Reagents, and Materials (Physical, Chemical, Biological); Test Systems with Test and Reference Items
As in 2	As in 2, with consideration for proper disposal of hazardous materials	As in 2	As in 2	As in 2
5. Reporting of Results
	360° Oversight Grades Staff Appraisal Reviews Tenure and Promotion Chairperson and Administration Review	Midterm Semester Yearly	According to University and legislative mandates	Academic Review and Reporting Staff Review and Reporting Infrastructural Review and Reporting by Technicians External Reviews and Audits All reports and records are archived in a secure location in hard copy or electronically
6. Archival
All oversight reporting authorities including staff, supervisors, faculty, administrators and technicians	Reports, Copies of SOP, Certifications, Validations, Corrections, and related documents are stored hardcopy or electronically.	Weekly, monthly, yearly as needed	In hardcopy or electronically with file identifiers as used in SOP	As in 5

Components of the GLP Plan

The primary components of GLP applicable to small colleges and universities are as follows:

Organization and Personnel

Facilities

Budget Operations (University and Grant Expenditures)

Equipment, Reagents, and Materials (Physical, Chemical, and Biological)

Reporting of Results

Archival

The success of GLP and related procedures depends upon full adherence by participants at all levels along with strong administrative support for the processes involved. Additionally, success requires that the core elements of GLP (Table 2) encompass (a) accessible and understandable policies, (b) transparency of operations, (c) segregation of duties, and (d) 360° oversight. Each of these is examined in greater detail below.

1. Organization and Personnel. Common problems faced by smaller academic institutions include lack of resources, lack of funding, and limited staff (7). The morphing of job duties to compensate for these deficiencies, along with employee turnover, can compound these challenges. Assessment of job performance depends upon defining duties and matching expertise to job functions. In the vast majority of academic settings, faculty are subject to the promotion and tenure system of checks and balances; whereas, staff and administrators undergo assessment of job performance as defined by University handbooks. In either case, the candidate for a specific position within the University must have the appropriate training and certifications. For example, a professor of biology is generally neither trained nor certified to calibrate and verify equipment operations (e.g., sterile hood). When jobs are assigned incorrectly within the University, the likelihood for error and mismanagement is high.

Overcoming Issues and Challenges at Small Colleges and Universities

With respect to laboratory operations, job duties and reporting structures must be clear and understandable. When they are not, maintenance and oversight of equipment and operations may be performed incorrectly by individuals who lack expertise or, in the worst case scenario, may not be performed at all. All user groups, from students to professors, must abide by SOP and sanitary SOP (SSOP). This requires training, acceptance, adherence, and understanding of reporting hierarchies. The “who, what, when, how, and where” of equipment use, sanitation, calibration, verification, repair, and removal must be accessible through established SOP, SSOP, and GLP policies.

In the absence of defined guidelines, departmental end users make incorrect assumptions regarding who cleans and maintains equipment, who reports equipment failure, how often maintenance is performed, how toxic waste is removed, what policies apply to common equipment, what policies apply to grant-acquired equipment, how accidents or equipment failures are reported, and other important issues. Failure to understand the legal ramifications pertaining to jobs, duties, and certifications does not preclude an organization from compliance with the law.

2. Facilities. Maintenance and sanitation require established prerequisite practices including SOP and SSOP. At many institutions such services are outsourced to private enterprises whose policies and reporting structures may not interface with those of the University. Regardless of whether or not facility operations are conducted by the University or are outsourced, oversight of laboratory functions is obfuscated when policies are not clear and/or accessible. As discussed above, lack of understanding the “who, what, when, how, and where” of facility use by all user groups leads to system failures. This in turn can precipitate serious safety issues.

3. Budget Operations (University and Grant Expenditures). While this component of the modified GLP shown in Table 2 is not addressed in industrial models, it needs to be included in academic models since laboratory equipment and reagents may fall into different budget designations. Before developing SOP and SSOP, decision trees can help identify governance and responsibilities pertaining to common use items and those belonging to individual labs. In the case of the latter, the principal investigator (PI) could exercise his or her discretion for shared use of grant-acquired equipment. Many universities return a portion of indirect costs to the PI and/or the department for discretionary use. Such monies can be used to purchase equipment. It is the responsibility of the University to establish policies regarding shared equipment acquired through grant awards or use of indirect costs to build infrastructure.

The Code of Federal Regulations (CFR) clearly defines what constitutes “equipment” purchased with grant monies and how such equipment is to be used even after the grant period has ended. There is a tendency for universities to appropriate equipment from grant-funded laboratories and to place such equipment into common use. Faculty and administrators should read the CFR and/or grant guidelines carefully to understand how grant-acquired equipment is used and governed. If equipment is acquired through grant funding, the PI may have the final word in how such equipment is used by faculty, staff, and students regardless of the wishes of administrators. Under some circumstances (e.g., TRIO grants) equipment may, in fact, belong to the United States government. Usually, depreciation reduces the value of the equipment to levels below the cutoff for defining equipment ($5,000) after which time said equipment remains within the institution.

To support GLP budgets, as shown in Table 2, purchase orders, routing, receiving, and use and disposal of equipment and reagents all require clarification. This is particularly important if equipment and reagents are shared by multiple users. Policies that safeguard end-user access help thwart collegial friction. For example, if two cases of pipettes are ordered and designated for a specific class or research activity, under a sound academic GLP plan a non-designated end user could not appropriate these items for a different class or activity. Similarly, SOPs that apply to use of equipment, shared or not, provide instructions regarding who can use the equipment, how the equipment is used, usage logs, and equipment failures. When usage logs are properly employed, shared access is orderly and culpability for breakage and misuse is clear. No one can say, “It broke” or preempt another user by “jumping the queue” for a centrifuge or thermocycler.

4. Equipment, Reagents, and Materials (Physical, Chemical, and Biological). It is essential that members of the department, including students, understand how equipment and reagents are used (SOP), in addition to safety checkpoints, equipment failure reporting, accident reporting, sanitary standard operating procedure (SSOP), and other related laboratory activities. SOP and SSOP are primary components of GLP. An example of an SOP can be found in Appendix 1.

An excellent resource for understanding what information should be included in an SOP can be found in an instructional guide developed by the Michigan Institute for Clinical and Health Research (MICHR) entitled Standard Operating Procedure (SOP) Template User Guide (8). This resource lists and discusses the sections that are included in an SOP: Purpose, Scope, Policy, Definitions, Roles and Responsibilities, Procedure, References, Appendices, Footer Information.

The SOP template adopted by an institution organizes information in a consistent and reproducible manner. Most sections are short with the exception of the Procedure, which is often written in cookbook fashion. A footer denotes the version and author and may include approval routing. Technical and safety information, company contacts, references, and supporting documents can be attached in appendices. Each section is precise. There is no doubt regarding the “who, what, when, how, and where.” As equipment and protocols change, so do the SOP; when modifications are made, the date, the author, and the reason for the modification are noted.

A manual of policies, SOP, and SSOP is developed for GLP and, where appropriate, a hard copy of the manual is placed next to sites of operation. Training for equipment and reagent use in academic settings is the shared responsibility of a designated safety officer, mentor, professor, and/or PI. All students must be fluent in their understanding of GLP, SOP, and SSOP. The safety of operations and longevity of equipment is linked to this understanding. Proper training of students reflects academic excellence and sets the stage for retention, increased enrollment, improved academic performance, and a competitive advantage for graduating students in the professional workplace.

5. Reporting of Results, and 6. Archival. These components of GLP do more than validate the model. In academic institutions, good record keeping is reflective of efficient and sound educational practice. Accreditation, grant acquisition, and placement of students in post-graduation career paths all parallel the guidelines established for GLP. In the same manner that a student’s performance is verified by a transcript, equipment performance is verified by a user log. It is incumbent upon each department to scaffold how and where reporting logs are kept and archived. These records allow new faculty and staff to modify and maintain GLP as the University evolves.

CONCLUSION

Developing and implementing GLP, SOP, and SSOP takes some initial effort. Those who recognize the value of GLP can begin by investigating templates for SOP and spearheading change. One or more department members can begin by taking inventory of equipment and creating SOP handbooks which provide the “who, what, when, how, and where” of procedures and policies. Handbooks and manuals can also include routing and reporting structures. When necessary, these documents should be translated into multiple languages as needed at individual institutions. In the long run, application of these practices vastly improves laboratory efficiency and safety. Understanding that accepted policies apply to everyone in the hierarchy removes the basis for incorrect assumptions. Most importantly, modeling GLP for students provides pre-professional training and fosters collegial respect.

Click here for additional data file.