Using citizen science to bridge taxonomic discovery with education and outreach.

PREMISE OF THE STUDY: Biological collections are uniquely poised to inform the stewardship of life on Earth in a time of cataclysmic biodiversity loss. Efforts to fully leverage collections are impeded by a lack of trained taxonomists and a lack of interest and engagement by the public. We provide a model of a crowd-sourced data collection project that produces quality taxonomic data sets and empowers citizen scientists through real contributions to science. Entitled MicroPlants, the project is a collaboration between taxonomists, citizen science experts, and teachers and students from universities and K-12.
METHODS: We developed an online tool that allows citizen scientists to measure photographs of specimens of a hyper-diverse group of liverworts from a biodiversity hotspot.
RESULTS: Using the MicroPlants online tool, citizen scientists are generating high-quality data, with preliminary analysis indicating non-expert data can be comparable to expert data. DISCUSSION: More than 11,000 users from both the website and kiosk versions have contributed to the data set, which is demonstrably aiding taxonomists working toward establishing conservation priorities within this group. MicroPlants provides opportunities for public participation in authentic science research. The project's educational component helps move youth toward engaging in scientific thinking and has been adopted by several universities into curriculum for both biology and non-biology majors.

The significance of biological collections of museums and academic institutions is well documented (Graham et al., 2004; Berendsohn and Seltmann, 2010) to have made countless contributions to science and to society in general, from environmental monitoring to traditional taxonomy and systematics to public understanding of biodiversity (Suarez and Tsutsui, 2004). Biological collections and their associated data also provide a unique resource for educators to teach core bioscience topics (Ellwood et al., 2015). With specimens in at least 1500 institutions in the United States alone, and in probably close to 5000 institutions worldwide, there are an estimated two billion specimens (Ariño, 2010), almost all with taxonomic, geographic, and temporal data (Page et al., 2015). Yet much of this vast storehouse of information is insufficiently described and thus untapped. In an analysis focusing on flowering plants, for example, Bebber et al. (2010) concluded that herbaria may be reservoirs of undescribed diversity, and thus emphasized the pivotal role of herbarium‐based taxonomic research as well as large‐scale digitization for increasing accessibility to biodiversity data. In recent years, substantial international efforts and resources have been invested into the digitization of natural history collections, with museums and herbaria routinely employing specimen‐level collection databases (Blagoderov et al., 2012). One example of large‐scale digitization—typically encompassing databasing, georeferencing, and imaging—is the National Science Foundation (NSF) program Advancing Digitization of Biodiversity Collections (ADBC), which supports the digitization of the physical specimens as well as specimen‐based data. The ADBC program is providing critical information on existing gaps in our knowledge of life on Earth (Page et al., 2015).

Specimen digitization (i.e., digitally capturing each component of the specimen label and, in some cases, the specimen) is a multi‐step process, and one of the most expensive and time‐consuming of those steps is transcribing the labels into textual formats essential for further description and querying (Hill et al., 2012). Recently, there have been a growing number of web‐based initiatives involving the general public to help overcome this impediment, with participants helping accelerate the process with label and ledger transcription, georeferencing from locality descriptions, and specimen annotation from images (e.g., Hill et al., 2012). Citizen science participants are thereby playing an increasingly important role in transcribing specimen label data (Ellwood et al., 2015). The field of public participation in digitization of biodiversity research specimens has largely been limited to the transcription or interpretation of data associated with the scientific label. However, there is great potential in using crowd‐sourced science coupled with online technology to unlock data and information from digital images of natural history specimens themselves. We know of only a few other projects that utilize crowdsourcing to record or interpret data beyond the scientific label from digitized natural history specimens. These projects known to us, including two using the Notes from Nature platform (https://www.notesfromnature.org/active-expeditions/Labs), focus on capturing phenological data. For example, Willis et al. (2017) reported preliminary results from a crowdsourcing tool that has been developed to collect phenological data from specimens. They demonstrated that phenological data collected from non‐expert users were comparable to those compiled by expert users, suggesting that it has the potential to be a powerful tool for the collection of detailed, accurate phenological data. However, the recording and interpretation of digital images by volunteers has been widely deployed in other scientific disciplines, none more so than in the field of astronomy. Using the popular online citizen science platform Zooniverse (https://www.zooniverse.org/), participants have classified or interpreted millions of images of galaxies, moon craters, stellar light curves, Mars surface images, meteor showers, and more (Lintott et al., 2008; Bonney et al., 2014), and in the process have made discoveries including hundreds of new supernovae candidates (Wright et al., 2017), exotic planets around distant stars (e.g., Schwamb et al., 2013), green pea galaxies (Cardamone et al., 2009), and brown dwarfs (Kuchner et al., 2017).

We hereby outline a novel approach connecting natural history collections to education and outreach through citizen science using a Zooniverse‐based platform. The project focuses on a hyper‐diverse liverwort genus, Frullania Raddi, a taxonomically complex genus with a worldwide distribution and over 2000 published names (von Konrat et al., 2010). It is often difficult to morphologically distinguish between many subgenera (e.g., Frullania subg. Microfrullania (R. M. Schust.) R. M. Schust. and some species within Frullania subg. Frullania), which are the focus of this paper. What began as a project to accelerate taxonomic discovery and bring the scientific practice of taxonomically classifying digitized specimens into the undergraduate classroom rapidly grew into a desire to create a real‐world, hands‐on scientific data analysis process to a broader audience through an ongoing citizen science project. The resulting participatory science project, named MicroPlants, has three interconnected purposes: (1) capture large data sets on a scale not otherwise possible, (2) engage participants of diverse ages and backgrounds in a biologically significant research project, and (3) expose participants to new analytic techniques as well as the traditional scientific method. We test the hypotheses that (1) these data can be used to complement or replace the work of a trained observer and (2) observed differences among lobules (the specialized leaf structures of these liverworts) are taxonomically informative. We also showcase how the MicroPlants online tool has been utilized in formal and informal education of students from middle and high schools to colleges and universities. K–12 teachers and educators also provide insight into the project's potential compatibility with the Next Generation Science Standards (NGSS). We also briefly highlight how the online platform has been developed into touchscreen technology as part of an interactive kiosk in a high‐profile exhibit at The Field Museum. Finally, we analyze and reflect on participants’ motivation and user feedback provided by children, adolescents, and adults.

METHODS

Plant vouchers

A total of 258 herbarium vouchers were used to generate the images used in this study. Most plant material belongs to the herbarium of The Field Museum (F), and a few images come from specimens loaned from the Auckland War Memorial Museum herbarium (AK). The images depicted in this article come from the following vouchers: NEW ZEALAND. North Island, Gisborne District, East Cape, deLange 11498 (F), Hicks Bay, deLange 11607 (F); Tolaga Bay, deLange 11454 (F); Kermadec Region, Raoul Island, Stanley s.n. (AK 294898).

Pilot studies using ImageJ

Pilot studies were conducted at three institutions (Northeastern Illinois University, Wilbur Wright College, and The Field Museum) in 2012 where students measured and catalogued observations from digitally rendered specimens of the liverwort genus Frullania. Pilot studies started on a very small scale with a handful of students before eventually scaling up to include more than 200 students in the pilot phase. A workshop meeting sponsored by the Encyclopedia of Life's Biodiversity Synthesis Center was convened involving students, educators, professors, instructors, and researchers to discuss best practices and improve and refine instructions for future pilots. The pilot proved successful, with both Northeastern Illinois University and Wilbur Wright College continuing to include the project in their curricula in 2013. Students observed and measured a series of morphological characteristics associated with Frullania lobules (specialized inflated leaves), spores, cells, and oil bodies from digital images using ImageJ (Abramoff et al., 2004). ImageJ is open source and freely available software developed by the National Institutes of Health that can display, edit, analyze, process, save, and print images. Measurements were entered into Excel spreadsheets where statistical analysis could be done. We quickly learned that if students were provided with clear instructions, their measurements and observations were comparable to those of the experts. After successful tests, we were confident that students could return potentially valid data and consequently scaled‐up the activity to approximately six classrooms per semester with about 24 students each, totaling 144 students per semester at Northeastern Illinois University alone.

Pilot studies of a web‐based tool

A significant outcome from the pilot phase and follow‐up workshops with K–12 educators, university and college professionals, and public outreach advocates was the recommendation of an online version that would vastly simplify what the project was trying to achieve, as well as reach and engage a broader number of participants, i.e., the general public. In 2013, we partnered with Zooniverse who designed the online tool to only measure lobule dimensions, one of the most informative characters that would also aid systematic research as well as one that had been thoroughly tested. Although moving the tool online eliminated the use of ImageJ, an aspect of the program that many instructors had found useful, it did free up the activity to be used on a larger scale and eased data collection. The prototype tool was trialed by students from Wilbur Wright College, Northeastern Illinois University, and Roosevelt University, as well as the general public at The Field Museum. Early studies revealed the potential for individuals to produce data with validity comparable to that of experts. Testing and refinements led to the current iteration of the website and project called MicroPlants (http://microplants.fieldmuseum.org). Students from Roosevelt University also manually tested data obtained from the online version using ImageJ against the same images.

Led by undergraduate interns, the website was further developed and a variety of materials were produced to support participant understanding, including identification guides, videos, background about the significance of the research, and FAQs (frequently asked questions). Educational modules or add‐ons, developed by undergraduates pursuing a professional teaching career, were built into the activity; these addressed issues such as building phylogenetic trees, making scientific drawings, and learning about the history and ecological importance of these early land plants. The website is also multi‐lingual, currently in Spanish, English, Polish, Portuguese, and Romanian.

Preliminary data cleaning

Raw data for a citizen science measurement consists of four points (four sets of x,y coordinates) corresponding to mouse‐clicks of the ends of user‐drawn lines on an image. The four points indicate the apex and base of the lobule and the left and right side of the lobule at its widest point, thus the four points define two lines corresponding to the length and width of the lobule. The number of measurable lobules in any single image ranges from one to 10 or more. Initial quality control included removal of measurements for which the angle between the long and the short axis was less than 80° (users were instructed to maintain an angle of approximately 90°) and removal of extreme outliers based on the long‐axis length measurement. These primarily included measurements of leaves and other objects in the images that were not lobules. Measurements for each image were then organized into lobules.

Data processing and comparison of citizen science measurements with trained observer measurements

The intersection of the two lines for each measurement was computed and these intersections were grouped into clusters (representing lobules in the image) using model‐based clustering implemented with ‘Mclust’ in the ‘mclust’ R package (Fraley et al., 2012), which was modified with a custom script to prevent over‐splitting of clusters (e.g., for lobules in which there was a broad range of intersection positions). Lobules with fewer than two measurements were removed from the data set. For each lobule, outliers were removed based on the size distribution of long‐axis measurements with a cutoff of two standard deviations from the lobule mean. The variance of the citizen science measures for each lobule was then calculated. This was completed by first assigning all the points for a lobule into four clusters corresponding to the apex, base, and two sides of the lobule using the ‘mclust’ method coerced to identify exactly four clusters. The centroid of each of the four clusters was calculated as the median X and Y value for all the points in the cluster, and then the mean Euclidean distance from each point to the centroid was calculated. The variance score for the lobule was calculated as the mean of these four variances. An arbitrary cutoff for the variance score was used to exclude messy lobules from further analyses. After extensive exploratory analyses, the cutoff was calculated as (M/2) + 13, where M is the number of measurements. For remaining lobules, the four cluster centroids were used to define a consensus of the citizen science measurements. In some cases, certain clusters did not correspond to usable lobules. These included instances where multiple users measured leaves rather than lobules or when a subset of lobules were out of focus in the image. These were identified by detecting outliers and bimodal size distributions among the consensus measurements within an image.

To test whether citizen science measures were different from a trained observer, we generated consensus measurements (as described above) for a subset of lobules that were also measured by a trained observer. We then conducted a two‐tailed paired t‐test with pairs consisting of the consensus and the trained observer measurements for the same lobule. The null hypothesis was that the mean difference between the consensus and the trained observer measurements did not differ from zero.

Testing whether observed differences are taxonomically informative

To test whether we could differentiate Frullania subg. Frullania and Frullania subg. Microfrullania using citizen science measurements, we compared the major and minor axes of 37 Frullania subg. Frullania (5056 replicates) and 42 Frullania subg. Microfrullania accessions (13,893 replicates). First, means of major and minor axes for each accession were calculated, then a two‐tailed t‐test was performed using R (version 3.4.1; R Core Team, 2016). Additionally, to test the differences in allometric relationship (major vs. minor axes) between Frullania subg. Frullania and Frullania subg. Microfrullania, we performed analysis of covariance (ANCOVA). We used the minor axis as a response variable, the major axis as a continuous predictor variable, and subgenera (Frullania subg. Frullania or Frullania subg. Microfrullania) as a fixed predictor variable. Model simplification was performed to find the most parsimonious model.

Interactive touch‐screen version of MicroPlants in a museum exhibit

In March 2017, the web version of MicroPlants was developed into touchscreen technology and formed part of an interactive kiosk in a temporary museum exhibit on biological specimens. We provide some preliminary analysis investigating the efficacy of the tool in an exhibit environment. Data on the exhibit kiosk were validated using similar methods to our MicroPlants web‐based project. Images measured by both participants and experts were selected for comparison. Data were sorted by eliminating all measures with intersections of less than 80° and all measures that did not contain intersecting lines. Data were not sorted by type of participant collector, nor were they sorted with regard to potential quality of participant measures. An aggregated average minor axis (width) and major axis (length) measure was used to conduct an independent two‐tailed Student's t‐test. Individual lobules were not directly compared.

Surveys and feedback

We sought public and student feedback throughout the early iterations of the project to improve the functionality of the tool and website. Approval from The Field Museum's Internal Review Board was sought as members of the general public were asked to complete a survey after participating in the citizen science project. Additional feedback was collected from students taking a botany course at Western Illinois University. A brief analysis is provided.

RESULTS

Summary statistics of web‐based tool

The web‐based citizen science project MicroPlants has been functioning in its present form from July 2013 to the present. An overview of the data is presented in Table 1, including number of images uploaded, number of measurements, number of participants, and the number of participating schools. From timestamps derived from the raw data output, we note that several individuals spent over one hour measuring.

Table 1

Summary statistics associated with the MicroPlants project as of 3 September 2017

Criterion	Summary statistic
Images uploaded	10,000
Images measured	9230
Total no. of measurements	90,098
Individual participants	ca. 8000
Participating schools/universities	11
Maximum length of stay	1 h

Testing data produced by citizen scientists can be used to complement or replace the work of a trained observer

A critical hypothesis was that data generated from the MicroPlants citizen science initiative can be used to complement or replace the work of a trained observer. Analysis employed a subset of the data dating back to 27 October 2014 based on 2296 images representing 407 specimens. Summary statistics are provided in Table 2. Figure 1 depicts the process for each individual image of data clustering derived from the measurements, subsequent consensus analysis of all citizen scientists’ measurements, and comparison with expert measurements.

Table 2

Summary statistics of lobule measurements after initial data cleaning and removal of missing data to test the hypothesis that valid data can be retrieved from citizen scientists

Attribute	No. of measurements
After cleaning out those with angle <80°	38,402
After removing missing data (e.g., measurements with <4 mouse clicks)	38,183
After removing images with a single measurement	37,190
After removing the tutorial image	35,342
After removing outliersa	33,690

After computing mean and variance for measurements of a single lobule, removing poor measurements for just that lobule.

Figure 1

Summary of the data processing. (A) Raw data (black) are assigned to clusters corresponding to lobules. (B) Clusters failing to meet quality criteria (clusters 6–9) are excluded, and a consensus (blue) among the remaining measurements is calculated for each lobule. (C) For a subset, the consensuses are compared to measurements made by a taxonomic expert (red). (D) The difference between citizen science measurements and expert measurements (dashed lines) is typically small relative to the size difference among lobules (distances among all points).

Without even accounting for the quality of images (e.g., out‐of‐focus lobules), in general, citizen science measurements were quite comparable to those undertaken by experts. In an analysis that included only the 130 lobules with high numbers of replicate citizen scientist measurements and an expert measurement, the mean percent difference between the consensus of the citizen scientist measurements and the expert measurement was only 3.68%. Figures 2 and 3 indicate a consensus analysis of three sets of images. The raw data represent citizen scientists regardless of participant type, i.e., without distinguishing between children, adolescents, adults, facilitated participants, online, biology or non‐biology students, etc. When the lobules are clearly in focus, easily interpreted, and typically longer than wide, the consensus analysis is that citizen scientist measurements and the expert measurements are comparable (Fig. 2). However, the shape of the lobule does clearly impact the interpretation of length and width (Fig. 3), rendering many participants’ data unusable. Summary statistics from the consensus analysis are provided in Table 3, including an assessment of the number of well, fairly, or poorly measured lobules.

Figure 2

Raw citizen science data (yellow line), consensus analysis (blue lines), and expert measurements (green dotted lines) with image of stem and lobules superimposed of two Frullania subg. Microfrullania vouchers: de Lange 11498 (A) and de Lange 11454 (B).

Figure 3

Raw citizen science data (yellow lines) and expert measurements (green dotted lines) with image of stem and lobules superimposed of Frullania subg. Frullania sect. Australes (voucher de Lange 11607).

Table 3

Summary statistics for three Frullania images (see, for example, Fig. 2) illustrating the data cleaning process. Lobules are included in downstream analyses only if they were measured multiple times and if the variance among individual measurements is low. Individual lobules are flagged (well, fair, poor) based on the variance among measurements of the individual lobule

Attribute	F. subg. Microfrullania	F. subg. Frullania	F. subg. Microfrullania
Voucher ID	deLange 11498	deLange 11607	deLange 11454
Citizen scientist measurements	93	23	46
Lobules in image	9	9	9
Lobules retained for analysis after cleaning	9	1	8
No. well measured	8	0	8
No. fairly measured	1	0	0
No. poorly measured	0	1	0
Mean quality index	7.75	0	6.23
Mean lobule length (μm)	147.99	NA	149.01
Variance of lobule lengths	21.87	NA	125.87

NA = not applicable.

Testing observed differences among lobule measurements is taxonomically informative

We used a subset of the cleaned data set including 79 accessions and 18,949 measurements representing two Frullania subgenera. We found that the means of the major and minor axis of Frullania subg. Frullania and Frullania subg. Microfrullania are significantly different from each other (Figs. 4, 5 [major axis: t = 16.628, df = 39.984, P < 0.001; minor axis: t = 14.583, df = 40.943, P < 0.001]). Additionally, the relationship between the major and minor axis was similar between Frullania subg. Frullania and Frullania subg. Microfrullania, but Frullania subg. Microfrullania lobules were consistently smaller than Frullania subg. Frullania (Fig. 6, Table 4).

Figure 4

Histogram of the minor axis (width) of Frullania subg. Microfrullania (gray bars) and Frullania subg. Frullania (black bars).

Figure 5

Histogram of the major axis (length) of Frullania subg. Microfrullania (gray bars) and Frullania subg. Frullania (black bars).

Figure 6

Scatterplot of the major axis versus minor axis of lobules for Frullania subg. Microfrullania (gray) and Frullania subg. Frullania (black) with line of best fit. The red dots represent a new, undescribed Frullania species.

Table 4

Results of analysis of covariance (ANCOVA) testing the effects of subgenus and major axis on minor axis

Factors	Sum of squares	df	F	P
Subgenus	471.5	1	18.7901	<0.001
Major axis	43,262	1	130.7428	<0.001
Residuals	25,148	76

df = degrees of freedom; F = F‐test statistic.

Colleges and university application of MicroPlants

Since the development of the measuring tool and the accompanying website, universities and colleges have implemented the project in a variety of courses, reaching to date more than 1200 undergraduate students (Table 5). The tool has been utilized in a broad array of courses, ranging from those with a primary goal of increasing the understanding of the scientific method and the generation of real data for analysis in the classroom to students majoring in the Learning Sciences. MicroPlants remains in the curriculum for several universities, including Roosevelt University where the professors have adopted MicroPlants as a real‐world industrial project where students have been improving work flows, removing invalid data, and trouble‐shooting ongoing aspects of the large data set of almost 90,000 data points, then reporting their findings to the primary investigators.

Table 5

A summary of how the MicroPlants tool has been utilized from 2013 to 2016 in five universities/colleges for which students received course credit or service learning hours

Course name	College/University	Justification	Estimated no. of students
Introductory Biology and Introductory Botany	Wilbur Wright College	Gained an understanding of how scientific research was conducted	720
Essential Skills for Biologists	Northeastern Illinois University	Improve students’ mathematical, computer, and analytical skills	300
Introduction to Natural History Collections and Research	Drexel University	Teaching about the process of species discovery and description based on natural history specimens	100
Introduction to Plant Biology	Western Illinois University	Measuring attributes, participating in “real” biology	75
The Changing Natural Environment	Northeastern Illinois University	An engaging activity to learn about plants and data management	50
Introduction to the Learning Sciences	University of Illinois (Chicago)	Produced assignments reporting functionality and analysis of learning modules	10
Mathematical Industrial Applications	Roosevelt University	Used the data from MicroPlants to form the basis of course projects exploring ways to automate analysis and other problems	20
Seminar in Natural Science	Roosevelt University	Tested the functionality of the pilot version of the online tool as part of an introduction to the methods and analytic framework of the natural sciences	7

K–12 application of MicroPlants

To date, two urban science educators from Illinois as well as The Field Museum constructed and implemented lessons aligned with NGSS and the MicroPlants project. The NGSS are content for science educators ranging from kindergarten through 12th grade. To date, 18 states have adopted and are implementing these standards. Table 6 summarizes the grades that were taught, the lesson, and alignments.

Table 6

Examples of how K–12 teachers/educators aligned the MicroPlants project to current Next Generation Science Standards (NGSS) or variously applied in the classroom

School	Grades	Lesson/Activity	Application in classroom
Chicago public schools	8–12	Plant unit in aquaponics Investigation puzzle in Forensics class	From molecules to organisms: Structures and processes (LS1) Ecosystems: Interactions, energy, and dynamics (LS2) Biological evolution–unity and diversity (LS4)
Elgin Academy	5–6	MicroPlants Plants Project	Discussed biodiversity, importance of changes over time, implications of extinction, students made measurements then read how their contribution was helping scientists
The Field Museum and participating schools	5–6	Activities, broadcast from scientist, MicroPlants	Apply scientific ideas to construct an explanation for real‐world phenomena, examples, or events (MS‐LS4‐2) Evidence of common ancestry and diversity (LS4.A) Patterns (e.g., MS‐LS4‐2) Systems and system models (e.g., MS‐LS1‐3)

Public outreach and engagement

In 2014, participating citizen scientists were solicited for feedback in the form of anonymous surveys as part of The Field Museum's “Meet a Scientist” event, in which scientists interact and engage with the general public. Eleven questions were asked, providing feedback about the tool, website, user motivation, and reactions. A total of 277 participants completed or partially completed a survey regarding the project. Of the 277 participants, 20% (55) were between the ages of 13 and 19. The remaining 80% (222) were over the age of 19. A thorough analysis will be provided in a forthcoming paper. Responses to two questions are provided in Figs. 7 and 9, which illustrate that most users felt they gained a better understanding of the processes in science (Fig. 7) and a large majority had a positive experience using the site (Fig. 8).

Figure 7

The distribution of participant responses, ranking from strongly agree to strongly disagree, about having a better understanding on the processes involved in research.

Figure 9

Scanning electron micrographs illustrating variation in leaf‐lobule shape, size, and position for a selection of Frullania species representing different subgenera (arrows indicating lobule feature). (A) Frullania (subg. Homotropantha) utriculata, (B) Frullania (subg. Frullania sect. Australes) anomala, (C) Frullania (subg. Frullania) pentapleura, (D) Frullania (subg. Microfrullania) microscopica, (E) Frullania (subg. Microfrullania) toropuku, (F) Frullania (subg. Diastaloba) hypoleuca, (G) Frullania (subg. Frullania) setchellii, (H) Frullania (subg. Frullania) probosciphora.

Figure 8

The distribution of how many participants found the MicroPlants project very enjoyable to very unenjoyable.

Kiosk results

The MicroPlants citizen science kiosk was put into operation in March 2017 and in six months had approximately 3752 different participants. We selected one image to test the concept that participant measures could be considered comparable to expert measures. From our preliminary comparisons, after we exclude invalid data, there was no significant difference between expert measures and participant measures (Table 7). However, we can see from the data that participant interaction with the kiosk is very different from their interaction with the online platform and that there are likely ways to improve both the quality of data coming from the kiosk and the user experience. In short, as we learned with many trials of the website, clear and easy instructions are crucial. For example, the image selected for comparison contained seven lobules, but only 31% of participants measured exactly seven lobules and 13% measured more than seven lobules. We are working on a way to clarify these results. Because the setting of the kiosk on the museum floor is so different from that of the website, participant interaction with the tool must be considered. In the exhibit, participant interaction with the tool is potentially limited by time, interest, and other waiting people. We are currently analyzing how these factors might influence both quality and quantity of data generated by the kiosk. We are also currently evaluating how to improve design for accuracy coupled with a good museum experience.

Table 7

Independent two‐tailed t‐test comparison of expert and participant lobule measures (μm) of image “Dna_F063_stem_100x_00119.jpg” on a museum exhibit kiosk

Axis	Participant		Expert		t test
	Mean (μm)	SD	Mean (μm)	SD
Major axis	145.9	54.7	138.3	1.8	1.81a
Minor axis	82.8	40.6	77.8	0.3	1.68a

SD = standard deviation.

P > 0.05, N = 190, t crit = 1.97.

DISCUSSION

Access to digitized natural history collections is increasing at great pace, with over 106 million specimen records and over 22 million media records accessible through the iDigBio portal alone (http://www.idigbio.org). Several successful web‐based initiatives have been developed to use citizen science volunteers to help transcribe label information into textual format—one of the most time‐consuming and expensive aspects of the digitization workflow (Hill et al., 2012). The integration of the internet into everyday life generally has led to a tremendous expansion of the number of citizen science projects that have been remarkably successful in advancing scientific knowledge (Bonney et al., 2009, 2014). With over 1.6 million registered citizen scientists, over 70 active online citizen science projects across the disciplines, and more than 120 research peer‐reviewed publications (https://www.zooniverse.org/publications), Zooniverse is the web's most successful collection of online citizen science projects. To date, the majority of citizen science projects using natural history collections have largely been limited to label transcription activities. Some exceptions include collection of phenological data from specimens using the platform CrowdCurio (http://www.crowdcurio.com) (Willis et al., 2017). With millions of images representing natural history collections, citizen scientists remain largely an untapped resource to unlock potentially useful quantitative and qualitative data from the physical specimens beyond the scientific label data.

Frullania is one of the most species‐rich liverwort genera, with over 500 accepted species (Söderström et al., 2016), and is often easily recognizable in the field by specialists and biologists generally. In the United States alone there are over 33,000 herbarium specimens accessed from the online portal Symbiota (www.bryophyteportal.org). Fortunately, there are suites of taxonomically informative characters that can often be used to distinguish between subgenera or phylogenetic clades, or even species. For example, the leaf‐lobules are remarkably diverse in size and form (Fig. 9), and can often be used to screen specimens to prioritize for further study. This is extremely helpful when faced with the daunting task of examining thousands of collections. Our ongoing studies of the Southern Hemisphere liverwort clade Frullania subg. Microfrullania (Carter et al., 2017), including species delimitation and biogeography, provided the impetus for the MicroPlants project. Investigators developed a series of web‐based tools to aid in the acceleration of taxonomic documentation and syntheses of thousands of collections and associated digital images.

The study as outlined here—the implementation of MicroPlants as an online citizen science tool—clearly indicates that non‐expert users can rapidly measure defined morphological characteristics such as the leaf‐lobule. After significantly reducing the data set of clearly erroneous records (Table 2), our preliminary analysis showed that data generated by non‐experts were comparable to experts. However, that analysis only removed data that were less than 80° (users were instructed to maintain an angle of approximately 90°) and extreme outliers based on the long‐axis length measurement. There may be other elements of the data that we could screen for as well. Analyses are ongoing to evaluate how many individual citizen scientists are required to provide measurements derived from the same image before that image can be retired. Moreover, this minimal number might also differ among different audiences, which is also being investigated. Since analysis of these data, we have also learned that increased accuracy by non‐experts can be obtained by screening the quality and orientation of the images. Some lobule shapes may not lend themselves to accurate measurement by non‐experts, but this consideration needs to be explored further. The comparison illustrated here between Frullania subg. Microfrullania and Frullania subg. Frullania indicates that non‐experts and citizen scientists can generate valid data to rapidly screen between clusters that reflect natural units. This then allows taxonomic experts to focus their limited time and resources on specific taxonomic groups and to prioritize specimen examination. Moreover, data derived from citizen scientists can be used to start building individual specimen profiles that could enhance database‐searching utilities.

We envision a number of extensions that would be seamless in their implementation, and we are exploring the prospect of including other morphological characteristics that participants could accurately observe and measure. Significantly, the online tool has great potential to be deployed across a vast array of organisms and test an endless number of research questions. One project currently being considered focuses on repurposing selected digitized fern and lycopod specimens to analyze polyploidy levels and morphological variation in species complexes as part of an investigation of the evolutionary history of flagellate plants (see http://flagellateplants.group.ufl.edu).

Connecting digitized natural history collections to education and outreach

Digitized and web‐accessible natural history specimens provide an opportunity for educators to promote participatory learning and provide students with a more authentic educational experience (Cook et al., 2014). Participation in authentic science research is an important component for engaging youth in scientific thinking and is a critical strategy for preparing them to enter a modern workforce where STEM plays a central role (National Research Council, 2010). There is a growing body of research documenting the positive impact of research experiences on participants’ career trajectories; for students at the undergraduate level, scholars have found that research experiences are a significant predictor of the extent to which students will sustain their interest in science (Schultz et al., 2011). The MicroPlants citizen science project provides hands‐on practice and reinforcement of observation and measurement skills, accuracy and precision, and plant morphology. There remains great potential in its application (as well as that of similar projects) in a variety of college‐level curricula as well as for K–12 educators, as we have illustrated. The MicroPlants project provides us with an opportunity to investigate the impact of engagement in hands‐on, authentic science during a critical period of transition in a young person's educational life—from secondary to post‐secondary schooling and into the working world.

The exhibit kiosk version of MicroPlants shows potential, but is not yet as efficient an interface as the web‐based tool. However, preliminary analysis indicates that some participants’ measurements are at least comparable to expert data (Table 7), a subject that will be explored more thoroughly in a future paper. Public participation in scientific research using MicroPlants, as shown by the surveys, has the capacity to enhance both public knowledge and understanding of science (education outreach) as outlined by Haywood and Besley (2014).

Exploring motivation and testing between audiences

Our analysis of the data, now almost 90,000 data elements, did not assess the differences among various audience groups. However, we have the capacity to investigate a number of outstanding questions, for example: How many measures must be taken by each kind of user group? Are there significant differences in measurement facility among children, adolescents, and adults? Are there significant differences between a facilitated audience and a purely online audience? Limited work has investigated the arc of engagement from secondary to post‐secondary education and into adulthood. Examining a cross‐sectional population set will allow us to study reasons and motivations of learner engagement moving from a formal to an informal setting. The potential also exists to use this project to explore how authentic research experiences can both develop student interest in science (McGee, 2008) and promote learning of biodiversity concepts (Gunckel et al., 2012).

CONCLUSIONS

The data generated in these authentic experiences are contributing to research on the morphological diversity of a hyperdiverse liverwort genus, Frullania, as well as generating underlying data associated with the individual specimens. The online tool is aiding in accelerating biodiversity discovery and documentation, connecting scientific collections to a broader audience, and encouraging similar activities in other organisms. Citizen scientists could contribute a great deal by carrying out quantitative and qualitative observations and measurements of digitized natural history collections to test research‐driven hypotheses.

AUTHOR CONTRIBUTIONS

M.v.K., T.C., M.G., M.B., L.T., and A.S. co‐conceived intellectual content of project, including early pilot projects. M.v.K. wrote the foundation of the paper with T.C., B.C. (Ben Carter), M.B., J.L., L.T., S.C., E.G., A.Q., E.R., T.L., T.S., T.P., C.Y., J.D., B.A., and A.S. incorporating specialized elements outlined below (text, analysis, education/outreach, K–12 and undergraduate programming). T.C., B.C., T.S., C.V., and S.C. conceived of and designed analyses. M.v.K., T.C., and J.L. co‐managed the project overall. L.T., Br.C. (Brian Carstensen), C.S., and A.S. engineered, designed, and implemented the MicroPlants website. M.G., E.G., M.B., S.C., A.Q., E.R., J.D., and T.L. designed and implemented elements described in the paper as part of outreach, K–12, and undergraduate programs. M.v.K., J.L., P.D.L., and B.A. collected over 2500 plant specimens directly associated with the project described in the paper. B.C., J.L., B.A. processed and assembled sequence data and analyzed the data used as part of the current project to ensure target taxa were investigated. J.L., T.W., C.D., and K.S. conducted early and critical pilot studies, as well as processed thousands of plant specimens, prepared microscopic slides, and provided the thousands of images that were the basis of the project. T.W. also used the data to explore cluster patterns and preliminary investigation of cryptic species. T.P. and C.Y. piloted early iterations, and then engineered, designed, and implemented the touchscreen technology as part of an exhibit and provided access to data. A.Q., Y.R., C.V., S.A., and J.M. conducted dozens of hours of observations, data organization, and compiled and analyzed survey data, as well as participated in public engagement activities.