| Literature DB >> 22383616 |
Julie Libarkin1, Gabriel Ording.
Abstract
We tested the hypothesis that engagement in a few, brief writing assignments in a nonmajors science course can improve student ability to convey critical thought about science. A sample of three papers written by students (n = 30) was coded for presence and accuracy of elements related to scientific writing. Scores for different aspects of scientific writing were significantly correlated, suggesting that students recognized relationships between components of scientific thought. We found that students' ability to write about science topics and state conclusions based on data improved over the course of three writing assignments, while the abilities to state a hypothesis and draw clear connections between human activities and environmental impacts did not improve. Three writing assignments generated significant change in student ability to write scientifically, although our results suggest that three is an insufficient number to generate complete development of scientific writing skills.Entities:
Mesh:
Year: 2012 PMID: 22383616 PMCID: PMC3292072 DOI: 10.1187/cbe.11-07-0058
Source DB: PubMed Journal: CBE Life Sci Educ ISSN: 1931-7913 Impact factor: 3.325
Figure 1.Coding rubric used to evaluate student papers. Students were provided with paper-specific versions of this rubric as part of each assignment.
Validity and reliability considerations applied to this studya
| Criteria | Description | This study |
|---|---|---|
| Content validity | Measure of whether or not items actually measure the trait they are intended to measure. | Components of a scientific paper evaluated here mirror those aspects of scientific writing considered necessary for scientific thinking by faculty with 10+ yr of science teaching experience. |
| Conclusion validity | Measure of ability to determine whether or not a relationship exists between the variables being studied. | We utilized only appropriate statistical measures in our correlation and difference analysis and chose an appropriate |
| Construct validity | Measure of whether or not strong support for the content of items exists. | The raters hired to score student papers agreed that the paper characteristics were appropriate. |
| Communication validity | While researchers often assume that participants will interpret questions as intended, explicitly considering participant interpretation can generate important insights (e.g., | The global scoring rubric was generated by one of the authors based on 10 yr of experience teaching nonmajor biology courses. This rubric is the culmination of years of revisions based on student feedback. |
| Transferability | A measure of the extent to which results can be generalized to populations outside the study. | Research findings are most directly applied to students enrolled in large, general education biology courses at the same university as the subsample. Findings can only be broadly applied to nonscience majors enrolled in biology courses nationwide. |
| Internal consistency reliability | The stability of test results across samples of similar populations, consistency in test results over time, and generation of similar results using slightly different forms all provide evidence that a survey is generating reproducible findings. | We considered each of the coded characteristics to represent an item on a scientific writing scale. For these five items, we calculated a Cronbach's alpha score of 0.78 using the average score for all three raters. This indicates that as a group, these characteristics are generally measuring a common variable. |
| Interrater reliability | Measure of the agreement between different raters analyzing the same data set. | Average intraclass correlation was calculated at 0.89, suggesting good agreement across codes. |
aExcept where noted, concepts of validity, reliability, and trustworthiness are adapted from Lincoln and Guba (1985), Litwin (1995), and Trochim and Donnelly (2007). Table is modeled after Clark and Libarkin (2011).
Example responses from a single student over the three writing assignments
| SCI | DATA | HYP | |
|---|---|---|---|
| Paper 1 | One paragraph introduction | Four references cited, all online sources | “Deforestation causes large breeding grounds for mosquitoes as well as provides cleared areas where the spread of malaria is more common.” |
| Two paragraphs of malaria background | |||
| Three paragraphs about deforestation and malaria | |||
| One paragraph conclusions | |||
| Paper 2 | One paragraph introduction | Four references cited, all online sources | “The negative impacts caused by the gypsy moth are too great to discontinue use of Bacillus thuringiensis.” |
| Three paragraphs of gypsy moth background. | |||
| Three paragraphs about defoliation and impact | |||
| Three paragraphs about | |||
| One paragraph conclusion | |||
| Paper 3 | One paragraph introduction | Seven references cited, including one newspaper article and one journal article | “DDT is a particularly controversial pesticide because it is questioned for its safety by many, while at the same time advocates of the pesticide feel if it is used properly it can be extremely helpful to control malaria.” |
| Two paragraphs of malaria background | |||
| One paragraph about DDT and malaria | |||
| Three paragraphs pros/cons of DDT use | |||
| One paragraph DDT background | |||
| Three paragraphs argument with support | |||
| One paragraph conclusion | |||
| Comments | Use of sources to describe the biological topic and human activities | Use of data from appropriate sources to support position, rather than as background information only, | Explanation of the purpose or hypothesis under discussion |
Figure 2.Average score for the five paper characteristics (SCI, HYP, CONN, DATA, CONC) for papers 1, 2, and 3. Students had the most difficulty stating the hypothesis or purpose of their paper (HYP); this difficulty remained despite practice writing three papers. Student ability to provide a scientific background (SCI) remained fairly stable across the semester, while student ability to report and discuss data (DATA) improved significantly.
Average scores (n = 30) across coders (n = 3) for five paper characteristics and total scorea
| SCI | HYP | CONN | DATA | CONC | SUM | |
|---|---|---|---|---|---|---|
| Paper 1 | 1.73 ± 0.39 | 1.99 ± 0.74 | 2.06 ± 0.53 | 1.73 ± 0.68 | 1.83 ± 0.55 | 9.34 ± 2.28 |
| Paper 2 | 2.02 ± 0.28 | 1.68 ± 0.73 | 2.02 ± 0.44 | 2.09 ± 0.48 | 2.08 ± 0.42 | 9.89 ± 1.67 |
| Paper 3 | 2.04 ± 0.36 | 1.97 ± 0.52 | 2.22 ± 0.48 | 2.22 ± 0.60 | 2.19 ± 0.46 | 10.65 ± 1.81 |
aStandard deviation is 1 sigma.
Correlations and statistical significance between paper variablesa
| HYP | CONN | DATA | CONC | SUM | ||
|---|---|---|---|---|---|---|
| SCI | Correlation coefficient | 0.22* | 0.64** | 0.64** | 0.66** | 0.78** |
| Sigma (two-tailed) | 0.035 | 0.001 | 0.001 | 0.001 | 0.001 | |
| HYP | Correlation coefficient | — | 0.20* | 0.18 | 0.34** | 0.57** |
| Sigma (two-tailed) | — | 0.05 | 0.08 | 0.001 | 0.001 | |
| CONN | Correlation coefficient | — | — | 0.61** | 0.63** | 0.79** |
| Sigma (two-tailed) | — | — | 0.001 | 0.001 | 0.001 | |
| DATA | Correlation coefficient | — | — | — | 0.61** | 0.79** |
| Sigma (two-tailed) | — | — | — | 0.001 | 0.001 | |
| CONC | Correlation coefficient | — | — | — | — | 0.83** |
| Sigma (two-tailed) | — | — | — | — | 0.001 |
*Correlation is significant at p ≤ 0.05.
**Correlation is significant at p ≤ 0.001.
a n = 90.