Effects of problem-based learning modules within blended learning courses in medical statistics - A randomized controlled pilot study.

Problem-based learning (PBL) allows students to learn medical statistics through problem solving experience. The aim of this study was to assess the efficiency of PBL modules implemented in the blended learning courses in medical statistics through knowledge outcomes and student satisfaction. The pilot study was designed as a randomized controlled trial that included 53 medical students who had completed all course activities. The students were randomized in two groups: the group with access to PBL modules within the blended learning course (hPBL group) and the group without access to PBL modules-only blended learning course (BL group). There were no significant differences between the groups concerning socio-demographic characteristics, previous academic success and modality of access to course materials. Students from hPBL group had a significantly higher problem solving score (p = 0.012; effect size 0.69) and the total medical statistics score (p = 0,046; effect size 0.57). Multivariate regression analysis with problem solving as an outcome variable showed that problem solving was associated with being in hPBL group (p = 0.010) and having higher grade point average (p = 0.037). Multivariate regression analysis with the medical statistics score as an outcome variable showed the association between a higher score on medical statistics with access to PBL modules (p = 0.045) and a higher grade point average (p = 0.021). All students in hPBL group (100.0%) considered PBL modules useful for learning medical statistics. PBL modules can be easily implemented in the existing courses within medical statistics using the Moodle platform, they have high applicability and can complement, but not replace other forms of teaching. These modules were shown to be efficient in learning, to be well accepted among students and to be a potential missing link between teaching and learning medical statistics. The authors of this study are planning to create PBL modules for advanced courses in medical statistics and to conduct this study on other universities with a more representative study sample, with the aim to overcome the limitations of the existing study and confirm its results.

Introduction

Medical doctors can be exceptional in their fields even if they do not know medical statistics, but they will be better if they do [1]. The study of Swift et al [2] showed that medical doctors considered medical statistics useful for “accessing clinical guidelines and evidence summaries, explaining risk levels to patients, assessing medical marketing and advertising material, interpreting the results of a screening test, reading research publications for general professional interest, and using research publications to explore non-standard treatment and management options”. Future physicians also thought that there was a need for a practical application of knowledge in medical statistics, not only its’ theoretical basis [3]. Lack of knowledge in medical statistics can lead to misinterpretation of clinical findings [4]. Statistical softwares, widely available now, enable an easy and comfortable analysis, but mistakes can be made when choosing the appropriate statistical test or assumptions for its’ application [5]. Medical students state that learning medical statistics through real life problems and the process of drawing conclusions can be more productive than traditional learning and knowledge assessment [6].

The process of education in medical sciences is most commonly based on traditional classroom lectures (face-to-face, lecture-based). In the past decade, there has been an increasing number of studies aiming to test, improve and introduce other forms of teaching, such as e-learning, blended learning, problem-based learning, team-based learning and flipped classroom [7-9].

A need to modify traditional classroom learning became a focal topic during the COVID-19 pandemic when teaching activities at universities worldwide were forced to shift to different types of online learning. It appears that in the future this challenge will be permanently changing the methods of physicians’ education [10]. The findings of new possibilities to transfer knowledge and skills through online learning modules and its’ constant improvement are receiving almost universal attention. In accordance with this, the implementation of problem-based learning in the online environment has shown similar success among students compared to in-person problem-based learning [11].

Problem-based learning enables students to learn through problem-solving experience [12]. During the learning process students’ main focus is on understanding and solving problems, rather than memorizing facts. Students develop critical thinking and clinical reasoning in concrete medical situation, which is very important for physicians’ daily practice [13]. There are positive experiences of combining problem-based learning (PBL) with lecture-based learning (LBL) and blended learning in education of future health care professionals [14, 15]. A combined model of PBL + LBL was shown to be efficient in increasing the knowledge score, skills score and students’ satisfaction. This hybrid approach in learning has been increasingly used in Chinese medical faculties recently [16].

To the best of our knowledge, there are no previous studies on problem-based blended learning method for teaching medical statistics.

The aim of this study was to evaluate the effectiveness of implemented problem-based modules within blended learning courses in medical statistics through the outcomes of knowledge and student satisfaction. We created problem-based modules in medical statistics, based on actual problems which contained all of the steps in statistical analysis (defining the problem, choosing and applying adequate statistical tests, interpreting the results and drawing conclusions) and implemented them within the blended learning course.

Materials and methods

The study was designed as a randomized controlled trial that included third-year medical students at the Faculty of Medicine, University of Pristina, Kosovska Mitrovica. The final analysis included 53 students who had completed all course activities out of 62 students who had been initially included in the study. Students were randomized in two groups: the group with access to problem-based modules within the blended learning curriculum (hybrid problem-based learning group–hPBL group) and the group with no access to problem-based learning modules–only blended learning course (blended learning group–BL group). The study began on October 1st, 2019, at the beginning of the academic year, and was completed at the end of the academic year (September 30th, 2020). As the research took part during the entire academic year, a part of the study was conducted during the COVID-19 pandemic. Classes in medical statistics and informatics were organized as a blended learning module.

Problem-based modules were conceptualized as an addition to the existing theoretical and practical curriculum in medical statistics within the blended learning course. PBL modules were created based on the technical solution verified by the board of the Ministry of Education, Science and Technological Development of the Republic of Serbia [17].

Blended learning course in medical statistics and informatics is based on a Moodle platform and contains 15 classes of theoretical lectures, 30 classes of practical exercises and 15 classes of other type, such as online readings or seminars. Total of 70% of the program of this course is comprised of medical statistics and this part of the course contains units on data types, descriptive statistics, confidence interval, probability and probability distributions, hypotheses testing, correlation and linear regression. Practical exercises are done using the statistical software Easy R (EZR) [18]. Students from both groups in our study had access to identical course activities (lectures and exercises), except for the access to the problem-based modules that were available only the students from hPBL group (Table 1). During the course, students receive grades for all existing activities: lectures, exercises, colloquium, seminars, solving problems and final test. Students can see all the points for each activity any time during the course. The maximal number of points is 100 (70 for statistics and 30 for informatics). Students need to obtain the minimum of 51 points to pass the course. Based on the total number of points (51–100) the passing grades students can receive vary from 6 to 10.

Table 1

Activities during medical statistics and informatics course.

Timeline	hPBL	BL
Weekly	Lectures	Lectures
Weekly	Practical exercises done using the statistical software	Practical exercises done using the statistical software
Weekly	Independent students’ assignments (interactive online lectures, Moodle)	Independent students’ assignments (interactive online lectures, Moodle)
Weekly	Problem-based learning module (Moodle)	–
During the course	Seminars	Seminars
During the course	Colloquium	Colloquium
At the end of the course	Problem solving	Problem solving
At the end of the course	Final test	Final test

Our study examined only the outcomes of medical statistics (70% of the course): problem solving score (5 problems with maximum score of the total of 25 points) and total medical statistics score (theoretical knowledge score, practical exercises score, problem solving score, independent students’ assignments score, seminars and colloquium; the maximal total medical statistics score was 70 points). The final grade in the course could not be compared because of the score in medical informatics, since the course contains both medical statistics and medical informatics. An anonymous online questionnaire using the five-point Likert scale (1 point- low satisfaction, 5 points- high satisfaction) was used to assess the students’ satisfaction in hPBL group with the PBL modules.

PBL modules were created based on the structure of the steps in statistical analysis (Fig 1). Statistical analysis of the research problem is based on the multiple successive steps which include the following: the definition of the problem and the research question, recognition of the data type, sample type and hypothesis, selection of the adequate statistical test, application of the test, interpretation of the results and conclusion related to the description of the data, statistical conclusion and implications of the results. The PBL modules use guiding questions following the steps of statistical analysis. The guiding questions consisted of: interactive multiple choice or open-ended questions and followed a similar principle to the one Brown et al. applied [19]. Guiding questions changed within each step, based on the type of the statistical problem, number of variables and the sample (examples can be seen online following the link provided in the text below). Students can understand the necessary components for statistical reasoning by answering these questions and learn how to solve the problem.

Fig 1

Basic structure of statistical analysis and the problem-based module.

PBL modules were incorporated in each unit within the blended learning curriculum in Medical statistics and informatics for students in hPBL group. For each unit, students had PBL modules to resolve and to synthesize knowledge from theoretical and practical modules. Each PBL module was created and moderated by the teacher (tutor in classical problem-based learning). All the necessary information was given to students during the lectures, while the exercises and materials were given in the blended learning course. Students can resolve the problems alone or in communication with other students within the group (sending messages on Moodle platform, by asking questions in the forum discussion specially designed for these purposes, or by addressing questions to the online moderator- teacher). Students can use all available materials from the blended learning course, as well as other online materials or books during problem solving. The success in each step is evaluated. After completing all the necessary steps, students receive points and depending on the aim of the module, they also receive correct answers. The system supports the possibility of repeating an attempt of solving the problem until the student achieves the minimal necessary knowledge level, or desired knowledge level. PBL module contains meta-cognitive characteristics such as planning, managing and application of the previously adopted knowledge.

Examples of PBL modules used in this study are based on the course Problem based modules in medical statistics and can be accessed via the following link: http://e-ucenje.med.pr.ac.rs/course/view.php?id=232 (username: user; password: user).

Ethical statement

The study was approved by the Ethical Committee of the Faculty of Medicine, University of Pristina, Kosovska Mitrovica (No. 09–3171). During the first week of the course (the first week of the semester), before the randomization in the groups was performed, students received written and oral explanations of the study, the processes and aims, the modalities of data gathering and data analysis. Students were explained that all the information gathered would be anonymous, that the participation was voluntary and that they could dropout of the study at any point. After this, the students gave an oral consent for their participation in the study, which was then verified in their records. A questionnaire on satisfaction was filled in after the course had been completed, as an online anonymous and non-obligatory questionnaire. All the data on the course outcomes and the data from the questionnaire on the students’ satisfaction were gathered by the administrator of the Moodle platform who was the only one with the access to the complete database. The authors of the study only had access to the anonymized database that is provided with the manuscript.

Statistical analysis

Based on the variable types and normality of distribution, description of the data was shown as number (n) and percentage (%), mean±standard deviation or median (range, minimum- maximum). T-test, Mann-Whitney test, Chi-square test or Fisher’s exact test were used to test the hypotheses. The effect size in t-test was examined with Cohen’s d. Linear regression was used to analyze the learning outcome (problem solving score and total medical statistics score) and its potential predictors. All the variables which were significant in the univariate models at the level of 0.05 were entered in the multivariate regression analyses. Statistical hypotheses were tested at the significance level (alpha) of 0.05.

Results

There were no statistically significant differences between the students from the hPBL and BL group concerning socio-demographic characteristics, grade point average and the modality of access to the materials within the blended learning course (Table 2).

Table 2

Characteristics of students included in the research.

Characteristics of students	Total	hPBL	BL	p-value
	(n = 53)	(n = 26)	(n = 27)
Age (in years), mean ± sd	21.4±0.9	21.4±1.0	21.4±0.9	0.934
Sex, n (%)
male	16 (30.2%)	8 (30.8%)	8 (29.6%)	0.928
female	37 (69.8%)	18 (69.2%)	19 (70.4%)
Grade point average, mean ± sd	7.7±0.7	7.7±0.7	7.8±0.8	0.611
The grade expected, n (%)
8	4 (8.7%)	2 (8.7%)	2 (8.7%)	0.787
9	13 (28.3%)	6 (26.1%)	7 (30.4%)
10	29 (63.0%)	15 (65.2%)	14 (60.9%)
Successfully completed the course, n (%)
Before COVID-19 pandemic	22 (41.5%)	13 (50.0%)	9 (33.3%)	0.218
During the COVID-19 pandemic	31 (58.5%)	13 (50.0%)	18 (66.7%)
Access to fast internet, n (%)	46 (100.0%)	23 (100.0%)	23 (100.0%)	1.000
Had any online module previously, n (%)	8 (17.4%)	3 (13.0%)	5 (21.7%)	0.699
Most common time of access to course materials, n (%)
When at faculty or from home	10 (21.7%)	4 (17.4%)	6 (26.1%)	0.475
Both when at faculty and from home	36 (78.3%)	19 (82.6%)	17 (73.9%)
The most commonly used device for access to the course materials, n (%)
PC or laptop	40 (87.0%)	20 (87.0%)	20 (87.0%)	1.000
Tablet or smartphone	6 (13.0%)	3 (13.0%)	3 (13.0%)
Self-rated computer skills (1- very poor, 5- very good), median (range)	4 (1–5)	4 (3–5)	4 (1–5)	0.227

Students in hPBL group had a significantly higher problem solving score (p = 0.012, effect size 0.69) and total medical statistics score (p = 0.046, effect size 0.57) (Table 3). Students in hPBL group had a significantly higher score on total satisfaction with the course (median 5, range 4–5) compared to the students in BL group (median 5, range 3–5), (p = 0.012).

Table 3

Outcomes among two groups of students.

Outcomes	Total	hPBL	BL	p-value
	(n = 53)	(n = 26)	(n = 27)
Problem solving score, mean ± sd	21.8±2.1	22.5±1.7	21.1±2.3	0.012
(range)	(16.1–25.0)	(18.7–25.0)	(16.1–24.7)
Total medical statistics score, mean ± sd	62.3±4.7	63.6±3.8	61.0±5.2	0.046
(range)	(52.6–68.9)	(55.5–68.9)	(52.6–68.9)

Multivariate regression analysis with problem solving as an outcome variable showed that problem solving was associated with being in hPBL group (p = 0.010) and having higher grade point average (p = 0.037) (Table 4 and Fig 2).

Table 4

Regression models with problem solving score as an outcome variable.

Variable	Univariate linear regression		Multivariate linear regression
	b	p	b	p
hPBL vs BL	1.434	0.012	1.455	0.010
Age	-0.230	0.473
Sex	-0.353	0.583
Grade point average	1.045	0.009	0.809	0.037
The grade expected	0.680	0.156
Successfully completed the course during vs before the COVID-19 pandemic	-0.896	0.131
Had any online module previously	-0.682	0.410
Most common time of access to course materials	1.181	0.117
The most commonly used device for access to the course materials	-2.078	0.022	-1.589	0.062
Self-rated computer skills	0.332	0.364

Fig 2

The relationship between the problem solving score and factors associated with it in multivariate regression model.

hPBL–hybrid Problem Based Learning, BL–Blended Learning.

Multivariate regression analysis with total medical statistics score as an outcome variable showed that the total medical statistics score was associated with hPBL group (p = 0.045) and higher grade point average (p = 0.021) (Table 5 and Fig 3).

Table 5

Regression models with total medical statistics score as an outcome variable.

Variable	Univariate linear regression		Multivariate linear regression
	b	p	b	p
hPBL vs BL	2.567	0.047	2.431	0.045
Age	-1.018	0.150
Sex	0.603	0.674
Grade point average	2.678	0.002	2.006	0.021
The grade expected	2.643	0.010	1.849	0.059
Successfully completed the course during vs the COVID-19 pandemic	-0.216	0.871
Had any online module previously	-2.642	0.141
Most common time of access to course materials	1.198	0.472
The most commonly used device for access to the course materials	-3.298	0.102
Self-rated computer skills	1.216	0.125

Fig 3

The relationship between the total medical statistics score and factors associated with it in multivariate linear regression model.

hPBL–hybrid Problem Based Learning, BL–Blended Learning.

All the students in hPBL group (100.0%) thought that the PBL modules helped them to achieve the desirable knowledge in medical statistics. On the five-point Likert scale (1 –the lowest satisfaction, 5 –the highest satisfaction) median grade for adequacy of the modules, modalities of solving problems, the assistance received in the process of learning medical statistics, the students graded problem solving modules with the median grade 5 (range 4–5). Median on the interest in problem solving modules was 4.5 (range 3–5) (Table 6).

Table 6

Students’ attitudes towards the implemented problem solving modules.

Question	median (range)	n (%) highest satisfaction
Problems are adequate for learning medical statistics	5 (3–5)	19 (73.1%)
Contents and structure of problem-based modules is interesting	4.5 (3–5)	13 (50.0%)
I like this modality of solving the actual statistical problems	5 (3–5)	19 (73.1%)
Step- by- step approach within the module is useful for learning medical statistics	5 (3–5)	20 (76.9%)
Problem based modules helped me to learn the steps for resolving actual statistical problems	5 (3–5)	21 (80.8%)
Problem-based modules helped me understand medical statistics	5 (3–5)	20 (76.9%)

Discussion

In this study, it has been shown for the first time that the implementation of problem-based learning into blended learning course in undergraduate medical studies contributes to better learning outcomes in medical statistics. The results of this research indicate that students of the hPBL group had a significantly higher problem solving scores and total medical statistics scores. The presented PBL modules enable active learning of medical statistics by solving actual statistical problems, through conceptual understanding, and they direct students to logical thinking. This direction is in line with recommendations from the Guidelines for Assessment and Instruction in Statistics Education (GAISE) Reports published by the American Statistical Association (ASA) [20]. Also, our results are consistent with the results of a systematic review that relates to the benefits of problem-based learning over traditional learning in biomedical education [21]. Like in our study, medical students had better solving scores and satisfaction [21]. The results of another meta-analysis showed that combined PBL and LBL learning of clinical medicine was significantly superior in achieving higher knowledge and skills scores, as well as learning satisfaction compared to LBL alone [22]. This meta-analysis included studies from China, and its’ authors suggested hybrid PBL to be gradually introduced into clinical medical teaching programs [22] since the results from previous meta analysis in China had shown that the competencies of students in medical statistics were insufficient and that they did not have sufficient ability to practically apply their statistical knowledge [23].

Teacher in medical statistics acted as a tutor in our study, he/she was obligated to answer students’ questions and to evaluate the outcomes of the problem based learning. In the study of Woltering et al [24] blended problem-based learning included the e-learning module complementary to the classic PBL modules, but without the inclusion of tutors. This study showed an increase in students’ motivation, subjective gains in knowledge and overall satisfaction among students in blended program-based learning. There was no significant difference in the successfulness of problem solving, which confirms that the PBL can be successfully implemented in online learning environments. Additionally, in the study which compared traditional classroom classes and online asynchronous PBL, de Jong et al [25] found that the absence of a formal tutor can force students to rely on themselves and teamwork, develop critical thinking, analytical and self-regulation skills.

The results of our study suggest that the modality used to access the course is a significant predictor of the problem solving score (if it is via PC/laptop or Tablet/Smartphone). Students who accessed the course via PC/laptop had better scores, which was expected because the problems are solved in the EZR software via PC, and the access via smartphone or tablet does not allow access to this feature. On the other hand, access to theoretical materials may be more comfortable/practical via tablets. However, the method of access is not a statistically significant predictor of total medical statistics score.

Self-rated knowledge through the expected final grade was significantly associated with the total medical statistics score in univariate regression model, which was, most likely, influenced by multiple accesses to PBL modules and students’ motivation. The motivation was not directly assessed in our study, but was assessed indirectly, through the expected final grade, as there is a proven positive association between the intrinsic motivation and perceived academic rank [26]. It is expected that highly motivated students would use the advantages of an online platform more and would have higher scores in solving actual problems and higher total medical statistics score.

Multivariate regression models showed a significant association between the problem solving score and total medical statistics score with hPBL group and higher grade point average. We expected that students with a higher grade point average have higher problem solving as well as the total medical statistics score, regardless of the study group. Although we did not have a large sample and we did not find a great difference between the means of the two groups, the effect size was moderate. However, the implemented PBL modules can be adapted to teaching in medical statistics, as they offer a modern method of solving practical problems that is appealing to students. Such an approach of directing them through a problem, from its’ recognition to the analysis of results, helps students to better understand medical statistics and can help them later during their scientific research work. Depending on the curriculum and concept of teaching medical statistics, PBL can be used as an addition to theoretical and/or practical classes. PBL can be easily implemented in different teaching models (classic, online, blended), which in our case proved to be extremely useful during the COVID-19 pandemic. PBL modules can be used not only during classes for learning or updating materials, but also for practicing assignments and self-assessment of knowledge by students. The PBL is completely independent from the modality of calculation (classical or software) or on platform (can be solved in a classroom, on paper, or using an online study platform). The optimal application of PBL is within online courses because they allow students to access materials when it suits them, as often as they need to (self-regulated learning). Educational reform that combined Moodle with the traditional way of learning medical statistics has achieved good results among students [27]. The meta-analysis showed that the blended learning methods were more efficient than traditional learning in medical sciences [28]. Also, PBL can be used for assessment of knowledge on the exam itself. This is supported by the results from our study, which showed that the time of taking the exam (before or during the COVID-19 pandemic) was not a statistically significant predictor of problem solving score and total medical statistics score. During the COVID-19 pandemic, there were also changes in testing methods [29] and the possibility of introducing the open-book examination [30]. The applied PBL modules can also be used for online assessment of problem solving from medical statistics with the open book model, since in a limited time linking of all information needed for problem solving cannot be compensated by searching on the Internet, books or any study materials.

The potential for the application of these modules is high, in the context of the current level of presence of e-learning methods in medical sciences and especially, due to the increase in digitalization during the COVID-19 outbreak. The PBL can be used for learning, repetition, exercise, and self-assessment, and finally, for the assessment in the exam. This increases its usefulness threefold.

The results from the meta- analysis showed that the courses which implemented the PBL were associated with long term knowledge retention, short term retention and application of clinical skills and thinking. Traditional approach is more convenient for short term knowledge retention which does not require further understanding [31]. We did not examine the knowledge retention, and it could be included in the further research.

Limitations

This study has a few possible limitations. Firstly, the study was conducted in only one educational institution, during one study course, with a small number of participants and this should be taken into account when generalizing the results in other courses, study programs or universities. Secondly, the PBL modules include only modules in the basic course of medical statistics. Although this concept and its technical solution can be applied to higher levels of education, there is a need for the assessment of its efficiency, which is our aim for further studies. Another potential limitation of the study we could not control was the fact that the students from the hPBL group could show the modules to the students in BL group. This risk can be minimized through the inclusion of a larger number of participants in the study and through the development of a larger number of the problems for students to solve. The enthusiasm of students and their reaction to the new type of learning, especially during the COVID-19 pandemic, can also affect higher grades among the hPBL group.

Future research

The authors of this study are planning to create PBL modules for advanced courses in medical statistics and to conduct the study on other universities with a more representative study sample, with the aim to overcome the limitations of the existing study and confirm its results.

Conclusion

The presented PBL modules can be easily implemented in the existing courses of the medical statistics developed on the Moodle platform, have high applicability and can complement, but not replace other forms of teaching. The PBL modules allow students to associate theory and practice, synthesize the existing knowledge and generate new knowledge and show how medical statistics can be thought through conceptual understanding via directing students through problems using the step-by-step approach. The problem-based modules were shown to be efficient in acquiring knowledge, are well accepted among students and can be a missing link in learning and understanding medical statistics.

(ZIP)

Click here for additional data file.

26 Aug 2021

PONE-D-21-22424

Effects of problem-based learning modules within blended learning courses in medical statistics - randomized controlled pilot study

PLOS ONE

Dear Dr. Bukumirić,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

While both reviewers recognized the value and contribution of your study, both noted significant issues with the manuscript, in particular around reporting of study's methodology. Take a detailed look at the provided comments, I hope you find the feedback useful and reviewers' comments constructive. I'm looking forward to reading the revised manuscript.

Please submit your revised manuscript by Oct 10 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see:  http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols . Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at  https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols .

We look forward to receiving your revised manuscript.

Kind regards,

Vitomir Kovanović, Ph.D.

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. We note that you have indicated that data from this study are available upon request. PLOS only allows data to be available upon request if there are legal or ethical restrictions on sharing data publicly. For information on unacceptable data access restrictions, please see http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions.

In your revised cover letter, please address the following prompts:

a) If there are ethical or legal restrictions on sharing a de-identified data set, please explain them in detail (e.g., data contain potentially identifying or sensitive patient information) and who has imposed them (e.g., an ethics committee). Please also provide contact information for a data access committee, ethics committee, or other institutional body to which data requests may be sent.

b) If there are no restrictions, please upload the minimal anonymized data set necessary to replicate your study findings as either Supporting Information files or to a stable, public repository and provide us with the relevant URLs, DOIs, or accession numbers. Please see http://www.bmj.com/content/340/bmj.c181.long for guidelines on how to de-identify and prepare clinical data for publication. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories.

We will update your Data Availability statement on your behalf to reflect the information you provide.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #3: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #3: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #3: No

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #3: No

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Thank you for the opportunity to review your manuscript. I have provided some potential suggestions for edits. Of note, it may be helpful to work with someone regarding the syntax and diction--there were several grammatical errors and challenges with the flow that could make the manuscript more impactful.

INTRO

- Provide a nice summary of applicable research and outline specifically the application to medical statistics. Would suggest to explore if there are publications on medical statistics / epidemiology teaching practices broadly in the literature. It may be helpful to put this within the context of other work there, not necessarily just PBL related.

- Several grammatical errors (e.g., "there were no studies on problem-based blended learning method for teaching the medical statistics)

METHODS

- Love that this was an RCT, we don't get to see that often! The main question is about the fidelity of the work--how did you prevent students from sharing information or resources?

- I don't understand the statement "the students passed the exam in medical statistics during the study". Is that more of a result?

- Can you provide more information about the context of the course this activity was embedded within? Also to double-check, there are normally 53 students in the program correct? Or was this the number who consented to participate.

- May be helpful to reference Figure 1 earlier in the methods when discussing the steps that the students went through.

- A summary table of the assessment activities may also be useful. Could you also provide a sample, maybe in an appendix to know what these activities looked like, time dedicated to them, etc.

- I have difficulty following how there was significant difference between the two groups or how they were sufficiently separated for the purpose of the study. Can you please provide more details about how their experiences differed based on the random assignment.

- It is not clear what assessment activities are included and how the relates to the linear regression. More details are needed. How were the problem solving and medical statistics score measured? What are the total possible scores? Have they been used in other settings that showcase how well they measure those two abilities? Etc. Since these are essential for interpreting the results more details are needed about the reliability and validity of these results.

RESULTS

- Can you please clarify "the grade expected" since these are number values; for international audience this may be useful if they are used to letters, etc.

- "The exam was passed" in the table is also confusing. I'm not sure what this means, the number of people who passed? What qualifies as passing the exam and when is the exam administered?

- Do you have the correlation between the problem solving and medical statistics scores, it would be curious to see the extent of their relationship to one another.

- For Likert scales, it's generally inappropriate to summarize at average/median levels. Instead, it is preferred to illustrate the number of individuals who responded with each choice.

DISCUSSION

- More comparison of statistical versus "clinical" significance may be necessary. The mean scores differed by 1-2 points, is that truly significant beyond statistically?

- Unfortunately, more details are needed about the methodology to evaluate the discussion appropriately.

Reviewer #3: Dear authors,

Congratulations on the work developed and submitted to this journal, on the effects of problem-based learning (PBL) modules within blended learning (BL) courses in medical statistics, by running a randomized controlled trial. Your aim was to assess the efficacy of PBL modules implemented within the blended-learning courses in medical statistics through the knowledge outcomes and student satisfaction.

Considering the above mentioned as a head start for the review, below you will find some comments on each section that should be addressed.

- Abstract Section:

Whilst being overall well written, this particular section should be restructured in order to be more appealing for the reader. There is an excess of abbreviations which lead to a sense of confusion. Additionally, it should mention briefly the future steps of this project.

- Introduction Section:

This particular section is underexplored. Some questions that should guide your thoughts while adding some important information are the reason why PBL methods can be important on the subject of medical statistics and the particular subset of skills that a medical student is supposed to develop from this curricula. Additionally, once you set out to explore knowledge outcomes and student satisfaction, there are previously conducted studies that explore these variables with future medical skills, so what is really the importance of student satisfaction on continuing learning profiles, e.g.? The last paragraph should be fully restructured, as it is one of the most important ones for the reader, since it is the last one before the next Materials and Methods section. I would advise to begin with the aim of the study and only then advance to the design made. Also, some english grammar errors were found, which are described below.

Line 37 - Where written "In the past decade, there are increasing (...)" change for "In the past decade, there is an increasing (...)"

Line 49 - Where written "During learning process the students (...)" change for "During the learning process, students (...)"

Line 50 - Where written "They" change for "The students" or "Students"

Paragraph 57 - Where written "To the best of our knowledge, there were no studies on problem-based blended learning method for teaching the medical statistics." change for "To the best of our knowledge, there are no previous studies on the problem-based blended learning method for medical statistics teaching."

- Materials and Methods Section:

First and foremost, once being a randomized controlled trial, it would be optimal to have it registered at the clinicaltrials.gov platform. Further on, the authors mention that an informed consent was obtained, however there is no reference on how. It is my opinion that these paragraphs 73/74/75 should add the environment in which the consent was obtained, to avoid any coercion interpretations. Additionally, you mention an anonymous online questionnaire for the assessment of the student satisfaction of the hPBL group. Was this preceded by a mandatory informed consent? How was the data managed and stored? Who had access to it? Were an IP addresses collected?

Overall, this section needs more work, mainly at the description of the method itself. I would a recomend a major revision of the english used, as well as the review of the duplicated information. Some sentences are in reality avoidable, because they are the continuation of the last.

Line 76 - Where written "The study began on the 1st of October, 2019 (...)" change for "The study began on October 1st, 2019 (...)

Line 77 - Where written "was completed at the end of school year (30.09.2020)" change for "was completed at the end of school year (September 30th, 2020)."

Lines 84/85 - Where written "Ministry of education, science and technological development of the Republic of Serbia" change for "Ministry of Education, Science and Technological Development of the Republic of Serbia"

Paragraph 86 - Should be rewritten, since there is repeated information.

Lines 106/107 - Rewrite the following sentence "All the necessary information is given to students during the lectures and exercises in the materials in the blended- learning course.". Suggestion: "All the necessary information was given to students during the lectures, while the exercises and materials where given at the blended-learning course."

Lines 126/127/128 - Where written "Anonymous online questionnaire with the five-point Likert scale (1 point- low satisfaction, 5 points- high satisfaction) was used to assess the satisfaction of the students in the hPBL group with the PBL modules." change for "An anonymous online questionnaire with the five-point Likert scale (1 point- low satisfaction, 5 points- high satisfaction) was used to assess the satisfaction of the students in the hPBL group with the PBL modules.".

- Results Section:

This section is well presented and clear. I would suggest that in line 139, where written "There were no differences between (...)", the authors change for "There were no statistically significant differences between (...)".

- Discussion Section:

The discussion section is well explored and written.

- Conclusions Section:

The conclusions are both supported by the results obtained in the present study, as well as previous studies.

This article has some flaws as pointed above, for each section. A general review of the english grammar and spell check should be performed, additionally to addressing the point mentioned.

Best regards,

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #3: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

4 Oct 2021

Response to reviewer 1:

Dear Madame/Sir,

Thank you for your time and assistance. We appreciate very much your professional advice and accepted comments and suggestions in the text.

We accepted all suggestions in Abstract, Introduction, Methods, Results, and Discussion section.

Also, we added the recommended references.

Comments to the authors:

Thank you for the opportunity to review your manuscript. I have provided some potential suggestions for edits. Of note, it may be helpful to work with someone regarding the syntax and diction--there were several grammatical errors and challenges with the flow that could make the manuscript more impactful.

INTRO

- Provide a nice summary of applicable research and outline specifically the application to medical statistics. Would suggest to explore if there are publications on medical statistics / epidemiology teaching practices broadly in the literature. It may be helpful to put this within the context of other work there, not necessarily just PBL related.

Thank you for your comment. We added the paragraph on the existing teaching practices in the introduction section.

‘Medical doctors can be exceptional in their fields even if they do not know medical statistics, but they will be better if they do [1]. The study of Swift et al [2] showed that medical doctors considered medical statistics useful for “accessing clinical guidelines and evidence summaries, explaining risk levels to patients, assessing medical marketing and advertising material, interpreting the results of a screening test, reading research publications for general professional interest, and using research publications to explore non-standard treatment and management options.” Future physicians also thought that there was a need for a practical application of knowledge in medical statistics, not only its’ theoretical basis [3]. Lack of knowledge in medical statistics can lead to misinterpretation of clinical findings [4]. Statistical softwares, widely available now, enable an easy and comfortable analysis, but mistakes can be made when choosing the appropriate statistical test or assumptions for its' application [5]. Medical students state that learning medical statistics through real life problems and the process of drawing conclusions can be more productive than traditional learning and knowledge assessment [6].'

1. Trajkovic G. “Introduction to medical statistics”. Lectures. Faculty of Medicine, University of Pristina in Kosovska Mitrovica. 2015.

2. Swift L, Miles S, Price GM, Shepstone L, Leinster SJ. Do doctors need statistics? Doctors’ use of and attitudes to probability and statistics. Stat Med. 2009;28: 1969–1981. doi:10.1002/sim.3608

3. MacDougall M, Cameron HS, Maxwell SRJ. Medical graduate views on statistical learning needs for clinical practice: A comprehensive survey. BMC Med Educ. 2019;20. doi:10.1186/s12909-019-1842-1

4. Windish DM, Huot SJ, Green ML. Medicine residents’ understanding of the biostatistics and results in the medical literature. J Am Med Assoc. 2007;298: 1010–1022. doi:10.1001/jama.298.9.1010

5. Masic I, Jankovic SM, Begic E. PhD Students and the Most Frequent Mistakes during Data Interpretation by Statistical Analysis Software. Studies in Health Technology and Informatics. IOS Press; 2019. pp. 105–109. doi:10.3233/SHTI190028

6. Astin J, Jenkins T, Moore L. Medical students’ perspective on the teaching of medical statistics in the undergraduate medical curriculum. Stat Med. 2002;21: 1003–1006. doi:10.1002/sim.1132

- Several grammatical errors (e.g., "there were no studies on problem-based blended learning method for teaching the medical statistics)

Thank you for your comment. We consulted the native English speaker to check the grammar and spelling after the revision.

METHODS

- Love that this was an RCT, we don't get to see that often! The main question is about the fidelity of the work--how did you prevent students from sharing information or resources?

Thank you for your comment. The administrator of the Moodle platform checked if the students assessed the PBLs from the same IP address, which was true for our students. We could not control if the students from PBL group shared the information on modules with the students from BL group. We expect that there were no such cases, or were only a few and that these cases did not affect the differences in scores. This was a pilot study and its’ results will be used in further research. We added this in the limitation section:

‘Another potential limitation of the study, we could not control for, was that the students from the hPBL group could show the modules to the students in BL group. This risk can be minimized through the inclusion of the larger number of participants in the study and through the development of a larger number of the problems for students to solve.’

- I don't understand the statement "the students passed the exam in medical statistics during the study". Is that more of a result?

Thank you for your comment. Our study lasted for 12 months, during which it is expected from all the students enrolled in any courses to pass the exams. The students pass the exam if they complete all the necessary course activities and if they obtain 51 or more points (out of the 100). We deleted this sentence, and provided the detailed explanation in the Materials and Methods section.

- Can you provide more information about the context of the course this activity was embedded within?

Thank you for your comment. We added the description in the Materials and methods section.

‘Blended learning course in medical statistics and informatics is based on a Moodle platform and contains 15 classes of theoretical lectures, 30 classes of practical exercises and 15 classes of other type, such as online readings or seminars. Total of 70% of the program of this course is comprised of medical statistics and this part of the course contains units on data types, descriptive statistics, confidence interval, probability and probability distributions, hypotheses testing, correlation and linear regression. Practical exercises are done using the statistical software Easy R (EZR) [18]. Students from both groups in our study had access to identical course activities (lectures and exercises), except for the access to the problem-based modules that were available only the students from hPBL group (Table 1). During the course, students receive grades for all existing activities: lectures, exercises, colloquium, seminars, solving problems and final test. Students can see all the points for each activity any time during the course. The maximal number of points is 100 (70 for statistics and 30 for informatics). Students need to obtain the minimum of 51 points to pass the course. Based on the total number of points (51-100) the passing grades students can receive vary from 6 to 10.

Table 1 – Activities during Medical statistics and informatics course

Timeline hPBL BL

Weekly Lectures Lectures

Weekly Practical exercises done using the statistical software Practical exercises done using the statistical software

Weekly Independent students’ assignments (interactive online lectures, Moodle) Independent students’ assignments (interactive online lectures, Moodle)

Weekly Problem-based learning module (Moodle) –

During the course Seminars Seminars

During the course Colloquium Colloquium

At the end of the course Problem solving Problem solving

At the end of the course Final test Final test

Our study examined only the outcomes of medical statistics (70% of the course): problem solving score (5 problems with maximum score of the total of 25 points) and total medical statistics score (theoretical knowledge score, practical exercises score, problem solving score, independent students’ assignments score, seminars and colloquium; the maximal total medical statistics score was 70 points).‘

Also to double-check, there are normally 53 students in the program correct? Or was this the number who consented to participate.

Thank you for your comment. There were total of 62 students in the third year of medical school and all of them gave the consent for participation in the study (30 hPBL group and 32 in BL group), but the 53 students completed all of the course activities and were included in the analysis. We considered this to be acceptable, without any influence on the results, as the drop- out from the study was less than 15%, considered acceptable for the longitudinal studies (https://www.cebm.ox.ac.uk/resources/levels-of-evidence/oxford-centre-for-evidence-based-medicine-levels-of-evidence-march-2009).

We clarified this in the abstract and in the Material and methods section:

Abstract

‘The pilot study was designed as a randomized controlled trial that included 53 medical students who completed all course activities.’

Material and methods

‘The final analysis included 53 students who had completed all course activities out of 62 students who had been initially included in the study.’

- May be helpful to reference Figure 1 earlier in the methods when discussing the steps that the students went through.

Thank you for noticing. We moved the figure 1 earlier in the methods section.

PBL modules were created based on the structure of the steps in the statistical analysis (Figure 1).

- A summary table of the assessment activities may also be useful. Could you also provide a sample, maybe in an appendix to know what these activities looked like, time dedicated to them, etc.

- I have difficulty following how there was significant difference between the two groups or how they were sufficiently separated for the purpose of the study. Can you please provide more details about how their experiences differed based on the random assignment

Thank you for your comments. We provided the detailed description of all the activities in the Material and methods section.

‘Blended learning course in medical statistics and informatics is based on a Moodle platform and contains 15 classes of theoretical lectures, 30 classes of practical exercises and 15 classes of other type, such as online readings or seminars. Total of 70% of the program of this course is comprised of medical statistics and this part of the course contains units on data types, descriptive statistics, confidence interval, probability and probability distributions, hypotheses testing, correlation and linear regression. Practical exercises are done using the statistical software Easy R (EZR) [18]. Students from both groups in our study had access to identical course activities (lectures and exercises), except for the access to the problem-based modules that were available only the students from hPBL group (Table 1). During the course, students receive grades for all existing activities: lectures, exercises, colloquium, seminars, solving problems and final test. Students can see all the points for each activity any time during the course. The maximal number of points is 100 (70 for statistics and 30 for informatics). Students need to obtain the minimum of 51 points to pass the course. Based on the total number of points (51-100) the passing grades students can receive vary from 6 to 10.

Table 1 – Activities during Medical statistics and informatics course

Timeline hPBL BL

Weekly Lectures Lectures

Weekly Practical exercises done using the statistical software Practical exercises done using the statistical software

Weekly Independent students’ assignments (interactive online lectures, Moodle) Independent students’ assignments (interactive online lectures, Moodle)

Weekly Problem-based learning module (Moodle) –

During the course Seminars Seminars

During the course Colloquium Colloquium

At the end of the course Problem solving Problem solving

At the end of the course Final test Final test

Our study examined only the outcomes of medical statistics (70% of the course): problem solving score (5 problems with maximum score of the total of 25 points) and total medical statistics score (theoretical knowledge score, practical exercises score, problem solving score, independent students’ assignments score, seminars and colloquium; the maximal total medical statistics score was 70 points).‘

- It is not clear what assessment activities are included and how the relates to the linear regression. More details are needed. How were the problem solving and medical statistics score measured? What are the total possible scores? Have they been used in other settings that showcase how well they measure those two abilities? Etc. Since these are essential for interpreting the results more details are needed about the reliability and validity of these results.

Thank you for the comment. We added the explanation to the Materials and methods section.

‘Our study examined only the outcomes of medical statistics (70% of the course): problem solving score (5 problems with maximum score of the total of 25 points) and total medical statistics score (theoretical knowledge score, practical exercises score, problem solving score, independent students’ assignments score, seminars and colloquium; the maximal total medical statistics score was 70 points).’

RESULTS

- Can you please clarify "the grade expected" since these are number values; for international audience this may be useful if they are used to letters, etc.

Thank you for noticing this. The ‘grade expected’ referred to the grade that student stated that is expected to get in Medical statistics and informatics. Students were asked at the beginning of the course which grade they are expecting to get at the end of it. The passing grades at our University vary from 6 to 10, and the grade 5 means that the student failed the exam. The passing grades are computed based on the total points obtained during the course (0-100). We provided the explanation in the methods section:

‘During the course, students receive grades for all existing activities: lectures, exercises, colloquium, seminars, solving problems and final test. Students can see all the points for each activity any time during the course. The maximal number of points is 100 (70 for statistics and 30 for informatics). Students need to obtain the minimum of 51 points to pass the course. Based on the total number of points (51-100) the passing grades students can receive vary from 6 to 10.’

- "The exam was passed" in the table is also confusing. I'm not sure what this means, the number of people who passed? What qualifies as passing the exam and when is the exam administered?

Thank you for the comment. We understand that it was not clearly described in the methods and in the results section. We added the explanation of the minimum of activities student needs to complete in order to complete the course and rephrased the ‘passing the exam’ to successfully completing the course in medical statistics in Table 2.

- Do you have the correlation between the problem solving and medical statistics scores, it would be curious to see the extent of their relationship to one another.

Thank you for the comment. The problem solving score is a part of total medical statistics score, which is why we did not examine the correlation between them. We described this in the material and methods section:

‘total medical statistics score (theoretical knowledge score, practical exercises score, problem solving score, independent students’ assignments score, seminars and colloquium; the maximal total medical statistics score was 70 points).’

- For Likert scales, it's generally inappropriate to summarize at average/median levels. Instead, it is preferred to illustrate the number of individuals who responded with each choice.

Thank you for your comment. The responses in our questionnaire did not have the names of the categories, but were numerical (1-5), which is why we considered that it is adequate to show them through median and range. Additionally, as seen in table, the minimum in our study was 3. However, in accordance with your comments, we added the column in table 5 with the frequency of the most frequent response.

Table 6. Students’ attitudes towards the implemented problem solving modules

Question median (range) n (%)

highest satisfaction

Problems are adequate for learning medical statistics 5 (3 – 5) 19 (73.1%)

Contents and structure of problem-based modules is interesting 4.5 (3 – 5) 13 (50.0%)

I like this modality of solving the actual statistical problems 5 (3 – 5) 19 (73.1%)

Step- by- step approach within the module is useful for learning medical statistics 5 (3 – 5) 20 (76.9%)

Problem based modules helped me to learn the steps for resolving actual statistical problems 5 (3 – 5) 21 (80.8%)

Problem-based modules helped me understand medical statistics 5 (3 – 5) 20 (76.9%)

DISCUSSION

- More comparison of statistical versus "clinical" significance may be necessary. The mean scores differed by 1-2 points, is that truly significant beyond statistically?

Along with the statistical significance, we calculated the effect size, as an approximation to the 'clinical' significance. In our study, the effect size was moderate (0.69 for problem solving score and 0.55 for total medical statistics score). The larger effect size is associated with the power of the statistical test used correctly reject a false null hypothesis, or probability that the test will identify a treatment effect if one really exists. In our study, the solving of PBL modules increased not only the problem solving score, but also the total medical statistics score. This can lead to the increase in the capability of the physicians to apply medical statistics in their daily work. We consider that this will be confirmed in the future research on the larger sample. Additionaly, the differences between the groups remained significant in the multivariate models as well.

- Unfortunately, more details are needed about the methodology to evaluate the discussion appropriately.

We hope that the new paragraphs added in the introduction and materials and methods clarify the methodology and enable easier reading of the discussion section.

################

Response to reviewer 3:

Dear Madame/Sir,

Thank you for your time and assistance. We appreciate very much your professional advice and accepted comments and suggestions in the text.

We accepted all suggestions in Abstract, Introduction, Methods, Results, and Discussion section.

Also, we added the recommended references.

Comments to the authors:

Dear authors,

Congratulations on the work developed and submitted to this journal, on the effects of problem-based learning (PBL) modules within blended learning (BL) courses in medical statistics, by running a randomized controlled trial. Your aim was to assess the efficacy of PBL modules implemented within the blended-learning courses in medical statistics through the knowledge outcomes and student satisfaction.

Considering the above mentioned as a head start for the review, below you will find some comments on each section that should be addressed.

- Abstract Section:

Whilst being overall well written, this particular section should be restructured in order to be more appealing for the reader. There is an excess of abbreviations which lead to a sense of confusion.

Additionally, it should mention briefly the future steps of this project.

Thank you for your comment. We rewrote the abstract in order to be more understandable. We also added sentence on the future research:

'The authors of this study are planning to create PBL modules for advanced courses in medical statistics and to conduct this study on other universities with a more representative study sample, with the aim to overcome the limitations of the existing study and confirm its results.’

- Introduction Section:

This particular section is underexplored. Some questions that should guide your thoughts while adding some important information are the reason why PBL methods can be important on the subject of medical statistics and the particular subset of skills that a medical student is supposed to develop from this curricula. Additionally, once you set out to explore knowledge outcomes and student satisfaction, there are previously conducted studies that explore these variables with future medical skills, so what is really the importance of student satisfaction on continuing learning profiles, e.g.?

Thank you for your valuable comment. We added the paragraph in the introduction section.

‘Medical doctors can be exceptional in their fields even if they do not know medical statistics, but they will be better if they do [1]. The study of Swift et al [2] showed that medical doctors considered medical statistics useful for “accessing clinical guidelines and evidence summaries, explaining risk levels to patients, assessing medical marketing and advertising material, interpreting the results of a screening test, reading research publications for general professional interest, and using research publications to explore non-standard treatment and management options.” Future physicians also thought that there was a need for a practical application of knowledge in medical statistics, not only its’ theoretical basis [3]. Lack of knowledge in medical statistics can lead to misinterpretation of clinical findings [4]. Statistical softwares, widely available now, enable an easy and comfortable analysis, but mistakes can be made when choosing the appropriate statistical test or assumptions for its' application [5]. Medical students state that learning medical statistics through real life problems and the process of drawing conclusions can be more productive than traditional learning and knowledge assessment [6].'

1. Trajkovic G. “Introduction to medical statistics”. Lectures. Faculty of Medicine, University of Pristina in Kosovska Mitrovica. 2015.

2. Swift L, Miles S, Price GM, Shepstone L, Leinster SJ. Do doctors need statistics? Doctors’ use of and attitudes to probability and statistics. Stat Med. 2009;28: 1969–1981. doi:10.1002/sim.3608

3. MacDougall M, Cameron HS, Maxwell SRJ. Medical graduate views on statistical learning needs for clinical practice: A comprehensive survey. BMC Med Educ. 2019;20. doi:10.1186/s12909-019-1842-1

4. Windish DM, Huot SJ, Green ML. Medicine residents’ understanding of the biostatistics and results in the medical literature. J Am Med Assoc. 2007;298: 1010–1022. doi:10.1001/jama.298.9.1010

5. Masic I, Jankovic SM, Begic E. PhD Students and the Most Frequent Mistakes during Data Interpretation by Statistical Analysis Software. Studies in Health Technology and Informatics. IOS Press; 2019. pp. 105–109. doi:10.3233/SHTI190028

6. Astin J, Jenkins T, Moore L. Medical students’ perspective on the teaching of medical statistics in the undergraduate medical curriculum. Stat Med. 2002;21: 1003–1006. doi:10.1002/sim.1132

The last paragraph should be fully restructured, as it is one of the most important ones for the reader, since it is the last one before the next Materials and Methods section. I would advise to begin with the aim of the study and only then advance to the design made. Also, some english grammar errors were found, which are described below.

Thank you for your comment. We rephrased the paragraph.

‘The aim of this study was to evaluate the effectiveness of implemented problem-based modules within blended learning courses in medical statistics through the outcomes of knowledge and student satisfaction. We created problem-based modules in medical statistics, based on actual problems which contained all of the steps in statistical analysis (defining the problem, choosing and applying adequate statistical tests, interpreting the results and drawing conclusions) and implemented them within the blended learning course.’

Line 37 - Where written "In the past decade, there are increasing (...)" change for "In the past decade, there is an increasing (...)"

Thank you for your comment, we corrected this.

Line 49 - Where written "During learning process the students (...)" change for "During the learning process, students (...)"

Thank you for your comment, we corrected this.

Line 50 - Where written "They" change for "The students" or "Students"

Thank you for your comment, we corrected this.

Paragraph 57 - Where written "To the best of our knowledge, there were no studies on problem-based blended learning method for teaching the medical statistics." change for "To the best of our knowledge, there are no previous studies on the problem-based blended learning method for medical statistics teaching."

Thank you for your comment, we corrected this.

- Materials and Methods Section:

First and foremost, once being a randomized controlled trial, it would be optimal to have it registered at the clinicaltrials.gov platform.

Thank you for your comment. However, our study is a randomized controlled pilot study, and the intervention and outcomes are associated with the education of medical students. As we did not examine any clinical outcome, we did not think that the study fulfills the criteria for registration at the clinicaltrials.gov.

Further on, the authors mention that an informed consent was obtained, however there is no reference on how. It is my opinion that these paragraphs 73/74/75 should add the environment in which the consent was obtained, to avoid any coercion interpretations.

Thank you for noticing this. We added the section regarding the ethics in our study.

'Ethics Statement

The study was approved by the Ethical Committee of the Faculty of Medicine, University of Pristina, Kosovska Mitrovica (No. 09-3171). During the first week of the course (the first week of the semester), before the randomization in the groups was performed, students received written and oral explanations of the study, the processes and aims, the modalities of data gathering and data analysis. Students were explained that all the information gathered would be anonymous, that the participation was voluntary and that they could dropout of the study at any point. After this, the students gave an oral consent for their participation in the study, which was then verified in their records. A questionnaire on satisfaction was filled in after the course had been completed, as an online anonymous and non-obligatory questionnaire. All the data on the course outcomes and the data from the questionnaire on the students’ satisfaction were gathered by the administrator of the Moodle platform who was the only one with the access to the complete database. The authors of the study only had access to the anonymized database that is provided with the manuscript.’

Additionally, you mention an anonymous online questionnaire for the assessment of the student satisfaction of the hPBL group. Was this preceded by a mandatory informed consent?

Thank you for your comment. The questioonare was not obligatory and the procedure is described in the ethical statement section:

‘A questionnaire on satisfaction was filled in after the course had been completed, as an online anonymous and non-obligatory questionnaire.’

How was the data managed and stored? Who had access to it? Were an IP addresses collected?

Thank you for the comment. We did not collect the IP addresses. The explanation was given in the Ethical statement:

‘All the data on the course outcomes and the data from the questionnaire on the students’ satisfaction were gathered by the administrator of the Moodle platform who was the only one with the access to the complete database. The authors of the study only had access to the anonymized database that is provided with the manuscript.’

Overall, this section needs more work, mainly at the description of the method itself. I would a recomend a major revision of the english used, as well as the review of the duplicated information. Some sentences are in reality avoidable, because they are the continuation of the last.

Line 76 - Where written "The study began on the 1st of October, 2019 (...)" change for "The study began on October 1st, 2019 (...)

Thank you for your comment, we corrected this.

Line 77 - Where written "was completed at the end of school year (30.09.2020)" change for "was completed at the end of school year (September 30th, 2020)."

Thank you for your comment, we corrected this.

Lines 84/85 - Where written "Ministry of education, science and technological development of the Republic of Serbia" change for "Ministry of Education, Science and Technological Development of the Republic of Serbia"

Thank you for your comment, we corrected this.

Paragraph 86 - Should be rewritten, since there is repeated information.

Thank you for your comment, we rephrased the paragraph.

‘PBL modules were created based on the structure of the steps in statistical analysis (Figure 1). Statistical analysis of the research problem is based on the multiple successive steps which include the following: the definition of the problem and the research question, recognition of the data type, sample type and hypothesis, selection of the adequate statistical test, application of the test, interpretation of the results and conclusion related to the description of the data, statistical conclusion and implications of the results. The PBL modules use guiding questions following the steps of statistical analysis. The guiding questions consisted of: interactive multiple choice or open-ended questions and followed a similar principle to the one Brown et al. applied [19]. Guiding questions changed within each step, based on the type of the statistical problem, number of variables and the sample (examples can be seen online following the link provided in the text bellow). Students can understand the necessary components for statistical reasoning by answering these questions and learn how to solve the problem. ’

Lines 106/107 - Rewrite the following sentence "All the necessary information is given to students during the lectures and exercises in the materials in the blended- learning course.". Suggestion: "All the necessary information was given to students during the lectures, while the exercises and materials were given at the blended-learning course."

Thank you for your comment, we corrected this.

Lines 126/127/128 - Where written "Anonymous online questionnaire with the five-point Likert scale (1 point- low satisfaction, 5 points- high satisfaction) was used to assess the satisfaction of the students in the hPBL group with the PBL modules." change for "An anonymous online questionnaire with the five-point Likert scale (1 point- low satisfaction, 5 points- high satisfaction) was used to assess the satisfaction of the students in the hPBL group with the PBL modules.".

Thank you for your comment, we corrected this.

- Results Section:

This section is well presented and clear.

I would suggest that in line 139, where written "There were no differences between (...)", the authors change for "There were no statistically significant differences between (...)".

Thank you for your comment, we corrected this.

- Discussion Section:

The discussion section is well explored and written.

Thank you for the comment.

- Conclusions Section:

The conclusions are both supported by the results obtained in the present study, as well as previous studies.

This article has some flaws as pointed above, for each section. A general review of the english grammar and spell check should be performed, additionally to addressing the point mentioned.

Thank you for the comment. We also consulted the native English speaker for the revised version of the manuscript.

Submitted filename: Response to Reviewers.docx

Click here for additional data file.

11 Jan 2022

Effects of problem-based learning modules within blended learning courses in medical statistics – randomized controlled pilot study

PONE-D-21-22424R1

Dear Dr. Bukumiric,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Gwo-Jen Hwang

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Thank you for the response to the comments--I feel they have greatly enhanced the publication.

14 Jan 2022

PONE-D-21-22424R1

Effects of problem-based learning modules within blended learning courses in medical statistics – a randomized controlled pilot study

Dear Dr. Bukumiric:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Gwo-Jen Hwang

Academic Editor

PLOS ONE