Use of a 3D virtual app and academic performance in the study of the anatomy of the musculoskeletal system among Peruvian medical students.

OBJECTIVE: To evaluate the association between the use of a 3D virtual App and academic performance among Peruvian medical students. In addition, factors associated with academic performance were also assessed.
METHODS: We conducted an analytical cross-sectional study in students enrolled in the Musculoskeletal System course during the first semester of 2019. Students filled out a data collection form and the "Self-directed learning readiness scale" (SDLRS) questionnaire adapted by Fisher, King, and Tangle. Linear regression models were carried out to assess the association between the appropriate use of the application and academic performance. Additionally, the factors associated with academic performance were evaluated using nested models, and β coefficients were calculated by manual forward selection.
RESULTS: A total of 187 medical students were included. The 61% were female and the median age was 21 [20-22] years. The average grade was 13.5 ± 2 and 21% reported an adequate use of a 3D App. No association was found between the use of the 3D App and academic performance in the adjusted model (aβ = 0.17; 95% CI: -0.45 to 0.80). We found that age (aβ = -0.22; 95% CI: -0.39 to -0.06), performing extracurricular activities (aβ = 0.75, 95% CI: 0.25 to 1.24) and having failed an anatomy/physiology course before (aβ = -2.11 to 95% CI: -2.9 to -1.8) were factors associated with academic performance.
CONCLUSION: The adequate use of a 3D application to study the anatomy of the musculoskeletal system was not significantly associated with better academic performance.

Introduction

During recent years, there have been several changes in medical education curricula, probably due to the increasing advances in technology and the greater variety of electronic tools available for medical students [1]. Although traditional teaching through books for the study of anatomy represents a widely used methodology [2], new contemporary teaching techniques [3], together with the trend of self-assisted education [4], have introduced new virtual tools that are part of the e-learning teaching technique [5]. Thus, universities and institutions have implemented interactive video classes, electronic material available on the web, and downloadable virtual applications [6], so that students can have a different, broad and detailed perspective of the human body to study human anatomy.

Some studies have assessed the utility of using e-learning tools in health professionals' academic training. In Norway, second-year medical students that used additional electronic material in an immunology course scored better results during their academic examination than those who did not [7]. Similarly, studies conducted on medical students in the Netherlands [8] and Canada [9] found that using books to study anatomy could cause difficulties and 3D visualization methods might be more useful [8]. However, other studies have not found an association between academic performance and using 3D applications to study anatomy. In South Korea, there was no statistically significant difference between the use of a 3D anatomy atlas and the results of the evaluation of an anatomy and embryology course among first-year medical students [10]. Similarly, a study conducted on students of the Major Program of Chiropractors in Australia found no association between their neuroanatomy evaluation score and the use of 3D applications [11].

We have found few studies in Latin America [12,13,14] regarding the implementation of e-learning in medical education, and knowledge about its association with students' academic performance remains an unexplored field. Likewise, it is important to mention that the majority was conducted five years ago or more [12,14]. Therefore, it is not coupled with new technological advances. In Peru, there are some reports of previous experiences regarding the implementation of e-learning as a tool for online scientific research courses [15,16]. However, its impact has not been specifically evaluated. Besides, we have not found studies that assessed the association between the use of virtual applications and academic performance in our country.

Musculoskeletal System is an in-class course where the teacher explains different anatomical pieces and their respective insertions, innervations, and movements. The importance of understanding this system's anatomy lies in that the acquisition of knowledge in a comprehensive way enables a better training of students to become competent professionals. The assessment method for this course consists of multiple groups and individual tests. Individual assessments consist principally of multiple-choice tests, which aim to measure the student's knowledge regarding the musculoskeletal system's anatomy, physiology, and pathology. On the other hand, group assessments are both oral and written. They determine how well prepared the students are regarding the knowledge of the different anatomical landmarks and conducting group discussions on clinical cases.

There is currently a wide range of virtual applications that offer three-dimensional technology for the study of human anatomy [17]. Using 3D virtual apps may appear as an alternative to develop the ability to visualize structures in space in a three-dimensional way. Besides, this understanding could be useful in subsequently performing surgical medical interventions [10]. This study aimed to evaluate the association between using a 3D application and the academic performance of Peruvian medical students, enrolled in the Musculoskeletal System course during the first semester of 2019. In addition, factors associated with academic performance were also assessed.

Methods

Study design

We conducted an analytical cross-sectional study in medical students from a Peruvian university enrolled in the Musculoskeletal System course during the first semester of 2019. This course gave the possibility to use different tools as a complement to theoretical knowledge. These included the use of 3D Apps and human anatomy models. There were no dissection classes during the semester.

Study population

The referential sample size was calculated in Epidat v4.2 using the following criteria: 95% of confidence level, 80% of statistical power, a ratio between sample sizes of 1, an expected standard deviation of 10.13 for the population that used traditional study techniques (books and lectures) and 10.03 for the population that used 3D Apps, with a difference of means of 4.69 [18]. The minimum sample size required was 148 students. However, we tried to enroll all the participants that met the eligibility criteria. It is pertinent to mention that the total number of students enrolled in the course in the first semester of 2019 was 292.

Regarding the eligibility criteria, we included medical students enrolled during the first semester of 2019 and who had taken the Musculoskeletal System course for the first time. We excluded students had withdrawn from the course before the first assessment or were enrolled in less than four courses during the first semester of 2019. Students who were working and studying simultaneously during the semester were also excluded.

Variables

Outcome: academic performance

We defined academic performance as the final grade point average (GPA: 0 to 20) of students enrolled in the Musculoskeletal System course during the first semester of 2019.

Exposure: use of 3D app

The use of a 3D App was defined as the self-report of using a 3D virtual application more than once to study and prepare for the Musculoskeletal System anatomy assessments. Given that the course mentioned above had two theoretical-practical sessions per week, for this study, students who used the application 4 to 7 times a week were considered as adequate use, and those who did not use or used it less than three times a week were considered as inadequate use.

Other variables

The following variables were also included in the analysis: age, sex, cohabitation, extracurricular activities, number of courses enrolled during the first semester of 2019, previous failure of an anatomy/physiology course, amount of hours dedicated to study Musculoskeletal System course, and image quality, frequency of use and name of the 3D App.

In addition, we evaluated student satisfaction regarding the use of the application using three variables: utility, perception, and preference. We used the score obtained in the Self-directed Learning Scale for assessing the preparation for self-directed learning.

Characteristics of the instrument and validation

We prepared a data collection sheet that included the previously mentioned variables. Besides, we used the “Self-directed learning readiness scale” (SDLRS) questionnaire adapted by Fisher, King and Tangle in 2001 [19], which consists of 40 items rated at 5 points on the Likert Scale, considering the value of 1 “strongly disagree” and 5 “strongly agree” [20]. This instrument evaluates the students' perceptions, motivation, and skills regarding autonomous learning, in which a score greater than 150 results in an indicator of a developed self-learning ability [19]. It is a reliable scale that was initially developed to measure self-directed learning in nursing students. It has been translated into Spanish and validated for its application in Latin-American medical students, obtaining a Cronbach's alpha of 0.89 [20,21].

We performed a qualitative validation of the data collection sheet and the Self-directed Learning Scale questionnaire. For that purpose, both instruments went through a expert validation process following a Delphi methodology. The selection of experts was defined by a background of academic training in university teaching (e.g., postgraduate programs) and/or having at least three publications in journals indexed in Scopus in the area of medical education during the last five years. The instruments were sent by email to five health professionals with experience in the subject. Each one evaluated the questions taking into account clarity, coherence, and relevance. These criteria were rated with a score from 1 to 4, with 1 “Does not meet the criteria”, 2 “Low level”, 3 “Moderate level” and 4 “High level”, considering the respective indicators for each category. After this process, we made the corresponding modifications according to the experts' recommendations until we reached a final consensus for both instruments.

Once we consolidated the final instrument, it was applied to a sample of 30 students who took the Musculoskeletal System anatomy course in the first semester of 2020 to assess its understanding. The scoring procedure ranged from 1 to 10, with 0 being “not understood at all” and 10 being “fully understandable”. Finally, we calculated Cronbach's alpha of the Self-Directed Learning Scale to measure the instrument's reliability.

Data collection

The final survey was imported into Google Forms. It included a data collection form and informed consent. After the surveys were applied, we obtained the GPAs throughout the Faculty of Medicine. It is important to mention that we exclusively included the students who accepted the terms listed on the informed consent.

Statistical analysis

The data obtained from Google Forms was exported into a Microsoft Excel spreadsheet. We carried out a coding process following the instructions of a dictionary of variables that was built for this study. This process was carried out by two people independently. Once completed, the databases were crossed to detect coding errors and possible non-plausible data. After quality control, the database was entered into the statistical package STATA v15.0 (StataCorp, TX, USA) for its analysis.

Descriptive results for numerical variables were presented as mean with standard deviation (SD) or medians with interquartile range (IQR), according to the presence or absence of normality, respectively. This was evaluated with the Shapiro Wilk test; however, histogram, quantile plot, kurtosis, skewness, and equality between mean and median were also assessed. Categorical variables were presented using frequencies and percentages.

According to academic performance (AP), the sample's characteristics were descriptively evaluated. Since AP was considered a numerical variable, we used student t-test and ANOVA when the crossing was with a dichotomous and polytomous variable, respectively. For any of the cases, the corresponding assumptions were evaluated, particularly normality and homoscedasticity. The first was evaluated using the procedures described initially, and for the second, we used the Levene test.

To evaluate the association between using a 3D App and academic performance, simple and multiple linear regression models were performed to calculate the crude β (cβ) and adjusted (aβ) coefficients. Both were reported with their respective 95% confidence intervals (95% CI) and p values < 0.05 were considered significant. In the multivariable model, the adjustment was made by the confounders reported in the literature: sex, history of failure on a previous anatomy/physiology course, and score on the self-directed learning scale. In addition, the Wald test was used to ensure that the variable of enrolled courses did not interact with the model. Likewise, we confirmed that the final model complied with the assumptions of linearity, normality, and homoscedasticity using the studentized residuals.

As an exploratory approach, we evaluated factors associated with academic performance. For this purpose, cβ and aβ were also calculated. We used the manual forward selection method, starting with the null model and then adding independent variables until obtaining the most parsimonious model. At first, we considered the Akaike information criterion (AIC), but the final decision was made using the log-likelihood ratio test and considering the variation of R2. After each model selection, we evaluated the presence of multicollinearity using the variance inflation factor (VIF), which for all cases, had a value smaller than 4.

Ethics

The Institutional Review Board of the Faculty of Health Sciences of the Universidad Peruana de Ciencias Aplicadas approved the study protocol (FCS/CEI 065-05-20). Informed consent was presented to all the participants, detailing the study's objectives and procedures. The first page of the survey had the informed consent. The students that agreed had to mark the option “I have read the consent form, and I agree with it. I would like to start the survey”. Only the participants who marked this option were able to continue to the next page. In addition, students who agreed to participate were asked to voluntarily provide their university code to later ask for their final GPA. No grades were requested without prior authorization from the participant.

Results

We enrolled 253 participants, from which 66 were excluded according to the eligibility criteria and elimination due to non-plausible data. Data from 187 participants was finally analyzed.

Among the students surveyed, 61% were female and the mean GPA was 13.5 ± 2.1. Almost a quarter of the participants (21%) indicated that they had previously failed a anatomy/physiology course and 40 students (21%) had an adequate use of a 3D App. Regarding the SDLRS, more than half of the surveyed students (66%) obtained a score greater than 150. Furthermore, we obtained a Cronbach's alpha of 0.97 for this instrument. The rest of the academic and sociodemographic characteristics are presented in Table 1.

Table 1

Descriptive characteristics of the surveyed students and self-directed learning readiness (n = 187).

Variables	n (%)
Academic performance (average)∗	13.5 ± 2
Age (years)†	21 (20–22)
Gender
Male	73 (39)
Female	114 (61)
Cohabitation
Lives alone	24 (13)
Lives with father and/or mother	126 (78)
Lives with another family	15 (8)
Lives with friends	2 (1)
Extracurricular activities
No	115 (57)
Yes	86 (43)
Previous failure of an anatomy/physiology course
No	54 (29)
Yes	133 (71)
Number of courses enrolled (total)∗	5.8 ± 0.6
Hours dedicated to studying Musculoskeletal System (total)†	8 (5–10)
Use of a 3D App
Inadequate use	147 (79)
Adequate use	40 (21)
Electronic Device
Cell phone	38 (26)
Tablet or iPad	73 (50)
Computer	13 (9)
Only used 2 devices	20 (14)
Used all of the devices	2 (1)
Name of the 3D App
Visible body	87 (60)
Essential Anatomy	5 (3)
Complete Anatomy	8 (5)
Easy Anatomy	1 (1)
Anatomy 3D Atlas	41 (28)
Other	4 (3)
Frequency of usage
Once a week or none	41 (22)
Two or five times a week	106 (57)
Six or seven times a week	40 (21)
Preparation for self-directed learning
Score ≤150	64 (34)
Score >150	123 (66)
Self-directed learning scale score	155 (148–165)

Mean ± standard deviation.

Median (interquartile range).

In the bivariate analysis, we found that those students who previously failed a course in anatomy/physiology had a lower GPA than the non-disapproved students (p < 0.001). Furthermore, we found that the students who carried out extracurricular activities had a higher GPA than those who did not (p = 0.003). There were no significant differences between the adequate use of the 3D application (p = 0.944), nor having obtained a score greater than 150 on the Self-directed learning scale (p = 0.073) with academic performance (Table 2).

Table 2

Academic performance according to the descriptive characteristics of the surveyed students (n = 187).

Variables	Academic performance∗	p-value
Gender
Male	13.6 ± 0.2	0.874
Female	13.5 ± 0.1
Extracurricular activities
No	13.1 ± 0.2	0.003
Yes	14.1 ± 0.2
Previous failure of an anatomy/physiology course
No	15.2 ± 0.3	<0.001
Yes	12.9 ± 0.1
Use of a 3D App
Inadequate use	13.6 ± 2.1	0.944
Adequate use	13.5 ± 1.9
Application usage frequency
Once a week or none	13.5 ± 2.3	0.988
Two or five times a week	13.5 ± 2.0
Six or seven times a week	13.5 ± 1.9
Self-directed learning scale
Score ≤ 150	13.1 ± 0.2	0.073
Score >150	13.7 ± 0.2

Mean ± standard deviation.

After adjusting for confounders in the multivariate model, we found that students who had adequate use of the 3D application had on average 0.17 more points on their GPA than those who had inadequate use or did not use it. However, this result was not statistically significant (p = 0.580) (Table 3). In the exploratory analysis, we found that age (aβ = -0.22; 95% CI: -0.39 to -0.06), performing extracurricular activities (aβ = 0.75, 95% CI: 0.25 to 1.24) and having failed an anatomy/physiology course before (aβ = -2.11 to 95% CI: -2.9 to -1.8) were factors associated with academic performance (Table 4).

Table 3

Association between academic performance in Musculoskeletal System course and the use of the 3D application.

Variables	Academic performance
	Crude model			Adjusted model∗
	cβ	95% CI†	p-value	aβ	95% CI†	p-value
Use of a 3D App
Inadequate use	Ref.			Ref.
Adequate use	-0.03	-0.75; 0.70	0.944	0.17	-0.45; 0.80	0.580

Adjusted by sex, history of the previous failure of an anatomy/physiology course and score obtained on the Self-directed learning scale.

Confidence interval.

Table 4

Factors associated with academic performance of students enrolled in the Musculoskeletal System course.

Variables	Bivariate analysis			Multiple regression, Parsimonious Model††
	cβ∗	95% CI†	p-value	aβ∗	95% CI†	p-value
Age	-0.40	-0.58;-0.21	<0.001	-0.22	-0.39; -0.06	0.008
Gender
Male	Ref.			-	-	-
Female	-0.05	-0.66; 0.56	0.87
Cohabitation
Lives alone	Ref.			-	-	-
Lives with father and/or mother	0.59	-0.30; 1.49	0.19
Lives with another family	-0.02	-1.35; 1.31	0.98
Lives with friends	-0.08	-3.07; 2.90	0.95
Extracurricular activities
No	Ref.			Ref.
Yes	0.90	0.30; 1.48	0.003	0.75	0.25; 1.24	0.003
Previous failure of an anatomy/physiology course
No	Ref.			Ref.
Yes	-2.36	-2.91;-1.80	<0.001	-2.11	-2.67; -1.56	<0.001
Number of courses enrolled	0.08	-0.39; 0.55	0.72	-	-	-
Hours of study	0.04	-0.01; 0.09	0.13	-	-	-
3D App use
Inadequate use	Ref.
Adecuate use	-0.03	-0.75; 0.69	0.94	-	-	-
Score in the Self-directed learning scale
Score ≤ 150	Ref.			-	-	-
Score >150	0.57	-0.05; 1.19	0.073

Beta coefficient.

Confidence interval.

Estimated by manual forward selection.

Discussion

Main findings

The use of a 3D application for the study of the Musculoskeletal System's anatomy was not associated with academic performance. Although most students had an adequate use of the application and recognized it as a useful study tool, it was not associated with obtaining higher GPAs at the end of the course.

Use of a 3D application and academic performance

Findings regarding the association between using a 3D virtual resource and academic performance vary within the literature. We found similar results to the ones obtained by Park S et al. (2019) and Meyer AJ et al. (2016) [10,11]. Likewise, a previous review reported that students' preferred study methods were books and lectures over applications [22]. On the contrary, other studies conducted in medical [23] and psychology [24] students that evaluated the association between the use of virtual applications with academic performance through pre-tests and post-tests showed an improvement in their performance of the students reflected in the score obtained after using the application compared to that of the control group (which used anatomy books). This would indicate that virtual applications to study anatomy could represent a useful tool for improving academic performance in the short term. However, it does not exclude other study methods preferred by the students.

The practical applications of integrating the use of virtual applications for the study of anatomy include the possibility of studying the various anatomical landmarks from different angles and in a three-dimensional way [8]. However, this study and others already mentioned could indicate that students might also prefer anatomy books or other traditional methods instead of 3D applications to study anatomy.

Factors associated with academic performance

We found that students who had previously failed an anatomy/physiology course had a lower GPA than those who had never failed before. We believe this could be because students who have not failed have a better planning of their learning schedules, in addition to a greater desire to learn compared to those students who have previously failed a course in their medical curricula.

Students who took extracurricular activities had a higher GPA compared to those who did not. Considering that the university's extracurricular activities where this study was conducted are mainly sports, this finding could be explained by a combination of various positive factors. For example, increased cerebral blood circulation, synthesis of new neurons, and the increase in neurotrophins and neurotransmitters along with variations in neuronal plasticity, might have an important role in learning and memory, which would be reflected in academic performance [25].

In contrast, we saw that the Musculoskeletal System course's final GPA was lower in older students. We have found similar studies that affirm the inverse relationship between age and academic performance [26,27]. This could be related to the fact that older students might have a history of failed courses. Hence, the GPA obtained could reflect their limited performance during the medical career.

Limitations

Some limitations should be highlighted. First, given the study's cross-sectional design, it was not possible to assess causality. Second, our findings' external validity is limited since we only focused on students of a single academic semester that undertook a specific anatomy course. Third, since we used a self-administered questionnaire, some answers might have errors, and there is the possibility of social desirability bias. However, this was not possible for the final GPA since the Faculty provided this information to ensure its validity and reliability.

Conclusion

Our findings indicated that using a 3D App to study the musculoskeletal system's anatomy was not associated with academic performance. Given the limited number of related research in Latin America, it is necessary to conduct more studies to clarify whether virtual anatomy applications can influence learning among medical students. We recommend that these studies should evaluate the existence of a long-term impact on academic performance, preferably with randomized approaches.

Declarations

Author contribution statement

Carlos Jesus Toro-Huamanchumo, Sol de La Barrera-Cantoni, Melanni Lizarbe-Lezama, Jhoel Rodriguez-Macedo, Tammy Carrillo-Levin and Maria Jaramillo-Ocharan: Conceived and designed the experiments; Performed the experiments; Analyzed and interpreted the data; Wrote the paper.

Funding statement

This work was supported by Dirección de Investigación de la Universidad Peruana de Ciencias Aplicadas, Lima, Peru (A-240-2021).

Data availability statement

Data will be made available on request.

Declaration of interests statement

The authors declare no conflict of interest.

Additional information

No additional information is available for this paper.