The efficacy of the new medical internship management network system.

With the increasing number of medical students and scattered host hospitals, the traditional clinical internship management system has been falling behind. Thus, we have independently developed a new clinical practice management system based on the browser/server structure.We aim to evaluate the efficacy of the new medical management system on the quality of the internship and survey the users' satisfaction.This study was conducted on the students of Second Clinical Medical College of Southern Medical University, including 672 interns (non-user group) who had not used the system and 315 users (user group). We analyzed the scores of their theoretical knowledge and clinical skills before and after the internship, which served as the quantitative standards for evaluating the efficacy of the system. User satisfaction surveys were also conducted, including by interns, clinical teachers, and teaching administrators.An analysis of scores showed that the user group had more significant improvement in both clinical knowledge and skills than the non-user group after the internship. Significant improvement was found in the user group, while there was a slight difference in the non-user group. A total of 310 students responded to the satisfaction questionnaires effectively, most of whom agreed with the system's benefits of learning theoretical knowledge (98.1%, n = 304) and clinical skills (93.9%, n = 291). Additionally, 115 clinical teachers and 27 teaching administrators were selected using a random sampling method for the survey. The results showed that 96.3% of the teachers considered the system to be helpful for improving teaching quality, and 92.6% administrators thought it was useful for implementing management. Over 90% of the participants in the 3 user groups reported a strong satisfaction of the system.With high level of user satisfaction, this system helps to strengthen the management for interns and improves the medical knowledge and skills of interns. Hence, it could be widely used in medical colleges and even in other fields.

Introduction

Medical internships are of great significance for medical students to cultivate clinical thinking and sharpen clinical skills and are an essential part of medical education. A high-quality internship is a critical transition from theoretical learning to clinical practice.[ However, most medical colleges still rely on hospitals to manage their students. This means that the quality of the internship is greatly dependent on the responsibility and teaching level of the hospitals. An internship survey conducted in 7 hospitals in Australia consisted of 53 satisfaction items on the training that the interns received. Emergencies and skill training were rated 2.76 and 2.50, respectively, on a scale of 1 to 5, which were much lower than the 30% to 40% of the students’ expectations.[ Most of the students believed that the number of diseases they studied during internship was less than their demand, only taking 70% or even less.[ With the traditional clinical internship management system, hospitals could not meet the students’ needs and syllabus regardless of whether the internship was undertaken in metropolitan or regional settings.[ In addition to the responsibility and teaching levels, increasing clinical mistakes and risks arise from the interns and make the administrators of hospitals very anxious that they would like to put interns into a peripheral place of medical work.[ A previous study suggested that 45% of interns admitted that they had spent over half of their internship on preparing for postgraduate examinations and employment. Moreover, the enthusiasm that the interns had has a great impact on the quality of the internship.[ Thus, reforming the internship system is clearly in order. In recent years, the management keeps changing from the traditional model, which relies on written exams and comments offered by teachers, to the new system that depends on a comprehensive evaluation consisting of modular assessments, logs and clinical works.[

We managed to construct a new medical internship management network system based on the browser/server structure to standardize the teaching contents, keep interns enthusiastic and improve the clinical practice quality. Based on modern information technology, the New Medical Internship Management Network System consists of 3 subsystems, including interns, hospital and clinical colleges.[ The subsystem of interns mainly includes 4 functional modules of the department rotation plan, internship log, feedback, and evaluation. Interns have to record what they have learned, which means that their learning tasks are required to be accomplished and logged on time in the current department. Additionally, they can view the evaluations from teachers, departments, and host hospitals, for which interns can also provide feedback through the system. With the benefit of the browser/server structure, the clinical teachers and teaching administrators can view all that the interns have uploaded through their own subsystem. All users can log in to their system using the browser entry (http://shixi.smu.edu.cn/). It can not only heighten tripartite engagement but can also realize real-time tracking of intern dynamics. This research aims at evaluating the efficacy of the system on improving the quality of the internship by comprising the enhancement of theoretical knowledge and clinical skills between the students of users and non-users. Meanwhile, we also surveyed the users’ satisfaction to understand their experience and estimate the practicability for expansion.

Methods

Ethical statement

This study conforms to the Helsinki Declaration and was also permitted by the ethics committee of Southern Medical University.

Selection of study subjects and measurements

This study is a retrospective cohort study. The interns of Grades 2010 to 2012 from the Second Clinical Medical College of Southern Medical University were included in this study. The non-user group included 672 participants from Grades 2010 and 2011, all of whom had not used the system. The user group consisted of 315 interns from Grade 2012 who tried out the new system. The selection criterion for interns is as follows: they were students from grades 2010 to 2012 (January 2014–December 2016) who majored in clinical medicine at the Second Clinical Medical College of Southern Medical University. Students signed informed consents and voluntarily joined this study. They were interns at grade-A tertiary hospitals, not in community settings, for 1 year. Intern users used the new medical internship management network system for 1 year and at least 3 times a week. The test results of the theoretical knowledge and clinical skills from the 2 groups were obtained.

All of the 315 intern users were surveyed after their internship using a self-reported satisfaction questionnaire.[ Combined with the specific situation of the medical students in our university, the questionnaire was designed referring to manuscripts on how to evaluate web-based learning systems. The questionnaire consisted of 15 items on the rationality (6 items), practicability (5 items), usability (3 items), and overall evaluation (1 item) of the system. As a result, 315 questionnaires were distributed and recalled, of which 310 were effective. The response rate was 100%, while the effective rate was 98.4%.

There are 23 hospitals that are attached to Second Clinical Medical College of Southern Medical University. Five teachers from each hospital were randomly selected to complete the questionnaires. The selection criterion for teachers was as follows: teachers from tertiary hospitals affiliated with Southern Medical University; teachers that taught interns from the Second Clinical Medical College of Southern Medical University for at least 2 years; teacher users who used the new medical internship management network system for one year and at least once a week. The questionnaire for teachers consisted of 12 items on rationality (3 items), practicability (5 items), usability (3 items), and overall evaluation (1 item). Among the 115 questionnaires that were distributed, 111 questionnaires were recalled, and 108 are effective. The response rate and effective rates were 96.5% and 97.3%, respectively.

In addition, every teaching administrator randomly selected from the teaching affairs office of each hospital, as well as 5 teaching administrators from the Second Clinical College, were surveyed. The selection criterion for the teaching administrators were as follows: teaching administrators from the Second Clinical Medical College of Southern Medical University; teaching administrators that occupied management positions for at least 2 years; teaching administrator users who used the new medical internship management network system for 1 year and at least once every day and who knew the system well. Their questionnaires consisted of 10 items on practicability (6 items), usability (3 items), and the overall evaluation (1 item). All of the 27 questionnaires distributed were recalled effectively. Both the response rate and the effective rate were 100%.

All of these questionnaires were designed on a 5-point Likert scale (1 = strongly disagree [SD], 5 = strongly agree [SA]). Items were excluded from analysis if they met any of the following criteria: any items in the questionnaire that had less than or more than 1 answer; all of the items in the questionnaire had the same answer; questionnaires that were completed in less than 30 seconds.

Statistical analysis

Statistical analysis was performed using SPSS 23.0 (SPSS, Chicago, IL).

With α=0.025, ß = 0.10, δ=0.50, σ (the standard deviation of clinical skills scores before internship) = 2.76 and m (the ratio of the number of non-users to all intern of Grade 2010 to 2012) = 0.68. The max minimal sample size was equal to 592, which was much smaller than the number of intern interviewees (n = 887) in this study.

The test of normality was used to assess the normal distribution of continuous variables with a P value of <.05. Continuous variables were represented as the means ± standard deviation. Student t test (independent-sample t test) was used for continuous variables with a normal distribution, whereas the Mann–Whitney U test was used for continuous variables without a normal distribution. A P value of <.05 was considered statistically significant, and the confidence interval was defined as 95%.

Results

Results of the independent samples test

The results of the test of normality showed a P value for each variable of <.05. In other words, the continuous variables were in line with the normal distribution, and Student t test (independent-sample t test) was used for continuous variables with a normal distribution. The analyzed test scores are represented as the means ± standard deviation in Table 1.

Table 1

Results of 2 sample t test between 2 groups.

Before the internship, both groups were similar in term of the Pre-internship Clinical Knowledge Scores and the Pre-internship Clinical Skills Scores. There was no significant difference in the tests of theoretical knowledge and clinical technique between the 2 groups before the internship. However, the post-internship Clinical Knowledge Scores and post-internship Clinical Skills Scores were significantly higher in the user group than in the non-user group (P = .006 and less than .001, respectively). A significant difference was seen in both tests between the groups after the internship.

Results of the questionnaires

As for the interns’ questionnaires, the 315 interns of Grade 2012 used the new system, and 98.4% (n = 310) of the interns offered an effective response to the questionnaires. Over 93% of the intern users felt that the learning missions in the system were set with a proper difficulty and a wide range. Additionally, they agreed that the introduction and aims of the learning missions were clearly stated, given that the missions corresponded with their learning requirements and it was possible for them to complete them in internship. Almost all of the intern users believed that, with the help of the system, it was possible for them to study with clearer objectives, continuous enthusiasm and at an appropriate learning pace, which was beneficial for theory and skills learning. The results of the questionnaires for interns are shown in Table 2.

Table 2

Results of the questionnaire for interns (n = 310).

As for the teachers’ questionnaires, 115 teachers were selected randomly from 23 hospitals, and 97.3% (n = 108) of them offered valid responses. Teachers generally had positive feedback on the task difficulty and the wide-ranging learning missions set in the system. They ensured that they were able to help interns accomplish all of their missions during internship. While using the system, more than 90% of the teachers confirmed that they could teach with clearer aims and more enthusiasm. Feedback from the interns did have an impact on their improvement and self-rectification. Most agreed that it helped interns promote theory and skills learning. The results of the questionnaires for teachers are shown in Table 3.

Table 3

Results of the questionnaire for teachers (n = 108).

As for the teaching administrators’ questionnaires, a total of 27 teaching administrators were surveyed in the study. Approximately 96% of the administrators thought it was positive that the system helped to roughly grasp the attendance of the interns and learn about them more comprehensively and in real time. The shortcomings of learning and teaching could be found earlier and more expeditiously. We were happy to find that all of the interviewees thought that they had taken advantages of learning their defects in time by the system. As a result, their works had become more efficient. The results from the questionnaires for the teaching administrators are shown in Table 4.

Table 4

Results of the questionnaire for teaching administrators (n = 27).

As for usability, over 90% of participants in each group thought highly of its simplicity, convenience, and accessibility. They also strongly endorsed the new system for intern management.

Discussion

In this study, we aimed to demonstrate that the new medical internship management network system could be conducive to the completion of the self-management and self-improvement of interns and clinical teachers and could be of great help to teaching administrators. The results of the study reaffirmed our thoughts.

The data showed that interns who had used the system were superior to the non-users in both theoretical knowledge and clinical skills, which means that by strengthening management, the interns had a better command of both sides. Additionally, both interns and clinical teachers subjectively agreed that the system could help clinical learning, which could be seen from the results of the questionnaire.

As we all know, clinical students often have to digest a great deal of theoretical knowledge in a very short period of time. For clinical practice learning in college, it seems tedious and lacks a sense of reality. Therefore, internship is needed to compensate for the weaknesses of classroom learning. Medical students come into contact with real patients and see fascinating cases during internship. In addition to the clinical experience dictated by teachers and the review of textbooks during their own time, students can integrate theory with practice more easily, deepen their comprehension and consolidate their memory of the theoretical knowledge. Thus, a new tool to enhance the efficacy of internship management would be of great help to clinical learning. This is the new medical internship management network system.[ The learning missions that interns must finish in every department are set in the system. They include diseases and skills that all interns need to be familiar with. Interns are required to record the completion of learning tasks in their logs. Using writing logs, interns can study and work more efficiently with clearer aims, keep accumulating experience, and review what they have acquired regularly or in every specific period, which could be of benefit to consolidate their clinical knowledge and skills. With these learning missions, interns would be able to learn with clear aims during clinical learning.[ In addition, clinical teachers, officers of departments and the internship teaching affairs office can also make their evaluation for interns. With the teachers’ assessments and self-assessments, interns learn about their weaknesses, adjust their learning state and improve themselves in real time, which drive the interns to be continuously positive and enthusiastic.[ Interns would learn more from their internship with greater enthusiasm.

On the other hand, interns can also offer feedback to clinical teachers, officers of departments and the internship teaching affairs office through the system. According to the results of the questionnaire for teachers, teachers said the feedback could help them understand the insufficiency of teaching, make corrections accordingly and improve their teaching quality. Hence, management of both interns and host hospitals could be implemented at the same time by using the system. The system heightens their engagement and keeps them passionate. It also brings about a better quality internship by solving the problems those interns who lack continuous passion use in the study and that the host hospitals use to perform their teaching without sufficient responsibilities. In addition, such a management system could reduce a teacher's workload such that they can pay more attention to their teaching.[

For teaching administrators, it is of the highest significance that the system not only enhances the efficacy of internship but also making their work more effective. Both benefits were confirmed by the teaching administrators according to the results of the questionnaire. Since the system records all of the interns’ information, they can check on attendance and supervise teaching jobs in real time without geographical limitations.[

Lastly, based on the browser/server structure, the system is convenient and accessible. Users can log into the system with any browser.[ We have set up the entrance for not only the web page but also Wechat (a Chinese social media) so that interns can use the system whenever and wherever they are. However, some users have suggested that the system still needs to be simplified. For example, when writing the logs, full details on the patients are needed to be filled out, which seems unnecessary and over-elaborated, though this process, in fact, attempts to imitate the medical record. Intern users believed that they often forgot the information of their patients and would choose to fill in false information. Moreover, the processes involved in written records can be a waste of time and result in overload caused by the extra paperwork, which would lead to negative feedback and become an obstacle for the popularization of the system.[ Thus, we must think twice before we take any measures to improve the system. Another flaw is that, although the new system helped interns clear their learning aims, it seems impossible for them to finish all of the learning tasks. Interns stay in every department for only 2 weeks, during which they may not meet patients with certain diseases they need to learn and have no chance of performing specific procedures as a result of the geographical and seasonal constraints. These findings imply that how we supervise and manage the internship still needs further improvement. Nevertheless, most of the interns, teachers and teaching administrators highly praised the system, especially regarding its practicability.

Conclusion

Taken together, the new medical internship management network system generally meets the demands of internship management today regardless of the number of interns and the geographical constraints. Improving the level of theoretical knowledge and the practical skills of interns by strengthening the management has been shown to be helpful. We can speculate that this internship system could satisfy the needs of colleges, hospitals, and interns in heightening the quality of internship. The new system is not limited to the medical internship and is expected to be applied in various fields.

Author contributions

Conceptualization: Wenjie Guo, Zhirong Zeng.

Funding acquisition: Tszhei Fong, Zhirong Zeng.

Software: Anqi Lin.

Supervision: Peng Luo.

Visualization: Ruolan Qian.

Writing – original draft: Yating Chen.

Writing – review & editing: Oudong Xia, Jiaqi Ye, Anqi Lin, Peng Luo, Zhirong Zeng.

Peng Luo orcid: 0000-0002-8215-2045.