Quasi-experimental study on the effectiveness of a house officer preparatory course for medical graduates on self-perceived confidence and readiness: a study protocol.

INTRODUCTION: Being a house officer (HO) is said to be associated with high levels of stress, leading to mental health problems and sometimes to quitting the medical profession altogether. In Malaysia, the number of HOs completing training on time is slowly declining, with increasing annual dropout rates. Feeling incompetent is one of the contributors towards this growing problem. This study aimed to evaluate the effectiveness of a 3-day pre-HO intervention module in addressing participants' confidence, readiness and psychological well-being in preparation for their HO training. METHODS AND ANALYSIS: The pre-HO intervention is the 'Medicorp' module that includes clerkship, experience sharing, hands-on skills training, common clinical cases and introduction of the local healthcare system. This is a pre-post quasi-experimental study lasting 1 year, with three assessment time points-at pretraining, immediately after training and 1 month into the participants' HO-ship. The study is currently ongoing and involves 208 participants who attended the course in Malaysia. Participants with known psychiatric illness, working HOs and medical students are excluded. A pretested, self-administered questionnaire that includes baseline sociodemography, adaptation of the International Medical University (IMU) Student Competency Survey and the Depression Anxiety Stress Scale has been adopted, and 1 month follow-up will be conducted by telephone. Data will be analysed using SPSS V.24. The primary outcome is change in confidence level, while the secondary outcomes are changes in the readiness and psychological well-being of the participants. ETHICS AND DISSEMINATION: This study protocol has received ethics approval from Ethics Committee for Research Involving Human Subjects Universiti Putra Malaysia and the National Medical Research Registry Malaysia. Written informed consent has been obtained from each participant. Results will be disseminated through journals and conferences, especially those involved in medical education specifically looking into the training of medical doctors. TRIAL REGISTRATION NUMBER: NCT03510195. © Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

The intervention focuses on relevant skills required to increase the confidence level of medical graduates to work as house officers (HOs), expanding to training after they graduate.

Follow-up during the initial part of working as an HO in this study would identify if the intervention addresses the needs of a working HO.

Limitations include the use of non-probability sampling, which is expected to impose selection bias even though it is economical and logistically advantageous.

Introduction

Psychological problems among house officers (HOs) are a known worldwide issue. For example, in Norway, 11% of HOs are reported to have mental health problems needing treatment.1 In the UK (Midlands), 46% of HOs are said to have clinical depression.2 Similar percentages are yielded by local Malaysian studies, where 31%–58% of the HOs are reported to suffer from various psychological conditions. For example, 31% of HOs are reported to be distressed, 36.6% indicated having a high level of emotional burnout, and 34% and 58% reported experiencing a level of stress in Kuala Lumpur and Kota Kinabalu, respectively.3–5

Many studies have linked the causes of HO stress with performance issues. This can be related to dealing with patient demands, workload intensity, mental strain and feeling overworked.1–3 6 Other authors have described stressors among HOs as ‘coping with diagnostic uncertainty’, ‘perceived lack of skills’, ‘fear of making mistakes’ and ‘feeling insecure’.1 3 5–7 Authors of a more recently published study conducted in the UK also reiterated that HOs felt less confident in their knowledge and skills to perform during their initial phase of training.8 This touches on the issue of HOs’ perceived confidence during training, as some felt unprepared for the work ahead, which includes clinical procedures, work demand and clinical knowledge.9 Reduced confidence has been reported to affect students’ psychological well-being, such as feeling anxious or distressed, especially during clinical training.6

There are many consequences of this psychological impact on HOs for the healthcare system and nation as a whole. The stress experienced by HOs in Malaysia is reflected in the decreasing numbers that complete training within the allocated timeframe of 2 years. In 2009, 86.4% of all HOs completed training in 2 years, whereas only 58.8% of them did in 2012.10 The dropout rate, which is defined as HOs not completing their training within 5 years, is said to be slowly increasing from 3.7% to 4.8% per batch year.10 High stress levels are likely to affect thoughts of quitting HO-ship by up to three times, as evidenced in 2016, when 1.2% of HOs either were terminated or quit due to the inability to cope with stress.4 10 This, in turn, leads to the economic burden associated with the high cost of training future doctors.11 12 The issue of stressed HOs also leads to questionable patient care if managed by those having psychological problems.2 13

Currently, in Malaysia, the waiting time to commence work from the time students graduate can be a few months, but it may take up to 1 year; the average is 6 months.10 The long waiting time can be attributed to the sudden increase in the number of medical graduates trained locally via private institutions, as well as those trained overseas.10 Hence, it is during this time that medical graduates turn to independent HO Preparatory Courses offered by independent bodies to address the aforementioned issues.14

Medicorp is a company that offers regular training up to 10 times per year, with 50–100 participants attending each course for the past 3 years. Medicorp takes into account that peer training is one of the top learning methods preferred by junior doctors, alongside textbooks and online materials.15 They are followed up after course completion via social media networks and social media application groups for further support.

Preparatory HO training is very scarce in Malaysia, especially programme aimed at helping HOs to be better prepared, motivated and familiar with the healthcare system. Extant research on these training modules is also highly limited.

We aim to evaluate this peer-led course to assess its effect on medical graduates’ self-perceived confidence and readiness, to better prepare them for HO training. With an improvement in these factors, it is hoped that the psychological stressors among junior doctors just starting work will be reduced, thereby giving them a head start in becoming more motivated doctors. The results of this study will further help to refine this module to be more comprehensive and effective. In addition, the adapted questionnaire can be used to create an assessment tool for future training modules.

Methods and analysis

Study design and setting

This pre-post quasi-experimental study is conducted over a 12-month period in Kuala Lumpur, Malaysia. Participants come from all over Malaysia, and the study centre is equipped with a lecture hall and boarding facilities. All study participants are HO preparatory course attendees, and their perceived confidence and readiness are evaluated at three assessment time points: at baseline (before the course), immediately after the course completion (only for level of confidence and readiness) and 1 month after starting work as an HO. No control group is included due to resource constraints.

Study participants

Participants who attend the Medicorp HO Preparatory Course are recruited and followed up from April 2018 to March 2019. The House Officer’s Preparatory Course was initially organised in early 2011 by a medical non-governmental organisation to address the above-mentioned issues. Since then, it has evolved and has been privatised to Medicorp, a company that specialises in junior doctor training, run by medical officers. The module has been further refined through feedback from participants, speakers and organisers.

Medicorp takes into account that peer training is one of the top learning methods preferred by junior doctors.15 Hence, this training course relies on feedback from participants and trainers, most of whom are its alumni. They are followed up after the course by the organisers via social media networks, such as WhatsApp groups, for further support. Per year, approximately 1000 participants join this training course. It is conducted around 10 times per year, with approximately 100 participants each time.

Eligibility criteria are based on the following:

Inclusion criteria

Participants who have registered to attend the Medicorp HO Preparatory Course.

Exclusion criteria

Participants who declared suffering from a psychiatric illness.

Participants who have not completed a medical degree (medical students).

Participants already working as an HO.

All individuals who fulfil the eligibility criteria and agree to participate are included in the study sample. Recruitment of participants is summarised in figure 1.

Figure 1

Flow of the recruitment and follow-up of participants.

Sample size

Sample size was calculated using G*Power 3.1 sample size calculator software. It is based on the confidence score reported in a previous study analysing pre-emergency and postemergency department junior doctor posting.6 The mean overall confidence score was 56.48 (SD=24.67) at the end of the first month and 62.78 (SD=28.69) at the end of the fourth month.6 Thus, for the present study, the estimated sample size was 208 participants after accounting for 80% power, a significance level of 0.05% and 30% attrition.

Intervention

The intervention in this study is the Medicorp HO Preparatory Course, which comprises a 3-day training programme. Content includes aspects of HO training that are needed for HOs to function in the workplace, in particular the nature of the HO job; explaining technical details such as the shift, on-call system and tagging period (where HOs in a new posting are required to follow a more senior HO for a timeframe determined by each specialty to help HOs adjust to their work scope); and assessments that HOs need to undergo during their HO training. The trainers are specialists in training, or medical officers, as well as HOs who come to share their experiences. Medicorp encourages their alumni to be involved in their training programme. The module, as described in table 1, is held every 1–2 months. Module content is moderated by Medicorp based on discussions with the board of directors, input from advisors and participant feedback.

Table 1

Intervention module

Day	Programme/ Lectures
Day 1	Everything You Need to Know about HO-shipContract HO, Choosing Hospitals and first Department, Tagging, Flexihour vs Oncall, Assessment and Extension
	The Straits Times
	Reviewing and Presenting Cases as an HOQuiz
	Excellent HO Forum
Day 2	General Clerking, Common Labratory Forms and Referring
	Surviving PaediatricsQuiz
	Designing Your Future; Further Career Options after HO-ship
	Doctors and Finance
	Balancing Family Commitments during HO-ship
	Express Physical Examination for HOs
	Assisting Surgery as an HO
	Excellent HO Traits
Day 3	Attending Unstable and Collapsed PatientsQuiz
	Requesting Scans from the RadiologistQuiz
	Practical Session Briefing
	Lunch
	Practical Training(Branula insertion, Venopuncture, CBD insertion, CPR training, Basic Suturing Skills)
	End of Programme

CBD, Continous Bladder Drainage; CPR, Cardiopulmonary Resuscitation; HO, House Officer.

The training programme takes the form of lectures with a simple quiz exercise at the end, along with a hands-on training session involving a mannequin. Before commencing the course, participants are included in a WhatsApp group for easier content sharing and course updates. They are encouraged to continue this networking even after course completion. HO hospital placements in Malaysia are conducted online and are opened for registration at specific times of the year via the e-Housemen website (ehousemen.gov.my). Future HOs need to register online and choose their preferred hospital placements, and the system will try to match their requests, where possible. Medicorp course attendees receive additional guidance in this process via Facebook and WhatsApp, specifically on reminders of the time to register and the choice of training placements. During the time of job commencement, Medicorp can use their database to guide participants into different WhatsApp groups, according to their place of work, for additional support.

Measures

The outcome measure will be determined via analysis of the responses given in the self-administered questionnaires by the participants. The primary outcome is competency, which involves previous clinical experience and confidence. The secondary outcome is readiness. There are several validated questionnaires assessing medical graduates’ confidence, readiness and preparedness for work. Understandably, these tools mainly aim to assess the undergraduate curriculum to determine if it prepares medical graduates for their working environment. This differs from our assessment of a short, preparatory HO course after graduation.8 9 16–21 We thus chose a questionnaire that focuses on preparation for the HO role, specifically prior to the commencement of work, and that has been validated for use in the local setting. Therefore, we adapted the IMU Student Competency survey, as the tool was used by other authors to assess preparatory programme, specifically in preparation for HO-ship.19–21 It is a valid and reliable tool, with a Cronbach’s alpha of 0.92 and intraclass correlation coefficient of 0.88–0.95.21 Content validity was also established by a panel of academic clinicians across seven disciplines, who are also student supervisors.

The IMU Student Competency survey is used to assess perceived competence, estimated experience in a range of skills and work readiness. It comprises self-perceived confidence in generic skills (7 items) and practical skills (15 items), estimated experience in practical skills (16 items), self-perceived competence in personal skills (7 items) and work readiness (2 items, 1 relating to the most daunting aspect and 1 to overall work readiness, respectively). All the items require responses on a Likert scale ranging from 1 to 5, with the exception of competence in personal skills, where the scoring is dichotomous (comfortable vs uncomfortable) and two points are given for ‘comfortable’ and one for ‘uncomfortable’. The marks on each subscale are totalled, whereby a higher score indicates better results.

For the adaptation of the questionnaire for this current study, content validity was assessed by specialists in the Ministry of Health and academics in local public universities involved in HO training. All decisions made to omit and add items were based on what the Medicorp HO Preparatory Course teaches as a module, taking into consideration its limited duration and resources. Some of the questionnaire items were more appropriate for assessment of the undergraduate curriculum.

As per the original questionnaire, the estimated experience in practical skills remained the same (16 items). Three items from the self-perceived confidence in generic skills section were removed—counselling patients/relatives on common diseases, answering questions from patients/relatives on admission and prioritising cases to be seen—as the module did not address these skills. An item on making management plans for new admissions was added. Therefore, the scale assessing self-perceived confidence in generic skills comprised five items.

Seven items assessing confidence in practical skills were omitted, namely the insertion of intravenous lines and blood taking for paediatrics; administering medications via intravenous, by intramuscular injection and per rectally; and the handling of blood containers, because these skills were not taught in Medicorp’s HO preparatory module. The item ‘performing an ECG’ was also excluded, as Medicorp has a separate module on ECG. Five items on practical skills were added, pertaining to confidence in the insertion of the urinary catheter for both men (one item) and women (one item), performing a basic suture and surgical tie (one item), conducting a comprehensive review of patients during ward rounds (one item) and referring cases to another department (one item).

Personal skills were assessed as confidence, and the seven items were reduced to five, whereby those pertaining to finding ward routines/protocols unaided and managing time on and off work were eliminated, as these topics were not taught in the course. The dichotomous choice from the original questionnaire was changed to a five-point Likert scale for uniformity in the assessment of confidence levels throughout the questionnaire used in the present study. In addition, one item for assessing overall confidence to start work the next day was added in order to evaluate the participants’ sense of confidence as a whole.

We retained the two original questionnaire items related to readiness, pertaining to the most daunting aspect and overall work readiness. All subscales were also scored on a five-point Likert scale, except for the most daunting aspect, where the participant is instructed to choose one answer. Hence, the adapted questionnaire for this current study only assesses self-perceived confidence and readiness, but not competency, as mentioned in the original questionnaire. The adapted questionnaire was pretested and the Cronbach’s alpha of all the subscales ranged from 0.92 to 0.96.

Psychological well-being is assessed using the Depression Anxiety Stress Scale-21 (DASS-21). It is a set of three self-reported scales aimed at assessing depression, anxiety and stress. It is a valid and reliable tool, with a Cronbach’s alpha of 0.96–0.97 for DASS-Depression, 0.84–0.92 for DASS-Anxiety and 0.90–0.95 for DASS-Stress.22 Each of the three scales has seven items requiring responses on a four-point Likert scale (ranging from 0 to 3). The scores for each of the scales are categorised into normal, mild, moderate, severe and extremely severe. The higher the scores, the more severe the condition. The outcome measures are presented in table 2.

Table 2

Outcome measures

Outcomes	T0	T1	T2
Confidence	x	x	x
Readiness	x	x
Psychological well-being	x		x
Additional information	Sociodemographic questionnaire Past clinical experience		Current workplace and posting Any suggested course improvements

T0=before the course; T1=right after course completion; T2=1 month after starting work as a house officer.

Procedures

A pretested, self-administered questionnaire is used to collect information from the participants. The questionnaire at baseline includes sociodemography, clinical experience, personal skills, confidence level, readiness level and the DASS-21.

Immediately after the participants have completed the course, their level of readiness and confidence is assessed via a self-administered questionnaire on the last day of the course. The participants will be followed up 1 month after they have been working as an HO in their respective hospitals and will be contacted by telephone. The organisers will continue to keep track of placements of all participants via social media applications, given that a component of the course is maintaining connections and informal training after the course has ended and as participants start the process of job application and employment. The questions asked during follow-up will probe into the participants’ level of confidence, readiness, DASS and workplace information (which hospital and posting), as well as the open-ended question: ‘Any suggestions to improve the course based on your current working experience?’ They will be given a copy of the questionnaire via email or the WhatsApp application to facilitate the interview process.

Possible determinants/confounders

Possible determinants/confounders may additionally be taken into account in this study. These will include sociodemographic variables, past clinical experience and working experience once the participants start working. Data collected will be based on participants’ self-reports. Finally, the working conditions of the participants in terms of qualitative data, which is an open-ended question, will be assessed during the final follow-up via telephone.

Planned statistical analyses

Data will be analysed using IBM Social Package for Social Science (SPSS) V.24. A descriptive analysis of participant demographic characteristics, clinical experience and baseline level of confidence, readiness and psychological well-being will be reported using means and SD or median and IQR for continuous variables (depending on the data distribution), and as frequencies and percentages for categorical data. Comparison between participants who completed and withdrew from the study will be made using the χ2 or Fisher’s exact test (for unbalanced data) for categorical variables and independent t-test for continuous data.

A repeated measures analysis of variance will also be conducted to determine intervention effectiveness within the groups across the study periods (baseline, immediately after the intervention and 1 month after starting work). Controlling for baseline measures will be done to determine any changes in the measured outcomes (level of confidence, readiness and psychological well-being) over time.

Patient and public involvement

This training module was designed by Medicorp based on feedback from its alumni (comprising doctors after completing HO-ships) in terms of improving content. Qualified specialists and specialists in training were asked to comment on the content of the module to make further improvements. A few of them were involved in the design and conception of this research study. Members of public and patients were also not involved in the development of the training module, nor the design and conception of this study. Final study results will be shared with stakeholders.

Ethical considerations

This study’s approval for ethical clearance was obtained from the Ethics Committee Involving Human Subjects Universiti Putra Malaysia and from the National Medical Research Register, Malaysia, and the Medical Research and Ethics Committee, as the participants will be working in Ministry of Health facilities during the 1-month follow-up. This study is also registered with the National Institutes of Health as trial registration.

Informed consent is obtained from each study participant, and they will be told that they reserve the right not to respond to the questions they do not want to respond to and can withdraw from the study at any time. All data obtained will be kept confidential, will be used for research purposes only and will not be shared. The data obtained will not affect participants’ future work prospects, as all information will be kept confidential. No personal data will be shared on social media platforms. Personal questions will be asked confidentially via telephone.

The benefits of the study include assessing issues in relation to HO well-being and determining the training needed for a functional HO. The potential risks, discomforts and inconvenience are negligible. However, should the DASS score be suggestive of depression or anxiety, the research team members will refer the participant appropriately. Should participants choose to withdraw from the study, they will be allowed to do so.

Dissemination

Results of this study will be disseminated by publication through peer-reviewed professional and scientific journals, as well as via presentations at meetings and conferences focusing on medical education and/or the psychological well-being of doctors. Participant data will be kept confidential and will not be shared with the public. If there are requests for data sharing for appropriate research purposes, this will be considered on an individual basis after trial completion and after the publication of the primary manuscripts.