Perceived learned skills and professional development of graduates from a master in dental public health programme.

INTRODUCTION: Assessing the impact of a training programme is important for quality assurance and further development. It also can helps with accountability and marketing purposes. This study evaluated the impact of King's College London (KCL) Master of Science programme in Dental Public Health in terms of graduates' perceived learned skills and professional development.
METHODS: An online questionnaire was sent to individuals who completed successfully the KCL Master of Science programme in Dental Public Health and had a valid email address. Participants provided information on demographic characteristics, perceived learned skills (intellectual, practical and generic) and professional development (type of organisation, position in the organisation and functions performed at work before and after the programme). Learned skills' scores were compared by demographic factors in multiple linear regression models, and the distribution of responses on career development was compared using nonparametric tests for paired groups.
RESULTS: Although all scores on learned skills were on the favourable side of the Likert scale, graduates reported higher scores for practical skills, followed by intellectual and generic skills. No differences in scores were found by sex, age, nationality or time since graduation. In terms of career development, there were significantly higher proportions of graduates working in higher education institutions and taking leadership/managerial roles in organisations as well as greater number and variety of functions at work after than before the programme.
CONCLUSION: This online survey shows that the programme has had a positive impact on graduates in terms of perceived learned skills and professional development.

Introduction

Public health is a continuously evolving field, with multidisciplinary dimensions, and requires professionals competent in a variety of skills 1. With new public health challenges in a globalised world, such as health inequalities, re‐emerging infectious diseases and shortage of trained workforce, a need for competent public health practitioners has been widely advocated 2, 3, 4. Postgraduate programmes in public health, including those in dentistry, are designed to produce a skilful public health workforce capable of coping with the new challenges in public health 5.

Postgraduate public health programmes have proliferated over recent years. They vary between institutions in terms of their length, structure, content and teaching methods. Because of the number and variety of these programmes, academic institutions and prospective students have become increasingly concerned about the quality of the teaching and training they provide, particularly in terms of programme outcomes and impact 5. Training programmes can be assessed at four increasing levels of complexity; level one (reactions) measures how the person feels about the course; level two (learning) measures the extent to which principles, facts and techniques have been absorbed; level three (behaviour) measures the application of the principles and techniques acquired on the job; and level four (results) measures the ends, goals and results desired 6, 7, 8.

Alumni's perception of skills or competencies after graduation and the extent to which those skills have contributed to career improvement are common indicators of levels two and three in Kirkpatrick's evaluation framework 5, 8. Perceived skills and career progression have been used widely to assess the impact of postgraduate public health programmes because they provide valuable information for quality assurance and improvement 9, 10, 11, 12, 13, 14, 15, 16, 17, 18. In some cases, they are also useful for accountability, academic recognition (accreditation) and marketing purposes.

The Master of Science in Dental Public Health of King's College London Dental Institute (KCLDI) aims to produce a highly knowledgeable individual capable and skilful in dental public health. It was established in the eighties to support the emerging new specialty of Dental Public Health in the UK. The programme seeks to develop an understanding of the basic concepts of dental public health; the major health problems (and their determinants) of a community; the organisation of oral health services; research methods including epidemiology and statistics; and approaches to promoting oral health and preventing oral diseases. Students need to complete 180 credits over 1 year full‐time or 2 years part‐time. At present, there are seven core modules in the programme which have changed from time to time according to staff expertise and emerging trends in the field. The current six taught modules are introduction to dental public health; principles of epidemiology; research methods; social and behavioural sciences as applied to medicine and dentistry; oral health promotion and education; and planning and evaluation of oral health, each counting for 20 credits. There is also a research module of 60 credits for which students need to complete and submit an original piece of research work for their dissertation. Practical experience is gained across some dental public health competencies by participating in various dental public health placements including teaching, research, health services and health promotion activities 19. The programme has produced over a hundred graduates over the years. However, its impact on graduates is yet to be documented. The aims of this study were to evaluate perceived learned skills and professional development of graduates from the KCLDI Master of Science programme in Dental Public Health.

Methods

Participants

A total of 141 students have graduated from the in‐house Master of Science programme in Dental Public Health at KCLDI since 1981. The inclusion criteria for the study were students (i) who completed the programme successfully between 1981 and 2012 (those with at least 2 years of experience after graduation from the programme), (ii) who had a valid email address (where the recruitment email could be sent) and (iii) who agreed to participate voluntarily in the survey.

The study protocol was approved by KCL Biomedical Sciences, Dentistry, Medicine and Natural & Mathematical Sciences Research Ethics Subcommittee (Reference BDM/12/13‐62). Return of the completed questionnaire was taken as implied consent.

Data collection

A list of graduates, with their contact details, was not available at KCLDI and one had to be created for the study. We therefore proceeded to create group spaces in relevant social network sites (Facebook and Linked‐in) through which we could advertise the survey and regain contact with programme's graduates. These social media spaces were developed and run by the programme administrator, who acted as the gatekeeper during the data collection period.

Graduates who expressed interest in the survey were sent a recruitment email containing the details of the study, an invitation to participate and a link to the online questionnaire. The gatekeeper also asked participating graduates whether they could put her in contact with fellow graduates they knew would be interested in participating in the survey (i.e. snowball sampling). A reminder was emailed to all participants a week before the end of the survey period.

The questionnaire was developed based on previous relevant surveys 9, 11, 12, 13, dental public health competencies 20, 21, 22 and the programme specification forms 19. An initial draft was circulated amongst past and present teaching and administrative staff involved in the programme for assessment. After amendments, the questionnaire was piloted with 5 graduates (i.e. past students who were either working or in further education at KCLDI) for validity assessment. Suggestions for improvements from the pilot study were incorporated in the final version of the questionnaire. The final format of the questionnaire was developed using the internet‐based tool SurveyMonkey®. The questionnaire collected information on graduates' background (sex, age, nationality and year of graduation), perception of learned skills gained during the programme, satisfaction with and attitudes towards the programme and professional development. Respondents ranked their learning of 6 intellectual, 8 practical and 6 generic skills during the programme using 5‐point Likert scales (1 = definitely disagree, 2 = mostly disagree, 3 = neither agree nor disagree, 4 = mostly agree, 5 = definitely agree). Graduates' professional development was assessed in terms of the type of organisation, position in the organisation and functions performed at work before and after the programme. Two final open‐ended questions asked participants to describe their highest professional appointment in dental public health and proudest accomplishment in dental public health to date.

Data analysis

All analyses were carried out in SPSS® Statistics version 20 for Windows (IBM Corporation, Armonk, NY, USA). We first presented the distribution of participants' scores for individual learned skills using the mean, standard deviation (SD), median and range of values. Scores were then compared by sex, age groups (<35, 35–44 and 45 + years), nationality (British, South Asian and Other) and time since graduation (before 2000, 2000–2010 and after 2010) using multiple linear regression. Bonferroni correction was used to reduce false positive results due to repeated comparisons over multiple outcomes – 20 learned skills – 23, 24.

For professional development, we compared the distributions of graduates' responses on type of organisation and position in the organisation after completion of the programme (observed values) with those before joining the programme (expected values) using the chi‐square goodness‐of‐fit test. We then compared the number and type of functions performed at work before and after the programme. We created a summative score for the number of functions reported by graduates at each time point and compared them using paired t‐test. The proportion of graduates performing each function at work before and after the programme was compared using the McNemar's test. Bonferroni correction was used here too to correct for multiple comparisons – 10 functions – 23, 24.

Results

Fifty‐seven graduates met the inclusion criteria and 44 of them completed the online questionnaire (77% response rate). The characteristics of participants are shown in Table 1. Most participants were women (54.5%), younger than 35 years (45.4%) and from South Asian countries (45.4%). The average time since graduation was 8 years, with 38.6% of respondents graduated after 2010.

Table 1

Characteristics of participants in the online survey

Characteristic	Groups	N	(%)
Sex	Women	24	(54.6%)
	Men	20	(45.4%)
Age group	<35 years	20	(45.4%)
	35–44 years	13	(29.6%)
	45 +  years	11	(25.0%)
Nationality	British	9	(20.5%)
	South Asian	20	(45.4%)
	Other	15	(34.1%)
Time since	After 2010	17	(38.6%)
Graduation	2000–2010	15	(34.1%)
	Before 2000	12	(27.3%)

There was variation in graduates' scores on learned skills, with the highest scores for practical skills, followed by intellectual and generic skills (Table 2). However, mean scores were on the favourable side of the Likert scale and above the neutral point (score of 3). Understanding the underlying philosophy of dental public health (mean: 4.56; SD: 0.84) and the ability to adapt to change (mean: 3.80; SD: 0.98) were, respectively, the intellectual skills with the highest and lowest scores. The application of health promotion principles to practice (mean: 4.32; SD: 0.69) and carrying out basic statistical analyses (mean: 3.95; SD: 1.00) were, respectively, the practical skills with the highest and lowest scores. And the use of information and communication technology (mean: 4.15; SD: 0.88) and the management of manpower, resource or time (mean: 3.68; SD: 0.91) were, respectively, the generic skills with the highest and lowest scores. No differences in scores were found by sex, age group, nationality or time since graduation (multiple linear regression, P > 0.05 in all cases).

Table 2

Intellectual, practical and generic learned skills of graduates from the King's College London's Master of Dental Public Health programme

Learned skills	Mean	(SD)	Median	Range
Intellectual
Ability to appraise scientific literature	4.39	(0.92)	5	1–5
Understanding the philosophy of dental public health	4.56	(0.84)	5	1–5
Ability to reflect critically on my own work	4.15	(0.91)	4	1–5
Ability to adapt to change	3.80	(0.98)	4	1–5
Ability to be self‐motivated	3.83	(0.95)	4	1–5
Capacity to think critically	4.20	(0.90)	4	1–5
Practical
Apply the key principles in dental public health	4.29	(0.78)	4	2–5
Carry out epidemiological oral health surveys	4.02	(0.91)	4	2–5
Find existing evidence relevant to research question	4.27	(0.81)	4	1–5
Carry out basic statistical data analysis	3.95	(1.00)	4	1–5
Apply social/behavioural sciences theories to my practice	4.00	(0.81)	4	2–5
Plan and evaluate oral health services and programs	4.05	(0.80)	4	2–5
Apply health promotion principles to my practice	4.32	(0.69)	4	2–5
Develop a research project	4.24	(0.86)	4	1–5
Generic
Take appropriate responsibility	3.88	(0.98)	4	1–5
Communication skills	3.88	(1.00)	4	1–5
Use of information and communication technology	4.15	(0.88)	4	2–5
Management of manpower, resource or time	3.68	(0.91)	4	2–5
Engage in continuous professional development	3.90	(1.11)	4	1–5
Presentation skills	3.90	(1.02)	4	2–5

Graduates' responses on professional development are reported in Table 3. There were significant differences in the type of organisation graduates were working before and after the programme (chi‐square goodness‐of‐fit test, P < 0.001). A 20% decrease in the proportion of graduates working in private practice was accompanied by a 10% increase in those working in higher education institutions and 5% increase in those pursuing further training. There were also significant differences in graduates' position in their organisation before and after the programme (chi‐square goodness‐of‐fit test, P < 0.001). The proportion of graduates working as staff members dropped by 22%, whereas the proportion of graduates working as head of department in an institution/organisation increased by 13%. Furthermore, there were significant differences in the number and type of functions performed at work before and after the programme. Graduates reported more functions after (mean: 4.9, SD: 3.3) than before the programme (mean: 2.8, SD: 2.3) (paired t‐test, P = 0.002). Of the 10 functions evaluated, a significantly greater proportion of graduates reported conducting studies (McNemar test, P = 0.001), working on health communication, education or health promotion (P = 0.007), training students/staff (P = 0.004) and making health plans (P = 0.007) whereas a significantly lower proportion of graduates reported providing clinical services (P = 0.002) after the programme. Finally, 35 (79.5%) and 32 (72.7%) provided answers to open‐ended questions on highest professional appointment and proudest achievement to date. Becoming a lecturer (22.9%), consultant (17.1%) and senior/chief dental officer (11.4%) were commonly reported as the highest professional appointment, whilst conducting research studies (18.8%), epidemiological dental surveys (15.6%) and oral health programmes (15.6%) were the most cited proudest achievements.

Table 3

Professional development of graduates from the King's College London Master of Dental Public Health programme in terms of the type of organisation, position in the organisation and functions performed at work before and after the programme

	Before the programme	After the programme
Type of organisation (%)
At home	9.8	17.1
Private practice	26.8	7.3
Public sector health services	41.5	39.0
Higher education institution	17.1	26.8
Private organisation	4.9	4.9
Others	0.0	4.9
Position in that organisation (%)
Staff	86.5	64.7
Head of Department in Institution	10.8	23.5
Head of Department in Ministry	2.7	2.9
Head/Deputy Head of Institution	0.0	2.9
Other	0.0	5.9
Functions performed at work (%)
Providing clinical services	79.4	41.2
Administrative work	23.5	47.1
Conducting studies	5.9	47.1
Health communication, education or promotion	29.4	61.8
Evaluating/managing curative services	14.7	38.2
Conducting health programs	29.4	41.2
Training students/staff	41.2	76.5
Making plans	14.7	47.1
Consulting on policies	11.8	32.4
Managing programs/projects	29.4	55.9
Other	14.7	8.8

Discussion

An overall positive impact of the Master of Science in Dental Public Health programme of King's College London Dental Institute was reported by this group of graduates, in terms of perceived learned skills and professional development. Graduates felt the programme has helped them to enhance their intellectual, practical and generic skills, which was the main aim of the programme. The findings also suggest that the programme has helped graduates with career improvement.

Some limitations of this study need to be borne in mind when interpreting the present results. First, participants were recruited using non‐random sampling and the final sample size was relatively small. Although participation rate was high, we could not recruit participants from all calendar years (especially amongst older cohorts). Thus, the present results cannot be generalised to the full population of graduates and they may reflect the short‐term rather than long‐term outcomes of the programme. Second, learned skills were measured through graduates' self‐reports, which are prone to overestimation and measurement bias. Responses could be influenced by graduates' emotional commitment to their alma mater and gratitude for receiving a professional degree 25, 26. However, the ability for self‐evaluation is a defining characteristic of professional practitioners 8, 27. Therefore, self‐assessment of skills must be appreciated as they enable a public health professional to internalise these skills and facilitate addressing the health needs of populations 12. Third, we measured professional development as reported by graduates. Although it is recommended to include feedback from supervisors and peers to validate graduates' responses 9, 17, interviews with peers and employers can also introduce bias, due to interviewees giving socially desirable answers but also because graduates change jobs often or are given more responsibilities 5. Observation seems preferable but would probably be more complicated to achieve for international programmes.

Although perceived scores for all learned skills were high, there was some variation amongst them. Graduates felt the programme had helped them to enhance practical skills the most; a finding that is consistent with other studies on the impact of master's programmes of public health 9, 10, 11, 12, 13, leadership 14, 15, 16, 17 and primary care 18. Scores were somewhat lower for generic skills though. The programme provides students with opportunities to learn generic skills like presentation skills, use of information and communication technology, communication skills, management of manpower, resource or time, taking appropriate responsibility and engaging in continuous professional development; however, they are not formally assessed. Graduates may feel they had those skills before joining the programme or that they did not simply join the programme aiming to learn those. The present findings thus yield new insights into areas for further improvement in the programme.

The comparison before and after the programme showed clear changes in graduates' organisation, position and duties. A significant proportion of graduates moved to work in higher education institutions or to pursue further training (speciality training or doctorates) after completion of the programme, although the main proportion of participants continue working in public health sector services (including armed forces). These changes came along with taking up leadership/managerial roles in the public, private and voluntary sectors. Keeping in mind that the majority of respondents graduated after 2010, the long‐term impact of the programme may not be completely apparent due to graduates' limited experience and short time since graduation. Furthermore, graduates reported a broadened scope of their roles and responsibilities at work after involvement in the programme, particularly in terms of conducting studies, making health plans, working on health communication, education or promotion, and training students or staff. Taken together, the above findings indicate that the programme has improved the practice of public health dentistry by its graduates. However, future studies should evaluate how often graduates use those skills in day‐to‐day work activities.

The present results thus have implications for quality assurance and future research. This survey is a first step to address an important dimension of programme quality and the impact of educational programmes on public health dentists. The survey represents part of an ongoing, continuous quality improvement process being applied to KCLDI Master in Dental Public Health programme. This is the first study evaluating the impact of a postgraduate programme in dental public health 5 and has offered an approach to evaluate other postgraduate taught programmes in the UK and abroad. Future studies should also consider organisational barriers and challenges faced by graduates in the practical application of skills acquired through this programme.

Conclusion

This study shows that the Master of Science in Dental Public Health programme of King's College London has made positive contributions towards developing individuals skilful in the practice of dental public health whilst also creating opportunities for professional development of its graduates, beyond the intended improvements in their existing knowledge and competencies.