Reforming medical regulation: a qualitative study of the implementation of medical revalidation in England, using Normalization Process Theory.

Introduction

Medical revalidation requires all licensed doctors to demonstrate they are up to date and fit to practise. Introduced in the UK in 2012,[1] it involved changes to the way organizations manage medical performance, and to the relationships between doctors, their employers and the professional regulatory body. Extending regulatory oversight of doctors’ practice throughout their post-qualification careers, revalidation has brought professional regulatory activity and oversight formally into the organizational sphere for the first time, providing considerable statutory legal powers and duties which fall upon doctors’ employers. Revalidation has repositioned where regulatory practices occur, now taking place in contexts in which complex and varied managerial and governance systems exist.

Before the introduction of revalidation, the UK General Medical Council (GMC) mainly regulated doctors’ practice and behaviour after qualification and admission to the register through its fitness to practise procedures in instances of alleged misconduct.[2] Employers in the National Health Service (NHS) mostly dealt with performance concerns informally, and their links to the professional regulator were limited.[3] This form of regulation became increasingly challenged, which resulted in debate over regulatory reform[4] that was further shaped by a series of high-profile incidents of gross medical misconduct in England.[5,6] In response, the governance of the GMC was reformed while fitness to practise proceedings were also changed, along with the introduction of arrangements for independent adjudication and appeals against unduly lenient decisions.[7-9] In parallel, and after an extended period of contentious policy development, a new system of medical revalidation was introduced, representing the most significant and far-reaching element of these regulatory reforms.

Revalidation is a continuing competency system carried out in five yearly cycles. All licensed doctors who are not in training must take part in the annual appraisal for which they need to provide and reflect on, among other things, continuing professional development (CPD), significant events, review of complaints and compliments, quality improvement activity and feedback from colleagues and patients. They have to be connected to one organization (‘designated body’), a ‘Suitable Person’ or report back to the GMC directly. Designated bodies have to provide a senior doctor as Responsible Officer (RO) who ensures that every doctor has a regular appraisal, is supported by an up to date appraisal system with sufficient trained appraisers, and that clinical governance systems are in place that provide supporting information, policies and systems for identifying and responding to concerns about doctors. Organizations have to link with other organizations where their doctors also work, so that information about their practice can be shared. The RO can make one of three possible revalidation recommendations to the GMC: revalidation, deferral (requesting the GMC to give more time for a recommendation to be made) or non-engagement (which can lead to the doctor losing their licence to practise).[10]

Our earlier work on the initial stages of the implementation of revalidation suggested that senior leaders gradually recognized the benefits of revalidation and became more accepting of it, but that revalidation had yet to be fully embedded, and scepticism about its value remained.[11] Little is known about the experience of those operationalizing revalidation ‘on the ground’, however, with formal evaluations of the introduction of revalidation in organizations lacking. Also, at the time of our earlier study, not all doctors had experienced revalidation. Indeed, the first full five-year cycle of revalidation was completed in 2017 only, making this paper, which analyses the implementation of how doctors and organizations experienced revalidation, particularly timely.

As in our previous study,[11] we used Normalization Process Theory (NPT) to help understand the implementation process through analysing interventions in relation to the work people do to implement them.[12] NPT has been successfully used to examine a wide range of clinical working practices, making it particularly applicable to the study of revalidation.[13] We used NPT to assess the implementation of revalidation across a wide range of organizations, and from the perspective of those people, clinical and non-clinical, operationalizing and ‘doing’ revalidation on the ground. Revalidation is a practice, and the medical profession’s acceptance and attitude towards it is crucial for implementation. NPT provides a lens to investigate the acceptability of this change and gain an understanding of how willing people are to engage in and adapt the practice of revalidation both in the medical profession and the organizations in which they work. We thus extend the existing use of NPT, which has predominantly been used to study the introduction of new clinical techniques, guidelines or practices at a micro level, by applying the theory to an organizational intervention introduced as a result of national health policy and legislation.[14,15]

Methods

This study draws on qualitative data to understand how revalidation has been implemented during the first five years. Ethical approval for this study was awarded by the University of Manchester Ethics Committee (REC 15028). Fifteen health care organizations were recruited through purposeful sampling following a national survey of ROs, to ensure coverage across types, settings, performance (appraisal rates; self-assessment) and geography.[16] In contrast to our previous study, which focused on policymakers and senior national professional leaders, this study centred on clinical and non-clinical staff who were directly involved in, or who experienced impact from, the implementation of revalidation in their organization.[11] We generated a list of potential interview participants based on information from organizational policies, board reports and role descriptions relating to revalidation or the management of medical work, resulting in 80 interviews with 79 participants, with one participant interviewed twice. Very few participants declined to be interviewed; when this did occur it was because individuals understood their role to not be relevant to the study, and predominantly coincided with the individual in question referring us to a more appropriate colleague. Roles held by participants included those with specific or obvious revalidation duties such as ROs, appraisal leads and appraisers, but participant recruitment also extended across wider organizational management structures including directors of fitness to practise, human resources, and professional practice, complaints managers, practice managers and medical directors. Twenty-one participants worked in primary care organizations and 58 in secondary care. The recruitment of participants varied according to organization type and size; it was guided by reviewing organizational structures and policies and drew on a snowball strategy or referrals from other interview participants. Participants were approached either via email or phone, depending on the contact details provided by gate keepers in each organization. The study and nature of data collection were fully explained and a participant information sheet and consent form provided in order to ensure participants could make an informed decision on whether or not to part take in the research. All participants provided written informed consent.

Interviews were carried out between 2016 and 2017 by seven researchers. Initially interviews were conducted in pairs to help provide consistency as well as an interview schedule being used by all researchers. Following this a combination of individual and paired researcher interviews were carried out with regular debriefing and discussion meetings to reflect on interviews and the interviewing experience. The research team developed an initial interview guide based on reviewing organizational policy and job description documents and from reviewing literature on the management of medical performance. Having conducted initial analysis on the first round of interviews, we developed a second interview guide to further explore emerging themes and to fill gaps in the data to reach a point of apparent data saturation.[17] Interviews were conducted in person or by telephone, according to participants’ preferences. All interviews were digitally recorded, transcribed and imported into Dedoose qualitative analysis software.[18]

We developed a coding framework from the four domains and sub-domains of NPT (Table 1), using an adapted version of the NoMAD instrument, which was developed to assess implementation processes.[13] This instrument was then applied to transcripts by coding evidence of the sub-domains. Four researchers analysed the transcripts independently. Dedoose was used to enable blind coding verification of code application to check consistency of analysis. Coding and interpretations were discussed at regular intervals throughout the analysis phase of the study during collaborative meetings with all authors.

Table 1.

Adapted NoMAD instrument.

Domains	Sub-domains	Sub-domain application to revalidation
CoherenceParticipants’ understanding, sense making and valuing of an intervention	Communal specification	Do participants have a shared understanding of the purpose of revalidation?
	Differentiation	How does revalidation differ from usual ways of working?
	Individual specification	How does revalidation affect the work for participants?
	Internalization	Can participants see the potential value of revalidation?
Cognitive participationCommitment and engagement by participants	Activation	Are participants willing to support revalidation?
	Initiation	Are there key people who drive the revalidation forward and get others involved?
	Enrolment	Are participants open to working with others in new ways for the purposed of revalidation?
	Legitimation	Do participants believe that being involved in revalidation is a legitimate part of their role?
Collective actionThe work participants have to do to make the intervention function	Interactional workability	Can participants easily integrate revalidation into their existing work?
	Contextual integration	Are sufficient resources available to support revalidation?
		Do management adequately support revalidation?
	Relational integration	Does being involved in revalidation disrupt working relationships?
		Do participants have confidence in other people’s ability to carry out revalidation?
	Skill set workability	Do participants believe work is assigned to those with appropriate skills to carry out revalidation?
		Is sufficient training provided to enable participants to enact revalidation?
Reflexive monitoringThe evaluative work people do to assess and understand the ways that a new set of practices affect them and others around them	Systemization	Are participants aware of reports about the effects of the revalidation?
	Communal appraisal	Do participants agree that revalidation is worthwhile?
	Individual appraisal	Do participants value the effects revalidation has on their work?
	Reconfiguration	Is feedback about revalidation used to improve it in the future?
		Do participants modify how they work with revalidation?

Source: Adapted from Finch et al. 2015.13

Results

We found evidence of all the NPT domains in our data. The degree and form of each domain’s presence provided insight into how the implementation of revalidation was experienced and operationalized in practice. It also highlighted how successful or not the organizations we investigated were at embedding the policy and securing acceptance of revalidation within the medical profession. Table 2 provides a selection of interview excerpts to illustrate and evidence the presence of each domain.

Table 2.

Analysis of interview data by Normalization Process Theory (NPT) domains: sample excerpts.

Domain and sub-domains	Sample excerpts
Coherence
Communal  specification	The purpose of revalidation is pretty straightforward. It’s embedded in the legislation. It’s essentially about protecting the public. (Joint Interview 1–3: Managing director, Operations director, RO)It's about the accountability, it's about the profession knowing that they are periodically, you know, they have to account for their practice. For me it's about that – it's that addition to the assurance process that they are competent to do the job, you know, fit and proper, competent to do the job that they are employed to do. (Int 42 – Director of HR)
Differentiation	We check doctors when they come into the trust now, we get our own exit information. We're much better at pre-employment checks. We have databases and systems now that will monitor all that… We've gradually just tightened it and tightened it, and the recording of everything and the documentation of everything really. You know, if someone says to me, where's someone's appraisal form for three years ago, I can produce that now, whereas before we couldn't. (Int 47 – Revalidation Manager)Before we had that process the only other route of referral was into performance and so I think we’re certainly resolving things quicker, faster and I think we see less performance concerns arise because certainly the quality of appraisal in this area is really high. (Int 58 – Head of Revalidation)
Individual  specification	The RO role means that you now have a seniorish management position looking at appraisal, in a way that they weren’t before, and perhaps looking more globally at complaints and intelligence and stuff, in a way that wasn’t looked at globally before. (Int 33 – AMD for Revalidation)Because of revalidation or because the RO role has been so well defined all these things [performance and concerns info] are brought together into one so that I'm able to oversee a lot of things that otherwise will not have been. So there is that type of tightening as well, it is worth bringing things together. (Int 43 – RO)
Internalization	I think there's a real acceptance now of it but I also think that doctors are seeing it as a really good way of reflecting and using that information to enhance their practice anyway… I think people are beginning to see the benefits of doing it. Certainly, there doesn’t seem to be anywhere the noise in the system now as there was when it first came out. (Int 26 – Deputy Director of Quality & Compliance)Revalidation is an admin function. That’s all it is. The revalidation itself and when it actually occurs is not at all important. What is important is that you’re doing your appraisals. The appraisal is all that matters. (Int 15 - Recruitment Director)
Cognitive participation
Activation	It’s very positive … our senior appraiser team are very positive about the team we have here, and how we’ve actually supported everybody, going forwards, and how we’ve worked consistently to ensure that doctors are supported … It’s been a very positive transition. There are plenty of doctors who really hate appraisal and revalidation. Interestingly, a lot of those were some of the older doctors … it’s transition, but for some of them who’ve found it very difficult, I think they’ve also had very positive feedback from us, because we have been supportive, we have tried to make sure that, you know, if they can’t have an appraisal, that they understand that it can be deferred, and they can do the evidence, that they can present, and we’re very, we try and do our utmost to ensure that they have a high quality appraisal, that’s supportive, but that doesn’t cause harm. (Int 62 – Senior Appraiser)What I’m finding in cause for concerns now is when we call the GP in after a complaint they’ve already done it. They already come. I had a GP two weeks ago who’d had quite a serious issue go through the GMC. He’s trying to get back on our list. We called him in. There it was, the reflective template, his statement, his learning, it was all there, whereas before that had been a battle to get it all there. And that’s the assurance NHS England ultimately looks for, to know that this GP’s learned, moved on, and now is safe to practice. (Int 60 – Senior Project Officer)
Initiation	Having the dedicated RO, who is the ex-medical director, has helped this organization because obviously he's been a position of authority, he’s well respected, he's very experienced. I think that’s helped in terms of the respect that, you know, people have for him… we had a couple of issues around this sort of early doors but I think the sort of strength of leadership from the medical director and the RO with [name] I think I've managed to sort of the majority of that. (Int 41 – Chief Nurse and Executive Director of Operational Clinical Services)We are really lucky in terms of resourcing that we’ve empowered our senior appraisers… they drive forward a number of different programmes of work around the appraiser training, the quality assurance for appraisals. I think that’s a really helpful and clever way to do things, because they take ownership of that and actually it’s then clinically led. So it’s led by the clinicians who are out in the patch working who can relate to the people they’re appraising, and the appraisers that they’re supporting as well (Int 58 – Head of Revalidation)
Enrolment	We’ve made quite a lot of changes. We’ve put in place a different leadership structure, so we’ve got myself, who supervises a number of Clinical Directors, who supervise a number of Clinical Leads, who supervise a number of Consultants. The line management stuff, the governance that comes through line management, now is aligned to revalidation and appraisal. (Int 20 – RO)In the first couple of years it was very hit and miss about getting information from other organizations. But as, I think over the last 12 months, especially, because I think it's been promoted by NHS England in the events, that we should be sharing information, we are getting that on a more regular basis. And the agencies are getting better. Because we quite often get doctors that we recruit from the agencies. So we are getting a lot more of the transfer information …you can request a reference from the revalidation, they call it revalidation reference. (Int 39 – Revalidation Manager)
Legitimation	We got slightly that initial, this is just a tick box exercise, why do I have to do participate in this? And I think as a generic move towards…yes, this is positive, I see I can do this and I can see it adds value to my career. So I think broadly now the majority of both doctors and appraisal staff would say it's a good thing. (Int 18 – Former RO)We had an appraiser group last year where we were banging on about whole practice appraisal forever. And they were talking one day about one of our appraisers had seen a doctor who has a private practice. He's very open about it. And she said to him, how many patients a year do you see? And he said to her, that's none of your business. (Int 47 – Revalidation Manager)
Collective action
Interactional  workability	When I joined there was absolutely nothing in place whatsoever, apart from an RO. So I literally had to start from scratch. And that meant, the first thing was to find out which doctors, within [organization], belong to us, and who are our connected doctors. So, it was a long process. I had to write out to all the hospitals… So, I got hundreds and hundreds of contacts back. And then [name] and I put together a letter with some questions, introducing ourselves and what we were doing, and this was sent out to all of these doctors, across the country, and across all the [organization] sites, asking them to complete and return to me. And that’s what we first did. So, then the list started to come together, so I had just a long list of who this doctor was, and where they practice, their specialty etc. And it was only then that we started to look at systems out there (Int 77 – Consultant Liaison and Revalidation Manager)So, for us here in [area], is was just a natural transition. We’d been planning for it, preparing for it, and when it finally came in, everyone’s going, thank goodness for that we’ve waiting ten years, we’re now here. And it allowed us to strengthen our systems as well, because it gave us, for the majority of doctors, it was a terrifying prospect, now most of them have been through it and understand it, it’s not so terrifying and they understand it. (Int 59– Programme Manager)It’s an administrative burden for doctors to do it well, that’s undoubted, even if he’s gathering information throughout the year, to write up a reflection. There isn’t that sort of amount of time spare to do it well, so it does rely on people taking timeout of their personal life. (Int 71 – RO)
Contextual  integration	I think the trust invested in appraisal from the start, so they didn't have to invest significantly more since the revalidation apart from endorsing what needs to happen. And because me and the other medical director we sit on the board and, in fact, the HR director sits on the board as well, we are well placed to actually highlight any issues. And if I said to the board, look, I need more money for revalidation purposes, they won't say no. … when we had that person who was revalidation appraisal and education lead, when we separated the two we ended up spending about 30-40 additional thousand pound to dedicate, just to say that that person is responsible for revalidation appraisal. So the trust has never shied away from actually supporting that purpose. (Int 49 – RO)It's at a stage where if anyone goes off sick, which inevitably happens, there just isn't the slack in the system anymore. Last year there was a 30 per cent cut in budgets for staff and the impact has been quite considerable, and the impact and stress on staff is significant. So if one person goes off ill it has a knock on effect on everyone else and the system does slow down. What we haven't got enough resource for … is the performance concerns that sit alongside all of this. (Int 57 – RO)
Relational  integration	When I started, some of the appraisees were unsure about what documentation they needed to put forward and how often, so like audits, how often they needed to do audits. I think there wasn’t the scrutinization as much, of the appraisal documentation, but since we’ve got a Responsible Officer on board he scrutinizers, so that’s improved. (Int 44 - Appraisal Administrator)I’ve got one person who manages the revalidations here, she takes the lead, does all the work, she’s very knowledgeable … if someone leaves, that’s all lost. And that’s where it becomes very difficult, when you don’t have more staff, because you’ve got no succession planning. (Int 59 – Programme Manager)
Skill set workability	We had to implement it from scratch, but it was not that difficult once we’d got the system going. Inevitably we were getting lack of understanding; people didn’t know much about it. But then, the way the GMC introduced it, was quite sensible, they let us select, for the first six months or so, who went forward for revalidation, so we could choose the people who actually went up for it. And then we put some of the more difficult cases for the latter part of the year and the following year. The ones we thought were fairly ahead of the game, so we got into it more gently in terms of revalidation (Int 74 – RO)We had some issues around identifying doctors who could carry out the revalidation, and I know we sort of struggled with that for a little while when it first was implemented, but I'm not aware of any further problems on that at all now… It was actually people being trained to be appraisers and their availability to do the training. (Int 27 – Deputy Director of Quality & Compliance)
Reflexive monitoring
Systemization	We carry out our audits of our revalidation process… around actually looking at the output of those revalidations and then collectively seeing if there's any sort of themes and things coming out of those that we can use for learning across the charity. Then, of course, what we haven’t done yet … is then triangulating that with outputs from the other quality information that’s available. (Int 27 – Deputy Director of Quality & Compliance)We had to get the numbers of appraisals up, and then we had to get the quality of the appraisals up. And some of that was about peer support so that the best appraisers were coaching the less good appraisers to get better, the use of external training. So we brought a trainer in to: what does a good appraiser look like? … We formed a quarterly appraisers' group, which was, you know, can somebody tell us some things about good practice and appraisals? Can some people discuss some areas where they're struggling with? And then more recently starting to look at a bit more formalizing. Actually can we measure what we mean by a good appraisal?… So we sit down and say: let's have a look at half a dozen round of appraisals, let's score them and say, well, why does that one score higher than that one, what can we learn from that? (Int 18 – Former RO)
Communal appraisal	We’re actually doing an end to end review of this one because there were some really serious patient safety concerns here relating to clinical practice and it hasn’t been flagged by anybody in the system at all. So we did the ones that we thought were higher risk at the beginning of our programme two years ago. This is towards the end of our programme. How have these two people actually continued to do this and put patients at risk? (Int 53 – Head of inspection – CQC)I fully endorse revalidation, I think it's absolutely essential. Obviously we graduate and then we work and work and work and work and then if we didn't have anything to really help us to continue and improve and learn and learn more and more we will live as we were like 20 years ago and not actually move in the right direction positively, and actually embrace the new development innovation and all of that. And I think revalidation is the key to that. (Int 49 – RO)
Individual appraisal	You hear about concerns much earlier now than you did before. Some staff say, well, actually staff on the ground feel they're being dealt with before, because there's always this bit of a thing, you know, doctors get away with it … Before, there was a real culture of, oh, well, he's always been like that. Unless you did anything serious clinically, but conduct things, you know, some people were rude and it was just how they were. We tackle those things now. …we always tackle clinical things but actually I think we're probably even sharper on that now. (Int 47 – Revalidation Manager)Definitely it has achieved the ability to identify struggling doctors and helping them, it definitely has done that much better… it's helped a lot of doctors with minor problems, doctors who need to just tighten up their record keeping, doctors who need to pay more attention to certain aspects of their practice; it's really helped a lot with that. Because the performance data shows these things, they disclose at appraisal, they formulate a smarter objective on how somebody supports them. So it's done that. So it probably has helped, I say low level. (Int 43 – RO)All it’s done is impact negatively and it’s taken time away from hands -on patient care, undoing all the stuff you need to do, and it’s driven people out of the medical profession prematurely… having this sort of five-year, oh, we’ll tick the box, isn’t going to stop anybody bad getting through, but it’s lost a lot of good people. (Int 22 – Appraiser)
Reconfiguration	The great thing about having a system it actually does help your implementation because that becomes it … first we started off by just saying that’s it, get the stuff in your portfolio. So you start off with it being a little bit of a tick box exercise because that’s the way of getting it into practice, and then you start working on the quality after that. I don’t think implementation has been especially troublesome because this is required if you want a license to practice, just get on with it. (Int 37 – AMD & Deputy RO)In terms of quality, when we had the appraiser feedback meetings, we asked the appraisers how they would feel about quality feedback, and we do that every time. And we’ve done it different in different years. So one year, we had some lay people in the trust actually sit in on people’s appraisals, so we might have the Medical Director, somebody from HR, sit on and view an appraisal, and then feedback afterwards to the appraiser, and we collate all the information. (Int 44 – Appraisal Administrator)

RO: responsible officer; HR: human relations; AMD: assistant medical director; CQC: Care Quality Commission..

The following four sections explore our findings further, covering each of the NPT domains and subdomains as set out in Table 2.

Coherence

Participants on the whole held a shared understanding of the objectives of revalidation. It was understood by most as a necessary form of regulation that aims to improve patient safety through ensuring doctors are up to date and fit to practise, and to identify and offer support for struggling doctors. There was less consensus in understandings of the outcomes of revalidation, especially deferral. Doctors in non-standard roles such as locum doctors (doctors who work short term in the place of the regular doctor when that doctor is absent or when a hospital or practice is short staffed), whose way of working conflicted with the formalized requirements of revalidation, were noted to be more likely to lack a full understanding of the process. Overall, however, knowledge had increased as people had gained experience of the revalidation process.

When differentiating the changes revalidation brought to ways of working, participants focused on the formalization of clinical governance systems, such as the linking or integration of previously separate areas of clinical governance. This was seen to have facilitated the collection of information for clinical governance and communication between teams, which positively impacted participants’ work. The introduction of the RO role in particular was stated to have significantly increased these connections. Participants also indicated that revalidation helped to clarify quality improvement activities, by creating a clear structure for them to function within.

Participants expressed a general acceptance of the need for greater scrutiny of the medical workforce and of medical revalidation as a legitimate means to achieve this. The value of revalidation was demonstrated through explicit comments and reports of engagement with the policy (Table 2). Positive perceptions of revalidation were understood to have increased over time as the profession and others involved in the policy gained greater clarity through having to operationalize it.

Not all valued revalidation, however, for two main reasons. Firstly, there was scepticism that it would identify poor performance or impact clinical practice. Participants expressing these views tended to see revalidation as purely administrative in function. A minority stated that revalidation would not deal effectively with very poor or dangerous practice by doctors who actively sought to conceal problems. Secondly, some thought that revalidation was simply duplicating other areas of clinical governance, making it unnecessary as its goals could be reached by existing systems or regulatory bodies.

Cognitive participation

All case study organizations had implemented a system of revalidation and revalidated the majority of their doctors. The compulsory nature of the policy meant that ‘willingness’ to support revalidation was in part because doctors and organizations had no choice. At an organizational level, support was evident through a willingness to appoint new posts, restructure ways of working and provide resources (to varying degrees). At an individual doctor level, most were reported to be willing to comply with this requirement, and increasingly so.

Although the majority of doctors were reported to engage willingly with revalidation, an ongoing push was still identified as necessary. Chasing was required, described as an ongoing task for revalidation teams, especially administrative staff. Non-engagement became being less tolerated and more directly challenged. Organizations were described as becoming stricter in their approach to engagement and being more likely to take action, and there was a perception that the GMC did so, too.

Key individuals driving revalidation forward were identified and described as pivotal to initial implementation, changing attitudes and the continued successful running of revalidation. ROs and their support staff (most frequently a medical staffing and revalidation team) were identified as those occupying these driving roles. Having strong leadership by an individual or team understood to be knowledgeable and supportive seemed to contribute to engagement levels. Those able to effectively communicate with their doctors and allay concerns appeared to be successful in ensuring engagement, making the process less onerous for all involved.

Revalidation was seen to have brought change to the way doctors and other staff worked with others both internally and externally. People were reorganizing themselves, new groups had been put in place to manage the requirements of the revalidation process alongside some new appointments. These changes were reported as being positively received and responded to, but this perception varied between organizations.

Change had also occurred in the ways that designated bodies worked with external partners. The passing of information between organizations as doctors moved was stated to occur to a better standard than previously and more frequently. At the same time, the ability to share information was seen to be hampered by inconsistent use of IT systems across organizations.

Revalidation and the changes it brought were perceived to be most legitimate by those involved in running the process, in particular ROs and revalidation teams. This is perhaps unsurprising given the individual investment in the policy that their roles required. While many doctors on the ground were said by participants to have accepted revalidation, for some it was seen as a bureaucratic, tick box exercise which detracted from their clinical role. There was also a perception by a small number of participants that some senior doctors were unwilling to or displeased about being appraised by others. Further, not all doctors accepted the disclosure of private practice as part of the appraisal process, which some viewed to be outside the remit of NHS designated bodies. This is despite appraisal requiring to cover doctors’ full scope of practice. A generational difference was also reported, which was seen to be the result of differences in training, with younger doctors identified as more accepting of such oversight than older doctors.

Collective action

Our data revealed that there had been some initial difficulty in setting up and integrating revalidation into existing work patterns, with four main reasons identified: initial resistance; the newness of the policy; poor existing clinical governance systems and preparation for the introduction of revalidation, and lack of resources. Integration became easier and more successful over time, but this was largely dependent on an organization’s preparation and planning for revalidation as well as organizational culture. Those for whom it meant the biggest change to existing ways of working found integration of revalidation most challenging, in particular those in poorly resourced or less supportive organizations. Similarly, doctors whose working context meant that they were distant from their organization, transient or did not work within an organization found integrating revalidation difficult due to the lack of organizational support they could access.

The availability of resources and management support for revalidation was key to how easily individual doctors were able to integrate it into their existing work, in particular protected time, IT systems and administrative support. Time was one of the most under-resourced areas and numerous individuals highlighted that their organization had not provided protected time for revalidation roles, or provided administrative support to operate the policy. This meant that the preparation revalidation required of individual doctors, those being appraised and those conducting appraisals, was experienced as a burden requiring work in personal time outside of formal working hours. Poor organizational and IT systems were also identified as problematic, making it difficult to gather the data doctors needed for their appraisal and revalidation. It was argued that if these systems were improved it would make undertaking appraisal and revalidation more straight forwarded. Lack of resources was not always within local control however; issues such as an inability to recruit the required staff were understood as challenges facing the health care system in England more broadly.

Many of those involved in operationalizing revalidation noted to lack confidence in the ability of others to carry out its requirements appropriately, referring both to doctors going through the system and those running it. This was mostly seen as an indication of adapting to the newness of the policy rather than specifically about revalidation itself. Lack of confidence was for example commonly noted to result from individuals being unable to fit training into their work, or that training opportunities provided by organizations were inadequate. However, confidence was seen to be ‘restored’ once identified training needs had been addressed and indeed, confidence in the ability of those assigned tasks to fulfil them was, for the majority, high. Concerns remained in regards to succession planning, with participants noting that knowledge and ability to run revalidation within organizations was held by and reliant on a few ‘in the know’. With the exception of succession planning, once the necessary systems had been put in place and training delivered, revalidation was described as running smoothly for most.

Reflexive monitoring

At the time of this study, the organizations being studied had revalidation systems up and running, with most of their doctors having completed the first five-year cycle of revalidation. Many organizations had undertaken audits of the quality and effectiveness of revalidation systems or these were underway, while formal assessments of the impact of revalidation on performance and patient care were, on the whole, yet to be undertaken.

Many participants spoke of the effects and impacts of revalidation on practice and the organizations in which they worked, based on their own informal assessment of the system as individuals and communally with colleagues. Most believed revalidation to be worthwhile, with different explanations provided. Revalidation was believed, for example, to have improved continued professional development and keeping doctors up to date; reflection, leading to improvements in patient safety; the quality of appraisal and the appraisal process (record keeping, formalization); doctors’ behaviour to other staff and the likelihood of concerns about or behavioural difficulties of a doctor being dealt with.

The majority of those interviewed spoke of the impact of revalidation at a distance from themselves, highlighting changes of the profession at large or at an organizational level rather than their own practices. Perceived impacts depended on the role of the individual. Those in management saw revalidation as offering them a better oversight of other doctors’ practices and more authority, enabling them to better perform management duties. But this also meant an increased workload. At the individual level, revalidation was mostly seen to have increased and improved reflection, as noted, and some identified this as improving practice although many did not make this link. Non-clinical staff involved in revalidation within organizations reported significant changes to roles, increased ability to get doctors to engage and more authority to ensure compliance.

We noted earlier that there was a minority of those interviewed who did not perceive revalidation to be worthwhile. Further, some participants noted an awareness of others in the profession who did not view revalidation to be of value, highlighting that negative views were discussed by colleagues across organizations. Questions about the effectiveness of revalidation were further fuelled by the observation in some organizations that revalidation had failed to pick up serious patient safety concerns.

The requirements of revalidation are uniform for doctors nationally, but organizations approached it differently, adapting it to fit best with existing systems, agendas and ways of working. The tailoring of revalidation in this way was often a practical approach to resource constraints and a result of ongoing assessments of revalidation systems both formally and informally. In most organizations, this came following an initial phase of ‘getting it in’. Once this point had been reached, a gradual increase of feedback was said to occur and was used to improve the efficiency and quality of revalidation systems. Interview participants spoke of an increased focus on the quality of systems in place and the information and material used for revalidation. This was most evident in relation to appraisal, particularly supporting information and appraisal reports. Work was also described as beginning in terms of strengthening the triangulation of information and better connecting revalidation to wider clinical governance systems to support organizations in their clinical governance and ability to monitor doctors and patient safety.

Discussion

In this paper, we have assessed the implementation of revalidation across a range of organizations using NPT as a framework for analysis. We found that, overall, in NPT terms, coherence was achieved by most organizations, although not completely. Much of the impact on organizations and doctors’ practice occurred as an indirect result of the work that revalidation required organizations to do on their existing systems. There remained a lack of coherence in terms of participants’ understanding of the revalidation process and its requirements.

Cognitive participation had also been achieved to a degree and it had enabled implementation, although this was in places limited. We found that all the organizations included in this study had sufficient support from clinical and non-clinical staff to get revalidation up and running. The mandatory requirement for revalidation was seen as essential to its implementation, especially since support and acceptance of it was gradually acquired via experience rather than in the initial stages. This suggests that if the process had been voluntary, it would have been unlikely to have been taken up or supported to the same degree.

Organizational context was the most significant factor determining doctors’ and non-clinical staffs’ experience of revalidation and the likelihood of successful collective action. Context here refers to organizational size and type, resourcing, culture and organizational history. These findings echo those of Spendlove’s recent research on revalidation, which focused on a single organization.[19] Reflexive monitoring had begun to take place, too. Organizational systems were planned to be appraised and audited by most organizations. There were informal discussions on the impact of revalidation on practice and performance but none of the organizations studied had formally assessed the impacts of revalidation as yet. Most, but not all, participants believed that revalidation was worthwhile due to perceived improvements since its introduction.

Approaches to revalidation varied across organizations, which were described as tailored to the needs, agendas and resources of each. There was thus limited consistency in the experience and delivery of revalidation. Given that many of the issues faced by organizations in the implementation of revalidation were seen to be the result of newness and unfamiliarity with systems, this multiplicity raises questions about what is legitimate variation and what is inconsistency. Variation between organizational approaches could be interpreted as revalidation being used by organizations to fit their own agendas, or as a consequence of the medical profession shaping revalidation in a way that might avoid the full scrutiny of the policy.[19] At the same time, a ‘one-size fits all approach’ of revalidation may be inappropriate, given that individual doctors’ practice and working circumstances vastly differ as do the organizations in which they work.[11,16] From this perspective, the tailoring of revalidation can be understood to be necessary for its implementation and potentially contributing to existing acceptance, while also beneficial to its chances of long term acceptance and embedding in everyday work.

Overall, our findings suggest that nationally developed and led policy initiatives such as revalidation need to be sufficiently flexible to allow organizations to implement them in a way which capitalizes on existing systems, and be appropriate for the specific organizational context. The success and ease of implementation of organizational change depend on a number of local factors, notably the existence of relevant organizational policies and processes, which means that a better understanding of such policies and processes would be beneficial to inform policy development. There is a particular need to better address the challenges faced by those groups of doctors for which revalidation has been identified as particularly problematic, such as locums and those with a portfolio career. In addition, organizations that experience tension between revalidation and the running of business, such as private agencies, require further attention. By changing the relationship between doctor and organization, revalidation has implications for professional identity in medicine and how it is enacted and performed.[20]

The use of NPT to explore the implementation of revalidation in this study has extended the usual parameters of the theory. We have noted earlier that existing work that has used NPT in health care has explored implementation of clinical and behavioural practices at an individual level rather than those at an organizational level.[15,21,22] Our use of NPT in this study demonstrates its value for exploring and analysing the implementation of interventions within the health care setting at organization and system levels and perhaps of other complex social interventions and policies outside of this setting.

Conclusion

This study explored the implementation of medical revalidation, which was introduced in the UK in 2012, analysing the change process and its consequences for doctors and organizations. We found that despite considerable early scepticism and overt opposition in the medical profession, revalidation has become gradually accepted, embedded and even valued over time. Its impact and effectiveness are still questioned by many stakeholders, and the focus of attention has now shifted towards revising and improving the way revalidation works in practice.