Recording problems and diagnoses in clinical care: developing guidance for healthcare professionals and system designers.

BACKGROUND: Accurate recording of problems and diagnoses in health records is key to safe and effective patient care, yet it is often done poorly. Electronic health record systems vary in their functionality and ease of use, and are not optimally designed for easy recording and sharing of clinical information. There is a lack of professional consensus and guidance on how problems and diagnoses should be recorded.
METHODS: The Professional Record Standards Body commissioned work led by the Royal College of Physicians Health Informatics Unit to carry out a literature review, draft guidance, carry out an online consultation and round table discussion, and produce a report including recommendations for systems. A patient workshop was held to explore patient preferences for mechanisms for sharing diagnosis information between primary and secondary care.
RESULTS: Consensus was reached among medical specialties on key elements of diagnosis recording, and draft guidance was produced ready for piloting in a variety of care settings. Patients were keen for better ways for diagnosis information to be shared. DISCUSSION: Improving the recording of diagnoses and problems will require a major effort of which the new guidance is only a part. The guidance needs to be embedded in training, and clinical systems need to have improved, standardised functionality. Front-line clinicians, specialist societies, clinical informaticians and patients need to be engaged in developing information models for diagnoses to support care and research, accessible via user-friendly interfaces. © Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Background

Detailed and accurate records of patient problems and diagnoses are essential for safe, effective patient care, but current practice and the design of electronic health records (EHRs) in the National Health Service (NHS) do not achieve this.1 Even where diagnoses are coded using a terminology, in secondary care the coding process is detached from clinical care and may not always convey the meaning intended by clinicians.2 Information for audit or research is often collected via a process separate from clinical care, resulting in wasteful duplication and potential inconsistency in data sources.3 Good record keeping guidance exists for general practice (GP),4 but there is no equivalent for secondary care or shared care. Improvements in diagnosis recording will require a clinician-led, system-wide multidisciplinary approach involving guidance, training and improvement in systems.5 6

Electronic recording of diagnoses using a terminology such as SNOMED CT (or previously the Read Clinical Terms) has been standard practice in GP in the UK for 30 years,7 but it does not represent details such as laterality, evidence and clinical manifestations in a structured way. These details are recorded only in free text despite their importance for clinical decision making.8 9 Conversion of free text into a structured form at the point of data entry has been proposed, and while it is relatively straightforward to extract simple clinical terms from text, the task of integrating this information with the existing record and structuring it appropriately is much more complex.10–12 EHR systems should enable the level of uncertainty along the diagnostic process to be faithfully represented.13

The need for improving the recording of diagnoses was highlighted at a workshop on diagnosis recording organised by the Royal College of Physicians (RCP) for medical specialty informatics leads in October 2017. Following on from this workshop, the Professional Record Standards Body (PRSB) and RCP Health Informatics Unit (HIU) conceived a project to develop professional guidance on recording problems and diagnoses in EHRs. This work builds on the 2015 US National Academies Report on improving diagnosis in healthcare,13 and focuses on the recording and communication of diagnoses, including a suggested programme of future work in this area. This article explains the rationale and methodology of the project, and recommendations arising from it.1

Problem-oriented documentation

Problem-oriented documentation was advocated by Weed more than 50 years ago.14 The fundamental principle is that clinicians curate a list of a patient’s pertinent health problems or diagnoses, and link them to other entries related to the problem, such as prescriptions or clinical notes.15 GP systems support problem oriented recording but the functionality differs between systems, with the result that well-organised records in one system may become disorganised and unwieldy when transferred to another. For example, the Egton Medical Information System (EMIS) allows problems to be categorised as ‘major’ or ‘minor’, whereas vision uses a numerical priority attribute. Clinicians need to adapt the way they record information according to how the vendor has designed the system.

Studies in the literature report benefits of problem lists, but greater clarity is needed on how they should be used.8 15–19 Clinicians have differing views on what counts as a ‘problem’18–22 and how to record recurrent conditions.23 Problem lists may become cluttered with minor or inactive problems19 and it may be unclear whose responsibility it is to curate the problem list, and how this activity fits into clinical workflows.21

Sharing and communicating problem and diagnosis records

For most NHS patients, longitudinal problem and diagnosis records exist only within their GP record. Secondary care organisations may have access to view the GP record, but currently the only way that information flows back to the GP is in a human-readable letter. Harmonisation of data models for problems and diagnoses between primary and secondary care, and electronic messaging standards (such as Fast Healthcare Interoperability Resources (FHIR) being developed for the CareConnect programme24) could potentially allow diagnoses to be recorded and transmitted in a structured and actionable format. Shared primary and secondary care problem lists do operate in some areas, but it can be unclear who has responsibility for maintaining entries in a shared problem list,25 and the clinical informatics community lacks a consensus view on the best framework for sharing.

Aim and scope of the project

This project aimed to develop system-wide guidance for the recording of medical diagnoses and problem lists in a structured form in patient records, to enable standardisation and uniformity across primary and secondary care. We sought to create a central focus to which all interested stakeholders could provide input. We prioritised the creation of concise, timely, implementable guidance with a roadmap for future work. Given the time and budgetary constraints, the project scope did not extend to detailed consideration of nursing or allied healthcare professional diagnoses, or detailed technical standards. The project was commissioned by the PRSB and led by the RCP HIU, with funding from PRSB and the RCP.

Methods

Project overview

A project board was convened by the HIU and the PRSB, comprising individuals with expertise in a range of clinical specialties (including medicine, surgery and GP), health informatics, project management, communication and engagement. The project board was chaired by the director of the HIU, and the clinical lead for the project was a hospital doctor with expertise in research using EHRs. The board contained members of key professional groups such as the Faculty of Clinical Informatics, the Royal College of General Practitioners Health Informatics Group and the British Computing Society Primary Health Care Specialist Group. The board met regularly in 2018 and early 2019, and the final project report was presented to the PRSB advisory board in July 2019.

Literature review and development of draft guidance

We developed guidance in an iterative process guided by relevant existing guidance (such as guidance for general practitioners4), evidence on the effectiveness or benefits of problem lists in the literature, capabilities of existing systems and personal clinical experience. We searched for professional guidance documents, information models and technical standards from websites of relevant organisations, via personal knowledge and contacts, and directly from vendors through board member involvement in the INTEROPen CareConnect programme.

We also carried out a review of the academic literature, searching for relevant articles in PubMed in English using the terms ‘problem list’ or a combination of ‘problem’, ‘oriented’, ‘medical’ and ‘record’ in the title. We searched for additional articles in the reference lists of articles reviewed. The draft guidance is published as Appendix 1 of the full project report.1

Consultation survey

The purpose of the consultation survey was to gather the views of a large variety of different clinical specialties and other professional groups, based on the draft guidance. We conducted the consultation using the SurveyMonkey online platform between 4 October and 16 November 2018. Consultees included healthcare professionals who had expressed an interest in health informatics and were on the HIU mailing list, specialist society informatics leads, Academy of Medical Royal Colleges (AoMRC) representatives on the PRSB Advisory Board, attendees of the 2017 RCP diagnosis recording workshop, specialist society informatics leads, the General Medical Council, British Medical Association, Chief Clinical Information Officers via the DigitalHealth network, NHS bodies, commissioners and clinical coders. We also asked lay reviewers to assess the clarity of the document.

Round table discussion

We convened a round table meeting on 4 December 2018, inviting key stakeholder representatives from among the survey consultees to review issues raised in the survey. The issues chosen for discussion were those where survey respondents had expressed concern, or where the project board felt that face-to-face discussion would be helpful to clarify and refine the guidance.

Patient workshop on diagnosis sharing

A patient workshop was not planned as part of this project, but during the round table discussion the issue of shared curation of problem lists surfaced considerable interprofessional uncertainty. If a secondary care clinician is able to contribute to a shared problem list curated by the general practitioner, this could enable more up to date and detailed information to be available in one place, but a disadvantage is that the problem list may become cluttered and unwieldy, and it may be unclear who has responsibility for errors.

As this is key to the future development of shared records, we sought to put this question to patients. A face-to-face workshop was chosen as the preferred format, as it would enable the options and implications to be discussed in detail. An opportunity arose to consult participants of a workshop, which was seeking views on the use of free text for research. Workshop participants were members of the University College London Hospitals Biomedical Research Centre patient and public involvement group. The concepts of different levels of sharing were explained using role play: no sharing, view only, ability to suggest and ability to edit. After facilitated discussion, participants were asked to vote anonymously for the option they would most support.

Final guidance and report

Recommendations for future work were derived by the project board based on the findings of this project. The final guidance and recommendations for future work were presented to the PRSB advisory board in July 2019.1 The board recommended that the guidance is piloted in different clinical settings, after which it will be submitted for endorsement by the AoMRC.

Results

We received 198 individual responses. The responses were broadly positive (70%–80%) in respect of each individual section (figure 1), but some respondents were concerned about the strength and feasibility of the recommendations, particularly given time constraints in the clinical setting, the applicability to fields such as mental health, and the clarity of some of the wording. More details are in Appendix 2 of the full report.1

Figure 1

Survey respondents’ satisfaction with aspects of the draft guidance on diagnosis recording.

We chose the following topics for discussion based on concerns raised in the consultation survey: definitions of terms for problems and diagnoses (and whether the word ‘condition’ could be an overarching term), what to include in a problem list, certainty and verification status of diagnoses, problem and diagnosis attributes, reviewing problem lists and maintaining problem lists. The meeting also included discussion on how the guidance would be disseminated, and recommendations for the design of clinical systems.

There were 19 participants at the round table discussion, including informaticians and clinicians from a range of clinical specialties, as listed in Appendix 3 of the full report.1 Topics discussed and decisions are outlined in table 1.

Table 1

Topics discussed in the round table meeting

Topic	Decision
Definition of a ‘problem’, and consideration of ‘condition’ as an overarching term	Use ‘problem’ as the overarching term to describe any type of entry in a problem list (diagnosis, symptom, social factor, etc).
What to include when creating problem and diagnosis lists	Guidance should focus on the functional requirements of a problem list, and provide examples.
Certainty of diagnoses	Classify diagnoses as ‘confirmed’ or ‘suspected’, and ensure that systems respect these attributes whenever information is transferred.
Problem and diagnosis attributes	Active/inactive, priority, severity, body site, laterality, stage, clinical behaviour, evidence, manifestation, aetiology.
Reviewing and maintaining problem lists	Clinicians should update problem list entries in their domain at each encounter. Systems should enable sharing of problem lists; clinicians should not need to transcribe them from one system to another.

The first topic of discussion was the terminology for problem-like concepts, which may include symptoms, diagnoses, examination findings, abnormal investigation results and social factors. The group considered the use of ‘condition’ as an umbrella term for such concepts, but concluded that ‘problem’ was more suitable, as the concept of a ‘problem list’ is already in common use.

The group considered what should be included on a problem list, but could not come to a consensus recommendation on the actual items to include, as they may vary by use case. Instead, the group recommended that the guidance should emphasise the functional requirements of problem lists. Coded problem lists can be viewed in different ways to suit different needs. Clinicians should focus on problem list entries in their domain, with the aim of facilitating continuing care.

The group also discussed standardisation of certainty qualifiers. The Emergency Care Data Set26 collects information collected on all emergency department attendances in England, and requires clinicians to qualify the diagnosis as ‘suspected’ or ‘confirmed’. This two-level classification has been well received; other options such as ‘possible/probable/proven’ were consulted on but found to be too complicated, and a ‘working’ diagnosis was not well understood. A way of flagging ‘disputed’ diagnoses was also thought to be important, particularly in psychiatry.

Laterality was considered to be an essential attribute, which should be standardised and persistent for clinical safety. Recording of evidence underlying a diagnosis was also thought to be particularly important. Other attributes common to many diagnoses included severity, stage, body site, clinical behaviour, manifestation and aetiology.

The issue of structured versus unstructured data in EHRs was also discussed. Although structured data has advantages (it can be used for clinical decision support and audited), it may be time-consuming to enter. Differential diagnoses and other elements of clinical thought were considered best suited to free text.

The group discussed sharing of information between primary and secondary care, which currently does not happen in a structured way. As a result, problem lists are duplicated, and it is time-consuming to maintain accurate and up-to-date problem lists in all settings, particularly in outpatient clinics where patients are seen infrequently. The recommendation for clinicians is to update problems and diagnoses in their domain, and prioritise problem list entries that will have most impact on care.

A national approach to developing data standards for diagnosis records was proposed. This would incorporate coordinated information models and SNOMED CT subsets, and should accommodate the information needs of primary and specialist care, disease registries and billing.

Seventeen participants attended the workshop and ranged in age from 25 to 84 years (median 67). Workshop participants appreciated the problems resulting from lack of information flow between hospitals and GPs. They were keen on patients being able to view problem lists and thought that this could empower them to feed back on any errors, thus improving the quality of the record. Issues that they raised included: GPs’ feelings about records being updated without their specific consent, what happens if there is more than one change suggested, what happens if the GP does not agree with the change, and how information could be shared between different specialties.

The outcome of voting was that 11/17 (65%) supported specialist access to edit the GP diagnosis list, 4/17 (24%) supported a ‘view and suggest’ model and only 2/17 (12%) supported a ‘view only’ model.

Key recommendations in final guidance and report

The guidance is based on the overall principle that problem lists should be complete, consistent, accurate, relevant, accessible, timely, unambiguous and (where appropriate and possible) linked to treatments and other information.1 Key recommendations are outlined in box 1.

When creating a problem list, bear in mind its primary function to ensure that important information about a patient’s health issues relevant to current care is readily available to treating clinicians.

When adding information to a problem list, amend or update existing problems where appropriate and avoid creating duplicates.

Create separate records for acute conditions that are exacerbations of chronic problems.

Ensure that problem titles are coded accurately to the relevant terminology, such as SNOMED CT. It is acceptable to use a broad term initially and then refine it as more clinical information becomes available.

Do not mix together suspected and differential diagnoses with confirmed diagnoses in the problem list. In most systems, suspected and differential diagnoses should be recorded in text notes rather than in the coded problem list.

Record the date of onset of the problem, if possible, and what evidence a diagnosis is based on.

When seeing a patient, review, verify and edit problem list entries within your domain, prioritising those that are most relevant to current care.

Problems that have resolved should be marked as such so that they do not clutter up the problem list. However, past conditions that require ongoing follow-up or consideration should remain ‘active’, to ensure they are not forgotten.

In secondary care outpatients, prioritise recording of problems being actively managed by the clinic or service.

Include the encounter problems (the primary problem(s) managed during the admission or visit) in any communications, and include other problems if they impacted on the patient’s care during the episode.

It is not necessary to include a comprehensive list of past, inactive problems that have no ongoing impact.

In order to be able to use problem lists in this way, clinical systems need to be safe and user friendly, and they must enable meaning to be maintained faithfully whenever information is entered, viewed or transferred. Problem lists should enable information to be recorded once and reused as needed, without a need for duplication or manual transcription. It should be possible to view historic versions of problem lists on which clinical decisions were based. We strongly recommend that the structure of problem lists is standardised according to information models defined by the clinical community.

Discussion

This project was able to achieve consensus among a broad range of medical specialties on the general principles for recording problems and diagnoses in EHRs.1 It has highlighted the complexity of the issue, where even seemingly basic steps (such as agreeing on terminology to describe concepts such as ‘problem’ and ‘diagnosis’) sparked much debate. The project has limitations; it was bounded in scope by time and budgetary constraints, and the findings may not be generalisable to other contexts.

Improvements in systems are needed, but their specifications need to be defined by the clinical community and not by vendors. In the past, there has been a vacuum of leadership in agreeing how information should be recorded, and vendors have stepped in and created their own models which are not interoperable and may not be fit for purpose. Professional standards developers such as the PRSB and open platforms such as openEHR27 need to be harnessed to enable the clinical community to define the structure and content of health records. A large number of stakeholders will need to pull together with the common aim of improving diagnosis recording, and agree on a common framework for specifying how this information is recorded (table 2).

Table 2

Suggested roles of major stakeholders in improving the recording of problems and diagnoses in the NHS

Stakeholder	Suggested role(s)
Professional Record Standards Body	Liaise with healthcare professional organisations to build consensus on professional requirements for problem and diagnosis recording.
Faculty of Clinical Informatics	Advocate and develop models to improve the recording and sharing of problems and diagnoses, and ensure it is embedded in undergraduate and postgraduate medical training.
Academy of Medical Royal Colleges	Ensure that accurate recording of diagnoses is a high priority in all medical specialties, endorse guidance for healthcare professionals, and encourage the use of standardised, routine data for national clinical audits.
Specialist societies	Align requirements for clinical diagnosis and audit with information models for problems and diagnoses.
OpenEHR community	Develop models for healthcare information (archetypes) that enable details of problems and diagnoses to be recorded and shared.
INTEROPen	Develop interoperability profiles which accommodate diagnosis information models, and encourage vendors to adopt them.
NHSX, NHS Digital and other central NHS organisations	Develop core informatics infrastructure and a programme of work for standardised recording of problems and diagnoses, including disease archetypes and terminology subsets.
NHS Trusts	Provide user-friendly EHRs that enable problems and diagnoses to be recorded in a standardised way, and support clinicians in using them.
Health Data Research UK and research funders	Fund work on data standards and improving usability of EHR systems, in order to improve the quality of data and its utility for medical research.
Medical schools and postgraduate deaneries	Ensure that medical students and doctors in training understand how to use problem lists and terminologies, and appreciate the need for accurate recording of diagnoses.

EHR, electronic health record; NHS, National Health Service.

How problem lists are used in practice will depend on how this information can be shared between healthcare professionals. Any proposed technical solution will have to be adaptable to local data sharing arrangements.

Patient views on sharing diagnosis information

Local health and care records (LHCR) in some parts of the UK currently enable secondary care users to view relevant information from a patient’s GP record. The PRSB core information standard28 will ensure consistency in the content of the information shared. This is an important first step, but patient participants in our workshop did not feel that viewing alone was sufficient; they preferred a structured way for GPs and specialists to collaborate on a patient-centric shared problem list. This would require agreement between NHS organisations to clarify responsibilities and resolve any conflicts between entries on the list.

Recommendations for future work

We recommend that the guidance is embedded in EHR user guides, incorporated into medical school and postgraduate training programmes, and piloted in a range of primary and secondary care settings.

Developing a technical data sharing and information model for problems and diagnoses

LHCR sharing is being implemented in many areas, but there is a need for functionality to share data outside the region, and also to share more detailed structured information about problems and diagnoses in a consistent way.

As a first step, we propose standardisation of the information model of problem and diagnosis records, and implementation of an application programming interface that will allow the retrieval of diagnosis and problem lists from the GP record or any other NHS EHR system (with appropriate permissions). In the future, problem list entries with links to other parts of the record (eg, investigations providing evidence for the diagnosis) should also be transferable between systems, using the National Record Locator Service.

The shared record framework will remove the need for manual transcription of problems or diagnoses; instead it should enable them to be copied, imported from, linked or referred to from another system (figure 2). It should enable any clinician to access a combined, detailed list of a patient’s problems and diagnoses from different sources, including diagnoses embedded in communications such as discharge summaries (subject to consent for sharing). This would include details such as dates of diagnoses, which are currently not included on discharge summaries. The system should enable specific restrictions to apply to sensitive problems. Information should be prioritised and presented in a way that is clinically safe and easy to assimilate, with diagnoses that are critical to patient safety always prominently visible. It should be possible to view previous versions of problem lists on which historical clinical decisions may have been made.

Figure 2

A potential model for shared problem and diagnosis records. GP, general practice; HFpEF, heart failure with preserved ejection fraction.

Facilitating transfer of problem list entries between systems without merging the lists maintains clarity of responsibility for individual lists, but may lead to duplication and conflicts between lists. It also requires each user to spend time curating their own problem list. Shared curation of a common problem list can be achieved by designating the GP problem list as the primary list and enable secondary care clinicians to edit or suggest changes, under a data-sharing framework. An alternative approach is to create a shared specialist diagnosis registry separate from the GP problem list. Further work in this area will be explored via a Special Interest Group of the Faculty of Clinical Informatics.

Engagement with healthcare professionals and specialist societies

Disease ‘profiles’ or ‘archetypes’ could be defined to enable user interfaces to display information relevant to a diagnosis (such as treatments and investigations) consistently across all clinical systems that support the disease profile. Such a system could also provide a user-friendly way to enter diagnosis details, perhaps using natural language processing,11 12 rather than trying to choose a complex precoordinated SNOMED CT term from a long list. This will help to improve safety by displaying problem lists in a more consistent way, so that important information is not missed.

We recommend that specialist societies and front-line healthcare professionals should be fully engaged in developing disease profiles to shape the way information is captured and displayed in their specialty. They must aim for audit and registry data to be collected in usual clinical care rather than as a separate process. Engagement with researchers, research funders and initiatives (such as the Health Data Research UK Phenomics theme) will also be important, to enable research grade data to be collected in clinical care.29

Implementing better diagnosis recording in practice

Vendors need to be engaged throughout the process to ensure that proposed solutions are feasible; this can be done through the INTEROPen community.30 The technical solutions need to be clinically assured for safety, and piloted before wider implementation.

We recommend long term, consistent funding for information modelling in order to yield continuous improvement in the usability of EHR user interfaces and quality of data recorded. Eventually, the new problem list specification should be incorporated into statutory NHS contracts, to ensure that it is applied consistently.

Conclusions

This project has highlighted the challenges and opportunities in recording problems and diagnoses in EHRs.1 A consensus has been reached among clinical professionals on some key aspects of professional practice and requirements for systems. However, guidance for clinicians is just the start of this process; major improvements in system capability, interoperability and usability are essential to realise the vision of better diagnosis information for patient care.