A network model of activities in primary care consultations.

OBJECTIVE: The objective of this study is to characterize the dynamic structure of primary care consultations by identifying typical activities and their inter-relationships to inform the design of automated approaches to clinical documentation using natural language processing and summarization methods.
MATERIALS AND METHODS: This is an observational study in Australian general practice involving 31 consultations with 4 primary care physicians. Consultations were audio-recorded, and computer interactions were recorded using screen capture. Physical interactions in consultation rooms were noted by observers. Brief interviews were conducted after consultations. Conversational transcripts were analyzed to identify different activities and their speech content as well as verbal cues signaling activity transitions. An activity transition analysis was then undertaken to generate a network of activities and transitions.
RESULTS: Observed activity classes followed those described in well-known primary care consultation models. Activities were often fragmented across consultations, did not flow necessarily in a defined order, and the flow between activities was nonlinear. Modeling activities as a network revealed that discussing a patient's present complaint was the most central activity and was highly connected to medical history taking, physical examination, and assessment, forming a highly interrelated bundle. Family history, allergy, and investigation discussions were less connected suggesting less dependency on other activities. Clear verbal signs were often identifiable at transitions between activities. DISCUSSION: Primary care consultations do not appear to follow a classic linear model of defined information seeking activities; rather, they are fragmented, highly interdependent, and can be reactively triggered.
CONCLUSION: The nonlinearity of activities has significant implications for the design of automated information capture. Whereas dictation systems generate literal translation of speech into text, speech-based clinical summary systems will need to link disparate information fragments, merge their content, and abstract coherent information summaries.

INTRODUCTION

Current generation electronic health records (EHRs) do not fit naturally with the flow of many clinical encounters. This misalignment between clinical work and EHR workflows is a result of the primarily documentation-focused design of EHRs that largely ignores other tasks taking place simultaneously with documentation, such as problem-solving and communication., As a consequence, the use of EHRs is associated with clinician burnout, increased cognitive load, information loss, and distraction from nondocumentation tasks in the clinical encounter. A recent study found that more than half of the primary care physician’s workday is spent using EHRs.

The rapid evolution of technologies that support human–computer interaction, including gesture-based modalities and speech recognition, provide opportunities to reimagine the way EHRs are designed and the functions they might perform. In particular, the notion of a digital scribe, which assists clinicians by automatically generating some or all of the documentation associated with a clinical encounter, has real potential to transform clinical information system design. Digital scribes take advantage of advances in artificial intelligence and natural language processing to automate some or all of the clinical documentation tasks currently conducted by humans. Whereas present generation speech recognition tools focus on dictation of notes (verbatim transcript creation) and template-based data entry, digital scribes should have the capability to interpret speech content in a way that permits more complex documentation tasks such as summarization of speech.

There are 3 broad stages to digital scribe evolution. Present day human-led documentation systems still require clinicians to create clinical documentation but support them with tools including dictation support, semantic checking, and use of templates. Mixed-initiative documentation systems are more sophisticated and are delegated at least part of the documentation task, converting the content of conversations in a clinical encounter into summaries in the electronic record. Computer-led systems would be delegated full control of documentation and request human interaction only when exceptions are encountered. Today’s state of the art is still in the first human-led documentation phase. Speech recognition is still primarily used to transcribe a clinician’s dictated speech verbatim rather than analyze and summarize that speech.

There are 2 major challenges to developing digital scribes that have a degree of autonomy. The first technical challenge arises from the difference between dictated speech—which is structured by the human speaker for a specific documentation purpose—and clinical conversation—which arises between humans to establish common ground, glean information, and solve problems. Conversational and dictational speech have different probability distributions over sequences of words. A comparison of conversational and dictational speech language models found that the dictation language model was more accurate when predicting the likelihood of a word appearing given a context. This was due to dictational doctor notes following a specific format and conversational speech being less structured.

Next, a digital scribe will need to understand not only the words that are spoken in a consultation but also their role or meaning. For example, a digital scribe may need to convert a physician’s lay explanation of disease to a patient into a technically appropriate clinical diagnosis. This is further challenged when similar concepts, such as a drug name, might appear in discussion of a patient’s past history, family history, or present complaint. Understanding the distinct contexts in which concepts are presented is crucial to understanding their meaning and where they should be put in a record.

To better understand the challenges of automatically converting clinical conversation into documentation, this study focuses on primary care consultations, which are a natural setting for digital scribes. Primary care consultations occur in defined physical spaces and involve a limited set of actors, typically a doctor and a patient. Whereas specialist outpatient clinics have similar characteristics, hospital wards present very different encounter structures and might require different scribe designs.

Most models of primary care consultation such as the Calgary-Cambridge Model, Pendleton’s Model, Neighbour Model, and Waitzkin’s Model have been developed for educational purposes. These models emphasize ideal rather than actual consultations and are designed to teach patient-centeredness, psycho-social factors, and shared decision-making. All describe well-structured, linear, and chronological representations of consultation covering history taking, examination, diagnosis, and treatment. Observational studies on primary care consultation structure have been shaped by these teaching models and typically describe consultations as a formal, predominately linear, process. This linearity does not fit well with our understanding of clinical conversations, which can be a fragmented and interruption-driven process.

Thus, there remains a substantial gap in our understanding of the way clinical activities unfold in consultations and how communication and activity patterns might constrain technology choice and design. To this end, this exploratory study seeks to characterize the structure of primary care consultations with a view of identifying typical activities and their unfolding relationships to each other. We sought to determine whether the spoken dialogue between patient and clinician in primary care can be deconstructed into a recognizable sequence of activities and how well these mapped to traditional models of primary care consultation. We also sought to identify a high-level structure of activities with their transition patterns and investigate the possibility that spoken dialogue would directly signal the beginning and end of any given activity. Our goal is to inform the design of automated approaches to clinical documentation involving natural language processing, initially in primary care, but with a longer-term view to supporting a wider variety of clinical settings and documentation needs.

MATERIALS AND METHODS

Study design

An observation study design was used to capture clinical activities taking place during primary care visits. Physicians and patients were recruited from 1 Australian primary care clinic in March and April 2018 using a convenience sampling strategy. Recruitment days were organized according to the availability of consenting primary care physicians on any weekdays. Inclusion criteria for physicians required them to be a primary care doctor and use an EHR for documentation purposes. Patients needed to be at least 18 years of age and have English language competency. All eligible patients scheduled for a consultation on recruitment days were invited to participate. Researchers obtained informed consent from all participants.

Data collection and analysis

Data collection involved 1) a single observer sitting discretely in the examination room and taking notes, 2) audio-recording of the doctor–patient conversation, 3) video capture of the EHR screen, and 4) a brief semi-structured interview with the physician at the end of sessions. Audio was captured using a portable Zoom Audio Recorder and edited with Audacity 2.2.2. Screen interactions were recorded with FastStone 9.0 software. The Macquarie University Ethics Committee approved the study.

The observers included a human–computer interaction researcher (14 sessions), a researcher with a background in public health (19 sessions), and a researcher with a medical and informatics background (7 sessions). After a pilot session, the observers agreed to record 1) the times when tools were used including papers, notebooks, and websites; 2) the occurrence of phone calls; 3) interruptions; 4) whether a patient is a new or a regular patient; and 5) whether a patient is accompanied by others. Primary analysis for this study utilized transcripts documenting the dialogue between patient and clinician. Observation notes and screen recordings were primarily used to obtain information such as visit type, to identify some consultation activities that were difficult to detect by audio recordings, and occasionally to resolve disagreements during data analysis.

All conversations were transcribed verbatim and then coded using NVivo 12. In a code-development phase, transcripts from the first 7 consultations were analyzed to identify distinct clinical activities with a common set of tasks offered by Waitzkin’s framework together with open coding. Waitzkin’s framework was chosen as the most suitable consultation model to guide coding because it takes a task-centered approach to describing consultation activities. Two researchers worked with a primary care doctor to agree on a final coding scheme after coding the transcripts of the first 7 pilot consultations. A single researcher, who was involved in coding the consultation transcriptions in the pilot phase, coded the rest of the transcripts according to the final scheme while using open coding as needed. The final decisions on the inclusion of newly generated codes were made by the 3 researchers. Observer notes and video-recordings were primarily used to help identify some activities that were difficult to detect by audio recordings only. The resulting coding scheme (Box 1) was used to code the consultations taking place after the pilot code development phase only (n = 31). The final analysis (n = 31) did not include the consultations from the pilot phase (n = 7). Additional details about the coding scheme and an extended table listing the consultation activities offered by Waitzkin’s original framework aligned with the activities in our final coding scheme are available at Supplementary Appendix 1 and 2, respectively. The coding process was focused on the communication between doctor and patient and the specific clinical activities underpinning these interactions. Therefore, nonverbal activities were identified only if there were any conversational cues to do so. Figure 1 shows a flow diagram of the data collection and analytical steps.

Figure 1.

The flow diagram showing the data collection and analytical steps.

Coding scheme for activities observed in a primary care consultation. Adapted from Waitzkin (1989)

Descriptive statistics were calculated for activity occurrences and transitions. Statistical analyses were not undertaken, given the small sample size and risk of bias from a single site study. An activity-transition analysis was performed using the transcriptions of doctor–patient conversations for all consultations. The analysis involved 1) labeling the sections of transcript with relevant activities, 2) identifying transitions between the activities, and 3) calculating the total number of transitions to and from each activity. An example of activity-transition analysis with transitioning phrases is depicted in Figure 2 in which present complaint has 1 connection with medical history, 4 with physical examination, 1 with drug history, 1 with assessment, 1 with treatment-prescribing, and 1 with other investigations. The analysis of 31 consultations resulted in a network diagram with activities as nodes and transitions between them as connections (using Gephi 0.9.2). Node sizes and link weights were calculated according to the total number of connections to and from an activity node for all activities, and Fruchterman Reingold centrality was used to draw the network diagram. The total number of connections between activity pairs to build the network diagram is available in Supplementary Appendix 4.

Figure 2.

An example activity structure from 1 single primary care consultation. Numbers reflect the temporal sequence of transitions from 1 activity to another. The case is a regular patient of the provider presenting with symptoms of gastroesophageal reflux.

An analysis of phrases identified typical phrases associated with termination or commencement of different activities. In this analysis, we examined the transitions between different activities in the spoken dialogue to see if there were clear spoken markers indicating that the activity was occurring, whether transition identification was subtler requiring an analysis of content shifts, or whether the switch was noticed by the observer from other signals in the room but could not be inferred from speech. Identifying transitions involved coding the relevant parts of transcripts based on the activity descriptions. A current activity code remained applicable to any new conversations until a verbal sign indicating the start of a new activity or a change in the current conversation topic was detected. Any uncertainties in assessments were resolved by relistening to the audio-recordings, checking the observer notes, conferring between annotators, getting opinions of other medical doctors, or searching medical knowledge databases.

RESULTS

In this study, 3 researchers observed a total of 40 individual patient consultations undertaken by 4 primary care physicians (2 male and 2 female) using the same EHR over 3 weeks. All physicians had at least 5 years of experience as a primary care physician using EHRs. Of all the recordings, 2 were excluded because of technical problems, and 7 recordings were used for the code-development phase only, leaving 31 consultations for the final analysis. The chief complaints of patients included reproductive health and urinary tract issues (6), musculoskeletal problems (5), preventive health concerns (5), migraines (3), skin problems (3), upper respiratory tract infections (3), dizziness (1), eye discomfort (1), gastroenteritis (1), and gastroesophageal reflux (1).

Activity frequency

There was wide variation in activity frequencies across consultations. A total of 370 separate activity occurrences were observed over 31 consultations (Table 1). Discussion of the present complaint was the most frequent, with a rate of 2.1 instances per consultation, accounting for 17.6% of all activity and appearing in 90.3% of consultations. Discussion of family history was the least common, with a rate of 0.2 instances per consultation, occurring in less than 20% of consultations and accounting for only 1.6% of all observed activities.

Table 1.

Frequency and rate of primary care consultation activities

Activity	n (%) of activity occurrences (total 370)	n (%) of consultations in which activity occurred (total 31)	Activity rate (n activity/31 consultations)
Present Complaint	65 (17.6)	28 (90.3)	2.1
Medical History	44 (11.9)	20 (64.5)	1.4
Assessment	34 (9.2)	21 (67.7)	1.1
Physical Examination	33 (8.9)	20 (64.5)	1.1
Nonpharmacological Treatment	30 (8.1)	17 (54.8)	1.0
Drug History	28 (7.6)	15 (48.4)	0.9
Treatment- Prescription	25 (6.8)	16 (51.6)	0.8
Treatment-Follow-up	22 (5.9)	21 (67.7)	0.7
Social History	21 (5.7)	16 (51.6)	0.7
Paperwork	16 (4.3)	15 (48.4)	0.5
Results Review	15 (4.1)	10 (32.3)	0.5
Allergies	12 (3.2)	4 (12.9)	0.4
Treatment Referral	12 (3.2)	7 (22.6)	0.4
Investigations	7 (1.9)	9 (29.0)	0.2
Family History	6 (1.6)	6 (19.4)	0.2

Activity sequence

Analysis of the temporal sequencing of activities and transitions in a consultation helped reveal the dynamic relationship between activities. Rather than following a linear order, with one activity completing and leading to the next, the data revealed a highly interactive, fragmented, and nonlinear process. Some activities were returned to several times and were highly interrelated with others. For example, in Figure 2, the time ordering of activities for a single consultation revealed that discussion of a patient’s present complaint triggered a physical examination, which, in turn, triggered further inquiry about the present complaint over several cycles.

To provide a high-level summary of the temporal ordering of activities, we generated a heatmap (Figure 3). Discussion of present complaint was the first activity for 84% of all the consultations in which it was present and reappeared as the third activity in 26% of these consultations. Discussion of past medical history was most likely to appear as the second activity (32%), but, like most of the common activities, it appeared at many points in activity sequences. Treatment activities, allergy, and paperwork discussions usually appeared in the last steps of the consultation process.

Figure 3.

The heatmap showing the temporal ordering of activities.

Activity network

Discussion about a patient’s present complaint was the most central activity in the consultation network and was highly connected to medical history taking, physical examination, and assessment suggesting that these together form a highly interrelated activity bundle (Figure 4). Remaining activities, such as family history or investigation discussions, were more peripheral and less connected than the central cluster suggesting less dependency on other activities. Supplementary Appendix 4 reports the underlying activity transition data with the total number of connections between each activity used in network construction.

Figure 4.

Network diagram summarizing connections between observed activities over all consultations. Node size and line widths are proportional to the frequency and number of connections for a given activity node, respectively.

Linguistic activity transitions

Our analysis of conversations revealed that transitions between activities could often be inferred from speech content. In some transitions, there were obvious verbal signs indicating a new activity, such as a doctor mentioning “family history” to signal that the conversation was now turning to taking family history. In other transitions, the receipt of specific information triggered an immediate suspension of the current activity and transition to another. For example, remarks by a patient about a previously used medicine triggered a transition from discussing the present complaint to taking a drug history. Not all transitions, however, involved such recognizable verbal activity- switching signals. Instead, a transition had to be inferred from the first few sentences within the next activity.

Although transition phrases took many forms (Table 2; Supplementary Appendix 3 with the full table), there seemed to be some underlying semantic similarities. For example, similar phrases indicated that the clinician was about to undertake a physical action—eg, “I’m now going to look/check/listen/put…” In our data set, most activity transitions were initiated by the doctors. Table 2 presents the examples of transition phrases used by doctors.

Table 2.

Example transition phrases between activities

Activity	Transition phrases
Present Complaint	What can I do for you today?
	What would you like to talk about today?
	Anything else I can do for you today?
Medical History	Do you have a history of …
	Do you have any other medical problems?
	Anything else in your history, like…
Drug History	Are you on any regular medication for anything?
	Do you take any medications?
	What medications are you taking at the moment?
Family History	Anything that runs in your family at all?
	Family history, mum and dad—are they alive and well and healthy?
	Any major health problems in the families or for your parents?
Social History	What sort of work do you do?
	What are you studying?
	How are you sleeping?
Allergies	Are you allergic to anything?
	Do you have any allergies to anything?
	Are you allergic to any medicine that you’re aware of?
Physical Examination	Alright, I’m going to put the speculum in.
	I’m just going to take a swab.
	Alright, let me have a listen to your chest.
Results Review	The x-ray doesn't look fantastic.
	I’ve got those blood test results here.
	You’ve come back to get your result?
Assessment	It’s probably just been strained and sometimes…
	So clearly that reflects that you’ve …
	There’s a chance that you could have something like…
Treatment-Prescription	I think probably the best option would be to try…
	I’ll give you the prescription for…
	It’s probably worth trying some tablets.
Treatment-Nonpharmacological	Try avoiding…
	Keep using the…
	You need to make sure that…
Treatment-Referral	Now the other thing is it might be worth seeing a specialist.
	Yeah, so you do need a referral.
	It’s probably worth it to get you to a specialist.
Treatment-Follow-up	Come back and see me in…
	You will get a reminder in…
	I could see you in 1 week then…
Investigations	I’ll give you a form for…
	I think the next step is probably to get an ultrasound of…
Paperwork	Do you need a medical certificate for that?
	I can give you a certificate.

DISCUSSION

We conducted an exploratory study to understand the types of activities that occur in a primary care consultation and the manner in which they unfold dynamically. Our findings suggest that consultation activities have fragmented structures resulting in a nonlinear flow. These results are in line with previous observational studies investigating activity structures and transitions. These studies focused on patient participation, characterization of doctor behavior, and socio-interactional considerations., They associate nonlinearity with problematic doctor–patient encounters or suggest that doctors should strive to follow consultation models with linear activity structures. In our data set, nonlinearity and fragmentation of activities were the norm and observed in almost all consultations (27 out of 31). Acknowledging the emergent and fragmented structures of activities as the norm may be essential for developing automated speech-based documentation systems capable of dealing with different consultation flows.

Our motivation behind this investigation on consultation activity structures was to support the design of digital scribe systems that can automatically generate some or all of the text needed to create a record of a consultation. Unlike classical dictation settings, wherein a clinician reports information that is transcribed verbatim, present day clinical consultations contain speech that is not intended to be directly translated into a record. The speech instead is used to ask questions, seek clarifications, and ensure there is shared understanding or common ground between patient and clinician. A digital scribe must therefore have the capacity to recognize not just the words that are spoken in a consultation but to understand their purpose. For example, similar clinical concepts will appear in the discussion of a patient’s past history, family history, and present complaint. Understanding the context in which the concepts are presented will be crucial to the generation of an accurate record.

Our analyses revealed that although consultations proceeded with an overall logic, the specifics of the patient’s problem and circumstances triggered the clinician to dynamically probe different parts of their history. The resulting fragmentation of activities like history-taking means that the conversation about different parts of a patient’s history may occur at multiple points in a consultation. Consequently, a digital scribe will need to not just recognize the meaning of speech but associate it with specific activities such as the different components of history-taking and aggregate similar components into a meaningful summary.

The content of the communication offers both local and high-level cues about the nature of the current activity. Locally, specific transition phrases may indicate that a transition is occurring and may also suggest the nature of the next activity (eg, transition phrases may indicate a physical examination is about to occur). Previous research has used conversational analysis to study specific activity transitions focusing on problem presentation, physical examination, diagnosis, treatment, and closing. Although not explored in this study, it may be possible to use nonverbal signs in doctor–patient interactions (eg, body gestures) in addition to verbal cues to detect activity transitions.

Our analysis identified higher level transition patterns across all activities. Future work can focus on developing probabilistic network models that capture the likelihood of transitions between defined activities as well as the likely clustering of some activity subsets. Approaches to modeling these different aspects of a consultation may involve both classic machine and deep learning approaches, which rely on significant volumes of training data, as well as using expert-driven heuristic algorithms.

Strengths and limitations

This study has several strengths. A pilot phase involving open coding informed the development of the final coding scheme. In the pilot code development phase, 2 researchers coded the transcripts and met regularly to exchange ideas and supporting research reflexivity. Multiple data collection methods involving observational data and audio and screen recordings were employed, which enabled the researchers to assess the activities performed more accurately.

There are also some limitations to this study. This was an exploratory study in a single practice involving a small number of doctors using a single EHR; it was not a definitive survey with any claim to comprehensiveness. Other consultation activities and activity structures are likely to be found in observations at other sites using a similar observational methodology. Such differences may relate to usage of different electronic records and decision support systems as well as differences in workflow, clinician training, and patient behaviors. Thus, whereas it is likely that the general characteristics of the clinical encounter recorded here have broad generalizability, it is unlikely that specific details would be as universal. General observations of high-level structure, such as nonlinearity, are thus more likely to be transferrable than observations such as specific transition cues between activities. The specific consultation styles observed may thus not be representative of the larger community of general practitioners. Since all the physicians in our study had at least 5 years of clinical experience, the network diagram reflected the competency of experienced primary care physicians. The network could be different for less-experienced physicians., Only a subset of the possible types of consultation or consultation activities was captured in this data set. For example, some classic activities like systems review or surgical history did not appear in our data set. The data set did not permit characterization of consultation differences such as those between first and repeat visits for a condition, patients with acute single conditions, and those with multi-morbidity or chronic conditions. Future work will require analyzing a larger sample of consultations with different characteristics.

The presence of recording equipment and an observer may have altered the nature of conversations and actions in the consultation and, thus, impacted on what was recorded. In this study, activity types and their transitions and relationships were inferred from transcripts of speech. It is possible that some activities which needed nonverbal data (such as gestures) to be detected were missed.

CONCLUSION

Clinical consultations in primary care appear to have a dynamic, fragmented, and nonlinear nature. In contrast to conventional linear models of primary care consultations, our analysis identified a nonlinear, sometimes reactive, network of activities. Developing automated documentation systems to translate the activities of a consultation into useful records and summary notes will need to consider these complexities in their design. Specifically, the task of automatically creating electronic records from digitally recorded signals, such as speech, will require systems with a rich underlying model of clinical consultations and not simply focus on limited tasks such as the translation of speech into text.

FUNDING

This research was supported by the National Health and Medical Research Council (NHMRC) grant APP1134919 (Centre for Research Excellence in Digital Health).

AUTHOR CONTRIBUTIONS

This study was designed by ABK, LL, EC, and SWi. Data collection was performed by ABK, HLT, and LL. Data coding and analysis was performed by ABK, LL, HLT, and EC. The first draft was crafted by ABK, LL, and EC; revisions and subsequent drafts were completed by ABK, EC, LL, HLT, SWh, JCQ, FR, and SWi. All authors approved the final draft.

SUPPLEMENTARY MATERIAL

Supplementary material is available at Journal of the American Medical Informatics Association online.

Click here for additional data file.

Consultation activity		Description
History-Taking	Present Complaint	Obtaining the chief complaint, problem history, symptoms, and any measures to relieve the symptoms
	Medical History	Obtaining past major medical illnesses and events
	Drug History	Obtaining use of any medications
	Family History	Obtaining deaths and major illnesses in the patient’s immediate family
	Social History	Obtaining occupation, education level, living situation, and lifestyle behaviors such as smoking and alcohol use
	Allergies	Obtaining any allergies such as drug or food
Physical Examination		Obtaining findings via use of observation, palpation, percussion, and auscultation
Results Review		Reviewing findings derived from test results
Assessment		Making diagnosis and diagnostic hypotheses
Treatment	Prescription	Prescribing drug intervention
	Nonpharmacological	Prescribing non-pharmacological treatment
	Referral	Generating referral to a specialist or another healthcare professional
	Follow-up	Making follow-up plans, eg, next appointment date
Investigations		Deciding on performing laboratory tests, x-rays, etc.
Paperwork		Generating patient letters such as medical certificates for school or work