Evaluating clinician acceptability of the prototype CanRisk tool for predicting risk of breast and ovarian cancer: A multi-methods study.

BACKGROUND: There is a growing focus on the development of multi-factorial cancer risk prediction algorithms alongside tools that operationalise them for clinical use. BOADICEA is a breast and ovarian cancer risk prediction model incorporating genetic and other risk factors. A new user-friendly Web-based tool (CanRisk.org) has been developed to apply BOADICEA. This study aimed to explore the acceptability of the prototype CanRisk tool among two healthcare professional groups to inform further development, evaluation and implementation.
METHOD: A multi-methods approach was used. Clinicians from primary care and specialist genetics clinics in England, France and Germany were invited to use the CanRisk prototype with two test cases (either face-to-face with a simulated patient or via a written vignette). Their views about the tool were examined via a semi-structured interview or equivalent open-ended questionnaire. Qualitative data were subjected to thematic analysis and organised around Sekhon's Theoretical Framework of Acceptability.
RESULTS: Seventy-five clinicians participated, 21 from primary care and 54 from specialist genetics clinics. Participants were from England (n = 37), France (n = 23) and Germany (n = 15). The prototype CanRisk tool was generally acceptable to most participants due to its intuitive design. Primary care clinicians were concerned about the amount of time needed to complete, interpret and communicate risk information. Clinicians from both settings were apprehensive about the impact of the CanRisk tool on their consultations and lack of opportunities to interpret risk scores before sharing them with their patients.
CONCLUSIONS: The findings highlight the challenges associated with developing a complex tool for use in different clinical settings; they also helped refine the tool. This prototype may not have been versatile enough for clinical use in both primary care and specialist genetics clinics where the needs of clinicians are different, emphasising the importance of understanding the clinical context when developing cancer risk assessment tools.

Introduction

In order to maximise opportunities to both prevent and detect cancer early, there is a need to identify people at higher risk, who may benefit from tailored screening and prevention [1-3]. There has been a growing focus on the development of multi-factorial cancer risk prediction algorithms, such as the Breast and Ovarian Analysis of Disease Incidence and Carrier Estimation Algorithm (BOADICEA [4-6]). The most recent version of BOADICEA combines genetic (e.g. rare genetic susceptibility variants and common genetic susceptibility variants in terms of a Polygenic Risk Score), family history, lifestyle and hormonal risk factors to calculate the future risk of developing cancer and the risk of being a carrier of a pathogenic variant [7]. The complex calculations included in BOADICEA have been validated in independent national and international prospective studies demonstrating good discrimination and accuracy [8-10].

Alongside the development of cancer risk prediction algorithms, researchers must also consider the design of graphical user-interfaces (GUIs) that enable the data entry required to populate the algorithm, the presentation of the risks calculated by the algorithm, and guidance on management of prevention strategies for that risk. These are fundamental to the successful implementation of risk prediction models into clinical practice. During the development stages of BOADICEA, a simple web-tool collected information to populate the algorithm. However, as the algorithm became more complex, there was a growing need for a more intuitive interface suitable for use in the clinical setting. As such, a new tool—CanRisk (www.canrisk.org), has recently been developed for healthcare professional use in a variety of clinical settings. Driven by the core principles of user-centred design [11], the core requirements of the CanRisk tool were initially influenced by clinicians from a wide range of clinical settings including primary, secondary and specialist care.

Acceptability is a multi-faceted construct that reflects the extent to which users consider an intervention appropriate, based on anticipated or experienced cognitive and emotional responses [12]. Whilst acceptability is a core facet of user-centred design, it is also an increasingly important component of evaluating the effectiveness of complex interventions [13, 14]. Furthermore, Sekhon et al. [12] suggest that successful implementation depends on the acceptability of the intervention to both intervention deliverers (healthcare professionals in the case of CanRisk) and receivers (patients). Assessing acceptability of any electronic risk assessment tool is particularly important during the beta stage of tool development as it enables further modifications to be made prior to evaluating the clinical utility of risk tools and how best to implement them [3]. Therefore, the aim of this study was to explore the acceptability of the prototype CanRisk tool among the healthcare professionals for whom it has been designed, in order to optimise further development, evaluation and, ultimately, implementation.

Methods

Design(s)

This study used a multi-methods approach. Participants completed a demographics questionnaire, two test cases (either face-to-face with a simulated patient or via a written vignette) and a semi-structured interview or equivalent open-ended questionnaire.

Participants

General Practitioners and Practice Nurses from primary care and Clinical Geneticists and Genetic Counsellors from specialist genetics clinics in London and the East of England were identified using an initial purposive sampling approach to account for clinical training and region, and enhanced by a snowball approach. Clinicians from specialist genetics clinics in France and Germany who were existing users of the previous simple web-based BOADICEA interface were also invited.

Ethical considerations

Following the receipt of an invitation to participate and a participant information sheet, participants gave full informed written consent. Participants were given the opportunity to withdraw up to and during study completion. As part of the study debrief, participants were reminded about the process for withdrawing their data. Ethical approval was sought and granted from the Cambridge Psychology Research Ethics Committee to cover all aspects of the study (PRE.2018.011). Additional approval for the study in England was granted by the Health Research Authority (Ref No: 18/HRA/1094); no additional approvals were required for the study in France and Germany.

CanRisk tool

The CanRisk Tool is a Web-based computer program developed by the Centre for Cancer Genetic Epidemiology at the University of Cambridge, and hosted on a University of Cambridge virtual server. The software is comprised of a number of components that form a series of graphical user interfaces (GUIs) to collect data required to run BOADICEA. This version of the CanRisk Tool was designed for use by healthcare professionals in a range of clinical settings. The tool prompts clinicians to enter the woman's risk factors (e.g. age, lifestyle, hormonal), genetic test results (if known) and family history data, and uses this information to calculate the woman's lifetime risk of breast and ovarian cancer and their risk of being a carrier of a pathogenic variant. Risk calculations are displayed in a variety of formats, including combinations of written text, graphs and icon arrays. For users in the UK, the tool also classifies the woman’s risk in accordance with the NICE guidelines [15]. No recommendations for risk management are provided. See Figs 1, 2 and 3 for screenshots of the prototype.

Fig 1

CanRisk tool initial page displaying the accordion menu of clinical topic areas included in the risk prediction calculation*.

* image corresponds to the revised version of the tool (not the beta version assessed).

Fig 2

CanRisk tool detailed view of one topic area: Personal Details*.

* image corresponds to the revised version of the tool (not the beta version assessed).

Fig 3

CanRisk tool detailed view of the interactive graphical pedigree editor [16]*.

* image corresponds to the revised version of the tool (not the beta version assessed).

Procedure

The research design aimed to standardise the content and quality of data collection across countries, so we used two methods of data collection for the study (face-to-face or on-line), determined by proximity to the participants. Participants were identified through local/national/international research networks (e.g. the Clinical Research Network in the UK) and were initially contacted by email. Participants were eligible for inclusion if they worked in a specialist genetics clinic or general practice and could give informed consent. Data were collected between April 2018 and August 2018, and the study was undertaken in participant’s clinical workplaces.

England

Those based in England participated face-to-face with the researcher. They completed an initial questionnaire collecting demographic data (age, gender, years in role, number of hours worked, any research qualifications) and information on their current use of computerised risk assessment tools (use of: web browsers, computer in clinic, electronic risk assessment tools, electronic tools to draw family tree). Participants then completed two simulated patient encounters (SPEs) using the CanRisk tool to produce a risk calculation. The SPEs were designed to emulate a typical appointment and were conducted in the participant’s normal clinical room with their normal computer. Two female professional medical role players took the part of the patients in the SPEs. Each role player was given a script (see Table 1 for an example) and rehearsed with the research team prior to the study. To standardise the SPEs across multiple participants, the actors were provided with the opening and closing lines of the consultation, information about the patient and their family and a list of comments to include (Table 1). Following completion of the SPE, participants took part in an audio-recorded face-to-face semi-structured interview (the interview schedule is available in S1 File).

Table 1

Example role player scenario and script used in England.

Summary	Sarah Jones has recently returned to (name and location) after spending a year teaching in Scotland. She has registered with the practice and made an initial appointment to see her GP requesting a mammogram.
Personal Details	Age:	49
	DOB:	4/2/1969
	Height:	5ft 5ins
	Weight:	9.4 stone
Family History	Daughter:	Olivia (DOB 3/4/97), lives locally. No history of cancer
	Mother:	Ann (DOB 2/2/1936). Died 2 years ago, aged 80. Initially diagnosed with breast cancer at 70. Underwent treatment and went into remission. Breast cancer returned aged 80 and she died within a couple of months.
	Father:	William aged 82 –No history of cancer
	Sister:	Jessica aged 55. She was diagnosed with breast cancer when she was 49. Now in remission.
	Maternal Aunt:	Helen aged 78 –no history of cancer
	Two paternal Uncles	Aged 78 and 76 –no history of cancer
Medical History	Children	1 Pregnancy—Gave birth to Olivia when she was 28. Breast fed for 8 months
	Oral contraceptives	Took the oral contraception pill aged 23–25 and 28–30 (6 years total)
	First period age	12
	Last period age	41
	Current medication	Hormone Replacement Therapy (Climmese)—has been taking for a year)
	Alcohol usage	1 glass of wine every couple of days
Ideas	She thinks that she is at high risk of getting breast cancer
Concerns	As both her mother and sister have a history of breast cancer, Sarah is worried that she is also at risk. Her sister was diagnosed with cancer when she was 49 and Sarah had her 49th birthday in February. Therefore, it is particularly playing on her mind that she could be at risk of getting cancer too.
Expectations	To be given a mammogram.
Opening Line	‘I’d like to have a mammogram please.’
Closing Line	‘Thank you for seeing me’ or ‘Thank you for referring me’ (depending on the outcome of the discussion) ‘What happens next?’
Other comments to include	If you see ovarian cancer on the screen–ask about it. ‘Why does it say ovarian cancer on the screen?’If shown the graph, point to the upper line and say ‘That looks much higher/bigger than expected’

International

Participants from specialist genetics clinics in France and Germany took part in an online version of the study via the Web-based survey tool Qualtrics [17]. They completed the same initial questionnaire, followed by two vignette-based test cases, adapted from the scenarios used in the English setting but with the same risk factors as in the SPE cases. Demographic information pertaining to a fictitious patient was provided for each case alongside details of their family history (see Table 2 for an example). Participants were asked to enter the information into the CanRisk tool and evaluate how it would work in a real-world setting. Participants were also prompted to take notes as they used the CanRisk tool in order to complete a detailed open-ended questionnaire (again drawn closely from the interview schedule) exploring their user experiences.

Table 2

Example vignette based case used in France and Germany.

Background	Sarah Jones is a new patient who has just moved from Scotland to London and wants to organise a mammogram because of her family history of breast cancer.
Personal details	Sarah was born 4 February 1969, is 165cm tall (5'5"), and weighs 60kg (9st4lb, BMI 22.0). She drinks one glass of wine every couple of days. She has no previous diagnosis of breast disease. Her first period was at age 12, and she took an oral contraceptive pill between the ages of 23 and 25, and again aged 28 to 30. She has one child, a daughter (born 1987, when Sarah was 28 years old), who was breastfed for 8 months. She had her last period when aged 41 years, and has taken hormone replacement therapy, Climesse, for the last year. She otherwise takes no medication, has no allergies and is in good health.
Family History	Sarah's mother (deceased age 80, born 1936) was diagnosed with breast cancer aged 70 and went into remission following treatment; the cancer returned aged 80. Her sister was diagnosed with breast cancer at 49, and she is now a healthy 55 year old. Her relatives are otherwise unaffected, including her daughter, father (aged 82), two paternal uncles (aged 78 and 76), and one maternal aunt (aged 78).

Analysis

Demographic data and information about current use of computerised risk tools were summarised. Qualitative data from the open-ended questionnaires and verbatim transcriptions from the semi-structured interviews were analysed inductively using thematic analysis [18]. In order to become familiar with the data, two authors (SA and FS) repeatedly read the interview transcripts, and notes were made about potential codes; these were subject to ongoing discussion and refinement. Following this, the same authors performed formal coding using NVivo [19], working through the transcripts and writing notes to support data patterns–these patterns were extracted as codes. Consistency of coding between SA and FS was discussed regularly through team meetings, but formal analysis of the concordance between reviewers was not completed. The codes were drawn together into a broad set of initial themes; these were refined with guidance of the senior author (FW). The final themes were mapped onto Sekhon et al.’s Theoretical Framework of Acceptability [12] which was developed to guide the assessment of acceptability from the perspectives of intervention deliverers and recipients both prospectively and retrospectively. The framework proposed seven component constructs: affective attitude, burden, perceived effectiveness, ethicality, intervention coherence, opportunity costs, and self-efficacy.

Results

Demographics

In England, 37 healthcare professionals participated in the study, including 11 General Practitioners (GPs), 10 Practice Nurses (PNs), 6 Clinical Geneticists (CGs) and 10 Genetic Counsellors (GCs) (Table 3). In their patient consultations, most (31/37, 84%) were already using computerised risk tools, relating to cancer (n = 14; GP = 2; CG = 4; GC = 8), cardiovascular (n = 18; GP = 10, PN = 8), and diabetes (n = 4; GP = 1, PN = 3) risk.

Table 3

Participant demographics.

	Primary Care			Specialist Genetics Clinics
	Total(n = 21)	General Practitioner (GP)n = 11	PracticeNurse (PN)n = 10	Total(n = 54)	Clinical Geneticist (CG) n = 18	Genetic Counsellor (GC) n = 36
Age
Under 30	2	2	0	13	1	12
31–40	8	5	3	24	9	15
41–50	5	3	2	13	7	6
51+	6	1	5	4	1	3
Gender
Male	3	3	0	11	5	6
Female	18	8	10	43	13	30
Years of Experience
Mean	11.1	9.7	12.6	7.3	7.8	6.8
SD	11.0	8.8	13.3	5.7	5.9	5.5
Range	0 to 45	1 to 25	0 to 45	0 to 25	2 to 23	0 to 25
Current users of a computerised risk tool
Cancer	2	2	0	48	15	33
Cardiovascular	18	10	8	0	0	0
Diabetes	4	1	3	0	0	0
Other	8	5	3	0	0	0
Nationality
English	21	11	10	16	6	10
French	0	0	0	23	3	20
German	0	0	0	15	9	6

In France and Germany, 68 people from specialist genetics clinics were approached, and 42 (62%) participated in the study. Four respondents were excluded from the analysis as they were not practising clinicians. The final sample of 38 healthcare professionals comprised 26 Genetic Counsellors (GCs) and 12 Clinical Geneticists (CGs) from specialist genetics services in France (n = 23) and Germany (n = 15). Most (36/38, 95%) were already using computerised risk tools in their consultations to calculate cancer risks.

For the purposes of this analysis, data from GCs and CGs in England, France and Germany were combined into the specialist clinical genetics sample (n = 54). The primary care sample was from GPs and PNs in England alone (n = 21). Detailed participant demographics for each of these groups are displayed in Table 3.

The thematic analysis generated 17 themes—these were organised around the seven theoretical constructs outlined by Sekhon et al., [12]; Fig 4 shows a thematic map.

Fig 4

Inductive themes mapped onto Sekhon et al’s Theoretical Framework of Acceptability [12].

The total number of times the theme occurred is presented after each theme name (e.g. n = x). The yellow (primary care) and green (specialist genetics clinic) colouring shows the proportional contribution of each sample to the theme.

‘Affective attitude’: How clinicians felt about the CanRisk tool

Affective attitude was directly influenced by participants’ first impressions. For specialist genetics clinicians, the majority of whom had used the previous simple web-based BOADICEA interface, there was a clear improvement:

“The CanRisk tool is an improvement to any other risk assessment tool based on BOADICEA. It is straightforward and has a simplistic design, and is therefore intuitive to use” (GC, Germany).

Clinicians from the specialist genetics setting anticipated that the CanRisk tool would be used frequently before and/or during their clinics, and would influence further testing and treatment decisions. In contrast, whilst some primary care clinicians had a positive outlook on the appropriateness of using the CanRisk tool in their setting, they perceived that the CanRisk tool would not be used frequently:

“I thought it was completely something reasonable for a GP to engage in or carry out. So perfectly reasonable consultations and appropriate and relevant. So it's the sort of thing that would happen, not frequently, but I would see these sorts of cases certainly” (GP, England).

Furthermore, several primary care clinicians were concerned that the CanRisk tool was not contributing to a management plan for their patient, and was simply providing more data on a risk they had already calculated personally:

“I’m not sure how useful it is in the primary care setting because all I’m doing is, I’m using that to collect data, and I’m just thinking it’s going to calculate my risk and it’s not using it to make any clinical decision” (GP, England).

Participants from both settings expressed views on whether the CanRisk tool could be, and should be, used in face-to-face consultation with patients. Some felt that the CanRisk tool was an asset for structuring complex conversations around factors contributing to cancer risk and involving patients in the consultation:

“It’s a good way of introducing conversation about risk and about risk perception. And as long as you’re doing it with them looking at the screen, so there’s that involvement” (GC, England).

Others were more concerned about using the CanRisk tool alongside patients due to the ‘real time’ nature of risk communication. Some clinicians described their anxieties about the lack of opportunity to interpret the risk scores before sharing the relevant information with the patient:

“I wasn’t sure what the risk was going to be calculated as, so I didn’t know what to expect, and we were going to get that at the same time, and I didn’t know how to convey that information when you can both see the numbers there and then” (GC, England).

In order to provide the best experience of care for patients in this time pressured environment, participants highlighted that they would need to be confident in their understanding, interpretation and communication of multi-factorial cancer risk; this is discussed further in a later theme on self-efficacy.

‘Ethicality’: How the CanRisk tool fits with the clinician’s value systems

Ethicality encompassed participants’ concerns about how the CanRisk tool changed the dynamic of the consultation. Clinicians from both settings were worried about issues such as reduced eye contact, reduced time spent building rapport, and an altered flow to their normal consultation.

“I think my interaction and communication with the patient was terrible. I didn't get to interact with them, I didn't get to deal with their anxieties around the issue. I was more focusing on inputting data into the tool” (GP, England).

Further to this, participants from the primary care setting were concerned about the depth and nature of some of the questions, particularly around pregnancy and baby loss, describing them to be intrusive and insensitive:

Participant: “That’s a bit horrible, isn’t it? Please include stillbirths, but it’s not necessary to include miscarriages”.

Interviewer: “How could we improve that do you think?”

Participant: “I don’t know. You want it to be clear and it’s better to be upfront. It’s easy for me in that I can see what you want, but then it’s a little bit… to ask that question. I didn’t ask either of those women that question” (PN England).

Clinicians in the specialist genetic setting were also concerned that the patients may feel judged for engaging in particular behaviours (i.e. alcohol), being overweight or for not breastfeeding their child:

“I’m thinking if I ask questions about breastfeeding and alcohol, it’s going to make no difference to the screening recommendations we make. It’s actually just going to make someone go away feeling a bit shit because they were told they drink too much and they didn’t breastfeed” (GC, England).

In order to allay these concerns, participants discussed the need for clear explanation about the CanRisk tool, how it works, and why particular types of information were needed in order to achieve the best possible risk estimation. To this end, time in the consultation to achieve this (or a lack of it in many cases) was seen as a major limitation, particularly in primary care, but not a complete barrier to use.

‘Burden’: The effort required to use the CanRisk tool

All participants discussed burden, or the alleviation of it. Some primary care participants described how the CanRisk tool could improve their clinical practice, and that ease of use was an important factor:

“It’s quite simple to use and I like the fact that it’s got the links, so I can link straight over to the NICE guidelines, I can go straight to the public health and look at the breast screening programme” (PN, England).

Other primary care participants felt that the CanRisk tool was confusing, particularly the aspects that were unfamiliar in their normal clinical practice (e.g. the completion of extended and detailed family histories); in some cases, they were concerned that this could impact on the risk calculation:

“It’s [family history] quite difficult to put in and then if we’re guessing, how accurate is the result? So, potentially, I could be putting someone in a much higher category than they actually are and then they’re going to go away and worry, and then they go to the hospital and actually they say, no you’re low risk” (PN, England).

Participants from both settings thought that using the CanRisk tool in clinical practice could complicate the consultation, particularly when patients might not have all of the information readily available. Clinicians from the specialist genetics setting felt that the CanRisk tool was unnecessarily complicated for a simple risk stratification as most of their patients have significant family histories that would not be moderated by other risk factors:

“So, this tool, in a way, made things a little more complicated for me than it needed to be, I suppose: there are certain categories of women that automatically will fall into that high risk, for me” (CG, England).

Primary care clinicians were clearly concerned about the amount of time needed to complete the CanRisk tool. They were apprehensive about their ability to complete the tool, achieve a good patient experience and initiate future management planning within a short consultation:

“It’s just whether you have time to do it. I don’t think you will have time to go through the whole thing without sort of establishing rapport initially, and finishing and closing at the end” (GP, England).

Specialist genetics clinicians rarely described the time needed to complete the tool as a limiting factor. Indeed, they reported having fewer time pressures due to the nature of their clinical set-up, which generally features longer appointments (up to 45 minutes in England) and often affords greater opportunities to prepare for each consultation.

‘Intervention coherence’: Understanding the CanRisk tool and how it works

Intervention coherence focused predominantly on participants’ beliefs about the accuracy of the risk assessment and the trust they had in the model. For some participants, endorsement of the CanRisk tool by NICE or other trusted authorities instilled greater trust:

“So, from the cancer risk point of view, obviously, Macmillan or anything like that, or Cancer Research I’d be thinking oh yes, or if it’s got NICE approval as well, then I think yes, okay, that’s good” (PN, England).

Adoption of the CanRisk tool within a clinical system (i.e. a hospital or GP practice) would also give some participants a feeling of assurance that it was appropriate for use in that clinical setting:

“If [Clinical IT Specialist] has deemed, and the doctors have deemed, it safe to embed into our templates, then I would trust that that's okay for me to use” (PN, England).

Whilst some participants were happy to rely on the risk calculation given by the CanRisk tool because of its reputation in their clinical community, others expressed concerns about receiving a risk calculation that did not fit with their clinical judgement:

“I have a figure that I’m expecting, and I find that very handy because if it’s wildly different, it alerts me that there could be concerns” (CG, England).

Specialist genetics clinicians found a discrepancy between the CanRisk calculation and their clinical judgement particularly challenging, as many were used to calculating risk based solely on family history information:

“So, you see, when we calculate risk, we don’t take that into account, and we don’t take HRT into account, or contraceptive pill into account, at the moment. So, it’s very much based on the family history. So, actually, having that be a bit higher, it just makes you think, actually, lifestyle factors may play a part, yes” (GC, England).

Some participants were also concerned about the impact of incomplete information about any of the risk factors, particularly lifestyle and hormonal factors, might have on the risk calculation:

“A lot of personal information is asked. What about patients [for] who we don't have all these items—the cancer risk is not as precise?” (GC, France).

Many of these concerns were underpinned by a lack of knowledge about how the calculation was made, leaving some participants feeling uncomfortable about the reliability and specificity of the risk calculation:

“[We’d] say, “This is one model we’ve got, one tool we’ve got for estimating your risk and it’s coming out saying your risk is about two percent.” We’d say that all datasets are incomplete, there may be new data and all sorts of things, but this is the current model. So I think in terms of regulation, it’s really important that that uncertainty is somehow captured when it’s being used in different clinical situations” (GC, England).

Some participants felt that the calculated risk figure needed to include a caveat to better represent the uncertainty. Participants wanted greater transparency to ensure that healthcare professionals and their patients were fully informed about the limits of both the BOADICEA model and the CanRisk tool.

‘Opportunity costs’: The sacrifice of benefits, profits and values

Opportunity costs were seldom discussed by participants in this study. Only data security, in terms of the safety of data entered into the CanRisk tool, was seen a potential problem. For some, there were concerns about data protection, specifically around that integration of risk prediction tools into existing IT systems:

“I think that’s a legitimate concern, we have electronic records and some people are worried about that, and so having online tools where you’re putting in all of this personal information, putting in your family members, that might spark some patient concerns” (GC, England).

Other concerns focused on the security of the Web more generally, the chances for data to be hacked, and whether any secondary use of the data was planned. None of the participants reported that these concerns would stop them from using the CanRisk tool, but they acknowledged that these points should be addressed in any guidance documentation.

‘Perceived effectiveness’: The ability of the CanRisk tool to achieve its purpose

Perceived effectiveness was a central tenet of acceptability for most participants in this study, with views on efficacy being linked to positive attitudes towards the CanRisk tool. Some participants believed that the use of the CanRisk tool promoted more thorough data collection and therefore improved the quality of information:

“So that's good, because you don't miss information and it's quite tidy. And, in your mind, you can understand where you're going with the information. And, actually, it's not even very alarming to the patient, because they can see where you're headed with the questions” (GP, England).

Following completion of the risk calculation, many participants talked about the benefits of having varied and versatile formats to help them communicate the patient’s risk:

“It was really helpful offering various versions [of risk communication]. At least one should fit for all of the patients—it is sometime helpful explaining the same message using different paths” (CG, Germany).

However, several PNs and GCs felt that the range of options for communicating risk were overwhelming. In some cases, the additional information was felt to be unnecessary as the risk calculation forms just one part of the wider consultation:

“Sometimes you won’t need to go into any more detail than moderately increased risk. Sometimes that’s absolutely fine. Moderately increased risk, you know, and actually the rest of the consultation might be about their feelings surrounding coming up to the age at which their mum was diagnosed and their thoughts about that” (GC, England).

In general, clinicians from the specialist genetics setting were confident about the next steps following a risk calculation; in some cases the tool was used to support a decision for a particular type of treatment (e.g. risk-reducing mastectomy). Conversely, primary care participants talked about their uncertainty about the next steps following the risk calculation and there was a general frustration about the lack of information on clinical management:

“I would know the risk stratification off the top of my head and I would know what to do next with it. But I'd know there is some guidelines somewhere that I need to follow in terms of whether or not they need a referral or not. But if we are going to use a risk stratification tool, then I would want some sort of, and you do this next, it's this is the risk, deal with it, GP” (GP, England).

Following this, some PNs were concerned about how and when to pass the case over to a GP colleague, particularly if the patient was identified as being at moderate or high risk. Some PNs were conscious of their lack of knowledge of the next steps, and this affected their confidence in being able to provide appropriate information to their patient:

“I don’t even know what, how quickly, people would get assessed for BRCA, I wouldn’t know if I send them here, if they’d have to go to Place 1, I really don’t know, so, because it’s not something I’ve done” (PN, England).

Whilst some participants believed that the CanRisk tool could help regulate the collection of data and structure conversations around risk, for the majority, the desire to develop confidence and competence when using the CanRisk tool was integral to its acceptability.

‘Self-efficacy’: Clinician’s confidence in using the CanRisk tool

Self-efficacy was predicated upon a need for additional training. Participants felt that training should focus on how to best use the CanRisk tool, promoting a sense of proficiency:

“We should be familiar with the tools that we’re using, so that we use them properly. And therefore, I think there should be a training component to it. Otherwise, you make mistakes” (CG, England).

All participants wanted to be able to feel comfortable about using the CanRisk tool in a consultation. Those from the specialist genetics clinics felt more confident due to their experience of using the simple web-based BOADICEA interface. In addition, their specialist training left them comfortable communicating cancer risk to their patients, but some struggled to describe the impact of non-genetic factors. Conversely, whilst most primary care participants were confident in their communication skills around multi-factorial risk, some were concerned about their limited genetics knowledge:

“Genetics, not part of my normal knowledge. Even in medical school you do a tiny, weeny bit of it, it's not, that is an absolute specialist area” (GP, England).

Whilst only one PN said they were ‘not so sure this would be an appropriate thing for somebody of my level of nursing to do’ (PN, England), many other primary care participants stated that they would need additional training on interpreting genetic risk and its communication:

“So you've got to have an education package that goes with it, to tell people what it's about, why all the things are on there and actually how to interpret it. I think most of us, as doctors, my sort of age, never had any consultations since training, we'd have never been taught how to convey risk” (GP, England).

Whilst the training needs of participants varied depending of the setting and/or the clinical training and experience of the participant, there was commonality in terms of a desire to provide the best care for their patients.

Discussion

Our findings suggest that the prototype CanRisk tool was broadly acceptable to the majority of participants. Most clinicians found the CanRisk tool easy to use due to its intuitive design. For geneticists and genetic counsellors who were users of the simple web-based BOADICEA interface, CanRisk offered a more user-friendly experience and the standardised format for data collection was of benefit to many. The foremost concern for clinicians from the primary care setting was the amount of time needed to complete and interpret a risk calculation, highlighting a potential barrier to the use of the CanRisk tool in this setting. Clinicians from both settings were also apprehensive about the impact that the CanRisk tool had on their consultation and the lack of opportunity to interpret the risk score before sharing it with the patient.

Many participants found the CanRisk tool to be intuitive; however, the majority were from specialist genetics clinics with previous experience of the simple web-based BOADICEA interface. All participants were concerned about potentially unhelpful changes to their consultation style influenced by the inclusion of detailed questions in the prototype that may appear to be unrelated to cancer risk, and increased time spent looking at the computer screen, consistent with the ‘third party effect’ [20]. In order to allay these concerns, a period of familiarisation and training prior to using the CanRisk tool for the first time in the clinical setting is advisable. This would allow healthcare professionals to navigate the CanRisk tool confidently, select which risk factors are important for that patient, and collect information on these in a way that is consistent with their consultation style [20, 21].

Many participants were also concerned about collecting the extra information needed for this multi-factorial cancer risk prediction tool. This was particularly evident when these topics are not routinely explored such as lifestyle factors by clinical geneticists and genetic counsellors and extended detailed family histories for GPs and practice nurses. Primary care clinicians were particularly concerned about the amount of information required for risk prediction as they had limited time available during their consultations [22]. In order to improve the acceptability of the CanRisk tool, the burden of data collection must be addressed. Some areas of the CanRisk tool could be pre-populated from existing clinical records; this would require the CanRisk tool to be embedded in electronic medical records (e.g. EMIS or SystmOne for primary care, or EPIC for hospital care). Another approach could be for patients to complete an electronic pro-forma. Whilst these two potential solutions may sound simple, the technical and regulatory requirements may make it difficult to navigate on a local and national level [23, 24], and may add further concerns around data security.

The study findings suggest that endorsement from appropriate clinical bodies is essential for confidence in adopting new technologies into the clinical setting. Whilst BOADICEA is referenced in the NICE clinical guidance for familial breast cancer (CG164) [15], clinicians may not be aware of the association between BOADICEA and the CanRisk tool. Including a reference or link to the CanRisk tool specifically within the guideline may increase visibility of the tool, strengthen the association between BOADICEA and the CanRisk tool, and increase adoption. The addition of CE marking to the CanRisk tool, indicating conformity with health, safety and environment protection standards for products within the European Economic Area [25], is also likely to support acceptability across the clinical settings.

Although some clinicians regarded the variety of risk presentation formats as a useful feature, others found them to be overwhelming. Further consideration of the risk presentation formats for each job role and/or clinical setting may bolster the clinician’s confidence, assist in quick interpretation and improve patients’ understanding and perception of risk [26]. In order to achieve shared understanding, incorporating layers of information, where details about the risk and its parameters are displayed in an incremental and controlled fashion, may be of benefit to both patients and staff [27]. Additional learning resources focusing on the communication of multifactorial risk for those in the specialist genetics setting and genetic risk for those in the primary care setting may also support clinicians to communicate the results more effectively. The development and testing of these resources should be considered prior to implementation.

The analysis highlighted a number of factors influencing participants’ views on the perceived effectiveness of the final CanRisk tool. Specifically, there were concerns about the validity of the output when there is missing information; more detailed information regarding the complex calculations performed in BOADICEA and the contribution of each risk factor to the model may be required. Information on the amount of error within each calculation should be included to better inform users about the level of uncertainty. This may help develop patient centred understanding of the risk score and cancer risk more broadly and what role this information may have in future care decisions [28]. The analysis also emphasised a need for the CanRisk tool to include clinical guidance on managing patients following a risk calculation. This was particularly important in the primary care setting, where most management of women found to be at population and moderate risk for breast cancer is likely to take place.

Limitations

This study collected data on acceptability from a variety of clinician groups, using a range of methods; whilst this allowed us to achieve both breadth and depth, it also resulted in a number of limitations. Firstly, this analysis was generated using data from both semi-structured interviews and questionnaires; the use of questionnaires to collect detailed experiential information was pragmatic, but the opportunity to explore some issues in more detail was limited. Secondly, no GPs contributed data in the international sample; this restricted the feedback from an integral group of clinicians and limits the generalisability of this research to the international primary care community. Thirdly, as the majority of clinicians from the specialist genetics clinics had previous experience of using BOADICEA in their clinical practice, some aspects of the CanRisk tool were already familiar; this may have amplified the differences between clinicians from primary care and specialist genetics clinics.

Next steps

Further development and testing of the CanRisk tool is required. The findings presented here have already underpinned a number of changes to the prototype CanRisk tool (see Table 4). Whilst these changes may go some way to improving the tool for clinical users in a variety of settings, further research exploring the patient experience of the tool and its use in clinical practice is necessary. Data collected in these studies would allow us to understand the acceptability of the CanRisk tool in this important stakeholder group and may inform future patient centred design.

Table 4

Summary of changes made to the prototype CanRisk tool.

Area of concern	Setting	Changes made to later version
Differences in practice between the two settings	Both	The tool has now been split into a CanRisk Genetics version for use in the specialist genetics setting and a CanRisk Core version for use in primary care.
Lack of approval from regulatory body	Both	The final version of the tool will carry CE marking.
Lack of information on how to manage the risk	Primary care	CanRisk Core will include better signposting to relevant management options and provide links to relevant parts of the NICE guidance.
Difficult questions leading to discomfort (e.g. pregnancy and baby loss) or judgement (e.g. breastfeeding, alcohol intake and weight)	Both	Questions collecting data on factors that do not currently contribute to the risk model have been removed.A description of why other data are collected has been included in the FAQs.
Completing a detailed family history	Primary care	Collection of a limited family history (i.e. first/second degree relatives) will be required for CanRisk Core.
Lack of information about how the data required for the calculation were to be used and stored	Both	Enhanced information on the privacy policy has been included at registration. An additional embedded webpage called Privacy and Cookies has also been included with detailed information.
Need for training to use the tool effectively	Both	Following registration, users are taken to a quick start guide to familiarise themselves with the main functionality of the tool. In addition, an updated and searchable section of Frequently Asked Questions has been included.
Further training on genetic risk communication	Primary Care	The CanRisk Core version of the tool will include refined risk communication formats, enabling easier interpretation in the time-limited environment.

Conclusion

The prototype CanRisk tool was generally acceptable to our participating clinicians. However, the findings highlight the challenges associated with developing a complex tool for use in different clinical settings. The next version of the CanRisk tool is likely to benefit from having core and clinical setting-specific components to increase acceptability across the clinical settings and, ultimately, future implementation.

(DOCX)

Click here for additional data file.

2 Jan 2020

PONE-D-19-33335

Evaluating clinician acceptability of the prototype CanRisk tool for predicting risk of breast and ovarian cancer: a multi-methods study

PLOS ONE

Dear Drs. Archer,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

We would appreciate receiving your revised manuscript by Feb 16 2020 11:59PM. When you are ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter.

To enhance the reproducibility of your results, we recommend that if applicable you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). This letter should be uploaded as separate file and labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. This file should be uploaded as separate file and labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. This file should be uploaded as separate file and labeled 'Manuscript'.

Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.

We look forward to receiving your revised manuscript.

Kind regards,

Alvaro Galli

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

http://www.journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and http://www.journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please include additional information regarding the survey or questionnaire used in the study and ensure that you have provided sufficient details that others could replicate the analyses. For instance, if you developed a questionnaire as part of this study and it is not under a copyright more restrictive than CC-BY, please include a copy, in both the original language and English, as Supporting Information.

3. Please provide additional details regarding participant consent.

In the ethics statement in the Methods and online submission information, please ensure that you have specified what type of consent you obtained (for instance, written or verbal, and if verbal, how it was documented and witnessed). If your study included minors, state whether you obtained consent from parents or guardians.

4. Thank you for sending us the data set underlying the results presented in your PLOS ONE submission. We notice that some of the information included in the data set may be potentially identifying. Please ensure that the data shared are in accordance with participant consent and provide only the data that are used in this specific study. To ensure patient confidentiality, we would recommend removing any identifying information in Box 2 of the manuscript. Additional guidance on preparing raw clinical data for publication can be found in our Data Policy FAQs (https://journals.plos.org/plosone/s/data-availability#loc-clinical-data).

5. In your Methods section, please provide additional information about the participant recruitment method and the demographic details of your participants. Please ensure you have provided sufficient details to replicate the analyses such as:

a) the recruitment date range (month and year),

b) a description of any inclusion/exclusion criteria that were applied to participant recruitment,

c) a description of how participants were recruited, and

d) descriptions of where participants were recruited and where the research took place.

6. We note that you have indicated that data from this study are available upon request. PLOS only allows data to be available upon request if there are legal or ethical restrictions on sharing data publicly. For information on unacceptable data access restrictions, please see http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions.

In your revised cover letter, please address the following prompts:

a) If there are ethical or legal restrictions on sharing a de-identified data set, please explain them in detail (e.g., data contain potentially identifying or sensitive patient information) and who has imposed them (e.g., an ethics committee). Please also provide contact information for a data access committee, ethics committee, or other institutional body to which data requests may be sent.

b) If there are no restrictions, please upload the minimal anonymized data set necessary to replicate your study findings as either Supporting Information files or to a stable, public repository and provide us with the relevant URLs, DOIs, or accession numbers. Please see http://www.bmj.com/content/340/bmj.c181.long for guidelines on how to de-identify and prepare clinical data for publication. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories.

We will update your Data Availability statement on your behalf to reflect the information you provide.

7. We note that you have a patent relating to material pertinent to this article.

Please provide an amended statement of Competing Interests to declare this patent (with details including name and number), along with any other relevant declarations relating to employment, consultancy, patents, products in development or modified products etc. Please confirm that this does not alter your adherence to all PLOS ONE policies on sharing data and materials, as detailed online in our guide for authors http://journals.plos.org/plosone/s/competing-interests by including the following statement: "This does not alter our adherence to  PLOS ONE policies on sharing data and materials.” If there are restrictions on sharing of data and/or materials, please state these. Please note that we cannot proceed with consideration of your article until this information has been declared.

This information should be included in your cover letter; we will change the online submission form on your behalf.

Please know it is PLOS ONE policy for corresponding authors to declare, on behalf of all authors, all potential competing interests for the purposes of transparency. PLOS defines a competing interest as anything that interferes with, or could reasonably be perceived as interfering with, the full and objective presentation, peer review, editorial decision-making, or publication of research or non-research articles submitted to one of the journals. Competing interests can be financial or non-financial, professional, or personal. Competing interests can arise in relationship to an organization or another person. Please follow this link to our website for more details on competing interests: http://journals.plos.org/plosone/s/competing-interests

8. PLOS requires an ORCID iD for the corresponding author in Editorial Manager on papers submitted after December 6th, 2016. Please ensure that you have an ORCID iD and that it is validated in Editorial Manager. To do this, go to ‘Update my Information’ (in the upper left-hand corner of the main menu), and click on the Fetch/Validate link next to the ORCID field. This will take you to the ORCID site and allow you to create a new iD or authenticate a pre-existing iD in Editorial Manager. Please see the following video for instructions on linking an ORCID iD to your Editorial Manager account: https://www.youtube.com/watch?v=_xcclfuvtxQ

9. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: N/A

Reviewer #2: N/A

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: No

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The authors presented the qualitative findings from their study of CanRisk, a web-based version of the BOADICEA model for predicting breast and ovarian cancer risk, among general practitioners and genetic specialists in the UK, France, and Germany. A strength of their study is their use of qualitative data from practitioners to propose changes to their tool to allow for greater applicability in the primary care setting. The authors should address the following comments:

1) The authors should report a kappa on the concordance of the coding of the transcripts between the two coders.

2) In the discussion, the authors described barriers to the use of the tool in the primary care setting largely because of competing demands and time constraints in clinical practice. The authors should acknowledge the limitations of using this tool in a busy primary care practice.

2) In future studies, the authors could consider evaluating the patient experience in using this tool, as patient participation is also vital to the success of such an intervention. There were concerns raised by practitioners using the tool regarding insensitive and intrusive questions (for example, regarding stillbirths and miscarriages); patient feedback in this area would be particularly relevant. The authors also discuss potentially using family history intake via the patient portal and patient feedback would also be helpful in creating this.

3) Another potential challenge to the implementation of this tool raised by authors is the interpretation and communication of risk estimates to patients by primary providers (not the genetics specialists). This is clearly an unmet need in primary care, and education regarding communicating cancer risk and options for risk management are clearly required for this tool to be useful in a generalist practice.

Reviewer #2: This manuscript describes a useful qualitative evaluation of a web-based tool to apply the BOADICEA risk prediction tool in clinical practice. The investigators tested the tool with and received comments from both general practitioners and specialists in genetics clinics, allowing them to assess the strengths and weaknesses of the tool in different practice settings. Overall, the paper was clearly written and the findings were thoughtfully interpreted.

Specific comments on the paper are:

1. Abstract: The summary of the results in the abstract is quite vague and doesn’t highlight some of the key findings described in the paper. In particular, the authors simply name the key themes and don’t really describe some of the major findings including concerns about the time needed to use the tool in primary care or how to address the next steps after risk assessment.

2. Introduction, first sentence (minor comment): The sentence states that “…there is a need to identify people at higher risk for tailored screening and prevention”, which is an ambiguous sentence structure. It would be preferable to state it as “…there is a need to identify people at higher risk, who may benefit from tailored screening and prevention”.

3. Box 1: There is inconsistency in the information presented regarding the subject’s birthdate (4/2/1969), her daughter’s birthdate (3/4/1997) and the age at which she gave birth (18). Based on the birthdates, the mother was 28, not 18, when she gave birth to her daughter.

4. Page 9, Analysis (minor comment): Clarify that “these were refined with guidance of FW” is another author.

5. Results: Although it is recognized that this was a qualitative analysis, it would be useful to quantify how frequently some of the comments were made. For example, how many times did clinicians express concerns about the time it would take to use the tool or how many had concerns about the questions being too intrusive or upsetting. It would help the reader to know whether the concerns were reported by the many clinicians or only by one or two people. If stratified by type of practitioner, it could quickly highlight differences in perceptions and needs by clinical setting.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files to be viewed.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org. Please note that Supporting Information files do not need this step.

5 Feb 2020

We have uploaded a response to reviewer comments document separately.

Submitted filename: Response to reviewer comments Final.docx

Click here for additional data file.

20 Feb 2020

Evaluating clinician acceptability of the prototype CanRisk tool for predicting risk of breast and ovarian cancer: a multi-methods study

PONE-D-19-33335R1

Dear Dr. Archer,

We are pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it complies with all outstanding technical requirements.

Within one week, you will receive an e-mail containing information on the amendments required prior to publication. When all required modifications have been addressed, you will receive a formal acceptance letter and your manuscript will proceed to our production department and be scheduled for publication.

Shortly after the formal acceptance letter is sent, an invoice for payment will follow. To ensure an efficient production and billing process, please log into Editorial Manager at https://www.editorialmanager.com/pone/, click the "Update My Information" link at the top of the page, and update your user information. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to enable them to help maximize its impact. If they will be preparing press materials for this manuscript, you must inform our press team as soon as possible and no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

With kind regards,

Alvaro Galli

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: N/A

Reviewer #2: N/A

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The authors have adequately addressed the reviewers comments. The manuscript reads very well and has been strengthened by addressing the review.

Reviewer #2: In responding to my comments about including more detailed results in the abstract, the authors inserted the sentence "Primary care were concerned about the amount of time...". There is a missing word -- it should be primary care clinicians or primary care providers.

Otherwise all of my comments were adequately addressed.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

28 Feb 2020

PONE-D-19-33335R1

Evaluating clinician acceptability of the prototype CanRisk tool for predicting risk of breast and ovarian cancer: a multi-methods study

Dear Dr. Archer:

I am pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please notify them about your upcoming paper at this point, to enable them to help maximize its impact. If they will be preparing press materials for this manuscript, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

For any other questions or concerns, please email plosone@plos.org.

Thank you for submitting your work to PLOS ONE.

With kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Alvaro Galli

Academic Editor

PLOS ONE