Exploring healthcare providers' experiences with specialty medication and limited distribution networks.

Integrated health-system specialty pharmacies (IHSSP) have shown high medication access, adherence, and provider satisfaction. The goal of this study was to explore healthcare providers' experiences with specialty medications distributed via Limited Distribution Networks (LDN) that do not include IHSSPs. We investigated healthcare providers' perceived impact of LDNs on clinic workflow, clinical practice, and patient outcomes. Interviews and focus groups were conducted with fourteen healthcare providers from four outpatient specialty clinics at an academic health system with an IHSSP. Qualitative analysis using an iterative inductive/deductive approach of coded transcripts was used to identify themes. Participants discussed requirements and barriers to communicating with insurance providers, drug manufacturers, and external pharmacies; time and effort required to navigate LDNs and impact on workload and clinic workflow; financial awareness of medication costs and methods for communication about financial information with patients; and advocating for patients to ensure access to necessary therapy and avoid missed doses or treatment lapse. Participants reported barriers to navigating LDNs that can interfere with clinic workflow and patient care. IHSSPs may reduce clinic burden by helping patients access, afford, and remain on therapy.

Introduction

Specialty medications, typically high-cost medications that treat rare or complicated diseases, offer life-changing outcomes for patients. Manufacturer distribution models for specialty medications vary: some can be dispensed by traditional retail pharmacies and others can only be dispensed by accredited specialty pharmacies. Some drug manufacturers employ limited distribution networks (LDNs), mandating that the drug can only be dispensed by select specialty pharmacies included in the network. LDNs can help manufacturers ensure product integrity and monitor its handling and administration, but also can benefit the manufacturer financially and may impede or delay medication access [1-3]. Patient access to specialty medications can be delayed by restrictions in insurance coverage or LDN requirements, which may create time-consuming and burdensome barriers for clinic staff to navigate [4, 5].

Many health-systems have developed integrated health-system specialty pharmacies (IHSSPs) to streamline patient access and improve adherence to specialty medications [6-8]. Some IHSSPs embed pharmacists and pharmacy technicians into ambulatory specialty clinics as part of the healthcare team to manage prescription access requirements. If the patient elects to use the IHSSP and the IHSSP is in network with both the drug manufacturer and patient’s insurance provider, the IHSSP dispenses the medication to the patient. Integrated specialty pharmacists and pharmacy technicians monitor for therapy safety and effectiveness through clinic visits and virtual monitoring (telephone and telehealth communications). Patient outcomes and any barriers to medication access or adherence are communicated to prescribers through the electronic health record. However, IHSSPs cannot dispense medications or perform comprehensive medication monitoring unless included in manufacturer dispensing networks and insurance provider (payer) contracts.

Previous research has described high medication access and adherence in patients who fill medication from an IHSSP compared with limited distribution drugs filled from an external pharmacy [9-12]. Healthcare providers rate higher satisfaction with IHSSPs than external [13]. Moreover, health systems have voiced concerns regarding negative impact of LDNs, including fragmented patient care, higher clinic burden, and delayed patient access to therapy [1, 2, 14]. However, no known study has investigated healthcare providers’ experiences with specialty medications in LDNs.

The goals of this study were to 1) explore healthcare providers’ experiences with LDNs, particularly those specialty drugs an IHSSP does not have access to dispense due to manufacturer distribution restrictions (henceforth referred to as limited distribution drugs [LDDs]), and 2) to investigate healthcare providers’ perceived impact of LDNs on clinic workflow, clinical practice, and patient outcomes.

Methods

Design and setting

This qualitative study was conducted at Vanderbilt University Medical Center, a large non- profit academic health system in the Southeast United States. As of July 2021, Vanderbilt Specialty Pharmacy (VSP) provides IHSSP services in 25 outpatient specialty clinical areas. Clinical pharmacists and pharmacy technicians help patients prescribed specialty medications access and afford therapy by assisting with insurance prior authorizations and appeals and linking patients with financial assistance programs when needed. If VSP is in network to dispense the specialty medication (based on manufacturer and payer contracts) and the patient chooses to fill medication from VSP, pharmacy staff will coordinate fulfillment and shipment to the patient. After patients initiate treatment, they are monitored through refill assessments, periodic pharmacist assessments, and clinic appointments. Per pharmacy accreditation guidelines, VSP uses robust clinical management protocols for specialty medications that can be dispensed. If patients use external specialty pharmacies for medication fulfillment, VSP pharmacists are unable to track medication access, fulfillment, and initiation. External specialty pharmacies maintain patient management protocols and follow legally mandated pharmacy dispensing requirements. Therefore, VSP cannot provide the same comprehensive services to patients managed by external specialty pharmacies to prevent duplication of efforts and patient confusion. This study was approved by the Vanderbilt University Institutional Review Board. Methods were developed and are reported using the consolidated criteria for reporting qualitative research (COREQ) checklist (S1 Table. ISSM COREQ Checklist).

Sampling

Purposive sampling was used wherein email invitations were sent to healthcare providers in five specialty clinics that prescribe a high volume of specialty medications. These clinics were selected to ensure participants were familiar with specialty medications that are dispensed through both the VSP and external specialty pharmacies. Participants were offered an interview or focus group, depending on convenience and preference.

Data collection

After reviewing and signing a consent form, participants completed a brief survey assessing their demographic characteristics, current role in clinic, and years’ experience with specialty medications. Semi-structured interviews or focus groups were then conducted, and follow-up questions were asked based on participants’ responses (S1 Text. Semi-Structured Interview/Focus Group Guide). Questions were developed by VSP pharmacists, leadership, and research team members/authors, then reviewed and annotated by Vanderbilt Qualitative Research Core personnel (not affiliated with VSP). Interview and focus group sessions followed a semi-structured guide focusing on the impact of LDNs on workflow, patient outcomes, and clinical practice. All interviews and focus groups were approximately 30 minutes in duration, took place at the workplace, and were audio recorded; the three-person focus group was video-recorded. A single female interviewer (MEP)who was a project manager within VSP, led all interviews and focus sessions. The interviewer holds a doctorate in behavioral sciences, and had minimal background in specialty pharmacy, no formal healthcare training, and no prior relationship with participants. The interviewer informed participants of her role at VSP and provided an overview of the study aims. All interviews were conducted at the workplace (Vanderbilt University Medical Center). Data saturation was not assessed as investigators determined a priori to perform all interviews/focus groups given the relatively small sample of clinics of interest based on use of LDD medications in the clinic.

Analysis

Audio recordings of the interviews and focus groups were transcribed by student pharmacists and reviewed for accuracy and to remove all personal identifiers by the interviewer (MEP). The Vanderbilt Qualitative Research Core was contracted to complete coding and analysis. A hierarchical coding system was developed iteratively using the interview questions and two randomly selected transcripts (S2 Table. Coding System). There were six major categories: 1) Participant descriptive details; 2) Steps involved in prescribing; 3) Other people involved in the process; 4) Factors that influence medication prescriptions; 5) Facilitators and Barriers; 6) Specific medications mentioned, and 7) Suggestions. Each category was further subdivided into 2–13 subcategories with several subcategories refined to a third level in the hierarchy. Definitions were written for each category and a codebook was developed using Microsoft Excel (S2 Table). There were nofield notes to incorporate into the analysis.

The transcripts were imported into Excel spreadsheets and formatted using a coding template. Each line in the spreadsheet was a speaking turn. Separate columns indicated the role of the interviewee (e.g., nurse, physician), participant identification, speaker, and quote. Each quote could be coded using up to 13 codes. The two transcripts used to develop the coding system were coded independently by two coders who then compared their results. This process was used to further refine the coding system, add additional categories, and refine definitions. The remaining nine transcripts were independently coded by two coders who reconciled their differences. The final coding consists of an agreement between two coders on how each quote should be characterized.

Qualitative analysis was performed using an iterative deductive/inductive approach [15, 16]. Deductively, authors used knowledge about clinicians and the health care system to structure our understanding of the coded quotes. Inductively, authors used the coded quotes to fill in details and to make connections between themes. Two qualitative researchers (PhD and MA not associated with VSP) reviewed the sorted quotes, identified themes, and flagged quotes for later use in presenting the data. Identification of themes was an iterative process starting with study questions and further modified by reviewing quotes. No interviews contradicted the overall conclusions about major themes.

Results

Participants

Fourteen providers from four specialty clinics participated in the study: a two-person focus group, a three-person focus group, and nine individual interviews. No providers refused to participate. As shown in Table 1, most participants were nurses (n = 6), White/Caucasian (n = 13) and women (n = 10). Median age was 37 years [interquartile range (IQR) = 33, 54], and participants had worked in their current role for a median of 4 years [IQR = 3, 7]. Transcripts were not returned to participants and participants did not provide feedback on the findings. No interviews were repeated.

Table 1

Characteristics of the sample (n = 14).

Characteristic	N (%) or Median [IQR]
Clinic
Endocrinology	3 (21%)
Hematology	3 (21%)
Neurology	4 (29%)
Pediatric Rheumatology	4 (29%)
Job Title
Physician	4 (29%)
Nurse Practitioner	4 (29%)
Nurse	6 (43%)
Proportion of work time spent assisting patients with accessing specialty medicines
Less than 25%	8 (57%)
25–50%	6 (43%)
Time spent in current role, years	4 [3.12–7.25]
Age, years	37 [33–53.5]
Race
Asian	1 (7%)
White/Caucasian	13 (93%)
Gender
Female	10 (71%)
Male	4 (29%)

Four themes were identified: communication, clinic workflow and workload, financial aspects of medication, and advocating for patients.

Communication

Participants discussed experiences communicating with health insurance providers, drug manufacturers, and external pharmacies, and compared communication regarding drugs dispensed by LDNs and non-LDNs.

Communication with insurance providers

For most patients with prescription drug insurance, clinic staff communicate with the patient’s insurance provider to approve coverage of medication. As demonstrated in the following exchanges, paperwork required by insurance providers can be cumbersome and, when not completed correctly, initiate back-and-forth between the clinic and insurance provider:

At the point that I want to prescribe [a specialty medication] I would have to go to the provider room to look for treatment forms, find them, and then fill it out, which usually [is] pretty complicated….like they have a patient assistance program or they have a nurse that [goes to the patient], and you check [a box on the form] sometimes for some conditions and not for others.

(Nurse practitioner)

Another participant expressed challenges with communicating with insurance providers via phone.

The hardest part of prior approval… is all the time on the phone [with insurance providers]. You could be on hold for 45 minutes, and that freezes you from doing anything else while you are waiting to speak to somebody to get whatever they request.

(Nurse)

One participant expressed difficulty reaching an insurance representative with medical knowledge and challenges with receiving conflicting or inaccurate information from representatives at insurance providers.

I know [with one specialty medication] there have been a lot of [patients with] lapses in therapy because of insurance where [the insurance provider] won’t request all of the documents that they need in the beginning. And then because [the specialty medication] is [available] in two concentrations, a lot of times they will approve one concentration, but they haven’t approved the other one, …The amount of time I have spent [communicating with insurance providers] the past two years is ridiculous. Just trying to get [someone on the phone], and the problem is you call these insurances and it’s not even someone medical, so they have no idea what you are talking about when you are trying to explain, they just say, ‘oh that’s fine, I’ll approve it.’ I’m like, you didn’t even approve the right [dose concentration]. Huge barrier there.

(Nurse)

With an IHSSP, the clinical pharmacist and pharmacy technician are familiar with requirements for appropriate use of each medication; thus the pharmacy team can complete required paperwork efficiently and serve as a point of contact for insurance providers.

A lot of [documentation and communication for LDDs] has now moved to the Vanderbilt Specialty Pharmacy with [the clinic specialty pharmacist]. She is doing a fantastic job with [specialty medication prescriptions]. She has taken over the certain meds that Vanderbilt Specialty Pharmacy can source for the patient… that works very well for medications that Vanderbilt can source for the patient.

(Nurse)

Communication with drug manufacturers

Participants also interact with drug manufacturers, either to start patients on therapy or to enroll eligible patients in manufacturer free drug assistance programs which may provide free medication to uninsured or underinsured patients.

One nurse described the complex process of communicating with a manufacturer to start a patient on treatment and the challenges that the back-and-forth correspondence placed on clinic workload.

Sometimes forms would be faxed with a script written on them but the doctor didn’t sign it…That would trigger a fax to come back, and it would… get in a stack with other faxes. So, you’ve got a stack of faxes… If one little thing is not on the prescription or not on the form like one box is not checked, it generates phone calls back…. Multi-page forms, different signatures, different things needed on there… The point is this is a complex role and it really needs a single point contact within the clinic. When it is just part of the routine clinic workflow and everybody is handling it, there are too many hands in the pie, things get dropped, boxes don’t get checked, and it creates an incredible amount of work and it really delays patient care.

(Nurse)

Manufacturers eligibility criteria and coverage rules for free drug programs that provide medication for patients initiating therapy or experiencing insurance lapses may vary. Knowing these criteria and the process of patient enrollment can be challenging. When IHSSPs are in the distribution network, the team can sometimes facilitate free medication storage, access, and fulfillment using a clinical management protocol. The IHSSP does not maintain protocols for LDN medications, so clinic staff and the manufacturer free drug assistance programs must coordinate patient enrollment and medication fulfillment and shipping for uninsured or underinsured patients prescribed LDDs. As discussed by the following participant, clear information about patient eligibility for these programs is not always available, which can cause patients to miss doses and experience severe complications:

In August, [the manufacturer free drug assistance program] [told] us [a patient receiving an LDD] is not eligible for [the manufacturer free drug assistance program] anymore because he has exceeded the maximum amount, which that was never communicated to us that there was a maximum amount because those changes [to assistance program eligibility] occurred… It was a lot of phone calls between me, [the manufacture free drug assistance program], and in the meantime, there was another pharmacy [added to the process]. Things were not straightforward. He did end up missing a dose a week for 3 weeks and required hospitalization.

(Nurse)

Communication with external pharmacies

When the IHSSP is not in network, prescriptions for the LDD must be sent to an external pharmacy to dispense medication to the patient. Participants expressed barriers to communicating with external pharmacies, who may not inform the clinic staff about delays or barriers to medication coverage.

The following exchange demonstrates communication delays with external pharmacies, which in turn prolongs insurance approval:

When [the clinic] send[s] a prescription electronically to [an external] pharmacy, usually unless the [external] pharmacist is going to pick up the phone and call us, we won’t know about any problems for at least a week to maybe even two weeks, because [the external pharmacy] will send a fax. Then the fax goes into the fax queue and it gets pulled off [by clinic staff] but then gets distributed by provider. It gets put into our [Clinic] baskets, if we are in satellite clinics, we don’t get it for a couple of days. So, there can be a week-long delay just to find out medicine needs prior authorization or needs a refill or any of those simple procedures.

(Nurse practitioner)

In other cases, an external pharmacy may not have a medication in stock but does not communicate this with the clinic staff, leading to delays or periods of treatment lapses.

[Before we knew a drug was limited distribution] we were sending the prescription to the patient’s pharmacy, and we weren’t getting calls back for quite some time that [clinic prescribers] couldn’t prescribe the drug because the pharmacies were trying to get the drug. It did lead to a delay in treatment when we [Clinic] weren’t using our [VSP] pharmacists to [assist with insurance prior authorization] because the communication was just so poor from the other pharmacies.

(Nurse practitioner)

[There is a] patient I have who is on [an LDD], there has been times when that limited distribution pharmacy didn’t have the medicine either. [The external pharmacy] had to order it and so it delayed it by another week… Because that specialty pharmacy didn’t communicate with us that they were having a delay in shipment, and we were told that the LDD pharmacies had the medicine on hand, which isn’t always accurate either. [The external pharmacy] didn’t communicate back to the family or to us. It basically came down to where [the patient was] a week late for [an] injection that can cause this patient to have [a] flare and joint disease.

(Nurse practitioner)

When the IHSSP has access to dispense a drug, these communication barriers are alleviated because the integrated pharmacist communicates with patients and clinic staff about medication barriers.

By [the clinic] being able to use VSP [to dispense the medication], it has cut down on that communication gap. Because, first, [VSP clinical pharmacists] are on the floor with us [in the clinic] so if there is a significant problem, the [VSP clinical pharmacist] will just come speak to us directly. Second, the [VSP clinical pharmacist] are able to use the message basket system and let us know immediately whether a patient’s insurance has lapsed, if [the patient] can’t get coverage, if it’s too soon to refill, if [the patient is] not refilling it appropriately, if the family has concerns or questions regarding the medicine, we [Clinic] are able to know that through VSP immediately. Which helps [the clinic] then communicate with the families quicker, get the situation resolved, and get the patient back on track.

(Nurse practitioner)

Clinic workflow and workload

Communication and administrative barriers associated with LDDs can interfere with clinic workflow and other patient care responsibilities.

Nurses discussed how communication barriers associated with LDNs interfered with their ability to complete other clinical tasks:

[Contacting manufacturer hub] would become very time consuming because then it requires calling the family, the hub, the insurance provider, calling the pharmacy because none of those people were communicating and we would become the point of contact and that was taking away from the flow of care in clinic.

(Nurse)

A physician participant also shared how their workflow and schedule were impacted when nurses were required to manage LDD prescriptions:

[Paperwork] was affecting us because that nurse was not available… all the time, so I would have to ask the patient to be in a waiting room. I would have to go to the workroom, call the nurse; the nurse would then come down. They had the paperwork that required my signature as well, so it did affect my schedule because I didn’t know when the nurse was going to come. Sometimes the patients can wait and sometimes they could not wait, and then all of this was happening over mail and faxes. Again, circling back to me for signatures. So definitely much more inconvenient back then [before VSP integration].

(Physician)

With IHSSPs managing these prescriptions, participants reported being able to spend more time counseling, educating, and supporting patients and their families.

[The IHSSP] has given us so much more time back with the patients and that impacts the care they get. We have more time to talk about lots of different aspects of their care instead of spending extended amounts of time on hold with insurance providers, not even with talking with people who are able to make decisions for the patient. So, we are extremely appreciative of that… I think we have more time to triage parents and are getting calls back faster than they were able to before.

(Nurse)

Financial knowledge and communication

Participants shared their knowledge and experiences with high-cost medications and how they discuss financial information with patients.

Financial knowledge

Participants consistently mentioned the high costs of specialty medications. With LDDs, clinicians and clinic staff may not be aware of all available financial assistance resources nor have time to help patients navigate these programs.

Most of the drug companies have some sort of patient assistance programs, but they are very different from one medicine to the next. Even within one medicine, they change their programs. So, it is very hard for me I never know who has this co-pay card or who has that [copay card].

(Physician)

When the IHSSP can dispense a drug, a dedicated pharmacy team helps patients who express need access financial assistance programs, which participants found helpful for themselves and the patients:

Specialty pharmacists have been very helpful at accessing grants for [patients in the Medicare coverage gap] or talking to the drug companies themselves to get compassionate use for the drugs so these patients can continue on it. It used to be where our social worker or myself or the nurses had to do all of that and we just didn’t have time. So, I think having the specialty pharmacy do it is extremely helpful, and it helps patients too.

(Nurse practitioner)

I know that pharmacy is going… to be able to help them with copay cards. It’s a big stress on family when we tell them that we are starting this medicine that is going to be expensive, but then we can give them recourses to connect them with different available co-pay programs. Pharmacy is great for that because I never know. I know [manufacturers probably have a co-pay card] but you have to call and give them the website to check or call their patient assistance program and check.

(Physician)

Financial communication

Participants also shared their practices for communicating with patients about medication costs and the benefits of the integrated pharmacist providing transparency regarding treatment costs for patients. With the integrated model, prescribers felt they could tell patients upfront about typical insurance/financial aspects of specialty medications, then refer them to the IHSSP team who can help navigate financial assistance if needed. Prescribers felt confident when the IHSSP can dispense the drug because they trusted that patients will receive support needed to overcome financial barriers and initiate therapy.

One physician felt that patients are more comfortable discussing financial information with the integrated pharmacist than with their healthcare provider:

I can tell you in general patients will never voice their financial concerns to the physician. They are more open in doing that with a pharmacist or nurse or a social worker. So, usually what I tell patients is that these medications are expensive, when we send the message to the pharmacist, they will walk you through everything and give you kind of what your options are…. So, I sort of give them a little bit of a prelude to that so they are not caught unaware. But VSP team really owns all of that for us.

(Physician)

This physician also introduced patients to the clinical pharmacist, who works alongside clinic staff, so patients have a single point-of-contact:

I let them know they will be contacted by [the clinic pharmacist]. I try to have [patients] actually meet [the integrated clinical pharmacist], either before the process and sometimes there’s logistical things to have them sign, or once they are on the therapy it’s nice to have a face with the name…. [The pharmacist] is helping these people and they usually want to thank her for all of the work.

(Physician)

The following participants discussed how, when the IHSSP can dispense the drug, they can refer patients to the clinical pharmacist who oversees the logistical steps of accessing medication.

[After deciding to prescribe a medication] I’ll message [the clinic pharmacist]…she will look into patient assistance programs because she knows about them, so in those situations, I can say to a patient, “I’d like to try [a specific drug], but it’s very expensive, and so that is prohibitive for some patients from using it, but it totally depends on your insurance and your financial situation, and there’s a lot of different options so I’m going to send it to our specialty pharmacist, who knows about all that stuff, and she will let me know what the options are, and then you guys can decide if you’re able to afford it.”

(Nurse practitioner)

Advocacy

Regardless of the drug’s network distribution status, participants prioritized obtaining the most appropriate drug for the patient, as determined by medical literature and/or patient needs or preferences, and, to the extent possible, expressed willingness to devote time and resources to ensure patients’ needs are met.

Choosing the right medication

Sometimes, clinicians have only one or few drugs to treat a patient’s condition, with no or few alternatives available and must do what is necessary to help patients access treatment.

They need the medicine they need and we just have to get it. I mean… whatever hoops we have to jump through to get that. It’s not like I have [multiple options] to choose from… in [some drug classes] you have a little bit more of a choice but not really, there are still differences that make individual ones be preferred in cases and I mean I am going to pick which ever one is best medically, we just would have to deal with the logistics of it…

(Physician)

One participant shared her willingness to ‘jump through hoops’ to ensure patients can access and afford medication:

We [clinic] pick the best drug for the patient in the situation they are in. If that means [the clinic has] to jump through 30 hoops to get that medication for [the patient], we will. Because, you know, in our practice it is the patient, they are what we are here for. They’re the reason why I am such an advocate for this, that drugs shouldn’t be limited in distribution mainly for the price issue. These families are already sometimes struggling to decide whether I come to the doctor versus buy groceries.

(Nurse practitioner)

Participants expressed empathy and concern for whether patients could afford medication but mentioned that for drugs that can be dispensed by the IHSSP, they trust that the clinic pharmacist will direct patients to financial assistance programs. This reduced provider concerns about treatment affordability and access.

I don’t want to sound naïve, but I still want to go for the best [medication for the patient]. Having said that, of course if a patient cannot afford it [I take that into account], but I still try for the first option clearly what I think is the best regardless of the cost and the logistics because we have such great support for the logistics.

(Physician)

Coordinating care to prevent medication access gaps

Nurses devoted time and effort to ensure that patients maintain access and adherence to therapy. When LDDs must be initiated during a hospitalization nurses ensure that paperwork has been completed so the patient has access to the medication after discharge:

When [the patient is prescribed a specialty medication while] inpatient, [the inpatient pharmacy] is able to give the medication inpatient… We just need to make sure before [the patient goes] home that a prior authorization has at least been started, and then [the manufacturer free drug assistance program] can send out the [free drug], but we also need to know when that is going to be sent out.

(Nurse)

The IHSSP maintains clinical patient management protocols for medications that are stocked and dispensed by the IHSSP, including sample medications, which are dispensed when deemed appropriate by prescribers to trial safety and/or efficacy. IHSSP staff gain expertise in navigating available patient assistance programs for medications dispensed.

[Whether patient’s health declines while waiting for drug approval] depends on the medication. Like with [non-LDD], we supply [non-LDD], so we can give you a sample until we wait the two months when insurance [will approve/cover the medication] With the [LDD] it’s not [dispensed by] our pharmacy, there’s a whole other playing field there.

(Nurse)

Discussion

This study details healthcare providers’ experiences with limited distribution specialty medications at an academic health center with an IHSSP. Healthcare providers in five specialty clinics discussed the challenges associated with LDNs, and the benefits of an IHSSP, specifically that IHSSPs enable providers and clinic staff to better focus on clinical care, streamline the patient’s access to medication, and provide essential financial assistance support that allow for confident specialty medication prescribing, and better patient outcomes. This is the first known study to qualitatively explore healthcare providers’ experiences with LDNs and investigate how LDNs influence providers, patients, and clinics. This study builds on previous reports that demonstrate the impact of LDDs on time to medication access, health system expenses, administrative and financial burden for clinics, and fragmentation in care [1–3, 10–12]. Understanding specialty providers’ perspectives on the perceived impact of LDNs is necessary for specialty pharmacy stakeholders to help improve efficient, equitable treatment for all patients.

Participants discussed the delayed and fragmented communication with external parties when prescribing a LDD, which interferes with patient’s ability to access medication. Previous studies have reported that for each specialty medication prescription, clinic staff spend approximately two hours completing paperwork requirements and coordinating with insurance providers and external specialty pharmacies [17]. One study reported that half of oncology prescriptions required five or more phone calls by clinic staff before the patient obtained the prescription [5]. If a drug can only be distributed by select pharmacies—as mandated by either the manufacturer LDN or the pharmacy benefit manager’s pharmacy restrictions—prescribers must send the prescription to a specific external pharmacy; this may be unknown to prescribers upfront [4, 18]. These requirements can delay patient access to medication, or cause patients to abandon the prescription, sometimes resulting in negative clinical outcomes for patients with specialty diseases [5, 18]. In our study, participants expressed that communication barriers were reduced when a drug could be dispensed by the IHSSP. This qualitative data corroborates previous findings showing that access time is significantly longer for specialty medications with LDNs compared with non-limited distribution, and that medication access time is reduced after the IHSSP is included in the LDN for multiple sclerosis, hematology, and neurology clinics [10-12]. Conversely, when a LDN that excluded the IHSSP was imposed, one health system found delays in drug acquisition and more variable and unreliable medication delivery times [2].

Participants also discussed barriers to patient adherence to LDDs, such as when the external pharmacy is out-of-stock of the medication, or the patient’s insurance benefits change. After patients initiate prescribed therapy, delayed and fragmented communication can cause patients to miss doses, which may cause urgent healthcare needs. When drugs can be dispensed by the IHSSP, the pharmacy team conducts monthly assessments, and prescribers are notified if patients report an issue with medication (adverse event, worsening symptoms) or encounter barriers to medication access, facilitating close monitoring and real-time communication with providers to address concerns or barriers. If the IHSSP is included in the LDN, clinical patient management protocols are in place and pharmacy staff have expertise in determining patient eligibility and enrollment requirements for free drug programs. Samples can also be stocked and dispensed as deemed appropriate by the prescriber for the patient to try the medication and evaluate safety and effectiveness. Previous studies have reported high medication adherence in patients who fill medication from an integrated pharmacy [7, 19–21] and discussed how clinical pharmacists at integrated specialty pharmacies can help patients navigate transitions in care to ensure medication adherence [22].

Many participants discussed the high cost of specialty medications and the impact of financial barriers on patient outcomes. Patients subject to high medication costs have lower adherence and experience financial toxicity—referring to distress and financial problems resulting from high medication costs—which is associated with worse patient outcomes and quality of life [23-25]. Participants felt confident prescribing medications when the IHSSP can dispense, as they trusted the pharmacy team would help the patient access treatment. Healthcare providers also discussed the benefit of a specialty pharmacist in clinic who could communicate with patients about financial assistance options. Patients filling medication from an IHSSP often pay low out-of-pocket medication costs due to the extensive knowledge surrounding available assistance programs for patients who express financial need [9, 19–21]. This in turn may lead to lower financial toxicity in patients who require specialty medication [23]. When patients lose or change insurance coverage, integrated specialty pharmacists coordinate care to ensure patients can obtain affordable medication during this transition [22]. IHSSPs often have access to internal safety net programs that can assist patients who qualify when other options are not available.

Despite the challenges and barriers associated with LDNs, participants prioritized advocating for their patients, expressing willingness to conduct extra work to identify and enroll patients in financial assistance programs when the external specialty pharmacy fails to provide adequate support to the extent possible to ensure patients access the best medication. These tasks, however, can interfere with other clinic responsibilities and patient care. When medications can be dispensed by the IHSSP, the pharmacy team oversees tasks needed for the prescriptions; thus, nursing staff have more time to spend with patients. IHSSP teams often must assist patients prescribed an LDD who are having issues obtaining medications from external pharmacies. However, lack of transparency into external patient pharmacy records and lack of knowledge of the operational failures causing the gap in patient care can make this process challenging. Additionally, IHSSP efforts to address external pharmacy issues substantially increase workload and are unable to be quantified to support adequate staffing models.

Given our study’s findings and other emerging research, it is worth exploring the rationale of LDNs without established and transparent criteria for network access. Drug manufacturers cite several reasons for using LDNs, including the ability to control medication product integrity, to closely monitor the medication’s handling and administration, to ensure patients receive disease management programs from specific specialty pharmacies, and to more easily ensure FDA requirements are met [1-3]. Manufacturers also realize financial benefits of LDNs, through closer inventory management, lower distribution fees, and higher revenue generation [1]. Manufacturers participating in Medicaid and Medicare must provide outpatient drugs to eligible health care organizations and covered entities at reduced prices under the 340B program. Allowing IHSSPs with capabilities that meet and can often exceed the criteria for inclusion into an LDN fulfills this requirement. Several healthcare providers in our study voiced the benefits of IHSSPs when managing specialty medications and quantitative data from previous literature demonstrates IHSSPs can provide the high-touch care delivery needed to ensure safety, administration, and monitoring requirements are met. Therefore, allowing eligible IHSSPs that have robust capabilities fulfills manufacturer distribution goals while ensuring a streamlined patient journey and optimal outcomes.

Future directions

Given the specific requirements for storing, handling, distributing, and monitoring the safety of specialty medications, the intent behind LDNs is reasonable. However, as noted by participants in this study, IHSSPs may augment manufacturers’ goals, including the ability to mirror high touch clinical trial conditions by helping patients afford, initiate and maintain on specialty medication. Recently, the American Society of Health System Pharmacists released recommendations from the Specialty Pharmacy State of Practice in Hospitals and Health Systems Future Directions Summit which included, “hospital and health-system specialty pharmacies should develop strategies with payers and manufacturers that promote access to and establish criteria for limited distribution networks, such as value-based contracting, cost-management strategies, medication affordability assistance, and care coordination” [26]. A structured collaboration between manufacturers and IHSSPs could allow the manufacturer to minimize costly resources that lead to optimal patient outcomes (e.g. education, ongoing monitoring, financial assistance) as these are often core services of an IHSSP. Additionally, better IHSSP/manufacturer collaboration and data sharing could enable the manufacturer to gain insight into the streamlined patient journey facilitated by IHSSPs, and provider perspectives on the benefits of this model. Introducing transparent, tailored criteria for inclusion into LDNs, along with data sharing agreements, could help ensure better use of the restricted network. More research is needed to understand specialty pharmacist and manufacturer perspectives on the impact of LDNs and potential solutions for improving access while maintaining patient safety.

Limitations

Participants were healthcare providers, mostly nurses with similar demographics, practicing within outpatient specialty clinics at single academic healthcare center with an IHSSP. Thus, experiences with specialty medications and LDNs may not generalize to all healthcare providers and care settings. The sample size of the study was small, but participants represented a range of specialty clinics and healthcare professions, and participants had experience working with internal and external specialty pharmacies. Larger focus groups would have allowed for richer data, however including more participants was limited by few clinicians/staff having the knowledge and experience needed to speak to the topic. Participants were individually selected by investigators based on their experience with prescribing or coordinating care for specialty medications that are distributed through an LDN; this recruitment method may have introduced some unknown selection bias. The investigators did not plan for interviewees/focus group participants to review or confirm the interview transcripts, which could impact data quality.

Having a non-health professional interviewer may have been beneficial to reduce potential bias as she did not have personal experience with LDNs and was able to approach the interview/focus group without assumptions regarding the LDN process. However, the lack of healthcare experience may have limited her ability to ask more comprehensive in-depth questions due to limited experience with the topic.

Conclusion

This study reports perspectives from specialty providers and clinic staff on the impact of limited distribution drug networks in real-world practice. As noted by participants, IHSSPs reduce clinic burden by helping patients access, afford, and remain on therapy. When IHSSPs are included in a drug’s LDN, healthcare providers report high satisfaction, more confidence in prescribing specialty medications, improved workflow efficiency, and better patient outcomes. Manufacturers who wish to distribute specialty medication through LDNs should create and communicate transparent criteria for specialty pharmacies to meet distribution goals.

ISSM COREQ checklist.

(PDF)

Click here for additional data file.

Coding system.

(DOCX)

Click here for additional data file.

Coded minimal data set.

(XLSX)

Click here for additional data file.

Semi-structured interview/focus group guide.

(DOCX)

Click here for additional data file.

18 Feb 2022

Submitted filename: Exploring HC Provider Attitudes_2021.09.03.FINAL.PT.docx

Click here for additional data file.

15 Mar 2022

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming.

RESPONSE:

We appreciate this reminder and have reviewed our manuscript to ensure it meets styling requirements. The template was appreciated.

File naming updated:

S1 Table 1. ISSM COREQ Checklist

S2 Text. Semi-Structured Interview.Focus Group Guide

S3 Table 2. Coding System (updated number to reflect new supplementary material provided)

File added with same naming format:

S4 Table 3. Coded Minimal Data Set

2. In the Methods section of the manuscript, please consider including more information on the number of interviewers, their training and characteristics; and please provide the interview guide used.

RESPONSE:

We have modified the interviewer description to be more detailed and clear (page 5, lines 108-112).

We have uploaded the semi-structured interview and focus group guide (S2 Text. Semi-Structured Interview.Focus Group Guide).

3. In your Data Availability statement, you have not specified where the minimal data set underlying the results described in your manuscript can be found. PLOS defines a study's minimal data set as the underlying data used to reach the conclusions drawn in the manuscript and any additional data required to replicate the reported study findings in their entirety. All PLOS journals require that the minimal data set be made fully available. For more information about our data policy, please see http://journals.plos.org/plosone/s/data-availability.

RESPONSE:

We have uploaded a minimal data set that was used for coding and analysis. Please let us know if this is not acceptable.

4. Please upload a copy of Supporting Information Table 2 which you refer to in your text on page 5.

RESPONSE:

I confirmed that this information was uploaded in the original submission. It is now labeled S1 Table 1. ISSM COREQ Checklist. Based on the journal guidelines, we thought this name was acceptable, but we did remove the Complete and the underscores please let us know if there is a preference for a different name. Guideline excerpt:

Authors may use almost any description as the item name for a supporting information file as long as it contains an “S” and number. For example, “S1 Appendix” and “S2 Appendix,” “S1 Table” and “S2 Table,” and so forth.

5. Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references.

RESPONSE: We have reviewed the reference list and confirm they are complete and correct. We updated reference #26 as this data has now been published

6. We have reviewed the study reporting perspectives from specialty providers and clinic staff on the impact of limited distribution drug networks in real-world practice. The authors have shown that participants noted that IHSSPs reduce clinic burden by helping patients access, afford, and remain on therapy. When IHSSPs are included in a drug’s LDN, healthcare providers report high satisfaction, more confidence in prescribing specialty medications, improved workflow efficiency, and better patient outcomes. Manufacturers who wish to distribute specialty medication through LDNs should create and communicate transparent criteria for specialty pharmacies to meet distribution goals. We find the study contributing to the knowledge in service delivery in healthcare. We have included the MS highlighting areas we think may need to be corrected or included. Key words and abbreviations may be included in the MS as well.

RESPONSE:We appreciate the reviewer’s comments and feedback on the manuscript. Suggested grammatical edits in the attached document that was provided have been incorporated in the updated manuscript. We have carefully reviewed the manuscript and added additional grammatical changes where appropriate.

Submitted filename: R2R.docx

Click here for additional data file.

14 Jun 2022

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Jonas Bianchi, DDD, MS, Ph.D

Academic Editor

PLOS ONE

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: (No Response)

Reviewer #2: (No Response)

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: N/A

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: We have made comments to the Authors in our initial review where we had accepted the MS for publication.

Reviewer #2: Thank you for the opportunity to review this manuscript. Overall, the manuscript reads well and is easy to follow. However, the following points need to be addressed before it gets published.

Methods:

It is unclear how the authors ensured data saturation, which is an important process for deciding how many people to interview in qualitative studies

It appears that the interviewee selections are mainly based on convenience rather than purposive sampling. Ten out of 14 participants (71.5%) are nurses/nurse practitioners. As such, the findings mostly represent nurses’ perspectives rather than the broader healthcare providers perspectives.

Although clinical pharmacists are the main players/stakeholders in LDN/VSP, the authors failed to incorporate pharmacists’ perspectives in the study, which is a missed opportunity to better understand the complexity of the IHSSP and LDN systems. Also, it would have been better to incorporate manufacturers’ perspectives. Unless we hear all stakeholders’ views, it would be difficult to come up with workable solutions.

It is unclear why the authors didn’t incorporate the field notes in their data analysis.

“Member check” is an important step in ensuring the trustworthiness/quality of qualitative data. However, the authors didn’t provide an opportunity for the interview to comment on the findings or edit/confirm the interview transcripts.

The focus group discussions only included two or three people. It is usually not recommended to have FGDs with less than five or six participants.

The authors should also discuss the pros and cons of a non-health professional interviewer. Although having a non-health professional as an interviewer may help to avoid bias, there are several limitations of this approach in health research, and this should be properly discussed in the discussion section.

Line 576-578: “Participants were individually selected by investigators based on their experience with prescribing or coordinating care for specialty medications that are distributed through an LDN; thus, sampling bias may have occurred.” I don’t think this is relevant for a qualitative study. In qualitative studies, bias cannot be avoided, they rather should be acknowledged. The concept of bias in qualitative and quantitative research is also different.

Don’t mix methods and results. Some of the methods section contain results, for example, lines 133 to 141 (including table 1).

What is the purpose of reporting the ethnicity/race of participants as well as their age and sex in Table 1? None of these characteristics was used to inform the data analysis. Perhaps, looking at the ethnicity/race of the participants, it appears that only practitioners from one ethnic group are recruited. Again, this is another example of missed opportunity by the researchers to include diverse perspectives!

Discussion

Please revise the limitation section of the discussion according to the above comments and suggestions.

Thank you.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Prof Greanious Alfred Mavondo

Reviewer #2: Yes: Dr Kebede Beyene

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

14 Jul 2022

Response to Reviewers

Reviewer Comment Authors Response

Reviewer #1

We have made comments to the Authors in our initial review where we had accepted the MS for publication. We appreciate the reviewer’s comments and previous feedback on the manuscript.

Reviewer #2

Thank you for the opportunity to review this manuscript. Overall, the manuscript reads well and is easy to follow. Thank you for these kind words and feedback.

It is unclear how the authors ensured data saturation, which is an important process for deciding how many people to interview in qualitative studies We appreciate you identifying this omission. We’ve added to page 6, line 130 that data saturation was not assessed. Because there was a relatively small sample of providers/clinics able to be interviewed, we chose a priori to perform interviews/focus groups on all potential participants regardless of data saturation.

It appears that the interviewee selections are mainly based on convenience rather than purposive sampling. Ten out of 14 participants (71.5%) are nurses/nurse practitioners. As such, the findings mostly represent nurses’ perspectives rather than the broader healthcare providers perspectives. The sampling was purposive as we used criteria that the clinic had to be familiar with and utilize LDDs in practice. The sample has a larger amount of nurses because they are typically the staff that address access issues for LDDs and therefore more knowledgeable on the topic. We’ve added to the limitations, page 28, line 586 that most of the participants were nurses, which contributes to limited generalizability.

Although clinical pharmacists are the main players/stakeholders in LDN/VSP, the authors failed to incorporate pharmacists’ perspectives in the study, which is a missed opportunity to better understand the complexity of the IHSSP and LDN systems. Also, it would have been better to incorporate manufacturers’ perspectives. Unless we hear all stakeholders’ views, it would be difficult to come up with workable solutions. We appreciate this perspective. The focus of this paper was to elucidate provider and clinic staff experience with LDNs rather than giving a sweeping view of the current LDN landscape. There is much more research to be done in this area as LDNs continue to increase and gathering opinions from pharmacists and manufacturers would be a great next step. We’ve added this to future directions (page 28, line 581).

It is unclear why the authors didn’t incorporate the field notes in their data analysis. We did not keep field notes for this study. Interviews were recorded and transcribed, providing the primary qualitative data. This is noted on page 9, line 155. We believe that the data provided meets PLOS One requirements of a minimal data set.

“Member check” is an important step in ensuring the trustworthiness/quality of qualitative data. However, the authors didn’t provide an opportunity for the interview to comment on the findings or edit/confirm the interview transcripts. Thank you for this feedback. We have added this as a limitation to our methods (page 29, line 596). The experience of the Vanderbilt Qualitative Research Core is that providers are busy clinicians with minimal time and usually lack of desire to further review transcripts. However, we did not note this in the manuscript as this anecdotal experience may vary.

The focus group discussions only included two or three people. It is usually not recommended to have FGDs with less than five or six participants. This is a great point. We were unable to have larger focus groups due to limited staff with the knowledge/experience needed to speak to the topic. Additionally, we wanted to keep the focus groups specific to each clinic so that we could understand if certain clinical areas are affected differently than others, further limiting our potential to combine areas and enlarge our focus group size. We’ve added this as a limitation (page 29, line 591)

The authors should also discuss the pros and cons of a non-health professional interviewer. Although having a non-health professional as an interviewer may help to avoid bias, there are several limitations of this approach in health research, and this should be properly discussed in the discussion section. We agree with the reviewer that this should be addressed and have added a discussion on pros/cons of this approach to the discussion (page 29, line 599)

Line 576-578: “Participants were individually selected by investigators based on their experience with prescribing or coordinating care for specialty medications that are distributed through an LDN; thus, sampling bias may have occurred.” I don’t think this is relevant for a qualitative study. In qualitative studies, bias cannot be avoided, they rather should be acknowledged. The concept of bias in qualitative and quantitative research is also different. Thank you for this feedback. We have updated the limitations to soften the language that that this recruitment method may have introduced limitation and changed verbiage to selection bias (page 29, line 595).

Don’t mix methods and results. Some of the methods section contain results, for example, lines 133 to 141 (including table 1). We have moved the participant information to results as requested.

What is the purpose of reporting the ethnicity/race of participants as well as their age and sex in Table 1? None of these characteristics was used to inform the data analysis. Perhaps, looking at the ethnicity/race of the participants, it appears that only practitioners from one ethnic group are recruited. Again, this is another example of missed opportunity by the researchers to include diverse perspectives! We have retained providing the race/ethnicity, age, and sex to give reader’s insight into the makeup of participants as we believe this is typical in most qualitative research studies. As previously noted, we were limited in our ability to include diverse perspectives based on the small number of healthcare practitioners with robust experience with LDNs at our institution. We certainly would have liked to have a more diverse sample to engage. We’ve added the limited diversity in our sample to limitations (page 28, line 586)

Please revise the limitation section of the discussion according to the above comments and suggestions. We have revised limitations as noted above.

Submitted filename: Response to Reviewers_2.docx

Click here for additional data file.

2 Aug 2022

Exploring healthcare providers' experiences with specialty medication and Limited Distribution Networks

PONE-D-21-28656R2

Dear Dr. Zuckerman,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Jonas Bianchi, DDD, MS, Ph.D

Academic Editor

PLOS ONE

4 Aug 2022

PONE-D-21-28656R2

Exploring healthcare providers' experiences with specialty medication and Limited Distribution Networks

Dear Dr. Zuckerman:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Jonas Bianchi

Academic Editor

PLOS ONE