Validation of the QualiPresc instrument for assessing the quality of drug prescription writing in primary health care.

INTRODUCTION: Adverse events related to drug prescriptions are the main patient safety issue in primary care; however there is a lack of validated instruments for assessing the quality of prescription writing, which covers the prescriber, the patient and the drug information.
OBJECTIVE: To develop and validate the QualiPresc instrument to assess and monitor the quality of drug prescriptions in primary care, accompanied by a self-instruction direction, with the goal of filling the gap in validated instruments to assess the quality of prescription writing.
METHODOLOGY: A validation study conducted in a municipality in Northeastern Brazil, based on prescriptions prepared in January 2021 by physicians assigned to 18 Basic Health Units and filed in 6 distribution/dispensing units. Four steps were covered: 1) Analysis of content validity of each indicator (relevance and adequacy); 2) Analysis of reliability via intra and inter-rater agreement of each indicator; 3) Analysis of the utility of each indicator; 4) Construction and analysis of the reliability of a weighted composite indicator based on effectiveness and safety scores for each indicator.
RESULTS: Twenty-nine potential indicators were listed, but only 13 were approved for validity, reliability and usefulness. Twelve indicators were excluded because of validity (<90% validity index) and four because they were not useful in the context of the study. Three weighted composite indicators were tested, but only one was approved for reliability and usefulness. The validated instrument therefore contains 13 indicators and 1 weighted composite indicator.
CONCLUSION: This study demonstrates the validity, reliability and usefulness of QualiPresc for the evaluation of prescription writing in the context of primary care. Application to contexts such as secondary care and tertiary care requires cross-cultural adaptation and new content validity. Educators, managers and health care professionals can access QualiPresc online, free of charge, to assess performance and provide feedback involving drug prescribers.

Introduction

Unsafe medication processes occur in all care settings, but they weigh more heavily in primary care. In this context, much of the harm related to health care seems to be associated with drugs prescription and with errors/delays in diagnosis and its avoidability is related more to medication, administrative practices and communication [1]. Further studies in primary care have found medication to be a critical point of patient safety [2-5]. Globally, up to 40% of patients suffer harm related to health care in the context of outpatient care and up to 80% of these are preventable [6].

Primary care has peculiarities that can make the medication process unsafe, but studies in this environment are less frequent than in the hospital environment [7]. There is a need for interventions related to information technology infrastructure, educational support for prescription management, clinical pharmacy services, medication reconciliation and strategies for measuring quality indicators [8].

In the medication process, prescription stand out as the step most susceptible to medication error [9-11]; therefore, they are a priority for the measurement of quality indicators. About 40% of users assisted in primary care do not need medication, however, 80% of these leave consultations with a prescription [12-14].

In the context of primary care, drugs demand/supply is high and technological advances are fast, which can lead to less than adequate prescription practices [15], due to the commitment and time investment demanded from prescribers in order to gain the appropriate knowledge inherent to this health technology, including the best scientific evidence about efficacy, safety and cost-effectiveness. This was evidenced in the Covid-19 pandemic, which required an investment of time by prescribers to obtain appropriate knowledge about the efficacy, safety and cost-effectiveness of prescribed drugs [16].

The preparation of a drug prescription involves structuring a document based on the technical specifications of the drug (prescription writing), following the prescriber’s therapeutic decision (therapeutic decision criteria). Ideal prescription writing encompasses the prescriber, the patient and the drug information; however, this information is often missing or incomplete [17-25]. In Brazil, the practice of prescription writing is based on recommendations from the Ministry of Health [26].

The quality of a prescription is reflected across various dimensions of the quality of health care, in this case, the safety, effectiveness, cost and patient-centered care, which results in improved care through user satisfaction, lower costs and a reduction of medication errors [27].

Thus, in order to reduce medication errors and reach the WHO global goal, Medication Without Harm [28], it is essential to evaluate and improve the quality of drug prescriptions. Failure to observe these aspects during the preparation of a prescription results in prescription error, a type of medication error related to decision-making or writing which, although unintentional, can reduce the efficacy of the treatment or increase the risk of mistakes [29].

Despite the abundance of studies on prescription quality in primary care according to therapeutic decision criteria [30-35], there is no valid, reliable and useful set of indicators that allows the assessment of prescription writing quality. The absence of a set of indicators to assess the quality of prescription writing is an incomprehensible gap, given that medication errors represent the main safety incidents resulting in adverse events in primary care [2-5]. Furthermore, this cannot be resolved through an electronic prescription system, even in well-resourced settings [36]. Therefore, it is extremely necessary to solve this deficiency, especially in primary care. Furthermore, the availability of a set of indicators to assess prescriptions from this perspective can be useful in health services and in teaching good prescribing practices.

Thus, this study aims to develop and validate a concise instrument to monitor the quality of prescription writing, to encourage continuous evaluation and improvement of the effectiveness and safety of pharmacological therapy in primary health care.

Methodology

Context and design of the study

This study is contextualized as a research/university extension project to improve the quality of medication prescription in the context of primary care, based on the cooperation between a Brazilian federal university and the health department of a municipality in Northeastern Brazil.

It is a validation study developed from March 2020 to March 2021, involving four steps: 1) Validity analysis; 2) Reliability analysis; 3) Utility analysis; 4) Attribution of scores to the approved indicators and analysis of the reliability of composite indicators. The flow of steps/products is shown in Fig 1.

Fig 1

Flow of steps and products for the development and validation of the QualiPresc.

As this study is a validation study of a measurement instrument for the management of quality in healthcare, it does not meet any of the specifications provided for in the EQUATOR Network, and it is therefore not possible to present the Research Reporting Guideline checklist here.

Step 1. Validity analysis

The validity analysis covers analysis of the relevance (face validity), adequacy (content validity) and theoretical support (criteria validity) of each potential indicator used to assess prescription quality. In this case, the relevance indicates how important the potential indicator is in assessing the quality of the prescription, the adequacy indicates how much the potential indicator is able to measure the quality of the prescription and the theoretical support refers to the literature on which the potential indicator is based. Thus, potential indicators were listed from a review of national and international literature on recommendations for the prevention of medication errors and safe prescribing practices [26, 27, 37–41]. Consensus on the indicators to integrate the instrument was established via the Delphi panel [42], which evaluated each item for adequacy and relevance as indicators of prescription quality.

Pharmacists, physicians, nurses and dentists with experience of work processes in academia and the health service were invited to participate in the Delphi panel. Thus, it covered a target audience that either evaluates the quality of the prescription and/or has its performance evaluated [40]. Among the 21 invited participants, 15 agreed to participate. The recruitment of specialists took place via telephone contact, during which the methodology of the Delphi panel was presented. After accepting the invitation, the material to be analyzed was sent via e-mail along with the informed consent form.

The Delphi panel was developed in three rounds, for each of which a Likert scale (0–9) was created for both relevance and adequacy of potential indicators. Each potential indicator was considered approved if at least 90% of the values assigned by experts where > 7 on the Likert scale [43], while those considered non-consensual were reassessed in subsequent rounds based on this same system. For the 1st round, specialists were given a period of 30 days to analyze the material, while, a period of 15 days was granted for both the 2nd and 3rd round. At the end of each time period, telephone contact was made with any specialists that in had not completed the analysis, in order to agree on a new delivery period, which contributed to 100% adherence.

In the 1st round, each potential indicator was analyzed as to its relevance, adequacy and theoretical support in assessing the quality of the prescription. The impact of each potential indicator on the safety and efficacy of the drug treatment was also considered in the 1st round, with the product of the score of these two items constituting the score assigned by each specialist. The final score of each potential indicator corresponded to the product of the average efficacy and safety from the 15 experts. For the 2nd and 3rd rounds, only the relevance and adequacy of each potential indicator were analyzed.

Step 2. Reliability analysis

The inter-rater reliability was assessed by two pharmacists through an independent review of 30 randomly selected drug prescriptions [44], based on duplicate prescriptions and special control prescriptions. These were prepared in January 2021 by 24 physicians assigned to Basic Health Units, and were filed in 5 district pharmacies and 1 pharmaceutical supply center in the municipality of the study.

This Brazilian city has an estimated population of 68,343 inhabitants [45] and, in the context of primary care, has 24 Family Health Teams, distributed in 18 Basic Health Units. It also features five district pharmacies and one pharmaceutical supply center, which centralizes the process of distribution/dispensing of prescription drugs in this context.

Inter-rater agreement was calculated from the % agreement (general agreement index) and Cohen’s kappa. The general agreement index is sufficient when agreement > 95%. A negative kappa or 0 indicates that agreement is low or non-existent; >0.2, mild; >0.4, moderate; > 0.6, good; > 0.8, high; and 1 indicates complete agreement. A low kappa (<0.4) should not be interpreted in the case of extreme presence of indicator compliance (<15% or >85%), as it may be biased. In this case, it was planned to intentionally select a sample with unknown extreme presence and recalculate kappa [46]. A kappa cut off point with 95% confidence interval (95% CI) ≥ 0.4 was considered.

Step 3. Utility analysis

After validating the indicators in terms of validity and reliability, they were analyzed in terms of the usefulness of their application. From this perspective, if the indicator is 100% in compliance with the pilot, it is not useful to monitor. If it is 0% compliant, it is necessary to ask if there is governance to include it in the monitoring. In this case, a pharmacist, a specialist in quality management and the municipal health secretary of that municipality were consulted. Where there was consensus about the usefulness of the indicator, it was considered useful; otherwise, it was excluded.

Step 4. Attribution of scores to the approved indicators and reliability analysis of the composite indicators

Three composite indicators were created: effectiveness, safety and quality of the prescription. These, made up of heterogeneous and uncorrelated indicators, correspond to a Formative Measurement Model, which advocates that the value of a construct is determined by the values of a set of indicators. In this perspective, the instrument’s score resulted from the sum of its items, which were given a weighting that reflected the contribution of each item to the construct [47].

For this purpose, a 3x3 matrix was built combining 2 dimensions, representing the impact of each item on the safety and efficacy of the treatment, with 3 levels each (absent = 1, moderate = 2, significant = 3). The 15 experts defined the values of this matrix during the 1st round of Delphi, and, based on it, the weighting for each potential indicator of the instrument, from the product of the average of the scores assigned to the safety dimension and the average of the scores assigned to efficacy dimension.

The three weighted composite indicators created were than analyzed for reliability through intra- and inter-rater agreement. In both cases, intraclass correlation coefficients (ICC) and Bland-Altman graphs were analyzed. In addition, histogram and box-plot graphs were calculated to analyze the usefulness of the indicators in identifying different levels of prescription quality and the difference between the three composite indicators [48, 49].

Ethical aspects

This study was approved by the Ethics and Research Committee of the Hospital Universitário Onofre Lopes under n° 64367517.3.0000.5292.

Results

Face, content and criteria validity

Initially, 29 potential indicators were listed: number of drugs/prescription, absence of erasure, patient’s full name, address, date of birth, prescriber’s full name, identification, institution’s name, institution’s address, date of prescription, use of abbreviations, use of acronyms/symbols, active ingredient, drugs with similar names, concentration, frequency of administration, record of allergy report, route of administration, dosage, pharmaceutical form, duration of treatment, directions on the use of the drug, non-pharmacological recommendations, medicine included in the institutional list officially approved, potentially dangerous drug, vague expression, drug with narrow therapeutic index, drug in therapeutic duplication and electronic prescription [26, 37–39].

Among the 15 experts who agreed to participate in Delphi were 6 pharmacists, 4 physicians, 3 nurses and 2 dentists. Of the pharmacists, four had dispensing experience in a hospital environment and two in an outpatient setting; meanwhile, three physicians had experience in prescribing in an outpatient setting and one had experience in a hospital setting. All three nurses had experience with care in both outpatient and hospital settings, while one dentist had experience in prescribing in an outpatient setting and the other in a hospital setting. In addition, everyone has experience of work processes in academia and the health services.

For the 1st round, nine indicators were agreed: prescriber identification, prescription date, concentration, dosage, pharmaceutical form, route of administration, frequency of administration, duration of treatment and directions on the use of the drug.

In the 2nd round, seven indicators were agreed upon: electronic prescription, absence of erasures, patient’s full name, date of birth, record of allergy report, active ingredient and non-pharmacological recommendations.

In the 3rd round, 1 indicator was agreed upon (medicine included in the institutional list officially approved), ending the Delphi panel with 17 (57.1%) indicators agreed among the 29 proposed.

The criterion validity was analyzed after expert consensus, based on the level of evidence of the reference that supports the recommendation for each indicator, if the related practice has a proven relationship with the prevention of medication errors. The detailed results of the relevance and adequacy analyses in the three Delphi rounds are shown in S1 Table.

Reliability and usefulness of indicators

The reliability assessment found inter-rater agreement for the 17 consensus indicators, whose kappa indexes ranged from 0.65 (95% CI 0.48–0.82) to 1.0 (95% CI 1.0–1.0) and overall agreement rates ranged from 87% to 100% (S2 Table).

The subsequent utility assessment resulted in the exclusion of four indicators, one because it did not apply to the reality of the local context (electronic prescription) and three because the prevalence of the two evaluators was 100% (absence of erasure, patient’s full name and date of prescription), which indicates that these items do not contribute to compromising the quality of prescription writing, since prescribers have already incorporated these items into their professional practice (S2 Table).

Once the validation process was completed, the final version of the instrument consisted of 13 indicators. As an integral part of QualiPresc instrument, in order to contribute to the optimization of the reproducibility of the QualiPresc application process, definitions/clarifications of the validated indicators were generated according to S3 Table.

Graphical analysis via histograms and box-plots showed that the distribution of scores for the composite indicators had a variability similar to the normal distribution (Fig 2), revealing the usefulness of monitoring the instrument. In this variability, the indicator is sensitive to the detection of low, medium and high performance in efficacy, safety and quality.

Fig 2

Matrix-plot for reliability and utility analysis of composite indicators.

Thus, the Shapiro-Wilk test and intraclass correlation coefficient (ICC) test-retest were calculated from the quality indicator, resulting in p = 0.38 and ICC = 1.0, respectively.

Additionally, the inter-rater reliability of the composite indicators was analyzed through Bland-Altman agreement analysis (Fig 2). In addition to the image, the agreement is reinforced by the absence of statistically significant differences between the measures of the two evaluators (p = 0.622, p = 0.633 and p = 0.615, respectively), and none of these three composite indicators (the data) have a proportion bias relating to the distribution of evaluation differences among participants that is evenly distributed along the mean line and between the lower and upper limits of the 95% CI (p = 0.574, p = 0.574 and p = 0.525, respectively).

After transforming the three composite indicators to base 100 and comparing the means of each one, they are highly correlated (p<0.05), and the p-value of the difference between the means is similar (p>0.05) in a comparison of paired averages, showing that it is not useful to measure the three composite indicators because the quality is sufficient to represent the concept with the weight of each indicator. This can also be seen in Fig 2, which reveals similar data.

QualiPresc instrument content

Once the validation process is complete, QualiPresc uses a concise template in Excel® format (S1–S3 Files). QualiPresc consists of 3 spreadsheets: the 1st is intended for the operational definition of indicators; the 2nd is for the tabulation of data collected from drug prescriptions; and the 3rd is for the evaluation and monitoring of the quality of these prescriptions, in general and by indicator, through trend graphs.

Discussion

General contributions

Drugs constitute one of the five foundations for quality care in a health system where accessibility, high quality and ordering primary care are three of the basic principles that are essential for this to become a reality [41]. From this perspective, QualiPresc contributes to monitoring the quality of drug prescriptions at this level of health care, proposing to overcome the dangerous lack of validated instruments to assess the quality of prescription writing, based on structuring from the prescriber, the patient and the drug information.

Thus, this instrument can contribute to the WHO efforts to reduce medication errors by 50% over 5 years [38], both through research and through its use in health service regulation strategies or collaborative quality improvement projects [41]. This initiative aims to improve the quality of drug prescriptions in primary care, where medication errors are the main safety concern [2-5].

Validity of QualiPresc in assessing the quality of prescriptions in primary care

Face and content validity were assessed using an internationally recognized consensus method [42], whose expert panel covered both a target-audience that measures quality and a target-audience that has its performance measured [40]. Additionally, since the kappa index has limitations [50], reliability was assessed in light of the prevalence of compliance, along with overall agreement. The sample size used in the expert committee and in the reliability pilot study are recognized as adequate in other studies [43, 51, 52].

Potential methodological challenges concerning the use of composite indicators were overcome through weighting by expert consensus, which is considered a valid option [46, 53]. In relation to a possible cut off point, as all indicators are considered essential, the quality goal is to reach 100%.

The validity of QualiPresc can also be verified by the justification of each indicator. For example, reference to the drug by trade name, as opposed to the “active ingredient”, goes against the WHO’s target of 100% of drugs being prescribed by generic name [37] in order to facilitate access and rational use of the drugs [38]. In relation to the indicator “medicine included in the institutional list officially approved”, failure to comply with the list of essential drugs implies the unavailability of the prescribed drug in the public sector, which forces it to be obtained via the private sector. This does not always represent the best scientific evidence of safety and efficacy [54]. In addition, the omission of the “concentration” of the prescribed drug which consists of a numerical unit and a unit of measure, expressed via an abbreviation/acronym/symbol, can compromise the efficacy and safety of the drug treatment. As “concentration”, “dosage”, “frequency of administration” and “duration of treatment” are interdependent items, failing to indicate any of these may result in an underdose or overdose [55]. “Pharmaceutical forms” require specific “routes of administration”, which are interdependent items that consider the convenience, stability, biological compatibility, speed of onset and duration of action and undesirable effects [55]. “Directions on the use of the drug” and “non-pharmacological recommendations” complement the other items mentioned above in order to contribute to the efficacy and safety of the pharmacological therapy [26, 56]. Additionally, “recording an allergy report” constitutes an impediment to the prescription of a drug to which the patient is allergic and cautions against other representatives of the same pharmacological class [56]. Furthermore, the “prescriber´s identification” enable contact with the prescriber should any clarification or pharmaceutical interventions be needed and the “patient’s date of birth” constitutes an additional patient identifier [56].

Although valid and reliable, the electronic prescription indicator was not considered to be useful in the context of primary care, as aspects of infrastructure and professional qualification need to be fulfilled prior to its implementation. In Brazil, only 51.2% of Basic Health Units have a computer and only 35.4% have internet access [57]. There is a tendency for municipalities with low human development Index to have more precarious Basic Health Units, where the most influential factor is the average municipal income. In this movement, the factor that negatively affects the infrastructure of Basic Health Units is that of being located in the North or Northeastern region, which constitutes very strong evidence of the regional inequality historically established in the country [58]. The evaluation of its effects in primary care found a reduction in prescribing errors when applied to a limited number of potentially dangerous drugs and with physician-pharmacist communication [59]. That said, electronic prescription can contribute to the reduction of prescription errors, as long as its implementation is properly planned and monitored, and its access authorized through prior training by specialists in patient safety [60, 61].

The potential readability indicator was excluded because it makes it impossible to apply the other indicators due to the impossibility of reading the prescription content. In addition, the readability test requires face-to-face administration of a questionnaire to at least 20 users, preferably from the target population of the drug, and excludes health professionals, in order to avoid results biased by their specialized knowledge [62, 63].

Although the panel of experts did not agree on the inclusion of potential indicators on the use of abbreviations/acronyms/symbols, when focusing on the structuring of the prescription document and the technical specifications of the drug, QualiPresc considers compliance with abbreviations/acronyms/symbols, as it imposes the mandatory presence of items such as active ingredient, dosage, concentration, route of administration, frequency of administration and duration of treatment. In addition, it considers that abbreviations/acronyms/symbols demonstrate errors related to the medication and are not accepted for compliance with the other indicators, in accordance with use/non-use recommendations [26, 64], as per S3 Table.

Practical application of the QualiPresc

This set of validated indicators can be applied both by external evaluators (inspectors, accreditors, certifiers, auditors) and internal ones (self-assessment and continuous improvement of the health service), preferably pharmacists, who have a specific assignment to evaluate the prescription with regard to pharmacological, legal and biopsychosocial aspects [56]. In addition, it can be an instrument for teaching patient safety and good prescription practices in the academic environment.

QualiPresc is an alternative to long and paper-dependent measuring instruments and is freely available and easily used by health professionals from any primary care service, for the automatic collection and analysis of data monitoring. An average of 1 hour is invested to analyze 30 prescriptions.

It also constitutes software prototyping to be developed for the aggregate analysis of the quality of prescriptions in primary care networks, periodically, for application of the results in quality control charts, with a view to monitoring and continuous improvement, which will be developed initially, in the same municipality where the reliability and usefulness of the potential QualiPresc indicators were analyzed.

Study limitations

Convergent validity was not assessed due to the absence of instruments to assess the quality of prescriptions aimed at observing prescription writing, which constitute the gold standard for this comparison.

Criteria validity was evaluated based on the references of each indicator, but none represents strong recommendations, despite the very high consensus on face and content validity. However, the availability of QualiPresc will favor the production of robust evidence for the criteria validity of the prescription writing when relating them to medication errors.

QualiPresc has been validated for use in the context of primary care in a country with continental dimensions, whereas the reality is very diverse. The application of QualiPresc in significantly different contexts, such as secondary care and tertiary care, requires cross-cultural adaptation and new content validity, for which the list of 29 potential indicators may be useful.

Conclusion

The study shows the validity, reliability and usefulness of QualiPresc for the assessment of prescription writing, in the scope of primary care. Application to significantly different contexts, such as secondary care and tertiary care, requires cross-cultural adaptation and new content validity. Educators, managers and healthcare professionals can access it free of charge online to monitor and provide feedback to drug prescribers.

Validity analysis of potential indicators.

(DOC)

Click here for additional data file.

Reliability and utility analysis of consensual indicators.

(DOC)

Click here for additional data file.

Definitions and clarifications of the validated indicators.

(DOC)

Click here for additional data file.

Portuguese QualiPresc.

(PDF)

Click here for additional data file.

English QualiPresc.

(PDF)

Click here for additional data file.

Spanish QualiPresc.

(PDF)

Click here for additional data file.

25 Jan 2022

Submitted filename: Plos one 202201.docx

Click here for additional data file.

16 Feb 2022

Journal Requirements / Authors' Response

1. When submitting your revision, we need you to address these additional requirements.

Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

1. This request has been fulfilled in accordance with https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please include captions for your supporting information files (Portuguese qualipresc, English qualipresc, and Spanish qualipresc) at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information.

2. This request has been fulfilled in accordance with the Supporting Information guidelines http://journals.plos.org/plosone/s/supporting-information.

This information has been added to the Discussion, line 362 and to the Supporting information, lines 597-599 of the manuscript.

3. Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the reference list and also include a citation and full reference for the retraction notice.

3. References 17-25 and 36 have been added to the manuscript (lines 81 and 99, respectively) and can be found in the References section, lines 440-467 and 497-500, respectively

Reviewer #1 / Authors' Response

1. (Abstract, line 28) What does “writing criteria” mean? Is this a commonly used phrase? By “writing criteria”, we mean prescription writing, which covers the prescriber, patient and drug information. This has now been clarified in line 28 of the Abstract, and the expression “writing criteria” has been replaced by the term “prescription writing” in the title and throughout the body of the article.

2. (Abstract, Methods) Please write about where this study has taken place, who are the target patients or medical institutes of the study, and their numbers.

The study was conducted in a municipality in Northeastern Brazil, based on prescriptions prepared in January 2021 by physicians assigned to 18 Basic Health Units and filed in 6 distribution/dispensing units. This information has been added to the Abstract (Methodology), lines 32-34.

3. (Methods, line 105) How did you create QualiPresc? Does “Step 1. Validity analysis” correspond to the method of creating QualiPresc? In that case, it is bizarre that how the QualiPresc was created is written in the section of validation analysis. In other words, you should write separately about the development and validation of QualiPresc.

No, the validity analysis does not correspond to the method used to create QualiPresc. The validity analysis covers analysis of the relevance (face validity), adequacy (content validity), and theoretical support (criteria validity) of each potential indicator used to assess prescription quality. In this case, the relevance indicates how important the potential indicator is in assessing the quality of the prescription, the adequacy indicates how much the potential indicator is able to measure the quality of the prescription, and the theoretical support refers to the literature on which the potential indicator is based. This information has been added to the Methods section of the manuscript, lines 123-128.

4. (Results) All the Figures are blurred, and difficult to see. Please increase the quality of the Figures. You should check it before submission. In addition, Figures 2-4 can be presented by Tables. Why did you present them in Figures?

We have made changes to the resolution of Figures 1 and 5 (resolution increased from 300 dpi to 600 dpi), and Figures 2-4 have now been converted to Tables (Tables 1-3), as requested. Thus, Figure 5 was renamed Figure 2 in the revised manuscript.

5. (Results, Figure 4) Does Figure 4 correspond to the finalized QualiPresc? Please write more about how to use it in practice.

No, Figure 4 (renamed Table 3 in the revised manuscript) does not correspond to the finalized QualiPresc, but is an integral part of the QualiPresc instrument. This has been clarified in the Results section of the manuscript, in line 251 (“As an integral part of QualiPresc instrument,”). QualiPresc consists of 3 spreadsheets: the 1st is intended for the operational definition of indicators; the 2nd is for the tabulation of data collected from the drug prescriptions; and the 3rd is for the evaluation and monitoring of the quality of these prescriptions, in general and by indicator, through trend graphs. This information has been added to the Discussion section of the manuscript, lines 365-369.

6. (Discussion) As you write in the limitations, the generalizability of QualiPresc may be limited. For example, it might be good to use it in other countries. Therefore, you should write about where this method can be applicable in the Conclusions and the Conclusions section of the Abstract.

The application of QualiPresc in significantly different contexts, such as secondary care and tertiary care, requires cross-cultural adaptation and new content validity, for which the list of 29 potential indicators may be useful. This information has been added to the manuscript, in the Discussion, lines 385-388; the Conclusion, lines 391-393; and the Abstract (Conclusion), lines 46-47.

Reviewer #2 / Authors' Response

Abstract: Objective, line 29-31: The sentence is too long, making it difficult to retain the main message from the beginning to the end of the sentence. Consider dividing into at least two sentences so that it is easier to comprehend.

We have changed this sentence in accordance with Proofreading.

Abstract: Line 36 & 38: Consider spelling the numbers "29" and "12", since they are at the beginning of the sentences.

We have changed this in accordance with your request (lines 39-40).

Introduction: Line 52: What is the "damage" in the sentence referring to? Is it injury or harm related to an unsafe medication process? It is not clear to me.

This term concerns the harm related to health care. This has now been clarified in the Introduction, line 55.

Introduction: Line 56: Is this "suffer harm" related to general harm from healthcare or medication-related harm?

It is harm related to health care. This has now been clarified in the Introduction, line 59.

Introduction: Line 67: "In this context .." What is the context? Please elaborate.

The context is primary care, and this has now been clarified in the Introduction, line 70.

Introduction: Line 67-68: "technological advances are fast ...". How does the fast technological advances lead to less adequate prescription practices? It is not clear to me. Fast technological advances demand commitment and time investment from prescribers in order to gain the appropriate knowledge inherent to this health technology, including the best scientific evidence about efficacy, safety and cost-effectiveness. This has now been clarified in the Introduction, lines 72-74.

Introduction: Line 72: "Writing criteria". Please consider briefly expanding what the writing criteria is in the context of primary care practice in Brazil. What should be the ideal practice for the writing criteria and why does the current practice warrant the need for assessing the quality of the prescription with the instrument?

Ideal prescription writing encompasses the prescriber, the patient, and the drug information; however, this information is often missing or incomplete. In Brazil, the practice of prescription writing is based on recommendations from the Ministry of Health. This information has now been added to the Introduction, lines 79-82. Additionally, the expression “writing criteria” has been replaced by the expression “prescription writing”, in the title and throughout the body of the article.

Introduction: Line 74: Not very clear to me what "the care of" means in the sentence.

The expression “the care of” has been replaced by the word “across” in the Introduction, line 83.

Introduction: Line 83-85: Could the use of electronic prescribing (e.g., via the use of an electronic health record system with prescription templates) in well-resourced settings contribute to the reason why there is "no valid, reliable and useful instrument that allows for the assessment of prescription quality in terms of writing criteria"?

The sentence “Furthermore, this cannot be resolved through an electronic prescription system, even in well-resourced settings.” has now been added to the Introduction, lines 98-99.

Introduction: Line 87: "results in ignorance about the impacts of these aspects on medication errors." Please can you briefly elaborate on the impact (e.g., patient harm or cost related to harm to patients) of the absence of a tool to assess prescription writing criteria on patient safety, if there is literature available?

The phrase “given that medication errors represent the main safety incidents resulting in adverse events in primary care” has now been added to the Introduction, lines 96-98.

Methods: Line 117: How did you select which specialists you called over the phone for recruitment?

The following text has been added to the Methods section, lines 134-137: “Pharmacists, physicians, nurses, and dentists with experience of work processes in academia and the health service were invited to participate in the Delphi panel. Thus, it covered a target audience that either evaluates the quality of the prescription and/or has its performance evaluated. Among the 21 invited participants, 15 agreed to participate.” The invited participants were considered to be key informants who met the inclusion criteria.

Methods: Line 118: "systematic of the study". Please it is not clear what this phrase means in the sentence. May need to rephrase to make the meaning clearer.

This sentence has been reworded to clarify that this relates to the methodology of the Delphi panel (see the Methods section, lines 138-139). “The recruitment of specialists took place via telephone contact, during which the methodology of the Delphi panel was presented.”

Methods: Line 119: "Informed Consent Form". Please consider using lower case letters at the beginning of each word hence 'informed consent form'.

We have changed this in accordance with your request (line 140).

Methods: Line 120. "Delphi Panel". Consider small letter at the beginning, hence 'panel'.

We have changed this in accordance with your request (line 141).

Methods: Line 120: "3 rounds". Consider spelling out 3 since it is a figure less than 10.

We have changed this in accordance with your request (line 141).

Methods: Line 142-146: The sentence is too long. Makes it difficult to read and understand. Please consider dividing into a minimum of 2 sentences.

We have changed this in accordance with your request (line 160).

General comment for the methods section: Overall, in the methods section, some of the paragraphs are very short (just 2 sentences) and it may appropriate to combine some paragraphs to reduce the total number of paragraphs.

We have changed this in accordance with your request.

Methods: Line 165: "If the response it yes, then ok." Please check and revise. Not clear to me.

The phrase “If the response it yes, then ok” has been replaced by the text “Where there was consensus about the usefulness of the indicator, it was considered useful; otherwise, it was excluded” in the Methods section, lines 181-182.

Methods: Line 172: "Weighing". Please check if you intended to write 'weighting'. If not, then consider checking if it will be appropriate.

We have changed this in accordance with your request (lines 189, 194 & 300).

Results: Lines 232 and 233: "is completed" and "consists". Please check if you need to use the past tense since it has happened.

We have changed this in accordance with your request (lines 250-251).

Discussion: Line 309: "UBS": Please the international readers may not understand the abbreviation. Consider writing out.

We have changed this in accordance with your request throughout the body of the article.

Submitted filename: Response to Reviewers.doc

Click here for additional data file.

22 Mar 2022

11 Apr 2022

Journal Requirements / Authors' Response

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

None of the papers cited in this manuscript were retracted.

Reviewer #1

6. Review Comments to the Author

1. (Tables) I think Tables 1-3 need to be shown in the main manuscript, not in supplementary information.

1. We agree. Tables 1-3 will be in the manuscript. They are not Supplementary Information. The current view at the end of the manuscript is only during the review process, as this is Plos One's organization during this process (tables and figures). We have included the titles of all tables and figures in the text to indicate the approximate location where they will be inserted. See manuscript template (Download sample manuscript body (PDF)).

2. (Discussion, lines 363) It is bizarre that the contents of QualiPresc are shown for the first time in Discussion. You should indicate it in the Methods or Results.

2. Changes made. This information is presented in the Results topic (lines 277-282).

3. In addition, QualiPresc in English version needs to be written in English, but not in another language.

3. Changes made. View Supplementary Files 1-3.

Submitted filename: Response to Reviewers.doc

Click here for additional data file.

14 Apr 2022

VALIDATION OF THE QUALIPRESC INSTRUMENT FOR ASSESSING THE QUALITY OF DRUG PRESCRIPTION WRITING IN PRIMARY HEALTH CARE

PONE-D-21-37829R2

Dear Dr. Mariz Batista,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Benito Soto-Blanco, DVM, MSc, PhD

Academic Editor

PLOS ONE

22 Apr 2022

PONE-D-21-37829R2

VALIDATION OF THE QUALIPRESC INSTRUMENT FOR ASSESSING THE QUALITY OF DRUG PRESCRIPTION WRITING IN PRIMARY HEALTH CARE

Dear Dr. Mariz Batista:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Benito Soto-Blanco

Academic Editor

PLOS ONE