Commonalities among dental patient-reported outcomes (dPROs)-A Delphi consensus study.

Improvement of patients' oral health-related quality of life (OHRQoL) is the main goal of oral health care professionals. However, OHRQoL is not a homogenous construct and how to assess it is challenging because of the large number of currently available instruments. Investigating available instruments and what they have in common would be necessary for consolidation and standardization of these instruments into a smaller set of tools. If the OHRQoL dimensions including Oral Function, Orofacial Pain, Orofacial Appearance, and Psychosocial Impact are the fundamental building blocks of the dental patient's oral health experience, then these dimensions should be measured by generic multi-item dPROMs. In this study, a panel of 11 international dentists use the Delphi consensus process to determine how well 20 of these instruments measured the four OHRQoL dimensions. All 20 dPROMs questionnaires assessed at least one OHRQoL dimension while all four OHRQoL dimensions were measured by at least one dPROM instrument, i.e., the four OHRQoL dimensions were essential components of the patient's oral health experience. This shows that the currently available generic multi-item dPROMs have a lot in common, in that they share Oral Function, Orofacial Pain, Orofacial Appearance, and Psychosocial Impact as targeted dimensions. Based on these commonalities, it is plausible and desirable to move towards a single four-dimensional metric to assess oral health impact in all clinical, community-based, and research settings. This step is necessary to advance evidence-based dentistry and value-based oral health care.

Background

Dental patient-reported outcomes (dPROs) represent what is important to dental patients [1,2]. Therefore, dental patient-reported outcome measures (dPROMs) determine which dental treatments are the most effective in addressing dental patients’ oral health problems. Two systematic reviews identified 155 multi-item dPROMs that capture oral disease impact [3,4]. These dPROMs provide a multitude of opportunities to measure oral disease impact on the quality of life, but they also present challenges to dentists and researchers such as which instrument to select and how to interpret instrument scores. The dPROMs essentially measure one or more of the four dimensions of OHRQoL Oral Function, Orofacial Pain, Orofacial Appearance, and Psychosocial Impact [5]. A search for commonalities among the currently available dPROMs could simplify and consolidate the complex array of existing dPROMs into a smaller and better set of instruments to measure the impact of oral disease. This would also facilitate the determination of treatment effectiveness in dental interventions comparable to the results of research studies. Consequently, evidence-based dental practice and value-based oral health care would greatly benefit from a standardization of patient-reported outcomes assessment.

A Delphi technique is such a structured process to derive a consensus about a topic such as the relationship between dPROMs and OHRQoL dimensions. The conceptual advantages of a Delphi process are accompanied by technical advantages of involving many dental experts with international representation and reasonable burden using electronic communication. This study implemented the Delphi process using a panel of international dental experts and aimed to investigate the commonalities among 20 generic multi-item dental patient-reported outcome measures (dPROMs).

Methods and material

In the present study, 20 questionnaires that measure oral disease impact were assigned to the four OHRQoL dimensions Oral Function, Orofacial Pain, Orofacial Appearance, or Psychosocial Impact by dental experts using a Delphi process after undergoing a reliability assessment and reaching consensus. The oral disease-generic dPROMs were identified as instruments that measured oral disease impact across of a broad range of patients [3]. The 20 questionnaires contained 36 unique dPROs. These dPROs were measured by 53 dPROMs [3].

Dental experts

The dental experts comprises of a group of 11 international dentists, known to one of the authors (MTJ). The selection criteria for dental experts were that they had to be a dentist, had to have practiced dentistry in the past year, and were fluent in the English language. Each dental expert received an invitation by email with detailed study instructions. The dental experts were requested to familiarize themselves with the contents of 20 dPROMs. They were provided the abstract of the published dPROM questionnaires by each author(s), a brief description of what the dPROM intended to measure, the reference of the dPROM, as well as the dPROM items. All dental experts who were approached to participate in the study had accepted the invitation, providing a response rate of 100%.

All dentists were full-time dentists, six were women and five men. Of those dental experts, six were from Europe and the rest from the United States. Five of the dentists were Doctorates of Dental Surgery (DDS) and a Doctor of Philosophy (PhD). The dental experts have had between five to 25 years of experience in the dental field. The major criterion for the dental experts were that they had to be a dentist. Dental experts should have had some understanding and experience about patients’ functional, pain-related, aesthetical, and psychosocial problems in regards to oral health [6].

The Delphi process

Calibration of dental experts

The definitions of the four dimensions of OHRQoL: Oral Function, Orofacial Pain, Orofacial Appearance and Psychosocial Impact were provided to all dental experts. They were asked to familiarize themselves with their meaning and content, even if they had an intuitive understanding of what these dimensions were. These dimensions were identified from 10,778 49-item Oral Health Impact Profile (OHIP-49) data of prosthodontic patients and general population subjects covering an age range of 40 years or more in 35 studies conducted in six countries [7-10]. The OHIP-49 data formed an aggregate secondary data set used for the factor analyses in the Dimension of Oral Health-Related Quality of Life Project [7].

Assignment of PROM to OHRQoL dimensions using the Delphi technique

The data was collected using an electronic survey in the English language. The survey was generated using Qualtrics software. Dental experts received an electronic link to the survey and provided their anonymous responses online by choosing a 7-digit number that was not to be shared with other dental experts or the study organizer. This number was used to link participants’ responses to the corresponding dental expert in both round one and round two. While the number identified a particular dental expert, completion of all ratings was anonymous.

The dental experts were requested to assess the 20 questionnaires one by one. The dental experts were asked to indicate with the four scores as to what extent the specific questionnaire assesses each of the four dimensions. The scores could range from 0 to 10 points, where 0 points meant the questionnaire does not measure an OHRQoL dimension at all and 10 points meant the questionnaire measures the dimension perfectly.

In subsequent rounds of the Delphi process, dental experts did not have access to their own previous ratings but were shown summarized group scores from other dental experts. For each questionnaire, rating medians and interquartile for all four dimensions were presented. The dental experts were asked to provide ratings again, but this time based on the knowledge of what the group’s assessment was. The dental experts were informed that group ratings can but do not have to influence their own new assessment. The Delphi process was stopped when a consensus was reached, which has been defined in the following section labelled as “Consensus—Validity assessment of panel expert ratings.”

Reliability assessment of dental experts’ ratings

To find the commonalities among dPROs using dental experts’ opinions, the expert ratings needed to be stable over time, i.e., an expert should have the same opinion when asked again. Therefore, a test-retest assessment was performed by asking dental experts twice within two weeks to assign questionnaires to dimensions without knowledge of previous results. Ratings from the first round of the Delphi process were used as “test” results. Before dental experts were given feedback on the group’s assessment and ask to continue with the second round of the Delphi process, another survey round (“retest”) was performed to determine test-retest reliability.

Out of 11 dental experts, only seven provided retest data. We calculated an intraclass correlation coefficient (ICC) according to Fleiss [11], using a one-way ANOVA for each expert and for each dimension, resulting in 28 ICCs from 7 (number of raters) times 4 (number of dimensions). Box plots were used to present central tendency and spread of ICCs overall and per OHRQoL dimension. The median ICCs were interpreted according to Fleiss [11]. An ICC <0.40 is considered “poor,” 0.40–0.75 represents “fair-to-good,” and >0.75 is seen as “excellent” reliability. The software that was used to interpret this data is called, Stata.

The internal consistency of the panel expert ratings was also determined. Cronbach’s alpha, the overall and by dimension, was computed for the dental expert responses when they were asked which dimension a dPROM belonged to. A Cronbach’s alpha value of 0.90 or higher was adopted as the threshold for sufficient panel expert ratings’ internal consistency [12].

Consensus–validity assessment of panel expert ratings

A Delphi study does not lead automatically to a consensus; it simply provides an opportunity to reach a consensus [13]. Only if dental experts agree on the topic, the result can be considered a consensus [13]. Therefore, we examined the “dimensionality” of dental expert ratings. Dimensionality refers to the number and nature of the attributes reflected in the dental experts’ responses. If dental expert responses could be considered unidimensional, i.e., they would only measure one attribute, it could be assumed that the dental experts have reached a consensus on the topic of commonalities among multi-item dPROMs that measure disease impact. In the case of dental experts’ response ratings being multi-dimensional, i.e., several attributes would be measured, it could be assumed that the dental experts would not have a homogenous opinion about the topic, indicating that a consensus was not reached.

To determine the dimensionality of dental expert ratings, a parallel analysis was performed suggested by Horn [14]. For 9 dental experts, this technique calculated the 9 dental expert’s ratings x 9 dental expert ratings correlation matrix from the 20 PRO questionnaires x 9 dental experts dataset. This technique created a scree plot [15]. The eigenvalues of the sample correlation matrix were plotted against their position from largest to smallest (1, 2, …, 9). The eigenvalue points related to a straight line. The parallel analysis also created a simulated dataset with 180 observations randomly sampled from 9 independent normal variates. It calculated a 9 x 9 correlation matrix for the simulated data and extracted the 9 eigenvalues, ordering them from largest to smallest. This step was repeated k times (in our case, k = 1000). From 1000 eigenvalues at positions 1, 2, …, 9, medians were calculated. The 9 eigenvalue medians were related to a dashed line and overlaid on the Scree plot. Finally, the intersection of the solid (actual data) and the dashed lines (simulated data) was the cutoff for determining the number of dimensions present in the data. Panel expert ratings would be considered unidimensional when only one eigenvalue would be above the dashed line of the simulated data.

Assignment of questionnaires to OHRQoL dimensions

Having reached a consensus, median ratings for the group of raters were calculated for each questionnaire and for all four dimensions. Consequently, each questionnaire received four median ratings. For example, the “Psychological Impact of Dental Aesthetics Questionnaire” [16] received 0, 0, 9, and 10 points for median ratings for the dimensions Oral Function, Orofacial Pain, Orofacial Appearance, and Psychosocial Impact, respectively.

From this example, it becomes clear that situations such as 0, 0, 0, and 10 points, indicating a perfect fit for only one dimension, could have rarely occurred. It is expected that dPRO questionnaires measure several dimensions. Therefore, in addition to considering only median ratings of equal to or larger than 5 on the 0 to 10 numeric rating scale as, the highest median rating (in previous example, the 10 points) and any other median ratings within 2 points of the highest median rating (in the previous example, the 9 points) were considered indicators of the dimension. Therefore, the example of the questionnaire “Psychological Impact of Dental Aesthetics Questionnaire” measured Psychosocial Impact (score of 10) and Orofacial Appearance (score of 9) but did not measure the Oral Function (score of 0), or the Orofacial Pain (score of 0).

Two hypotheses about the presence of commonalities among dPROMs

To interpret commonalities among the 20 questionnaires, two hypotheses were generated:

All 20 dPRO questionnaires assess at least one OHRQoL dimension.

All four OHRQoL dimensions are measured by at least one questionnaire.

Confirmation of these two hypotheses would be considered evidence for the commonalities among generic multi-item dPROMs that measure oral disease impact.

Results

Reliability of dental expert ratings

Overall, temporal stability of the questionnaire to OHRQoL dimension assignment was “excellent” according to Fleiss’ guidelines [11]. This was indicated by a median reliability coefficient of 0.86 for all dimensions and all dental experts combined. For three of the four dimensions, the median reliability coefficient exceeded the 0.75 threshold, considered “excellent” reliability, indicative of temporal stability. For the fourth dimension Orofacial Pain the coefficient was 0.74 (Fig 1).

Fig 1

Reliability coefficients (N = 28) examining the panel expert ratings’ temporal stability for the four OHRQoL dimensions.

The dispersion of the reliability coefficients varied across the four dimensions. More variable responses were observed for the dimension Psychosocial Impact compared to the other dimensions Oral Function, Orofacial Pain, and Orofacial Appearance. One ICC estimate (0.20) was an outlier compared to 27 estimates that ranged from 0.56 to 0.99. A Cronbach’s alpha of 0.95 was calculated for all the dental experts’ ratings, and an estimate of 0.90 for all the dimensions. These findings indicated internal consistency sufficient for “clinical application.” [12].

Validity of dental expert ratings

For all dimensions combined and for each of the four dimensions separately, a dominant general factor of uni-dimensionality among the underlying dental expert ratings was present (Fig 2). The differences among dimensions were small. The four plots showed only one eigenvalue exceeded the dashed line of the random eigenvalues, thus illustrating the uni-dimensionality of each dimension. The validity assessment findings indicated that the dental experts reached a consensus on the topic.

Fig 2

Horn’s parallel analysis to determine expert panelists’ responses dimensionality for the four OHRQoL dimensions.

Hypothesis 1—All 20 dPRO questionnaires assess at least one OHRQoL dimension

Questionnaires to OHRQoL dimensions assignment

Seven of the 20 dROMs measured a single dimension, seven measured two dimensions, five measure three dimensions, and one instrument measured all four dimensions, confirming the hypothesis, found in Table 1.

Table 1

dPRO questionnaires measuring OHRQoL dimensions.

”x” indicates a median expert rating ≥5 points and ≤2 points within the highest dimension rating. Empty cells indicate median expert ratings >2 points lower than the highest dimension rating.

No.	Questionnaire	Abridged name	Authors	Publication year	Oral Function	Orofacial Pain	Orofacial Appea-rance	Psycho-social Impact
1	Rand Dental Health Index [17]	Rand DHI	Gooch, Dolan, Bourque	1989		x	X	x
2	Geriatric Oral Health Assessment Index [18]	GOHAI	Atchison, Dolan	1990	x	x		x
3	Jaw Disability Checklist [19]	JDC	Dworkin, LeResche	1992	x
4	Dental Impact Profile [20]	DIP	Strauss, Hunt	1993	x			x
5	Mandibular Function Impairment Questionnaire [21]	MFIQ	Stegenga, de Bont, de Leeuw, Boering	1993	x
6	Oral Health Impact Profile [22]	OHIP	Slade, Spencer	1994	x	x		x
7	Subjective Oral Health Status Indicators [23]	SOHSI	Locker, Miller	1994	x	x		x
8	Dental Impacts on Daily Living [24]	DIDL	Leao, Sheiham	1996	x	x	x	x
9	Oral Health Quality of Life Inventory [25]	OH-QoL	Cornell, Saunders, Paunovich, Frisch	1997	x			x
10	Oral Impacts on Daily Performance [26]	OIDP	Adulyanon, Sheiham	1997	x			x
11	Oral Health Related Quality of Life-UK [27]	OHQoL-UK	McGrath, Bedi	2001	x			x
12	Manchester Orofacial Pain Disability Scale [28]	MOPDS	Aggarwal, Lunt, Zakrzewska, Macfarlane, Macfarlane	2005	x	x		x
13	Psychological Impact of Dental Aesthetics Questionnaire [16]	PIDAQ	Klages, Claus, Wehrbein, Zentner	2006			x	x
14	Jaw Functional Limitation Scale [29]	JFLS	Ohrbach, Larsson, List	2008	x
15	Chewing Function Questionnaire—Alternative Version [30]	Alt-CFQ	Baba, John, Inukai, Aridome, Igarahsi	2009	x
16	Modified Symptom Severity Index [31]	Mod-SSI	Nixdorf, John, Wall, Fricton, Schiffman	2010		x
17	Orofacial Esthetic Scale [32]	OES	Larsson, John, Nilner, Bondemark, List	2010			x
18	Brief Pain Inventory-Facial [33]	BPI- F	Lee, Chen, Urban, Hojat, Church, Xie, Farrar	2010		x		x
19	New Chewing Function Questionnaire [34]	New-CFQ	Peršić, Palac, Bunjevac, Čelebić	2013	x
20	Craniofacial Pain and Disability Inventory [35]	CF-PDI	La Touche R, Pardo-Montero, Gil-Martínez, Paris-Alemany, Angulo-Diaz-Parreño, Suarez-Falcon, Lara-Lara, Fernandez-Carnero	2014		x		x

Hypothesis 2—All four OHRQoL dimensions are measured by at least one questionnaire.

Oral Function, Orofacial Pain, Orofacial Appearance, and Psychosocial Impact were measured by 14, 9, 4, and 13 questionnaires, respectively, confirming the hypothesis.

Confirmation of these two hypotheses was considered evidence for the existence of commonalities among the 20 questionnaires for adult dental patients, found in Table 1.

Discussion

An international panel of dental experts provided reliable and valid ratings so that a consensus could be reached in a Delphi study regarding how well 20 dental patient-reported outcome (dPRO) questionnaires fit the four dimensions of oral health-related quality of life (OHRQoL). Each questionnaire assessed at least one OHRQoL dimension, and each of the four dimensions was at least assessed by one questionnaire. Based on these results, it was concluded that multi-item dPROMs have the four OHRQoL dimensions Oral Function, Orofacial Pain, Orofacial Appearance, and Psychosocial Impact in common.

Comparison with literature

The commonalities among multi-item dPROMs assessing disease impact are fundamental for the meaning and scoring of questionnaires used to measure perceived oral health because they represent the major attributes measured by these instruments. While dimensionality for individual dPROMs has been studied, e.g., the Oral Esthetics Scale (OES) [36] the dimensionality of larger groups of dPROMs have not been investigated. Therefore, a direct comparison with similar studies is currently not possible.

However, previous studies which assigned names of dPROMs to the four OHRQoL dimensions are informative for interpretation of the current findings. A previous exploratory study, which investigated multi-item generic dPROMs, generated the research hypothesis for the present study [3]. In that study, the names for 53 generic dPROMs could all be assigned to the four OHRQoL dimensions. A second study which used the same methodology of relating dPROM names to the four dimensions through use of a different set of questionnaires yielded identical results [4]. In this study, all specific dPROMs were collected, i.e., instruments designed to measure impact for a specific oral disease and all (N = 102) disease-specific dPROMs could also be linked to the OHRQoL dimensions. A study investigating PROMs for pediatric dental patients came to similar conclusions [37]. In this systematic review, authors concluded the twelve pediatric dPROMs assessed included items that could be mapped to the four OHRQoL dimensions [37].

From these studies it can be concluded that, although oral disease-specific and generic dPROMs may target a different set of oral diseases in different dental patient populations like adult and pediatric, there is no difference in what they have in common as measurement targets—the four OHRQoL dimensions.

Because dPROMs capture what matters to patients, their common measurement targets should be related to the reasons why patients seek treatment for their oral health problems. An international study conducted in all six World Health Organization health regions corroborated this [37]. This study found that Oral Function, Orofacial Pain, Orofacial Appearance, and Psychosocial Impact capture dental patients’ perceived impact of oral health problems worldwide, regardless of whether the patient currently suffers from oral diseases or intends to prevent them in the future [3]. While these findings applied to all dental patients, a subsequent analysis restricted to pediatric dental patients confirmed these results [37].

Although OHRQoL is only one among many concepts in the larger group of dPROs, it is the most important concept [1]. Therefore, comparing our dimensionality findings for dPROMs with those of OHRQoL instrument could provide deeper insight into dPRO dimensions. Globally, the OHIP is the most widely applied OHRQoL instrument [5]. As the most extensive OHRQoL instrument and being a special case of a dPROM, therefore it should measure the same four dimensions. Indeed, OHIP’s four dimensions Oral Function, Orofacial Pain, Orofacial Appearance, and Psychosocial Impact were studied with several methodological approaches and all agreed on OHIP’s four dimensions [38].

Strengths and limitations

Methodological pros and cons of this study are centered around the basic components of the Delphi study—the expert panel, the investigated dPROMs, and methods used to achieve and demonstrate consensus.

In this present study, a small convenience sample of dental experts was selected—an approach typically adopted in Delphi studies. The participating dental experts may not represent all dentists worldwide.

Although specific dPROMs that deal with the specific aspects of oral diseases also exist, this study focused on the generic dPROMs because, conceptually, they are applicable to all oral diseases. As we studied dPROMs for adults, results may only apply to this age group. Nonetheless, the results may be applicable to older children because, the dPROMs for these children measure the same major attributes as determine by adult dPROMs.

To assess whether consensus on the commonalities among the dPROMs and OHRQoL dimensions was achieved—a necessary condition to have interpretable results—we used a method that is often applied to investigate the dimensionality of PROMs [39]. This study found strong evidence that dental experts’ ratings on the commonalities of dPROMs were unidimensional, indicating that a consensus among dental experts was indeed achieved and that the Delphi study results are interpretable.

Conclusion

Oral Function, Orofacial Pain, Orofacial Appearance, and Psychosocial Impact were solidly identified as the commonalities among generic dPROMs, supporting them as the building blocks of the dental patient’s oral health experience. These findings pave the way towards a four-dimensional metric to assess oral disease impact that is essential for pragmatic implementation oral disease outcome measurement to achieve patient-centered and evidence-based clinical decision-making.

3 Mar 2022

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Kelvin Ian Afrashtehfar, M.Sc., D.D.S.,Dr. med. dent., FRCDC

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf.

2. Thank you for stating the following in the Acknowledgments Section of your manuscript:

[We thank Drs. Akanksha Gupta, Christian Hirsch, Daniel Reissmann, Hina Mittal, Ira Sierwald, Katrin Bekes, Oliver Schierz, Radwah Sobieh, Subha Giri, Thiago Nascimento for their help with the study. We thank Dr. Rener-Sitar for her comments to the manuscript draft.

The research reported in this publication was supported by the National Institute of Dental and Craniofacial Research of the National Institutes of Health, USA, under the Award Numbers R01DE022331 and R01DE028059.]

We note that you have provided funding information that is not currently declared in your Funding Statement. However, funding information should not appear in the Acknowledgments section or other areas of your manuscript. We will only publish funding information present in the Funding Statement section of the online submission form.

Please remove any funding-related text from the manuscript and let us know how you would like to update your Funding Statement. Currently, your Funding Statement reads as follows:

[The research reported in this publication was supported by the National Institute of Dental and Craniofacial Research of the National Institutes of Health, USA, under the Award Numbers R01DE022331 and R01DE028059.]

Please include your amended statements within your cover letter; we will change the online submission form on your behalf.

3. Thank you for stating the following in your Competing Interests section:

[NO authors have competing interests.]

Please complete your Competing Interests on the online submission form to state any Competing Interests. If you have no competing interests, please state "The authors have declared that no competing interests exist.", as detailed online in our guide for authors at http://journals.plos.org/plosone/s/submit-now

This information should be included in your cover letter; we will change the online submission form on your behalf

Additional Editor Comments:

Dear Authors,

After appraising the authors’ manuscript in great detail with the effective assistance of 9 reviewers, it is my pleasure to inform you that their manuscript has been granted “ major corrections" status.

Thus, please address our suggestions and provide the revised version at your earliest convenience for a 2nd round of revisions.

Some additional literature that might support your introduction or discussion sections follows: Br Dent J. 2022 Feb;232(4):192. doi: 10.1038/s41415-022-4005-4; Evid Based Dent. 2016 Dec;17(4):109-110. doi: 10.1038/sj.ebd.6401202; Evid Based Dent. 2021 Dec;22(4):143-145. doi: 10.1038/s41432-021-0216-9; Int J Dent. 2020 Dec 29;2020:6621848. doi: 10.1155/2020/6621848.

Thank you for submitting such outstanding work to PlosOne.

Best Regards,

Academic Editor

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: No

Reviewer #4: Yes

Reviewer #5: Partly

Reviewer #6: Yes

Reviewer #7: Partly

Reviewer #8: Yes

Reviewer #9: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

Reviewer #4: Yes

Reviewer #5: I Don't Know

Reviewer #6: Yes

Reviewer #7: Yes

Reviewer #8: Yes

Reviewer #9: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: No

Reviewer #3: Yes

Reviewer #4: Yes

Reviewer #5: No

Reviewer #6: Yes

Reviewer #7: Yes

Reviewer #8: Yes

Reviewer #9: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

Reviewer #4: No

Reviewer #5: Yes

Reviewer #6: Yes

Reviewer #7: No

Reviewer #8: Yes

Reviewer #9: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: This research is very interesting, in that it focuses on the quality of life. To often, we think that elimination of disease is what is important and tend to ignore the other aspects that make up good oral health. I just have a few comments in regards to the methodology. Firstly, it says 11 international dentists participated, so were only 11 dentists approached? Or were there more dentists who were approached and only 11 participated, and if so please indicate the response rate?. Secondly, it would be helpful if more information about these 11 dentists could be provided, such as their highest level of education, years of work experience ( mean - work experience of study group), full time or part time employed and where. Just to provide much more clarity.

Strengths and limitations need to be explained much more clearly. For instance, the fact that convenience sampling was employed and one of the author recruited the participants, instead of random sampling, there could be biasness in the responses.

Reviewer #2: What was the criteria for selection of the inetrnatinal experts. what was the geographical distrubution of the experts. why is no mention of the details of the selection citeria for the expert no dissussed in the text.

Reviewer #3: Comment #!

The relevance of the study was not clear.

The authors mentioned as an important factor the Delphi process using a panel of international dental experts, however the nationality of the experts was not mentioned, also the sample size of experts were small.

The methods were not concise.

Comment #2

The Future directions and the impact of the study outcomes was not clear. Importantly, the potential of this study, regarding the proposition of a new validated, clear and optimized tool for dental patient-reported outcomes was not reported.

Reviewer #4: The term ‘international data’ has been used for OHIP-49 data (ref.2), as the countries where the 35 studies were conducted represented all continents. Is it the same reason for mentioning the raters as the international dental experts i.e., did they represent all continents??? Even if that is the case, one should be cautious about using the term ‘International’ due to the prevailing diversity in different countries and within a particular country as it is related to the generalizability of the study findings.

Abstract: If we say ‘A Delphi consensus process with 11 dental experts determined how well 20 of these instruments measured the four OHRQoL dimensions, we have to present in our results as to what score on a scale of 0 to 10 had different questionnaires for a particular dimension. Actually, the aim of the study had been ‘to investigate the commonalities among 53 generic multi-item dental patient-reported outcome measures (dPROMs) used in 20 questionnaires’.

Background: The number of dPROMS investigated in one of the two systematic reviews must be rechecked. Also, the reference number assigned to various references in the reference list, the relevancy of a reference to a topic in the text, the names of authors given in Table 1 have to be reviewed.

It is suggested that the authors should collectively and thoroughly revise the manuscript for clarity, brevity, and proper sequencing of sentences and paragraphs. They are requested not to consider the amendments (in green) proposed in the attached modified version of the manuscript, as final. Some sentences (in red) in the text need to be rephrased or explained further to enhance comprehension of the manuscript.

Reviewer #5: Reviewer: This manuscript can contribute to consolidation and standardization of available instruments patients’ oral health-related quality of life.

Please see below some comments:

1. Background, page 4: In the background all references used are from articles by only a researcher.

It would be important to consider studies by others researchers.

The authors used the acronyms: dPROs; dPROMs; OHRQoL; EBDP; VBOHC

We use acronyms because its use saves space and prevent repetition. But, if the reader is not familiar with the acronym and if a paper contains too many, that can be distracting and confusing in itself. Its use will likely detract from the readability of the paper.

Then, we should be prudent in their use of abbreviations. Avoid acronyms in the unless the acronym is used multiple times in the text.

Dental patient-reported outcomes (dPROs)

dental patient-reported outcome measures (dPROMs)

Oral Function, Orofacial Pain, Orofacial Appearance, and Psychosocial

Impact (OHRQoL)

evidence-based dental practice (EBDP)

value-based oral health care (VBOHC)

2. Material and Methods, page 5:

This sentence needs a reference: “They were provided the abstract published by the questionnaire author(s), a brief description from the author on what the questionnaire intends to measure, the reference, as well as the questionnaire items.”

Material and Methods, page 5:

In this sentence: “These dimensions were identified through a systematic collection of OHIP-49 data in typical dental patients and general population subjects of both genders covering an age range of 40 years or more.”

This acronym (OHIP) lacks the initial definition.

Material and Methods, page 05 and page 06:

Where the data may be found? “Study instructions of dental experts”, the survey and results.

3. Results: the authors could bring more graphic elements to illustrate the results.

4. Discussion:

According to the authors, direct comparison with similar studies is not possible. But the authors could bring similar studies from other areas of health. Furthermore, it is noted that 15 references are citations of an only researcher.

References:

References 10, 12, 20, 24 and 25: Check references. Be consistent with referencing Vancouver style across the document.

Reference 40:

Fink-Hafner D, Dagen T, Doušak M, Novak M, Hafner-Fink M. Delphi Method: Strengths and Weaknesses. Metod Zv [Internet]. 2019;16(2):1–19. Available from:

https://www.researchgate.net/publication/337570516

Reference 40: insert link to the original source of the article:

https://mz.mf.uni-lj.si/article/view/184/287

Reviewer #6: This manuscript applied the Delphi consensus method to study commonalities among 20 pre-selected patient-reported outcomes in the dental treatment category.

The setup description and data analysis methods are sufficiently presented in the text. The Discussion of the results and the cited literature is satisfactory to the claims of the manuscript.

Nonetheless, the methods and discussion can be improved and some important points need to be clarified. Additionally, while the study satisfies its stated hypothesis, it lacks on the delphi method details and its limitations are not sufficiently presented.

1. Better clarification is needed on why only 20 questioners were chosen from a previous meta analysis by the same authors, what were the inclusion/exclusion criteria? What constituted "a generic dPROM|"? wouldn't assorted specific dPROMs for the most common dental treatment options be better and offer more power to the sample?

2, as this article builds its sample from a previous meta-analysis, a better discussion should include the methodological issues and difficulties associated specific to metal-analysis of dPROs. There is a recent publication by YUN-CHENLIU et al. on this specific issue which I recommend citing here,.

3. The Delphi method limitations and biases need to be discussed in a better, more detailed manner, as one assumes it is the main focus of this paper. additionally, more details (in the methods and discussion) need to address the characteristics of the selected experts and the issues of selection bias and group consensus vs. Individual opinion.

4. the limitations of dPROMs and how do they compare/complement Disease-oriented outcome measurements deserves to be mentioned in the intro and the in the discussion.

5. I spotted some typos and grammatical errors in the manuscript (e.g. This study implementeds... in the intro) , please re-review.

Reviewer #7: The authors aimed to " to investigate the commonalities among 20 generic multi-item dental patient-reported outcome measures (dPROMs)" using the Delphi process. However I would like to humbly suggest some editions:

1-There are a few spelling, typos and inaccuracies within the text and the references, I would suggest to review the paper;

2-In the 4th line of background it inform 153 dProms, in methods it's says 53 dProms were evaluated, am not sure if this is correct;

3-Please describe the dimensions definitions

4- Reference 1 is now published, please update

5- I'm not sure this statement would be correct "the determination of treatment

efficacy of dental interventions would greatly improve..." the questionnaires are assessing the patients' point of view, not treatment efficacy, as per your first statement "Dental patient-reported outcomes (dPROs) represent what is important to dental patients", these tools will represent what is important to the patient which not necessarily will represent what the patient need and treatment efficacy.

6-The dentists chosen to participate in the study are described as dental experts, it would be nice to have more information about this, are they experts in qualitative studies?

7-Am not sure the conclusion is precise, as stated "Oral Function, Orofacial Pain, Orofacial Appearance, and Psychosocial Impact were solidly identified as the commonalities among generic dPROMs, supporting them as the building blocks of the dental patient’s oral health experience." The four dimensionalities were the only options for the dental experts assign each questionnaire, therefore obviously these commonalities would be identified.

Reviewer #8: The presented manuscript discloses pertinent results for dental research, especially, considering the current relevance of the patient-related outcomes. The paper is properly organized and presented. Some adjustments are suggested, as follows:

1) I believe there was a mistyping error of the word “implemented” in the background section phrase “This study implementeds the Delphi process…” (page 04).

2) I understand that the “Inclusion criteria were that they had to be a dentist, had to have practiced dentistry in the past year, and were fluent in the English language” (page 04). However, did you consider and/or collect any further information on the dentists’ expertise level? Considering the possible methodological impact of the expertise variation among the participants.

3) The Fleiss reference (page 07) should be checked. I consider it should be number 16 as per the references list: “The median ICCs were interpreted according to Fleiss(5).”

4) The median ratings used for the “Assignment of questionnaires to OHRQoL Dimensions” (page 09), are based on the rate of all the experts (eleven), or only on the seven ones that provided test/re-test data? Consider clarification, as it seems relevant, especially for the abstract section methods description.

5) When mentioning a systematic review on pediatric patients (page 16), consider verifying the reference number (3) and changing it for another (37).

6) Consider checking the phrase: “Because disease-specific instruments intend to characterize the very specific impact aspects of from a particular disease, they are conceptually less suited to study the overarching themes underlying all dPROMs.” (Page 18).

Reviewer #9: The manuscript deals with a relevant topic, considered one of the dimensions of the tripod of evidence-based practice, which is the patient's perspective regarding their own oral health. A consistent survey was carried out, identifying and testing the main dimensions in 20 existing instruments. This study integrates a line of research, with other works already published by the same group. The results presented are relevant, they bring a contribution in relation to the formulated research question and the possibility of adopting a single instrument. This would allow better comparability between different studies. On the topic of discussion, the authors point out that one of the weaknesses of the study lies in the intentional sample and criteria for choosing the experts to integrate the Delphi study conducted. I agree and this seems to me to be the main weakness of the study. Even so, I suggest that the authors mention if in the criteria for choosing the specialists they included aspects such as time since graduation and practice of the profession, if they act as specialists, if they are from different specialties, if they exercise the profession inserted in different socio-cultural and economic contexts. . Perhaps these are some issues that could eventually impact the results.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Dilan Arun Gohil

Reviewer #2: Yes: Pankaj Gupta

Reviewer #3: No

Reviewer #4: Yes: Haleem A.

Reviewer #5: No

Reviewer #6: No

Reviewer #7: No

Reviewer #8: No

Reviewer #9: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Submitted filename: PONE-D-22-02098-converted1.docx

Click here for additional data file.

19 Apr 2022

Thank you for taking time to review the manuscript. Please see response to reviewer document.

Submitted filename: Response to Reviewers.docx

Click here for additional data file.

9 May 2022

Commonalities among dental patient-reported outcomes (dPROs) – a Delphi consensus study

PONE-D-22-02098R1

Dear Dr. Chanthavisouk,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Kelvin I. Afrashtehfar, M.Sc., D.D.S.,Dr. med. dent., FRCDC

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Dear Respected Authors,

Your manuscript has been assessed by four reviewers this time and myself.

Fortunately enough, the process has gone smoothly without delays.

Congratulations on your acceptance.

I look forward to seeing more of your publications in PLoS One.

Regards,

The Academic Editor

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #4: All comments have been addressed

Reviewer #7: All comments have been addressed

Reviewer #8: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #4: Yes

Reviewer #7: Yes

Reviewer #8: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #4: Yes

Reviewer #7: Yes

Reviewer #8: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #4: Yes

Reviewer #7: Yes

Reviewer #8: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #4: No

Reviewer #7: Yes

Reviewer #8: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Authors have addressed the comments appropriately. The methodology and strengths/limitations are now more in depth. Overall, good interpretation of findings are there, and well understood.

Reviewer #4: Dear Authors

Thanks for addressing the major issues. However, the following should be looked into to rectify some minor errors.

Page 5: Data is plural as pointed out in my previos reviewed version of the manuscript.

Page 7: the statistical software Stata was used to anlyse the data, not to interpret.

Page 9: From this example, it became (not becomes) clear----you have been using past tense in this paragraph through out.

Please review thoroughly for such minor errors.

Reviewer #7: Dear authors, Thank you for taking time to review the manuscript and address the reviewers' comments/suggestions.

Reviewer #8: The authors have answered and made appropriate alterations to improve the clarity of their manuscript.

For this, I recommend its publication.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Dilan Arun Gohil

Reviewer #4: Yes: ABDUL HALEEM

Reviewer #7: No

Reviewer #8: No

13 Jun 2022

PONE-D-22-02098R1

Commonalities among dental patient-reported outcomes (dPROs) – a Delphi consensus study

Dear Dr. Chanthavisouk:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Kelvin I. Afrashtehfar

Academic Editor

PLOS ONE