A Method for Designing a Patient Burden Questionnaire in Dermatology.

INTRODUCTION: In recent years, the concept of "disease burden" has been given a central role in evaluating patient care, particularly in skin diseases. Measuring patient-reported outcomes (PRO) such as symptoms and disease burden may be useful. AIM: To present a methodology that facilitates the development and validation of burden questionnaires for patients suffering from skin diseases.
METHODOLOGY: Based on past published burden questionnaires, a methodology for designing skin disease burden questionnaires was to be developed.
RESULTS: Based on 16 burden questionnaires developed and published over the last 10 years, the authors propose a standardized methodology for the easy design and validation of disease burden questionnaires for patients with chronic skin diseases. The authors provide detailed guidance for the conception, development and validation of the questionnaires, including reliability, internal consistency, external validity, cognitive debriefing, testing-retesting, translation and cross-cultural adaptation, as well as for statistical analysis.
CONCLUSION: The proposed methodology enhances the design and validation of disease burden questionnaires in dermatology. Burden questionnaires may be used in clinical research as well as in daily clinical practice.

Introduction

The “disease burden” concept is central for evaluating patient care, particularly so in skin diseases.1 It was introduced in 2010 and has since been proven to be particularly useful for quantifying population health and determining action priorities in the public health field.2 It focuses on the “individual disease burden”, deemed to assess disease “disability” in its broadest sense, including psychological, social, economic and physical features thus differing from Quality of Life issues covering five dimensions: physical wellbeing, material wellbeing, social wellbeing, emotional wellbeing, and development and activity.3,4

Certain chronic diseases require specific care that can onerous, costly and modify daily life.5 Thus, they can exclude the patients from their social and/or family environment. Moreover, other collateral effects, such as reduced working hours, loss of employment in more serious cases, and relationship and parental crises, frequently may not be considered by the patient as impacting their lives. Environmental factors may also have an impact on symptoms, including pain and the occurrence of complications such as photosensitivity, deficiencies, the cancer risk and infections.6–12

Individual disease burden has already been evaluated in several skin diseases using patient questionnaires such as topic dermatitis, vitiligo hand eczema, haemangioma, inherited ichthyosis, sensitive skin and very recently psoriasis.13–17

Even though it has become the main focus in the management of dermatological diseases, no methodology allowing the design of patient-oriented burden questionnaires in dermatology has been proposed so far.

The aim of the present article is to provide guidance regarding the design of disease-related burden questionnaires for patients with skin diseases.

Methodology

The authors provide a detailed step-by-step methodology for building a patient-reported burden questionnaire, based on the authors’ recommendations for designing self-administered burden questionnaires described in articles published over the last 10 years for atopic dermatitis, inherited ichthyosis, haemangioma, vitiligo, epidermolysis bullosa, albinism, sensitive skin, psoriasis and hand eczema.14–30

The methodology follows that proposed by Seidenberg et al and Leidy et al for the building of a multiple ability self-report questionnaire and of health-related quality of life claims.31,32

Even though this method was developed prior to the publication of the COSMIN risk of bias checklist in 2018, items were verified in posteriori against this checklist.33

Results

Any patient self-completion questionnaire must allow the target patient or care-giver-population to easily understand and answer all of its questions.

Ambiguous wording or complicated phrasing should be avoided and the number of questions should be limited to a reasonable number. The length of questions and their number should correspond to the target population; ie questionnaires and questions might be shorter for children or elderly patients than for adults or care-givers.

To ensure rigour throughout the development process, questionnaires should be designed by a multidisciplinary team, consisting of experts in the drafting and development of questionnaires, health care professionals (specialised physicians, general practitioners, nurses and patient associations), all having expertise in the management of patients suffering from the target pathology.

The design of burden questionnaires follows three steps: conception, development and validation (Figure 1).

Figure 1

Flow chart for building a burden questionnaire.

Conceptual Phase

The conception of a verbatim is based on an extensive literature review and semi-structured face-to-face interviews with health care professionals in a first step, and with patients suffering from the target pathology in a second step.

Once the principal topics to be addressed have been identified, the proposed list should be confirmed by a target patient group.

This target patient group should be homogenous and recruited primarily through specialists, GPs, pharmacists or patient associations.

This confirmation identifies the main concerns raised by the patients, such as impact on work in general, impact on day-to-day work, stress, daily life, every day care, relationships with others, isolation, perception of other people, and the economic impact of treatment (direct and indirect impact-managing consequences).

Based on confirmed topics, a first questionnaire is designed by the working group, allowing the conversion of individual comments or complaints into simple and direct questions. The working group may meet several times in order to simplify the questionnaire and to check for duplications.

Should no consensus be reached on a given question, then both versions should be listed; the most suitable question should be confirmed during development.

Development Phase

During the first part of the development phase, questions are developed based on items identified in the conception step leading to an almost finalised questionnaire content. The questionnaire’s responses are determined using a Likert scale including six responses (“never”, “rarely”, ‘sometimes’, “often”, “very often”, “always”).34 Even though there are other means, a 6-point Likert scale may be used, as this tool is easy to use and, based on our experience, the best adapted. Likert scales are frequently used in self-completion questionnaires and this response method is commonly identified by working groups as being the most pertinent method for the questionnaire in development.20,26,35,36 In order to limit missing data, an additional “not applicable” response should be added to the first six possible responses. Reliability of an evaluation scale increases with the number of response categories. Therefore, Guilford and Sellitz et al suggested standardising responses, thereby reducing the risk of distorted scores.37,38

During the second step, the questionnaire should be tested on a sample of target participants (patients or care givers). Care should be taken to avoid too large population diversity as this may lead to an alteration or weakening of the questionnaire’s sensitivity risks. Results may result in the replacement, deletion or merging of several questions into one single question. This may be due to the format of the question (too close or considered a complementary question). Moreover, items having too much or insufficient data should be rejected. The threshold value determining this rejection is determined based on the results of the analysis of the different items. Items for which the percentage of data is missing or “Not applicable” is too high should be removed; this should also apply for misunderstood or badly perceived items. Moreover, items correlating highly or too low with other items (>0.7 or 0.8 and<0.4, Pearson, Spearman, polychoric coefficient) should be rejected as well. The potential risk from retaining highly correlated items is that they may lead to the creation of dimensions uniquely based on these duplications and not on the real dimensions of the tool’s underlying structure. The risk from retaining items with low correlation is keeping those that do not measure the same underlying concept and would not allow for the factor analysis.

The assessment of the distribution of responses allows for the rejection of all items presenting with limitations or which result in non-adapted responses. The role of the working group during this second step is essential in order to ensure that significant items are not rejected or excluded or that there is a potential discrimination of conceptual aspects. For example, an item that is of interest for daily care may not be pertinent at the time the questionnaire is created. However, it may become pertinent once the treatment or management has been made available. Similarly, an item of interest for professional activities may not be pertinent to senior participants, but be applicable to others.

Factor analysis is undoubtedly the oldest method still used in psychometrics to confirm connections between factors and latent properties. It was introduced at the beginning of the 20th century by Spearman.39

Factor analysis offers two different techniques: principle component analysis and correspondence analysis.

When designing questionnaires, the principle component analysis technique is directed by the decision to use variables that are modelled as continuous with a Likert-type score, ranging from “not at all” to “completely”, with the possibility of intermediate responses and an additional category for “not applicable” to avoid non-responses as a result of hesitation or discomfort.

The principal component factor analysis is to simplify variables obtained from the transcripts of patient interviews by organizing them into groups. Multiple variables may measure the same one-dimensional construct (whether in reality or as a hypothesis suggested by experts). For example, “self-image”, “withdrawal”, or “relationship with others “, or that the same concept has multiple dimensions (eg the economic aspect would include work absenteeism and non-reimbursed medical and non-medical fees).

Thus, the exploratory factor analysis allows different items that measure the same “effect” to be grouped together, providing a method for summarizing these items into a synthetic score. Prior to an exploratory factor analysis, a Kaiser-Meyer-Olkin (KMO) analysis should be performed. This is to ensure that data are appropriate for the factor analysis. Therefore, all questions presenting a KMO of less than 0.5 should be excluded. Following this analysis, a scree plot is to be used in order to determine the number of dimensions (number of groups) in which each variable will be gathered.

Following this, an exploratory factor analysis will be performed. Once construct validity has been completed, reliability should be assessed through internal consistency checking and test–retesting.

To ensure internal reliability, the correlation of items to factors should be of 0.5 or above and item cross-correlations should not exceed 0.2. It is important to ensure that the questionnaire really measures what has been agreed to be measured.

Internal consistency

The burden questionnaire is designed using multiple questions that are all different, or at least appear to be.

Therefore, analysing each question, the relationship between all questions, as well as the relationship between the questions and the overall questionnaire score, is essential.

This requires testing in order to confirm the consistency of the tool and that each of the components responds coherently to any given answer. For a given individual, the answers to questions such as “I feel lonely”, “I am isolated,” “I have no social relations” should not be radically different, even though they may be nuanced to achieve a level of sensitivity which is capable of recognizing a difference in the experiences of two different subjects.

Thus, internal validation corresponds to a confirmation that the questionnaire is capable of measuring a given construct in a coherent fashion. The Cronbach alpha fulfils this role.40 It provides a synthetic calculation of the average correlation between responses to different questions within a questionnaire, and of the average correlation between responses to questions that have been identified as measuring the same dimension. It provides a score between 0 and 1 with 1 corresponding to perfect consistency between questions. The internal validity of a questionnaire is considered to be adequate if the Cronbach’s alpha is 0.7 or higher. Conversely, a Cronbach’s alpha above 0.95 may be considered as suspect, and, in such a case, questioning the reasons behind this excessive validity may be adequate.41

A confirmatory factor analysis with a higher order factor should be performed. This is to ensure that subscales obtained using the exploratory factor analysis are appropriate and that items can be summed as an overall measure. Three values should be presented: root-mean-square error of approximation (RMSEA), Goodness-of-fit index (GFI) and Benley’s comparative fit index (CFI). GFI and CFI should be superior to 0.9 and at least 0.8, RMSEA should be inferior to 0.5. RMSEA can be highly dependent on the number of items, showing a tendency to be higher when the number of items is low.

An item response theory should be performed to ensure that Likert scales are correctly ordered and that each level of every scale corresponds to its own level of difficulty.

Cognitive debriefing

Cognitive debriefing (CD) removes potential ambiguity leading to a question that could easily be misunderstood. It ensures that each question is understood in the same way by participants, regardless of their socio-cultural status.

CD may be performed according to recommendations made by Collins and McColl et al.42,43

CD should be applied to each newly designed questionnaire. Once CD has been completed, the final questionnaire should be tested on a sample of the native speaking population during a personal CD interview, in order to determine issues related to the wording of the questions and responses, such as ambiguity, misunderstanding, acceptability, etc. This testing should be managed by specialised institutions.

A CD produced by specialised institutions allows us to verify that the participant understands the questionnaire and to identify and resolve potential problems, such as the translation of a question that may be confusing or hard to understand.44 The goal is not to change the sense of the original question, but to express it clearly in the language into which it is being translated. A native examiner with a strong background in cognitive interviewing techniques should conduct each interview. Respondents should be sufficiently representative of those for whom the questionnaire has been designed and should be natives of the questionnaire’s language. This simple-to-use and easy-to-understand questionnaire should be considered the final version to be validated.

Verification should be performed according to the principles of good practice for the translation and cultural adaptation process for patient-reported outcome measures.45

Validation Phase

External validity

External validity measures the degree of certainty that the questionnaire achieves when measuring a given construct.

Several types of validation may be performed to establish the claim of external validity for a questionnaire. Nomological validity should be preferred. Indeed, nomological validity supports the claim of external validity for a questionnaire, and also provides a certain amount of additional, relevant information regarding the context and the environment of the subject.

The external structure’s validity is based on correlations of the tool, with one or more tools that have already been validated and for which pertinence has already been confirmed. The choice of questionnaire should be considered thoroughly. It is important that the questionnaire is validated and that it examines the agreed concepts with the specifically created tool.

External validity of the questionnaire should be studied by analysing its correlation with a specific questionnaire and with non-specific questionnaires such as the Short-Form health questionnaire-12 (SF-12), using the Pearson correlation.46 Correlation should range between 0.6 and 0.9. A correlation exceeding 0.9 may indicate that the questionnaire does not differ from the reference questionnaire.

Another common good practice is to ascertain the correlation of the questionnaire with variables known for their clinical impact on the patient’s burden, such as severity or specific localisation related to the disease’s burden such as the face or the extremities.

Test–Retest

Test–retesting should be performed by distributing the same questionnaire to the same participant sample at two different times, to confirm that there are no significant changes between the two measures. The time interval between the original test and the retest should be short.

Reproducibility studies should be conducted to confirm the stability of scores in stable (state of health and treatment) subjects between two administrations. The calculation of the intra-class correlation coefficients (ICC) of responses given by the same individual qualified as stable at two distinct moments evaluates the Test–Retest stability.

Coefficients of at least 0.40 are considered correlated regardless of the chosen method. Results should help to understand how different items come together to form the dimensions. Different rotations (eg Varinax and Pronax) may be performed in order to allow determining factors. During this stage, items that do not belong to any dimension will be removed. Once these items are excluded, exploratory techniques should be applied once again and some items may be removed to produce a stable structure. An intra-class coefficient (ICC) of ≥0.7 may be considered satisfactory.

Translation and cross-cultural adaptation

The Principles of Good Practice for the Translation and Cultural Adaptation Process for Patient-Reported Outcomes should be followed to generate versions in different languages and to account for cross-cultural validation.45

For each language, linguistic and cross-cultural validation should be conducted. This process aims to refine the translation, considering the nuances of the original version. Several changes can be implemented throughout the validation process, without modifying the content. The aim is to improve the first idiomatic rendering.

Linguistic and cross-cultural validation steps are summarized in Table 1.

Table 1

Linguistic and Cross-Cultural Validation45

Steps	Details
1. Preparation	Evaluation of the source text from a linguistic and cultural point of view including definition of concepts
2. Forward translation	Forward translation into required target language by two independent translators
3. Reconciliation	Comparison of the tow forward translations to provide the best adaptation and produce a draft version of the text
4. Back translation	Translation of the draft forward translation back into the targeted language without reference to the original language
5. Bac translation review	Comparison of the original text and the back translation to verify that the meaning of the draft translation is equivalent to source
6. Analysis and implementation of back-translation review report	Analysis of the back-translation review report to verify if there are changes required to the draft forward
7. Pilot testing	Clinical review and cognitive debriefing
8. Review of cognitive debriefing or clinical review results	Review of the results from the cognitive debriefing or clinical review to identify translation modifications necessary for improvement
9. Proofreading and finalisation	Last stage, which aims at a cross-cultural and validated translation of the questionnaire

Discussion

Over the last decade, PROs have become more and more important to assess the impact of diseases on the patient’s Quality of Life, the mental health status and self-esteem and on daily and professional life, especially in visible, chronic or hereditary diseases.47–50 This is because chronic or hereditary skin diseases, impact on appearance, add a psychological burden to the physical burden, thus occupying an important part in the patient’s daily life.51–55

Although, until today, no standardized method for designing a patient-reported outcome (PRO) questionnaire for patients with skin diseases has been proposed, several studies have assessed in the past the burden of specific skin diseases in patients.14–29,56

The methodology proposed herewith allows to design PRO questionnaires using a validated and easy-to-follow method. Building such questionnaires requires the involvement of different specialists, especially dermatologists, statisticians, institutions and professionals specialized in the testing of questionnaires and the recruitment of potentially large panels of participants, including patients. Moreover, several conceptual and internal, as well as external, validation steps are mandatory for making a questionnaire become a powerful and recognized tool for assessing PROs.

Furthermore, burden scores should be evaluated for responsiveness and sensitivity to change their minimal clinically important difference (MCID) which can be defined as the smallest difference in score which patients perceive beneficial.57 Ideally, the score should be used in clinical trials or clinical setting and should correlate with improvement after treatment. Although this step is not always performed, it is essential. Another important step is defining the cut-off for severity allowing translating any patient-reported outcome scoring into severity. This can be done using anchorage methods with a patient global severity score and might help for patient selection when conducting a clinical trial.

One major limitation may be the fact that we did not follow COSMIN guidelines and did not use the latest version of the COSMIN checklist to validate our approach. Indeed, the presently proposed method was developed based on PRO questionnaires in dermatology developed and published prior to COSMIN recommendations and the COSMIN checklist. Thus, we were unable to respect guidelines proposed the time of its creation. However, we checked our method using this checklist allowing to validate the proposed methodology.

To our knowledge, this article provides the first validated methodology for designing a patient burden questionnaire in dermatology, based on published questionnaires used to assess the burden of patients suffering from different skin diseases and the respective burden of their care givers.