Patient reported outcomes in the assessment of premature ejaculation.

The term 'Patient Reported Outcome', abbreviated as PRO, was introduced by the US Food and Drug Administration (FDA) which proposed guidance on the development and validation of PROs. Previously PROs were known as self-report diaries, event-logs, self-administered questionnaires, and clinician administered rating scales. PROs seek to capture the subjective perceptions of patients and/or partner's related to their specific symptoms, degree of bother, efficacy of a medication or psychotherapy intervention, and quality of life issues related to a specific condition. This article reviews the essential psychometric and regulatory agency requirements in the development of PROs. The constructs of reliability, various forms of validity, sensitivity, and specificity as well as concerns with translating a PRO into a different language are reviewed. Three PROs, the Premature Ejaculation Profile (PEP), the Index of Premature Ejaculation (IPE) and the Premature Ejaculation Diagnostic Tool (PEDT) all used in the assessment of premature ejaculation (PE) are discussed. These questionnaires meet or exceed all the psychometric requirements and have been employed in clinical trials and observational studies of men with PE. The article concludes on discussing some of the limitations of PRO use and recommendations for the future.

Introduction

The term ‘Patient Reported Outcome’, abbreviated as PRO, was introduced by the US Food and Drug Administration (FDA) which proposed guidance on the development and validation of PROs (1). Previously PROs were known as self-report diaries, event-logs, self-administered questionnaires and clinician administered rating scales. PROs seek to capture the subjective perceptions of patients and/or partners related to their specific symptoms, degree of bother, efficacy of a medication or psychotherapy intervention, and quality of life issues related to a specific condition. Research in premature ejaculation (PE) obtains both objective data such as Intravaginal Ejaculatory Latency Time (IELT) gathered by stopwatch assessment as well as subject’s responses on PROs assessing their PE symptoms, degree of bother and other relevant concerns.

This manuscript will specifically focus on PROs used to diagnose and detect treatment benefits in patients with PE.

Key psychometric concepts

Prior to launching into a discussion of the PE PROs themselves, it is necessary to define and clarify the psychometric or measurement properties that are essential in constructing PROs. In order to be considered psychometrically valid, PROs must possess the essential characteristics of reliability, validity, sensitivity, and specificity. The validation process is an iterative, ever evolving process that increases clinicians’ and regulatory agencies confidence in the quality of the data.

There are two forms of reliability—test-retest and internal consistency. Test/retest reliability refers to the ability of the questionnaire to measure the same phenomenon in a similar fashion at two or more points in time. For instance, a PE questionnaire is deemed reliable, if holding all conditions constant, a subject’s scores on specific constructs, like voluntary control, are similar from week to week. Internal consistency measures the degree to which all questions are measuring the same phenomenon or construct (i.e., how all the questions in a distress subscale are related to one another).

There are multiple types of validity that contribute to the overarching concept of validity. These include: face, construct, discriminant, known-groups and predictive validity.

Face validity is the extent to which a measure is subjectively viewed as covering the construct it purports to measure. It refers to the transparency or relevance of the PRO as it is perceived by subjects responding to it. Simply stated a measure is said to have face validity if it appears to be assessing the construct under consideration (e.g., sexual satisfaction) (2,3).

Construct validity is composed of three aspects consisting of discriminant (known groups or convergent/discriminant), predictive (response to treatment) and content (clarity, relevance, construct comprehensiveness) validity (Derogatis L. Measurement of outcomes with PRO’s. personal communication, 2014). The FDA places great value on content validity in approving PROs to be used in clinical trials. They recommend qualitative research with focus groups be conducted to determine if all the relevant constructs pertinent to what is being assessed have been captured as well as the subjects understanding of each question and response choice. Questions and response sets are developed based on the results of focus groups. An initial PRO is constructed knowing in advance that many of the questions will not be suitable to be brought forward. The measure is administered to subjects diagnosed with a specific disorder (i.e., PE) and subjected to factor analysis to determine the domain structure of the questionnaire and the relationship of specific questions to the domain(s) and other questions. Questions are removed from the draft PRO based on redundancy, poor psychometric performance, failing to fit into the factor structure and subjects not understanding the intent of the question.

After revision the new measure is given to known groups such as men diagnosed as suffering from PE and those without PE. To assess convergent and divergent validity, the subjects also complete questionnaires that evaluate similar PE symptoms (i.e., Golombok-Rust Inventory of Sexual Satisfaction) and questionnaires that are unrelated to the focus of the PRO (i.e., SF-36, a measure of health status). One would expect to see different scores from men with PE versus those without PE on the PE relevant scales. The percentage of cases that are correctly classified by the PRO is known as sensitivity (e.g., 95% of cases with rapid ejaculation are classified as having the dysfunction) while specificity refers to the percentage of non-cases that are correctly identified (e.g., 87% of non-rapid ejaculators are classified as not being dysfunctional).

The next phase of validation concerns predictive validation or the PROs ability to measure treatment efficacy. Responses of subjects given placebo versus an active medication are examined expecting the active medication group to demonstrate statistical and clinical significance.

After the initial psychometric work on reliability and validity further research can determine a PROs cutoff score, the point at which one would classify a person as suffering from PE. Cutoff scores are used in screeners such as the Premature Ejaculation Diagnostic Tool (PEDT) which classify men as having or not having PE.

Questionnaires developed in a specific language or cultural context cannot be assumed to be valid when translated to a different language or cultural setting. Specific procedures such as forward and back translation are used to assure that the intended meaning of a question appears in the translated item. Additionally, the translated questions are judged by focus group participants to confirm that all items have adequate content validity (i.e., have the same meaning and significance).

Premature ejaculation (PE) patient reported outcome (PROs)

Several PE questionnaires assessing lifelong and acquired subtypes have been described in the literature (4-8), although only a small number have undergone extensive psychometric testing and validation. Five validated questionnaires have been developed and published to date. Currently, there are two questionnaires that have extensive databases meeting most of the criteria for test development and validation—The Premature Ejaculation Profile (PEP) (6) and the Index of Premature Ejaculation (IPE) (4). A third brief diagnostic measure (PEDT) has also been developed, has a modest database and is available for clinical use (7,9). Two other measures—The Arabic Index of Premature Ejaculation and Chinese Index of Premature Ejaculation (5,8) have minimal validation or clinical trial data available. details these instruments in terms of number of questions, domains and psychometric properties.

Table 1

Characteristics of recommended patient reported outcomes for premature ejaculation

Name	No. of questions	Domain names	Reliability studies	Validity studies	Advantages	Limitations
Premature Ejaculation Profile (PEP)	4	Perceived control over ejaculation; satisfaction with sexual intercourse; personal distress related to ejaculation; interpersonal difficulty related to ejaculation	Yes	Yes	Assesses outcome; brief, easy to administer; evaluates the subjective and clinically relevant component domains	Lack of validated cutoff scores; only one question per domain; not based on DSM-5 PE definition
Index of Premature Ejaculation (IPE)	10	Control; sexual satisfaction; distress	Yes	Yes	Assesses outcome; relatively brief and easy to administer. Evaluates the subjective and clinically relevant domains	Lacks norms and diagnostic cutoffs; not based on DSM-5 PE definition
Premature Ejaculation Diagnostic Tool (PEDT)	5	None	Yes	Yes	Screening questionnaire with cutoff scores; brief and easy to administer	Not based on DSM-5 PE definition

There are no validated PROs for assessing acquired PE or the provisional subtypes of subjective or variable PE. Additionally, all of the validated PROs ask about penile vaginal intercourse and are therefore not appropriate for use with gay men. There remains a need to fill these voids and develop appropriate endpoints for these populations of men.

Premature Ejaculation Profile (PEP)

The PEP is a 4-question PRO that asks a respondent about his subjective sense of control over ejaculation, distress related to PE, interpersonal difficulty and satisfaction with sexual intercourse. Each question is answered on a 5-point Likert-type scale and an index score is derived by averaging the responses to the 4 questions. The PEP has been extensively employed in the Dapoxetine and PSD-502 clinical trials as well as in observational studies (10-14).

The PEP has good test-retest reliability and known groups’ validity. One of the limitations of the PEP is the reliance on a single question to represent a domain. Does that one question sufficiently cover all the man’s concerns with that specific domain (i.e., control)? Another concern regarding the PEP is that the original validation of the PEP was based on the DSM-IV-TR PE criterion (15). Although there is not a time dimension in the DSM-IV-TR, the authors defined a man as a premature ejaculator if his IELT was 2 min or less. The current DSM-5 includes an IELT time criterion of approximately one minute. It is likely that the PEP requires a revalidation if the PE group is defined by DSM-5 criterion.

The Patient Outcome for Premature Ejaculation (POPE) is a recent unpublished revision of the PEP. The POPE modified the wording of the distress question; the three other questions remain the same (Disbrow J, personal communication, 2015).

Index of Premature Ejaculation (IPE)

The 10-item IPE was developed as a measure to evaluate sexual satisfaction, control and distress in men with PE (4). Like the PEP, the initial validation of the IPE used men with a stopwatch assessed IELT of 2 min or less. Subsequently the ISSM proposed, and DSM-5 accepted, a new more specific time dimension for diagnosing PE limited to an IELT of approximately 1 min. A second validation effort was undertaken using only men who met the 1 min or less IELT condition. The IPE demonstrated the same domain structure as the initial validation. Reliability was good for both internal consistency and test-retest reliability. Convergent validity using IELT as the standard was excellent: control domain (0.75), sexual satisfaction domain (0.60), and distress domain (0.68). Known-groups validity was adequate, all domain mean scores were statistically significantly worse in men with PE compared with the men reporting no PE problems (16).

Each of the three IPE domains contains several questions thus assessing a broader expanse than the single item domains of the PEP. The IPE has also been used in clinical trials for PSD-502 (10,11).

Premature Ejaculation Diagnostic Tool (PEDT)

The PEDT is a 5-item tool developed to systematically apply the Diagnostic and Statistical Manual, 4th Edition, Text Revision (15) criteria in diagnosing the presence or absence of PE. By employing a three tiered cutoff score it diagnosis PE (≤8), possible PE (9 or 10), and no PE (≥11). Possible PE calls for the clinician to conduct a further investigation to delineate the presence of absence of the dysfunction. The PEDT works best as a screener for PE rather than a measure of assessing the impact of an intervention. Because it contains only 5 questions it can be rapidly completed by patients and offers the clinician a valid assessment of his PE status.

Future work on the PEDT will require a re-validation with men who meet the DSM-5 criterion, rather than the older DSM-IV-TR standards. The PEDT has been used in several research studies to define groups of men suffering from PE.

Premature ejaculation (PE) patient reported outcome (PROs)—the future

The IPE and PEP have been used with both lifelong and acquired forms of PE using the DSM-IV-TR criterion. With the advent of the DSM-5 definition that contains the approximately 1 min IELT criterion for lifelong PE and the recommendation from the ISSM that the definition of acquired PE be changed to include a reduction in IELT of approximately 3 min (17), different measures, or revalidation of the existing measures are necessary to for populations of men diagnosed with these PE subtypes. Similarly, a PE PRO for use with a homosexual population is necessary so that gay men can be included in clinical trials and/or be properly diagnosed. Finally, it may be worthwhile to consider a measure that can categorize men with subjective and variable PE as well as lifelong and acquired forms of PE.

Limitations of patient reported outcome (PROs)

PROs should never substitute for an in-depth clinical assessment of a patient’s condition. While PROs provide useful and reliable information, only a face-to-face discussion allows for further elaboration of each man’s PE status and impacts. Partner perspectives are also very helpful in planning PE interventions and no questionnaire reliably assesses their perspective regarding the man’s PE.

Conclusions

PROs are a useful adjunct to diagnose and detect treatment benefits in men with PE. This article reviewed the important psychometric features of PROs that insures that the measure will be consistent and accurate. The PEP, IPE and PEDT were reviewed with recommendations made for future revalidation of these measures based on the new DSM-5 definition as well as the ISSM recommended definition for acquired PE and the inclusion of multiple subtypes of PE.