A new measure of patient satisfaction with ocular hypotensive medications: the Treatment Satisfaction Survey for Intraocular Pressure (TSS-IOP).

PURPOSE: To validate the treatment-specific Treatment Satisfaction Survey for Intraocular Pressure (TSS-IOP).
METHODS: Item content was developed by 4 heterogeneous patient focus groups (n = 32). Instrument validation involved 250 patients on ocular hypotensive medications recruited from ophthalmology practices in the Southern USA. Participants responded to demographic and test questions during a clinic visit. Standard psychometric analyses were performed on the resulting data. SAMPLE: Of the 412 patients screened, 253 consented to participate, and 250 provided complete datasets. The sample included 44% male (n = 109), 44% Black (n = 109) and 57% brown eyed (n = 142) participants, with a mean age of 64.6 years (SD 13.1) and a history of elevated IOP for an average of 8.4 yrs (SD 7.8). A majority was receiving monotherapy (60%, n = 151).
RESULTS: A PC Factor analysis (w/ varimax rotation) of the 31 items yielded 5 factors (Eigenvalues > 1.0) explaining 70% of the total variance. Weaker and conceptually redundant items were removed and the remaining 15 items reanalyzed. The satisfaction factors were; Eye Irritation (EI; 4 items), Convenience of Use (CofU; 3 items), Ease of Use (EofU; 3 items), Hyperemia (HYP; 3 items), and Medication Effectiveness (EFF; 2 items). Chronbach's Alphas ranged from.80 to.86. Greater distributional skew was found for less common experiences (i.e., HYP & EI with 65% & 48.4% ceilings) than for more common experiences (i.e., EofU, CofU, EFF with 10.8%, 20.8% & 15.9% ceilings). TSS-IOP scales converged with conceptually related scales on a previously validated measure of treatment satisfaction, the TSQM (r =.36 to.77). Evidence of concurrent criterion-related validity was found. Patients' symptomatic ratings of eye irritation, hyperemia and difficulties using the medication correlated with satisfaction on these dimensions (r =.30-.56, all p <.001). Clinicians' ratings of IOP control, severity of side effects and problematic medication use correlated with patients' satisfaction scores on these dimensions (r =.13-.26, all p <.01).
CONCLUSIONS: This study provides initial evidence that the TSS-IOP is a reliable and valid measure, assessing patients' satisfaction with ocular hypotensive medications.

Background

Introduction

Around the world, Patient Reported Outcomes (PRO's) are becoming an increasingly important set of criteria with which to evaluate the adequacy of treatment outcomes [1-6]. The relative success or failure of medical treatments can, at least in part, be judged by inquiring about patients' perceptions of their treatment experiences and changes in the impact of illness on their daily lives. . PRO self-report assessments have been developed to assess patients' perceptions of the type and severity of symptoms, the functional impact of illness, utility and preference measures for treatment options, the impact of illness on health-related quality of life and well-being, and various types of patient/treatment satisfaction.

More specifically, patient satisfaction has been used as a way to include patients' perceptions and preferences when evaluating the success of both medical treatments and systems of healthcare delivery [7-10]. Moreover, an individuals' satisfaction has been shown to affect health-related decisions and treatment-related behaviors, which in turn impact the success of treatment outcomes and the costs associated with treatment failure [11,12]. Patients' satisfaction with services has been shown to predict treatment success, medical compliance, follow-through with treatment plans, and appropriate use of services [13-15]. In a similar way, satisfaction with medication predicts patients continuance on pharmaceutical treatment, correct medication use and compliance with medication regimens [16-19].

The adverse effect of low treatment satisfaction on medication compliance has been found to be particularly problematic among persons with chronic disease conditions [14,20]. It has been estimated that up to one half of patients with chronic and/or asymptomatic illness will make medication-related decisions without seeking medical advice, becoming 'non-adherent' to such an extent that they compromise the effectiveness of treatment and eventually place further utilization demand on broader systems of care [20]. In contrast, more acutely ill patients who perceive an immediate threat to their physical well-being may be more willing to tolerate short-term aggressive treatment regimens in hopes of restoring their former health. Primary open angle glaucoma is a disease where patient adherence to therapy is important since the disease is by and large chronic, asymptomatic and can lead to irreversible vision loss. Patient compliance with therapy is necessary for optimal long-term outcomes. The objective of this research is to validate a treatment-specific Treatment Satisfaction Survey for Intraocular Pressure (TSS-IOP).

Measurement Issues

Due to the central importance of consumer satisfaction to the success of both health care services and pharmaceutical products, conceptual advances in the field have lead to a proliferation of satisfaction measures across many disease states [21,22]. These measures can be grouped into those addressing patients' satisfaction with discrete aspects of medical treatments (treatment satisfaction) and those focusing on more systemic aspects of programmatic care [15,23-27]. As discussed in two articles describing the development of a general model of treatment satisfaction, patients' satisfaction with their medication (TS-M) can be thought of as a specific sub-dimension of treatment satisfaction (TS) [28,29]. In turn, TS is viewed as a subset of patient satisfaction (PS) that covers more general and systemic aspects of both medical treatments and interpersonal aspects of clinical care. Thus one may inquire about patients' satisfaction across different aspects of both interpersonal care and medical treatments or more specifically about satisfaction with medication.

Adding to the complexity of this hierarchical model of PS, TS, and TS-M, each class of instrumentation can be operationalized using measures that differ on a context-specific to context-general continuum. Borrowing from conceptual work in the field of Quality of Life [14], items and scales of TS-M measures can be thought of as existing on one of three levels of generality-specificity (see Table 1). Level 3 scales contain items that refer specifically to a particular set of circumstances and events related to a particular type of treatment or disease state (e.g., How satisfied or dissatisfied are you with the way in which medication X has relieved symptom Y associated with condition Z?). Such items and scales do not rely heavily on respondents' interpretation of an items' meaning due to the situational specificity of the content, thus both respondents and scale assessors can be fairly certain of what is being rated.

Table 1

Levels of Generality-Specificity of PRO Items and Scales

Levels of PRO Item and Scale Specificity	Content Specificity & Referential Certainty	Respondents' Inference or Interpretation of Meaning	Normative Index of Personal Relevance
Global: Level 1	None	High	High
General: Level 2	Domain Specific Only	Moderate	Moderate
Specific: Level 3	Domain & Event Specific	Low	Low

Level 2 scales are made up of more general items that assess a particular domain of treatment (e.g., effectiveness, side effects or convenience) but are not specific to a certain illness or type of medication. In response to Level 2 items, respondents interpret the meaning of the item from the vantage point of their particular experiences of treatment (e.g., How satisfied or dissatisfied are you with how well your medication has relieved the symptoms associated with your condition?). As a result, such items are appropriate for use with a wider range of patients with different illness and treatment conditions. The quality of these items depends heavily on the face validity of the perceived relevance of the content. The items are interpreted in such a way that they are understood to be personally relevant, and the resulting response is reflective of a salient aspect of ones' experiences. The wording of Level 2 items refers generally to the dimension being evaluated (e.g., Effectiveness, Side Effect, Convenience), not the particular disease-specific treatment experiences within each dimension.

Level 1 scales are the most general or global of all. In addition to being seen as relevant across many different types of patient populations, illness conditions, and treatment approaches within a domain of satisfaction, Level 1 items and scales elicit a global appraisal or judgment across numerous domains of measure (e.g., Taking all things into account, how satisfied or dissatisfied are you with your medication overall?). As one moves from Level 3 to Level 1, greater personal interpretation and judgment is implicitly required from respondents. They make contextual sense out of the more generally worded items using sets of personally relevant experiences which stand out in their mind. Moreover, these experiences have emotional relevance, which may explain the stronger correlations between emotional variables and Level 1 scales than Level 3 scales. In general, responses to more global scales have been shown to exhibit higher correlations with affective constructs than more specifically worded items [30]. An exception to this observation may occur if a specific item is relevant to the majority of a sample. It remains to be seen whether such emotive associations and general appraisals of satisfaction can be shown to predict behavioral variables as they have in other populations [31].

Conversely, the content specificity of Level 2 and particularly Level 3 items and scales is higher, and it is often easy from reading these items to be fairly certain of what respondents' are referring to when making ratings. As a result, Level 3 items are often viewed more favorably providing evidence to substantiate specific claims regarding particular treatment or aspects of care [32].

PRO Measurement in Glaucoma

The importance of patients' perceptions of both clinical and non-clinical factors affecting the outcomes of ophthalmology has lead to the development of various PRO measures for use with glaucoma patients. PRO instruments have been used to assess patients' perceptions of visual functioning [33-35], visual disabilities [36], visual symptoms [37], patient preference and treatment satisfaction [38], and Health-Related Quality of Life [39]. As one might expect, an inter-relationship has been shown between various types of PRO outcomes. For example, in addition to patients' reports of their visual function [40], TS-M (particularly the side effects domain) has been shown to affect patients' health-related quality of life scores [41].

Patient satisfaction measures have been used to assess glaucoma patients' experiences with surgical procedures [38,42], pharmaceutical interventions [43], and various aspects of service delivery [44]. To date, only one valid measure of TS-M for ocular hypotensive treatments exists, the Comparison of Ophthalmic Medication for Tolerability Questionnaire (COMTOL) [45]. However, this earlier questionnaire places a heavy emphasis on vision-related functional outcomes and does not adequately cover the side effects that became apparent with the emergence of prostaglandin treatments in 1996.

Glaucoma Treatments and Patient Experience

The reduction of intraocular pressure (IOP) in patients with glaucoma helps prevent the progression of the disease which may lead to visual loss and potential blindness. In addition, reduction in IOP is used to help prevent the progression of OH to glaucoma [46,47]. Unfortunately, the topical treatments for OH are often accompanied by significant side effects [48,49]. Similar to observations in other areas of medicine, the factors of cost, convenience and side effects of pharmacotherapies can influence a patient's lifestyle, quality of life, non-compliance with medication regimens, and ultimately, their clinical effectiveness [11-13,48,49].

In addition, multiple medications and multiple daily administrations may be a necessary inconvenience and, for a subset of patients with dexterity problems, present significant difficulties to its use. Easy to use delivery systems that permit accurate dosing of topical agents are important to minimize the wastage associated with missing the eye or instillation of multiple doses. The costs and inconveniences associated with such waste are a substantial concern for some patients.

Several clinical classes of medications are available to treat elevated IOP in patients with POAG and OH. A frequently administered class of medications is the prostaglandin analogs. These medications have the advantage of once daily dosing, are highly efficacious and have a low incidence of systemic side effects. The most common ocular side effects are conjunctival hyperemia and iris pigmentation changes [50-52]. Another commonly prescribed class of medicine is the topical beta-adrenergic blockers. These medicines are generally slightly less effective than prostaglandins and are dosed once or twice daily. The beta-adrenergic blockers may be associated with pronounced systemic side effects in some patients, including worsening of reactive airway disease and aggravation of cardiac conduction disease [46,53,54].

Topical carbonic anhydrase inhibitors (CAI) are available as monotherapies or as a fixed combination with timolol maleate, a beta-blocker, and may be dosed two to three times daily. The CAI medicines, although less effective than beta-blockers, provide an excellent systemic safety profile but are commonly associated with mild ocular burning and stinging upon instillation [55-57]. Brimonidine is a centrally acting alpha-agonist that is usually dosed two to three times daily and has similar efficacy to dorzolamide. Brimonidine may occasionally cause systemic side effects, such as blood pressure changes or neurological symptoms and may cause ocular intolerance in approximately 10%-26% of cases [57,58]. All the above agents are often dosed as un-fixed combinations that increase dosing complexity and the likelihood of adverse events.

Unfortunately, no existing measure of treatment satisfaction adequately assesses the subjective impact of ocular side effects and inconveniences associated with different IOP medications. In order to address this gap, the objective of this study was to design a measure of TS-M specifically to assess patients' satisfaction with various aspects of topical ophthalmic treatments within a sample of patients with glaucoma or ocular hypertension – the Treatment Satisfaction Questionnaire for Medications for Intraocular Pressure (TSS-IOP).

Study methods

This study occurred in two stages. The first portion was conducted to identify the item content for the new measure, based on information gleaned from a literature review and four focus groups consisting of patients receiving topical ophthalmic treatment to control IOP. This content was used to develop an initial pool of items that would be psychometrically tested in the second stage of the study. This second, larger psychometric study was used to select the final items to be included in the TSS-IOP and to examine the performance of the new scales.

Stage I: Patient Focus Groups Qualitative Research Methodology

The primary objective of the focus groups was to refine and finalize the content pool for the TSS-IOP test items. The methodological approach used to plan and conduct the patient focus groups was consistent with Goldman's group depth interview model [59], in which information is gathered from a number of interacting individuals who share a community of interests. These groups are facilitated using a trained moderator who employs a combination of probing as well as direct- and non-direct inquiry techniques.

Prior to implementing the focus groups, a discussion guide was developed to direct the collection of data. The guide consisted of nine sections: (1) orient participants to the purpose of the discussion, (2) guide patient introductions, (3) discuss satisfaction with ocular hypotensive medications, (4) identify determinants of medication satisfaction and dissatisfaction, (5) explore three targeted satisfaction domains (effectiveness, side effects, and convenience/ease-of-use/delivery method), (6) discuss compliance, (7) inquire about doctor visits and the continuum of care, (8) review a prototype TSS-IOP mockup, and (9) probe for final thoughts, including what, if any, additional domains could be added that might impact satisfaction/dissatisfaction. The focus group data were collected using the moderator's notes, notes taken by two observers seated behind the one-way mirror in the focus group facility, and via review of the session videotapes.

Focus Group Composition

Thirty-two patients with primary open-angle glaucoma (POAG) or ocular hypertension (OH) participated in one of four, 90-minute focus group sessions. These sessions were composed of a heterogeneous sample that represented a diversity of patient experiences with common ocular hypotensive medications used to treat OH and POAG. Focus group participants included: Those experiencing hyperemia associated with ophthalmic prostaglandin medications (PG) within the last 3–6 months; patients who were newly treated in the past 3–6 months; PG naive patients who received some form of topical therapy other than PG's; patients who had used medications that required multiple daily dosing (e.g., timolol, brimonidine) or used multiple types of ophthalmic medications daily; and patients using novel forms of medication delivery aids.

Twelve individuals reported problems with medication effectiveness and seven admitted that they did not always use their medication as prescribed. Fourteen participants reported specific problems with the side effects of their current medication, while 16 of patients reported having experienced at least one ocular side effect associated with their medication over time. Eight individuals were on two or more medications at the time and of these, two reported problems specifically associated with multiple medication use (primarily side effects). Ten participants indicated having at least some minor problems with respect to the medication they have used in the past.

Thematic Content of Focus Group Discussion

Information gathered within the four focus group sessions was tabulated by thematic content (see Table 2). These themes were used to develop thirty-one items for psychometric testing in the second stage of this study.

Table 2

Thematic Content Analysis of Factors Relevant to Patients' Satisfaction With Their Medication Use

Content Area	Prevailing Themes and Sub-Themes
Medication Effectiveness	• The eye pressure readings are the only way one can tell
	• Some report improvements in their vision, including:
	Ability to read (small print) without glasses
	Vision is clearer/not as blurred or cloudy
	Distance vision is clearer
	Able to see better at night
Unintended Medication Effects	• Burning, Itching, Grittiness/Sandiness, Dryness, Tearing of eyes
	• Redness of eye, Darkening of iris of eyes
	• Swelling, Crustiness, Stickiness of eyelids
	• Visual Changes (e.g., "clear ropes" in eyes, loss of center of vision, sensitivity to light)
	• Systemic affects associated with allergenic reaction or use of oral treatments: shortness of breath, restlessness/inability to sleep, excessive perspiration, low energy, migraines
Convenience and Ease of Medication Use	• Discomfort putting things in eyes
	• Strong "blink reflex" making it difficult to instill the drops
	• Difficulty learning to instill drops
	• Miss the eye when administering the medicine
	• Unable to feel whether a drop has gone into their eye
	• Inadvertently dispense more than one drop, or dispense just one more to be sure
	• Require assistance if elderly or physically impaired (e.g., have Parkinson's)
	• Trouble remembering to use the medicine, particularly on trips or vacations
	• Instillation twice a day, this is less convenient than once
	• Frustration with the daily dosing and, as a result, sometimes not taking their medicine
	• More inconvenient to administer evening than morning doses, sometimes too tired in evening
	• Delay taking medication in evening till returning home
	• Difficult to tell when their medicine is about to run out

Stage II: Validation Study

Study Methods

Participants in the validation study consisted of 250 patients who were consecutively recruited from participating Ophthalmology clinics in 5 different clinics in the Southeastern U.S. These patients had either open-angle glaucoma or ocular hypertension and were currently using marketed topical IOP-lowering medication(s) in at least one eye (as defined by AAO diagnostic codes). In order for patients to be included they were also required to meet the following inclusion-exclusion criteria: Be 18 years of age or older; willing to comply with the investigator's and protocol's instructions; consent to participate; be treated with a topical ophthalmic hypotensive drop medication in at least one eye; and possess adequate visual acuity and mental ability to read and understand English. Individuals were excluded if they had any clinically significant medical/psychiatric condition or had participated in any investigational ophthalmic trials within the previous 30 days. Patients who had ocular surgery within the last 60 days were also excluded.

Consenting participants were asked to complete the 31 draft treatment satisfaction items as well as a supplemental questionnaire gathering demographic and treatment-related information. A study staff member reviewed the materials for completeness prior to the end of their visit. Participants' physicians also provided clinical information about the level of side effects, degree of OH control, and difficulties their patients had with compliance and self-administration of their medication. In addition, patients' current treatment information from their medical records was merged with their records in the study dataset. This provided information on the types of topical medications to treat OH. As a follow-up, twenty-five patients were asked to complete the TSS-IOP and supplemental questionnaire twice, with assessments taken one week apart. The resulting information allowed for evaluation of the test-retest reliability of the measure.

Statistical Methods

The sample size required for the study was based on the requirements of the factor analytic procedure, which (as a rule of thumb) requires 10 subjects per question [60]. All statistical procedures and methods that were used in this study followed the generally accepted guidelines for the psychometric validation of PRO instrumentation [61-63]. This included the examination of construct validity using factor analysis and internal consistency of resulting scales assessed using Chronbach's Alpha coefficients. Computed scale scores allowed for assessment of the clinical-criterion and convergent validity of the instrument. The clinical criterion-related validity coefficients were based on known differences in patient's clinical condition and treatment experiences. The convergent validity of the instrument was assessed using a previously validated measure of treatment satisfaction, the Treatment Satisfaction Questionnaire for Medication (TSQM) [29]. Interclass correlations were used to assess the temporal stability of the scales under no change conditions.

Results

Sample Characteristics

Of the 412 patients approached, 252 patients consented, and 250 provided complete datasets. The majority of those who declined participation cited time constraints as the major reason (n = 91), some (n = 39) were unable to complete the survey without assistance due to current iris dilation procedures, and some declined because they thought the information was too personal (n = 32). Participants had a mean age of 64.6 years (SD 13.1) and a history of elevated IOP for an average of 8.4 yrs (SD 7.8). The sex ratio was about equal, with females representing 56.4% (141) of the sample. A slim majority of the sample was Caucasian (138, 55.2%) with 109 (43.6%) being Black and 3 (1.2%) Hispanic. The iris color of the sample was predominantly brown (142, 56.8%), followed by blue (67, 26.8%), and other light colors (41, 16.4%). Fifty-four percent of the sample (n = 134) were retired, 39.6% were working either full- or part-time and 6.8% were unemployed. A majority were receiving topical monotherapy for OH (60.4%, n = 151). Almost 80% (197) reported to have taken systemic forms of medication to treat other comorbid conditions in addition to their eye drops within the last 30 days.

Construct Validity & Scale Score Distributions

A principal components factor analysis (w/ varimax rotation) yielded 5 factors (Eigenvalues > 1.0) explaining 70% of the total variance. Weak or ambiguous items were removed and the remaining 15 items reanalyzed. The final factor analysis converged in six iterations and the five factors, Eye Irritation (EI), Convenience of Use (CofU), Ease of Use (EofU), Hyperemia (HYP) and Effectiveness (EFF), explained 71.9% of the total pooled variance (see Table 3).

Table 3

Final Five Factor Solution of the TSS-IOP Items

TSS-IOP Items	Factors*
	I	II	III	IV	V
EI_1: Bothered by prolonged burning or stinging	.784			.204
EI_2: Bothered by grittiness or sandiness in eyes	.778
EI_3: Bothered by dry eyes	.765
EI_4: Bothered by unpleasant feelings in/around eyes	.744			.268
CoU_1: Satisfaction w/ time of day to take medication		.898	.217
CoU_2: Satisfaction w/ times per day require to take med		.855	.206
CoU_3: Ease of remembering to take medication		.764	.270
EoU_1: Ability to accurately deliver drop in eye			.881
EoU_2: Ability to deliver the right amount of medication		.233	.858
EoU_3: Ease of positioning of head		.318	.756
HYP_1: Bothered by others reactions to your red eyes				.878
HYP_2: Self-conscious of eye redness	.350			.825
HYP_3: Concern over cosmetic appearance of eyes				.775
EFF_1: Prevention of future vision problems					.761
EFF_2: Reduction of current visual problems					.752

* Note: Factor loadings of less than .2 have been omitted. Factor I Eye Irritation 17.2% Factor II Convenience of Use 16.1% Factor III Ease of Use 15.3%Factor IV Hyperemia 15.2% Factor V Effectiveness 8.1%

Individual scores were computed by equating the scale range of items, adding the scale values of items within a factor, and transforming the resulting value into a score between 0 and 100. Higher scores were indicative of greater satisfaction. Examination of the distributional characteristics of the resulting scales (Table 4) revealed the presence of the data skew that is a typical characteristic of treatment satisfaction data. As noted elsewhere [29], the magnitude of the ceiling effect and accompanying skew is greatest among scales measuring less common negative events, with 53% (n = 131) of respondents reporting that they did not experience any hyperemia and 25% (n = 63) reporting no form of eye irritation. Given the small number of items in each scale, one would expect low internal consistency estimates. However, the Chronbach's Alphas for each of the TSS-IOP scales were quite high, an indication of conceptual coherence between scale items. The one-week test-retest reliability coefficients (intra-class correlations, ICC) were also adequate for all but the EFF scale, and ranged from .71 to .86. The EFF scale which manifested some score instability over a one-week period possessed an ICC of .41. Nevertheless, these values should be interpreted with caution as larger samples are typically required for adequate estimation of test-retest statistics.

Table 4

Score Distribution and Internal Consistency Characteristics of TSS-IOP Scales (n = 250)

TSS-IOP Scales	Mean (SD) Statistic	Number of Items	Chronbach's Alpha	Skewness Statistic	% Ceiling	Test-Retest Reliability (ICC's)
Hyperemia	91.3 (17.7)	3	.84	-2.89	65.3%	.86
Eye Irritation	91.2 (14.3)	4	.80	-2.71	48.4%	.71
Convenience of Use	80.2 (15.4)	3	.86	-.53	20.8%	.78
Ease of Use	68.9 (21.0)	3	.86	-.58	10.8%	.86
Effectiveness	77.0 (16.7)	2	.83	-.95	14.9%	.41

Table 5 presents the Spearman Rho intercorrelations between the five scales of the TSS-IOP. As might be expected, the greatest conceptual overlap was observed between EofU and CofU (r = .56). EI and HYP were also correlated at .40. The correlation of EofU and CofU with EFF was bit higher than expected (r = .40 and .43 respectively) although low enough to suggest a degree of conceptual distinctiveness of these measurement constructs.

Table 5

Inter-scale Spearman Rho Correlations on the TSS-IOP

	Effectiveness	Eye Irritation	Hyperemia	Convenience of Use
Eye Irritation	.19***
Hyperemia	.27***	.40***
Convenience of Use	.43***	.21***	.30***
Ease of Use	.40***	.24***	.22***	.56***

* p < .05 level (2-tailed) ** p < .01 level (2-tailed). *** p < .001 (2-tailed)

Table 6 presents the item-to-scale correlations for each of the five scales. Strong loadings of items on its respective scale replicate observations of high internal consistency of scale items and the factorial distinctiveness of scales. Similar patterns of intercorrelations between items and unrelated TSS-IOP scales reveals a moderate association between CofU and EofU, also observed in the inter-scale correlation table presented above.

Table 6

TSS-IOP Item-Scale Spearman Rho Correlations

TSS-IOP Items	Effectiveness	Eye Irritation	Hyperemia	Convenience of Use	Ease of Use
EFF_1	.92	.18	.21	.40	.40
EFF_2	.95	.18	.26	.34	.40
EI_1	.17	.63	.34	.17	.22
EI_2	.16	.68	.26	.14	.11
EI_3	.16	.70	.39	.18	.17
EI_4	.12	.68	.32	.23	.18
Hyp_1	.23	.38	.88	.15	.26
Hyp_2	.20	.23	.71	.10	.17
Hyp_3	.21	.28	.72	.24	.30
EofU_1	.33	.22	.20	.89	.51
EofU _2	.36	.15	.18	.88	.45
EofU _3	.32	.28	.22	.86	.52
CofU_1	.49	.18	.23	.49	.89
CofU_2	.40	.17	.26	.49	.92
CofU_3	.30	.21	.28	.50	.86

* All correlation is significant at the .05 level (2-tailed).

Convergent Validity

Scales of the TSQM, a previously validated TS-M instrument, were used to examine the convergent validity of the new TSS-IOP scales. Conceptually related dimension on the TSQM and TSS-IOP were expected to exhibit moderate to large correlations (.5–.8) with one another. Table 7 reveals that this was indeed the case for the satisfaction scores on EFF, CofU and EofU. The lower correlations between the TSQM Side Effects scale and the TSS-IOP EI and HYP scales may suggest that patients think about side effect items on these two instruments somewhat differently. A final observation was that the pattern of correlations of the TSQM Global scale with both the TSQM specific scales and the TSQM-IOP specific scales was very similar

Table 7

Convergence of the TSS-IOP Scales and the TSQM (Spearman Rho Correlations)

	TSQM GLOBAL	TSQM Effectiveness	TSQM Side Effects	TSQM Convenience
TSQM Scales
Effectiveness	.52***
Side Effects	.29***	.31***
Convenience	.40***	.30***	.36***
TSS-IOP Scales
Effectiveness	.50***	.77***	.34***	.34***
Eye Irritation	.20**	.18**	.45***	.32***
Hyperemia	.21***	.19**	.36***	.34***
Convenience of Use	.48***	.40***	.35***	.68***
Ease of Use	.41***	.36***	.28***	.62***

* p < .05 level (2-tailed) ** p < .01 level (2-tailed). *** p < .001 (2-tailed)

Criterion-Related Validity: Subgroup Comparisons

The expected correlations were found between patients' frequency ratings of specific problems associated with treatment and their ratings of satisfaction with the side effects of treatment. These associations between the frequency endorsement of undesirable events and satisfaction levels were observed using both instruments (i.e., TSQM Global and Side Effects scales and the TSS-IOP IE and HYP scales), with stronger correlational associations found on Level 3 scales than the more general Level 2 or Level 1 scales. Interestingly, clinical ratings of the severity of unintended medication effects were significantly correlated with relatively few of the patients' frequency ratings, the notable exceptions were, red eyes (r = .21, p < .001), twitching/tight eye lids (r = .16, p < .05), iris pigmentation (r = .14, p < .05) and darkening of the eye lids (r = .17, p < .01).

In a similar manner, patients' reported problems with self-administration of their medication were weakly correlated with relatively few of physicians' ratings of these problems (problems of self-administration, r = .15, p < .05; anticipatory blinks, r = .15, p < .05; and medication spillage, r = .12, p < .05). In contrast, the correlations between patients' difficulty ratings of medication administration and their satisfaction on the TSQM Convenience scale and the CofU and EofU TSS-IOP scales were stronger (Table 9). Inspection of Level 3 TSS-IOP ratings revealed an experiential distinction between EoU and CoU by the type of self-administration problem, these distinctions were not discernable at more general levels of abstraction (Levels 2 and 1). Again, the correlations between specific experiences and more general Levels 1 and 2 TSQM satisfaction ratings were weaker than on the Level 3 scales of the TSS-IOP.

Table 9

Convenience Satisfaction Ratings Correlated with Frequency of Specific Difficulties with Administration (Spearman Rho Correlations)

	TSS-IOP Ease of Use (Level 3)	TSS-IOP Convenience of Use (Level 3)	TSQM Convenience (Level 2)	TSQM GLOBAL (Level 1)
Problems self-administering	-.52***	-.30***	-.42***	-.28***
Requiring assistance	-.23***	-.02	-.17**	-.06
Frequency missing eye	-.44***	-.29***	-.33***	-.09
Anticipatory blink and spillage	-.30***	-.23***	-.30***	-.15*
Trouble positioning head	-.43***	-.34***	-.34***	-.19**
Delivering too much medication	-.45***	-.29***	-.27***	-.16**
Forgetting to use medication	-.17**	-.38***	-.26***	-.11

* p < .05 level (2-tailed) ** p < .01 level (2-tailed). *** p < .001 (2-tailed)

The correlations between patients' satisfaction ratings and physicians' ratings of their patients on the core treatment dimensions of IOP control, severity of side effects, compliance, and difficulties with self-administration (see Table 10) provided evidence for the concurrent clinical criterion-related validity of the TSQM and TSS-IOP scales. Stronger associations were observed between physicians' ratings and the most conceptually related treatment satisfaction scales. Interestingly, doctors' ratings of the severity of side effects and problems with self-administration of medication were more highly correlated with patients' satisfaction in these areas than they were with the frequency or degree of any actual events. A final observation was that physicians' ratings of compliance were not significantly correlated with any dimension of patients' satisfaction ratings.

Table 10

Convergent Validation of Patients' Satisfaction Ratings Using Physicians' Ratings of Patient Case (Spearman Rho Correlations)

	PHYSICIANS' RATINGS
	Degree of IOP Control	Severity of Side Effects	Compliance w/ Medication Regimen	Problems w/ Self-Administration
TSQM Scales
GLOBAL	.18**	-.13*	.00	-.12
Effectiveness	.26***	-.14*	.00	-.06
Side Effects	.16*	-.35***	.04	-.17**
Convenience	.06	-.03	.04	-.23***
TSS-IOP Scales
Effectiveness	.26***	-.16*	.03	-.09
Eye Irritation	.08	-.22***	.10	-.11
Hyperemia	.11	-.18**	.01	-.16*
Convenience of Use	.18**	-.05	.06	-.16*
Ease of Use	.07	-.08	.04	-.13*

* p < .05 level (2-tailed) ** p < .01 level (2-tailed). *** p < .001 (2-tailed)

With the exception of HYP and EI, patients' self-reported level of resistance to using their medication was negatively correlated with all aspects of their satisfaction with treatment. The same was found for another subjective or emotionally based measure, patients' ratings of their acceptance of their illness (Table 11). Of note, these emotionally based appraisals of illness acceptance and treatment resistance most correlated with the Level 1 global scale scores. Patients' ratings of their tendency to forget to use their medication were most strongly correlated with the TSQM Convenience scale and particularly the CofU scale of the TSS-IOP.

Table 11

Acceptance of Illness and Resistance to Using Medication by Satisfaction Levels (Spearman Rho Correlations)

	Acceptance of Illness	Resistance to Using Medication	Forgetting to Take Medication
TSQM Scales
GLOBAL	.38***	-.32***	-.12
Effectiveness	.29***	-.28***	-.18*
Side Effects	.26***	-.18**	-.20*
Convenience	.19**	-.16*	-.28**
TSS-IOP Scales
Effectiveness	.27***	-.29***	-.26**
Eye Irritation	.08	-.07	-.05
Hyperemia	.05	-.09	-.12
Convenience	.27***	-.24***	-.39***
Ease of Use	.22***	-.15*	-.19*

* p < .05 level (2-tailed) ** p < .01 level (2-tailed). *** p < .001 (2-tailed)

Evidence of Known Groups Validity: Satisfaction by Medications Groups

A comparison of persons on single (60%, n = 151) versus multiple topical medications (40%, n = 99) by the dimensions of treatment satisfaction revealed that the monotherapeutic group was more satisfied than the polytherapeutic group on the TSQM Side Effects scale (93.4 (12.7) vs. 88.7 (15.2), F(1, 243) = 6.67, p = .01), and the TSS-IOP EI scale (93.4 (11.1) vs. 87.5 (17.8), F(1, 243) = 10.4, p = .001), CofU scale (82.5 (14.2) vs. 77.1 (16.8), F(1, 243) = 7.47, p = .007) and the EFF scale (79.1 (15.4) vs. 73.7 (18.0), F(1, 243) = 6.19, p = .014). Monotherapeutic respondents on Beta Blockers (n = 34) and Prostagladins (n = 80) reported the highest satisfaction levels with CofU, followed by those on Carbonic Anhydrase (n = 22) and Alpha Agonists (n = 12), (85.3 (14.5), 83.6 (14.0), 79.3 (14.3) and 73.6 (11.1) respectively, F(3,144) = 2.62, p = .05). Respondents on Beta Blockers also reported the highest satisfaction with HYP, followed by Carbonic Anhydrase Inhibitors, Prostagladins and Alpha Agonists (99.3 (3.2), 93.6 (8.1), 90.7 (17.8) and 88.2 (27.2) respectively, F(3,144) = 2.79, p = .04).

Of those on monotherapy, 11% (n = 16) reported administering their medications in the morning, 46% (n = 69) in the evening and 41% (n = 62) administered them in both the morning and evening. A comparison of respondents based on the time of day of medication administration affirmed focus group discussion and revealed that among monotherapeutic patients, the lowest CofU ratings occurred for those using medications both morning and evening, followed by evening administration, with the highest satisfaction among morning users (77.6 (SD15.9), 83.8 (13.4), 89.6 (12.5), F(3) = 7.31, p = .001).

Discussion

This initial psychometric analysis of the TSS-IOP revealed the instrument possesses a sound conceptual structure (construct validity), all but one TSS-IOP scale possessed reliable assessment characteristics, and, on most dimensions the scales manifested the expected convergent validity using an established measure of TS-M. Some construct divergence was observed from the TSQM Side Effects construct, which was manifested by lower than expected correlations between the HYP and EI scales of the TSS-IOP and Side Effects scale of the TSQM. A supplemental analysis of TSS-IOP EI and HYP scales with TSQM Side Effect items revealed low item to scale correlations on TSQM items pertaining to the impact of side effects on the mental and physical health of patients. Neither the physical and mental impacts of OH/POAG treatments were emphasized as important aspects of topical OH treatments by a significant number of focus group participants and thus were not covered by content of the TSS-IOP side effect scales.

Of note, there was significant debate among researchers over the meaningfulness of including the two EFF items in the TSS-IOP since it is believed that patients cannot reliably detect or report on treatment-related changes in vision associated with reduction in intraocular pressure. It was eventually decided that these items should be included because a significant number of focus group members, particularly those in the earlier stages of disease, emphasized the importance of changes in their visual acuity as a result of treatment. Since the clinical meaningfulness of patient reported changes in visual problems associated with OH treatments has yet to be established or refuted, inclusion of the EFF scale can, at the very least, be considered an attempt to retain the face validity of the TSS-IOP to patients – who recognize this as the most important reason for taking their medication in the first place. The poor test-retest stability of the EFF scale may reflect patients' inability to reliably discern and report on this dimension of IOP treatment. Alternatively, such 'instability' may reflect real fluctuation or changes in clinical measurement of IOP. Supporting this possibility, a significant correlation was observed between patients' EFF satisfaction scores and clinician's ratings of IOP control, suggesting that patients derive at least some of the information with which to make satisfaction judgments directly from results reported during the clinical assessment process. Notably, a significant number of focus group members knew their IOP levels as communicated by their physicians at visits.

The concurrent criterion-related validity of the TSS-IOP two side effect scales and two convenience scales were demonstrated by a clear association with a fundamental set of criterion measures, namely patients' ratings of the frequency and severity of various problems associated with their use of the medication. These criterion-related validity results suggest that patients' satisfaction ratings on the IE and HYP scales were differentially based on somewhat distinct aspects of patient experience, as were the two convenience scales. In addition, the TSS-IOP provides further conceptual distinction to the TSQM Convenience construct, in that the new instrumentation successfully discriminates between patients' satisfaction with a treatment scheduling from satisfaction with the ease of use of delivery technology. Evidence for the 'known groups' validity of the EI, HYP, CofU and EofU scales was found since TSS-IOP scores on these scales differed significantly between classes of medication as well as by the frequency of daily medication administration. These differences were consistent with differences in patient satisfaction that are known to occur within clinical practice.

Compared to the Level 1 scale of the TSQM, the higher correlations of the TSS-IOP (Level 3) scales and patients' reports of specific treatment difficulties support the commonly assumed measurement benefits of using more specific TS-M scales as an means for the differentiation of pharmaceutical products; in this case, the dimensions of EI, HYP, CoU, and EoU associated with topical ophthalmic medications. The greater event/situational specificity of Level 3 scales allows for more focused statements to be made about the associations between problematic effects of treatment and patients' dissatisfaction.

Despite earlier concerns raised about the content validity of the side effects and convenience scales of the TSQM in this population, certain advantages were found to using these more generally worded items and scales of the TSQM. The Level 2 and Level 1 scales of the TSQM seemed to possess greater explanatory power with respect to clinicians' ratings of treatment side effects and difficulties their patients had with medication use. In addition, Level 1 and, to some degree, Level 2 TSQM scales were more strongly correlated with patients' ratings of their own resistance to regular medication use than the Level 3 scales of the TSS-IOP. There is a need for future clinical research to assess the degree to which dimensions and levels of TS-M assessment are able to predict actual medication-related behaviors such as compliance and persistence with medication regimens, as well as a need to more clearly specify how non-compliance affects broader health outcomes among persons with POAG. Such associations could have important ramifications for numerous stakeholders involved in health care delivery, including those in the pharmaceutical industry, regulatory agencies, health management organizations, and professionals providing clinical care.

A final observation was the scarcity of significant correlations between patients' frequency ratings of undesirable treatment experiences and clinicians' ratings of problems with side effects of medication use. This might suggest that the involved practitioners use relatively few specific indicators to assess these dimensions of treatment impact. Also contributing to the low correlations, specific treatment-related events often vary across individuals, thereby reducing the correlations observed between measures of specific events and practitioners' general clinical assessments across a heterogeneous sample of patients. Interestingly, patients' satisfaction ratings were more highly correlated with physicians' ratings of side effect than with patients' frequency ratings of undesirable aspects of treatment. This raises the possibility that patients' expression of satisfaction or dissatisfaction in the clinical setting may influence physicians' clinical impression on certain clinical assessment dimensions, this in turn suggests that treatment satisfaction could play a role in clinical decisions in regards to prescribing, adjusting, and/or switching medication regimens. Further research is required to examine the clinical usefulness of treatment satisfaction assessments and their role in the clinical decision-making process.

Conclusions

Results from this initial validation study of the TSS-IOP indicate that the measure is psychometrically sound and provides a means to assess important aspects of patients' experiences with the two dimensions of side effects and two dimensions of convenience associated with topical eye medications used in the control of OH. Results from the two side effect scales (Hyperemia and Eye Irritation) demonstrate their ability to differentiate between medications clinically known to differ on these dimensions. Results from the Effectiveness scale suggest that this scale reflects patient reported perception of effectiveness, likely a result of professional opinion about a medication's ability to manage OH.

Authors' contributions

MJA, Principle Investigator, Overall Project Management, Study Design & Planning, Psychometric Design & Analysis, Primary Authorship

WCS, Study Planning, Primary Clinical Investigator, Second Authorship

JMF, Study Design & Planning

JAS, Clinical Study Coordinator, CRF Preparation, Data Management, Manuscript Reviewer

RD, Study Design

EM, Study Design

JL, Design of Qualitative Methodologies, Discussion Guide, Focus Group Facilitator

Table 8

Respondents' Dissatisfaction Ratings Correlated with Specific Problematic Treatment Effects (Spearman Rho Correlations)

	TSS-IOP		TSQM
Frequency of...	Hyperemia (Level 3)	Eye Irritation (Level 3)	Side Effects (Level 2)	GLOBAL (Level 1)
Itching	-.24***	-.35***	-.35***	.08
Burning	-.28***	-.37***	-.25***	.15*
Stinging	-.36***	-.41***	-.31***	.14*
Grittiness	-.26***	-.56***	-.32***	.08
Tearing	-.21***	-.32***	-.20**	.14*
Dryness	-.31***	-.54***	-.26***	.14*
Puffiness-Swelling	-.35***	-.41***	-.24***	.14*
Red Eyes	-.52***	-.41***	-.35***	.20***
Twitching-Tight Lids	-.20***	-.31***	-.24***	.10
Degree of...
Eye Lash Growth	-.12*	-.21***	-.14*	.01
Baggy Eye Lids	-.41***	-.32***	-.29***	.15*
Iris Pigmentation	-.24***	-.23***	-.16**	.01
Darkened Eye Lids	-.29***	-.25***	-.26***	.12

* p < .05 level (2-tailed) ** p < .01 level (2-tailed). *** p < .001 (2-tailed)