Construct validation of the Amblyopia and Strabismus Questionnaire (A&SQ) by factor analysis.

BACKGROUND: The Amblyopia and Strabismus Questionnaire (A&SQ) was previously developed to assess quality of life (QoL) in amblyopia and/or strabismus patients. Here, factor analysis with Varimax rotation was employed to confirm that the questions of the A&SQ correlated to dimensions of quality of life (QoL) in such patients.
METHODS: Responses on the A&SQ from three groups were analyzed: healthy adults (controls) (n = 53), amblyopia and/or strabismus patients (n = 72), and a historic cohort of amblyopes born between 1962-1972 and occluded between 1968-1974 (n = 173). The correlations among the responses to the 26 A&SQ items were factor-analysed by Principal Component Analysis (PCA). As the development of the A&SQ was intuitive-deductive, it was expected that the pattern of correlation could be explained by the five a priori hypothesized dimensions: fear of losing the better eye, distance estimation, visual disorientation, diplopia, and social contact and cosmetic problems. Distribution of questions along the factors derived by PCA was examined by orthogonal Varimax rotation.
RESULTS: Data from 296 respondents were analyzed. PCA provided that six factors (cutoff point eigenvalue >1.0) accumulatively explained 70.5% of the variance. All A&SQ dimensions but one matched with four factors found by Varimax rotation (factor loadings >0.50), while two factors pertained to the fifth dimension. The six factors explained 33.7% (social contact and cosmetic problems); 10.3% (near distance estimation); 8.7% (diplopia); 7.2% (visual disorientation); 6.3% (fear of losing the better eye); and 4.3% (far distance estimation), together 70.48% of the item variance.
CONCLUSION: The highly explained variance in the A&SQ scores by the factors found by the PCA confirmed the a priori hypothesized dimensions of this QoL instrument.

Introduction

We previously developed, in an intuitive-deductive manner, the Amblyopia and Strabismus Questionnaire (A&SQ) to measure the QoL in amblyopia and strabismus patients [1]. The A&SQ is intended to measure patients’ factors of QoL that might parallel with five dimensions, i.e. a priori hypothesized domains: fear of losing the better eye, distance estimation, visual disorientation, diplopia, and social contact and cosmetic problems. These domains originated from an inventory of day-to-day complaints that an amblyopia and/or strabismus patient experiences. Twenty-six questions were formulated for these hypothesized domains with an intuitive, deductive approach. Having set up working definitions for the five kinds of QoL to be measured, the questions formulated had to fit these definitions (Table 1). The A&SQ is similar in format to the National Eye Institute Visual Function Questionnaire 25 (VFQ-25) [2]. A&SQ discriminant validity and reliability was assessed previously in a comparison between the A&SQ, the VFQ-25 and the Short-Form 12 Health Survey. In that study, three groups were compared: controls, outpatients, and a historic cohort of patients who had been treated for amblyopia 3 decades ago [1]. Clinical validation of the A&SQ in the historic cohort showed that the visual acuity of the amblyopic eye was the dominant clinical determinant for all five domains [3].

Table 1

Results of factor analysis (PCA and Varimax rotation) within the five A&SQ dimensions; fear of losing the better eye (Q 1–Q 3), distance estimation (Q 4–Q 13), visual disorientation (Q 14–Q 16), diplopia (Q 17–Q 21), social contact and cosmetic problems (Q 22–Q 26)

	Six-factor solution*
Questions	1	2	3	4	5	6
1 I can see equally well with both eyes	0.13	−0.01	0.01	−0.10	0.64	0.26
2 I worry about losing my better eye	0.12	0.12	−0.04	0.11	0.90	0.06
3 I worry something might gets into my better eye	0.13	0.13	0.03	0.12	0.90	0.04
4 I can estimate distances well	0.09	0.24	0.24	0.18	0.24	0.75
5 I have good depth perception	0.12	0.33	0.19	0.16	0.26	0.70
6 I feel unsure when putting something on a table	0.12	0.77	0.18	0.05	0.03	0.17
7 I miss the other person's hand when shaking hands	0.20	0.63	0.09	−0.05	−0.07	0.27
8 I have difficulty parking my car	0.16	0.12	−0.13	0.24	−0.03	0.67
9 I find it difficult to put the cap on a pen or marker	0.05	0.74	0.25	0.07	0.13	0.01
10 I find it difficult to put power plug into a socket	−0.06	0.69	0.16	0.36	0.16	0.02
11 I have difficulties pouring drinks	0.17	0.75	−0.03	0.10	0.09	0.26
12 I have difficulties walking down stairs	0.15	0.46	0.30	0.23	0.07	0.13
13 I have difficulties playing ball games	0.27	0.34	0.21	0.25	0.11	0.51
14 I have difficulties finding my way in a shopping mall	0.13	0.13	0.14	0.86	0.05	0.26
15 I have difficulties finding my way in a department store	0.13	0.14	0.21	0.89	0.08	0.16
16 I have difficulties finding my way in a train station	0.11	0.16	0.22	0.84	0.03	0.18
17 I see double	0.14	0.15	0.88	0.10	0.02	0.12
18 Double vision disturbs me in my daily activities	0.14	0.07	0.87	0.20	−0.00	0.10
19 When I am tired, I must be very careful not to miss what I reach for	0.17	0.44	0.73	0.11	−0.04	0.07
20 I have to do things more slowly when I am tired, because of my eyes	0.15	0.25	0.82	0.19	0.03	0.04
21 I have to squint or shut one eye in bright sunlight	0.12	0.06	0.30	0.13	0.32	0.40
22 I have difficulty making eye contact in a one-on-one conversation	0.77	0.17	0.16	0.24	0.07	0.12
23 I have difficulty making eye contact with people in a group conversation	0.77	0.18	0.16	0.29	0.08	0.13
24 My eyes are misaligned (one or both eyes cross, or turn out or up)	0.77	0.10	0.17	−0.01	0.14	0.10
25 Because of my misaligned eyes I feel insecure	0.86	0.12	0.04	0.06	0.16	0.12
26 If I did not have misaligned eyes, I would have more self-confidence	−0.87	−0.02	−0.09	0.02	−0.05	−0.10
Unrotated solution
Eigenvalue	8.75	2.68	2.27	1.86	1.64	1.13
% Variance explained	33.65	10.30	8.70	7.17	6.31	4.33
% Cumulative variance explained	33.65	43.94	52.65	59.82	66.13	70.48
Rotated Solution
Eigenvalue	3.71	3.55	3.41	2.92	2.40	2.40
% Variance explained	14.21	13.61	13.05	11.18	9.21	9.21

Factor loadings >0.50 are in bold. The table shows that the six non-rotated factors explained 33.7%, 10.3%, 8.7%, 7.2%, 6.3% and 4.3%, together representing 70.48% of the item variance. All hypothesized A&SQ domains but one matched with the underlying factors found by the factor analysis.The items of one hypothesized domain, distance estimation, were distributed over two factors, according to whether the questions related to near- or far-distance estimation (Q 4–Q 13).

Decrease in QoL in unilateral amblyopia plays a major role in cost-effectiveness of screening and treatment of amblyopia [4]. In a birth cohort study, a decrease in quality of life could not be detected through population-level outcomes in education, employment, sport and social functioning, general and mental health [5]. However, it seems likely that a reduction in QoL (mostly minor in amblyopia and strabismus) can only be detected at the level of daily functional restrictions, as assessed by the A&SQ.

Here we evaluate the construct validity of the A&SQ—the correlation of its constructed domains and questions to the QoL of amblyopia and/or strabismus patients—by factor analysis. This makes it possible to identify whether the A&SQ assesses separate QoL dimensions [6]. Themes that were uniquely and directly attributable to amblyopia and strabismus produced the five hypothesized domains during the design of the A&SQ. Formulated questions within each hypothesized domain had to be as broadly varied and as realistic as possible. It was expected that the questions within each domain would be indicators of a common patient disposition. This implied the necessity that the responses to the questions within the hypothesized domains each correlated uniquely to corresponding factors, i.e. QoL dimensions. Rotation to simple structure (Varimax) was performed to discern the distribution of A&SQ questions along the derived factors.

Methods

Subject groups and data

The factor analysis was performed on data from our previous study [1], in which three groups of subjects had filled out the A&SQ. The first group were 53 healthy controls, some with minor eye problems like wearing glasses. Mean age was 32.8 years (SD = 12.4 years); 48.1% were male. The second group were 72 adult unilateral amblyopia and/or strabismus patients, visual acuity ≤0.5 D, from our ophthalmology outpatient clinic. Mean age was 44.1 years (SD = 16.1 years); 47.8% were male. The third group was a historic cohort of 173 patients born between 1962-1972. Mean age was 35.9 years (SD = 2.8 years); 51.2% were male. This historic cohort was derived from 471 patients who had all been treated for amblyopia with occlusion therapy between 1968–1974 for amblyopia and strabismus in the ophthalmology outpatient clinic of the Waterland Hospital in Purmerend. The historic cohort has been previously described as an almost non-select sample of amblyopes, because the 471 patients comprised almost all patients with strabismus and amblyopia occluded at that time in Waterland, a rural area north of Amsterdam [3]. Of these 471 patients, 203 could be traced and were sent the A&SQ. 173 responded (36.7%). Orthoptic re-examination of 137 patients from the historic cohort followed in 2003. At the beginning of their occlusion treatment, 98 of the 137 patients (71%) had amblyopia caused by strabismus. In 2003, acuity of the amblyopic eye had slightly improved in the 98 strabismic amblyopes, had slightly deteriorated in the anisometric amblyopes, and had deteriorated in the combined-mechanism amblyopes as compared to the acuity at the end of the occlusion therapy, 30–35 years ago [7].

Answers on a five-point scale from the three groups of respondents on the 26 A&SQ were processed to obtain a complete dataset. Non-applicable questions (to be skipped) were valued as «none of the time». The answer-alternative of an activity scored as not relevant was replaced by the group mean; that of an eye condition scored as not relevant by the score «none of the time». Answers that were then still missing were imputed by a hotdeck-method that uses responses from similar patients in the same dataset [8].

Factor analysis

Two QoL questions are expected to highly correlate if their answers are indications of some common underlying factor. Then, two response variables can virtually be replaced by one single variable constructed to best explain the correlation between the two responses. Similarly, Principal Component Analysis (PCA) is applied to the matrix of correlations of a set of variables to find a set of common underlying factors, called principal components [9]. Principal components explain the correlations between all observed responses. The eigenvalue of a principal component is its variance and the eigenvector its covariance with each of the (standardized) responses. The principal components system can be viewed as a coordinate system wherein each question has its place; the closer the questions are to each other, the higher their correlation. The principal components are so derived that they are uncorrelated with each other, and the first component accounts for the highest possible variance, the second the second highest variance, and so on. The outcome of such derivation still has to be made more informative and unique in order to be of use for a specific QoL instrument. First, the principal components with an eigenvalue smaller than 1.0, i.e. a variance smaller than the variance of a single standardized item response, are ignored. We are then left with a smaller number of components that best explain the correlations between the responses. Second, the coordinate system is rotated to a simple structure (if rotated with orthogonal angles: Varimax) such that each response is uniquely associated with a single component, i.e. loaded as much as possible on a single factor. Factor loadings were calculated (ORBACLAN, version 2.0 [10]) as correlations between the item responses and the principal components or factors. Finally, the scale unit of each principal component is changed such that each has a variance of one. Consequently, the associations of the observed responses with these new components could now be interpreted substantively in terms of the underlying factors measured by the questions.

Because of its construction in five dimensions it was expected that the A&SQ could be explained by five factors. The appearance of five unidimensional factors that correspond to the assignments of our items to our hypothesized domains is considered as evidence for the construct validity of the A&SQ.

Results

Data from 296 respondents were analysed by PCA. Excluded from factor analysis were two respondents from the historic cohort with too many missing answers. We extracted six factors that explained a substantial amount (70.5%) of item variance (eigenvalue >1.0). The effect of additional factors was negligible. The square of each factor loading corresponds to the proportion of variance that the item responses and the factor have in common. Table 1 gives the factor loadings of the Varimax-rotated six-factor solution. The responses to items in the hypothesized domains are generally appropriately associated with the factors. The six (non-rotated) factors explained 33.7%, 10.3%, 8.7%, 7.2%, 6.3% and 4.3% of the item variance respectively. The constructed, hypothesized A&SQ domains, matched with that of the underlying factors that were found by the factor analysis, provided that the items of one hypothesized domain, distance estimation, were distributed over two factors, according to whether they related to near- or far-distance estimation (Table 1, Q 4-Q 13).

The amount of item grouping within the factors was provided by orthogonal Varimax rotation. Twenty-four of the 26 A&SQ items were mainly unidimensionally distributed along the six factors with eigenvalue >1.0. After rotation, the six factors explained 20.2%, 19.3%, 18.5%, 15.9%, 13.1% and 13.1% of the item variance respectively (variance 100% in total), or explaining 14.2%, 13.6%, 13.1%, 11.2%, 9.2% and 9.2% of the item variance. Together, the six factors explained 70.5% of the variance. The now-found structure was not completely congruent with the hypothesized domain structure, but still showed a high degree of grouping (Table 1).

Two questions: “walk down stairs” (Q 12) and “squinting one eye” (Q 21) had a factor loading of <0.50 on all factors. The five questions “eye contact one-on-one conversation” (Q 22), “eye contact group conversation” (Q 23), “misalignment” (Q 24), “feel insecure” (Q 25), and “self-confidence” (Q 26) had the highest factor loadings (>0.70) on the first factor, which accounted for 14.2% of the variance (Table 1). The five questions “put something on table” (Q 6) “miss other person’s hand when shaking hands” (Q 7), “put cap on pen” (Q 9), “put power plug into socket” (Q 10), and “pouring drinks” (Q 11) had high factor loadings on the second factor, which explained 13.6% of the variance. The questions “see double” (Q 17), “disturbing double vision” (Q 18), “reaching things when tired” (Q 19), and “doing things when tired” (Q 20) loaded high on the third factor, explaining 13.1% of the variance. The questions “find way in mall” (Q 14), “find way in store” (Q 15), “find way in train station” (Q 16) had high loadings on the fourth factor, which explained 11.2% of the variance. The three questions “seeing equally well both eyes” (Q 1), “worry losing better eye” (Q 2), and “worry something gets into better eye” (Q 3) had high loadings on the fifth factor, accounting for 9.2% of the variance. The four questions “estimate distances well” (Q 4), “good depth perception” (Q 5), “parking car” (Q 8), and “play ball games” (Q 13) had high loadings on the sixth factor, accounting for 9.2% of the variance.

Accordingly, it could be determined that the found six factors were associated with the hypothesized construction of the five A&SQ domains, and the so-arranged questions within these hypothesized domains could be retained. Four factors were entirely associated with four hypothesized domains that could be left unchanged; the first with social contact and cosmetic problems, the third with diplopia, the fourth with visual disorientation, and the fifth with fear of losing the better eye. One hypothesized domain, distance estimation, had to be separated into two domains: near distance estimation and far distance estimation.

Discussion

The 70.5% of the item variance that was explained, while the six factors found largely matched the five A&SQ dimensions, strongly supported the construct validity of the A&SQ. For comparison, the VFQ-25 had 46.37% of the variance explained by four factors [11], the Impact of Visual Impairment questionnaire had 57.2% of the variance explained by three factors [12], and the GO-QOL 64.6% of the variance explained by four factors [13].

Based on the factor loadings of the items on the six factors, the questions in the four hypothesized domains social contact and cosmetic problems, diplopia, visual disorientation, and fear of losing the better eye could be assigned to these dimensions as expected, since all questions with factor loadings >0.50 showed a high degree of ordering on four factors.

The remaining factors two and six correlated to the questions in the second hypothesized domain distance estimation. The second factor related to problems with near distance estimation [<1 m between eyes and the to be (touched) object]: pertaining to putting something on a table (Q 6), shaking other person’s hand (Q 7), capping a pen (Q 9), putting power plug in socket (Q 10), and pouring drinks (Q 11). The sixth factor related to problems with far distance estimation: Pertaining to estimating distances well (Q 4), subjective depth perception (Q 5), parking car (Q8) and playing ball games (Q 13).

The one question (in the domain diplopia) with a low factor loading (< 0.50), “squinting one eye in bright sunlight” (Q 21), might better be excluded from the questionnaire. The low factor loading could be explained by the fact that most divergent strabismus patients who squint do not suffer from diplopia. It is unclear why they have to close one eye, even when they do not experience diplopia.

The A&SQ was developed in an intuitive-deductive fashion. Some arguments have been raised against the process by which the A&SQ was developed. Firstly, it has been argued that it is unclear whether the A&SQ applies to QoL in patients with either amblyopia or strabismus, and whether it would not have been better to construct a questionnaire for either amblyopia or strabismus patients [14-16]. The A&SQ has been developed in amblyopia and strabismus patients to jointly assess QoL in such patients. It seems exceedingly difficult to obtain truly representative samples of patients with pure amblyopia or of patients with pure strabismus. In our historic Waterland cohort, we found that the two conditions are so much intertwined that a joint assessment of the two conforms better to clinical reality and to the population-level state of amblyopia and strabismus; a reality of which the ophthalmologists are aware of at the visual screening in children.

Secondly, it has been said that a QoL questionnaire should preferably be developed by selecting a limited number of questions from a large pool of questions, without pre-conceived concepts, by factor analysis. This quantitative-inductive approach to develop the questionnaire was used, for example for the VFQ-25. However, an equally appropriate approach to develop a QoL instrument is by classifying collected patient complaints into themes, subsequently in domains and then to formulate questions for the domains. This qualitative-deductive (intuitive) method, applied to the development of the A&SQ, is used in constructing psychometric instruments [12]. The themes, domains and questions were deduced along the causative theory of sign, organ failure, disease, loss of function, functional restriction leading to decrease in quality of life [17]. As a result, factor analysis outcomes showed that the response variance on the A&SQ questions could for a large amount be explained without redundant domains, which implied that the collected complaints were all-encompassing, in spite of the fact that the number of subjects was not very large [14].

Thirdly, the question is whether the results of factor analysis can be generalized to all other subjects with amblyopia and/or strabismus because they were derived from three different specific subject groups (controls, outpatients and historic cohort patients) and, possibly, measurement variance may have occurred.

As an argument for generalization, the historic cohort (previously been described as an almost non-select sample of amblyopia and/or strabismus patients) contained patients with amblyopia caused by strabismus, anisometropia or both, amblyopia caused by deprivation, and various kinds of strabismus [3]. In addition, it is likely that the 174 patients who had filled out the A&SQ represented the entire group of 471 occluded amblyopia and/or strabismus patients. Finally, moving out of the region was relatively rare in this rural area; urbanization occurred only after 1980.

In conclusion, the A&SQ [18] is a sufficiently validated instrument to assess the QoL in all amblyopia and/or strabismus patients, and the highly explained variance in the A&SQ scores by the factors found by the PCA confirmed the a priori hypothesized dimensions of this QoL instrument.