Validation of the Portuguese version of the Personal Outcomes Scale.

BACKGROUND/
OBJECTIVE: Quality of life (QOL) is actually helpful to organizations for guiding personalized support plans and enhance personal outcomes. However, there is a lack of consensus about the QOL construct in the field of intellectual disability (ID). This study aims (a) to analyze the first-order factor structure (i.e., eight domains) of the Portuguese version of the Personal Outcomes Scale (POS), and (b) to compare two alternative models of the higher factor structure of the QOL construct (i.e., Salamanca and Schalock).
METHOD: Both aims were investigated by examining self-report and report-of-others measures. Data were collected from 1,264 adults with mild or moderate levels of ID and respective proxies.
RESULTS: Results from confirmatory factor analysis (CFA) showed evidence of suitable psychometric properties of the QOL dimensions. Findings also highlighted that the first-order model was more robust than either of the two second-order models. Nevertheless, the Schalock structure was stronger than the Salamanca structure.
CONCLUSIONS: Although further research is needed, the results reveal that the Portuguese POS may be a valid and reliable instrument to measure QOL of adults with ID. The implication of these results are discussed highlighting the scale as a useful tool to serve as the basis for planning and evaluating personalized supports.

The examination of the construct of quality of life (QOL) has been a consistent topic in the literature over the past three decades. The concept has been used as a key desired outcome in education or rehabilitation and has been a guide about personal values (Schalock, Bonham, & Verdugo, 2008). QOL has been the basis of an increasing amount of research focused on understanding the underlying concept of QOL in the field of intellectual disability (ID; Claes, Van Hove, Van Loon, Vandevelde, & Schalock, 2010). QOL is “a multidimensional phenomenon composed of core domains influenced by personal characteristics and environmental factors. These core domains are the same for all people, although they may vary individually in relative value and importance” (Schalock, Keith, Verdugo, & Gómez, 2010, p. 21). The concept is progressively being used as a sensitizing notion, social construct, and overarching theme for planning, delivering, and evaluating personalized supports (Claes et al., 2012a, Claes et al., 2012b, Schalock et al., 2011, Van Loon et al., 2013). The QOL framework aims to meet challenges and overcome barriers that people with ID have been facing, as well as to improve public policies or service practices to address their needs and choices.

In the field of ID, different QOL frameworks have been reported in the literature and “there is usually no indication that the component parts have some form of hierarchical structure or causal sequence” (Cummins, 2005, p. 701). Each approach has an impact on the QOL assessment and many tools have been developed for this purpose (e.g., Van Loon et al., 2009, Verdugo et al., 2010, Verdugo et al., 2014). Whereas there is a consensus that QOL is a multidimensional concept, no common conceptualization has been accepted about the number of the core domains and indicators (Simões, Santos, & Claes, 2015a).

Our research is based on Schalock and Verdugo's (2002) conceptual framework, because this construct of QOL is one of the most cited, used, and has further provided a pivotal impact on research or practice related to people with ID (Gómez, Verdugo, Arias, & Arias, 2011). According to Schalock and Verdugo (2002), the concept of QOL: (a) is multidimensional, (b) has objective and subjective indicators, (c) has etic (universal) and emic (culture-bound) properties, and (d) is influenced by personal and environmental characteristics. The authors also suggested that QOL is composed by eight core domains, including: personal development, self-determination, interpersonal relations, social inclusion, rights, emotional well-being, physical well-being, and material well-being. The domains (i.e., latent variables) and respective indicators (i.e., observed variables) determine the construct of QOL, yet domains and indicators’ importance vary upon individual preferences or desires (Claes et al., 2010, Schalock et al., 2008, Schalock and Verdugo, 2002). There has been consistent verification and validation of this multidimensional model of QOL (e.g., Carbó-Carreté et al., 2015, Gómez et al., 2011, Jenaro et al., 2005, Schalock et al., 2005, Wang et al., 2010).

The analysis of the factor structure or hierarchical nature remains a critical element of the QOL construct (Gómez et al., 2011, Wang et al., 2010). It seems to be enough evidence of the eight core QOL domains, but there are few studies about the factor structure validation (Gómez et al., 2011). Furthermore, although only one first-order structure (i.e., multidimensional model) exists, there are two potential hierarchical factor structures (i.e., Salamanca versus Schalock). Both solutions have three slightly different second-order constructs. The Salamanca model (Jenaro et al., 2005) consists of personal well-being (i.e., emotional well-being, interpersonal relations, self-determination, and personal development), physical and material well-being, and empowerment (i.e., rights and social inclusion). The Schalock model (Schalock et al., 2005) consists of independence (i.e., personal development and self-determination), social participation (i.e., interpersonal relations, social inclusion, and rights), and well-being (i.e., emotional well-being, physical well-being, and material well-being).

The first unique contribution of the current research is related to the context in which the data were collected (i.e., Portugal). In some European countries, the need to know more about QOL has been arising due to the fact that this concept is paramount to the Quality Certification process and the intervention strategies undertaken by service organizations that provide support to the people with ID (Simões & Santos, 2014), which require the use of QOL assessment instruments. Consistently, the QOL assessment is now emerging in the Portuguese context as a way to promote a new source for establishing personalized programs and intervention design, which should be more focused in individual perspectives and needs of each person with ID (Simões et al., 2015a). In addition, Portuguese people with ID experience restrictions in participating in social activities and limitations due to environmental barriers (Santos, 2014). Thus, a validated scale based on the principles for assessing QOL among people with ID is of paramount importance in Portugal.

There are several instruments to measure QOL but, most of them, are not validated or do not relate to a clearly articulated QOL theory (Verdugo et al., 2014). Furthermore, scales assessing the QOL need to be rigorously validated (Li et al., 2013, Townsend-White et al., 2012). In our country persists the inexistence of suitable scales to evaluate the QOL of people with ID. The Escala Pessoal de Resultados (EPR) is a Portuguese version of the Personal Outcomes Scale (POS) (Claes et al., 2010, Van Loon et al., 2009). The EPR was developed for the purpose of assessing specific indicators associated with each of Schalock and Verdugo's (2002) domains (see Table 1). When developing the EPR, two initial researches were conducted to verify its validity and reliability properties (Simões and Santos, 2014, Simões et al., 2015b). This was the first instrument specifically developed/adapted for Portuguese adults with ID.

Table 1

Quality of life conceptual framework.

Domains	Indicators
Personal development	Education status; personal skills; adaptive behaviour; ADLs; IADLs
Self-determination	Choices/decisions; autonomy; personal control; personal goals
Interpersonal relations	Social networks; friendships; social activities; relationships
Social inclusion	Community inclusion and participation; community roles
Rights	Human (respect, dignity, and equality); legal (legal access and due process)
Emotional well-being	Safety and security; positive experiences; contentment; self-concept; lack of stress
Physical well-being	Health status; nutritional status; recreation/physical exertion
Material well-being	Financial status; employment status; housing status; possessions

Note. ADLs = Activities of daily living; IADLs = Instrumental activities of daily living.

The second unique contribution of this study is related to the verification of the multidimensional or hierarchical structure of the QOL concept in the Portuguese population with ID. First, the multidimensional solution enables to understand the QOL construct's meaning and boundaries (Schalock et al., 2010). This is especially important because “the implementation of individualized supports has led to the expectation of enhanced personal outcomes, which are typically related to QOL domains and indicators” (Schalock et al., 2008, p. 184). Second, the hierarchical solution of the QOL construct enables to understand how the domains are correlated to each other and to summarize the ratings of adults with ID (Wang et al., 2010). Thereby, the analysis of the two higher factor structures (i.e., Salamanca and Schalock) is crucial to evaluate the best goodness-of-fit of these second-order solutions in the Portuguese context.

The last unique contribution is related to using two parallel versions of data. The active participation of people with ID remains a critical element in the assessment and validation of the QOL construct. In the last few decades, there has been a proliferation of self-report measures for people with ID, who became essential in this field of research because they take an active part in assuming their role as citizens (Simões & Santos, in press). Assessing the individual's perceptions about their own personal outcomes is only available through self-report measures (Cummins, 2005, Perry and Felce, 2002). Although personal viewpoints have been recognized as key elements of QOL (Simões & Santos, 2014), people with ID have communication difficulties, problems for understanding the questions or providing responses (Balboni et al., 2013, Claes et al., 2012a, Claes et al., 2012b, Simões and Santos, in press). Strictly speaking, the way how the questions are formulated is of crucial importance for assessment purposes. The QOL scales should use a simple, clear, and brief language that could be understood by people with ID (Simões et al., 2015b). Report-of-others may be used, in combination with self-report measures, to provide an estimate of the accuracy of people with ID report (Straughen, Caldwell, Osyka, Helmkamp, & Misra, 2013) or to provide information about their different living environments (Simões & Santos, in press). Nevertheless, some researchers have reported concerns about the precision of such data (Perry & Felce, 2002).

The QOL scales for people with ID should assess each indicator by two methods: subjectively, involving the person himself or herself as the primary respondent; and objectively, on the basis of proxy reports of the person experiences and circumstances (Van Loon et al., 2009). This procedure allows observing potential differences between people with ID and support staff or family members (Claes et al., 2012a, Claes et al., 2012b). Nevertheless, the majority of the studies only used proxy measures and has not included the viewpoints of the people with ID. Previous research showed the adequateness of eight domains for objective assessment (Gómez et al., 2011), but not for a subjective evaluation. Therefore, evidences of subjective measurement of QOL are still needed. In this sense, examining both self-report and report-of-others perspectives may represent an important step to increase the continued understanding of how to assess and improve QOL among people with ID.

This study addresses the following aims: (a) to analyze the first-order factor structure of the EPR, and (b) to compare two alternative models of the higher factor structure of the QOL construct (i.e., Salamanca and Schalock). Both aims were checked by examining self-report and report-of-others measures. We hypothesized that (a) QOL should be a multidimensional construct supported by the first-order factor structure, (b) QOL should demonstrate a suitable higher factor structure, and (c) both self-report and report-of-others measures should reveal two aforementioned QOL structures (i.e., multidimensional and hierarchical). Despite the debate that emerges internationally, to our knowledge, this research is the first that tries to explore the factor solution of a QOL scale with Portuguese people with ID. Moreover, this information is important, as there are no data available in the original POS (Claes et al., 2010, Van Loon et al., 2009).

Method

Setting and participants

The sample comprised 1,264 adults with ID who had been previously diagnosed by a multidisciplinary team, according to the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, Text Revision (American Psychiatric Association, 2000). The inclusion criteria of the participants included having a formal diagnosis of ID and being 18 years or older. The participants’ age ranged from 18 to 66 years (M = 31.36; SD = 10.55). Seven-hundred and sixty-nine participants (60.84%) were diagnosed as having mild ID and 495 (39.16%) as having moderate ID. All adults had the verbal skills to answer the self-report measure. Six-hundred were female (47.47%) and 664 were male (52.53%). Less than one-tenth lived in their own home (n = 93; 7.36%) and about one-fifth lived in care facilities (n = 247; 19.54%), and the majority lived in family homes (n = 924; 73.10%). Their daytime activity was classified as vocational training (n = 481; 38.05%), occupational activities (n = 558; 44.15%), and activities developed with the family members (n = 87; 6.88%). The remainder adults were employed, having paid jobs (n = 138; 10.92%).

Respondents on EPR report-of-others (n = 333) were members of the support staff (n = 224) or family members (n = 109). The support staff had known the person with ID for at least two years. The age of the key institutional workers ranged from 22 to 64 years (M = 41.46; SD = 9.87). Regarding gender, 179 were female (79.91%) and 45 were male (20.09%). Support staff included monitors (n = 129; 57.59%), psychologists (n = 56; 25.00%), psychomotor/occupational therapists (n = 16; 7.14%), social workers (n = 12; 5.36%), and special education teachers (n = 11; 4.91%). The age of the family members ranged from 21 to 89 years (M = 57.37; SD = 13.64). Seventy-four were female (67.90%) and 35 were male (32.10%). More than half were the respective mothers (n = 56; 51.38%) and the remainder were fathers (n = 24; 22.02%), siblings (n = 15; 13.76%), and other relatives (n = 14; 12.84%).

Instrument

The QOL assessment was made by the EPR (Simões and Santos, 2014, Simões et al., 2015b). The EPR structure is divided into two sections: the self-report part (a set of items to be answered by people with ID) and the report-of-others part (the same set of items, but to be answered by proxies who had known the person for at least two years). Each part has forty-eight items including six items per domain. All items from the EPR were measured on a three-point Likert scale (i.e., 3 = always, 2 = sometimes, 1 = seldom or never).

Similar to the POS (Claes et al., 2010, Van Loon et al., 2009), the scale is focused on the assessment of QOL of individuals with ID aged 18 and over. The EPR is a multidimensional tool based on the eight core QOL domains (see Table 1) proposed by Schalock and Verdugo (2002). The exploratory factorial analysis has supported this multidimensional structure of the EPR (Simões et al., 2015b). These indicators were cross-cultural evaluated through content validity procedures in a Portuguese sample of adults with ID (Simões & Santos, 2014). According to Simões et al.’s (2015b) scores, the EPR revealed adequate test-retest (i.e., r's ranged from .67 to .92), internal consistency (i.e., α = .87 for self-report and α = .90 for report-of-others), and inter-rater reliability (i.e., r's ranged from .40 to .88).

Procedure

The Ethic Committee of Centro Hospitalar de São João approved the research and the World Medical Association's (2008) ethical principles were guaranteed. Oral and written information about the study were given to all participants. Participants who wished to participate signed an informed consent form prior to becoming participants of the study and were free to leave at any time. Agencies providing services to adults with ID were asked by email to collaborate in the research. The sample was conveniently obtained from 45 Portuguese organizations that agreed to participate in the study. Individuals with ID, support staff, and family members were contacted by key stakeholders. The EPR was administered through an interview by qualified professionals who had been previously trained to administer the scale. Training sessions were performed to practitioners who agreed to contribute as interviewers. Specific training was given concerning the ethical principles and the importance of QOL in the ID field, as well as administration guidelines of the EPR. Before the participants answered the questions, the interviewers read standard instructions and collected socio-demographic data. The scale takes approximately one hour to one hour and a half to administer and was applied in Portuguese organizations or in family homes.

Data analysis

Data were analyzed using SPSS 21.0 and AMOS 21.0. A confirmatory factor analysis (CFA) was conducted to verify the factor structure of the EPR. As the EPR is a 3-point numeric scale, we used the optimal scaling procedure. This method allows “multivariate analysis when the data do not satisfy the classic quantitative measurement requirements but are qualitative” (Meulman, Kooij, & Heiser, 2004, p. 67). First, the outliers were analyzed through the squared Mahalanobis distance (Byrne, 2000, Marôco, 2014). Next, a CFA with the Unweighted Least Squares (ULS) estimation method was conducted to confirm the measurement model (Verdugo et al., 2015). The validity of QOL constructs was estimated through the composite reliability, the average variance extracted (AVE), and the discriminant validity (Marôco, 2014). The appropriateness of the model was assessed through a variety of absolute and incremental goodness-of-fit indexes (see Results section).

Results

The first phase of our research involved examining the first-order measurement model of QOL. On the second stage, two alternative models of the higher order factor structure were analyzed (i.e., Salamanca and Schalock models). To address the principles for assessing QOL on people with ID (Claes et al., 2010), the results of the research were presented with two major data sets.

Assessment of the first-order factor models

Based in the literature (Claes et al., 2010, Van Loon et al., 2009), a hypothesized model was proposed on each EPR section (i.e., self-report and report-of-others), including eight first-order domains and forty-eight indicators. The CFA results showed that the factor loadings of eight items failed to exceed the cut-off point of .40 (Matsunaga, 2010). Table 2 presents the items removed after the CFA using the self-report measure, as well as additional reasons for deleting the items ensuring the QOL construct. Likewise, the same eight items with lower scores with self-report section yielded a cut-off point lower than .40 with report-of-others section. As Table 2 also shows, the item-reduction seems to be related to the existence of unnecessary questions (Hooper et al., 2008, Matsunaga, 2010). Consequently, these eight items were eliminated in both QOL sections, because they did not provide theoretical consequences since the measurement model has all indicators presented in Table 1. The reduced item scale-data (i.e., with forty items) became the measurement model.

Table 2

Items removed by the confirmatory factorial analysis.

Domains	Self-report items	Additional reasons
Personal development	Do you have an opportunity to demonstrate skills you have?	In emotional well-being domain the CFA supported a similar item: “Do you feel successful in the things that you do?”
Self-determination	Do you control at least part of your money?	In material well-being domain the CFA supported a similar item: “Do you have enough money to make choices (e.g. what to wear, what to buy)?”
Interpersonal relations	Do you know whom to ask for help, advice or supports if you need it?	The remaining five indicators of the interpersonal relations domain supported by the CFA were also related with the social networks (e.g., contact or relationships with friends and family)
Social inclusion	How many neighbors in the area do you know by name?	In social inclusion domain the CFA supported a similar item: “Do you talk to or visit people living near you?”
Rights	Are you allowed to be together as much as you want with your partner/friend/girlfriend?	In rights domain the CFA supported a similar item: “Can you have a girlfriend or boyfriend if you want?”
Emotional well-being	How frequently do you express love, fondness, or affection toward others?	The remaining five indicators of the emotional well-being domain supported by the CFA includes safety and security, positive experiences, contentment, self-concept, and lack of stress
Physical well-being	Do you have concerns with being hurt or in pain?	The remaining five indicators of the physical well-being domain supported by the CFA includes health status, nutritional status, and recreation/physical exertion
Material well-being	Do you have the key for your home?	In rights domain the CFA supported a similar item: “Do you control the key to your home or apartment?”

Note. CFA = Confirmatory factor analysis.

First-order factor models: self-report measure

As Figure 1 illustrates, all factor loadings of self-report measure were statistically significant (p < .001), ranging from .42 (item 34) to .93 (item 2, 18, 24, and 27). Table 3 presents the composite reliability, the AVE, and the squared correlation test of discriminant validity for self-report version. The composite reliability, similarly to Cronbach's alpha coefficient, indicates the overall reliability of a collection of similar items (Arias, Verdugo, Navas, & Gómez, 2013). Scores of composite reliability above .70 were deemed reliable (Hair, Black, Babin, Anderson, & Tatham, 2005). All composite reliability coefficients were within Hair et al.’s (2005) standards for acceptable internal consistency ranging from .75 to .91. The AVE indicates how accurately the construct is measured (Arias et al., 2013). With exception to the domain of physical well-being, the AVE scores were equal or above the recommended cut-off point of .50 (Fornell and Larcker, 1981, Hair et al., 2005, Marôco, 2014). Thus, convergent validity of the self-report measure was generally accepted. Table 3 also shows the square correlations among constructs supporting discriminant validity between eight first-order constructs, given that all AVE scores exceeded the square correlations for each associated domain (Farrell, 2010, Fornell and Larcker, 1981, Hair et al., 2005).

Figure 1

The confirmatory factor analysis of eight first-order factors by the self-report measure.

Note. PD = Personal development; SD = Self-determination; IR = Interpersonal relations; SI = Social inclusion; R = Rights; EW = Emotional well-being; PW = Physical well-being; MW = Material well-being; E = Error.

Table 3

Composite reliability, average variance extracted, and square correlations by the self-report measure (below the diagonal).

Domains	CR	AVE	PD	SD	IR	SI	R	EW	PW	MW
PD	.90	.60	1
SD	.86	.51	.30	1
IR	.89	.56	.20	.16	1
SI	.85	.50	.23	.24	.47	1
R	.89	.57	.17	.14	.10	.11	1
EW	.87	.52	.00	.02	.09	.10	.02	1
PW	.75	.34	.04	.04	.11	.14	.02	.25	1
MW	.91	.63	.17	.13	.16	.15	.15	.05	.06	1

Note. CR = Composite reliability; AVE = Average variance extracted; PD = Personal development; SD = Self-determination; IR = Interpersonal relations; SI = Social inclusion; R = Rights; EW = Emotional well-being; PW = Physical well-being; MW = Material well-being.

In addition, the fit indexes of the first-order QOL construct on self-report measure are presented in Table 4. Due to the limitations of Chi-square (χ) when using large samples, we relied on the ratio of χ to its degrees of freedom (df; Hair et al., 2005), and a good fit was assumed with scores below 5.00 (Hooper et al., 2008). The Root Mean Square Error of Approximation (RMSEA) less than .06 was indicative of good fit (Byrne, 2000). Self-report results corroborate Hooper et al. (2008) and Kline's (2010) recommendations that the Normed Fit Index (NFI) and the Relative Fix Index (RFI) should be ≥ .95. The Root Mean Square Residual (RMSR) analysis, which should be less than .10 and low as possible (Hooper et al., 2008, Kline, 2010), was within the required scores. Finally, the Goodness of Fit Index (GFI) and Adjusted Goodness of Fit Index (AGFI) were higher than ≥ .95 (Hooper et al., 2008, Kline, 2010, Schreiber et al., 2006).

Table 4

The fit indices of quality of life models by the self-report measure.

Fit Indices	Multidimensional model	Salamanca model	Schalock model
χ2**	2147.86 (p = .000)	3572.86 (p = .000)	3451.98 (p = .000)
df	712	729	729
χ2/df	3.02	4.90	4.74
RMSEA	.048	.058	.055
RMSEA interval (95%)	.044–.053	.055–.062	.051–.059
NFI	.97	.94	.95
RFI	.97	.94	.95
RMSR	.051	.065	.064
GFI	.98	.96	.96
AGFI	.97	.96	.96
AIC	6045.89	6087.02	6049.11

Note. ** p < .001; χ = Chi-square; df = Degrees of freedom; RMSEA = Root Mean Square Error of Approximation; NFI = Normed Fit Index; RFI = Relative Fix Index; RMSR = Root Mean Square Residual; GFI = Goodness of Fit Index; AGFI = Adjusted Goodness of Fit Index; AIC = Akaike Information Criterion.

First-order factor models: report-of-others measure

As Figure 2 shows, all factor loadings for report-of-others were higher than .40 and were statistically significant (p < .001). The scores ranged from item 35 (λ = .41) to item 2 (λ = .95). Table 5 illustrates that the composite reliability of the first-order constructs were within the Hair et al.’s (2005) standards. The scores ranged from .72 (physical well-being domain) to .92 (personal development domain). Generally, proxies’ measure showed higher coefficients of composite reliability at the domain level, compared to self-report measure. Likewise, the same domain with lower AVE with self-report measure yielded lower AVE with report-of-others measure. The AVE results on proxies ranged from .31 to .66, therefore convergent validity was generally accepted. Analogous to people with ID findings, report-of-others scores supported the discriminant validity between eight first-order constructs (see Table 5).

Figure 2

The confirmatory factor analysis of eight first-order factors by the report-of-others measure.

Note. PD = Personal development; SD = Self-determination; IR = Interpersonal relations; SI = Social inclusion; R = Rights; EW = Emotional well-being; PW = Physical well-being; MW = Material well-being; E = Error.

Table 5

Composite reliability, average variance extracted, and square correlations by the report-of-others measure (below the diagonal).

Domains	CR	AVE	PD	SD	IR	SI	R	EW	PW	MW
PD	.92	.66	1
SD	.89	.58	.33	1
IR	.90	.62	.15	.17	1
SI	.88	.55	.23	.26	.54	1
R	.90	.60	.21	.20	.12	.16	1
EW	.85	.51	.02	.04	.22	.17	.01	1
PW	.72	.31	.03	.03	.15	.14	.00	.23	1
MW	.90	.62	.12	.19	.10	.15	.15	.06	.06	1

Note. CR = Composite reliability; AVE = Average variance extracted; PD = Personal development; SD = Self-determination; IR = Interpersonal relations; SI = Social inclusion; R = Rights; EW = Emotional well-being; PW = Physical well-being; MW = Material well-being.

The fit indexes for the first-order of the EPR report-of-others measure are present in Table 6. Similar to the results of self-report measure, data analysis also revealed suitable goodness-of-fit scores on proxies’ measure based on conventional guidelines (Byrne, 2000, Hair et al., 2005, Hooper et al., 2008, Kline, 2010, Schreiber et al., 2006).

Table 6

The fit indices of quality of life models by the report-of-others measure.

Fit Indices	Multidimensional model	Salamanca model	Schalock model
χ2**	1958.38 (p = .000)	3777.38 (p = .000)	3543.59 (p = .000)
df	712	729	729
χ2/df	2.75	5.18	4.86
RMSEA	.047	.060	.054
RMSEA interval (90%)	.043–.053	.056–.064	.048–.059
NFI	.98	.95	.95
RFI	.97	.94	.95
RMSR	.048	.069	.067
GFI	.98	.96	.96
AGFI	.98	.95	.96
AIC	5646.85	5683.83	5678.08

Note. ** p < .001; χ = Chi-square; df = Degrees of freedom; RMSEA = Root Mean Square Error of Approximation; NFI = Normed Fit Index; RFI = Relative Fix Index; RMSR = Root Mean Square Residual; GFI = Goodness of Fit Index; AGFI = Adjusted Goodness of Fit Index; AIC = Akaike Information Criterion.

Alternative models of higher order factor structure

Overall, the measurement model of the two EPR sections was within the required criteria and showed adequate psychometric properties. Consequently, further analyses were conducted with alternative models of higher order factor structure in order to examine the construct of QOL.

Second-order factor models: self-report measure

Additionally, it was also evaluated the best hierarchical structure (i.e., Salamanca or Schalock) on self-report measure. Table 4 shows that the RMSEA was the same (.06) for the second-order analyzed models, which was a suitable score (Byrne, 2000, Matsunaga, 2010, Schreiber et al., 2006). Nevertheless, incremental fit indexes (i.e., NFI and RFI) were higher in the Schalock model. According to several guidelines (Byrne, 2000, Hair et al., 2005, Hooper et al., 2008, Kline, 2010, Schreiber et al., 2006), the model showed acceptable goodness-of-fit results. Moreover, in the Salamanca structure we observed problems with the estimation of model’ parameters, and the covariance matrix was not positively defined. The Akaike Information Criterion (AIC) was also analyzed to compare the models, and smaller scores suggest the better fit of the data (Hooper et al., 2008, Kline, 2010, Schreiber et al., 2006). As Table 4 shows, the second-order model with the lowest AIC was also the Schalock hierarchical structure, and was regarded more suitable. However, multidimensional model showed higher absolute and incremental goodness-of-fit indexes, compared to Schalock model. Furthermore, AIC criterion suggests a better fit for the multidimensional model.

As Figure 3 illustrates, in Schalock model the factor loadings between the second-order constructs and the eight domains ranged from .40 to .79. Also, the factor loadings between the domains (i.e., first-order constructs) and the items (i.e., observed variables) ranged from .41 to .95. A high correlation between the second-order constructs was observed between social participation and well-being (Φ = .86). All scores were statistically significant (p < .001).

Figure 3

The second-order model of Schalock by the self-report measure.

Note. I = Independence; SP = Social participation; WB = Well-being; PD = Personal development; SD = Self-determination; IR = Interpersonal relations; SI = Social inclusion; R = Rights; EW = Emotional well-being; PW = Physical well-being; MW = Material well-being; D = Disturbance (structural error); E = Error.

Second-order factor models: report-of-others measure

Concerning to comparison of the second-order models (i.e., Salamanca and Schalock), the pattern of results on report-of-others measure was very similar to data collected through self-report ratings (see Table 6). In regard to the scores based on report-of-others: (1) the goodness-of-fit scores were higher in the Schalock model, (2) the Salamanca model showed problems with the estimation of model’ parameters, and (3) the multidimensional model was more suitable than the Schalock solution.

In report-of-others measure (see Figure 4), the factor loadings between the second-order constructs and the domains ranged from .48 to .85. The factor loadings between the domains and its respective items ranged from item 35 (λ = .38) to item 2 (λ = .94). The highest correlation was observed between social participation and well-being (Φ = .85). All scores were statistically significant (p < .001).

Figure 4

The second-order model of Schalock by the report-of-others measure.

Note. I = Independence; SP = Social participation; WB = Well-being; PD = Personal development; SD = Self-determination; IR = Interpersonal relations; SI = Social inclusion; R = Rights; EW = Emotional well-being; PW = Physical well-being; MW = Material well-being; D = Disturbance (structural error); E = Error.

Discussion

As there are few studies using the CFA with QOL scales in ID field, this research extends the knowledge in the literature by (a) developing a valid scale that Portuguese practitioners can use to improve QOL for individuals with ID, (b) confirming the factor structure used by previous studies (e.g., Gómez et al., 2011, Jenaro et al., 2005, Schalock et al., 2005, Wang et al., 2010) for examining QOL among people with ID, and (c) exploring the construct of QOL from both perspectives (i.e., self-report and proxies).

Regarding the first aim of examining the construct validity in order to use and implement the EPR based on standard guidelines, the CFA confirmed the same factorial structure of the scale in both parts (i.e., self-report and report-of-others). The CFA confirmed eight factors that have conceptual and empirical meaning, as well as practical importance: personal development, self-determination, interpersonal relations, social inclusion, rights, emotional well-being, physical well-being, and material well-being. The results of our research highlighted that QOL is a multidimensional construct, supported by the first-order measurement model, and has etic properties. Findings were similar to those found in other studies that employed the CFA technique (Gómez et al., 2011, Verdugo et al., 2010, Verdugo et al., 2014, Wang et al., 2010). In other words, statistic scores showed empirical support of the eight core QOL domains. Findings allow credibility to improve eight main dimensions on Portuguese individuals with ID.

However, only forty of the forty-eight items were supported on self-report and report-of-others measures. An explanation for the reduced item scale supported by CFA might be that the goodness-of-fit is negatively related with higher number of items (Matsunaga, 2010), and CFA technique reduces the number of observed variables (Schreiber et al., 2006). Furthermore, as highlighted by Matsunaga's (2010) criteria, quality items were selected and unnecessary questions were removed. Thus, “domain concepts are more clearly represented and their scores are more easily interpreted when items are clearly identified with the intended domain” (Skevington, Lotfy, & O’Connell, 2004, p. 303). Our main concern was that the items of QOL domains assess only the specific construct, based on evidence of discriminant validity. In this regard, “by deleting indiscriminant items fit is likely to improve and is advantageous in that it is unlikely to have any major theoretical repercussions” (Hooper et al., 2008, p. 56).

The QOL indicators of the EPR were chosen based on adequate factorial, discriminant, and convergent validity (Farrell, 2010, Fornell and Larcker, 1981, Hair et al., 2005, Marôco, 2014). The composite reliability scores showed that the coefficients were suitable in all domains (Hair et al., 2005). Nevertheless, scores with lower AVE were found in the physical well-being domain, in both sections of the EPR. It seems that these results were not influenced by the lack of understanding of the questions, once they were shared by adults with ID and proxies. Furthermore, it is important to note that the indicators were validated by different cross-cultural studies (e.g., Carbó-Carreté et al., 2015, Claes et al., 2010, Gómez et al., 2011, Jenaro et al., 2005, Schalock et al., 2005, Van Loon et al., 2009, Verdugo et al., 2010, Wang et al., 2010). Our findings were also consistent with previous CFA research of the GENCAT-scale (Gómez et al., 2011, Verdugo et al., 2010).

Verdugo et al. (2010) have observed that three items of the GENCAT-scale simultaneously contributed to physical well-being and other domains (i.e., rights and material well-being), which explains the lower scores on aforementioned dimension of QOL. Nevertheless, this explanation is not supported in our research, given that the QOL indicators were chosen for their discriminant properties. It seems that Portuguese results of the physical well-being domain can be explained because people with ID “present a greater variety of health disorders than individuals without such disabilities” (Schalock & Verdugo, 2002, p. 146). Additionally, those with ID do not properly identify their body image (Ayaso-Maneiro, Domínguez-Prado, & García-Soidan, 2014). Further research should also be performed to analyze if the scores of physical well-being domain can be influenced by the comorbidity of health disorders or the possible confusion between concepts (i.e., diagnosis of ID versus illness condition).

Concerning the second aim of the study, two second-order models (i.e., Salamanca and Schalock) were analyzed for examining the QOL among adults with ID. The CFA identified three higher factors of the QOL construct: independence (i.e., personal development and self-determination), social participation (i.e., interpersonal relations, social inclusion, and rights), and well-being (i.e., emotional well-being, physical well-being, and material well-being). Despite the results also supported a second-order solution, the model presented shortcomings. The high scores between the second-order constructs, observed in both self-report and report-of-others measures, showed the lack of discriminant validity of the second-order structure and, therefore, more caution should be given to this matter. This finding will represent a contribution to the continuous improvement of knowledge on QOL models.

Furthermore, the nature of the QOL models presents disagreements in the literature. Whereas Wang et al. (2010) have argued that QOL has a hierarchical structure, Gómez et al. (2011) stated that the hierarchical solutions are no needed because multidimensional first-order factor models explained the underlying construct. Our data supported the idea that QOL is a construct with eight first-order domains and further researches should be performed to improve suitable discriminant validity in the second-order model. Likewise Gómez et al.’s (2011) results, our findings confirmed that the first-order model was more robust than either of the two second-order models, but that the Schalock solution was stronger than the Salamanca solution. Practically speaking, two factor solutions have been confirmed, yet the multidimensional model is the one obtaining the better fit.

Our research involved examining the first-order or second-order models through self-report and report-of-others measures. The main assumption of this analysis was the huge discussion presented in the literature about from to whom obtain the QOL ratings. Similar to the original version (Claes et al., 2010, Van Loon et al., 2009), the EPR includes two parallel sections that tried to address twofold concerns in the field of ID: (a) the difficulty in obtaining self-report ratings of all potential participants (Balboni et al., 2013, Claes et al., 2012a, Claes et al., 2012b) and (b) the validity of proxy data (Perry & Felce, 2002). In addition, the literature presents some controversy about the disagreement (Perry & Felce, 2002) or suitable agreement (Balboni et al., 2013, Claes et al., 2010, Claes et al., 2012a, Claes et al., 2012b, Simões and Santos, in press, Van Loon et al., 2009) between people with ID and proxies’ information. Nevertheless, there is silence among the researchers about similarities/differences across the factor solutions based on different QOL respondents. Moreover, studies that employed the CFA techniques were mostly performed by proxies’ information.

The final measurement model supported the same indicators in both self-report and report-of-others measures. The results are encouraging because they show that people with ID are valid respondents in the QOL assessment. Furthermore, it is obvious the significance of getting the viewpoints of people with ID, whose interests the service system is supposed to attend. The people with ID should participate in their own assessment for exercising their right of self-determination (Verdugo et al., 2015). Our data supported that it is essential to get more than one perspective in the assessment process. It can be said that different perspectives between people with ID and proxies can be advantageous in terms of improving communication and problem solving. However, it is unacceptable to use proxy responses instead of self-reported QOL.

In summary, the EPR will retain the POS structure. First, the first-order structure confirmed eight core domains. Second, the second-order solution retained three QOL factors. Finally, the analyzed psychometric scores supported both self-report and report-of-others sections. To preserve a valid and reliable QOL assessment, the scores of domains, factors, and scale index should be calculated only with the forty items supported by the CFA.

The findings obtained in this research have implications for QOL assessment, services, and practices among people with ID. First, the development of QOL scales should include multi-perception strategies. Second, the indicators should be chosen, among other psychometric scores, by their discriminant properties. Accordingly, we have observable variables that effectively measure what is intended in each QOL domain. Third, researchers should continue to analyze the first-order and second-order structures, in order to understand the complexity of the construct.

Concerning to services and practices towards people with ID, the EPR is crucial for evaluation and intervention purposes. The QOL structures supported by our research (i.e., multidimensional and Schalock models) allow to reorganize the Portuguese services based on Van Loon, Claes, Vandevelde, Van Hove, and Schalock's (2010) approach: input (goals, choices, perspectives of people with ID), throughput (personalized support plans based on QOL construct), and output (enhanced personal outcomes related to the first-order and second-order factors). The QOL models should act as a guide to change the program practices and supports of people with ID (Claes, Vandevelde et al., 2012; Schalock et al., 2011, Van Loon et al., 2013). It is important to note that those with ID seek more than to be passive receivers of practitioners and need different supports to fully participate in society (Thompson, Schalock, Agosta, Teninty, & Fortune, 2014). It can be said that the main goals of services and practices should be developing the independence, social participation, and well-being with individuals with ID, including different perspectives reported by each person, family members, and support staff.

Nevertheless, the study has some limitations. The psychometric properties of the physical well-being domain suggest the need of scale refinement in future research. Although the selection of the indicators was derived from previous literature (Schalock & Verdugo, 2002) and the original POS (Claes et al., 2010, Van Loon et al., 2009), one can argue that future research should include additional new items pertaining to a Portuguese culture. As a convenience sample was used with mild and moderate levels of ID, further research is needed to confirm the results in other levels of ID.