Early intervention and perceived quality: Refinement of the inventory of quality in early intervention centers.

The current perspective on early intervention revolves around consideration of the family as a cornerstone, its opinion being essential in providing a quality service. Early intervention centers require an evaluation of the services they perform. The aim of this study was to examine the psychometric properties of the short version of the Inventory of Quality for Early Intervention Centers (IQEIC) and to obtain evidence of its validity and reliability. The sample consisted of 887 families from 21 early intervention centers in Spain, which were randomly divided into 2 groups to conduct a cross-validity analysis: exploratory factor analysis with the first group (n1 = 440), and confirmatory factor analysis with the second group (n2 = 447). A 8 factor structure was obtained in the confirmatory factor analysis that showed a good fit. Both the internal consistency (composite reliability ranging from 0.84 to 0.90) and the convergent (AVE values ranged from 0.12 to 0.50) and discriminant validity were adequate. Lastly, a multigroup analysis (n1 and n2) showed the invariance factorial through the difference in the CFI index. The IQEIC showed satisfactory reliability and validity in this study confirming the proposed model is a valid tool to assess the quality of the service provided in early intervention centers, therefore recommending its application for both research and management.

RCT Entities:   Population Interventions Outcomes

Introduction

Early intervention (EI) is a comprehensive service provided for children with developmental delay or for those at a high risk of developmental delay, to improve their cognition, emotion, behaviour, and ability to adapt to the society.[ It implies the active participation of families in the intervention process,[ although the family has generally been considered as a resource rather than as part of a global intervention objective.[ However, the current perspective revolves around consideration of the family as a fundamental axis in child health care, and specifically in EI services, advocating that the family is the expert on its children and makes decisions according to their needs and characteristics, supported by the professional in the process of attending to the child.[ This change is reflected in the objectives of new research, where professionals are the means of ensuring that children and families continue to benefit from the progress made in this area.[ According to McCollum[ and Shonkoff,[ EI consists of multidisciplinary services to promote child health and wellbeing, enhance emerging competencies, minimize developmental delays, remediate existing or emerging disabilities, prevent functional deterioration, and promote adaptive parenting and overall family functioning.

Currently in Andalusia, the geographical area in the South of Spain where the study sample was obtained, EI programs are provided in EI centers (EIC), where a multidisciplinary team provides services aimed at serving children with developmental disorders, or at risk, as early as possible, from birth to 6 years of age.

Quality assessment is an essential aspect of health organizations at present, from the point of view of both providers and service users.[ Incorporation of the opinion of the user into the functioning of a health system is not something new.[ From the users’ perspective, satisfaction analysis addresses their cognitive assessment of and emotional reaction toward their experience with health services.[ Most health service satisfaction assessment proposals are based on marketing theories, identifying the differences between expectations and perception of service.[

Parasuraman et al[ define perceived quality as the difference between expectations and consumer perceptions with regard to a service. The authors defined 5 aspects of any service that influence the quality, and popularized SERVQUAL. Other authors suggest that, depending on the type of business, the dimensions of service quality may be considered to be different.[

However, organizational systems that manage quality often operate simply on the basis of rational measurement, without regard for intangible or relational dimensions. Management models must take into account experiential aspects to provide more stable theoretical building blocks that help to understand and manage quality in daily clinical practice.[ These aspects are particularly relevant in EI, given the characteristics of children with a developmental disorder, since their progress can be seriously compromised.[ In this sense, families, as external customers of the EIC, are considered an indispensable element of care and quality improvement programs.[

From this perspective, specific tools have been developed to measure different aspects of EI, including the Early Intervention Services Assessment Scale (EISAS)[, the 2 scales proposed by Fisher et al,[ or the Systematic Exploration and Process Inventory for health professionals in early childhood intervention services.[

After a review of the scientific literature, a lack of service quality studies in EI is detected. It is necessary to propose improvements in services based on measurements performed using validated instruments.[ In this context, and given that work on perceived quality in EI is scarce, an evaluation tool that allows collection of relevant aspects about the service quality provided in the EIC is missing. Therefore, the IQEIC was proposed in order to gather families’ perceptions as users of these centers. The gathered data can be a key element in evaluation and subsequent decision-making regarding quality improvement activities. Therefore, the main objective of this research is the validation of a short scale to measure the quality perceived by users/families at the EIC.

Methods

Participants

The population was composed of families of children between 0 and 6 years old who received EI for at least 1 month in an EIC in Andalusia (southern Spain). A margin of error of 5% was used and a confidence level of 99% (Z = 1.96) was assumed for the selection of the sample. Finally, a total of 887 families participated in the study, 73.1% (648) being female and 23.7% (210) male, with ages ranging from 20 to over than 70 years, most often in the age range 30 to 39 years (n = 498; 56.1%). In terms of kinship, mothers represented 70.7% (n = 628) and fathers 21.7% (n = 193) (Table 1).

Table 1

Sample distribution and characteristics.

Measures

We used the Inventory of Quality in Early Intervention Centers,[ constructed from the Questionnaire of Perceived Quality in Early Intervention Physiotherapy[ and the Satisfaction of the Client Family Questionnaire.[ This instrument is made up initially of 48 items and 6 dimensions: center facilities (13 items), treatment rooms and materials (9 items), user services (7 items), qualified staff (8 items), general information (5 items), and technical or specific information (6 items). The response format for all items is a 5-point Likert scale rated from 1 (strongly disagree) to 5 (strongly agree). In the present study, the general information dimension was removed due to the collection of anachronistic information in the current process of referral to de EIC.

In the original study,[ the dimensions showed good reliability (α values higher than 0.70, with the exception of the general information dimension), the lower value justified by the small number of items of the scale. Exploratory factor analysis was performed independently for each dimension and offered 10 factors, showing a Kaiser–Meyer–Olkin (KMO) index >0.80 and a total variance explained above 50%, with the exceptions of the user services and general information dimensions.

Procedure

The researchers contacted 21 EICs located in the province of Malaga (Andalusia, Spain), offering one meeting with managers who were interested, with the purpose of explaining the objectives and the methodology of the study, the questionnaire and the data collection. Data were collected using the IQEIC, including a demographic characteristics section. The evaluators attended 2 training sessions prior to the start of the assessment period and visited the EIC every day over one month to deliver the questionnaires to participants, and they would read the different items so that the participants could mark the answer in the questionnaire, thus avoiding unanswered questions. Participants provided informed consent after reading the description of the study in accordance with the Declaration of Helsinki, and the anonymity of confidentiality of their responses was assured. Each participant took ten to fifteen minutes to complete the questionnaire. Permission to undertake this study was gained from the Research Ethics Committee at Malaga University (code 32-2017-H).

Data analysis

A cross-validation analysis was performed, a usual procedure in psychometric studies. To do this, the dimensionality of the ICCAIT was analyzed, splitting the sample into 2 random subsamples.[ With the first half (n1 = 440; 310 female: 70.5%; 104 male: 23.6%; 26 did not disclose: 5.9%), an exploratory factor analysis (EFA) was conducted utilizing principal components extraction and oblique rotation was applied to interpret the obtained factors. In order to analyze the factor loadings of every item, 0.40 was taken as the recommended cut-off point.[ We had previously tested the factorization conditions using the Bartlett and KMO tests. The second subsample (n2 = 447; 338 female: 75.6%; 106 male: 23.7%; 3 did not disclose: 0.7%) was analyzed using confirmatory factor analysis (CFA) to test the adjustment of the structure obtained in the first half, using the maximum likelihood estimation (ML) method. The fit indices utilized were as follows: the comparative fit index (CFI), Tucker–Lewis index (TLI), and incremental fit index (IFI),[ where values above 0.90 indicate good fit. With root mean square error of approximation (RMSEA), values equal or lower to 0.06 indicate a well-fitting model,[ while with the χ2/degree of freedom ratio, values below 3.0 are considered a good fit.[ Factor loadings of 0.50 and above, with significant p-values, and modification indices (MI), were used to locate any problematic items that contributed to misfit to the data.[

In addition, the internal consistency of the constructs was measured through composite reliability (CR).[ Convergent validity was evaluated through the average variance extracted (AVE), while discriminant validity was established when AVE for each construct exceeded the squared correlations between that construct and any other.[

Finally, a multi-group analysis was performed to test the difference between 2 invariance models.We do not rely on the χ2, as it is judged to be too restrictive; instead we rely on the change in CFI value (ΔCFI[), which has to be lower than 0.01 so that the hypothesis of invariance is not rejected.[ The Chi-square difference test was also reported for the comparison. Data were analyzed with Statistical Product and Service Solutions 20.0 (SPSS 20.0) and Amos Graphics statistical software 20.0.

Results

Exploratory factor analysis

The results for the EFA were adequate in the first subsample (n1 = 440). KMO measure (KMO = 0.94) and significance of Bartlett test (χ2 = 11477.23; df = 903; P < .001) revealed that the items had adequate common variance for factor analysis. Then, principal components analysis using oblimin oblique rotation yielded 8 factors explaining 64.08% of common variance across items. Reliability coefficients for all the obtained factors were higher than 0.70[, with values between 0.77 and 0.91, the global scale being at 0.94.

Confirmatory factor analysis

We used the other half of the sample (n2 = 447) to conduct CFA and cross-validate model obtained in the EFA. The adjustment showed a poor fit to the data in the 8-factor model with 43 items and 5 dimensions [χ2(794) = 2052.59; χ2/gl = 2.58; RMSEA = 0.060 (CI = 0.057–0.063); CFI = 0.89; TLI = 0.87; IFI = 0.89]. Although the RMSEA value was indicative of a good fit, the CFI, TLI and IFI values were below the threshold of 0.90.[ These results indicated that the model should be adjusted. For this purpose, a revision was made based on the factor loadings and modification indices. After selecting the items whose factor loading was greater than 0.50,[ items 3, 4, 9, and 13 (center facilities) were deleted, as well as items 24 to 29 (user services). Item 23 (“the attention that the user receives in the center is adequate”) of the customer care dimension moved to the qualified staff dimension. Finally, several items showed high modification indexes (MI) with different factors, so we proceeded to eliminate items 7 (MI = 70.60), 8 (MI = 134.82), 10 (MI = 17.45), 16 (MI = 44.55), 17 (MI = 23.01), 19 (MI = 24.29), and 34 (MI = 46.80). The result was a scale composed of 4 dimensions and 26 items with an acceptable fit to the data in confirmatory factor analysis [χ2(263) = 667.49; χ2/gl = 2.53; RMSEA = 0.059 (CI = 0.053–0.064); CFI = 0.94; TLI = 0.92; IFI = 0.94]. According to these results, the modified model had an excellent fit.[ All items showed high factor loadings, ranging from 0.53 to 0.93, providing evidence that each item appropriately captured its respective factor (Table 2).

Table 2

Factor loadings (λ), composite reliability (CR) and average variance extracted (AVE).

Internal consistency was assumed for all constructs, with composite reliability values greater than value 0.80,[ specifically ranging from 0.84 to 0.90. The AVE and squared correlation tests for discriminant validity are reported in Table 3. The AVE values were adequate and the squared correlation values ranged from 0.12 to 0.50, indicating discriminant validity in all constructs.

Table 3

Average variance extracted (AVE) and discriminant validity.

Multigroup validation

A multi-group CFA analysis was conducted with the testing sample (n1 = 440) and the validation sample (n2 = 447). The fit of the unconstrained model was acceptable [χ2(526) = 1330.08; χ2/gl = 2.53; RMSEA = 0.04; CFI = 0.94; TLI = 0.92; IFI = 0.94], and for the model with constrained factor loadings [χ2(543) = 1385.05; χ2/gl = 2.55; RMSEA = 0.04; CFI = 0.93; TLI = 0.92; IFI = 0.94]. As the CFI change between the models was lower than 0.01, we could assume factor loading invariance between the 2 groups.[

Discussion

To measure quality of service is not an easy task, and sociocultural variables surrounding participants should be taken into account during its measurement, since these subjects may have different interpretations of what quality means. As a consequence, it should be remembered that assessing the quality of the service in different areas and taking a unique scale or universal questionnaire as a reference is not possible.[

The purpose of the current study was to validate the reduced version of the Inventory of Quality in Early Intervention Centers, a questionnaire specifically designed to measure the quality perceived by families served in EICs. This questionnaire can be used in any EIC, regardless of the number of professionals or disciplines that make up the model, this feature being one of the advantages of the instrument above others, as proposed by Medina et al.[ Similarly, the Systematic Exploration and Process Inventory for health professionals in early childhood intervention services[ only collects information from the EIC professionals and evaluates not the quality of service but the needs and resources of families attending EI services. Also, the EISAS,[ presented as an instrument that measures the quality of the practice of general EI services, has not yet been validated. On the other hand, Fisher et al[ proposed 2 scales which collect information from parents and which evaluate only intervention programs in EI, one of them with 135 items. Even so, these did not collect other essential aspects of the service, such as the intangible elements.

The relevance of measuring perceived quality in EI services is that it allows identification of opportunities for improvement. Overestimation of the quality of care in an EIC can perpetuate a suboptimal performance, while exact measurements of the current action and a realistic comparison with other more successful centers could provide the motivation to improve the quality, as stated by Palazón-Cabanes,et al.[

According to the results obtained, it can be concluded that the IQEIC-R is a pragmatic and reliable tool that measures the perceived quality of service in EICs, encompassing both the healthcare and nonhealthcare processes provided in the EIC, since it covers all aspects of the service from the family arriving at the EIC to the child leaving. Moreover, the tool has the advantage of a small number of items: it can therefore be answered in a short time, estimated to be around 10 minutes.

Assessment of perceived quality is a complex, multifaceted area of study, and collecting information about satisfaction with services is routine within service systems. Since researchers usually do not report or share the study’ s results with their participants, in future research we propose to learn what the collected information means, how it is used by providers, and how families might use it to advocate changes to existing services and support, which could be empowering. This could benefit both the consumer and the provider agency, as stated by Copeland et al,[ Jemes-Campaña et al.[

One of the possible limitations of the study is the lack of evaluation of sensitivity to changing the instrument, although this was not the aim and the design does not allow it. It is considered that the administration of this questionnaire on several occasions is required in order to assess the response to changes in the centers to improve the quality of service offered. Also, the factor structure should be checked in other health care contexts where EI is organized within different parameters for example, in other European countries or even in the United States of America. An analysis of factorial invariability is not conducted according to sociodemographic characteristics such as age or gender, which could cause changes in perception of the perceived quality of the construct. In this sense, and regarding future lines of research, a factor invariance analysis could be of interest, given potentially different interpretations in different cultural and social contexts. As it stands, this type of study is in its beginning stages, affording numerous investigative opportunities.

Conclusion

In conclusion, this study demonstrates that the short version of the IQEIC is a reliable, reproducible, and valid tool to evaluate the quality of the service perceived by the users of the Spanish Early Intervention Centers.

Author contributions

Conceptualization: Rita Pilar Romero-Galisteo, Ángel Blanco-Villaseñor, Noelia Moreno-Morales, Pablo Gálvez-Ruiz.

Data curation: Rita Pilar Romero-Galisteo, Noelia Moreno-Morales.

Formal analysis: Ángel Blanco-Villaseñor, Pablo Gálvez-Ruiz.

Methodology: Ángel Blanco-Villaseñor, Pablo Gálvez-Ruiz.

Resources: Rita Pilar Romero-Galisteo.

Software: Pablo Gálvez-Ruiz.

Supervision: Pablo Gálvez-Ruiz.

Validation: Ángel Blanco-Villaseñor, Noelia Moreno-Morales, Pablo Gálvez-Ruiz.

Visualization: Pablo Gálvez-Ruiz.

Writing – original draft: Rita Pilar Romero-Galisteo, Ángel Blanco-Villaseñor, Noelia Moreno-Morales, Pablo Gálvez-Ruiz.

Writing – review & editing: Rita Pilar Romero-Galisteo, Ángel Blanco-Villaseñor, Noelia Moreno-Morales, Pablo Gálvez-Ruiz.