The design and validation of a parent-report questionnaire for assessing the characteristics and quality of early intervention over time.

This article concerns a parent-report repeat questionnaire to evaluate the quality of multiprofessional early intervention following early identification of deafness. It discusses the rationale for the design of the instrument, its theoretical underpinnings, its psychometric properties, and its usability. Results for the validity and reliability of the instrument are based on completion by 82 parents. The questionnaire is divided into four sections. "The description of the structure of professional services" demonstrated good face and content validity; the "content of intervention scale" yielded high internal consistency (Cronbach's alpha = 0.88) and reliability (6-month test-retest correlations, rho = 0.88, r = .68; 12-month test-retest correlations, rho = 0.60, r = .82); the "process of intervention" scale yielded high internal consistency (Cronbach's alpha = 0.93) and high reliability on test-retest administration (6-month test-retest correlations, r = 0.64; 12 month test-retest correlations, r = .82); and the short "overall impact" questions were answered well. The Trait Emotional Intelligence Questionnaire was used to control for influence of parental disposition on ratings of quality of intervention. Evaluating the goodness of fit between early intervention and parental priorities/values is discussed as a vital component in improving child and family outcomes.

This paper concerns parental assessment of the quality of multiprofessional early intervention services following the early identification of their children's deafness through universal newborn hearing screening. Assessing the quality of early intervention in this context is of particular significance because the developmental advantages associated with the detection of childhood deafness are firmly predicated on early identification being combined with quality early intervention (Yoshinaga-Itano, 2003; Yoshinaga-Itano, Coulter, & Thomson, 2001). Early detection leading to early diagnosis is of little consequence unless deaf children and their families receive services which are able to optimize the advantages of that very early recognition (Young, 2009; Young & Tattersall, 2007).

However, the issue of what counts as “quality” early intervention is in some respects very well developed and, in others, very poorly understood. If one considers quality from the perspective of structural features, that is, frameworks that support services (Aytch, Cryer, Bailey, & Selz, 1999), then the introduction of universal programs of newborn hearing screening has helped promote better specified and coordinated programs of early intervention, as well as better training and skills recognition for those who deliver them (Bodner-Johnson & Sass-Lehrer, 2003; Stredler Brown 2005; White, 2003). The US Joint Committee on Infant Hearing (JCIH) (2007) position statement contains sets of principles (such as confirmation of hearing loss by 3 months of age and commencement of early intervention by 6 months of age) as well as nine quality indicators (including the involvement of deaf and heard-of-hearing people in service planning and provision) (JCIH, 2007). In England, a comprehensive quality assurance mechanism is in place associated with both screen and follow-up multiprofessional intervention services. It takes the specification of quality standards to another level, in that it actively monitors the delivery of those standards (Newborn Hearing Screening Programme [NHSP], 2008).

Similarly, if one considers quality from the perspective of process features, that is, how services are provided (Aytch et al., 1999; King, Rosenbaum, & King, 1996) then universal newborn hearing screening has been a considerable catalyst for focusing attention on philosophy, style, and approach in working with families. A considerable literature now exists on, for example, the promotion and practice of informed choice (Young et al., 2006; Young, Carr, Hunt, Tattersall, & McCracken, 2008) and family centered practice (FCP) (Law et al., 2005; Sass-Lehrer & Bodner-Johnson, 2003). The distinctions between these paradigms is of less consequence than the overarching point that attention has been paid to quality of intervention in terms of how it is delivered as a marker of quality, not just what it might contain.

However, quality of early intervention is less well understood, in relation to deaf children and their families, and certainly less well investigated, if one starts to consider quality in more subjective terms. Parental and family values, beliefs, culture, expectations, and previous life experiences, as well as current features of social ecology, will influence what is determined as effective, significant, and valued in relationships with service providers and in response to specific interventions (Aytch et al., 1999; Dyke, Buttigieg, Blackmore, & Ghose, 2006; King et al., 1996; Young et al., 2006). From this perspective, the effectiveness of early intervention is in part a function of those subjective features of perspective, appraisal, and experience. They will influence the meaning parents/families give to features of intervention, the extent to which it is seen to match needs, the extent to which they are able to make use of it, and the impact they might ascribe to it (Young, 2002). As King et al. (1996) remark: “Parental perspectives mediate between provision of care and the outcomes of that care” (p. 758).

Many of parents’ own narrative accounts of bringing up a deaf child are replete with examples of how professionals or interventions were prized because they matched or suited those families’ “way of doing things” (Fletcher, 1987). More recently, the “parent-driven” organization in the United States, Hands and Voices, has strongly promoted to parents and professionals alike the key principle: “what makes the choice work for your child and family is what makes the choice right” (Seaver, 2004, unpaginated). Indeed, how well early intervention (in its structure and process) might align with parental/family values, beliefs, and preferred ways of doing things, has been suggested as a primary indicator of the quality of early intervention (Aytch et al., 1999). Such goodness of fit has also been regarded as a vital influence on program effectiveness (Fitzpatrick et al., 2007) and, thus, ultimately child outcomes (Tomblin & Hebbeler, 2007).

However, attempts to investigate these subjective elements of interactions with services and their uptake have been subject to limitations as well as strengths. Qualitative interviewing of parents of early-identified deaf children about their intervention experiences has revealed features that influence the acceptability, feasibility, and appropriateness of service provision (Fitzpatrick et al., 2007; Russ et al., 2004; Young & Tattersall, 2005, 2007). This work has gone on to be influential in service improvement. However, qualitative approaches alone are of limited value in measuring intervention effects in a manner that can be operationalized in large-scale quasi-experimental or correlational studies of the impact of intervention on child and family outcomes. More structured approaches to ascertaining parent preferences for service provision such as the use of Conjoint Analysis (Fitzpatrick et al., 2007) are helpful in regularizing subjective data into objectively weighted factors. However, they too are limited by expressed preference itself being a construct that is dependent on the extent of knowledge and experience. Therefore, in the same way as surveys of satisfaction, conclusions are only as strong as one's confidence that those asked are aware of the full range of possibilities available. Structured surveys of parental needs and the extent to which they are met are helpful in providing a standard way of appraising service provision and monitoring its delivery (e.g., Bailey & Simeonsson, 1988, 1990; Dalzell, Nelson, Haigh, Williams, & Monti, 2007). However, in such instruments, categories/type of need is predefined, thus moulding perceived need. Furthermore, the underlying influences on parents’ differential appraisals of need are not uncovered.

Additionally, in those countries or States/Provinces where screening has been established for several years, a new kind of quality of intervention question is emerging. Originally, a key concern had been to demonstrate that programs of early intervention following routine early detection of deafness led to demonstrably better outcomes for children and their families. Now there is greater attention to understanding what might account for the variability of outcome (Eisenberg et al., 2007) and a questioning what works for whom in which circumstances? There is an attempt to discern which elements of intervention are effective, rather than treating effectiveness of intervention in whole program terms. However, as we have already rehearsed, the answer to such questions are in part a product of how parents/families are able to use any particular intervention content or process, given their particular psychosocial context.

It is, perhaps, hardly surprising that in several studies which have sought comprehensively to evaluate the outcomes for deaf children and their families of the early identification of childhood deafness, this notion of quality of intervention as uptake has largely been ignored. By uptake, we mean the interaction of subjective influences (of parent, family, sociocultural context) with those objectively determined features of intervention (input), which taken together will determine the quality of intervention from parents’ perspective. Studies to date have largely treated quality of intervention as an independent variable to be determined by the standards/features of the programs in which families are engaged (e.g., Kennedy et al., 2006). Whether and how individual children and families could make use of the intervention and why and how they may regard the intervention as quality for them is not relevant within the study design (e.g., Ching, 2006). Family variables, such as socioeconomic status, ethnicity, educational level, and more rarely degree of parental engagement with an intervention, are often treated as independent variables and investigated for their relationship with child outcomes (e.g., Kennedy et al., 2006; Wake et al., 2005). However, this approach, which in part acknowledges effectiveness as a function of family, child and socio-demographic variables (Eisenberg et al., 2007), is not the same as treating the quality of intervention as itself a function of features of the family context.

In what follows, we will describe how these concerns about the strengths and weaknesses of different approaches to understanding quality early intervention influenced the questionnaire we designed. This questionnaire is referred to throughout as the MVOS (My Views on Services)—the title given to it in the copy participating parents received. We outline the underpinning rationale for scale development and present results from an evaluation of the properties, validity, and reliability of the questionnaire.

Methods

The Wider Study

The questionnaire described in this paper and the results presented are drawn from a wider study concerning early-identified deaf children, their families, services, and developmental outcomes. This study, “Positive Support,” is a prospective, longitudinal study of early-identified deaf children and families in England (http://www.positivesupport.info/). Its overall research questions are

To what extent do the current service improvements in Health, Education, and Social Services appear to deliver improved outcomes for deaf children?

How do the type, extent, and quality of support and intervention affect these outcomes?

How do the intervention parameters interact with audiological, social, and family variables? Thus, which children and families are likely to benefit from which interventions?

What are the most effective strategies for the active involvement of parents/carers, given the diversity of deaf children and their families, in the ongoing assessment of their deaf children's progress and intervention needs?

Infants were eligible for inclusion in the study if they had permanent bilateral deafness classified as moderate or greater and were identified through the national NHSP (http://hearing.screening.nhs.uk/), and diagnosis was confirmed by 6 months of age. In the English context, early intervention refers to coordinated multiprofessional support including pediatric audiology, specialist family support (usually provided in the first instance by teachers of the deaf working with families in the home setting) and additionally (although not on a universal basis) involvement of other specialists such as social workers, speech and language therapists, Deaf/deaf role models, genetic counselors, and allied health professionals (http://www.earlysupport.org.uk).

The overall study collected data in the domains of child language and other developmental outcomes, child and family variables, family functioning, service provision (from professional perspectives), audiology-related information, as well as parental evaluation of quality of early intervention. A range of standardized and newly designed instruments was employed. In what follows, only the parental evaluation of service quality data will be addressed. Further details of the other aspects of the full study are available in Bamford et al. (2009).

Sample

The sample used for purposes of validation of the MVOS was drawn from the wider Positive Support study. Following appropriate processes of ethical approval, parents were recruited via professionals passing on information/invitation to participate, through parental self-selection to participate in response to advertising, and via information being distributed to those on the newborn hearing screening programme database. A total of 105 parents of eligible children consented to be involved in the study: 82 provided initial data, of whom 52 provided follow-up data 6 months later, and of these, 23 also provided follow-up data at 12 months after study entry. The decreasing number of returns did not primarily result from study attrition. Not all parents who completed MVOS at entry into the study were able to complete it at subsequent time points because of when they entered the study and when the data collection window closed. [The wider study was funded for 3 years only, with data collection running over a period of 20 months.]

Of the 82 infants whose parents provided initial data, 32 (39%) had a moderate hearing loss, 17 (21%) had a severe hearing loss, and 32 (39%) had a profound hearing loss. In one case, degree of deafness was reported as mild resulting from Auditory Neuropathy Spectrum Disorder. Twenty-six children (32%) spent more than 48 hrs on a Neonatal Intensive Care Unit (NICU). Twenty-three children (28%) were reported by parents to have disabilities; of these, 15 (65%) had spent more than 48 hrs on NICU.

The average age of the children when parents first completed the MVOS was 11.7 months (SD 6.3 months, range 0.6–27 months). Seventy-two of the 82 children were in two parent families. Seventy-five mothers (92%) and 61 fathers (85%) described themselves as White British with a further eight parents from “White other” backgrounds. A further five parents were from Asian-British backgrounds and two parents were Chinese. All parents were able to complete the questionnaire in English. Eight parents reported having a personal experience of disability. In addition, five reported having a hearing loss and one family of which were British Sign Language (BSL) users but who opted to complete the questionnaire in English. Socioeconomic status was skewed to the higher end with 43% of families earning over £35,000 per year.

Questionnaire Design and Underpinning Rationale

The MVOS was designed to be used as a repeated measure within this wider study enabling the investigation of the same dimensions of interest at different points as the child grows older and as configurations of professional involvement with families may undergo change. It consists of four sections: (a) a description of structure of professional services evaluated according to timeliness and availability; (b) the content of intervention evaluated according to quantity, importance, and satisfaction; (c) the process of the intervention evaluated according to extent of professionals’ performance and importance; and (d) the overall impact of the intervention. All elements of the questionnaire including its visual design and ease of use were piloted by means of a focus group (nine participants) and through five individual interviews with Deaf parents of deaf children.

Section 1: A description of the structure of services evaluated according to timeliness and availability.

In Section 1, parents are asked to confirm which professionals have been involved with them in the past 6 months from a list of 17 possible and average contact hours per month. They are also able to add other professionals to the list. Additionally, parents are asked to indicate, via tick boxes, whether particular services were offered to them, if they refused them, if they wanted contact with them at the present time and the extent to which any given service was difficult to access. This latter category was added following the pilot phase with the parent focus group. In terms of content validity, it was the view of parents who participated in the pilot that timeliness and availability were to some extent mediated by effort required on families’ part to obtain some kinds of provision. Consequently, this element of difficulty of access was also included.

The combination in Section 1 of actual description with evaluations of timeliness and availability were designed to ensure that parents’ self-perceived needs and preferences could be judged alongside basic information about the quantity and extent of service provision. In considering the instrument as one amenable to repeat use over time, this inclusion of timeliness and availability is important. It enables the capturing of changing (or stable) descriptions of actual service provision alongside changing (or stable) attitudes to the appropriateness of those elements of professional involvement as children develop and family experience changes. Please see Figure 1 for a partial screen shot of Section 1 of the questionnaire as it appears to parents completing it (the full list of professionals is not shown).

Figure 1

Partial screen shot of Section 1 of the questionnaire as it appears to participants.

Section 2: The content of intervention evaluated according to quantity, importance, and satisfaction.

Section 2 consists of 22 items relating to the “content” of the actual intervention, where content items refer to the delivery of all elements of intervention as a whole, rather than to what individual professionals might do. Content items were generated from a comprehensive review of relevant literature concerning professional intervention with deaf children and their families as well as early intervention more broadly with families with disabled children. From this review, 177 statements were generated. As our aim was both to develop a brief questionnaire and not to investigate particular methodological approaches to intervention with deaf children, statements that were too specific (e.g., Information about Sign Language; Information about Cochlear Implants) were dropped from the pool. A remaining 121 relevant items were carried forward. These were then grouped independently by two members of the research team into 14 preestablished categories of similar items, and the groupings were compared. Both research team members were, in addition to their academic credentials, also qualified practitioners who had worked with families with deaf children. From the final groupings of similar category items, a statement was generated that could stand for each relevant “category” of intervention content, for example, “knowledge about how to play with and enjoy my deaf child”, and “comprehensive assessments, for example, language, development, hearing.” These item statements were then further reviewed by members of the wider research team for comments on clarity of expression and salience. Finally, hearing parents by means of focus group (total participants = 9) and Deaf parents by means of individual interview commented on the clarity, relevance, and format of the presentation of items. In the case of Deaf parents, they were commenting on a BSL version of MVOS; however, as no Deaf sign language users were in the sample on which this validation is based, feedback regarding the BSL MVOS is not included in this paper.

In these ways, the face validity and content validity of this part of the questionnaire were strengthened: (a) we sought to ensure that items represented well the potential universe of available items pertinent to the content of early intervention for early-identified deaf children and their families; (b) we did this by engaging “experts” in judgments about whether the items chosen tapped the construct of interest (where experts included parents as experts); (c) we assessed the appropriateness of the presentation of such items in the chosen format (was the questionnaire easy to use?); and (d) we sought to be nonpartisan, in that item statements could be interpreted as relevant to whichever kind of communication methodology or intervention approach parents happened to have chosen at the time of completing the questionnaire.

In terms of the scale for this part of the questionnaire, parents are invited to rate each content of intervention item according three dimensions: importance, quantity, and satisfaction. These dimensions were chosen in order to capture subtle distinctions parents might make which would be important to track over time. For example, although a parent might report a large amount of a particular element of intervention she/he may not regard it as important, although that judgment may change as the child develops and new parent needs become apparent. Similarly, degree of satisfaction may have nothing to do with quantity of delivery, or for some parents it might. Not all elements of an intervention might be regarded as being of equal importance; however, parents might record high satisfaction regardless. However, over time and with greater knowledge, some parents might become more discerning about their satisfaction ratings. Capturing such distinctions in an accessible way is an important step toward a more differentiated understanding of quality effects from parents’ perspective that could be linked to their child and family circumstances and ultimately to data on their child's outcomes.

Therefore, for each content of intervention statement participants rate the importance that each specific content had for them on a 4-point scale, whether the amount provided had been adequate on a 4-point scale, and their satisfaction with each particular content of intervention on a 5-point scale. A final appraisal of satisfaction with overall content of intervention is collected on a 5-point scale at the end of the 22 statements. Parents in the focus group commented on the ease of use of the scale, confirming that it was not regarded as a significant barrier to have three scales one after another for each content item. However, they did point out that the color of the fonts should be darker for readability purposes. See Figure 2 for a partial screen shot of section of the questionnaire (not all 22 items are listed) using the title for this section as it appears on the questionnaire.

Figure 2

Partial screen shot of Section 2 of the questionnaire as it appears to participants.

Section 3: The process of intervention.

Section 3 addresses how intervention was provided by professionals, rather than what (the content) had been provided. Items associated with the process of intervention were based on the concept of FCP, which has become so prevalent in relation to working with parents of early-identified deaf children and their families (Law et al., 2005). Standardized instruments do exist already that are aimed at capturing parents’ perceptions of the family centeredness of professional practice, for example, the extent to which professionals work collaboratively to empower parents; the extent to which services are respectful of family priorities. The most commonly used of these is the Measure of Process of Care (MPOC) (King et al., 1996, 1997). However, we chose not to use or modify this instrument for three reasons.

First, the MPOC includes items that we regarded as more appropriate to the content of intervention section in how we had structured the MVOS. Second, we were concerned that there were specific dimensions of early intervention with deaf babies and their families that would not be captured by simply transposing the MPOC. Literature is beginning to identify features of specific importance in this context, such as providing an optimistic view of the future (Young & Greally, 2003). Third, the MPOC enables parents to rate the extent to which professionals or services are displaying certain desired behaviors/processes, but not how relevant or significant these may be to the parents at the time. We were concerned with both dimensions: extent of professionals’ performance and importance. Within a longitudinal framework, observing how the relationship between these dimensions on any given set of items changed over time would be an essential element of understanding how particular elements of intervention are effective given changing child and family circumstances. Interestingly, the significance of looking at perceived importance as well as perceived extent was a conclusion others were also reaching, be it in different child service contexts. For example, Nijhuis et al. (2007) adapted the MPOC-NL for parents simultaneously to rate the importance to them of each item. This was not published at the time we were designing MVOS but results from the same kind of concern we had also identified.

Thus, Section 3 on the process of intervention is an 18-item scale consisting of professional behaviors known to be associated with promoting FCP in the context of deaf children and their families. These items were generated, and their content validity strengthened, following the same procedure as that previously described in relation to Section 2. Ratings consist of a 7-point scale to describe the extent to which professionals had engaged in the particular FCP item and additionally a 4-point scale to enable participants to rate the importance to them of such a practice. See Figure 3 as a partial screen shot of Section 3 of the questionnaire (not all 18 items are listed) as it appears on the questionnaire.

Figure 3

Partial screen shot of Section 3 of the questionnaire as it appears to participants.

Section 4: The overall impact of the intervention.

Section 4 consists of six questions concerning the overall impact of intervention on a 5-point scale and the perceived direction of that impact (e.g., is that difference positive?). Just because a parent perceives there to be, a big impact does not mean they would regard that impact as necessarily helpful. The question also invites separate responses in relation to child, parent, and family as a whole. We made these distinctions in recognition of the fact that parents have reported, with hindsight, the effect of the new experiences (including intervention) associated with having a deaf child on them as individuals (e.g., DesGeorges, 2003; Young, 2002). It is also well recognized that families as a system undergo growth and change in response to admitting and/or resisting the potential changes that the deaf child's experience brings, including involvement with professional services (Gregory, Bishop, & Sheldon, 1995; Young & Greally, 2003). Therefore, in inviting assessments of impact, we wanted also to invite parents to consider similar or different degrees of impact for their child, themselves, and their family as a whole. In this way, we would also be able to see if the balance between perceived impact across those three domains might change at different points over time.

In the pilot phase of questionnaire development, however, we encountered difficulties in getting the exact wording of such simple questions right. The inclusion of words such as “helped,” “assisted,” or “supported” in the phrasing of the initial statement were perceived to embody a predetermined assumption that the intervention was positive and it was just a question of how positive. Consequently, our interest in the direction, not just the extent of impact, was seen by parents as confusing or contradictory. However, a more neutral word such as “impact” which could be interpreted as either positive or negative was not liked as parents remarked on its associations with such events as a car crash. These discussions at the pilot phase led us to the simple solution of using the word “difference” (Figure 4).

Figure 4

Section 4 of the questionnaire as it appears to participants.

Finally an open-ended question invites parents to comment on anything else they might want to.

Additional Measures

A key criticism of evaluative questionnaires requiring participants to make a subjective evaluation based on experience is that responses are heavily influenced by the personal disposition of the rater. For example, participants with a more positive outlook on life may give higher ratings of impact and more positive evaluations of effect. In order to investigate and control for bias deriving from personal disposition, parents were also asked in the study to complete a short form of the standardized instrument: Trait Emotional Intelligence Questionnaire (TEIQue) (Petrides & Furman, 2001). In the version given to parents to complete, it was termed “My approach to life.” The TEIQue is a 30-item questionnaire rated on a 7-point scale of agreement. Petrides (2006) reported good internal consistency in its four subscales—emotionality (Cronbach's α = 0.78), self-control (α = 0.79), well-being (α = 0.83), and sociability (α = 0.81)—and global trait emotional intelligence (α = 0.90). Parents completed the TEIQue on an individual basis on one occasion only—at entry to the study. The completion of this additional standardized questionnaire would enable us to test out the extent to which subjective ratings of quality of intervention were closely allied with specific parental personality traits known to mediate individuals’ interpretations and life experience. Parents also completed a short demographic questionnaire referred to as “Your family.”

Results

Section 1: A Description of the Structure of Services Evaluated According to Timeliness and Availability

A glossary of professionals was included in the questionnaire to assist parents in completing Section 1 as some parents would not necessary know the professional job title of the interventionist. Parents encountered no significant difficulties in completing all columns relevant to each professional they had indicated contact with. Average amount of missing data across 16 professional categories was 3.2%. (The exception was the category “keyworker” which recorded 98.8% missing data as a result of a printing error separating the category from the natural end of section 1.) However, the initial column where parents had to tick “yes” or “no” to indicate current contact before completing other columns of data associated with the given professional did prove problematic. If parents were not seeing that particular professional at that time, they did not always tick “no.” This meant that the interpretation of subsequent data (e.g., “please tick if you were offered this service”) less reliable. In this instance without knowing whether the parent actually had the service currently, it would not be possible to discern the relationship between actual provision and offered provision. To overcome this problem, individual respondents were contacted again for clarification. However, in future printing of the questionnaire, greater emphasis will be put on the significance of completing this column.

The list of professionals offered in this section was generally regarded as exhaustive; however, 31 (38%) parents chose to add another professional service provider group which had not been listed but was of importance to them at the time. The additions were “portage worker” (a term used to indicate peripatetic play specialist usually working with physically or learning disabled children), voluntary sector workers (i.e., staff employed by charities or deafness related private organizations), physiotherapist, and communication method-specific professionals such as cued speech and auditory–verbal therapists. One of the additions made by parents reflects an error on our part in drawing up the initial list in that pediatrician had not originally been included.

On the whole, there were few problems in parents completing the average contact hours of each professional. A greater amount of incomplete data on hours of contact tended to occur in relation to those services with whom very few parents had contact (e.g., social workers; speech and language therapists) and those services that were not universal (e.g., contact with deaf mentors). In these instances it could be surmised that contact was either so infrequent or so variable as to make the specification of hours difficult.

These results demonstrate that the face and content validity of this section of the questionnaire was adequate. The format was flexible enough to enable amendment of the professional list. The multiple questions about each professional service did not prove off-putting. The data generated were also confirmed as amenable to the application of statistical tests such as one-way analysis of variance and linear regressions to investigate relationships such as the numbers of professionals involved and hours of intervention for different age groups of children.

Section 2: (The Content of Intervention) Structure Analysis

The sample size obtained was not large enough for meaningful factor analysis. Exploratory cluster analysis of variables was carried out using parents’ (n = 73) ratings of importance for the 22 statements in this section of the questionnaire in order to identify components of early intervention for parents with deaf infants. Different methods—hierarchical cluster analysis with Ward's linkage and between-group linkage and two-step cluster analysis were used to find consistent results. Internal consistency of identified subscales was estimated using Cronbach's alpha. Interscale/judge reliability was estimated using Pearson's correlation. Test–retest reliability was estimated using Spearman's correlation and Pearson's correlation.

Two main clusters were identified underlying the structure of the 22 items. We term these Cluster 1 “Supporting a deaf child” which included items relating to specific support associated with parenting a “deaf” child. Items were linked to what might be different or new for a parent because of the child's deafness. Cluster 2, which we term “Supporting parents,” (SP) covered items pertaining to less deaf-specific support, which relates more to SP as individuals or supporting the family more generally (Table 1).

Table 1

Content of intervention subscale clusters

Cluster 1: “Supporting a deaf child”	Cluster 2: “Supporting parents”
1. Information about how to communicate with my deaf child	1. Help to understand how professionals support system work
2. Help to encourage my child communication skills	2. Advocacy, for example, professionals help me to make my needs known and to fight for things if necessary
3. Comprehensive assessments (e.g. language, development, hearing	3. Referrals to other professionals and services
4. Knowledge about how to play with and enjoy my deaf child	4. Contact with other parents of deaf children
5. Knowledge about how deaf children grow up	5. Contact with deaf people
6. Confidence building in parenting a deaf child	6. Assistance to claim welfare benefits
7. Information about deaf children's needs and potential	7. Emotional support for you and your family (partner, siblings)
8. Information about deafness	8. Support for my whole family, not just me and my deaf child
9. Coordination of all the services, and professionals involved with my child and family	9. Support to make decisions about your deaf child and your family
10. Information about available services	10. Support to help others understand my child's deafness
	11. Full consideration of my whole family's strengths and needs
	12. Respite care, for example, support for childcare to enable carers to take a break

Both subscales demonstrated adequate internal consistency (Cronbach's α = 0.88 and 0.86, respectively) as did the global scale (Cronbach's α = 0.91). Correlation between the subscales was also high, r = .75. Test–retest correlations after 6 months for the subscales and global scale were high and statistically significant after 6 months (rho = 0.88, r = .68, r = .74) and 12 months (rho = 0.60, r = .82, r = .90), thus demonstrating the stability of the scale over time.

Regarding the data collected on satisfaction with content of intervention, the reliability of the data on both subscales was very high (Cronbach's α = 0.91 and 0.89, respectively) as was the global scale (Cronbach's α = 0.94). The stability of the scores over time was good with test–retest correlations on both subscales after 6 months (r = .68, r = .64) and 12 months (r = .69, r = .56).

Section 3: (Process of Intervention) Structure Analysis

Tests on the internal structure of the questionnaire items in this section revealed no apparent subscales within the 18 statements. The scale demonstrated high internal consistency (Cronbach's α = 0.93) with all 18 statements necessary to achieve such a high level. Reliability did not improve with the deletion of any statements, suggesting a high construct validity for the scale. Test–retest correlations after 6 months were high and statistically significant (r = .64) and after 12 months, r = .82.

Section 4: (Impact)

Parents had no difficulties in completing this section three times in relation to their child, themselves, and their family as a whole. Ninety-three percent of parents did so in its entirety, demonstrating good face validity.

Completion of the TEIQue

No significant difficulties were encountered by parents in the completion of the TEIQue. Of the 82 returns, 79 (96%) contained a completed TEIQue also. The acceptability to parents of completing a standardized psychological test alongside an instrument specific to their child and family circumstances was, thus, confirmed.

Discussion

Other publications from this study are concerned with the analysis of results from the MVOS and the relationship between these results and child and family characteristics and outcomes (e.g. Bamford et al., 2009). It has been the purpose of this paper to discuss the rationale for the instrument, its theoretical underpinnings, its psychometric properties and its usability. A central goal in designing this instrument was to ensure that parents were asked straightforward rather than complex questions, with the complexity of the information being yielded through how the different components of the questionnaire fitted together, rather than through asking complex questions. Hence, an approach that encouraged parents both to rate what they experienced and to evaluate it against selected dimensions, the relationship between which had been identified as pertinent to parental appraisal of the quality of intervention.

At the most basic level, results from this sample demonstrate that this approach and structuring of the questionnaire does work and is amenable to high-quality completion. However, the sample of parents who participated in this study are not representative of the full diversity of families, were self-selecting and, therefore, motivated, and were able to complete the questionnaire autonomously. It will be important to go on to test out the “acceptability and feasibility” (Fitzpatrick, Davey, Buxton, & Jones, 1998) of the questionnaire in populations where literacy may be a challenge or where facilitated support might be required to complete it. It is also, like any other questionnaire, prey to cultural norms of expression that would require modification in other service settings.

The strong construct validity of the content of intervention scale (Section 2) and the process of intervention scale (Section 3) will mean that robust relationships can be investigated between parents’ ratings associated with intervention and their child/family characteristics and outcome measures. Furthermore, the test–retest reliability supports the use of these scales in a periodic and longitudinal fashion. They allow for the discrimination of particular features of intervention, mediated by parental appraisal to be investigated for their influence on outcomes in a way that has not been possible before in longitudinal studies of the outcomes of early-identified deaf children. Although it has long been acknowledged that it is the quality of intervention in combination with early identification that makes a difference, it has been much harder to quantify that quality dimension in a way that can be operationalized within complex longitudinal study designs. The MVOS offers one possibility.

Finally, although this instrument was designed with that research goal in mind, it has some potential for use by practitioners/early interventionists to involve parents in the assessment of the quality of intervention they are experiencing. Particularly because it is an instrument amendable to repeat use, it can provide descriptive and comparative information about the saliency for families at particular points in time of features of intervention content, process, and professional configuration of services. While the statistical analysis of the relationship between various dimensions of evaluation may not be of immediate interest, the simple structure of the instrument allows for descriptive comparisons to be made between ratings at different times for individual families or, from a services’ perspective, between different families. Differences in ratings at different points in time would be immediately visually apparent as well in comparing questionnaires which potentially could provide an accessible means of parent-professional (or indeed parent to parent) discussion over changing needs, perceptions, and preferences.

Conclusion

As practitioners know, the effectiveness of early intervention depends to a very great extent on its reception by families (not its provision). That reception, as we have argued, must fundamentally be understood in terms of what is meaningful to families, fits changing needs and circumstances, and tunes in to priorities, values, and strengths. The design of this questionnaire has been an attempt for research finally to catch up with that understanding and provide a means to take it into consideration in longitudinal studies of child and family outcomes following early identification of deafness.

Funding

The Big Lottery Fund.