Development and psychometric testing of a scale for the assessment of the quality of developmental care in neonatal intensive care units in Iran.

INTRODUCTION: Developmental care provided to infants hospitalized at neonatal intensive care units (NICU) help weaken environmental stressors and reduce infant morbidity rates. Assessments are the first step to improving the quality of any type of care. Therefore, this study was conducted to design and assess the psychometric features of a scale designed for measuring quality of developmental care in the NICU in Iran.
METHODS: This study was conducted from December 2014 through September 2015 in Tehran, Iran. The present mixed-methods sequential exploratory (quantitative-qualitative) study used the Delphi method to design an initial questionnaire through a review of the literature and by using the input of experts. The validity of the questionnaire was ensured by assessing then validity of its content (qualitative-quantitative), face (qualitative-quantitative), and construct (exploratory factor analysis with 500 NICU personnel from 34 hospitals in Tehran), and its reliability was ensured by assessing its internal consistency (using Cronbach's alpha) and by assessing its stability through the test-retest method.
RESULTS: The qualitative stage of the study resulted in a 93-item questionnaire with eight domains. After performing the content and face analyses, a factor analysis was performed on 90 items of the questionnaire, yielding a 76-item questionnaire with five domains, including "sleep, pain and stress management," "routine care," "the family," "management," and "sensory care," which explained 62.5% of the variance. The reliability of the questionnaire was confirmed with a Cronbach's alpha of 0.9 and its stability was confirmed by an Intraclass Correlation Coefficient (ICC) of 0.93.
CONCLUSION: The questionnaire developed for the assessment of developmental care in the NICU covered all of the dimensions of this type of care, and it is a valid and reliable tool for assessing and improving developmental care in the NICU.

1. Introduction

Several infants are admitted to NICUs every day for different reasons, with premature birth being the main reason (1). According to statistics, in 2010, one in every 10 infants was born preterm, which adds up to a total of 15 million premature births throughout the world (2). The human brain is developed largely in the last trimester of fetal life and away from direct environmental effects. Nutrition, body temperature control, and many internal regulatory systems help set up a fetal biological rhythm inside the uterus (3). In preterm infants, this support system is replaced with completely different environmental stimuli in the NICU. The NICU acts as a harsh sensory stimulant that is unsuitable for the demands of the developing nervous system and thus exposes the preterm infant to a wide range of morbidities (4). Therefore, modifying the NICU environment can reduce the ill effects of these stressors. Developmental care is comprised of a wide range of interventions used for minimizing stress in the NICU that include single or multiple components, such as the control of external stimuli, a series of nursing care services, and the proper positioning of the infant (5). Developmental care is provided during a highly critical period by a specialized team, and the quality of these services is of crucial importance. Qualitative assessments are an integral part of improving patient care, including developmental care, which can be performed through various tools and methods (6). Only a few studies have discussed the tools used for measuring the quality of developmental care. In Ghina, a 48-item tool, was used to measure neurodevelopmental functioning in infants hospitalized at NICUs within five standards, including “flexible and individualized care,” “parental and family involvement,” “collaboration of healthcare providers,” “environmental control,” and “hospital-level developmental care involvement.”(7). In a study conducted in Italy, the quality of developmental care in NICUs was measured using a 9-item checklist within two domains, i.e., “infant-oriented care” and “infant pain management” using two separate indices (8). In 2011, Newborn Individualized Developmental Care and Assessment Program (NIDCAP) Federation International published a manual for the assessment and licensing of hospitals that provide developmental care. The questionnaire proposed in this manual assesses four dimensions, i.e., “physical ambience of the hospital and the NICU,” “infant-oriented care measures,” “family-oriented care measures,” and “personnel and medical system-oriented care measures and considerations.” Each dimension and sub-dimension contains items scored from 1 to 5, and the instructions on scoring also are provided in the manual. This questionnaire contains a total of 121 items, and its use requires sufficient training and skills. The manual does not discuss the validity and reliability of the questionnaire, but it asserts that, despite being a tool with international applicability, some of its sections can be eliminated based on the cultural structure of the country in which it is being used (9). Since providing developmental care is influenced by the cultural setting of the country and the opportunities it provides, the present study was conducted to design a tool for measuring the quality of developmental care according to the local setting in Iran.

2. Material and Methods

2.1. Setting

The present mixed-methods sequential exploratory study used a qualitative-quantitative approach to design a tool for measuring the quality of developmental care provided in NICUs Iran and to assess the psychometric features of the tool. The study was conducted in tertiary NICUs at 34 hospitals across Tehran, including teaching, public, and private hospitals with the highest rate of neonatal admissions to the NICU. Considering the sufficient samples for factor analysis, 500 nurses who had at least six months of work experience in the NICU and provided verbal consent to participate took part in this study.

2.2. Qualitative Stage

The qualitative stage of the study examined the developmental care provided in NICUs in Iran in three rounds using the Delphi method. Delphi method is a systematic approach in research that extracts the views of a group of experts about a certain issue or question and seeks to answer the question of what the issue actually is, what it can be and what it ought to be (10, 11). This stage involved the participation of 30 experts with academic background and relevant work experience in developmental care who were counted among the authorities of this field in the country. These experts included six neonatal specialists, six neonatal subspecialists, two pediatricians, three occupational therapists, two physiotherapists, two audiometrists, one neurologist, and eight nurses and midwives from across the country. In the first round, all 30 of the experts were asked about the elements they deemed necessary for the assessment of the developmental care provided in NICUs, and 18 participants responded. The questionnaire’s initial items were developed according to the views expressed by the experts and through a review of the literature on the subject. This version of the questionnaire included 156 items within eight domains. In the second round of the Delphi method, the experts were requested to prioritize the items in each domain based on their degree of importance to the assessment of the quality of developmental care provided in NICUs. The experts had to give a total score of 100 to each domain and divide that score between all of the domain’s items according to their perceived importance and level of priority. Then, the mean score of each item was calculated in all of the domains, and the experts, with the guidance of the research team, kept the items with a mean priority score of 50% or more and eliminated those with lower mean scores. In the third round, the final questionnaire was developed with 93 items in eight domains. Then, it was distributed to the experts once again, and it was approved by all 12 of the experts left in this stage.

2.3. Quantitative Stage

This stage of the study examined the validity and reliability of the questionnaire. The validities of the face, content, and construct of the questionnaire were measured to ensure its validity.

Face Validity: The face validity of the questionnaire was determined using qualitative and quantitative methods. In the qualitative part, face-to-face interviews were conducted with 10 NICU nurses, and the questionnaire’s level of difficulty, suitability, ambiguity, and probability of item misinterpretation or failure to understand the meaning of the words were assessed. In the quantitative part, the item impact method was used, and all of the items of the questionnaire were assessed and scored based on a 5-point Likert scale with options including absolutely important (5 points), somewhat important (4 points), moderately important (3 points), a little important (2 points), and not important at all (1 point). Items given a score of 1.5 or higher were kept for further steps of the analysis (12).

Content Validity: The content validity of the questionnaire was determined using qualitative and quantitative methods. In the qualitative part, the views of eight experts with knowledge of neurodevelopmental care were used to make any necessary modifications to the questionnaire. In the quantitative part, the Content Validity Index (CVI) and the Content Validity Ratio (CVR) were measured based on the relevance and necessity of the items. The CVI measures the relevance, clarity, and simplicity of the items on a 4-point Likert scale (13). For this part of the study, the questionnaire was provided to 10 new experts. The CVI score was calculated by summing the scores for each item that had been given a score of 3 or 4 (the highest scores) divided by the total number of experts, and, then, items with CVI scores higher than 0.79 were accepted. To determine the CVR of the questionnaire, 10 experts were asked to place each item on a three-point spectrum (necessary, useful but not necessary, and not necessary). The content validity ratio was therefore calculated as 0.6 based on Lawshe’s table (14, 15).

Construct Validity: The exploratory factor analysis was used to assess the construct validity of the questionnaire. This method comprises one of the most important measures that should be taken in the design of tools, and it also is useful for grouping relevant questions in a scale. Each group or factor consists of a group of variables with higher intra-class correlations than interclass correlations, and it signifies a relatively unique feature that helps explain the grouping of the variables (16). Performing a factor analysis requires three to 10 samples per item; however, when the percentage of variance is calculated and the factor loading exceeds 0.8, three samples also suffice (17). In this study, exploratory factor analysis was performed on 93 items within eight domains with the help of 500 nurses.

Reliability: The reliability of the questionnaire was ensured by measuring its internal consistency and stability. Cronbach’s alpha values were calculated for each item and for the entire questionnaire in order to determine the questionnaire’s internal consistency. The test-retest method was used to measure the stability of the questionnaire, and the questionnaire was completed twice by 30 nurses within an interval of two weeks, and the correlation coefficients between the scores obtained in the two tests for each factor and for the entire questionnaire also were calculated.

2.4. Statistical Analysis

In this study, we used descriptive and inferential statistics to analyze the data. The measure of Cronbach’s alpha was used to assess the reliability and internal consistency of the questionnaire, and Pearson’s correlation coefficient was used to assess its test-retest reliability. Both measures deemed values higher than 0.7 to be acceptable. The exploratory factor analysis was performed using the Kaiser-Meyer-Olkin (KMO) index of sampling, the analysis of the principal components, and Bartlett’s test of sphericity. The Scree Plot and Eigenvalue methods were used to calculate the number of factors constituting the questionnaire. The criterion used for the classification of the factors was an inflection point of 0.4 as the minimum factor loading required to keep the item in the factors extracted through factor analysis, and eigenvalues greater than one were accepted. The varimax rotation-an orthogonal rotation-was used for the simplification and interpretability of the factor constructs. The data were analyzed using SPSS-19 at a significance level of 0.05.

3. Results

The qualitative stage of the study resulted in a 93-item questionnaire with eight domains, including 11 items on sensory care (olfactory, taste, auditory, touch, light and sound), 15 on environment management, 15 on parental involvement, 7 on positioning care, 6 on sleep care, 13 on pain and stress management, 10 on skin care, and 15 on nutritional care. The nurses scored the items based on a Likert scale, with 1 signifying the lowest and 10 signifying the highest degree of developmental care provided in the NICU. Once the content validity of the questionnaire was determined and the recommended modifications were made, the content validity index was measured to determine the questionnaire’s validity. For this purpose, the questionnaire was distributed among 10 experts, and two items were eliminated because they received scores less than 0.79. The CVI was calculated as 0.8 to 1 for the remaining items and as 0.87 for the entire questionnaire. One item with a CVR below 0.62 was eliminated from the questionnaire. A 90-item questionnaire with eight domains was ultimately produced. To determine the construct validity of the questionnaire, 498 NICU nurses completed its final version. Table 1 presents the nurses’ mean age, work history, work experience in the NICU, mean number of infants managed by each nurse, and familiarity with developmental care and means of introduction to it. The exploratory factor analysis was carried out on 90 items using the principal components method. The Kaiser-Meyer-Olkin value was calculated as 0.958, and Bartlett’s test of sphericity was measured as 849.26817, which was significant at the 0.0001 level and which justified the factor analysis performed based on the correlation matrix yielded by the study sample (Table 2). With an eigenvalue greater than 1, the factor analysis showed the number of factors constituting the questionnaire using the scree plot shown in Figure 1. The exploratory factor analysis produced 76 items within five domains, i.e., “sleep, pain and stress management;” “daily care;” “the family;” “management;” and “sensory care,” which explained 62.5% of the variance with a minimum factor loading of 0.4 (Table 3). The research team decided to eliminate items with a factor loading of less than 0.4 and those in irrelevant domains. Table 4 presents the domains and distribution of the final questionnaire items Table 3 presents the Cronbach’s alpha coefficients and intra-class correlations obtained for the different domains and for the entire questionnaire, which indicate the acceptable reliability of the questionnaire.

Table 1

Participants’ Personal Information

Variable		n	%
Education	Bachelor’s	445	90.4
	Master’s or PhD	47	9.6
Familiarity with developmental care	Fully familiar	145	30.6
	Relatively familiar	273	57.6
	No familiarity	56	11.8
Means of introduction to developmental care	University courses	132	34.4
	Hospital workshops	149	38.8
	Domestic and foreign conferences	24	6.2
	Personal study	6	1.6
	Other means	36	9.4
	More than one means	37	9.6
Variable	Mean and Standard Deviation	Highest	Lowest
Age (in year)	33.79 ± 6.049	22	58
Work history (in year)	8.41 ± 5.615	1	33
Work experience in the NICU (in year)	5.73 ± 4.586	1	29
Number of infants managed by each nurse	3.48 ± 1.275	1	6

Table 2

Factor analysis, Kaiser-Meyer-Olkin index of sampling, and Bartlett’s test of sphericity results

KMO	0.958
Bartlett’s test of sphericity	Chi-square approximation	26817.849
	Degree of freedom	4005
	Significance level	0.001

Figure 1

Factor analysis and scree plot used for determining the number of factors constituting the questionnaire

Table 3

The total variance determined for the tool’s five factors, Cronbach’s alpha coefficients and intraclass correlation coefficients

Factor	Rotation Sums of Squared Loadings			Reliability
	Total	Variance Percentage	Cumulative Percentage	Cronbach’s Alpha	ICC
Sensory care	5.408	6.009	62.506	0.86	0.89
Sleep, pain and stress management	13.791	15.323	15.323	0.95	0.97
Routine care	13.634	15.149	30.472	0.91	0.93
The family	12.696	14.106	44.578	0.90	0.94
Management	10.728	11.920	56.498	0.91	0.95
Entire Questionnaire				0.90	0.93

Table 4

The number and distribution of the final questionnaire items

Factor (Construct)	Domain	Sub-Domain	Number of Items (Questions)	Total Number
1	Sensory care	Noise control	3	10
		Light control	2
		Odor control	2
		Touch control	3
2	Management	Assessment of the provided care	4	13
		Hospital management’s support of the provided care	3
		Facilitating personnel training	3
		Teamwork	3
3	The family	Family involvement in neonatal care	3	15
		Family training	6
		Providing the family with information	3
		Respect for the family	3
4	Sleep, pain and stress management	Regulation of sleep timing	3	18
		Sleep status	2
		Assessment of pain and stress	4
		Reducing pain and stress	6
		Personnel training	3
5	Routine care	Nutrition assessment	4	20
		Nutrition training	3
		Nutritional preparations	7
		Skin care	4
		Body positioning	2
Total				76

4. Discussion

The questionnaire developed in the present study was designed with 76 items in five domains and has confirmed face, content (qualitative-quantitative), and construct (exploratory factor analysis) validities, A confirmed reliability was determined through the measure of internal consistency (Cronbach’s alpha), and a confirmed stability was determined through the test-retest method. The results of factor analyses identified five domains for the assessment of developmental care in NICUs in Iran, including “sleep, pain and stress management;” “daily care;” “the family;” “management;” and “sensory care.” The theoretical framework and support model of developmental care used in the present study were based on the Universe of Developmental Care Model (2009), which identified five main domains of developmental care in NICUs, i.e., sleep care, pain and stress assessment and management, routine care, suitable environment and family-oriented care (18). In 2013, the Neonatal Integrative Developmental Care Model was introduced based on the Universe of Developmental Care Model, proposing seven domains instead of five and emphasizing “improved nutrition” and “protecting body positioning and movement” in addition to the five domains of “improved ambience”, “parental involvement”, “sleep protection”, “pain and stress reduction” and “skin protection” for accomplishing a better performance in NICUs (19). Given the accurate and comprehensive data obtained in the Delphi stage of the study, the constituent domains of the initial version of the questionnaire covered all the aspects of the theoretical framework of developmental care. The exploratory factor analysis identified three important domains, including “the family,” “sensory care,” and “management,” and it combined skin, body positioning, and nutrition care into the more general domain of “routine care.” Two important aspects of developmental care, i.e., “sleep care” and “pain and stress management,” which have been proposed as two separate domains in theoretical models, were combined into one domain by the factor analysis performed in this study. Thus, the questionnaire designed in the study includes all the main dimensions of developmental care and resolves the deficiencies existing in the questionnaires used in studies by Monitirosso (8) and Zhang (7), which covered limited dimensions of developmental care in their tools. Moreover, none of the developmental care assessment tools designed to date has undergone a comprehensive psychometric assessment. The questionnaire designed in this study is a valid and reliable tool for assessing the quality of developmental care provided in NICUs. The limitations of this study included the absence of a valid tool for assessing the convergent validity of the questionnaire and the large number of items developed in the questionnaire.

5. Conclusions

The questionnaire designed in the present study is a valid and reliable tool for the assessment and improvement of the quality of developmental care provided in NICUs in Iran, and it contains 76 items developed within five domains.