The Development and Psychometric Evaluation of the Electronic Fetal Monitoring Knowledge Scale.

BACKGROUND: The aim of this study was to develop and assess the psychometric properties of the Electronic Fetal Monitoring Knowledge Scale (EFMKS), a self-report and short instrument measuring knowledge concerning Electronic Fetal Monitoring (EFM). Methods. The EFMKS was developed in a three-phase process by using an integrated mixed-methods approach that included literature reviews, professional focus groups, expert consultations and a psychometric survey evaluation. The psychometric evaluation was conducted by recruiting a sample of 128 professionals (midwives and doctors). Content validity, exploratory factor analysis, discriminant and construct validity, test-retest reliability and internal consistency were explored.
RESULTS: The expert panel determined that the content validity was satisfactory. The final 10-item scale consisted of three factors explaining 73% of the total variance in the data. Discriminant validity was satisfactory. Internal consistency reliability (α = 0.89) and test-retest reliability (0.85) were satisfactory. The majority of the midwives and the obstetricians had a good level of knowledge while approximately one third of them had a low level of knowledge in EFM.
CONCLUSION: The EFMK demonstrated good content validity, an easily interpretable three-factor structure, high internal consistency, high test-retest reliability, and satisfactory discriminant and construct validity with sample characteristics. The EFMKS may be used for evaluating the EFM knowledge of health professionals and for identifying the areas of their knowledge gap. Based on study findings, an annual multi-professional CTG training is necessary for all intrapartum staff and in particular for the midwives and doctors with shorter clinical experience in the labor ward.

INTRODUCTION

The continuous use of EFM during labor of low risk women and poor EFM interpretation may lead to an increased Cesarean section rate without a significant reduction in cerebral palsy or infant mortality (1,2,3). Errors in the interpretation of cardiotocography (CTG) traces and failure to identify and manage pathological tracings are recognized causes of adverse obstetric outcomes (4,5,6,7). Additionally, cases of false interpretation and unsuitable management of cardiotocographic (CTG) traces may also lead to large financial costs (8). Based on a 10-year report on maternity claims of the English National Health Service Litigation Authority (NHSLA) approximately 1 in 1000 births ends in litigation with the three most frequent causes of litigation relating to management of labor (including CTG interpretation), cerebral palsy and cesarean section (9). The Royal College of Obstetricians and Gynecologists (10) stated on the English NHSLA report that the failure of the role of training and use of guidelines in claims needs to be assessed. Therefore, it can be concluded that CTG education is essential for reducing the incidence of hypoxic injuries during labor and for avoiding litigation.

Fetal surveillance education programs exist in Great Britain, the United States, Australia and New Zealand. To ensure adherence to national and international guidelines, a Greek CTG education and assessment program was developed by the University of West Attica in 2015. Several reports are available on the content of CTG training programs and many publications regarding the impact of CTG education programs on professionals’ skills (7,11,12,13). However, we were unable to identify any published studies regarding the development of validated CTG tools measuring knowledge of midwives and obstetricians regarding the EFM. A lack of validated assessment methods has been indicated by Pehrson, Sorensen and Amer-Wahlin, too (12).

AIM

The aim of this study was to develop and validate a self-report and short instrument measuring knowledge concerning electronic fetal monitoring. This instrument has the potential to provide a validated and feasible method of briefly assessing EFM knowledge in general.

METHODS

This study was designed for scale development. The scale was prepared in the Greek language, and its development included three main phases: item generation, item reduction, initial validity testing (content validity testing), construct validity testing (exploratory factor analysis (EFA), reliability testing and criterion-related validity).

Phase I: Item generation

The goal of phase I was to generate the items for the instrument from two main sources: a) an extensive literature review of the international guidelines on EFM and b) a focus group including six experienced midwives and six experienced obstetricians. Finally, a 25-item pool of items regarding fetal physiology, interpretation, classification and management was established.

Phase II: Content validity testing and item reduction

Once the item pool was developed from phase I, the goals of the phase II were to assess the content validity and reduce the number of questions for further scale development by assembling a panel of four experts on EFM. The items that were kept were the most clear and concise. As a result, nine items were deleted and a 16-item instrument resulted.

The CVI was calculated using a four-point ordinal scale ranging from 1 (not relevant) to 4 (very relevant). A 0.75 CVI value was used as the acceptable minimal CVI value. The experts returned their rating scales, and four more items with CVI ranging from 0.50 to 0.69 were deleted. This resulted in a revised instrument with 12 items and the total CVI score was 0.85.

Phase III: Reliability and validity testing

The goals of phase III were to test the performance of the 12 items in a sample of midwives and medical doctors by testing the reliability, exploring the factor structure of the instrument comprising the scale, testing the discriminant and construct validity and determine the scale’s test-retest reliability and stability in a repeated administration. Reliability was assessed by computing an internal consistency coefficient. Internal consistency was determined: (a) by using Cronbach’s alpha and (b) by examining the change in Cronbach’s α coefficient if an item was deleted from the scale. A minimum Cronbach’s α value of 0.70 for group comparisons is acceptable. In addition, poor items are defined as those that, when deleted, increase the coefficient α by 0.1 or more.

Cronbach’s alpha coefficient for the total scale with 12 items was 0.86. Two items proved problematic in terms of their item-total correlation’s namely item Q21: “evaluate the pattern of contractions” and item Q16: “interpret a fetal scalp-blood sample”. The deletion of these two items improved Cronbach’s alpha value. Therefore, items 9 and 4 were excluded from further interpretation in the reliability assessment and the factor analysis. This resulted in a revised instrument with 10 items.

The study was conducted in two public maternity clinics in Greece with an annual birth rate of approximately 3500 births. Data were collected between April and July of 2016. During the recruitment period, all eligible professionals (midwives and doctors) who worked in the labor ward during at least the past 12 months (N=156) were invited to participate in the study, and a total of 128 professionals agreed to participate and completed the questionnaires (response rate 82%). Non-participation was mainly due to lack of time of the professionals. The total of 128 professionals, including midwives (32 midwives and 32 student midwives undertaking their practical training) and doctors (32 obstetricians and 32 resident obstetricians), were recruited for the survey through random sampling.

The test-retest reliability of the questionnaire was calculated for 60 (who agreed to complete the retest of the EFMKS) out of 128 professionals who agreed to repeat EFMKS one week after the first administration (68 participants did not agree to complete retest the EFMKS due to lack of time).

Demographic data

Data on gender, age, employment status (professional or student/resident), and duration of clinical experience were collected using a specially designed form. Two dichotomous questions asked participants: a) their confidence about interpreting CTG traces and b) their feeling of having adequate training for CTG usage.

Electronic Fetal Monitoring Knowledge Scale (EFMKS)

The version of the EFMKS that emerged from phases I, II, and III of the instrument development process consisted of items designed to measure the level of knowledge regarding the electronic fetal monitoring. Every question was a multiple choice question, asking to select one or more than one answer, and for every correct answer a point was allocated. Professionals with higher scores were classified as having better level of knowledge. The cut-off was defined by the scale midpoint rather than the sample median because external criteria for ‘good’ and ‘poor’ knowledge were not available. Permission for the use of the entire EFMKS can be obtained from the corresponding author at the request of professionals or organizations who wish to use it.

Statistical analysis was performed using SPSS version 24.0. Descriptive statistics, such as means, standard deviations, and frequencies, were used to present the demographic characteristics of the participants and to describe the scale.

Factor structure of the EFMKS

Exploratory factor analysis (EFA) was used to assess the construct validity of the instrument and explore the factor structure of the EFMKS. The EFA was conducted by using Principal Components Analysis (PCA) with Varimax rotation. The appropriateness of the factor model was evaluated based on three criteria. The magnitude of Kaiser–Meyer–Olkin (KMO) test was computed to measure sampling adequacy, which should be greater than 0.70 for a satisfactory factor analysis to proceed, communalities should be above 0.35, and the Bartlett Sphericity test was also applied to the data and should be statistically significant. The statistical criteria guiding the determination of the number of the factors to retain were eigenvalues greater than 1.0 and the visual inspection of Catell’s scree test, looking for the break point where the curve flattened out. The next step involved interpreting the rotated solution by identifying which items loaded substantially on each retained factor.

Discriminant and construct validity

Discriminant validity was assessed by examining the intercorrelations between the factors of the EFMKS. Construct validity was established by assessing the ability of the EFMKS to distinguish between subgroups of each profession (midwives and obstetricians) known to differ in knowledge and clinical competences (subgroup of student midwives and resident obstetricians vs professional midwives and obstetricians respectively). Pearson correlation coefficients were used to measure the linear associations among the EFMKS factors. Any factor that correlated by > 0.7 was considered to overlap conceptually.

The study protocol was reviewed and approved by the Elena Benizelou-Alexandra Hospital Research and Ethics Committee (No 12/14-10-2015). Eligible participants were also assured about the confidentiality and anonymity of their responses. Written consent was taken from all the participants before filling in the questionnaires.

RESULTS

The majority of the professionals (71.9%, n= 46) had clinical experience in the labor ward for no more than five years (53.2% of midwives, n= 17, and 90% of the obstetricians, n=29). The midwifery students (n=32) participated in the last training year and the resident obstetricians had a mean duration of clinical experience of 2.5 years. The majority of the professionals (67.2%, n= 43) reported that they felt confident about interpreting CTG traces (59.4% of midwives, n= 19, and 75% of the obstetricians, n=24). However, approximately only half of the professionals (54.7%, n= 35) reported that they felt that their training adequately prepared them for CTG usage (56.3% of midwives, n= 18, and 53.1% of the obstetricians, n=17).

Reliability assessment: Internal Consistency and Test-Retest Reliability.

The Cronbach’s alpha coefficient of internal consistency for the 10-item scale was 0.89 and none of the items improved the scale’s Cronbach’s alpha estimate if deleted. The test-retest reliability of the scale for the two administrations was correlated at 0.85 (p<0.01).

Factor structure of the EFMKS

The Kaiser-Meyer-Olkin value was 0.824 and Bartlett’s test of sphericity reached statistical significance (χ2 = 747, df = 45, p < 0.001). These findings indicated that the data were suitable for a factor analysis. All initial communalities were ≥ 0.35, and all of them ranged from 0.629 to 0.885 (Table 1). The exploratory factor analysis suggested three factors with eigenvalues greater than 1, accounting for 72.99% of the variance (Table 1). All factors with an eigenvalue more than >1 showed consistency with the visual scree plot.

Table 1.

Factor structure of the Electronic Fetal Monitoring Knowledge Scale (EFMKS)

Item/ Item statement	Loading	Communality
Factor 1 (Eigenvalue = 3.32, variance explained = 33.27%, α = 0.85
3/ Key characteristics of a reactive NST	0.812	0.771
4/ Definition and attribution of variable decelerations	0.805	0.669
1/ Range and determinants of FHR baseline	0.778	0.793
4/ Range and determinants of FHR variability	0.751	0.832
Factor 2 (Eigenvalue = 2.41, variance explained = 24.12%, cumulative variance explained = 57.39%, α = 0.82
10/ Classification of CTG traces from compensatory to abnormal	0.860	0.885
6/ Definition and management of severe bradycardia	0.725	0.635
5/ Definition and attribution of late decelerations	0.712	0.680
Factor 3 (Eigenvalue = 1.56, variance explained = 15.6%, cumulative variance explained = 72.99%, α = 0.80
7/ Progressive hypoxia and CTG traces	0.785	0.637
8/ Identification of risk for neurological defect and ph	0.749	0.629
5/Apgar score, accelerations and risk of acidosis	0.745	0.769
NST: Non stress test, FHR: Fetal Heart Rate, CTG: Cardiotocography

Factor interpretation and naming

Inspection of the derived factors revealed meaningful groupings of the items. Factor 1 had four strongly loading items: on Q1 “range and determinants of fetal heart rate (FHR) baseline”, on Q2 “range and determinants of fetal heart rate variability”, on Q3 “key characteristics (accelerations) of a reactive NST”, and on Q4 “identification and attribution of variable decelerations” (Table 1). Substantively, the considered interpretation of this factor seems best focused on the midwives’ and doctors’ knowledge on key elements of CTG and on identification of normal CTG patterns. The factor was therefore named as “key elements of CTG and normal CTG patterns”.

Factor 2 had three strongly loading items: on Q5 “identification and attribution of late decelerations”, on Q6 “management of bradycardia during labor” and on Q10 “classification of CTG traces from compensatory to abnormal”.two items (5, 6) also loaded on Factor 1, but they were assigned to the factor with the highest loading. For Factor 2, the underlying concept seemed to be on abnormal CTG and on CTG during labor. The factor was therefore designated as “suspicious and abnormal CTG patterns”.

Factor 3 had three strongly loading items: on Q8 “risk of neurological defect and acidosis”, on Q7 “association between Apgar score, accelerations during labor and acidosis” and on Q9 “association between progressive hypoxia and CTG traces”. The factor was therefore designated as “hypoxia, acidosis and CTG traces”.

Discriminant and construct validity

The significant and positive associations between the mean score of EFMKS scale and the professional experience of participants suggested that EFMKS achieved discriminant and construct validity. The sensitivity analysis detected a significant difference in mean test score between obstetricians and residents (t= 5.717, p = 0.020) and between midwives and student midwives (t=4.553, p = 0.033), indicating acceptable test construct abilities.

The score of the 10-item EFMK scale ranged from 0 (indicating no correct answers) to 10 (indicating all correct answers), and professionals with scores of greater than five (the mid-point) were classified as having good knowledge about EFM, and those with scores of four and below were classified as having poor knowledge. The rationale for using a cut off of 5 for the knowledge scale was pragmatic as 5 represented the midpoint and the median of the scale.

The mean score for the total EFMKS was 6.84 (SD = 3.11), for the subscale measuring the “key elements of CTG and normal CTG patterns” with range from 0 to 4, the mean score was 2.31 (SD=1.7); for the “suspicious and abnormal CTG patterns” subscale (range from 0 to 3), the mean score was 2.21 (SD=0.96), and for the “hypoxia, acidosis and CTG traces” subscale (range from 0 to 3), the mean was 2.31 (SD= 0.85). Taking into consideration the midpoint of the scale (5), the mean score of our sample indicated a good level of knowledge. More specifically, 73.4% of the total sample (71.9% of the professional midwives and 75% of the professional obstetricians) had a good level of knowledge (score of more than 5 points).

DISCUSSION

It has been well-documented that when electronic fetal monitoring is performed by professionals without having the appropriate knowledge and skills, the possible consequences are an increased rate of Cesarean sections and increased litigation regarding avoidable intrapartum asphyxia (13,14). However, an exhaustive search of the literature has failed to identify previous studies on developing or using a validated instrument for measuring midwives’ and obstetricians’ knowledge and skills related to electronic fetal monitoring.

The purpose of this study was to develop and validate a self-repost measure of knowledge, skills and clinical decision-making related to electronic fetal monitoring. Through an iterative, rigorous instrument development process, the EFMK scale was developed and tested. The EFMKS was developed by using an integrated mixed-methods approach that included literature reviews, international guidelines, professional focus groups, expert consultations and a psychometric survey evaluation. The EFMK scale demonstrated good content validity, an easily interpretable three-factor structure, high internal consistency, high test-retest reliability, and satisfactory discriminant validity with sample characteristics.

Exploratory factor analysis is a useful analytic method, primarily a data-driven approach, which is generally used when no sufficient theoretical or empirical basis exists to make strong assumptions about how many constructs or factors underlie a set of items (15). The results of the exploratory factor analysis suggested that a discriminative capacity existed among the items and that a three-factor solution was the most appropriate. The underlying concepts of the three-factor structure were interpreted by the authors and labeled “key elements of CTG and normal CTG patterns”, “suspicious and abnormal CTG patterns” and “hypoxia, acidosis and CTG traces” in accordance with the essence of the items and these factors. This finding indicated that the knowledge and competence skills related to electronic fetal monitoring were multidimensional, variable and related not only with CTG trace interpretation and classification but also with fetal physiology and acid-base balance in relation to cardiotocography.

All the items loaded satisfactorily in factors with loadings ranging from 0.71 to 0.86 and had item-total correlation values higher than 0.30 but without exceeding 0.80 indicating that all the items measured a relevant but not the same underlying construct. The three-factor solution of the EFMK 10-item scale cumulatively accounted for the 72.9% of variance. This result indicated that the developed EFMK scale explained a high proportion of the tested knowledge variables. An initial data analysis should use EFA to identify potential factors.

Convergent validity (e.g. the extent to which a test correlates with other variables with which it theoretically should correlate) could not be assessed because, according to the authors’ knowledge, no other validated instrument measuring EFM knowledge was available.

Taking into consideration the midpoint of the scale (5 points), the mean score of our total sample indicated a good level of knowledge on EFM. More specifically, 71.9% of the professional midwives and 75% of the professional obstetricians had a good level of knowledge (score of more than 5 points). Despite differences in basic training and philosophies, an independent t-test demonstrated no difference between midwives and obstetricians knowledge related to EFM. This finding was similar to the findings of previous studies (16,17). However, it was noteworthy that approximately 30% of the professional midwives and 25% of the obstetricians, both of the professional groups with clinical experience in the labor ward, had a low level of knowledge in EFM. These results support the findings of previous studies, which suggest that knowledge regarding EFM remains intact only for some months after a training program (18) and that an annual update course in fetal monitoring is recommended and necessary (7,19,20,21).

Results of this study need to be interpreted within the light of some limitations. First, convenience sampling was used. Additionally, the sample of professionals was drawn only from two public hospitals from the capital of Greece. Thus, the results of this study may have introduced a selection bias and produced a non-representative sample of midwives and obstetricians in Greece so that the results are not likely to be generalizable. Therefore, it is essential to explore the psychometric properties and assess this scale among professionals from different settings (e.g., private hospitals) and different geographic regions.

CONCLUSION

In conclusion, the EFMKS was found to have satisfactory psychometric properties with a meaningful three-factor structure, good internal reliability and good discriminant and construct validity. Based on the study findings, the EFMKS may be used for evaluating, in a multidimensional way, the EFM knowledge of midwives and of obstetricians and for identifying the areas of knowledge gap (e.g., acid-base balance) among professionals who use it. Taking into consideration the study findings it can be concluded that an annual multi-professional CTG training is necessary for all intrapartum staff. It is hoped that through training and consequently the improvement of knowledge and competences regarding EFM the nationally increasing number of Cesarean sections may be reduced. In addition, future research should investigate the factorial structure of the EFMKS and must be verified in another sample of professionals through a confirmatory factor analysis.

Phase I: Establishment of twenty-five item pool	Phase II: Item reduction because of semantic similarityAction Justification
Q1. Define normal range/ determinants of FHR* baseline	Remains	Q1 remains as it not redundant
Q2. Define abnormal range/ determinants of FHR tachycardic baseline	Deleted	Q2 is a redundant item with Q1 in terms of meaning
Q3. Define abnormal range/ determinants of FHR bradycardic baseline	Deleted	Q3 is a redundant item with Q1 in terms of meaning turned into a sub-response of Q15
Q4. Define normal range/ determinants of FHR variability	Remains	Q4 remains as it not redundant
Q5. Define sinusoidal pattern	Deleted	Q5 is a redundant item with Q4 in terms of meaning and turned into a sub-response of Q18
Q6. Define abnormal/decreased variability	Deleted	Q6 is a redundant item with Q4 in terms of meaning
Q7. Define abnormal/marked variability	Deleted	Q7 is a redundant item with Q4 in terms of meaning
Q8. Define accelerations	Deleted	Q8 is a redundant item with Q9 in terms of meaning
Q9. Key characteristics of a reactive NST*	Remains	Q9 remains as it not redundant
Q10. Define decelerations	Deleted	Q10 is a redundant item with Q11, Q12, Q13 in terms of meaning
Q11. Define and attribute of variable decelerations	Remains	Q11 remains as it not redundant
Q12. Define and attribute of late decelerations	Remains	Q12 remains as it not redundant
Q13. Define and attribute of early decelerations	Deleted	Q13 is a redundant item with Q11, Q12 in terms of meaning and turned into a sub-response of Q11 and Q12
Q14. Classification of CTG* traces from compensatory to abnormal	Remains	Q14 remains as it not redundant
Q15. Definition and management of severe bradycardia in second stage of labor	Remains	Q15 remains as it not redundant
Q16. Interpret a fetal scalp-blood sample	Remains	Q16 remains as it not redundant
Q17. Differences between metabolic and respiratory acidosis	Deleted	Q17 is a redundant item with Q16, Q19, Q20 in terms of meaning and turned into a sub-response of Q19
Q18. Progressive hypoxia and CTG traces	Remains	Q18 remains as it not redundant
Q19. Identification of risk for neurological defect and ph	Remains	Q19 remains as it not redundant
Q20. Apgar score, accelerations and risk of acidosis	Remains	Q20 remains as it not redundant
Q21. Evaluate the pattern of contractions	Remains	Q21 remains as it not redundant
Q22. Differences between CTG and STAN*	Remains	Q22 remains as it not redundant
Q23. Operate CTG equipment and application of transducers	Remains	Q23 remains as it not redundant
Q24. Application of fetal scalp electrode	Remains	Q24 remains as it not redundant
Q25. Advantages and disadvantages of internal and external monitoring	Remains	Q25 remains as it not redundant
* Fetal Heart Rate (FHR), Non stress test (NST), Cardiotocography (CTG), ST-Analysis (STAN)