Nationwide implementation of the non-invasive prenatal test: Evaluation of a blended learning program for counselors.

This study assesses the results of a mandatory blended learning-program for counselors (e.g. midwives, sonographers, obstetricians) guiding national implementation of the Non-Invasive Prenatal Test (NIPT). We assessed counselors' 1) knowledge about prenatal aneuploidy screening, 2) factors associated with their knowledge (e.g. counselors' characteristics, attitudes towards NIPT), and 3) counselors' attitudes regarding the blended learning. A cross-sectional online pretest-posttest implementation survey was sent to all 2,813 Dutch prenatal counselors. Multivariate linear regression analyses were performed to identify associations between counselors' knowledge and e.g. their professional backgrounds, work experience and attitudes towards NIPT. At T0 and T1 1,635 and 913 counselors completed the survey, respectively. Overall results show an increased mean number of correct answered knowledge questions; 23/35 (66%) items at T0 and 28/37 (76%) items at T1. Knowledge gaps on highly specific topics remained. Work experience and secondary care work-setting were positively associated with a higher level of knowledge. Most counselors (74%) showed positive attitudes towards the blended learning program. The mandatory blended learning, along with learning by experience through implementation of NIPT, has facilitated an increase in counselors' knowledge and was well received. New implementations in healthcare may benefit from requiring blended learning for healthcare providers, especially if tailored to professionals' learning goals.

Introduction

Non-Invasive Prenatal Testing (NIPT) is increasingly available worldwide, as the field of prenatal screening for fetal anomalies is rapidly implementing new genetic technologies [1]. NIPT is a maternal blood test using cell-free DNA, mostly targeted at the common aneuploidies Down-, Edwards- and Patau syndrome. In most countries, NIPT is readily commercially available, although it has increasingly been offered as an aneuploidy screening test, within nationwide healthcare programs, in which mostly primary maternity care providers offer accompanying pre-test counseling [1-3]. Given this context, obstetric care providers have had to adapt by updating their knowledge about NIPT rapidly. Such knowledge, e.g. understanding the advantages and limitations of NIPT compared to other prenatal tests and knowing how to interpret the results, is essential to allow an adequate pregnant women’s informed choice regarding prenatal screening [4-7].

Given the importance of counselors’ up-to-date knowledge about NIPT, education of counselors has been an active topic for policymakers and educators. This has resulted in the development of programs aimed at improving counselors’ knowledge and communicational skills [5, 8]. In a public healthcare system, counselors’ education on NIPT typically involves a diverse palette of educational activities, so-called blended learning. Blended learning includes, for instance, face-to-face training, e-learning, assessments, and web-based information such as guidelines and Option Grid decision aids [9-12]. One of the challenges of educational programs is that counselors’ professional backgrounds (e.g. obstetrician, midwife, sonographers) and therefore professional knowledge differs significantly [9]. Furthermore, despite efforts to educate counselors properly, certain aspects such as the origin of cell-free fetal DNA used for NIPT and discordant/false-positive rates of NIPT seem difficult to understand [6, 9, 13]. The resulting knowledge gaps are problematic since insight into NIPT’s test characteristics is fundamental for understanding the clinical implications of the test results and the fact that NIPT is not diagnostic, diagnostic options and thereby for providing adequate counseling that supports informed decisions [4, 6, 14, 15].

Alongside these difficulties in education, providers offering counseling for prenatal aneuploidy screening report additional challenges such as biased education by commercial test providers, lack of nationwide consensus about the educational content, and trouble maintaining up-to-date knowledge about NIPT and its target anomalies [2, 5]. The challenges also result from the rapidly widening scope of NIPT, the diversity of analyses used by different laboratories, and the resulting lack of sustainable, reliable knowledge about NIPT that accompanies the evolving technical accuracy of NIPT [16, 17]. Furthermore, relatively little is known about efficient and effective educational programs accompanying implementation of NIPT, to address the educational difficulties counselors face. Finally, there is limited knowledge about the impact of counselors’ attitudes regarding NIPT on outcomes of the educational programs [18]. Such information is essential to learn more about how educational programs can be developed, implemented and improved to provide efficient and effective strategies for counselor education.

In the Netherlands, national studies (TRIDENT-1, TRIDENT-2) were launched to evaluate the organizational implementation of NIPT, test characteristics and outcomes, women’s perspectives as well as the counselors’ perspectives, and the educational requirements. In 2017, alongside the transition of NIPT from a second-tier test, only available for women with a high risk pregnancy based on the First trimester Combined Test results or medical indication, to a first-tier screening test for all pregnant women, the Dutch government set new educational requirements for healthcare professionals who provide counseling for prenatal aneuploidy screening [19, 20] (S1 Appendix). In addition to existing requirements for continuing education, counselors were obliged to participate in a blended learning program about NIPT to improve their knowledge and skills on this topic [11, 21] (Fig 1). This program addressed the first three of the four levels of Miller’s Pyramid of Professional Competence (knows, knows how, shows how and does) [22]. Our study aimed to assess the results of this mandatory blended learning program guiding national implementation of NIPT by evaluating: 1) counselors’ knowledge about prenatal aneuploidy screening, 2) factors associated with knowledge (e.g. counselor’s characteristics, attitudes towards NIPT), and 3) attitudes regarding the blended learning program. Although the blended learning program addressed the first three levels of Miller’s Pyramid of Professional Competence, here we only evaluated the effects of the blended learning program on the first level ‘knows’.

Fig 1

Overview of the blended learning for counselors and timing of knowledge assessment (T0 and T1) NIPT, Non-Invasive Prenatal Testing.

Materials and methods

The current study was part of the TRIDENT-2 study to evaluate the introduction of NIPT as first-tier screening in a governmentally-funded prenatal aneuploidy screening program [9, 19, 23–25].

Study design and procedures

A pre- and post-implementation survey study design was developed using an online knowledge questionnaire distributed to all registered prenatal counselors. The first assessment (T0) took place February-March 2017, before the start of the NIPT blended learning program and implementation of NIPT as first-tier aneuploidy screening test. To facilitate tailoring of the blended learning activities, we provided Regional Centers for prenatal screening (RCs) with the overall anonymous results of pre-implementation knowledge scores of the counselors. The second measurement (T1) took place approximately nine months later, after implementing the blended learning program and first-tier NIPT (November 2017-January 2018). Study participation was voluntary for completion of both questionnaires; no incentives were given.

Measures

We asked for counselors’ characteristics (age, profession, work experience, region), counseling education and number of counseling sessions a month. Furthermore, at both T0 and T1, counselors’ attitudes towards NIPT as first-tier aneuploidy screening test were measured using one, closed-ended question "What do you think about offering NIPT as a first-tier screening test?" (answer options: ’good’, ’not good’, ’neutral’).

Knowledge questionnaire

The knowledge questionnaire was developed based on questionnaires used in previous Dutch studies [6, 13, 26–28] and international literature [4, 6, 29] (S2 Appendix). Development was done by a multidisciplinary team, consisting of representatives from midwifery, gynecology, psychology, clinical genetics and health sciences. After T0, the 38- item questionnaire was adjusted based on feedback from participants, and policy officers from the Centre for Population Screening at the National Institute for Public Health and the Environment (RIVM-CvB) and the RCs; unclear questions (N = 3) were excluded, and missing knowledge domains were added (N = 2). The resulting 35-item knowledge questionnaire at T0 and a 37-item knowledge questionnaire at T1 required the answers ’true’, ’false’ or ’I do not know’, and consisted of seven themes/domains: 1) Prenatal aneuploidy screening program, 2) Test characteristics of NIPT, 3) NIPT versus FCT, 4) Additional findings from NIPT and FCT, 5) In- and exclusion-criteria for NIPT and FCT, 6) Follow-up tests, and 7) Fetal structural Anomaly Screening.

Attitude regarding the blended learning education

At T1, counselors’ attitudes towards the mandatory blended learning were measured by one question ’What do you think about the mandatory blended learning education on providing the knowledge and counseling skills you have to complete to stay certified as a counselor?’ to be answered as ’good’, ’not good’, or ’other’ including space to explain the given answer.

Data collection

The questionnaire was distributed to all 2813 certified Dutch counselors [30]. Counselors received an email from their RC with a request to participate. The email contained a link to the online questionnaire and information about the research, such as the study’s purpose, voluntary participation and privacy guarantee. With this method, we aimed to prevent as much selection and information bias as possible. Two and four weeks after the questionnaire’s distribution, reminders were sent for both T0 and T1 requesting the completion of the questionnaire.

Analyses

Descriptive statistics (N, %, mean (M), or standard deviation (SD) where appropriate) were used to describe the background characteristics of participating counselors, including attitudes towards NIPT. We compared characteristics of the respondents of T0 versus T1, and to available characteristics of the Dutch counselor population to examine the representativeness and comparability of different professions (e.g. midwife, gynecologist, sonographer) and regions. Counselors’ professional background were dichotomized according to their work setting into ’primary care’ (e.g. midwives and sonographers) and ’secondary/tertiary care’ (e.g. gynecologists, medical doctors in resident and nurses).

Counselors’ pre- and post-blended learning NIPT knowledge

Before the analyses, the answers to the knowledge questions were dichotomized into ’correct’ or ’not correct’ categories; the answer ’I do not know’ was coded as ’not correct’. Subsequently, numbers and percentages of correct answers were calculated per question, per knowledge theme and as an average of the total amount of correctly answered questions. Analyses were subsequently repeated for counselors connected to each of the RCs. If participants had answered at least 70.0% of the questions correctly, the same cutoff as used in the national e-learning and assessment developed by the RIVM-CvB (https://www.pns.nl/professionals/nipt-seo/scholing-counselors/e-learning), their knowledge was considered sufficient.

For analyzing the potential influences of the NIPT blended learning program and nine months of experience with NIPT as a first-tier test on counselors’ knowledge, frequencies of correctly answered questions were calculated per knowledge item. In addition, the sum of correctly answered items for T0 and T1 were calculated separately for the whole group and each RC.

We used multivariable, linear regression analyses to identify differences between results of the blended learning education of counselors’ knowledge at T0 and T1. Separate models were analyzed for T0 and T1, because we did not want to lean on the assumption that the same participants were included at both assessments. Backward selection was performed on the initial multivariable model for the sequential removal of variables (profession, work setting, work experience, number of counseling sessions a month, and attitude towards NIPT). In each step, the variable with the highest p-value was removed until the model contained only statistically significant variables (two-sided p < 0.05). One sample proportion tests were performed to identify significant differences between the results of T0 and T1.

Attitudes regarding the blended learning

Attitudes regarding the mandatory blended learning education for counselors were described by applying descriptive statistics (N, %, M, SD where appropriate). All analyses were carried out using SPSS 24.0, and a significance level of p≤0.05 was maintained.

Ethical considerations

The study protocol was approved by the Medical Ethical Committee of the VU University Medical Center, Amsterdam, the Netherlands (no. 2017.165). We collected online written consent using a ‘yes’ or ‘no’ button from all participating counselors, before they could contribute in the online survey.

Results

Table 1 (Counselors’ characteristics pre- (T0) and post (T1) NIPT implementation and participation in the blended learning program) shows the background characteristics and participation of respondents. At T0, 1,635 counselors completed the knowledge questionnaire (response 1,635/2813 = 58%) and at T1, 913 counselors did so (response 913/2,813 = 32%). At both measurements, most counselors worked in primary care (83% and 80%, respectively); the majority worked as a midwife. At T1 significantly more counselors (86%) showed a positive attitude towards NIPT as a first-tier aneuploidy screening compared to T0 (75%). At T0, self-reported mean years of work experience as a counselor was M = 9.7 (SD = 5.6) years and at T1, M = 8.6 (SD = 5.6) years. At T1, the number of counseling consultations was M = 9.2 (SD = 7.6) per month.

Table 1

Counselors’ characteristics pre- (T0) and post (T1) NIPT implementation and participation in the blended learning program.

Characteristics	T0	T1	Dutch counselor population
	N = 1,635	N = 913	N = 2,8135
Age mean (SD) in years	37.0 (10.9)	38.0 (11.1)	-
Experience (in years), n (%)
≤5 years	395 (24.2)	268 (29.4)	-
6–10 years	487(29.8)	406 (44.5)
≥11 years	753 (46.0)	239 (26.1)
Profession1, n (%)
Midwife primary care	1,358 (83.1)	727 (79.6)	2,125 (79.1)
Midwife secondary care	110 (6.7)	41 (4.5)	79 (3.0)
Gynecologist	91 (5.6)	41 (4.5)	40 (1.5)
Sonographer2	244 (14.9)	52 (5.7)	441 (16.4)
Other3	80 (4.9)	47 (5.1)
Regional Prenatal Screening Center4, n (%)
Region 1	323 (19.7)	216 (23.7)
Region 2	255 (15.6)	138 (15.1)
Region 3	190 (11.6)	78 (8.5)
Region 4	147 (9.0)	80 (8.8)
Region 5	302 (18.5)	185 (20.3)
Region 6	234 (14.3)	156 (17.1)
Region 7	207 (12.7)	87 (9.5)
Attitude toward NIPT, n (%)
Positive	1,228 (74.7)	724 (85.6)
Negative	117 (7.1)	35 (4.1)
Neutral	299 (18.2)	87 (10.3)
Participation in blended learning activities
E-learning completed, n (%)
Yes	Not	909 (99.6)
No	applicable	4 (0.4)
Attended face-to-face seminar, n (%)
Yes	Not	910 (99.7)	-
No	applicable	3 (0.3)

Valid percentages are shown

1 Participants might have had more than one profession and answered accordingly; therefore, numbers add up to more than N = 1,635/N = 913

2 Most sonographers were also midwives

3 Gynecologist in residence, nurse, reproductive physician

4 Data of the two Regional Prenatal Screening Centers in Amsterdam are merged

5 Annual Report of all national Regional Prenatal Screening Centers 2018.

Counselors’ pre- and post- blended learning NIPT knowledge

Table 2 (Counselors’ correctly answered knowledge questions at T0 (N = 1,635) and T1 (N = 913) shows counselors’ pre- and post-blended learning of NIPT as percentage of counselors who correctly answered the items in various knowledge themes. Analyses of sum scores of the respectively 35 and 37 items at T0 and T1 showed that counselors answered significantly more items correctly at T1 compared to T0; overall mean score of M = 28 (76%; SD = 3.0) correct answers versus M = 23 (66%; SD = 3.8), respectively. The overall mean scores per RC are shown in Table 3 (Overall mean scores of counselors’ knowledge represented per Prenatal Screening Region). Mean scores per RC at T0 varied between 22.5–24.0 (64%-69%) correctly answered questions, and at T1 between 28.0–29.6 (76%-80%) correctly answered questions. There were no significant difference in knowledge scores between RCs.

Table 2

Counselors’ correctly answered knowledge questions at T0 (N = 1,635) and T1 (N = 913).

Knowledge Theme	Question	Answer	T0: N (%)	T1: N (%)
Prenatal anomaly screening program	The pregnant woman is free to decide whether or not to have a fetal anomaly scan.	True	1,634 (99.9)	911 (99.8)
	If a pregnant woman is NOT considering terminating the pregnancy, then it is NOT necessary to discuss the prenatal, anomaly screening options.	False	1,625 (99.4)	910 (99.7)
	The pregnant woman has the right to decide whether or not to receive information about prenatal screening options.	True	1,623 (99.3)	907 (99.3)
	As a counselor, it is important to advice the pregnant woman whether or not to opt for prenatal anomaly screening, because a counselor has a better insight into the risks than the pregnant woman herself.	False	1,579 (96.6)	887 (97.2)
	If a woman chooses NIPT, she can also have a nuchal fold measurement taken as part of the screening program.	False	650 (39.8)	823 (90.1)
	Women who had previously been pregnant with a child with trisomy 21, 18, 13 only receive counseling in a prenatal diagnosis center for screening for trisomy 21, 18 and 13.	True	417 (25.5)	627 (68.7)
	If NIPT finds signs of cancer in a pregnant woman, this is always reported to her even if she did not opt for disclosure of incidental findings.	True	not applicable	622 (68.1)
Trisomy 13, 18, 21	Most infants with Edward’s syndrome die before birth or shortly after birth.	True	1,602 (98.0)	900 (98.6)
	The most common form of Down syndrome is hereditary.	False	1,515 (92.7)	850 (93.1)
	Approximately half of all infants born with Down syndrome have a heart defect.	True	1,361 (83.2)	815 (89.3)
	Delays in the development of motor skills in an infant with Down syndrome does NOT affect other areas of development.	False	1,117 (68.3)	614 (67.3)
	Most babies with Down syndrome are born to women under the age of 36.	True	1,019 (62.3)	633 (69.3)
	5–10% of infants with trisomy 13 survive beyond the first year of life.	True	693 (42.4)	449 (49.2)
Test characteristics of NIPT	NIPT determines whether the fetus is healthy.	False	1,631 (99.8)	910 (99.7)
	If the result of NIPT is: "negative for trisomy 21,18 and 13", then there is NO chance that the baby has trisomy 21,18 and 13.	False	1,313 (80.3)	753 (82.5)
	If NIPT gives a positive result for trisomy 21 in the initial screening, there is an average 25% chance that the infant does NOT have trisomy 21.	True	921 (56.3)	655 (71.7)
	The cell-free fetal DNA in maternal blood used for NIPT comes from the placenta.	True	899 (55.0)	827 (90.6)
	A failed result in NIPT (no result) occurs in 2 out of 100 tests.	True	867 (53.0)	657 (72.0)
	If NIPT gives a positive result for trisomy 13 in the initial screening (Patau syndrome), there is an average of a 75% probability that the infant does NOT have trisomy 13.	True	174 (10.6)	428 (46.9)
NIPT versus FCT	NIPT has a higher sensitivity than the first trimester combined test.	True	1,463 (89.5)	880 (96.4)
	When using NIPT as the initial screening test, fewer pregnant women are sent for follow-up testing than after the first trimester combined test.	True	1,363 (83.4)	830 (90.9)
	For initial screening, the personal cost for the first trimester combined test and NIPT are approximately the same as of 1-4-2017.	True	1,224 (74.9)	882 (96.6)
Additional findings NIPT and FCT	Additional findings (other than trisomy 21, 18 or 13) can result from the first trimester combined test.	True	1,119 (68.4)	505 (55.3)
	For initial screening with NIPT, the pregnant woman can choose whether she wants to hear additional findings.	True	840 (51.4)	901 (98.7)
	If the pregnant woman DOES want to know additional findings, chromosomes other than chromosomes 21, 18 and 13 can also be examined. NIPT results can include more than trisomy 21, 18 and 13.	True	766 (46.9)	856 (93.8)
	As an incidental finding, abnormalities of the placenta can also be detected by NIPT.	True	not applicable	741 (81.2)
Inclusion and exclusion criteria for NIPT and FCT	The first trimester combined test CANNOT be performed if a woman is pregnant with monochorionic twins.	False	898 (54.9)	640 (70.1)
	A thickened nuchal fold (≥3.5 mm) is NOT an indication for NIPT.	True	814 (49.8)	794 (76.0)
	NIPT CAN be performed if a woman is pregnant with monochorionic twins.	True	741 (45.3)	481 (52.7)
	Monochorionic twin pregnancies are monozygotic.	True	489 (29.9)	577 (63.2)
	If a pregnant woman has a chromosomal abnormality herself, she may NOT eligible for NIPT.	True	384 (23.5)	649 (71.1)
	If a pregnant woman is 17 years old, she CAN be screened by NIPT.	False	348 (21.3)	719 (78.8)
Follow-up tests	Chorionic villus sampling and amniocentesis can demonstrate with more certainty than NIPT whether there is a trisomy in the fetus.	True	1,484 (90.8)	885 (96.9)
	If the first trimester combined test shows an increased risk of 1 in 200 or higher, the woman can then still opt for NIPT.	True	1,521 (93.0)	826 (90.5)
	If the nuchal fold measurement in the first trimester combined test is ≥3.5 mm, and the karyotyping appears normal in the subsequent invasive test, the parents can be reassured.	False	1,081 (66.1)	633 (69.3)
Fetal structural Anomaly Scan	A Fetal Anatomy Scan is used to investigate physical abnormalities in an unborn baby.	True	1,613 (98.7)	901(98.7)
	The primary responsibility for making a medical indication for Advanced Ultrasound Examination lies with the counselor.	True	1,498 (91.6)	840 (92.0)

NIPT = Non-Invasive Prenatal Test, FCT = First trimester Combined Test. Grey fields indicate that <70% of the counselors answered an item correctly.

Table 3

Overall mean scores of counselors’ knowledge represented per prenatal screening region.

Region1	Overall mean knowledge scores at T0	Overall mean knowledge scores at T1
	M (SD)	M (SD)
1	24.3 (3.9)	29.6 (2.7)
2	22.5 (3.7)	28.9 (2.8)
3	22.6 (3.5)	28.7 (2.8)
4	22.9 (3.5)	28.0 (4.3)
5	23.4 (3.7)	28.8 (3.1)
6	23.4 (3.7)	28.6 (2.8)
7	24.0 (4.0)	28.2 (2.9)

1 Data of the two Regional Centers for prenatal screening in Amsterdam are merged.

Considering the scores on the separate knowledge themes, at both T0 and T1, all items of the themes NIPT versus FCT and Fetal structural Anomaly Screening (FAS) were correctly answered by more than 70% of the participating counselors. At T0 all six items of the theme In- and exclusion-criteria NIPT and FCT were answered correctly by 21% - 55% of the counselors. At T1, after the blended learning program, at least 70% of the counselors answered four of these 6 questions correctly. The remaining two items of this theme (on monochorionic pregnancies) were answered correctly by 53% and 63% of the. Half of the items on the anomalies screened for (Trisomy 13, 18 and 21) were answered correctly by less than 70% of the counselors at T1.

Multivariate linear regression analyses showed that both at T0 and T1, more years of work-experience and a secondary-care work setting were associated with significantly more correctly answered knowledge questions (T0: b = 0.068, p < 0.0001 and b = 0.003, p < 0.001; T1: b = 0.056; p = 0.002 and b = 0.002; p < 0.0001, respectively).

Attitudes regarding the blended learning program

At T1, most counselors had a positive attitude towards the mandatory blended learning program for counselors; 648 (74%) valued the blended learning as ’good’, 130 (14.9%) as ’not good’, and 97 (11.1%) counselors answered ’other’. Examples of remarks were: ’once every two years a blended learning training is too often’, ’okay, if online education is used where appropriate’, ’too much regarding counseling skills training’, and ’okay, only if major changes in (medical) developments have been made’.

Discussion

Our study aimed to assess the results of a mandatory blended learning program on prenatal screening counselors’ knowledge as part of the Dutch national implementation of NIPT. The learning program was designed to bridge knowledge gaps identified at the time NIPT was transitioned from a second-tier test, for women at an increased risk for common trisomies based on the FCT or medical history, to a first-tier test available to all pregnant women. The program was intended to be a flexible and effective framework incorporating a variety of media and methods such as e-learning and face-to-face seminars. The second voluntary knowledge assessment took place approximately nine months after NIPT transition and followed administration of the blended learning program.

Counselors, the majority of whom were midwives (80%), answered more knowledge items correctly after NIPT implementation and blended learning (76% versus 66%). Knowledge levels were positively associated with years of work experience and a secondary care work-setting. The majority (75%) had positive attitudes towards NIPT at the time of the first assessment; this rose significantly to 85% in the second survey. About three-fourth of the participating counselors had a positive attitude towards the mandatory blended learning program.

In line with previous studies in the Netherlands, counselors’ knowledge was generally considered sufficient [6]. Although counselors’ knowledge overall improved after NIPT education and several months of experience with NIPT as a first-tier test, specific areas still showed a need for improvement, despite the tailored education provided. Knowledge on the theme Trisomy 13, 18 and 21 did not improve over time; only three out of six questions were answered correctly by >70% at both T0 and T1. The incorrectly answered questions concerned development delays in Down syndrome, the fact that most children with Down syndrome are born to women younger than 36 years old, and the survival rates of a child with trisomy 13. The theme test characteristics of NIPT showed significant improvement, with 2/6 questions answered correctly by >70% at T0 increasing to 5/6 questions answered correctly at T1. The question most counselors struggled with in this theme was the false positive rate of Trisomy 13. Still, the percentage of respondents who correctly answered this question rose from 11% to 47%. The theme additional findings of NIPT and FCT also showed significant improvement in rates of correctly answered questions. The items about monochorionic pregnancies in the theme In- and exclusion-criteria NIPT and FCT were still only answered correctly by 53–63% of the counselors, at T1. This could be explained by the fact that most of the counselor work in a primary care setting and did not counsel monochorionic pregnancies, who were mostly counselled in hospitals.

The finding that not all post-implementation questions were answered correctly by >70% of the counselors may be due to the fact that some questions contained detailed knowledge of rare conditions and precise numbers which might be hard to remember, given the lack of daily use of such information. Nevertheless, optimisation of knowledge is essential since women rely on information provided by counselors to make an informed decision about prenatal aneuploidy screening, and can also influence their perception of the quality of life for children with anomalies included in NIPT. The need for providing more information on living with anomalies such as Down syndrome has been emphasised [31].

From the literature, we know that the baseline level of a counselors’ knowledge is associated with their professional background, and ongoing participation in educational programs for counselors is effective in improving knowledge levels [6, 13]. A Dutch survey study among primary care midwives showed that a positive attitude regarding prenatal screening for Down syndrome was positively associated with knowledge levels [32]. Although we observed a rise in the percentage of counselors who showed positive attitudes towards NIPT, we found no association between these attitudes and counselors’ knowledge. One explanation might be that the majority of counselors already had positive attitudes. In addition, although the uptake of NIPT differs substantially between regions (ranging from 32% to 68%) [33], according to our results, the level of counselors’ knowledge did not.

Consistent with other research [5], our respondents valued participating in educational programming regarding prenatal aneuploidy screening to improve their skills and stay up-to-date. However, also in line with other research [34], some counselors commented that the blended learning program should add something to their competencies, which might be independent of the frequency of once every two years. This outcome was also acknowledged by Dutch policymakers, resulting in a thorough evaluation of the blended learning program in 2019, and the ongoing redesign of content and program frequency in 2021 [35].

A strength of our study was the relatively high response rate. Furthermore, we recruited counselors from all over the country, resulting in a study population that was representative of the Dutch counselor population. However, we could not link the individual answers of participants in the first questionnaire to those who participated in the second one. Therefore, we do not know how much counselors did participate at both measurements. Moreover, the sole impact of the blended learning program cannot be distinguished from the nine months of ‘learning by doing’ through the implementation of first-tier NIPT. Nevertheless, our results suggest that knowledge levels in most topics were significantly improved after nine months, which by itself is promising and maybe a starting point for future research into combining a blended learning educational program with clinical practice. Finally, at T1 the counselors’ mean years of working experience was one year less compared to participants at T0, which might have led to an underestimation of knowledge levels during the second assessment, since knowledge was positively associated with the amount of work experience.

Regarding practical implications, policy makers and educators could use our study results to reflect upon the body of knowledge that counselors for prenatal aneuploidy screening should have. An interdisciplinary panel of experts could help develop such body of knowledge; through interdisciplinary dialogue, counselors can learn about the effect of their counseling on clients views and questions raised once they received a positive screening result. Furthermore, our results stress the importance of assessing knowledge levels of counselors through questionnaires that contain essential and unambiguous content, with a focus on knowledge that all counselors should have. Therefore, the development and use of diagnostic tools to accurately measure counselors’ knowledge may help to increase the awareness of personal knowledge gaps and the formulation of individual learning goals, before participating in educational training programs.

Conclusions

A mandatory blended learning program alongside the implementation of NIPT as a first-tier screening test in the Dutch prenatal screening program significantly improved counselors’ knowledge about prenatal aneuploidy screening. However, certain knowledge areas showed no or little improvement, revealing that educational tools need to be better tailored to identify and bridge the knowledge gaps for prenatal aneuploidy counselors. Individualized learning goals might optimize the effectiveness of the NIPT blended learning program. More research is needed to evaluate the effect of the blended learning program on higher levels of learning such as ’knows how’, ’shows how’ and ’does’, given the essential role counselors have to comprehensively inform pregnant women about prenatal testing.

Dutch prenatal screening setting.

(DOCX)

Click here for additional data file.

Development of the blended learning program.

(DOCX)

Click here for additional data file.

Members of the Dutch NIPT consortium.

(PDF)

Click here for additional data file.

9 Feb 2022

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Antonio Simone Laganà, M.D., Ph.D.

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please provide additional details regarding participant consent. In the ethics statement in the Methods and online submission information, please ensure that you have specified what type you obtained (for instance, written or verbal, and if verbal, how it was documented and witnessed). If your study included minors, state whether you obtained consent from parents or guardians. If the need for consent was waived by the ethics committee, please include this information.

3. We note that the grant information you provided in the ‘Funding Information’ and ‘Financial Disclosure’ sections do not match.

When you resubmit, please ensure that you provide the correct grant numbers for the awards you received for your study in the ‘Funding Information’ section.

4. Please ensure that you refer to Figure 1 in your text as, if accepted, production will need this reference to link the reader to the figure.

Additional Editor Comments:

The topic of the manuscript is interesting. Nevertheless, the reviewers raised several concerns: considering this point, I invite authors to perform the required major revisions.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: No

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: No

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Dear Authors, I found your paper interesting. I believe strong effort should be put in the implementation of campaign to acknowledge health care professionals regarding NIPT.

This test has been a large diffusion, but, women do not perform an adequately informed choice.

Your paper highlight the importance of programs to improve de adequacy of health care professionals counseling regarding NIPT, therefore good job!

I have some minor revisions:

1) I would try to ameliorate the abstract readability, is a bit tortuous, abstract is the first insight into a paper therefore describe your job in an easier way.

- I would have talk about prenatal aneuploidy screening not anomaly screening

- I would have specified who do you mena by Duch prenatal counselors (obstetricians, midwives, family doctors

- Sentence linke " were positively associated with more correctly answered knowledge questions"

I would have said positively associated with a higher level of knowledge for example

In the mean text

The introduction

1) Again discuss about prenatal screening for aneuploidies not anomalies

2) line 62 is essential to facilitate informed choice for expectant parents regarding prenatal

63 screening" I would have said to allow an effective/adequate pregnant women's informed choice

line 89

I would suggest to mention this paper PMID: 33111167, that again shows how an not-adequate counseling from "counselor" associate with a poor understanding of women regarding NIPT, often considered as "diagnostic" and not "screening"

I would recommend your paper to be published with minor revisions

Reviewer #2: This manuscript evaluations the impact of a training program. These kinds of evaluations are useful to inform other providers of similar educational initiatives. The manuscript can be improved by addressing the following issues:

1. Introduction: Provide the reader with a brief description of what Non-Invasive Prenatal Testing is.

2. Introduction: What are first and second-tiers? Anyone from a different health system does not know what this means.

3. Introduction: line 75 - I would not use the word 'unacceptable' - replace it with something like 'problematic'

4. Introduction: line 80 - what is interfering education by commercial test providers?

5. Introduction: line 86-87 It is not correct to say that little is know about how efficient and effective educational programs are for healthcare implementations. There is plenty of literature about education on other topics.

6. Materials and Methods: line 134 - what is RIVM-CvB and RC (spell out acronyms at first use)

7. Materials and Methods line 176 - an unpaired t-test is not the correct statistical test. The sample is presumably mostly the same participants at both assessments. I understand that they can't be matched, but the mean score on the first assessment could be used as the threshold score for the comparison of the posttest scores using a one-sample t test.

8. Materials and Methods line 178-179: Why are separate regressions used for the pre and post?

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

1 Mar 2022

March, 1st 2022

Dear reviewers,

Thank you for your expert review of our manuscript titled “Nationwide implementation of the Non-Invasive Prenatal Test: Evaluation of a blended learning program for counselors”, PONE-D-22-02421.

We have used your comments to improve our paper further, and we are pleased to send you our revised manuscript. Our response after each comment in italics are provided below.

Reviewer #1:

Dear Authors, I found your paper interesting. I believe strong effort should be put in the implementation of campaign to acknowledge health care professionals regarding NIPT.

This test has been a large diffusion, but, women do not perform an adequately informed choice.

Your paper highlight the importance of programs to improve the adequacy of health care professionals counseling regarding NIPT, therefore good job!

Thank you for your encouraging words!

I have some minor revisions:

1) I would try to ameliorate the abstract readability, is a bit tortuous, abstract is the first insight into a paper therefore describe your job in an easier way.

We do agree with the reviewer that the abstract could be made more readable. Therefore, we changed is according to the suggestions below (line 29-43 of the manuscript): “This study assesses the results of a mandatory blended learning-program for counselors (e.g. midwives, sonographers, obstetricians) guiding national implementation of the Non-Invasive Prenatal Test (NIPT). We assessed counselors' 1) knowledge about prenatal aneuploidy screening, 2) factors associated with their knowledge (e.g. counselors' characteristics, attitudes towards NIPT), and 3) counselors’ attitudes regarding the blended learning.”

- I would have talk about prenatal aneuploidy screening not anomaly screening

We do understand this suggestion. Since we measured not only knowledge about aneuploidy screening but also about structural anomaly screening we used the term prenatal anomaly screening. However, most knowledge items concerned aneuploidy screening and the article is about implementation of NIPT. Therefore, we changed our wording into prenatal aneuploidy screening (line 32; see inserted text comment 1).

- I would have specified who do you mean by Duch prenatal counselors (obstetricians, midwives, family doctors).

We implemented this suggestion (line 30): “This study assesses the results of a mandatory blended learning-program for counselors (e.g. midwives, sonographers, obstetricians) guiding national implementation of the Non-Invasive Prenatal Test (NIPT). We assessed counselors' 1) knowledge about prenatal aneuploidy screening, 2) factors associated with their knowledge (e.g. counselors' characteristics, attitudes towards NIPT), and 3) counselors’ attitudes regarding the blended learning.”

- Sentence linke " were positively associated with more correctly answered knowledge questions". I would have said positively associated with a higher level of knowledge for example.

Thank you for this more readable suggestion (see line 43): “Work experience and secondary care work-setting were positively associated with a higher level of knowledge.”

In the mean text

The introduction

1) Again discuss about prenatal screening for aneuploidies not anomalies.

In line with our response to the earlier comment on this topic we changed our wording into prenatal aneuploidy screening thoughout the manuscript.

2) line 62 is essential to facilitate informed choice for expectant parents regarding prenatal

63 screening" I would have said to allow an effective/adequate pregnant women's informed choice

We agree with the reviewer that the suggested text improves the manuscript. We therefore edited the text into: “Such knowledge, e.g. understanding the advantages and limitations of NIPT compared to other prenatal tests and knowing how to interpret the results, is essential to enable pregnant women making an adequate informed choice regarding prenatal screening (line 63).”

3) line 89

I would suggest to mention this paper PMID: 33111167, that again shows how an not-adequate counseling from "counselor" associate with a poor understanding of women regarding NIPT, often considered as "diagnostic" and not "screening"

Thank you for putting our attention towards this interesting article. We now refer to this article at line 75-78: “The resulting knowledge gaps are problematic since insight into NIPT's test characteristics is fundamental for understanding the clinical implications of the test results and the fact that NIPT is not diagnostic, diagnostic options and thereby for providing adequate counseling that supports informed decisions [4,6,14,35].”

Reviewer #2: This manuscript evaluations the impact of a training program. These kinds of evaluations are useful to inform other providers of similar educational initiatives. The manuscript can be improved by addressing the following issues:

1. Introduction: Provide the reader with a brief description of what Non-Invasive Prenatal Testing is.

We thank the reviewer for this relevant suggestion. In lines 55-57 we briefly describe NIPT:

“NIPT is a maternal blood test using cell-free DNA, mostly targeted at the common aneuploidies Down-, Edwards- and Patau syndrome. In most countries, NIPT is readily commercially available, although it has increasingly been offered as a second-tier pr first-tier an aneuploidy screening test, within nationwide healthcare programs, in which mostly primary maternity care providers offer accompanying pre-test counseling [1-3].”

2. Introduction: What are first and second-tiers? Anyone from a different health system does not know what this means.

NIPT can be offered as a first-tier screening test to all pregnant women regardless of their risk level, or as a second-tier test to women with increased risk for fetal aneuploidies only. This has been added to the introduction (line 97-102):

“In 2017, alongside the transition of NIPT from a second-tier test, only available for women with a high risk pregnancy based on the First trimester Combined Test results or medical indication, to a first-tier screening test for all pregnant women, the Dutch government set new educational requirements for healthcare professionals who provide counseling for prenatal aneuploidy screening [18,19]”

3. Introduction: line 75 - I would not use the word 'unacceptable' - replace it with something like 'problematic'

We replaced the word ‘unacceptable’ for ‘problematic’, according to the reviewer’s suggestion (line 75-76).

4. Introduction: line 80 - what is interfering education by commercial test providers?

By intefering education we ment biased education and changed the text accordingly (see line 81).

5. Introduction: line 86-87 It is not correct to say that little is know about how efficient and effective educational programs are for healthcare implementations. There is plenty of literature about education on other topics.

We agree with the reviewer that line 86-87 is not correct. We corrected into (line 89):

“Furthermore, relatively little is known about efficient and effective educational programs accompanying implementation of NIPT, ….etc.”

6. Materials and Methods: line 134 - what is RIVM-CvB and RC (spell out acronyms at first use)

In line 141, in the revised manuscript, we spell out the acronym RIVM-CvB (Centre for Population Screening at the National Institute for Public Health and the Environment). In 123 we already spell out RC (Regionals Centers for prenatal screening).

7. Materials and Methods line 176 - an unpaired t-test is not the correct statistical test. The sample is presumably mostly the same participants at both assessments. I understand that they can't be matched, but the mean score on the first assessment could be used as the threshold score for the comparison of the posttest scores using a one-sample t test.

Thank you for this remark! In line with the reviewers comment we did discuss which statistical test we should use. By reading the manuscript again we do see a discrepance in our work. In line 185 we wrote: Mean scores at T0 and T1 were statistically evaluated using an unpaired sample t-test. However, in line 194 we wrote: One sample proportion tests were performed to identify significant differences between the results of T0 and T1.

We did the latter, but unfortunately we did not remove line 185 from the manuscript. Now we did.

8. Materials and Methods line 178-179: Why are separate regressions used for the pre and post?

By using separate models for T0 and T1 we took a conservative approach, because we did not want to lean on assumptions about overlapping groups within T0 and T1. To clarify this to the readers we edited the text (line 188-190) into:

“Separate models were analyzed for T0 and T1, because we did not want to lean on the assumption that the same participants were included at both assessments.”

Submitted filename: Response to Reviewers_LH.docx

Click here for additional data file.

18 Apr 2022

Concerning Submission of Research Nationwide implementation of the Non-Invasive Prenatal Test: Evaluation of a blended learning program for counselors

PONE-D-22-02421R1

Dear Dr. Martin,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Antonio Simone Laganà, M.D., Ph.D.

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

I carefully evaluated the revised version of this manuscript.

Authors have performed the required changes, improving significantly the quality of the paper.

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: (No Response)

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Well done, I think the paper has been improved. The topic as I've already commented is actual and interesting therefore

i will recommend it for publication

Reviewer #2: The authors have done a good job of responding to the review feedback. I have no further suggestions.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

22 Apr 2022

PONE-D-22-02421R1

Nationwide implementation of the Non-Invasive Prenatal Test: Evaluation of a blended learning program for counselors

Dear Dr. Martin:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Antonio Simone Laganà

Academic Editor

PLOS ONE