Intra-observer reliability of Prechtl's method for the qualitative assessment of general movements in Taiwanese infants.

[Purpose] The aim of this study was to examine the intra-observer reliability for Prechtl's General Movements Assessment in Taiwanese infants. This includes the global General Movements Assessment, the Optimality List for Preterm General Movements and Writhing Movements, and the Assessment of Motor Repertoire-3 to 5 Months.
[Subjects and Methods] Fifty-nine videos of 37 infants were observed and rated by one physical therapist twice.
[Results] The intra-observer reliability ranged from good to very good for the global General Movements Assessment. The overall intra-observer reliabilities for the total score of the Optimality List from preterm up to postmenstrual age 46 weeks and the total score of the Assessment of Motor Repertoire for postmenstrual age 49 to 60 weeks were both good.
[Conclusion] The results suggest that the intra-observer reliability of a certified physical therapist was satisfactory for Prechtl's method in Taiwanese infants.

INTRODUCTION

Prechtl’s method, also known as the General Movements Assessment (GMA), has frequently been used as a functional assessment tool for the young nervous system1,2,3). Unlike other early neuromotor instruments, such as the Alberta Infant Motor Scale4) and the Test of Infant Motor Performance5), it assesses the variability and complexity of spontaneous motor behavior, called general movements (GMs), in young infants in the supine position without elicited or intrusive handling at a much younger age, even before full term1). Previous studies suggested that the original global GMA has good sensitivity, specificity, and consistency in early infancy6,7,8). It is not only a useful clinical instrument for early identification of brain insults, as in children with cerebral palsy9), but also a good predicator of later cognition and behavior at school age10). Currently, clinician judgment by watching videos and visual gestalt perception are still the most common methods for categorizing or scoring the GMA. Therefore, establishing a good level of inter-observer and intra-observer reliability prior to using the GMA for clinical application and/or research is important.

The inter-observer reliability of the global GMA was established in training courses and was excellent (k=0.84–0.92), as reported by systematic reviews6, 8, 11). However, the accuracy of some patterns and conditions appeared more difficult to obtain. In a study by Valentin et al., certified trainee records were analyzed, and it was found that the inter-observer reliability was lower for poor repertoire (PR) GMs and abnormal fidgety movements (FMs), suggesting that observers and even certified trainees may need to focus more attention on the videos of infants closer to term. In addition, the correctness of PR GMs and abnormal FMs were both below 75%11). On the other hand, most intra-observer reliability studies, like other studies related to the global GMA, are frequently conducted by GMA experts, tutors, or experienced observers who have extensive research background and are highly skilled at evaluating GMs6, 8). Perfect test-retest reliability by the same observer for 20 videos over 2 years has been reported in the GMA manual as 100% agreement for the reliability of global GMA results1). Since only one GM expert participated in this study, the results can serve as evidence of intra-observer reliability of the global GMA for an experienced observer12). In a study by Van Kranen-Mastenbroek et al., very good intra-observer reliabilities were reported for three observers of one global judgment of GMs13). However, the nine pre-selected category items used in this study were somewhat different from the items stated in the manual. In a study by Bernhardt et al., the intra-observer reliabilities of the global GMA for five therapists in Switzerland who had completed a GMA training course and passed the certification examination were reported to be moderate to very good for the normal/abnormal ratings, and for the six response category ratings over a 9-week interval3). The overall findings across the above studies indicated good inter-observer reliability for the above two-level category discrimination of the global GMA; however, limited information was found about the intra-observer reliability of the original global GMA, except among experienced and certified observers6, 8). Good inter-observer reliability of GMA can be considered the result of good postgraduate education provided by the specific prefectural society14), but does not guarantee consistency for novice observers. Intra-observer reliability is related to the consistency of the examiner. In the context of visual gestalt perception, it is important to establish acceptable intra-observer reliability, which may be affected by factors such as the examiner’s experience, tiredness, and lack of recalibration.

Two optimality scoring methods based on the age-specific character of the GMA were developed with the intent of providing detailed information regarding specific patterns of GMs for evaluating change and/or intervention effects15, 16). Semi-quantitative approaches were used for both scoring methods. The first scoring method is suitable for preterm and term ages, and is called “GM Optimality List for Preterm GMs and Writhing Movements” (GM Optimality Score, hereafter, GMOS)1, 17). The test-retest reliability of the GMOS showed 85% agreement for the repeated assessment of 20 videos within a 2-year interval, as stated in the manual. Again, since the detailed methodology of this test-retest reliability was unavailable, it is difficult to judge whether it could serve as evidence of the intra-observer reliability of the GMOS. The second scoring method is called “Assessment of Motor Repertoire—3 to 5 Months” (Motor Optimality Score, hereafter, MOS); it is suitable for assessing the motor behaviors of 3- to 5-month-old infants1, 18,19,20). The overall inter-observer reliability of the MOS is very high, but it is low in the “Posture” subcategory; however, its intra-observer reliability is not yet available19). Since intra-observer reliability is important for quantifying the consistency of an observer who predicts infant outcomes for intervention in longitudinal follow-ups, the purpose of this study was to examine the intra-observer reliability for the global GMA, the GMOS, and the MOS for infants in Taiwan.

SUBJECTS AND METHODS

Fifty-nine videos were recorded of 37 infants (18 girls and 19 boys; post-menstrual age (PMA) at the first video: 35–60 weeks, median 52 weeks) (Table 1). The inclusion criteria were as follows: (1) the parents signed the consent form, (2) the infant’s PMA was 28–60 weeks, and (3) the infant was able to lie in the supine position without any assistance devices such as a ventilator. The exclusion criteria were as follows: (1) the infant’s PMA was 47–48 weeks, and (2) high risk infants with known movement disorders such as congenital malformations and chromosomal abnormalities. The infants were recruited either from the neonatal intensive care unit (NICU) and/or outpatient clinics of Lin-Kao Chang Gung Memorial Hospital-Children’s Hospital or acquaintances of investigators from 2009 to 2011 in Taiwan. The Institutional Review Board (IRB) of Chang Gung Medical Foundation, Lin-Kao, Taiwan reviewed and approved the protocol of this study before the recruitment of subjects as a part of a longitudinal follow-up study of young infants (IRB No. 101-3548C). The videos in this study were recorded from a wide spectrum of young infants with various chronological ages, birth weights, gestational ages, and risk factors for later neurological impairments (Table 1). Infants with these possible risk factors included 8 with retinopathy of prematurity, 7 with congenital heart diseases, 20 with respiratory problems, 4 with gastroesophageal reflux disease, 10 with neurological concerns (e.g., periventricular leukomalacia), and 9 with a history of hyperbilirubinemia, totaling 26 infants (70.3%). All except one exhibited more than one risk factor. No identified risk was noted in the remaining 11 infants (29.7%). In addition, videos of 10 infants were obtained more than once (mean=1.6 ± 1.5 videos). As a result, 59 videos were collected. All parents were asked to provide as much developmental information as possible, such as the age of walking alone. A follow-up telephone call was made once the infant reached the age of 2 years. Out of 37 infants, only one contact was lost.

Table 1.

Basic information for all infants

ID	Gender	GA(weeks)	BBW(g)	PMA for 1st video (weeks)	First				Second				Final Dichotomous		Walk alone (months)
					Global GMA		GMOS	MOS	Global GMA		GMOS	MOS
					Six nominal	Dichotomous			Six nominal	Dichotomous
19	Female	24.6	710	35	CS/F-	Abn	8	15	CS/F-	Abn	8	10	Abn		21
7	Male	28.9	1,410	36	PR	Abn	12		PR	Abn	14		Abn		18
1	Female	27.3	1,195	37	CS	Abn	8		CS	Abn	9		Abn		18
8	Female	28.0	700	38	PR	Abn	15		PR	Abn	15		Abn		14
16	Female	26.0	742	38	PR	Abn	13		PR	Abn	13		Abn		16
32	Female	25.1	695	38	CS	Abn	8		CS	Abn	8		Abn		18
31	Male	36.0	2,490	40	PR	Abn	14		N	N	17		Disagree		18
36	Female	37.9	3,080	40	N	N	18		N	N	18		N		13
11	Male	28.1	1,275	41	PR	Abn	11		PR	Abn	11		Abn		13
27	Male	37.3	3,300	41	N	N	18		N	N	18		N		13
35	Female	40.1	3,215	41	N	N	18	28	N	N	18	28	N		11
25	Female	31.3	880	43	CS/F-	Abn	9	6	CS/F-	Abn	9	6	Abn		18
34	Female	40.0	3,300	45	N	N	18	28	N	N	18	26	N		12
10	Female	25.7	617	46	PR/F-	Abn	15	15	PR/F-	Abn	13	15	Abn		Passed away
30	Male	40.0	2,250	49	N	N		28	N	N		28	N		13
14	Male	24.3	685	51	F-/AF	Abn		6	F-/AF	Abn		15	Abn		13
26	Male	28.0	928	51	F-	Abn		9	N	N		23	Disagree		12
15	Male	39.4	3,210	52	N	N		26	N	N		26	N		18
17	Female	34.0	1,360	52	N	N		24	N	N		26	N		15
20	Female	40.0	3,046	52	N	N		28	N	N		28	N		13
21	Male	23.6	650	52	N	N		28	N	N		28	N		13
23	Female	40.7	3,505	52	N	N		28	N	N		26	N		14
5	Male	25.1	750	53	N	N		28	N	N		28	N		24
6	Male	25.1	820	53	N/F-	N/Abn		17	F-	Abn		11	Inconsist / Disagree		30
22	Male	23.9	770	53	AF	Abn		16	AF	Abn		10	Abn		16
24	Male	39.0	-	53	F-	Abn		5	F-	Abn		5	Abn		Missing
2	Male	39.0	3,285	54	N	N		28	N	N		28	N		12
18	Female	38.7	3,920	54	F-	Abn		8	F-	Abn		8	Abn		Unable
3	Female	39.1	2,580	55	F-	Abn		11	F-	Abn		7	Abn		Unable
12	Male	27.6	762	57	F-	Abn		15	F-	Abn		15	Abn		17
33	Male	25.3	670	57	N	N		28	N	N		26	N		17
4	Female	38.3	3,334	58	F-	Abn		5	F-	Abn		5	Abn		Passed away
13	Male	39.0	3,495	58	F-	Abn		6	F-	Abn		7	Abn		14
29	Female	38.6	3,100	58	N	N		28	N	N		28	N		14
9	Female	25.7	600	59	N	N		26	N	N		26	N		13
28	Male	37.0	3,600	59	N	N		26	N	N		26	N		17
37	Male	25.0	771	60	F-	Abn		6	N	N		18	Disagree		15
Mean		32.2	1,880.6	48.9			13.2	18.6			13.5	19.0
SD		6.5	1,216.2	7.8			4.0	9.4			4.0	9.0
Median		31.3	1,317.5	52.0			13.5	20.5			13.5	24.5
IQR		13.5	2,469.8	13.5			9.3	19.8			9.0	17.5
Female		23–27		ELBW
18 (48.6%)		14 (37.5%)		16 (43.2%)
Male		28–32		VLBW
19 (51.4%)		5 (13.5%)		4 (10.8%)
		33–36		LBW
		2 (5.4%)		2 (5.4%)
		>37
		16 (43.2%)

GA: gestational age; BBW: birth body weight; PMA: postmenstrual age; GMA: General Movements Assessment; GMOS: Prechtl’s Method on General Movement Assessment GM Optimality List for Preterm GMs and Writhing Movements; MOS: Assessment of Motor Repertoire – 3 to 5 Months; N: normal age-specific GMs; PR: poor repertoire of GMs; CS: cramped-synchronized GMs; AF: abnormal fidgety movements; F-: absence of fidgety movements; Abn: abnormal; Inconsist: inconsistent results among different videos; Disagree: disagreement between the two observations from same video; N: normal; Unable: unable to walk up to 24 months old; IQR: interquartile range; CI: confidence interval; ELBW: extremely low birth weight (<1,000 g); VLBW: very low birth weight (1,000–1,500 g); LBW: low birth weight (1,500–2,500 g).

For infants who had more than one video during the specific time window, only the first video was presented in this table unless inconsistent findings and/or disagreements were noted.

One physical therapist who had just completed the basic training course and passed the certification examination for the Prechtl GMs assessment in 2011 served as the observer for this study. He possessed 8 years of experience in pediatric physical therapy, including 8 years in the NICU.

According to the GMA manual, all infants were video-recorded awake, without crying, in the supine position1). Interaction between infants and adults was not allowed during the video recording. The videos were recorded either by the physical therapists, including the observer for this study, at pediatric clinics or at the child’s home. Following the video recordings, the observer used the global GMA to classify the infants’ GMs, along with either the GMOS or the MOS based on their PMA1, 13, 14, 17,18,19,20). The observer was blind to the infants’ code and viewed each video twice. The tapes were reviewed in a different order. The time interval between the scoring of the two videos was at least 7 days.

The global GMA assessed the critical periods of specific movements from preterm birth to 5 months of age. According to the GMA, GMs were divided into three “key age” periods: (1) the preterm GMs period (PMA 10–38 weeks), (2) the period of writhing movements (term to 6 weeks post-term), and (3) period of fidgety movements (9 weeks to 20 weeks post-term). According to the age of the infants, their GM behavior can be classified as normal age-specific GMs (N), fidgety movements (FMs), hypokinesis (H, no GMs during the recording), poor repertoire of GMs (PR), chaotic GMs (Ch), cramped-synchronized GMs (CS), abnormal fidgety movements (AF), and absence of fidgety movements (F-). PR, Ch, and CS constitute abnormalities of the preterm GM period and the period of writhing movements. For the period of fidgety movements, AF and F- are judged as abnormal1, 21). The global GMA uses a binary classification for normal and abnormal.

The GMOS consists of eight items: “quality” (4 characteristics), “sequence” (2 characteristics), “amplitude” (4 characteristics), “speed” (4 characteristics), “space” (2 characteristics), “rotatory components” (2 characteristics), “onset and offset” (2 characteristics), and “tremulous movements” (2 characteristics)1, 17). This is a semiquantitative evaluative list in scoring the GMs for infants aged from birth to the end of the first month after term. All items of the GMOS are rated on a 2-point scale with the exception of the “quality” item. A score of 2 is given for the optimal characteristic of GMs, such as variable in “sequence,” while non-optimal characteristics are only given a score of 1. For the “quality” item, the presence of normal, variable, and complex characteristics is scored as 4, a poor repertoire as 2, and others as 1. The GM Optimality score is the summation of the scores for eight subcategories. The minimum for the GM Optimality score is 8 and the maximum is 18.

The MOS consists of five items: “fidgety movements” (3 characteristics), “repertoire of coexistent other movements” (3 characteristics), “quality of other movements” (3 characteristics), “posture” (3 characteristics), and “movement character” (3 characteristics). It is also a semiquantitative evaluative list. For the “fidgety movements” item, a normal amount of fidgety movements is scored as 12, abnormal as 4, and absent/sporadic movements as 1. For the other items, the presence of optimal characteristics is scored as 4, less-optimal characteristics as 2, and obviously abnormal characteristics, such as “cramped-synchronized” in the “movement character” item, as 1. Subcategories of each time in the 3- to 5-months scoring list are ordinal-scaled. The total score of the MOS ranges from low (5) to high (28). It is appropriate to use the MOS for infants aged PMA 49–60 weeks1, 18, 19).

Intra-observer reliability was determined using percentage agreement (%), Cohen’s Kappa (k), and Intra-class Correlation Coefficient (ICC(3,1))22). The results of the Kappa coefficient and percentage agreement were interpreted as a k value of <0.40 or agreement of <60% as poor agreement, a k value of 0.40–0.60 or agreement of 60–80% as moderate, a k value of 0.61–0.80 or agreement of 80% as good, and a k value of >0.80 and agreement of >0.80 as very good6, 14). The following ICC(3,1) interpretation scale was used: <0.50 as poor, 0.50–0.75 as moderate, and >0.75 as good. In addition to the overall reliability analyses, the initial plan was to divide the videos into five age groups within a one-month interval (PMA <30, 30–34, 34–38, 38–42, 42–46 weeks) for the preterm GMs and period of writing movements, and three groups (PMA <52, 52–56, 56–60 weeks) for the period of fidgety movements, according to the infants’ PMA, for subgroup reliability analyses. However, only two age groups for the preterm GMs and period of writhing movements (the GMOS) were analyzed because only three infants of PMA 34–38 and no infant of PMA <34 weeks participated in this study. Consequently, five age subgroups (two age groups for the GMOS and three groups for the MOS) were analyzed.

RESULTS

Summarized statistics are presented for the global GMA, the GMOS, and the MOS in Tables 2, 3, 4. Overall, the intra-observer reliability ranged from good to very good for the global GMA, with k between 0.727 and 0.842. The agreements for each GM pattern were above 80%, except Ch, which no infant exhibited (Table 1).

Table 2.

Intra-observer reliabilities of the global GMA

			Present study		van Kranen -Mastenbroek et al.13)	Bernhardt et al.3)	Einspieler et al.12)
	Observer(s)		1		3	5	1
	PMA (weeks)	35–46	49–60	Total		31–54
	Time interval		7-days		>2 months	9 weeks	2 years
	Videos	N=22	N=37	N=59	N=50	N=30	N=20
	Six nominal categories
Percentage	N	83%	95%	91%
agreement (%)	PR	83%		83%
	Ch
	CS	100%		100%
	AF		100%	100%
	F-		85%	85%
	Overall	86.4%	91.9%	89.8%	-	60%–90%	-
Kappa (k)		0.776	0.842	0.832	-	0.44–0.86	-
	Dichotomous
Percentage	Abn	90.0%	86.7%	88.0%
agreement (%)	N	83.3%	95.5%	91.2%
	Global	86.4%	91.9%	89.8%	85%–100%	70%–93%	100.0%
Kappa (k)		0.727	0.830	0.792	0.69–1.0	0.40–0.86	-

GMA: General movements assessment; PMA: postmenstrual age; N: normal age-specific GMs; PR: poor repertoire of GMs; CS: cramped-synchronized GMs; AF: abnormal fidgety movements; F-: absence of fidgety movements; Abn: abnormal.

Table 3.

Intra-observer reliabilities of the GMOS

PMA	38–42 weeks	42–46 weeks	Total
Videos of infants	N=11	N=8	N=22
	ICC(3,1)	ICC(3,1)	ICC(3,1)
Quality	0.556	1.000	0.808
Sequence	1.000	1.000	0.923
Amplitude	1	0.756	0.909
Speed	0.522	0.260	0.516
Space	1.000	1.000	0.869
Rotatory components	1.000	0.655	0.886
Onset and offset	0.808	1.000	0.904
Tremulous movement	1.000	*	0.869
GM Optimality Score	0.886	0.961	0.936

GMOS: GM Optimality List for Preterm GMs and Writhing Movements; PMA: postmenstrual age; ICC: intraclass correlation coefficient; *: No statistics were computed because variables were constants. No infants in the PMA 28–32 weeks group were recruited in this study due to time constraints.

Table 4.

Summary of intra-observer and inter-observer reliabilities of the MOS

	Intra-observer								Inter-observer
	Present Study								Fjortoft et al.19)
	PMA49–52 weeks		PMA52–56 weeks		PMA56–60 weeks		Total		3–5 months post-term
Observer(s)	1								4
Videos of infants	N=8		N=13		N=16		N=37		N=24
	Kappa (k)	ICC(3,1)	Kappa (k)	ICC(3,1)	Kappa (k)	ICC(3,1)	Kappa (k)	ICC(3,1)	Kappa (k)	ICC(2,1)
Fidgety movements	0.750	0.775	0.806	0.822	0.893	0.867	0.842	0.821	0.75–0.91
Repertoire of co-existent other movements	0.314	0.648	0.524	0.646	0.706	0.843	0.543	0.718	0.51–0.69
Quality of other movements	0.515	0.586	0.667	0.954	0.434	0.629	0.537	0.733	0.51–0.84
Posture	0.600	0.655	0.649	0.848	0.756	0.908	0.699	0.828	0.39–0.56
Movement character	0.385	0.736	0.735	0.781	0.776	0.875	0.675	0.809	0.54–0.84
Motor optimality score		0.805		0.976		0.912		0.900		0.87(0.80–0.94)

MOS: assessment of motor repertoire—3 to 5 months; PMA: postmenstrual age; ICC: intraclass correlation coefficient.

The intra-observer reliability of the GMOS for 22 videos from 14 infants was very good for eight items, with the exception of the poor “speed” item (ICC(3,1)= 0.516), and the total score was good (ICC(3,1)= 0.936). If only considering the first video of 14 infants, the total score of the GMOS was also good (ICC(3,1)=0.952).

The intra-observer reliability of the MOS for 37 videos from 28 infants ranged from moderate to good with an ICC(3,1) between 0.718 and 0.828 for five items, and the total score was good (ICC(3,1)=0.900). The total score of the MOS for the first video of 28 infants was also good (ICC(3,1)=0.886).

DISCUSSION

The results indicated that the overall intra-observer reliabilities of the global GMA, the GMOS, and the MOS by a relatively novice certified observer for Taiwanese infants were satisfactory. The intra-observer reliability of the global GMA in this study was consistent with previous studies12). Previous results suggested that the inter-observer reliability of the certified observer for the Global GMA was moderate to very good (Table 2)1, 3, 13, 19). In addition, the intra-observer reliabilities of the summation scores of the GMOS and the MOS were also good for this novice certified observer. However, some specific items were prone to inconsistency, even by the same observer. Therefore, the results of this study need to be interpreted with caution.

First, the “speed” item of the GMOS was less consistent compared to other items. By reexamining the scores of the “speed” item, problems were found to arise from five inconsistent judgments out of 22 videos, based on judgments of “variable speed” (score 2) the first time, and that of “mainly one speed, not variable” (score 1), the second time. Those videos were reviewed consecutively. It has been suggested that judgment accuracy may decrease when many abnormal infants are assessed in a series without viewing a normal video in between2). Therefore, to improve the accuracy of the “speed” item, the results indicate that the observer should revisit normal videos more frequently as recommended by the manual1). In addition, when a video is classified as normal for the “speed” item, scoring twice may be necessary before a final decision is made. Furthermore, an experienced observer reported that identical scores were obtained in 17 out of 20 infants (85%), whereas different scores were obtained for the remaining 3 infants for one point11). Compared with the above study11), only 12 out of 22 infants (55%) obtained the same score in the present study. Although there were slight differences between the two scoring lists, it is interesting to assess clinical decisions made by novice and experienced observers.

Second, the results suggested that the intra-observer reliability of the total MOS was satisfactory for clinical use with similar concerns reported in the inter-observer reliability study by Fjortoft et al19). The results showed that the videos of those infants in the PMA 49–52 weeks group had less reliability than did those from the other age groups (the PMA 52–56 weeks and the PMA 56–60 weeks groups). The intra-observer reliabilities of the five items of the MOS were moderate to good in terms of their k values (k=0.537–0.842), as well as their ICC(3,1) values (0.718–0.821) for 37 videos (Table 4). The k values and the ICC(3,1) of the “repertoire of coexistent other movements” and “movement character” items for the PMA 49–52 weeks group were poor according to Cohen’s Kappa (k=0.314 and 0.385, respectively); however, the intra-observer reliabilities of those two items were moderate according to their ICC(3,1) values (0.648 and 0.736, respectively). Since the scoring of these items using the MOS as an ordinal scale was more appropriate, the intra-observer reliabilities of these items were also satisfactory. However, because the infants in this age group had writhing movements combined with fidgety movements, larger and longitudinal follow-up examinations of GMs for infants aged PMA 49–52 weeks are recommended, especially for eight of the infants in this age group for this study21).

Some limitations of the present study must be acknowledged. First, as previously mentioned, the number of subjects in some age groups was limited. No infants in the PMA 28–32 weeks group were recruited in this study because of time constraints. In our clinical experience, infants at PMA 28–32 weeks tend to rely on some kind of assistance device/equipment, such as a vital signs monitor and a central intravenous catheter, for survival. It would also be worthwhile for future studies to determine the impact of medical assistance devices/equipment on gestalt perception. Second, most of the video recordings were recorded by the same observer. The influence of an observer being present who knew the infants and family history during the video recording was not determinable at this time. Third, compared to previous intra-observer reliability studies, which had various time intervals ranging from 9 weeks to 2 years, the time interval for this study was relatively short1, 3, 13). A longer time interval may be needed. Finally, longitudinal follow-ups for the infants are needed to confirm the predictive validity of the global GMA, the GMOS, and the MOS in the future.