Intra- and inter-rater reliability in a comparative study of cross-sectional and spiral computed tomography pelvimetry methods.

BACKGROUND: Different low-dose computed tomography (CT) pelvimetry methods can be used to evaluate the size of birth canal before delivery. CT pelvimetry might generate an acceptable low fetal radiation dose but its measurement accuracy is unknown.
PURPOSE: To investigate intra- and inter-rater measurement reliability of cross-sectional and two spiral CT pelvimetry methods: standard spiral and short spiral.
MATERIAL AND METHODS: Ten individuals (age ≥60 years, body mass index ≥30 kg/m2) having a CT scan of the abdomen also had CT pelvimetry scans. Three radiologists made independent measurements of each pelvimetry method on two occasions and also in consensus for a reference pelvimetry computed from the standard-dose CT scan of the abdomen. Inter- and intra-rater reliability was analyzed by intraclass correlation coefficient.
RESULTS: Measurements in the short spiral pelvimetry demonstrated excellent intra- and inter-rater reliability, intraclass correlation coefficient ≥0.93, and good to excellent 95% confidence interval 0.87-0.99. Corresponding results of the standard spiral and cross-sectional pelvimetry showed good to excellent intraclass correlation coefficient ≥0.85 and ≥0.76, and 95% confidence interval was least good and moderate 0.73-0.98 and 0.59-0.97, respectively. Intraclass correlation coefficient between reference pelvimetry and other CT methods showed analogous results.
CONCLUSION: The short spiral pelvimetry demonstrated high and best reliability in comparison to other methods. Standard spiral method showed also good measurement reliability but the short spiral pelvimetry generates lower fetal radiation dose. This method might be suitable for measurements at narrow pelvis. Patient acceptance and attitude to CT pelvimetry should be investigated.

Introduction

Pregnant women with a narrow pelvis or who have the fetus in breech presentation risk prolonged labor. Emergency Cesarean section might be needed to prevent suffocation of the fetus (1,2). Cesarean section is associated with maternal and neonatal morbidity with prolonged hospitalization and mortality; the risk of complications is increased with an emergency Cesarean section (3–7). To prevent emergency Cesarean section, pelvimetry imaging can be performed before delivery (8–12). Pelvimetry is a diagnostic examination that provides measurements of the birth canal in order to rule out a narrow pelvis and to decide upon a vaginal delivery or a planned Cesarean section (11,13,14).

Different imaging techniques can be used for the pelvimetry, e.g. plain X-ray, computed tomography (CT), and magnetic resonance imaging (MRI) (15–18). MRI is a useful alternative as no ionizing radiation is used, but it is expensive and with limited availability (15); some patients might experience anxiety and distress (19). Plain X-ray and CT represent radiographic techniques, but CT has a potential of generating high measurement accuracy (10) and our previous study results have demonstrated acceptable low fetal radiation dose (20). Various methods can be used in CT pelvimetry, for example cross-sectional method (20–22) and spiral method (10,20,23). The cross-sectional method has a comparable fetal radiation dose to plain X-ray (20). The spiral method has an increased radiation dose to the fetus (20,24) but it has been considered to have a low margin of measurement error (10) and shortening the spiral scan (e.g. short spiral method) reduces the radiation dose (20). Still, the measurement accuracy of different CT scans should be compared and also considered in relation to the radiation dose in order to decide upon the most optimal pelvimetry method.

The aim of this study was to investigate intra- and inter-rater measurement reliability of cross-sectional and two spiral CT pelvimetry methods. It was hypothesized that the short spiral pelvimetry demonstrates sufficient measurement reliability compared to the standard spiral pelvimetry and is considerably higher in comparison to the cross-sectional method.

Material and Methods

Participants

Ten female patients undergoing a CT examination of the abdomen at a radiology department at a hospital in Sweden, during February 2017 and February 2018, were included in the study. Inclusion criteria were women aged ≥60 years with a body mass index of (BMI) of ≥30 kg/m2. The cut-off age was determined in order to not include women of childbearing age. A BMI ≥30 kg/m2 might be related to the body mass of a pregnant women at ≥35 weeks of gestation (25). Communication in Swedish was necessary to provide informed consent. Exclusion criteria were: osteoporosis; skeletal metastases in pelvic area; metal hip prosthesis; multimorbidity; and having severe cognitive impairment. The study was approved by the Swedish Ethical Review Board (Dnr 2016/675) and the Radiation Protection Committee.

Procedure

Besides the CT abdomen examination, women received additional pelvimetry scans as follows: posteroanterior (PA) and lateral (LAT) pre-view imaging scans; a cross-sectional single-slice image; and a standard spiral pelvimetry scan. Scan setting details are presented in Table 1.

Table 1.

Scan settings of the CT pelvimetry examinations.

CT scan	Tube voltage (kV)	Tube current (mA)	Ref. eff. mAs	Ref. kV	Dose modulation	Collimation (mm)	Pitch	Slide	Rotation time (s)
Pre-view PA (bottom projection)	100	20	–	–	–	–	–	–	–
Pre-view lateral (lateral projection)	120	35	–	–	–	–	–	–	–
Cross-sectional single-slice	120	–	14	–	CARE kV off CARE Dose 4D on	1 × 5	–	–	–
Standard spiral	100	–	13	–	CARE kV off CARE Dose4D on	128 × 0.6	1.0	–	0.5

The PA and LAT images scanned pelvis from fifth lumbar vertebra including the tuber ischiadicum (Fig. 1a and b). The cross-sectional single-slice image was centered at fovea caput as a landmark (Fig. 1c). The standard spiral was a scan of the pelvis including the fifth lumbar vertebra through the tuber ischiadicum (Fig. 1d). The scans were performed by one study researcher while the CT scan of the abdomen was carried out by the radiographers according to the diagnostic protocol at the radiologic department. All scans were performed on a SOMATOM Definition Flash CT (Siemens Healthineers, Forchheim, Germany). The scan settings for the CT abdomen scans are presented in Table 2.

Fig. 1.

Additional scans for the study purpose: (a, b) a posteroanterior (PA) (a) and lateral (b) pre-view image; (c) a cross-sectional single-slice image centered at fovea caput femur; and (d) a standard spiral scan of the pelvis.

Table 2.

Scan settings of the diagnostic protocol of the abdominal CT examination.

Image	Tube voltage (kV)	Tube current (mA)	Ref. eff. mAs	Ref. kV	Dose modulation	Collimation (mm)	Pitch	Slide	Rotation time (s)
Pre-view PA (top projection)	100	35	–		–	–	–	–	–
Abdominal CT scan	–	–	250	100	CARE kV onCARE Dose 4D on	128 × 0.6	0.6	7	0.5

CT pelvimetry methods

Three different CT pelvimetry methods were evaluated: cross-sectional; standard spiral; and short spiral. Each method provides pelvimetry measurement points for four distances: the interspinous and intertuberous distance (Fig. 2a and b), the anteroposterior (AP) pelvic inlet diameter (Fig. 2c), and the AP pelvic outlet distance (Fig. 2d) (23,26). The distances a–d in each CT pelvimetry method are presented in Figs. 3–5.

Fig. 2.

Measurement distance (a–d) needed for pelvimetry. Projections with intertuberous (a) and interspinous distance (b) measured on a PA projection to the left and anteroposterior (AP) pelvic inlet diameter (c) and the AP pelvic outlet distance (d) measured on a lateral projection to the right. Intertuberous distance (a) is the widest distance between the ischial tuberosities, interspinous distance (b) is the shortest distance between both ischial spines, AP pelvic inlet (c) is the shortest distance between promontory and symphysis while the AP pelvic outlet distance (d) is the distance between the inferior inner aspect of the symphysis and the distal end (the joint) of the sacrum.

Fig. 3.

Measurements in the cross-sectional method: (a) PA image for measurement of the intertuberous distance; (b) LAT for measurement of the AP pelvic inlet and outlet distance; (c) a cross-sectional single-slice image for measurement of the interspinous distance. The white lines in the figure are the measurement distance/s performed in the image. The black dashed line in (a) represents the location for the single-slice cross-sectional image.

Fig. 4.

Measurements in the standard spiral method: (a, b) the intertuberous and the interspinous distances were measured from the spiral scan and/or transverse reconstruction images; (c) and the AP inlet and outlet diameters were measured in sagittal reconstructions. An oblique reconstructed image of 150-mm thickness (d) was also included for the measurement of the intertuberous distance. The white lines represent the measurement distances in the images.

Fig. 5.

Measurements in the short spiral method: LAT image (a) of the pelvis for measurement of the AP pelvic inlet diameter (b, c) From the spiral scan and/or transverse reconstruction images, the intertuberous and the interspinous distances were measured. The AP pelvic outlet diameter, was measured in sagittal reconstructions (d). An oblique reconstructed image of 150-mm thickness (e) was also included for measurement of the intertuberous distance. The dark dashed lines mark the area of a short spiral. The white lines represent the measurement distances in the images.

The cross-sectional method contained the PA, LAT, and cross-sectional single-slice image of 5-mm thickness (Fig. 3). The standard spiral method contained the spiral scan of the pelvis with transverse images of 0.75-mm thickness, transversal, coronal, and sagittal reconstruction images of the spiral scan in 3/3-mm thickness, and one oblique reconstruction of 150-mm thickness (Fig. 4). The short spiral method contained the same set of images as the standard spiral method but with a view of the bone structures only from the hip joint through the tuber ischiadicum and the LAT image of the pelvis (Fig. 5). The set of images for the short spiral method corresponds to approximately the halved scan length compared to the standard spiral method. In addition, a reference pelvimetry method was computed from the CT abdomen. The reference pelvimetry images and reconstructions were the same as for the standard method with the exception of the radiation dose used allowing an optimal visualization of bone structures.

Analysis

Three radiologists (observers) independently conducted pelvimetry measurements on the three CT methods. The measurements were performed on two separate occasions (T1, T2) with a minimum of two weeks between them. Measurement on the reference pelvimetry was executed in consensus by all observers >1 month after they finished their T2 measurements. Each observer obtained a separate folder in the Picture Archive and Communication System (PACS) containing images of the cross-sectional, standard spiral, and short spiral pelvimetry methods, respectively. The images of the reference pelvimetry method were delivered in a separate folder. Patient data were pseudonymized and all data in the folders were assigned a randomized code in the range of 100–5000; participants were placed in a random order in each folder and method.

Two of the observers were experienced radiologists with >10 years of experience in pelvimetry measurements; one of the observers was undergoing specialist training in radiology. Before entering the study, the observers discussed the scan images, measurement points, and distances. The T1, T2, and reference measurement procedures were executed in clinical routine; thus, the radiologists decided when to execute the measurements, how many at the time, which images within each method to use, and with a possibility to create additional reconstructions. All measurements were executed on a PACS Sectra IDS7 workstation version 19.1.18 3696 (Sectra, Linköping, Sweden).

All 760 measurement values with one imputation were entered into a database for analysis. The imputation was a repetition of the interspinous distance measurement (b) for one participant in the short method from timepoint T2 to timepoint T1 of one observer. Two additional reconstructions were performed by the same observer at timepoint T1 in the short spiral method. All the data were normally distributed. IBM SPSS Statistics version 24 was used for the reliability analyses. Measurement values were analyzed in centimeters with two decimal points without rounding up according to pelvimetry in clinical practice. The inter- and intra-rater reliability was assessed using of intraclass correlation coefficients (ICC), two-way mixed, absolute agreement, and single measures (27,28). ICC values in the range of 0.50–0.75 were considered moderate reliability, whereas values in the range of 0.75–0.90 indicate good reliability, and values ≥0.90 indicate excellent reliability (28).

Results

Intra-rater reliability for pelvimetry methods

The intra-rater reliability results between measurement occasions T1 and T2 for the three methods under study are presented in Table 3. Repeated measurements in the short spiral method demonstrated excellent ICC ≥0.95 for all three observers and a 95% confidence interval (CI) of 0.88–0.99, indicating good to excellent reliability. Repeated measurements in the standard spiral method showed excellent reliability for two observers (ICC = 0.96) and good reliability (ICC = 0.85) for the third observer; the 95% CI 0.73–0.98 also indicated good to excellent reliability. Intra-rater reliability was excellent (ICC 0.93) for only one observer in the cross-sectional method, otherwise it was good (ICC = 0.76 and 0.78); 95% CI 0.59–0.97 demonstrated moderate to excellent values.

Table 3

. Intra-rater reliability between two timepoints, T1 and T2, conducted by each observer for each CT pelvimetry method.

Observer	Cross-sectional pelvimetry	Standard spiral pelvimetry	Short spiral pelvimetry
1	0.93 (0.84–0.97)	0.96 (0.93–0.98)	0.96 (0.88–0.99)
2	0.76 (0.59–0.87)	0.85 (0.73–0.92)	0.95 (0.90–0.97)
3	0.78 (0.61–0.88)	0.96 (0.92–0.98)	0.98 (0.95–0.99)

Values are presented as ICC (95% CI).

*Two-way random effects, single rater measurement, absolute agreement.

ICC, intraclass correlation coefficient; CI, confidence interval.

Inter-rater reliability for pelvimetry methods

The inter-rater reliability results for all observers for each method and for timepoints T1 and T2 are presented in Table 4. Measurements between all observers and at both occasions showed excellent reliability (ICC ≥0.93) for the short spiral method; 95% CI 0.87–0.98 indicated good to excellent reliability. The inter-rater reliability results for measurements in the standard spiral method demonstrated good to excellent reliability (ICC = 0.88 and 0.93, respectively) and 95% CI 0.81–0.96 also indicated good to excellent reliability. The inter-rater reliability results for measurements in the cross-sectional method showed good reliability (ICC = 0.77 and 0.82, respectively) and 95% CI 0.65–0.89 also indicated moderate to good reliability.

Table 4.

Inter-rater reliability between all observers for each CT pelvimetry method and for timepoints T1and T2.

Observer	Cross-sectional pelvimetry	Standard spiral pelvimetry	Short spiral pelvimetry
1 + 2 + 3 (T1)	0.82 (0.71–0.89)	0.93 (0.88–0.96)	0.96 (0.94–0.98)
1 + 2 + 3 (T2)	0.77 (0.65–0.86)	0.88 (0.81–0.93)	0.93 (0.87–0.96)

Values are presented as ICC (95% CI).

*Two-way random effects, single rater measurement, absolute agreement.

ICC, intraclass correlation coefficient; CI, confidence interval.

Inter-rater reliability between reference and each pelvimetry method

Tables 5 and 6 show the results of inter-rater reliability between the reference measurement and measurements in each CT pelvimetry method by each observer and at timepoints T1 and T2, respectively.

Table 5.

Inter-rater reliability between reference measurement and in each CT pelvimetry method conducted by each observer at measurement timepoint T1.

Observer	Cross-sectional pelvimetry (T1)	Standard spiral pelvimetry (T1)	Short spiral pelvimetry (T1)
1 + reference	0.72 (0.53–0.84)	0.91 (0.83–0.95)	0.92 (0.85–0.96)
2 + reference	0.72 (0.53–0.84)	0.89 (0.80–0.94)	0.91 (0.84–0.95)
3 + reference	0.66 (0.44–0.80)	0.92 (0.86–0.96)	0.91 (0.83–0.95)

Values are presented as ICC (95% CI).

*Two-way random effects, single rater measurement, absolute agreement.

ICC, intraclass correlation coefficient; CI, confidence interval.

Table 6.

Inter-rater reliability between reference measurement and measurements in each CT pelvimetry method conducted by each observer at timepoint T2.

Observer	Cross-sectional pelvimetry (T2)	Standard spiral pelvimetry (T2)	Short spiral pelvimetry (T2)
1 + reference	0.67 (0.46–0.81)	0.90 (0.79–0.95)	0.92 (0.82–0.96)
2 + reference	0.58 (0.34–0.76)	0.89 (0.81–0.94)	0.88 (0.79–0.94)
3 + reference	0.60 (0.36–0.77)	0.90 (0.82–0.95)	0.92 (0.86–0.96)

Values are presented as ICC (95% CI).

*Two-way random effects, single rater measurement, absolute agreement.

ICC, intraclass correlation coefficient; CI, confidence interval.

Measurements at timepoint T1 between the short spiral method and reference values demonstrated excellent inter-rater reliability (ICC ≥ 0.91) and 95% CI 0.83–0.96 showed good to excellent inter-rater reliability. Measurements between the standard spiral method and reference values presented good to excellent inter-rater reliability at timepoint T1 (ICC = 0.89–0.92, 95% CI 0.80–0.96). Measurement results between the cross-sectional method and reference values demonstrated a moderate inter-rater reliability (ICC = 0.66–0.72) 95% CI 0.44–0.84 indicated a moderate to good inter-rater reliability at timepoint T1 (Table 5).

Measurements at timepoint T2 between the short spiral method and reference values presented good to excellent inter-rater reliability (ICC = 0.88–0.92, 95% CI = 0.79–0.96). Measurements between the standard spiral method and reference values presented good to excellent inter-rater reliability at timepoint T2 (ICC = 0.89–0.90, 95% CI = 0.79–0.95). Measurement results between the cross-sectional method and reference values demonstrated a moderate inter-rater reliability (ICC = 0.58–0.67); 95% CI 0.34–0.81 indicated a poor to good inter-rater reliability at timepoint T2 (Table 6).

Discussion

Measurements in the short spiral method showed high and best reliability results compared to the other CT pelvimetry methods. Results from both spiral methods demonstrated sufficient measurement reliability, particularly compared to reference values. The cross-sectional method showed less reliable measurements; these were hypothesized. Our previous study results (20) demonstrated acceptable low radiation dose to the fetus in the short spiral pelvimetry method, which, together with the current study results, argues for the suitability of this method.

Scientific evidence regarding the measurement reliability of different CT pelvimetry methods is scarce, thus strengthening the need of such evaluation studies. One previous study (29) investigated the cross-sectional CT pelvimetry method. The results pointed out a risk of measurement error in the method, probably due to sensitivity of the method to patient position in the CT modality. It was concluded that pelvimetry measurements in the cross-sectional CT method are uncertain and not reliable to use in clinical practice, which is in line with our results. Another available study in the field of measurement reliability in pelvimetry investigated reliability in MR pelvimetry measurements (16). The inter-rater reliability results showed that measurements in MR pelvimetry provide a reliable examination, which provides evidence for the use of MRI in pelvimetry. However, the troublesome aspect is that MRI is expensive, with limited availability (15), and some patients might find the examination unpleasant (19). Although CT pelvimetry provides reliable measurement results, how it is accepted by patients is unknown. Patients may be concerned about the radiation dose generated by the X-ray examination; therefore, this should also be investigated. The fetal radiation dose for the short spiral method (approximately half the scan length compared to the standard spiral method) was examined on a phantom in a previous study (20). In the current study, only a lateral topogram is included and the short spiral scan was performed with lower kV (100 kV) which can be presumed to give a lower fetal radiation dose. However, the current study involves elderly women and one can only assume that pregnant women’s skeleton at a younger age provides better image quality, meaning that lowering the reference mAs can be made, which in turn further reduces the fetal radiation dose.

Several analyses have been used in evaluation studies to investigate reliability, including Pearson correlation coefficient and ICC (27,28). The ICC reflects a degree of correlation between measurements of the same variable and absolute agreement between measurements (28). In the analyses one imputation of the data was made, which presumably had a low effect on the reliability outcome.

The rule of thumb emphasizes that the value of measurement reliability should be at least ≥0.90 for acceptable measurements for a patient in clinical practice. Lower values might be satisfactory for group level measurements or for research purposes (30). Keeping this line of reasoning, results from the study demonstrate that the short spiral method provides reliable pelvimetry measurements for a woman in order to decide upon a vaginal delivery or planned Cesarean section. An accurate measurement is important to prevent prolonged labor and emergency Cesarean section, which is associated with maternal and neonatal morbidity as well as prolonged hospitalization and mortality (1,2).

To control for bias in measurement error in the study, all pelvimetry scans and required reconstructions were executed by the same researcher in the study and the scans were performed in the same CT scanner. However, it is important to bear in mind that reliability results from the study remain only if the method is correctly performed in clinical practice as well. Different radiographers usually conduct pelvimetry examinations; thus, it is important to adhere to the radiographic method by clear methodology guidelines and training. Before pelvimetry measurements, the observers discussed the measurement landmarks and reconstructions needed; this is also important to do in clinical practice, for example in clinical training of new colleagues. According to clinical practice, the observers were allowed to compute additional reconstructions that happened twice in the short spiral method at measurement timepoint T1. This might have happened due to the inexperience of using short spiral pelvimetry. In addition, the pelvimetry measurements were supposed to mirror clinical practice where conducting additional reconstructions is a common routine.

The sample size of 10 individuals may be considered low, but it was important to keep the number of women that would receive additional X-ray radiation in the study as low as possible but still be able to secure the strength of statistical analyses. According to the guidelines, at least 40 observations are needed to analyze measurement reliability (31). The sample size of 10 women delivered four measurement values for each individual and pelvimetry method, which resulted in 120 observations at one measurement timepoint for inter-rater analysis and 80 observations for intra-rater analysis. Measurement reliability between the reference pelvimetry and each method under study relied on 80 observations. It follows that the data provided sufficient samples for reliability analysis (31) and no unnecessary radiation scans were performed. In other words, a larger sample could be interpreted as more reliable but redundant in terms of sufficient reliability analyses and therefore not justified in a view of ionizing radiation.

In conclusion, the short spiral pelvimetry demonstrated the highest and best reliability in comparison to other CT methods investigated. The standard spiral method showed good measurement reliability in comparison to the reference but the short spiral pelvimetry method generates a lower fetal radiation dose. Therefore, we consider this method might be suitable for measurements of the birth canal before delivery. However, patient acceptance and attitude to CT pelvimetry should be investigated.