Radiographic Texture Reproducibility: The Impact of Different Materials, their Arrangement, and Focal Spot Size.

BACKGROUND: Feature reproducibility is a critical issue in quantitative radiomic studies. The aim of this study is to assess how radiographic radiomic textures behave against changes in phantom materials, their arrangements, and focal spot size.
METHOD: A phantom with detachable parts was made using wood, sponge, Plexiglas, and rubber. Each material had 1 cm thickness and was imaged for consecutive time. The phantom also was imaged by change in the arrangement of its materials. Imaging was done with two focal spot sizes including 0.6 and 1.2 mm. All images were acquired with a digital radiography machine. Several texture features were extracted from the same size region of interest in all images. To assess reproducibility, coefficient of variation (COV), intraclass correlation coefficient (ICC), and Bland-Altman tests were used.
RESULTS: Results show that 59%, 50%, and 4.5% of all features are most reproducible (COV ≤5%) against change in focal spot size, material arrangements, and phantom's materials, respectively. Results on Bland-Altman analysis showed that there is just a nonreproducible feature against change in the focal spot size. On the ICC results, we observed that the ICCs for more features are >0.90 and there were few features with ICC lower than 0.90.
CONCLUSION: We showed that radiomic textures are vulnerable against changes in materials, arrangement, and different focal spot sizes. These results suggest that a careful analysis of the effects of these parameters is essential before any radiomic clinical application. Copyright:

Introduction

Radiomics is a new advanced image processing issue that aims to quantify medical images for several purposes. In the radiomic studies, several features are extracted and are used for clinical applications such as tissue characterizations, therapy response assessment, survival analysis, and response and metastasis prediction.[12345] A wealth of data has been published on the clinical applications of radiomics, but there are concerns on the use of these features due to their vulnerability against changes in radiomic process including image acquisition, image processing, image reconstruction, image segmentation, and data analysis.[6789]

A considerable amount of literature has indicated that radiomic features extracted from imaging modalities including computed tomography (CT), positron emission tomography, single-photon emission CT, and magnetic resonance imaging (MRI) could vary against change in the imaging settings, radiomic settings, and data analysis,[101112131415] but few studies are available on the radiomic reproducibility in radiographic studies. In our previous study, we showed that radiomic features extracted from digital bone radiography change against variations in exposure parameters (kV and mAs), filters, source-to-film distance, and tube angles.[9]

As a simple rule, radiography images are attenuation shadows of the objects to be imaged. In this regard, the object's atomic number, mass density, electron density, and thickness are the natural issues for determining the image quality. On the other hand, exposure parameters and focal spot sizes are other important factors for determining image quality. Focal spot is the area of the target that is struck by the electrons in X-ray tube. It is essential that focal spot has to be sufficiently large to avoid local overheating and anode damage. In addition, it has to be as small as possible to achieve maximum sharpness in the radiographic image. There are different focal spot sizes in X-ray tube called small and large focal spots. The focal spot sizes commonly employed are 0.3 mm and 0.6 mm, usually for mammography, and 1.0 mm and 1.2 mm, usually for general radiography.[16]

In the present study, for the first time, we aimed to assess how radiographic texture features may vary with change in the focal spot size, different phantom materials, and change in the arrangements of phantom materials. In several clinical situations, there are several variations in the tissue materials and arrangements that would be important in radiomic analysis. For example, tissue arrangements are different in posteroanterior, anteroposterior, and oblique views in projection radiographic imaging. In addition, based on the location of tissue being studied, the materials and their arrangement vary. For example, in imaging of tissues that are located in the lung, abdomen, and pelvis, the type and order of the materials in the path of radiation beam are different. It will also be a major issue for people with different body sizes and people with prostheses.

To our knowledge, there is no study on this topic. To analyze our aims, we made a simple phantom using wood, Plexiglas, rubber, and sponge. The radiomic features extracted from images are assessed in terms of reproducibility and agreement using statistical tests including coefficient of variation (COV), Bland–Altman tests, and intraclass correlation coefficient (ICC). The results of this study may be interesting in the field of radiomics.

Materials and Methods

Phantom preparation

To facilitate the study of the variability of radiomic texture against focal size, different materials, and their arrangements, we developed a simple phantom with detachable parts. The materials were chosen to produce a wide range of radiomic texture values in the range of values found in human tissues. These materials were wood, sponge, Plexiglas, and rubber. Wood is selected because has a variable atomic number and could be issued as heterogeneous tissue. In addition, wood is made up of carbon, which is equivalent to many tissues in the body. Plexiglas is an approved tissue equivalent material that is used in several imaging phantoms. Rubber is equivalent to some human tissue such as muscles. Sponge is also a water equivalent material that could be considered as a homogenous tissue. In addition, these materials are used as phantom in some radiomic studies.[1718]

The size of phantom materials was the same and 1 cm, because different thicknesses change the radiomic feature values and induce a new bias. These parts were detachable, and we were able to change their arrangements. We showed the phantom and its component in Figure 1.

Figure 1

The phantom and its components

Image acquisition

Several image acquisitions were obtained: (1) imaging each phantom's part separately with focal spot size 0.6 mm and with (2) focal spot size 1.2 mm and (3) imaging the whole phantom with different arrangements of phantom's parts (24 modes) with focal spot size 0.6 mm and with (4) focal spot size 1.2 mm. All these issues were repeated three folds at consecutive time. The imaging parameters were Kv, 40; mAs, 4; source-to-image distance, 100 cm; and filter, 1 mm Al. All images were acquired using a general digital radiography machine (Mehranteb, Iran). The machine is equipped with flat panel detector, with maximum voltage 150 and focal spot 0.6 and 1.2 mm. We also checked the machine in terms of quality assurance, consistency, and calibration before imaging. In addition, electronical and mechanical issues were checked and validated.

Image preprocessing and feature extraction

To assess texture feature reproducibility, all images were inputted into the MaZda texture analysis software (MaZda 3.20, The Technical University of Lodz, and Institute of Electronics) and textures were extracted from a same size region of interest (ROI) drawn on the all images. Before feature extraction and in order to noise reduction, increasing sensitivity, and normalizing the intensities, all image intensities are normalized between μ ±3 σ, where μ is the mean value of gray levels inside the ROI. We extracted 22 texture features, namely Percentile, AngScMom, Contrast, Correlat, DifEntrp, DifVarnc, Entropy, InvDfMom, SumAverg, SumEntrp, SumOfSqs, SumVarnc, Fraction, GLevNonU, LngREmph, RLNonUni, ShrtREmp, Teta, WavEnHH, WavEnHL, WavEnLH, and WavEnLL. The full names and more details on these features are available in Table 1. These features were intensity histogram, gray-level co-occurrence matrix (GLCM), gray-level run-length matrix (GLRLM), autoregressive model, and wavelet feature sets.

Table 1

Texture features

Full name	Abbreviation	Feature set
Percentile	Percentile	Intensity histogram
Angular Second Moment	AngScMom	GLCM
Contrast	Contrast	GLCM
Correlation	Correlat	GLCM
Difference Entropy	DifEntrp	GLCM
Difference Variance	DifVarnc	GLCM
Entropy	Entropy	GLCM
Inverse Different Moment	InvDfMom	GLCM
Sum Average	SumAverg	GLCM
Sum Entropy	SumEntrp	GLCM
Sum Of Square	SumOfSqs	GLCM
Sum Variance	SumVarnc	GLCM
Fraction	Fraction	GLRLM
Gray Level Non Uniformity	GLevNonU	GLRLM
Long Run Emphasis	LngREmph	GLRLM
Run Length Non Uniformity	RLNonUni	GLRLM
Short Run Emphasis	ShrtREmp	GLRLM
Teta (Θ)	Teta	Autoregressive model
Wavelet Energy High High	WavEnHH	Wavelet
Wavelet Energy High Low	WavEnHL	Wavelet
Wavelet Energy Low High	WavEnLH	Wavelet
Wavelet Energy Low Low	WavEnLL	Wavelet

GLCM – Gray-level co-occurrence matrix; GLRLM – Gray-level run-length matrix

Reproducibility analysis

To assess texture feature reproducibility, we conducted three statistical tests as the following:

COV, obtained by Eq. (1):

Where standard deviation (SD) and mean are SD and mean of feature value over applying different focal spot sizes, phantom's materials, and arrangement, respectively. This test was done to assess feature reproducibility against focal spot size, change in the phantom's materials, and arrangement of these materials.

Four groups of COV including a very small (COV ≤5%), small (5% < COV ≤ 10%), intermediate (10% < COV ≤ 20%), and large (COV >20%) were assessed to categorize variation. Texture features with COV ≤5% were considered as most reproducible features. To show these COVs on the figures, we considered numbers 1–4 for COV ≤5%, 5% < COV ≤ 10%, 10% < COV ≤ 20%, and COV >20%, respectively.

ICC obtained by Eq. (2):

Where BMS and RMS represent the between-subjects and residual mean squares, and d is the total number of variables. This test was done to assess feature test–retest reproducibility for each phantom's materials in three repeated consecutive times for both small and large focal spot sizes.

Bland–Altman analysis

This was applied to evaluate the mean, SD, and upper/lower reproducibility limit (URL/LRL) for radiomic features in response to change in focal spot size. The URL/LRL was obtained by Eqs. (3) and (4), respectively, as the following:

We used U/LRL as criteria to classify the level of reproducibility: high –1%≤ U/LRL ≤30%, intermediate –30%< U/LRL ≤45%, and low –45%< U/LRL ≤50%. We considered any feature below the low level as nonreproducible (NR). We used this criterion as was proposed by Atlazi et al. and was reported by several clinical studies.[19]

All statistical analyses were performed using GraphPad Prism (version 8.0.2 (263), 2019) and MedCalc Statistical Software version 19.0.5 (MedCalc Software, Ostend, Belgium; http://www.medcalc.org; 2019].

Coefficient of Variation Results

Our results on COV for radiomic features against changes in focal spot size, phantom's materials and arrangements are depicted in Figure 2.

Figure 2

Results on COV for radiomics features against changes in focal spot size, phantom's materials and arrangements. 1: COV .5%, 2: 5% < COV < 10%, 3: 10% < COV< 20%, and 4: COV >20%. COV: Coefficient of variation

Results show that 59% of all features (13 of 22) are most reproducible (COV ≤5%) and 27% (6 of 22) have the large variations (COV >20%) against change in focal spot size. We also showed that all GLRM and four textures from GLCM texture sets including Entropy, SumAverg, SumEntp, and SumOfSqs are most reproducible (COV >20%) over the change in focal spot size. In addition, Teta and three wavelet textures including WavEnHL, WavEnLH, and WavEnLL have the very small variations and are most reproducible (COV >20%).

For radiomic reproducibility against change in the arrangement and phantom's materials, the following results were obtained [Figure 2]:

When the focal spot size was small (0.6 mm)

For the arrangement analysis, we observed that 50% of all features (11 0f 22) have the smallest variations (COV ≤5%) and 36% (8 of 22) have the largest variations (COV >20%). All GLRLM textures, Teta, WavEnLL, and four textures from GLCM texture sets including Entropy, SumAverg, SumEntp, and SumOfSqs were most reproducible (COV ≤5%).

For the reproducibility analysis against change in phantom's materials, we found that just five textures including SumAverg, SumOfSqs, SumVarnc, RLNonUni, and WavEnLL are most reproducible (COV ≤5%). From all textures, just AngScMom had the very large variation (COV >20%).

When the focal spot size was large (1.2 mm)

For the reproducibility analysis against change in the arrangement of phantom's materials, our results were exactly the same when the focal spot size was large.

For the phantom's materials analysis, we observed that just three textures including SumAverg, ShrtREmp, and Teta were most reproducible (COV ≤5%) and three textures, namely SumAverg, WavEnLH, and WavEnHH, had very large variations (COV >20%).

For all settings, SumAverg was the most reproducible feature (COV ≤5%).

Intraclass Correlation Coefficient Results

Our results on ICC including test–retest reproducibility analysis for each phantom's materials in three repeated consecutive times for both small and large focal spot sizes are depicted in Figure 3. For this section, the following results were obtained:

Figure 3

Results on ICC including test-retest reproducibility analysis for each phantom's materials in three repeated consecutive times for both small and large focal spot sizes. ICC: Intraclass correlation coefficient

For sponge (S), the ICCs were >0.98 (range: 0.98–1). For wood (W), there was a feature, SumOfSqs, that had ICC, 0.5697, and ICCs for other features were >0.97 (range: 0.97–0.99). For rubber®, SumAverg had ICC, 0.7319, and the range of ICC for others was 0.92–0.99. For Plexiglas (P), we found that two features, SumAverg and SumOfSqs, had ICC of 0.628 and 0.8146, respectively. The range of ICC for other features was 0.97–0.99.

For sponge (S), there were three features, namely SumEntrp, SumOfSqs, and LngREmph, with ICC of 0.6367, 0.7136, and 0.7492, respectively. The ICC was >0.99 for other features. For wood (W), rubber®, and Plexiglas (P), ICCs for features were >0.9 and its range was 0.99–1, 0.97–1, and 0.91–1, respectively.

Bland–Altman results

Results on Bland–Altman analysis are shown in Table 2 and in Figures 4-8 for all texture features. We observed that there is just a NR feature and the 21 others have high reproducibility against change in the focal spot size. This feature is SumVarnc and belongs to GLCM texture feature set.

Table 2

Bland-Altman analysis of radiomic features

Feature set	Features	Mean difference±SD	LRL	URL	Level of reproducibility
Intensity histogram	Percentile	−0.7061±7.269	−14.95	13.541	High
	AngScMom	8.294±0.4303	−0.7738	1.6191	High
	Contrast	0.4782±0.4499	−1.3818	0.3772	High
	Correlat	−24.45±59.43	−9.6214	1.19	High
	DifEntrp	−1.748±0.08921	1.5729	1.9226	High
	DifVarnc	0.3638±0.4171	−1.1814	0.4537	High
GLCM	Entropy	−1.468±0.02564	1.4174	1.5179	High
	InvDfMom	1.229±0.3243	−1.8646	−0.5933	High
	SumAverg	−0.4747±0.007418	0.4601	0.4892	High
	SumEntrp	−1.438±0.2090	1.028	1.8474	High
	SumOfSqs	−0.8866±0.01528	0.8567	0.9166	High
	SumVarnc	−1.550±0.3096	61.4281	116.6446	Nonreproducible
	Fraction	−0.5519±0.8899	−1.1924	2.2961	High
	GLevNonU	3.927±3.945	−3.806	11.6601	High
GLRLM	LngREmph	1.512±2.651	−3.6848	6.7088	High
	RLNonUni	−1.522±2.278	−2.9435	5.9881	High
	ShrtREmp	−0.4227±0.6104	−0.7738	1.6191	High
Autoregressive model	Teta	−0.4421±0.9330	−2.271	0.387	High
	WavEnHH	1.69±4.037	−6.215	9.609	High
Wavelet	WavEnHL	1.210±1.375	−8.2625	19.9328	High
	WavEnLH	4.611±5.904	−6.9614	16.1834	High
	WavEnLL	1.476±0.7253	0.05468	2.8978	High

GLCM – Gray-level co-occurrence matrix; GLRLM – Gray-level run-length matrix; SD – Standard deviation; LRL – Lower reproducibility limit; ULR – Upper reproducibility limit

Figure 4

Bland-Altman analysis on Percentile, a feature from Intensity histogram feature set

Figure 8

Bland-Altman analysis on Wavelet feature set including a) WavEnLL b) WavEnLH, c) WavEnHL and d) WavEnHH

Bland-Altman analysis on GLCM feature set including a) AngScMom, b) Contrast, c) Correlation, d) DiffEntp, e) DiffVarnc, f) InvDiffMom, g) SumAvrg, h) SumEntp, i) SumOfSqrs, and i) SumVarnc

Bland-Altman analysis on GLCM feature set including a) Fraction, b) LngREmph, c) GlevNonU, d) ShrtREmp and e) RlNonUni

Bland-Altman analysis on Teta, a feature from Autoregressive model feature set

Discussion

Biomarker discovery using radiomic features is a promising issue for personalized medicine applications. However, there are concerns due to radiomic feature changes over the change in imaging, radiomic, and data analysis settings. To address this issue, a number of studies have suggested that radiomic features have to be checked in terms of reproducibility, repeatability, and robustness over variations in the imaging and radiomic stings.[101120]

To our knowledge, the radiomic feature reproducibility against change in focal spot size, different materials, and different arrangements of the materials has not been evaluated, although many studies have focused on other imaging and radiomic settings. In the present study, we investigated these parameters. This work was conducted as a phantom study, and 22 texture features from different feature sets were examined. We observed that several features have variations against changes in the focal spot size, phantom materials, and the arrangement of the materials. This result may be beneficial for clinical studies.

According to COV analysis, we found that 6 features out of 11 GLCM features are not most reproducible (COV ≤5%) and have variations against change in the focal spot size. GLCM is a matrix that is defined as the distribution of co-occurring pixel values (grayscale values or colors) at a given offset. Based on some previous studies, some GLCM features have been reported as the highest reproducible and predictive radiomic features against variation in imaging ad image processing settings.[2122] On the other hand, all GLRLM texture features were fund as high reproducible features. GLRLM is a matrix that gives the size of homogeneous runs for each gray level. The element (i, j) of GLRLM corresponds to the number of homogeneous runs of jj voxels with intensity ii in an image. Some of these feature sets have been introduced as feasible biomarkers for clinical diagnosis, prognosis, and prediction.[512]

As an interesting part of our study, we identified that radiomic texture features have different behaviors against change in the phantom's materials and arrangement when the focal spot size changes. We found that for both focal spot sizes, there are same results for change in the arrangement of phantom's materials. While for the change of materials, the variations in radiomic features are greater when the focal spot size changes. The reasons for this variation are not clear, but it may be suggested that change in the image quality due to large focal spot size has great impact on the features. The other reason may be due to nature of the features. Another reason could be that geometric blurring increases with increasing focal spot size.

In another part of our study, we did test–retest radiomic reproducibility analysis for repeated imaging of phantom materials separately. We observed that the ICCs for more features are >0.90 and there were few features with ICC lower than 0.90. We found that these features are belonging to GLCM feature set and there was a feature from GLRLM feature set. In addition, the effect of focal spot size on this issue was different. To explain this event, we suggest that nature of features, phantom's material structures, atomic number, and density may have role in the feature variations. However, further studies with sufficient imaging data including different phantom materials are needed to more explain these issues.

Although our results are significant, this study has some limitations. We tested just four materials and with an imaging machine. Further studies with more phantom materials and multicenter collaborative efforts are needed to find most reproducible features. In addition, more radiomic features from different feature sets could be tested. In another way, these testes may be done by other imaging modalities such as CT scan and MRI with modified settings.

Conclusions

This study examined the reproducibility of several radiomic features extracted from radiographic images of a detachable phantom composed of wood, sponge, Plexiglas, and rubber in response to the variation of parameters including focal spot size, phantom's materials, and phantom material arrangement. According to our results, several radiomic features within the scope of this study were highly affected by variations of such parameters. These results suggest that a careful analysis of the effects of these parameters is essential before any radiomic clinical application.

Financial support and sponsorship

None.

Conflicts of interest

There are no conflicts of interest.

BIOGRAPHIES

Younes Qasempour is a BS student of Radiology Technology at Kerman University of Medical Sciences (KMU), Kerman, Iran. He is member of Student Research Committee and Radiology Scientific Association at KMU. His research interests re Medical Image Analysis, Radiobiology and Machine Learning.

Email: qasempouryounes@gmail.com

Amirsalar Mohammadi is a BS student of Radiology Technology at Kerman University of Medical Sciences (KMU), Kerman, Iran. He is member of Student Research Committee and Radiology Scientific Association at KMU. His research interests are Medical Imaging, Radiobiology and Biotechnology.

Email: amirasmk530@gmail.com

Mostafa Rezaie is a BS student of Radiology Technology at Kerman University of Medical Sciences (KMU), Kerman, Iran. He is member of Student Research Committee and Radiology Scientific Association at KMU. His research interests are Radiology, Image Processing and Radiobiology.

Email: mostafarezaei70@yahoo.com

Parisa Pouryazadanpanah is a BS student of Radiology Technology at Kerman University of Medical Sciences (KMU), Kerman, Iran. He is member of Student Research Committee and Radiology Scientific Association at KMU. His research interests are Radiobiology, Image Analysis and Genetics.

Email: lianayazdan7899@gmail.com

Fatemeh Ziaddini is a BS student of Radiology Technology at Kerman University of Medical Sciences (KMU), Kerman, Iran. He is member of Student Research Committee and Radiology Scientific Association at KMU. His research interests are Medical Imaging, Image Processing and Deep Learning.

Email: fatemeh9964@gmail.com

Alma Borbori is a BS student of Radiology Technology at Kerman University of Medical Sciences (KMU), Kerman, Iran. He is member of Student Research Committee and Radiology Scientific Association at KMU. His research interests are Image Processing, Image Analysis and Biomedical Engineering.

Email: fatemeh9964@gmail.com

Isaac Shiri received his M.Sc. degrees in Medical Physics from Iran University of Medical Science, Iran. He is now PhD student of Medical Physics, in Geneva University, Geneva, Switzerland. He is researcher in PET Instrumentation & Neuroimaging Laboratory (PINLab) and Radiology & Medical Informatics located in Geneva University Hospital. His main research interests are medical image analysis using conventional and machine/deep learning algorithms.

Email: Isaac.Shiri@etu.unige.ch

Ghasem Hajianfar received the MSc of Medical Physics from Kermanshah University of Medical Silences, Kermanshah, Iran. In 2019 he joined the Biomedical and Health Informatics Research Group at Rajaie Cardiovascular Medical and Research Center as a researcher. His work focuses specifically on data analysis, machine/deep learning and radio/genomics analysis in medical images.

Email: hajianfar.gh@gmail.com

Azam Janati Esfahani is an Assistant Professor of Medical Physics at Medical Biotechnology Department, Qazvin University of Medical Sciences, Qazvin, Iran. Her background is in Medical Physics, with specific expertise in the biological effect of ionizing radiation, radiotherapy, imaging, hyperthermia and simulation.

Email: janaty.azam@gmail.com

Sareh Ghasemirad serves as Assistant Professor of Emergency Medicine at Kerman University of Medical Sciences (KMU), Kerman, Iran. Her background is in General Medicine. She currently works at Emergency Medicine Departments in Bahonar and Afzalipour Medical Centers at KMU.

Email: sareh.ghasemirad@gmail.com

Hamid Abdollahi is an Assistant Professor of Medical Physics at Kerman University of Medical Sciences (KMU), Kerman, Iran. His research interests are Radiomics, Radiogenomics, Radiobiological Modelling and Machine Learning.

Email: hamid_rbp@yahoo.com