PURPOSE: A radiomics features classifier was implemented to evaluate segmentation quality of heart structures. A robust feature set sensitive to incorrect contouring would provide an ideal quantitative index to drive autocontouring optimization. METHODS: Twenty-five cardiac sub-structures were contoured as regions of interest in 36 CTs. Radiomic features were extracted from manually-contoured (MC) and Hierarchical-Clustering automatic-contouring (AC) structures. A robust feature-set was identified from correctly contoured CT datasets. Features variation was analyzed over a MC/AC dataset. A supervised-learning approach was used to train an Artificial-Intelligence (AI) classifier; incorrect contouring cases were generated from the gold-standard MC datasets with translations, expansions and contractions. ROC curves and confusion matrices were used to evaluate the AI-classifier performance. RESULTS: Twenty radiomics features, were found to be robust across structures, showing a good/excellent intra-class correlation coefficient (ICC) index comparing MC/AC. A significant correlation was obtained with quantitative indexes (Dice-Index, Hausdorff-distance). The trained AI-classifier detected correct contours (CC) and not correct contours (NCC) with an accuracy of 82.6% and AUC of 0.91. True positive rate (TPR) was 85.1% and 81.3% for CC and NCC. Detection of NCC at this point of the development still depended strongly on degree of contouring imperfection. CONCLUSIONS: A set of radiomics features, robust on "gold-standard" contour and sensitive to incorrect contouring was identified and implemented in an AI-workflow to quantify segmentation accuracy. This workflow permits an automatic assessment of segmentation quality and may accelerate expansion of an existing autocontouring atlas database as well as improve dosimetric analyses of large treatment plan databases.
PURPOSE: A radiomics features classifier was implemented to evaluate segmentation quality of heart structures. A robust feature set sensitive to incorrect contouring would provide an ideal quantitative index to drive autocontouring optimization. METHODS: Twenty-five cardiac sub-structures were contoured as regions of interest in 36 CTs. Radiomic features were extracted from manually-contoured (MC) and Hierarchical-Clustering automatic-contouring (AC) structures. A robust feature-set was identified from correctly contoured CT datasets. Features variation was analyzed over a MC/AC dataset. A supervised-learning approach was used to train an Artificial-Intelligence (AI) classifier; incorrect contouring cases were generated from the gold-standard MC datasets with translations, expansions and contractions. ROC curves and confusion matrices were used to evaluate the AI-classifier performance. RESULTS: Twenty radiomics features, were found to be robust across structures, showing a good/excellent intra-class correlation coefficient (ICC) index comparing MC/AC. A significant correlation was obtained with quantitative indexes (Dice-Index, Hausdorff-distance). The trained AI-classifier detected correct contours (CC) and not correct contours (NCC) with an accuracy of 82.6% and AUC of 0.91. True positive rate (TPR) was 85.1% and 81.3% for CC and NCC. Detection of NCC at this point of the development still depended strongly on degree of contouring imperfection. CONCLUSIONS: A set of radiomics features, robust on "gold-standard" contour and sensitive to incorrect contouring was identified and implemented in an AI-workflow to quantify segmentation accuracy. This workflow permits an automatic assessment of segmentation quality and may accelerate expansion of an existing autocontouring atlas database as well as improve dosimetric analyses of large treatment plan databases.