Use of Receiver Operating Characteristic (ROC) Curve Analysis for Tyrer-Cuzick and Gail in Breast Cancer Screening in Jiangxi Province, China.

BACKGROUND Breast cancer is a malignant tumor derived from breast gland epithelium. The screening and early diagnosis of breast cancer in high-risk populations can effectively suppress its threat to women's health and improve treatment efficiency, and thus has critical importance. Using various evaluation models, the present study evaluated cancer risk in 35-69-year-old women, and the usefulness of models in breast cancer prevention was compared. MATERIAL AND METHODS A total of 150 infiltrative breast cancer patients who were diagnosed with breast cancer at our hospital were recruited, along with 130 healthy women as the control group. A retrospective study was performed to collect information. The 5-year risk of breast cancer was evaluated using the Gail and Tyrer-Cuzick models. Diagnostic results were analyzed to plot ROC curves for comparing the value for screening between Gail and Tyrer-Cuzick models. RESULTS The Gail model has 53.33% sensitivity and 77.69% specificity, with 73.39% positive prediction value, 59.06% negative prediction value, 64.64% accuracy, and 0.31 Jordon index. The Tyrer-Cuzick model had 66.00% sensitivity, 86.92% specificity, 85.34% positive prediction value, 68.90% negative prediction value, 75.71% accuracy, and 0.53 Jordon index. The area under the curve (AUC) was 0.665 for the Gail model (95% CI: 0.629~0.701) and 0.786 for the Tyrer-Cuzick model (95% CI: 0.757~0.815). CONCLUSIONS Both Gail model and Tyrer-Cuzick models can be used to evaluate breast cancer risk. The Gail model has relatively lower accuracy in evaluating breast cancer risk in Jiangxi province of China and the Tyrer-Cuzick model had relatively higher accuracy.

Background

Breast cancer (BC) is a malignant tumor derived from breast gland epithelium and is the most common cancer in women [1-3]. Most BC patients are already at advanced or terminal stage at the time of primary diagnosis. Due to rapid progression and high metastasis and recurrence rates, treatment efficacy is not satisfactory. Distal metastasis toward other vital organs an important cause of mortality in BC patients [4,5]. Therefore, effective screening of BC and timely identification of BC patients can reduce BC morbidity and mortality and improve treatment efficiency, thus having important clinical implications.

The feasibility of large-scale screening for BC is limited by medical resources and clinical technique. Therefore, the development of rapid, easy-to-use, and effective screening tools for BC has become the critical challenge for BC prevention and treatment. To date, various BC screening models have been developed, including the Gail, Tyrer-Cuzick, Couch, Frank, BRCAPRO, and Claus models. Among these, the Couch, Frank, BRCAPRO, and Claus models focus more on high-risk populations and special situations, thus having limited usefulness [6-8]. The Gail model was initially proposed in 1989, and has advantages of rapid progress, easy manipulation, low cost, and effectiveness, and thus has become the most widely used and standard risk evaluation model for BC, with wide clinical application [9-11]. The Gail model was derived from the USA, and mainly targets white females, with less study in other regions, thus making its applicability in Chinese women unclear. Compared to the Gail model, the Tyrer-Cuzick model has not been widely assessed in large-population studies. The Tyrer-Cuzick model includes multiple factors such as age at menopause, body-mass index (BMI), benign disease of breast gland, BC in relatives, family history, and hormonal replacement therapy into a comprehensive investigation [7,12,13]. A previous study found the Tyrer-Cuzick model has relatively higher accuracy [14]. The present study took place in Jiangxi province, China and evaluated the utility of the Tyrer-Cuzick and Gail models in BC screening.

Material and Methods

Research objects

This study used a risk evaluation model to analyze BC risk among 35–69-year-old women. The disease group consisted of infiltrative BC patients who received ultrasound or X-ray examination with pathology confirmation at the Affiliated Jiujiang Hospital of Nanchang University (N=150, average age=52.6±11.8 years). The control group consisted of healthy women who received breast ultrasound or X-film examination in our hospital (N=130, averaged age=54.3±12.6 years). No significant difference was found in age distribution, economic status, occupation, ethnic group, or education level between the 2 groups (p>0.05).

This study was approved by Ethics Committee of the Affiliated Jiujiang Hospital of Nanchang University and all enrolled objects signed informed consent.

Research methodology

Well-trained investigators were employed to perform retrospective information collection using a uniform questionnaire to collect general information in the control and disease groups, including age, history of breast gland disease, family history, menstrual age, pregnancy age, biopsy of breast gland, and ethnic group. Age and breast gland biopsy information were collected based on recall of the last 5 years. Other information regarding risk factors of BC pathogenesis for Gail model were also collected.

Well-trained investigators were employed to perform retrospective information collection using a uniform questionnaire to collect general information in the control and disease groups for the Tyrer-Cuzick model, including age, height, body weight, menopause age, menstrual age, pregnancy age, history of benign breast gland disease, estrogen exposure experience, family member with BC, and number of bilateral BCs. All information and data related to BC risk factors in the Tyrer-Cuzick model were collected. Personal information was input based on the questionnaire to calculate 5-year BC risk prediction value.

Statistical analysis

All questionnaire data were input into computer software. Using a diagnostic trial approach, sensitivity and specificity of the Gail or Tyrer-Cuzick model in BC risk evaluation were calculated. The 5-year BC risk was estimated using both models for disease and control groups to calculate the ROC AUC, along with the paired chi-square test. A statistically significant difference was defined when p<0.05.

Results

Evaluation results using Gail model

Gail model analysis was performed among all 280 individuals included. High BC risk was defined as higher than 1.2% cancer risk, and other scenarios were defined as low BC risk (boundary level of risk was the upper limit of the ROC curve). The Gail model had 53.33% sensitivity, 77.69% specificity, 73.39% positive prediction value, 59.06% negative prediction value, 64.64% accuracy, and 0.31 Jordon index (Table 1). The difference was statistically significant by paired chi-square test (χ2=16.980, p<0.001, Table 2).

Table 1

Estimation of Gail model diagnostic trial.

Index	Calculated value	95%CI
Sensitivity	53.33%	45.62~61.04%
Specificity	77.69%	69.28~86.10%
Positive prediction value	73.39%	65.51~81.27%
Negative prediction value	59.06%	51.08~67.04%
Jordon index	0.31	0.27~0.35
Accuracy	64.64%	59.16~70.12%

Table 2

Gail model diagnostic trial results.

Risk	Disease		Control		χ2	P
	N	%	N	%
High	80	53.3	29	22.3	16.980	<0.001
Low	70	46.7	101	77.7
Total	150	100.0	130	100.0

Tyrer-Cuzick model evaluation results

Tyrer-Cuzick model analysis was performed among all 280 individuals included. High BC risk was defined as higher than 1.59% cancer risk, and other scenarios were defined as low BC risk (boundary level of risk was the upper limit of the ROC curve). The Tyrer-Cuzick model had 66.00% sensitivity, 86.92% specificity, 85.34% positive prediction value, 68.90% negative prediction value, 75.71% accuracy, and 0.53 Jordon index (Table 3). The difference was statistical significant by paired chi-square test (χ2=17.000, p<0.001, Table 4).

Table 3

Estimation of diagnostic trial parameters in Tyrer-cuzick model.

Index	Calculated value	95%CI
Sensitivity	66.00%	59.68~72.32%
Specificity	86.92%	81.55~92.29%
Positive prediction value	85.34%	78.59~92.09%
Negative prediction value	68.90%	61.73~76.07%
Jordon index	0.53	0.48~0.58
Accuracy	75.71%	71.06~80.36%

Table 4

Tyrer-cuzick model diagnostic trial results.

Risk	Disease		Control		χ2	P
	N	%	N	%
High	99	66.0	17	13.1	17.000	<0.001
Low	51	34.0	113	86.9
Total	150	100.0	130	100.0

ROC curve analysis for diagnostic values of Gail and Tyrer-Cuzick model

ROC curves were plotted for Gail and Tyrer-Cuzick model to evaluate their predicted risk values for BC diagnosis. ROC analysis showed the area under the curve (AUC) was 0.665 for the Gail model (95% CI: 0.629~0.701, Figure 1A) and 0.786 for the Tyrer-Cuzick model (95% CI: 0.757~0.815, Figure 1B).

Figure 1

ROC curve analysis for diagnostic value of Gail and Tyrer-Cuzick models.

Discussion

The incidence for BC is increasing since 1970s worldwide. It is estimated that newly discovered BC cases account for about 25% of all cancers in women, and BC-induced mortality accounts for 15% cancer-related death in women [15]. There are about 1 500 000 newly diagnosed BC patients every year worldwide and about 500 000 (one-third) of these women died from BC [16]. Therefore, BC has become the primary morality factor threatening women’s health. In both developed and developing countries, BC is the primary cause of death among malignant tumors in women [17]. Compared to Western countries, BC incidence in China is relatively lower. However, in recent years, with lifestyle and dietary changes, increased mental and work pressure, environmental pollution, and endocrine disruption, BC incidence in China is rapidly increasing. Currently, BC has become the most common malignant tumor in Chinese women [18-20]. The early pathogenesis of BC is relatively insidious, frequently without typical symptoms or body signs. Therefore, misdiagnosis and neglect are frequent. In the early stage, BC progresses rapidly and is prone to metastasis toward peripheral tissues and major organs, including bone, lung, lymph node, skin, and brain, leading to high rates of recurrence and mortality rates. Therefore, timely deployment of BC screening and discovery and diagnosis of BC patients can effectively decrease the threat of BC to women’s health and improve treatment efficiency, thus having clinical importance.

Currently, various developed countries have initiated early screening programs for BC. For example, the USA has begun early screening of BC in community-wide physical exams. In China, BC screening is mainly achieved by large-scale screening in major hospitals, in the form of breast gland X-ray, ultrasound, or even biopsy. These approaches are difficult and have poor patient compliance, making them hard to promoted widely. The Gail model was proposed by Gail in 1989. Researchers analyzed clinical data from more than 1 000 000 women and predicted the 5-year BC rate and lifetime risk. It is also used for recommendation for further screening or drug prevention in high-risk populations. The Gail model was co-designed by the National Cancer Institute (NCI) and the National Surgical Adjuvant Breast and Bowel Project (NSABP), and is accredited by the National Comprehensive Cancer Network (NCCN) as a strategy for prevention and diagnosis [21,22]. Compared to other screening approaches, the Gail model is easy to carry out. However, due to its origin in the USA and because it mainly targets white women, the sensitivity and accuracy of BC screening in China still require further investigation. Compared to the Gail model, the Tyrer-Cuzick model mainly targets British, Australians, and New Zealanders, and incorporated menopause age, BMI, history of breast gland benign disease, family history of BC, and hormonal replacement therapy, thus increasing its accuracy [7,12,13]. In sharp contrast with the Gail model, the Tyrer-Cuzick model has not been widely studied in large-population investigations, so its sensitivity and accuracy are unclear.

Results of our study show that the Gail model had 53.33% sensitivity and 77.69% specificity, with 73.39% positive prediction value, 59.06% negative prediction value, 64.64% accuracy, 0.31 Jordon index, and 0.665 ROC AUC. These results reveal that the Gail model has relatively higher sensitivity and specificity in predicting BC in a high-risk population of Jiangxi women. However, the Gail model does not consider information such as family history, lack of secondary relative, contralateral BC information, relative disease onset age, lactation or abortion, or history of hormonal replacement therapy, all of which can form risk factors of BC onset, leading to its relatively lower predictive value. Therefore, the present study used the Tyrer-Cuzick model to investigate its value in screening of female BC. Results showed that the Tyrer-Cuzick model had 66.00% sensitivity, 86.92% specificity, 85.34% positive prediction value, 68.90% negative prediction value, 75.71% accuracy, 0.53 Jordon index, and 0.786 ROC AUC. Warwick et al. [12] showed the ROC AUC was 0.62 using the Tyrer-Cuzick model in female BC screening, similar to our results. Our results showed that the Tyrer-Cuzick model had higher sensitivity, specificity, and diagnostic value for Jiangxi female BC screening than the Gail model. Brentnall et al. [7] observed higher diagnostic value of the Tyrer-Cuzick model compared to the Gail model, further supporting our results. However, in contrast to Boughey et al. [13], who recruited nearly 10 000 cases, our study had a relatively small sample size, compromising its representativeness. Moreover, this was a retrospective case study, and may involve memory bias for information collected 5 years ago. Therefore, large-sample prospective studies are required for to compare the Gail and Tyrer-Cuzick models in risk evaluation of BC. In summary, we found that the Gail model had relatively lower accuracy in evaluating BC risk of Jiangxi women, while the Tyrer-Cuzick model had higher accuracy for evaluating BC risk in our province.

Conclusions

The Gail model has relatively lower accuracy in evaluating BC risk in women in Jiangxi province and the Tyrer-Cuzick model has higher accuracy in evaluating BC risk among Jiangxi women.