Anti-TSH receptor antibodies (TRAb): Comparison of two third generation automated immunoassays broadly used in clinical laboratories and results interpretation.

Graves' disease (GD) is the most common cause of hyperthyroidism in iodine-replete populations. It is an autoimmune disease caused by autoantibodies to the TSHR (TRAb). Although the diagnostic is mainly clinical, measuring TRAb improves accuracy and provides valuable prognostic information. The aim of this study was to compare the performance of two of the most widely used immunoassays i.e., EliA™ anti-TSH-R and Elecsys® anti-TSH-R. We have carried out a comparative study measuring TRAb by the two immunoassays in consecutive sera samples referred to the laboratory for TRAb measurement. Autoantibodies were measured in all samples in parallel by the two techniques. The two techniques were highly concordant as demonstrated by a Cohen's kappa of 0.82. At the manufacturer recommended cut-off, sensitivity of Elecsys® TRAb test was higher (100% vs. 96.6%), while specificity of the EliA™ TRAb test was higher (99.4% vs. 95.3%). In most patients TRAb are detected by any of two tests which are both well suited for Clinical Laboratories use. However, a higher specificity may constitute an advantage for measurement used not for screening but for diagnostic purposes, as anti-TSH-R is.

Introduction

Graves’ disease (GD) is an organ-specific autoimmune disease of the thyroid gland that affects predominantly women (ratio about 8:1) between 30 and 50 years old and is the most common cause of hyperthyroidism in iodine-replete populations [1]. The mechanism of hyperthyroidism in GD is the production of autoantibodies to the Thyroid Stimulating Hormone Receptor (TSH-R) that mimic the effects of the thyrotropin. The TSH-R belongs to the family of 7TM G-protein coupled receptors and is expressed by thyroid follicular cells and, to a much lower level, by thymocytes and fibroblasts of retro-orbital tissue [2]. Based on their functional effect on TSH-R, three types of antibodies can be considered: stimulating, blocking, and cleaving (“neutral” in biological activity terms) TRAb [3]. The stimulating TRAb are the most common and the cause of hyperthyroidism in GD patients [4].

The autoantibodies to the TSH-R are of high affinity but remain at a low absolute concentration as they are produced by a limited number of clones of B lymphocytes and plasma cells and this may explain why in a few patients, the response may shift from producing predominantly stimulating to blocking or neutral antibodies with the corresponding change in the state of the thyroid function and the clinical symptoms [5, 6].

Like other autoimmune diseases, GD is a chronic condition that requires early diagnosis to prevent permanent tissue damage e.g., osteoporosis, ophthalmopathy, myopathy, and personality changes. The diagnostic is based on the recognition of the symptoms of hyperthyroidism, goiter, eye signs, and the measurement of thyroid hormones and TSH.

Measurement of TRAbs is important to confirm GD and rule out other causes of hyperthyroidism making the diagnosis much more accurate [7-9] and can be critical in cases of Graves’ Ophthalmopathy (GO) [10, 11] without hyperthyroidism. Since the discovery of TRAb more than 60 years ago as long-acting thyroid stimulator (LATS) by Adams and Purves [12], the measurement of TRAb has progressively improved in sensitivity, specificity, reliability, and usability and this has expanded its clinical application as reflected in many recent guidelines and surveys [13-17].

TRAb tests can be divided into main two categories depending on the detection method used: competition immunoassays and bioassays. Competition immunoassays detect all types of anti-TRAbs by measuring their ability to compete with a labeled ligand (TSH or a monoclonal antibody (MoAb) to TSH-R) for binding to the TSH receptor. Bioassays can detect the stimulating or blocking effect of the TRAb by measuring the production cAMP, the TSH-R intracellular signal, by cells expressing the TSH-R [11, 14]. Even if discrimination among the types of TRAbs could be of great interest in given clinical situations, such as in cases of unexplained changes in the thyroid function during or after pregnancy [18], competition immunoassays, that are easier, faster, and can be automated, are the tests commonly used in clinical diagnostic laboratories.

TRAb are autoantibodies and as such, they are not a molecularly defined analyte but a mixture of high-affinity IgG that bind selected epitopes of the TSH-R that varies among individuals and fluctuates within one individual. Small changes in the level, affinity, or fine specificity of the TRAb can result in major changes in their capacity to activate the TSH-R. Measuring TRAbs is, therefore, challenging, and generations of tests using different TSH-R preparations and ligands have been developed over the years, while in parallel, labeling and detection methods have also improved for the immunoassays in general. Therefore, the number of reports comparing tests in terms of sensitivity, specificity, safety, and cost-effectiveness as applied to different populations, have grown over the years [19-25] reflecting that none of them have yet met all the expectations of the clinical endocrinologist.

Therefore, the objective of this work is to compare the accuracy in terms of sensitivity and specificity of two automated 3rd generation immunoassays in use for TRAb testing of a clinical laboratory.

Materials and methods

A comparative study was designed and conducted from February 2019 to July 2019 and the STARD guidelines were followed. The project was carried out at the Clinical Laboratories of Hospital Universitario Dr. Peset in Valencia (Spain), ISO 9001:2015 certified.

Informed consent was not requested as per ethics committee indication. The reason is that data was appropriately anonymized and all determinations were requested by specialists as part of the routine diagnostic testing. The study just included the determination requested by the specialists performed with two different methods. Medical decisions were taken according to the Hopitals’ routine testing method results.

Patients

Baseline demographics and clinical characteristics of patients are given in Table 1.

Table 1

Demographic and clinical data.

	NDGD	GD-R	GD-T	AIT	MNG	OTD	MISC	TOTAL
Number	29	27	76	18	35	56	60	301
Female (%)	86.2	92.6	71.4	83.3	88.6	71.4	73.3	80.1
Age years (mean ± SD)	49.4 ±18.5	48.8 ±14.8	48.1 ±15.1	43.2 ±13.7	62.9 ±14.0	52.4 ±17.2	50 ±19.5	50.9 ±17.1

Abbreviations: GD, Graves’ Disease GD-R, in remission; GD-T, under treatment; GDGO, GD Orbitopathy; NDGD, newly diagnosed; AIT Autoimmune Thyroiditis; MNG multinodular goiter; OTD, other thyroid diseases; Misc, Miscellaneous patients

Participants were divided in different clinical groups according to their diagnosis (Table 1 and Fig 1):

Fig 1

Patient distribution in different clinical groups according to the final diagnosis.

Group 1: 29 newly diagnosed untreated GD patients (NDGD)

Group 2: composed by 27 cases of GD patients in remission (GD-R)

Group 3: composed by 76 cases of GD patients under treatment, GD-T

Group 4: 18 cases of Autoimmune Thyroiditis (AIT), composed by 11 classical Hashimoto’s thyroiditis and 7 postpartum thyroiditis, as assessed by TPO positive antibodies, imaging and hormones levels.

Group 5: 35 patients with non-toxic Multinodular Goiter (MNG) as assessed by history, image, and hormone levels.

Group 6: Other Thyroid Diseases (OTD), 56 patients with thyroid diseases other than GD, i.e., non-immune hyper or hypothyroidism, iatrogenic thyroid disorders, single thyroid nodules, and thyroid cancer.

Group 7: Miscellaneous (MISC), 60 cases referred to our laboratory because of diseases unrelated to thyroid pathology. This group was considered the control group in the study.

We did not make a category of patients with Graves’ Orbitopathy (GO), due to the low number of patients with GO (n = 3).

Thyroid pathology diagnoses were established by qualified endocrinologists from Endocrinology and Nutrition Service from Hospital Universitario Dr. Peset, according to the American Thyroid Association and European Thyroid Association guidelines [13, 15].

Samples

Eligibility criteria

Consecutive samples received at the laboratory for TRAb testing from both Primary Care and Endocrinology specialists. To be included samples had to conform to the general pre-analytical requirements of the laboratory as established in the corresponding Standard Operating Procedure. None was excluded because of hemolysis, lipemia, or bilirubin, as these features do not interfere with the tests according to the specifications of the method. A total of 307 samples were initially tested but 6 were excluded from analysis, 2 due to incomplete clinical information, and 4 because they were duplicates. Results from 301 samples were used for the statistical analysis.

TRAb test measurement methods

TRAb were measured in freshly obtained or -80ºC kept serum samples in parallel by two immunoassays, the fluorescence enzyme immunoassay (FEIA) EliA™ anti-TSH-R that uses the Phadia 250 auto-analyzer (Phadia A.B, Thermo Fisher Scientific, Uppsala, Sweden) and the electrochemiluminiscent immunoassay (ECLIA) Elecsys® anti-TSH-R method run in a COBAS 6000 auto-analyzer (Roche Diagnostics International Ltd, Basel, Switzerland).

The EliA™ anti-TSH-R test is an immunoassay that detects both stimulating and blocking antibodies. It is a competitive assay based on a human recombinant TSH-R (rTSH-R) immobilized to a solid phase (wells). Serum samples are dispensed to the wells for TRAb autoantibodies to bind to the TSH-R. A β-galactosidase labeled mouse monoclonal stimulating anti-TSH-R antibody is added in a second step. This MoAb binds the TSH-R epitopes that remain free after the patients’ antibodies binding. After washing the free antibodies, the substrate for generating the fluorochrome is added and after a short incubation, the reaction is arrested and fluorescence measured: intensity is inversely proportional to TRAb in the problem sample. Finally, the fluorescence readings are converted into a titer of anti-TSH-R antibodies using a 6-point calibration curve [24].

The Elecsys® anti-TSH-R method (Roche Diagnostics Ltd, Basel, Switzerland), is also an immunoassay that works using the same competitive principle as EliA™ test, but with three major differences: it uses an immobilized purified porcine TSH-R (pTSH-R) as antigen, a human monoclonal stimulating autoantibody, M22, labeled with ruthenium as a competitor, and a chemiluminescent based detection. The more antibodies present in the patient serum sample, the less luminescence is detected in the final measurement [25].

Both assays were carried out according to manufacturer specifications. Their most relevant technical parameters are showed in Table 2.

Table 2

Manufacturer specifications of the compared assays.

	Elecsys	EliA™
Negative	≤1.75 IU/L	<2.9 IU/L
Grey zone	None	2.9–3.3 IU/L
Positive	>1.75 IU/L	>3.3 IU/L
LoQ	0.8 IU/L	1.5 IU/L
UpQ	40 IU/L	68 IU/L
Calibration standard	90/672 NIBSC	08/204 NIBSC
Assay time	Depending on the system used	Depending on the system used
Sample volume	45 μL	50 μL
Functional sensitivity	1.5 IU/L	0.3 IU/L
Precision (CV)	Intra-run: 6.1% Inter-run: 6.13%	Intra-run: 4.7% Inter-run: 6.31%

Abbreviations: LoQ = Lower limit of Quantitation. UpQ = Upper limit of Quantitation; NIBSC, National Institute Biologicals Standards and Controls (UK)

Statistical analysis

The number of patients required for this study was estimated by extrapolation from reference reports describing similar accuracy comparative studies for TRAb measurement (24,25).

For calculating the diagnostic accuracy of the tests, results from patients already under treatment were not included as anti-thyroid drugs may reduce TRAb levels. Results in the grey zone for EliA™ test were considered negative in this comparison.

Statistical analysis was performed using SAS system 9.4 (SAS, Cary, USA) and GraphPad Prism (GraphPad Software, San Diego, USA) software packages. A two-sided value of p < 0.05 was considered statistically significant.

The results were compared as quantitative and qualitative variables, by applying Spearman’s correlation and Passing-Bablok respectively. TRAb values below the lower limits and over the upper limits of quantitation (LoQ and UpQ respectively) were considered equal to limit values for statistical purposes. Bland-Altman test was not applicable as the Shapiro-Wilk test showed that TRAb results were not normally distributed. Results are given as median with range and the subsequent statistical analysis was performed with Cohen’s kappa to study the agreement of categorical variables (positive vs. negative) and Kruskal-Wallis test to compare the groups of patients.

Receiver operating characteristic (ROC) curves were plotted and analyzed using Analyse-it Version 4.60 for Microsoft Excel (Leeds, UK) to compare the area under the curve (AUC) by both methods. Clinical sensitivity, specificity, predictive value, and likelihood ratio (LR) were also calculated.

Results and discussion

Anti-TSH-R immunoassays performance comparison

The main results are summarized in Table 3. Spearman correlation showed significant positive linearity of results for both tests (0.725). It was therefore adequate to apply the Passing-Bablok test where the titers from both tests were shown not to be comparable as they showed systematic and proportional differences (Fig 2).

Table 3

Statistical analysis comparing results from Elecsys® and EliA™ assays.

TEST		RESULT
Spearman correlation		rho = 0.725 p < 0.0001
Passing-Bablock		A = -1.6891 (-1.9436 to -1.4675)
		B = 1.2732 (1.1750 to 1.3786)
Shaphiro-Wilk		p < 0.0001
Cohen’s Kappa		K = 0.8176 (0.7486 to 0.8866)
Kruskal-Wallis	Elecsys ®	NDGD vs GD, AIT, MNG, OTD, MISC p < 0.001
	EliA TM	NDGD vs GD, AIT, MNG, OTD, MISC p < 0.001
ROC	Elecsys ®	AUC 0.995
		Cut-off 1.75 IU/L
		Sensitivity 100%
		Specificity 95.3%
		LR+ 21.1
	EliA TM	AUC 0.996
		Cut-off 3.3 IU/L
		Sensitivity 96.6%
		Specificity 99.4%
		LR+ 163.2

Abbreviations: AUC, area under de curve; GD, Graves’ Disease GD-R, in remission; GD-T, under treatment; GDGO, GD Orbitopathy; NDGD, newly diagnosed; AIT Autoimmune Thyroiditis; MNG multinodular goiter; OTD, other thyroid diseases; MISC, Miscellaneous patients

Fig 2

Passing-Bablok regression.

Correlation between EliA™ and Elecsys® TRAb. TRAb are displayed in UI/l. Dotted line (- - - -) corresponds to identity. Blue line corresponds to Passing Bablock regression and double dotted line (-- -- --) to 95% CI bands. Tests values are displayed as orange circles.

Bland-Altman plot was performed for a specific descriptive purpose, as the result differences were not normally distributed according to Shapiro-Wilk test p<0.0001 (Fig 3) and therefore this is not the most adequate representation for the results of these tests.

Fig 3

Bland-Altman plot.

Inter-assay agreement between EliA™ and Elecsys® TRAb. The dashed line (........) corresponds to bias. Lower (-8.036) and upper (6.195) 95% limits of agreement are plotted as speckled lines. Y-axis plots the difference scores of EliA™ and Elecsys® TRAb measurements against the mean for each studied sample.

Cohen’s kappa index for categorized values was k = 0.8176, indicating strong agreement which was higher for negative than for positive results (99.5% and 78.5%, respectively).

The comparison between paired clinical groups, performed by Kruskal-Wallis test (Fig 4) showed that NDGD is statistically different from all the other groups for both Elecsys® and EliA™ methods, with the logical exception of GD-T.

Fig 4

Box-plot of Kruskal-Wallis analysis.

A. Elecsys® TRAb levels in the different groups enrolled in the study. B. EliA™ TRAb levels in the different groups enrolled in the study. TRAb levels are displayed as data ranks obtained by transforming to continue data ranks using Roche’s test. For each box plot, the central line represents the median; the boxes limits represent the upper and lower quartile; black points are outliers (a value more than 1.5 times the interquartile range above/below the interquartile values); and the black lines are the whiskers, which extend from the interquartile ranges to the maximum values that are not classed as outliers.

NDGD, newly diagnosed untreated GD patients; GD-R, GD patients in remission; GD-T, GD patients under treatment; AIT, Autoimmune Thyroiditis; MNG, Multinodular Goiter non-toxic; OTD, Other Thyroid Diseases; MISC, Miscellaneous.

According to ROC analysis (Fig 5), the optimal cut-off (OCO) for EliA™ method is 3.2 IU/L, at which the EliA™ method has a sensitivity of 96.6% and a specificity of 99.4%

Fig 5

ROC (Receiver operating curve) analysis for EliA™ and Elecsys®.

ROC curve of both methods at manufacturer cut-off values.

The sensitivity and specificity of Elecsys® method at the manufacturer given cut-off value, 1.75 IU/L, are 100% and 95.3% respectively.

The area under the curve (AUC) for Elecsys® was 0.995 and 0.996 for EliA™. They showed the high diagnostic efficiency of both methods with a minimal non-statistically significant difference.

The value of TRAb measurement in GD diagnosis and treatment monitoring is widely accepted in clinical practice and it is now included in scientific society guidelines. Even though currently used competitive assays do not distinguish between the different types of TRAb activities, overall binding TSH inhibition measurement is sufficient for supporting clinical diagnosis in most cases. The exceptions are a few complex clinical situations such as neonatal hyperthyroidism where a bioassay could be useful [14].

Given the different calibration standards used, Elecsys® and EliA™ results are not directly comparable. This lack of direct comparability among 3rd generation methods has already been reported [21]. Articles comparing different TRAb assays including the EliA™ or the Elecsys® tests have been published, but to our knowledge, this is the first head-to-head direct comparison (Table 4).

Table 4

Test performance values obtained from comparative studies for TRAb measurement methods including EliA™ and/or Elecsys® evaluation.

Study	Methods	Cut-off	Sensitivity	Specificity
Current study (2019)	EliA™ T-Fisher	3.2 IU/L**	96.6%	99.4%††
	Elecsys® Roche	1.75 IU/L*	100%	95.3%††
Struja et al. (2019)	EliA™ T-Fisher	3.3 IU/L*	71.1%	97.9%††
Villalta et al. (2018)	EliA™ T-Fisher	3.8 IU/L**	94.7%	99.6%†
Doroudian et al. (2017)	Elecsys® Roche	1.75 IU/L*	95%	97% † (90%†††)
Luther et al. (2017)	EliA™ T-Fisher	3.3 IU/L*	79%	97.7%††

*Manufacturer suggested cut-off.

**Optimal cut-off according to ROC.

†Specificity value comparing GD to Healthy Controls.

††Specificity value Comparing GD to Disease Controls.

†††Specificity value comparing GD to Healthy and Disease Controls (MTNG/Primary Autoimmune Hyperthyroidism).

In our analysis, the Elecsys® shows slightly higher sensitivity while the EliA™ test a slightly higher specificity. Villalta et al. [24] compared EliA™ with TRAK™ RIA, Brahms (Thermo Scientific, Hennigsdorf, Germany) and Immulite™ TSI assay (Siemens Healthcare, Llanberis, UK) but not the Elecsys. For the EliA™ test the reported specificity of 99.6% is very similar to the 99.4% in our study, but in our series the optimal cut-off of 3.2 IU/L is closer to the upper limit of normality provided by the manufacturer, and slightly lower than the optimal cut-off of 3.8 IU/L reported in their paper.

In the report of Struja et al. [25], the EliA™ test was the most specific (97.9% at the manufacturer cut-off of 3.3 IU/L) and showed a specificity of 97.7% at the same cut-off in Luthers’ work [26]. Sensitivity was much lower in Struja and Luther series (71.7% and 79% respectively), but both studies included GD patients under treatment while we did not.

In the paper of Villalta et al. [24] the EliA™ sensitivity for untreated GD was of 94.7% which is slightly lower than our result of 96.6%. Adopting their cut-off point to our series, the sensitivity would be down to 73.8% because some of our untreated GD patients (5/29) are in the low positive range, from 3.3 to 3.8 IU/L.

In relation to Elecsys® anti-TSH-R method, Dourudian et al. found a sensitivity of 95% and a specificity of 97% when comparing their GD patients to a healthy control group [23] However, when they compared GD patients to a mixed group of healthy and disease controls, the specificity decreased to 90%. The sensitivity of Elecsys® in our series is 100%, as we did include any case of seronegative GD during the period of the study. We may have to re-assess this issue in future work.

There is a good concordance between Elecsys® and EliA™ for negative results and a total concordance for clearly positive results. The controversy is focused on Elecsys® low positives that could be equivocal/negative for EliA™. But this is an expected result when comparing one assay that prioritizes sensitivity with another prioritizing specificity. In their comparative study, Autilio et al. [27] found the four near cut-off false positives Elecsys® corresponding to chronic autoimmune thyroiditis cases. We also found six near cut-off positives by Elecsys® that by EliA™ were five negative and one equivocal, all of them corresponded to the OTD group.

A patient from OTD group with a nodular thyroid gland having a single dominant nodule was positive for both methods, but in this case, we cannot strictly consider this a false positive as overlapping cases of GD and multinodular thyroid are not rare.

The overall impression considering quantitative and qualitative results is that both Elecsys® and EliA™ are good methods for TRAb measurement in clinical laboratories. Whether to prioritize sensitivity or specificity and how to manage the results around cut-off and how to deal with grey zone results are difficult questions. The fundamentals of the two methods are different for these aspects. EliA™ prioritizes specificity and defines a grey zone to report intermediate results. Elecsys®, at the manufacturer’s cut-off value of 1.75 IU/L, clearly prioritizes sensitivity.

In this study, we have focused on test performance, however, other test characteristics like assay time, sample volume, and precision are also worthy of evaluation when selecting the appropriate test for a specific laboratory. Nevertheless, it is difficult to draw general conclusions, as their impact mainly depends on the type of laboratory system used, and the laboratory workflow. Sample volume is not critical in daily routine, however, lower volume sample needs are always better for patients.

As other studies have suggested [28], our results support the use of a higher cut-off point for the Elecsys® test in clinical laboratory use, as our preference is to prioritize a higher specificity. In principle, for low prevalence diseases as most autoimmune diseases, a small reduction in specificity is translated into an excess of false positives, as compared to false negatives caused by a small reduction in sensitivity. Moreover, this test is much more a confirmatory test than a screening test in the diagnostic strategy of GD, addressing a parameter in which positivity has been sometimes considered pathognomonic [29, 30].

In the same line with the results from Villalta et al. [24] series for Hashimoto’s thyroiditis patients, we observed an absence of TRAb positivity in our AIT patients group.

In relation to establishing a single cut-off or considering grey zones, both ways to report results are perfectly valid if the interpretation is carefully reported to the clinicians. It is also important that clinicians understand the limitations of the technique in use, and this is the role of clinical laboratory specialists.

Our study has two main limitations, first, we did not include a healthy patients group as the negative control, which is always good to have. Second, we did not evaluate the performance of both tests for patients’ disease follow-up and relapse prediction. This is also a key point when deciding which test to include in the daily routine, and our next objective as follow-up performance data is scarce on TRAb measurement methods.

Conclusions

In conclusion, both methods are perfectly valid and useful for routine TRAb analysis, with a trend to specificity in the case of EliA™. Moreover, it is very important to raise awareness to the clinicians on the proper interpretation of results falling in the grey zone or close to cut-off values. Given the increasing agreement on literature about TRAb utility in GD diagnosis and management and in order to minimize costs, TRAb measurements must be appropriately integrated into the routine of GD diagnosis and management.

(TIF)

Click here for additional data file.

8 Mar 2022

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Purvi Purohit

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please provide additional details regarding participant consent. In the ethics statement in the Methods and online submission information, please ensure that you have specified what type you obtained (for instance, written or verbal, and if verbal, how it was documented and witnessed). If your study included minors, state whether you obtained consent from parents or guardians. If the need for consent was waived by the ethics committee, please include this information.

Additional Editor Comments (if provided):

The manuscript needs major revisions and is not acceptable in its current format. If the authors wish to continue they should address the queries raised by the reviewer.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: José María López Ortega et al. manuscript "Anti-TSH receptor antibodies (TRAb): comparison of two third generation automated immunoassays broadly used in clinical laboratories and results interpretation" studied the comparison of accuracy in terms of sensitivity and specificity of two automated IIIrd generation immunoassays in use for TRAb testing of a clinical laboratory, the recent EliATM method (ThermoFisher Scientific, Germany) and the Elecsys® /Cobas ECLIA (Roche Diagnostics Ltd, Switzerland). This paper is easy to understand and read. Some of the aspects to be improved to favor its understanding is the methodology and result sections. I would like to add some comments and suggestions to this paper.

1. Please write a full form of abbreviation (TRAb) {Page No.4; Line No. 67}

2. I think the cost per sample matters a lot here, it is very important to compare it, today most of the third world countries are facing the lowest level of health services, so the lowest cost of any diagnostics test is to reach every poor patient.

3. Apart from this, by doing a comparative analysis on the following points, their explanation can be helpful in improving this Manuscript.

(A) Assay Time

(B) sample volume

(C) Measuring Range

(D) Functional Sensitivity

(E) Cutoff

(F) Precision

4. Please correct typing error. {Page No.15; Line No. 277}

"For the EliATM test the reported specificity of 99.6% is very similar to the 99.4% in our study, but in our series the optimal cut-off of 3,2 IU/L is closer the upper limit of normality provided by the manufacturer, and slightly lower than the optimal cut-off of 3.8 IU/L reported in their paper."

5. Please explain "disease control" in the manuscript.

6. In Figure 3, denote each specific sample in relation to disease.

7. Many figure captions do not explain what colouring and boxing mean in many figures. The figure legend needs a lot of improvement.

8. I would like to see a critical discussion on limitations in the current study.

9. The authors perform very similar analyses to https://doi.org/10.1186/s12902-019-0363-6, except I would argue the linked paper is more thorough since they use more rigorous statistical analysis. The authors should justify what their paper adds to the field that this paper doesn’t.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

7 Jun 2022

Dear reviewers,

Thank you very much for the time you've dedicated to evaluate our work. Please find below the response to all the comments:

2. Reviewer 1

Comment from the author: all changes applied to the manuscript due to reviewer suggestions are highlighted in the manuscript in light blue.

2.1. Comment 1

Please write a full form of abbreviation (TRAb) {Page No.4; Line No. 67}

Response

The full form abbreviation has been written. See below:

“functional effect on TSH-R, three types of TRAb or Thyrotropin Receptor Autoantibodies (TRAb) can be considered:”

2.2. Comment 2

I think the cost per sample matters a lot here, it is very important to compare it, today most of the third world countries are facing the lowest level of health services, so the lowest cost of any diagnostics test is to reach every poor patient.

Response

We completely agree with the reviewer comment, but we were not able to conduct a health economic evaluation. It must be taken into account that, in our country, health assistance is free for all citizens, so we are focusing on offering solutions that provide better aid in the diagnosis of patients. Moreover, the price of a test is subjected to many variables (public tenders, test volume, etc.).

A reader from third world countries must evaluate the results of this study in the context of the country-specific prices for the methods, and select the best option for them.

2.3. Comment 3

3. Apart from this, by doing a comparative analysis on the following points, their explanation can be helpful in improving this Manuscript.

(A) Assay Time

(B) sample volume

(C) Measuring Range

(D) Functional Sensitivity

(E) Cut-off

(F) Precision

Response

Again we agree with reviewer comments. Some of the aforementioned assay characteristics were already described in table 2 (measuring range, functional sensitivity, and cut-off) but we have added the ones missing (assay time, sample volume, and precision). Also, we have added the following paragraph to the discussion (page 16, line 310)

“In this study, we have focused on test performance, however, other test characteristics like assay time, sample volume, and precision are also worthy of evaluation when selecting the appropriate test for a specific laboratory. Nevertheless, it is difficult to draw general conclusions, as their impact mainly depends on the type of laboratory system used, and the laboratory workflow. Sample volume is not critical in daily routine, however, lower volume sample needs are always better for patients.”

2.4. Comment 4

Please correct typing error. {Page No.15; Line No. 277}.

Response

The typing error has been corrected. See below:

“99.4% in our study, but in our series the optimal cut-off of 3.2 IU/L is closer the”

2.5. Comment 5

Please explain "disease control" in the manuscript.

Response

We did not include a “disease control group” if it is defined as “healthy population”. However, we included 60 samples received for TRAb measurement corresponding to diagnostics unrelated to thyroid pathologies. With the exception of a couple of patients with Diabetes Mellitus type I, samples at this group corresponded to patients under control and not affected by autoimmune diseases or under clinical processes of a minor entity. We have modified the text accordingly to make it clear (pag 7, line 131):

“Group 7: Miscellaneous (MISC), 60 cases referred to our laboratory because of diseases unrelated to thyroid pathology. This group was considered the control group in the study.”

2.6. Comment 6

In Figure 3, denote each specific sample in relation to disease.

Response

We would like to denote each specific sample in relation to disease as the reviewer comments, however a modification like the proposed requires the involvement of the statistician and, due to the short timespan we’ve had, It has not been possible. As it is a clear improvement, but we do not consider it critical, we leave it to your criteria to include the modification for the final publication if you think it is essential, and of course, the manuscript is accepted.

2.7. Comment 7

Many figure captions do not explain what coloring and boxing mean in many figures. The figure legend needs a lot of improvement.

Response

We completely agree with reviewer's comments. We have modified figure captions in order to better explain them. See below the modified figure captions:

“Fig 1. Patient distribution in different clinical groups according to the final diagnosis.

Fig 2. Passing-Bablok regression. Correlation between EliA™ and Elecsys® TRAb. Trab are displayed in UI/l. Dotted line (- - - -) corresponds to identity. Blue line corresponds to Passing Bablock regression and double dotted line (-- – – –) to 95% CI bands. Tests values are displayed as orange circles.

Fig 3. Bland-Altman plot. Inter-assay agreement between ELIA™ and Elecsys® TRAb. The dashed line (........) corresponds to bias. Lower (-8.036) and upper (6.195) 95% limits of agreement are plotted as speckled lines. Y-axis plots the difference scores of ELIA™ and Elecsys® TRAb measurements against the mean for each studied sample.

Fig 4. Box-plot of Kruskal-Wallis analysis. A. Elecsys®TRAb levels in the different groups enrolled in the study. B. EliATM TRAb levels in the different groups enrolled in the study. TRAb levels are displayed as data ranks obtained by transforming to continue data ranks using Roche’s test. For each box plot, the central line represents the median; the boxes limits represent the upper and lower quartile; black points are outliers (a value more than 1.5 times the interquartile range above/below the interquartile values); and the black lines are the whiskers, which extend from the interquartile ranges to the maximum values that are not classed as outliers.

NDGD, newly diagnosed untreated GD patients; GD-R, GD patients in remission; GD-T, GD patients under treatment; AIT, Autoimmune Thyroiditis; MNG, Multinodular Goiter non-toxic; OTD, Other Thyroid Diseases; Misc, Miscellaneous.

Fig 5. ROC (Receiver operating curve) analysis for EliATM and Elecsys®. ROC curve evaluates the accuracy of both methods at manufacturer cut-off values, showing the tend to the specificity of EliATM assay.”

2.8. Comment 8

I would like to see a critical discussion on limitations in the current study.

Response

We especially agree with the reviewer. All manuscripts must include a description of the limitations of the study in the discussion section and our manuscript does not include it. Thus, have included the following paragraph in the discussion (page 13, line 248):

“Our study has two main limitations, first, we did not include a healthy patients group as the negative control, which is always good to have. Second, we did not evaluate the performance of both tests for patients' disease follow-up and relapse prediction. This is also a key point when deciding which test to include in the daily routine, and our next objective as follow-up performance data is scarce on TRAb measurement methods.”

2.9. Comment 9

The authors perform very similar analyses to https://doi.org/10.1186/s12902-019-0363-6, except I would argue the linked paper is more thorough since they use more rigorous statistical analysis. The authors should justify what their paper adds to the field that this paper doesn’t.

Response

Struja’s manuscript is really interesting for groups working in the field. However, although Struja’s comparative study includes EliATM anti-TSH-R, it does not include Elecsys® anti-TSH-R. EliATM and Elecsys® are two of the main used methods in our environment and that's why we decided to perform this specific comparison, to have a head-to-head comparison between two methods that have never been compared before, and that can be useful for professionals working in the field.

In fact, we highlight the novelty of the comparison in the manuscript (pag 13, line 261).

“Articles comparing different TRAb assays including the EliATM or the Elecsys® tests have been published, but to our knowledge, this is the first head-to-head direct comparison (Table 4).”

Submitted filename: Response to Reviewers.docx

Click here for additional data file.

21 Jun 2022

Anti-TSH receptor antibodies (TRAb): comparison of two third generation automated immunoassays broadly used in clinical laboratories and results interpretation

PONE-D-21-37268R1

Dear Dr.  José María López Ortega

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Purvi Purohit

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Congratulations to the authors. The manuscript has been revised satisfactorily and can now be accepted for publication.

14 Jul 2022

PONE-D-21-37268R1

Anti-TSH receptor antibodies (TRAb): comparison of two third generation automated immunoassays broadly used in clinical laboratories and results interpretation

Dear Dr. López Ortega:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Purvi Purohit

Academic Editor

PLOS ONE