A blinded international study on the reliability of genetic testing for GGGGCC-repeat expansions in C9orf72 reveals marked differences in results among 14 laboratories.

BACKGROUND: The GGGGCC-repeat expansion in C9orf72 is the most frequent mutation found in patients with amyotrophic lateral sclerosis (ALS) and frontotemporal dementia (FTD). Most of the studies on C9orf72 have relied on repeat-primed PCR (RP-PCR) methods for detection of the expansions. To investigate the inherent limitations of this technique, we compared methods and results of 14 laboratories.
METHODS: The 14 laboratories genotyped DNA from 78 individuals (diagnosed with ALS or FTD) in a blinded fashion. Eleven laboratories used a combination of amplicon-length analysis and RP-PCR, whereas three laboratories used RP-PCR alone; Southern blotting techniques were used as a reference.
RESULTS: Using PCR-based techniques, 5 of the 14 laboratories got results in full accordance with the Southern blotting results. Only 50 of the 78 DNA samples got the same genotype result in all 14 laboratories. There was a high degree of false positive and false negative results, and at least one sample could not be genotyped at all in 9 of the 14 laboratories. The mean sensitivity of a combination of amplicon-length analysis and RP-PCR was 95.0% (73.9-100%), and the mean specificity was 98.0% (87.5-100%). Overall, a sensitivity and specificity of more than 95% was observed in only seven laboratories.
CONCLUSIONS: Because of the wide range seen in genotyping results, we recommend using a combination of amplicon-length analysis and RP-PCR as a minimum in a research setting. We propose that Southern blotting techniques should be the gold standard, and be made obligatory in a clinical diagnostic setting. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

Introduction

In 2011, an expansion of a GGGGCC-repeat in the gene ‘Chromosome 9 open reading frame 72’ (C9orf72) was identified as a cause of amyotrophic lateral sclerosis (ALS, OMIM614260) and frontotemporal dementia (FTD, OMIM105550).1 2 The following 3-years series of publications reported that a large proportion of ALS (1–30%) and FTD (6–30%) Caucasian patients carry a C9orf72 repeat expansion,3–5 making this mutation the most common known genetic cause of ALS and FTD, and one of the most frequent genetic alterations causing neurodegenerative diseases overall. In one of the initial reports, a combination of amplicon-length analysis, repeat-primed PCR (RP-PCR) assays, and Southern blot (SB) was used for detection and calculation of the repeat numbers.1 SB is regarded as the gold standard for detecting large polynucleotide repeat expansions,6 but it is relatively expensive, cumbersome and time consuming, and up to 10 µg of high-quality DNA is needed for a single analysis. It is not surprising, therefore, that in nearly all studies published during 2011–2013, the much simpler, cheaper and faster-to-perform PCR-based screening methods were used.2 By using amplification primers flanking the repeat motif, the amplicon-length analysis allows determination of the exact repeat numbers of alleles with up to 30 repeats, and thus, is able to exclude a pathological repeat expansion if two different alleles in the wild-type range are detected. In RP-PCR, at least two primers are used: one primer that hybridises outside the repeat motif, and one primer that binds to the repeat motif itself. In most protocols a third primer is applied that hybridises to an oligonucleotide tail of the repeat motif binding primer, also leading to the term triplet-primed PCR.7 In this RP-PCR, a large GGGGCC-repeat expansion typically gives rise to a ‘saw-tooth’ or ‘stutter’ pattern, which has been taken as evidence for the presence of a disease-associated expansion.1 2 Since the first reports of this mutation, a large number of studies have reported on the epidemiology, clinical, psychological and imaging features, and postmortem neuropathology of C9orf72 repeat expansion carriers with ALS, FTD, Alzheimer disease, Parkinson's disease, and other neurodegenerative diseases, as well as in healthy individuals.8 In many of these studies, the C9orf72-genotype was assessed using RP-PCR alone without a confirmatory SB analysis. Only recently, large-scale studies using SB analysis have been published.9 10 PCR-based C9orf72 screening is also used in clinical diagnostic testing of affected individuals, as well as in predictive testing of healthy individuals at-risk of ALS and FTD. The aim of this study is to determine the sensitivity and specificity of different C9orf72 genotyping methods, and to establish recommendations for molecular testing of the GGGGCC-repeat expansion in C9orf72. Fourteen experienced genetic laboratories participated in this study, and all laboratories were requested to perform RP-PCR with or without amplicon-length analyses on the same 78 samples.

Materials and methods

Study design

At the initiative of the ALS research laboratory at Umeå University, Sweden, 20 laboratories with extensive experience in performing DNA analysis for neurodegenerative diseases, including ALS and FTD, were invited to participate in this study. Eleven research laboratories and two diagnostic laboratories agreed to participate, and including Umeå University a total of 14 laboratories participated. Umeå University sent 400 ng DNA from 78 familial ALS or FTD cases to 13 laboratories: eight in Europe, four in North America, and one in Asia. All participating research laboratories had already established the methodologies for identifying the GGGGCC-repeat expansion in C9orf72 and published at least one manuscript regarding C9orf72. Each laboratory was asked to analyse the 78 DNA samples according to their own procedures and classify the results. Umeå University collected all results independently and analysed them in a blinded fashion. Eleven laboratories used RP-PCR and amplicon-length analysis, and three laboratories used RP-PCR alone. Independently to RP-PCR and amplicon-length analysis, SB was performed in three laboratories.

Patients and DNA extraction

Blood samples of 78 familial ALS or FTD patients from 32 families living in the Nordic countries, Switzerland and Portugal were collected. The sex ratio was 1.75 males per female. The ALS patients were diagnosed according to the EFNS consensus diagnostic criteria,11 and the FTD patients according to the Neary criteria.12 Autopsies were performed in 13 patients confirming the diagnosis of ALS or FTD neuropathologically. Whole venous blood was drawn into EDTA-containing vacuum tubes and following centrifugation, the buffy coat was isolated. The samples were collected during the time period 1993–2012 and stored as buffy coat in −80°C freezers until DNA extraction. The DNA was extracted according to the manufacturer's protocol with the DNA extraction kit NUCLEON BACC2 (GE Healthcare, Piscataway, New Jersey, USA) and DNA from the same extraction batch was sent to all laboratories. Ethical review boards in Sweden (The Regional Medical Review Board for Northern Sweden), Switzerland (Ethikkomission des Kantons St Gallen), and Portugal (Hospital de Santa Maria Ethics Committee, Lisbon) approved this study, and all participants gave informed written consent.

RP-PCR, amplicon-length analyses, and SB

The RP-PCR and amplicon-length analyses were done according to each laboratory's own method, and these are listed in online supplementary table S3. The SB protocols are listed in online supplementary table S5.

Results

Genotyping results: comparison of the results in 14 laboratories

The overall results of 14 laboratories (laboratory A–N) are presented in table 1. When combining amplicon-length analysis and RP-PCR, sensitivity and specificity above 95% were found in seven laboratories (A–E, G and L; 50%). The mean sensitivity of the combined results was 95.0% (73.9–100%), and the mean specificity was 98.0% (87.5–100%). Using RP-PCR analysis alone, a sensitivity and specificity of more than 95% were found in six laboratories (A–D, G and L; 42.9%). The mean sensitivity of RP-PCR alone was 94.3% (71.7–100%) and the mean specificity was 97.3% (87.5–100%). The classification determined by RP-PCR alone, therefore, changed eight genotype calls in four laboratories (E, F, J and K). Or put differently, 6 of 14 laboratories (42.8%) failed to correctly classify some samples when they performed RP-PCR analysis only. By combining RP-PCR with fragment-length analysis, four laboratories still reported false negative or false positive samples. A significant number of samples (1–10) were unclassifiable/difficult to classify in eight of the laboratories even when RP-PCR and fragment-length analysis were combined.

Table 1

The results of 14 laboratories on 78 samples

	Laboratories
	A	B	C	D	E	F	G	H	I	J	K	L	M	N
RP-PCR results
GGGGCC-repeat expansion
Yes	46	46	46	46	42	48	45	43	43	45	34	44	41	43
No	32	32	32	32	32	28	33	32	32	30	33	32	33	28
Q	0	0	0	0	4	2	0	3	3	3	11	2	4	7
False positive	0	0	0	0	0	2	0	0	0	2	1	0	0	0
False negative	0	0	0	0	1	0	1	0	0	0	2	0	1	0
Sensitivity (%)	100	100	100	100	91.3	100	97.8	93.5	93.5	93.5	71.7	95.7	89.1	93.5
Specificity (%)	100	100	100	100	96.8	87.5	100	100	100	93.8	96.9	100	100	87.5
Unclassified (%)	0	0	0	0	5.1	2.6	0	3.8	3.8	3.8	14.1	2.6	5.1	9.0
Amplicon-length analysis results
Numbers of amplicon
0	0	0	1	0	0	0	0	0	0	0	0	N.D.	N.D.	N.D.
1	54	54	55	56	56	54	54	51	56	54	51	N.D.	N.D.	N.D.
2	24	24	22	22	22	24	24	24	22	24	24	N.D.	N.D.	N.D.
Q	0	0	0	0	0	0	0	3	0	0	3	N.D.	N.D.	N.D.
RP-PCR+amplicon-length analysis results
GGGGCC-repeat expansion
Yes	46	46	46	46	46	48	45	43	43	43	34	44	41	43
No	32	32	32	32	32	29	33	32	32	30	34	32	33	28
Q	0	0	0	0	0	1	0	3	3	5	10	2	4	7
False positive	0	0	0	0	0	2	0	0	0	0	0	0	0	0
False negative	0	0	0	0	0	0	1	0	0	0	2	0	1	0
Sensitivity (%)	100	100	100	100	100	100	97.8	93.5	93.5	93.5	73.9	95.7	89.1	93.5
Specificity (%)	100	100	100	100	100	90.6	100	100	100	93.8	100	100	100	87.5
Unclassified (%)	0	0	0	0	0	1.3	0	3.8	3.8	6.4	12.8	2.6	5.1	9.0

No, sample numbers without large GGGGCC-repeat expansion; N.D., not determined; Q, sample numbers of undecided or unclassified samples Yes: sample numbers with large GGGGCC-repeat expansion.

Genotyping results: comparison of genotyped samples

The genotyping results of all 78 samples are summarised in online supplementary tables S1 and S2 available online. For 50 samples, the results among all laboratories were consistent. In the RP-PCR-only analysis, there were three individuals (6.5, 26.3 and 32.3) who were either misclassified or failed to classify in RP-PCR; these three samples had between 23 and 32 repeats on the wild-type allele, in addition to a large expansion on the other allele. In the amplicon-length analysis, two samples (samples 8.1 and 8.2) from the same family (number 8) were hard to classify: six laboratories identified two amplicons, four laboratories identified one amplicon, and one laboratory classified the samples as ‘undecided’. We sequenced these samples and revealed that one allele was a wild-type allele with two repeats, and the other allele had six repeats with a complex of 15 bp deletion/17 bp insertion mutation just after the repeat motif (online supplementary figure S1A,B). We found one more sample (sample 32.1) that carried the same nucleotide variant on one allele, and this sample also carried a large repeat expansion on the other allele (online supplementary figure S1C,D). This mutation is located in the low-complexity sequence region of C9orf72,13 and consequently, may interfere with the PCR-based genotyping method.

As a reference, SB was performed on all samples in a blinded fashion by three laboratories. There was complete concordance among the three laboratories: 46 samples carried a repeat expansion and 32 samples lacked the expansion (online supplementary figure S2).

Discussion

Our blinded multicenter study demonstrates limitations of PCR-based techniques used to assess C9orf72 GGGGCC-repeats, emphasising the need of detailed technical consensus guidelines for diagnostic and research settings.

Comparison of the methods of RP-PCR and amplicon-length analysis

Based on RP-PCR results alone, 50 samples (64.1%) showed congruent results among the participating laboratories. There are many variable RP-PCR protocols (see online supplementary table S3) and a comprehensive comparison is difficult. In summary, the RP-PCR protocols of the four laboratories (A–D) with 100% sensitivity and specificity, we found that laboratories A and C used almost identical methods. Laboratory B used primers that were a modification of the ones previously published,1 and they deleted the unspecific linker region between the fluorescence tag and C9orf72-specific sequence. In laboratory D, the RP-PCR was performed with only two primers. The primers sets P1, 2, 3 and P4, 5, 6 were the most commonly used with five laboratories using each set, and two of the laboratories that obtained 100% sensitivity and specificity used primers set P1, 2, 3. Among the laboratories that failed to classify samples in concordance with the SB results (having false positive and/or false negative), we found that one used a very short PCR elongation time, the PCR products were diluted extensively before capillary electrophoresis, and the concentration of deaza-dGTP was very low, or deaza-dGTP was not used at all.

In the amplicon-length analysis alone, 72 samples (92.3%) had concordant results, highlighting the reliability of this technique, regardless of the differences in PCR reactions and PCR protocols. The set of primers were the same in all laboratories except one, but the PCR reactions and PCR protocols were all different between the laboratories (see online supplementary table S3). Thus, it is not possible to identify specific parameters that could explain the incorrect results for the six samples that were not concordant between the laboratories.

Analyses of the results

All five laboratories that obtained full concordance of the PCR-based and SB results used RP-PCR in combination with amplicon-length analysis. None of the laboratories that performed RP-PCR alone reported the correct genotype in all samples. The sensitivity and specificity increased, and the percentage of unclassified samples decreased in three laboratories (E, F and K) when they performed RP-PCR and amplicon-length analysis. Accordingly, a combination of amplicon-length analysis and RP-PCR methodology is recommended to obtain the highest level of sensitivity and specificity, but it should be emphasised that a high risk of misclassification as either false positive or false negative (6 samples in four laboratories) still exists.

The RP-PCR results alone seemed difficult to interpret if one allele with a relatively large number (20–32) of GGGGCC-repeats was present in combination with a large repeat expansion on the other allele. This was demonstrated by three individuals who could not be genotyped using RP-PCR alone in five laboratories, possibly because the saw-tooth pattern curve of the large expanded alleles were hidden behind the peaks of the allele with 20–32 repeats (figure 1). Similarly, this intermediate 20–32 repeat allele could easily mimic an expanded allele and become a cause of false positive results.

Figure 1

Results from RP-PCR and amplicon-length analysis of sample 6.5. (A) A-1 is the full shape of RP-PCR and A-2 is a 16-times scale-up (Y-axis) shape of A-1. It is possible to see the clear saw-tooth pattern after scale-up. A-3 is the result of amplicon-length analysis and there is a number of small peaks before the highest last peak. It may be hard to identify the amplicon numbers because of these small peaks. (B) The RP-PCR figures and classifications of the same sample in different laboratories. The scale of the Y-axis is the same as in A-2.

Based on our data, there is no common definition of what a ‘GGGGCC-repeat expansion’ in C9orf72 is, and laboratories classify it in different ways. For example, some laboratories used clear cut-offs, for example, of more than 24 or 30 repeats, while other laboratories used the definition that a saw-tooth pattern in RP-PCR corresponds to a GGGGCC-repeat expansion (see online supplementary table S4). In this study, there was no false positive result based on different classifications of what is an ‘expansion’. This study was designed to compare the genotyping results of C9orf72 among laboratories using the same DNA. An identical amount of DNA was send to all laboratories, but the concentration and quality of DNA may have changed during shipping and handling. Another possible limitation of the present study is that most reactions were only performed once in each laboratory, and the accuracy may be improved if reactions were repeated in case of doubt, for example, with more DNA (however, only two laboratories requested more DNA).

Additionally, all laboratories that participated in this study used their own protocols, and hence, this study was not designed to thoroughly assess every single step in the protocol under similar circumstances within laboratories.

Proposed methods for GGGGCC-repeat expansion genotyping in C9orf72

There is an urgent need for broad consensus on analysing GGGGCC-repeat expansions in C9orf72, which is particularly important in a clinical setting (for diagnosing ALS or FTD, or when performing predictive testing of at-risk individuals), but also for research purposes. A possible algorithm for C9orf72 genotyping is presented in figure 2. In conclusion, we recommend, that as an absolute minimum, a combination of amplicon-length analysis and RP-PCR should be performed. We recommend using good quality and quantity of DNA and primers, an appropriate concentration of deaza-dGTP, and a minimum elongation time of 3 min. When results are questionable, we suggest (1) expand the analysed scale in the analysis software, (2) use a higher concentration of PCR products in the capillary electrophoresis and (3) repeat RP-PCR with a higher amount of DNA. Though five of the 14 laboratories got full concordance with SB using PCR-based techniques only, the high risk for misgenotyping using only PCR-based techniques as performed here in nine laboratories, and the devastating consequences misgenotyping may have in clinical practise, make us conclude that SB should always be employed in a diagnostic setting, and should be the preferred method in a research setting of smaller number of samples (eg, analysis of the expansion in autopsy tissue specimens). RP-PCR plus amplicon-length analysis should be used in a research setting and when many samples are to be analysed, for example, in an epidemiological study. Optimally, also is such studies should samples with a saw-tooth pattern be confirmed to have an expansion by SB (figure 2).

Figure 2

Flow chart for C9orf72 genotyping in a scientific setting.