Attribute Analytics Performance Metrics from the MAM Consortium Interlaboratory Study.

The multi-attribute method (MAM) was conceived as a single assay to potentially replace multiple single-attribute assays that have long been used in process development and quality control (QC) for protein therapeutics. MAM is rooted in traditional peptide mapping methods; it leverages mass spectrometry (MS) detection for confident identification and quantitation of many types of protein attributes that may be targeted for monitoring. While MAM has been widely explored across the industry, it has yet to gain a strong foothold within QC laboratories as a replacement method for established orthogonal platforms. Members of the MAM consortium recently undertook an interlaboratory study to evaluate the industry-wide status of MAM. Here we present the results of this study as they pertain to the targeted attribute analytics component of MAM, including investigation into the sources of variability between laboratories and comparison of MAM data to orthogonal methods. These results are made available with an eye toward aiding the community in further optimizing the method to enable its more frequent use in the QC environment.

Introduction

The use of mass spectrometry (MS) for protein therapeutic characterization has been steadily increasing over the past two decades, with the number of attributes evaluated using this technology also growing.[1] MS methodologies can be used to characterize protein therapeutics at all levels of molecular structure and are amenable to the analysis of a host of attributes. For example, molecular weight can be evaluated with the intact molecule, the location of many post-translational modifications can be ascertained via peptide analysis, and sequence verification can be performed at the amino acid level. In recent years, the introduction of the Multi-Attribute Method (MAM) by Amgen has opened the door for taking MS beyond use as a characterization tool and bringing it into the quality control (QC) lab.[2−6]

MAM can be used to perform two main functions, the first of which is attribute monitoring.[7] This process uses liquid chromatography (LC)–MS analysis of peptides to monitor critical quality attributes (CQA). MAM also functions as a purity test, which utilizes the comparison of aligned mass-to-charge (m/z) values and retention times of a product-specific reference standard to the test sample. This process of “new peak detection” (NPD) enables the detection of species that are present in one sample but not the other or those that have a difference in abundance between the reference and test samples above a set threshold.

Because MAM is capable of providing information on multiple attributes of interest, it holds promise as a single platform replacement for a number of conventional single-attribute assays within the QC space.[7] Current QC analytics rely upon the use of multiple assays to capture the body of information required for product release. Glycan heterogeneity may be assessed by hydrophilic interaction liquid chromatography (HILIC), clips may be detected using reduced capillary electrophoresis (rCE-SDS), and charge variation may be monitored using capillary zone electrophoresis (CZE), cation exchange chromatography (CEX), or capillary isoelectric focusing (cIEF). MAM not only provides the capability to inform on these and many more attributes, but also offers attribute-specific information that many conventional assays do not. For example, CEX with ultraviolet (UV) detection is often used to monitor deamidation, where an increase in the acidic peak of the chromatogram is assumed to signify increased deamidation. However, species bearing attributes such as sialic acid, cysteinylation, and glycation have also been shown to contribute to acidic peaks.[8−12] Because MAM relies upon MS detection, attributes of interest can be unambiguously identified and quantified even when coeluting species are present. In addition, data may be queried in future analyses when new attributes of interest arise, negating the need to reacquire the data.

The site-specific, direct monitoring of CQAs provided by the use of MAM is well-aligned with Quality by Design (QbD) principles that outline the need to establish correlations between CQAs and critical process parameters (CPP) with the goal of producing high-quality drug products while maintaining efficient manufacturing practices.[13,14] The most immediate applications of MAM for characterization and for tracking quality attributes during both upstream and downstream process development have been demonstrated,[4,15] and the feasibility of using MAM to analyze design of experiment in-process samples is aided by the development of automated sample preparation workflows capable of processing large numbers of samples in a short turnaround time.[15−17] MAM also has potential as a process analytical technology to monitor CQAs during production and provide feedback useful for making decisions regarding changes to CPPs.[18] Continual evolution of the method toward in-line, on-line, or at-line measurements with direct feedback loops to bioreactor process parameter settings is a future goal for MAM.

Following the introduction of the MAM platform to the industry, the MAM Consortium (www.mamconsortium.org) was established to bring members of the biopharma community together to share knowledge and experience in the development of MAM and in doing so enable acceleration of its adoption in the biopharmaceutical industry. Recently, members of the consortium came together in collaboration with the National Institute of Standards and Technology, the Institute for Bioscience and Biotechnology Research and Just – Evotec Biologics to perform a round robin study focused on the MAM platform. The overall goal of this international, interlaboratory study was to gather information regarding the instrumentation and software being used for MAM, assess the industry’s current standing regarding platform performance, and provide valuable insight for those members of the industry who are at the beginning stages of developing MAM. This study addressed both the attribute analytics and NPD functions of MAM. Those results pertaining to attribute analytics are presented here, while those pertaining to NPD were reported previously.[19]

Participants in the MAM round robin study were supplied a tryptic digest of the NISTmAb on which they performed MAM attribute analytics. The resulting data were collated and analyzed with a view toward evaluating the reproducibility of the method across laboratories. The results presented herein provide a survey of the status of MAM attribute analytics performance across the industry in addition to suggested best practices to consider.

Experimental Section

Reporting, Extraction, and Analysis of Data

Participants performed MAM using their individual LC and MS instrumentation, software platforms, and data processing parameters. A Data Reporting Template was provided to each participant for reporting requested information and data as previously described.[19] Submission of raw data files in addition to the Data Reporting Template was optional for participants. Data Reporting Templates and any raw data files submitted were given to the study organizers through a third party [National Association for Proficiency Testing (NAPT) (Edina, MN)], who first anonymized the reports. Thus, the study organizers performed collation and review of data without knowledge of which participant had submitted which data.

Summed extracted ion chromatogram (XIC) areas were reported for each of the 15 Calibration Peptides for each of the three injections. Total relative abundance values were automatically calculated in the Data Reporting Template as described in the Results and Discussion, eq .

Data acquired from one MS1-only mode injection of the Reference Sample were reported by each participant for 15 NISTmAb peptides. These data included the uncharged monoisotopic mass of each Reference Peptide (calculated from the most abundant charge state observed), the retention time and the summed XIC area. Total relative abundances of the 15 Reference Peptides were calculated by the study organizers from the reported XIC values as described in the Results and Discussion, eq .

The relative abundance of several predefined NISTmAb attributes (heavy chain N-terminal pyro-Glu, C-terminal Lys-loss, Met255 oxidation, Asn387/Asn392 deamidation, and Asn387/Asn392 ammonia loss) were calculated by each participant as described in the Results and Discussion, eq . Relative abundance values for the three most abundant glycopeptide species were calculated by the study organizers using the XIC values reported by participants, as described in the Results and Discussion, eq .

Data and information submitted by participants using the Data Reporting Template were extracted using R statistical software[20] onto a master spreadsheet. Box plot graphs were generated in Excel using the Real Statistics Resource Pack add-in (Release 5.5) (Copyright 2013–2018, Charles Zaiontz, www.real-statistics.com) with an outlier multiplier value of 1.5.

ASTM Standard E691-18 (Standard Practice for Conducting an Interlaboratory Study to Determine the Precision of a Test Method)[21] was used to calculate the interlaboratory repeatability standard deviation (sr) and reproducibility standard deviation (sR). Formulas are described in Supplemental Appendix S1 (Section A).

Reagents

Formulation Buffer: 12.5 mmol/L l-histidine monohydrochloride monohydrate (J.T. Baker no. 2081-06), 12.5 mmol/L l-histidine (J.T. Baker no. 2080-05), pH 6.0. Denaturing Buffer: 6 mol/L guanidine HCl (Sigma no. RDD001), 1 mmol/L ethylenediaminetetraacetic acid (Fluka no. 39692) in 0.1 mol/L Tris, pH 7.8. Digestion Buffer: 1 mol/L urea (Sigma no. U0631) in 0.1 mol/L Tris, pH 7.8. Tris buffer (0.1 mol/L, pH 7.8): 67.5 mmol/L Tris(hydroxymethyl)amino-methane hydrochloride (Sigma no. T5941), 32.5 mmol/L Tris(hydroxymethyl)amino-methane (Sigma no. T6066). dl-Dithiothreitol (DTT) (Pierce no. 20291) and Iodoacetamide (IAM) (Sigma no. A3221) were reconstituted to the noted stock concentrations using LC/MS-grade water (Fisher Chemical no. W6212). Acetic acid (Honeywell Fluka no. 49199). 0.1% formic acid (FA) in water (Fisher Chemical no. LS118-1).

Tryptic Digestion of Reference Sample

NISTmAb 8671 Lot 14HB-D-002 (in Formulation Buffer at 10 μg/μL) was diluted to 1.01 μg/μL with Denaturing Buffer, followed by addition of 500 mmol/L DTT to a final DTT concentration of 5 mmol/L. The sample was reduced for 1 h at 4 °C.

Alkylation was achieved by addition of 500 mmol/L IAM to a final IAM concentration of 10 mmol/L. The sample was alkylated for 1 h, in the dark, at 4 °C, then exchanged to freshly prepared Digestion Buffer using Zeba Spin desalting columns (Thermo Scientific, no. 89894).

Protein concentration was measured using the NanoDrop 2000 spectrophotometer (Thermo Scientific, no. P087). The “concentration blank” used for background subtraction comprised a “sample” prepared in parallel with and in the same manner as the NISTmAb, except that Formulation Buffer was used instead of the 10 μg/μL NISTmAb.

Digestion was performed using lyophilized trypsin (Roche no. 03708985001) resuspended to 1 μg/μL in 0.05 mol/L acetic acid. The enzyme solution was added to the Reference Sample at a trypsin:IgG mass ratio of 1:18. The sample was allowed to digest for 4 h at room temperature and then placed on ice. Trypsin activity was stopped by the addition of a volume fraction of 0.1% formic acid (FA) in water such that the final NISTmAb concentration was brought to 0.25 μg/μL.

The digested Reference Sample was kept on ice while aliquots of 100 μL each were placed into storage vials (Thermo Scientific no. 3741). Samples were stored at −80 °C until shipment on dry ice to participants.

Calibration Sample

The Calibration Sample comprised the commercially available Pierce Peptide Retention Time Calibration Mixture (5.0 pmol/μL) (Thermo Scientific no. 88321) thawed at room temperature and then diluted to 0.5 pmol/μL with a volume fraction of 0.1% FA in water. Aliquots of 20 μL each were placed into storage vials and kept on ice during the process. Vials were stored at −80 °C until shipment on dry ice to participants.

Blanks

Preformulated 0.1% FA in water was distributed into aliquots of 100 μL each and placed into storage vials. Vials were stored at −80 °C until shipment on dry ice to participants.

Disclaimer

Certain commercial equipment, instruments, or materials are identified in this paper to specify the experimental procedure adequately. Such identification is not intended to imply recommendation or endorsement by the National Institute of Standards and Technology, nor is it intended to imply that the materials or equipment identified are necessarily the best available for the purpose. NISTmAb values and information reported herein should not be construed as certified values and do not replace or supersede those presented in any Report of Investigation (ROI) for the Reference Material. NISTmAb users should always refer to the ROI associated with their specific material lot for the most up-to-date values and uncertainty ranges (https://www-s.nist.gov/srmors/view_detail.cfm?srm=8671).

Results and Discussion

Study Design

All materials to be analyzed for the study were prepared by the organizers with the intention of minimizing variation in sample preparation and thereby allow the study to focus on the capabilities of the instrument and data processing software used across the industry for MAM. Participants were supplied with a kit to perform their MAM analyses. The components of the kit utilized for the attribute analytics portion of the study included (1) a Calibration Sample to be run in triplicate for evaluation of instrument performance; and (2) a tryptic digest of a humanized monoclonal antibody reference material NISTmAb RM 8671 Lot 14HB-D-002[22] (the Reference Sample). Additional components of the kit not discussed here were used for evaluating the NPD component of MAM; a full description of those kit components and the results of those analyses were previously described.[19]

To allow this study to emphasize the performance of the state-of-the-art instrumentation and software being used across the industry for MAM, certain components of the study were controlled including the NISTmAb and its digest preparation, the column used, analytical LC gradient, column temperature, and MS acquisition type.[19] Parameters that could not be controlled and thus remained variable included the MS and LC instrumentation, the MS source and MS acquisition parameters, and certain autosampler parameters (e.g., participants held samples at the same temperature while in queue but drawing speed of samples from the vial was not harmonized). The software platforms used to process data as well as quantitation methods used to determine attribute relative abundance were also left to participants’ discretion.

Participants were provided with a Data Reporting Template and instructions for experimental data acquisition as described previously.[19] The collation and statistical treatment of data by the study organizers are described in more detail in the Experimental Section.

Demographic Overview

The demographic overview of the participants contributing to the study as well as the instrumentation and software platforms used were presented in detail previously.[19] Briefly, the study engaged 28 participants representing the biopharmaceutical industry, mass spectrometry instrument vendors, and government agencies. A defined set of analytical results were submitted by each participant, and 16 participants also provided their corresponding raw data files (10.18434/mds2-2497).

The mass spectrometry (MS) instrumentation used for the study included high-resolution quadrupole time-of-flight (QTOF), linear ion trap (LIT)-Orbitrap and quadrupole (Q)-Orbitrap platforms. The software packages used for attribute relative quantitation included vendor specific (i.e., software platforms available from instrument vendors for use with their instruments’ raw data formats; used by 54% of participants) and vendor neutral (available from independent software vendors for use with any raw data format; used by 29% of participants) platforms, with some participants using multiple platforms of a given type (17% of participants) (Supplemental Figure S1). This birds-eye view indicates that MAM attribute analytics are being performed across the industry using the full array of available software platforms, both vendor-specific and vendor-neutral.

Quantitation Approaches

A survey of relative abundance quantitation approaches employed by study participants revealed that when multiple charge states were observed for a given peptide, the most common approach (>80% of all participants) was to include all charge state species in the calculation of that peptide’s abundance (i.e., the XIC peak area) (Supplemental Figure S2). Less than 20% of participants restricted their quantitation to the one or two most abundant charge state species or to a predefined range of charge states (e.g., z = 1 to z = 6).

Participants were also found to utilize differing strategies regarding the isotopes included for calculating abundance values. Almost half of the participants (46%) summed the abundance of all observed isotopes to determine a peptide’s total abundance (Supplemental Figure S3), while 21% included only the monoisotopic peak for quantitation, and 11% considered the single most abundant isotope. The remaining quantitative approaches (22%) involved the inclusion of a defined number of isotopes (e.g., up to isotope x), or the sum of all isotopes, adducts and in-source artifacts (i.e., in-source dehydration) to determine a peptide’s abundance.

Repeatability and Reproducibility of Calibration Sample Total Relative Abundances

System suitability evaluation prior to running MAM analyses may include the use of a simple peptide mixture, similar to the Calibration Sample used in this study, which must meet predefined specifications with regard to retention time, mass accuracy, and peak area. In this study participants provided the retention times and XIC peak areas of each Calibration Peptide, using the individual quantitation methods described in Supplemental Figures S2 and S3, from all three injections and the uncharged monoisotopic mass observed for each peptide from one injection.

Results pertaining to community precision in retention time and mass accuracy measurements of the Calibration Peptides were previously reported and indicated that the intralaboratory retention time repeatability was <0.25 min and mass accuracy for each participating laboratory was <±5 ppm.[19]

Peak area measurements cannot be evaluated between laboratories using absolute abundance values due to the differences in instrumentation used. Instead, the total relative abundance of each Calibration Peptide in relation to all 15 peptides was used as a surrogate. The total relative abundance of each Calibration Peptide was automatically calculated in the Data Reporting Template using the summed XIC area values reported by each participant and averaged over three injections. Total relative abundance for a given peptide was calculated as the abundance of that peptide relative to the total abundance of the 15 peptides:where peptide abundance is the summed XIC area of a given peptide, which was calculated using methods specific to each participant (see Supplemental Figures S2 and S3).

The average total relative abundance of each Calibration Peptide calculated over three injections by each participant was compared via box plot graph (Figure ). The peptides were present in the mix at equimolar concentration and thus each had a theoretical total relative abundance of 6.67%. However, the observed total relative abundances averaged across all participants ranged from 0.94% (peptide LSSEAPALFQFDLK) to 10.92% (peptide GLILVGGYGTR). The measured values reported by each participant were expected to deviate from their theoretical value due in part to the differences in ionization efficiencies inherent to each peptide. In addition, synthetic impurities (e.g., additional or unlabeled residue, truncation) are known to account for a small percentage of the actual abundance of each peptide;[19] therefore, some deviation from the theoretical 6.67% value likely arose due to the exclusion of peptide impurities from the total relative abundance calculation.

Figure 1

Total relative abundance of Calibration Sample peptides. Total relative abundances (RA) were calculated by each participant for each of three injections, and then the average was taken for each peptide. These average relative abundances were used to generate the box plot (see Figure S15). The dashed line at 6.67% represents the theoretical total relative abundance of the 15 peptides which were provided at equimolar concentration. Symbols noting outlier data points are unique for each participant.

Interlaboratory repeatability (sr) and reproducibility (sR) standard deviations of Calibration Peptide total relative abundances were measured according to ASTM standards for interlaboratory studies[21] and are presented in Figure a and Supplemental Table S1 [see Supplemental Appendix S1 (Section A) for statistical equations]. Variability is often evaluated in the industry using the coefficient of variation (CV) rather than, or in addition to, sample standard deviation. Because the sr and sR do not represent the sample standard deviation required for calculating CV (e.g., sr is essentially the average repeatability of the laboratories, while sR incorporates the variability of the means into the sr) traditional CV values cannot be provided here. However, we have calculated “repeatability CV” (CVr) and “reproducibility CV” (CVR) values based on the sr and sR, respectively, for descriptive purposes only [Figure b and Supplemental Table S1; see Supplemental Appendix S1 (Section A), for statistical equations]. Note when reviewing these data that the statistical properties and inferences associated with the standard definition of CV do not apply to the CVr and CVR values presented here and that CVr and CVR are not associated with the ASTM standard.

Figure 2

Total relative abundance variability of Calibration Sample peptides. The total relative abundance values of each calibration sample peptide were reported by participants for three injections. (a) Repeatability (sr) and reproducibility (sR) standard deviations were calculated for each peptide; (b) coefficient of variation (CV) values (expressed as percentages) were calculated based on repeatability (CVr) and reproducibility (CVR) standard deviations. Note that because sr and sR are not sample standard deviations, the statistical properties and inferences associated with the standard definition of CV do not apply to CVr and CVR. Data points are summarized in Supplemental Table S1. Equations for sr, sR, CVr, and CVR are provided in Supplemental Appendix S1 (Section A).

For all peptides, sr values remained below 1% standard deviation, while sR values varied between 1% standard deviation for peptide LSSEAPALFQFDLK and 4.35% for peptide HVLTSIGEK. All but one of the CVr values calculated for relative abundance fell below 15% which is a common acceptance level criterion for precision (albeit, CV acceptance criteria are typically applied using absolute, rather than relative, abundance).[23,24] The exception was peptide LSSEAPALFQFDLK, which had a CVr of 46.26%. CVR values ranged between 16.27% and 47.67%, except for peptides DIPVPKPK and LSSEAPALFQFDLK which had CVR values of 75% and 99%, respectively.

Sources of Variation in Calibration Peptide Total Relative Abundance Values

One possible explanation for the variation in the relative abundance values of the Calibration Peptides is that participants were asked to use their own unique optimal source parameters and thus conditions were not harmonized between instruments. While different peptides will exhibit different ionization efficiencies under the same source conditions, the same peptide will exhibit different ionization efficiencies under different source conditions. For example, differences in source voltages, temperatures or ion optics likely account for some of the increase observed in interlaboratory reproducibility (sR) values (ranging between 1.0% and 4.35% total relative abundance) for each peptide as compared to their corresponding sr values (<1.0%) (Figure a, Supplemental Table S1).

The Calibration Peptide DIPVPKPK was among those with lower reproducibility in total relative abundance values between laboratories (sR = 3.85%). Available raw data were examined to determine the source of total relative abundance variation between participants for this peptide, and the presence of in-source fragments was noted. To evaluate the effect of differential levels of in-source fragmentation on observed total relative abundance, XIC values for DIPVPKPK and its prominent in-source fragments PVPKPK and PKPK were generated from 15 raw data files. Percent in-source degradation was calculated and plotted against the total relative abundance values reported by each participant for DIPVPKPK (Supplemental Figure S4). The results show a general trend of decreasing observed total relative abundance values with increasing in-source fragmentation. In-source fragmentation has also been observed for other peptides in this sample (e.g., HVLTSIGEK, GLILVGGYGTR, IGDYAGIK, LTILEELR, NGFILDGFPR), albeit to a lesser degree than for DIPVPKPK which suggests additional causes for variation in relative abundances are yet to be determined.

The exclusion of abundant charge species was noted as an additional source of variation in total relative abundances for the DIPVPKPK peptide. For 73% of the raw data sets reviewed, the z = 3 ion was either the most predominant charge state detected for the DIPVPKPK peptide or the z = 3 ion had fairly equal abundance to the z = 2 species (data not shown). However, one participant set the lower MS scan range limit at 350 m/z which precluded detection of the z = 3 species (301.1920 m/z). This latter participant was found to be an outlier in Supplemental Figure S4 (noted by the symbol ⬟ in the figure). If following the trendline in this graph, the comparatively low in-source fragmentation calculated for Participant ⬟ (13.52%) is expected to correspond to a calculated total relative abundance of ≈5%, but for this participant the total relative abundance value was reported as 0.34%. The exclusion of the +3 ion from the calculation was the likely cause for this participant reporting a low total relative abundance value despite having a low level of in-source fragmentation.

We were unable to determine the reason for the high CVr and CVR values observed for LSSEAPALFQFDLK but have considered the possibility that the source of variability arises from the properties of the peptide itself rather than with instrument parameters or data processing. For example, this peptide may be what has been described as a “sticky peptide” which is characterized by a steady decrease in peak area when back-to-back injections are made.[25] This phenomenon is presumably due to the affinity of the peptide for the inner surface of the autosampler vial and may be mitigated by sonication of the sample after being placed in the vial or by the addition of guanidine hydrochloride (GnHCl) to the sample to a final GnHCl concentration of 1.85 mol/L.

NISTmAb Reference Peptides

A complex digest representative of the test sample is often used for system suitability testing to evaluate overall instrument performance while taking into account sample preparation. The Reference Sample was evaluated in this light for the current study. Each participant reported the observed masses and retention times for 15 NISTmAb peptides detected in the single MS-only injection of the Reference Sample (see the Experimental Section). Peptides were chosen such that they spanned the full 65 min analytical gradient. Using a box plot to compare retention times revealed that most values reported by Participant ■ were outliers relative to other participants (Supplemental Figure S5).

Reproducibility and repeatability could not be evaluated per the ASTM standard because replicate values were not collected from participants for these Reference Sample data. In lieu of these calculations, the interlaboratory standard deviation (s) values were determined for each reference peptide retention time [Figure , Supplemental Table S2, Appendix S1 (Section B)]. The s values ranged from 1.36 min standard deviation (for NQVVLK) to 1.94 min standard deviation (for ALEWLADIWWDDKK). This was within the same range previously reported for the reproducibility standard deviations of the 15 Calibration Peptides (sR= 1.40 min for SSAAPPPPPR and 2.03 min for NGFILDGFPR).[19] While these retention time differences would be considered rather large for repeated injections on the same instrument, this wider range of values was expected when considering instrument-to-instrument variability. Although the same column, flow rate and gradient were used by all participants, different lengths of tubing within the flow path as well as different pump systems could have caused the several minute differences in retention time observed in these data. If multiple injections were to be made, we would expect that Reference Peptide intralaboratory repeatability would result in standard deviations lower than those measured for interlaboratory reproducibility as was the case for the Calibration Peptides.

Figure 3

Interlaboratory reproducibility of NISTmAb Reference Peptide retention times. The observed retention times of 15 NISTmAb Reference Peptides were reported by participants. The interlaboratory standard deviation (s) in retention time was calculated for each peptide. Data points are summarized in Supplemental Table S2; the equation for s is provided in Supplemental Appendix S1 (Section B).

Mass accuracy values calculated for the NISTmAb reference peptides averaged near ±2 ppm with s values of ≈ ± 2 ppm for each peptide (Figure , Supplemental Table S2), with the same outlier participant (□) averaging close to ±9 ppm across the 15 peptides (Supplemental Figure S6). These mass accuracy values were on par with those reported previously for the Calibration Peptides,[19] where the majority of individual data points are < ± 5 ppm. In addition, these mass accuracy values are within the expected level of performance for the high-resolution mass spectrometry instrumentation used.

Figure 4

Interlaboratory evaluation of NISTmAb Reference Peptide mass accuracy. The observed mass of each NISTmAb Reference Peptide was reported by participants for one injection. Absolute ppm values were calculated from the observed and theoretical masses of each peptide. The interlaboratory average |ppm| value (x̿) for each peptide is noted by the “X”, with error bars indicating the interlaboratory standard deviation (s). Data points are summarized in Supplemental Table S2; equations for x̿ and s are provided in Supplemental Appendix S1 (Section B).

Notable mass accuracy excursions were values of ±79 ppm and ±26 ppm reported by Participant ⊕ for light chain peptide SGTASVVCLLNNFYPR and heavy chain peptide ESGPALVKPTQTLTLTCTFSGFSLSTAGMSVGWIR, respectively. These aberrant data were reported by the participant as due to temporary interference with the internal reference mass during data acquisition.

Absolute peak areas of the Reference Peptides cannot be compared directly between different instruments because of the inherent differences in intensity values, thus total relative abundances of the 15 peptides were used as surrogate data. Summed XIC values (peak areas) reported for the Reference Peptides were used by the study organizers to calculate the relative abundance of each peptide in relation to the total abundance of all 15 peptides:where peptide abundance is the summed XIC area of a given peptide, which was calculated using methods specific to each participant (see Supplemental Figures S2 and S3).

Average total relative abundance values ranged from 3.17% (peptide NQVVLK) to 11.80% (peptide TPEVTCVVVDVSHEDPEVK) (Supplemental Figure S7, Supplemental Table S2). Interlaboratory standard deviation values (s) ranged from 0.98% (peptide DSTYSLSSTLTLSK) to 3.34% (WQQGNVFSCSVMHEALHNHYTQK) (Figure a, Supplemental Table S2). These values showed a similar level of variability as observed for the Calibration Peptides (sR = 0.92% to sR = 4.60%; Figure a), albeit using slightly different measurements of variability (e.g., the reproducibility standard deviation also takes the repeatability standard deviation into account). Interlaboratory CV values (CV) for the Calibration Peptides ranged from 15.55% (peptide DSTYSLSSTLTLSK) to 73.17% (FSGSGSGTEFTLTISSLQPDDFATYYCFQGSGYPFTFGGGTK) (Figure (b), Supplemental Table S2), with 11 of these falling within the same range as observed for the majority of Calibration Peptide CVR values (Figure b, Supplemental Table S2).

Figure 5

Total relative abundance variability of NISTmAb Reference Peptides. The observed peak areas of NISTmAb reference peptides were reported by participants and used to calculate the total relative abundance of 15 peptides. (a) Interlaboratory standard deviation (s) in total relative abundance and (b) interlaboratory coefficient of variation (CV) values were calculated for each peptide. Data points are summarized in Supplemental Table S2; equations for s and CV values are provided in Supplemental Appendix S1 (Section B).

Like the Calibration Peptides, differences in ionization efficiencies between instrument sources are one likely source of variation in total relative abundance values between participants for the Reference Peptides. The complexity of the Reference Sample may add to the level of variability in ionization efficiencies, since coeluting peptides may differentially affect the ionization of a given peptide under different source conditions.

Deviation of relative abundance values from the theoretical value of 6.67% may also in part be reflective of digestion efficiency. Assuming no sequence variants, each string of amino acids that generated the Reference Peptides upon enzymatic digestion is present in the intact molecule at equal molar ratio. If digestion were 100% efficient, the theoretical total relative abundance of each peptide would be 6.67% when only the 15 peptides are considered in the calculation. However, inevitable missed cleavages, nontryptic cleavages, and even nonenzymatic clips are likely to occur on each peptide at a different rate and will contribute to differences in total relative abundance values between the peptides. In addition, the Reference Peptides may each be subject to low levels of modification, resulting in different total relative abundances for each unmodified tryptic peptide.

Raw data were available for a few outlier participants for investigation as to possible additional sources of variation in relative abundance. Peak area data reported by Participant ◪ generated a total relative abundance of 0.002% for peptide SGTASVVCLLNNFYPR, although the interlaboratory average was 9.46% total relative abundance (Supplemental Figure S7). This participant reported that only the z = 4 charge species was included in the summed XIC value. However, review of the raw data showed the presence of doubly and triply charged ions, which were the two most abundant species, as well as a low abundant singly charged species. Signal from the highly abundant charge states (z = 2,3) appear to have saturated the detector and suffered a loss of mass accuracy (e.g., ≈ 80 ppm) which may have prevented these charge states from being included in the total XIC value reported. Recalculation of the total relative abundance of this peptide to include all detected charge states (z = 1,2,3,4) brought the total relative abundance to 9.48%.

Similarly, Participant ◇ was an outlier for peptide TPEVTCVVVDVSHEDPEVK (Supplemental Figure S7) due to the exclusion of the most abundant z = 3 charge state when calculating total peak area. Without this highly abundant species the total relative abundance value for the peptide was 0.36%, while the interlaboratory average was 11.80%. However, including all three observed charge states (z = 2,3,4) in the peak area calculation increased the total relative abundance of this peptide to 15.63% for this participant. Evaluation of the raw data did not reveal why the z = 3 charge state may have been excluded. Without knowledge of the software processing steps that generated the final results from the raw data it is not possible to make conjectures as to potential causes.

Participant Δ had an outlier value of 1.50% total relative abundance for peptide TVAAPSVFIFPPSDEQLK, while the interlaboratory average was 9.95% (Supplemental Figure S7). Review of the raw data did not shed any light on a potential source for this outlier. Extraction of peak area values from this raw data set by the organizers and subsequent recalculation of total relative abundance values for this participant produced a total relative abundance of 10.98%. Similarly, evaluation of outlier values for Participant □ (for peptide VTITCSASSR) and Participant ◒ (for peptide ALEWLADIWWDDKK) (Supplemental Figure S7) did not provide insight into the reason for their anomalous values.

NISTmAb Attribute Analytics: Overview

The monitoring of quality attributes is one of the two functions of MAM. Here, five attributes were chosen for “monitoring” by study participants, with the abundance of each attribute calculated relative to the unmodified counterpart. The interlaboratory reproducibility of the attribute relative abundance quantitation of these NISTmAb attributes were evaluated: heavy chain N-terminus pyro-Glu (cyclization of glutamine at peptide QVTLR), heavy chain C-terminus Lys-loss (peptide SLSLSPGK), oxidation of heavy chain Met255 (peptide DTLMISR), deamidation and ammonia-loss (succinimide intermediate) of the PENNY peptide (GFYPSDIAVEWESNGQPENNYK), and N-glycosylation at heavy chain Asn 300 (peptide EEQYNSTYR). Attribute relative abundance was quantified according to the following equation (with the exception of glycopeptide attribute relative abundance, which is described in eq ):where peptide abundance is the summed XIC area of a given peptide, which was calculated using methods specific to each participant (see Supplemental Figures S2 and S3) and may be the sum of multiple species containing the residue of interest (e.g., no missed cleavage peptide plus one missed cleavage peptide, parent peptide plus in-source event, peptide of interest with modification other than attribute of interest).

NISTmAb Attribute Analytics: Pyro-Glu

The interlaboratory average attribute relative abundance (x̿) [see Supplemental Appendix S1 (Section B)] of pyro-Glu was 99.42%, with the quantitation of this modification being highly reproducible between laboratories (s = 1.51%,CV = 1.52%) (Figure , Supplemental Figure S8 and Table S3). One participant (▲) did not detect the modified or the unmodified N-terminus of the NISTmAb heavy chain while another participant (□) reported an outlier attribute relative abundance value of 92.0% pyro-Glu (Figure a).

Figure 6

Interlaboratory evaluation of NISTmAb Reference Peptide relative abundance. The relative abundance (RA) of each monitored attribute in the NISTmAb Reference digest was reported by each participant for one injection. The interlaboratory average relative abundance value (x̿) for each attribute is noted by an “X”, with error bars indicating the interlaboratory standard deviation (s). Data points are summarized in Supplemental Table S3; equations for x̿ and s are provided in Supplemental Appendix S1 (Section B). Note that error bar ranges for EEQYNSTYR+A2G2F, DTLMISR and GFYPSDIAVEWESNGQPENNYK are smaller than the boundaries of the “X” symbol marking the average.

Figure 7

Quantitation of NISTmAb attributes. For (a–d) the relative abundance (RA) of each modification was calculated as the ratio of the peak intensity of the modified peptide to the sum of peak intensities of modified and unmodified peptides (see Supplemental Figures S9–S12, respectively, for peptides included by each participant). For (e) the RA of glycopeptides was calculated as the ratio of the individual glycopeptide species to the sum of the three most abundant glycan species found on heavy chain N300 (see Supplemental Table S4 and Figure S13 for glycopeptides included in the calculation).

The outlier participant (□) reported using the z = 2 species of both the modified and unmodified QVTLR peptide to calculate the attribute relative abundance of pyro-Glu in the sample (Supplemental Figure S9). A review of the accompanying raw data shows the z = 1 charge state of the modified peptide to be ≈20 times more abundant than the z = 2 charge state, while the more abundant charge state of the unmodified peptide was z = 2. The reported attribute relative abundance value was thus considerably lower than other participants’ values because the highly abundant z = 1 modified species was excluded from the computation. Recalculation of the attribute relative abundance of pyro-Glu for this participant to include both the z = 1 and z = 2 species, resulted in an attribute relative abundance value of 99.59%, consistent with that of the other participants.

Five participants (⊡, ☼, ◇, ⬟, and ⬠) detected the pyro-Glu form of the heavy chain N-terminal peptide but not the low abundant, unmodified QVTLR peptide (Supplemental Figure S9). Three raw data sets were available for examination, and for one participant (⬟), the data indicated that the unmodified peptide may have gone undetected due to the m/z scan range used for data acquisition. As described above, the unmodified species was more abundant in its doubly charged state, with an m/z of 308.6930, but this participant acquired MS data within a 350 m/z to 2000 m/z window. For Participant ⬠, z = 1 and z = 2 of the unmodified peptide were found in the raw data, but due to their low abundance may have been below the peak detection threshold set by this participant. Attribute relative abundance of pyro-Glu as calculated from this participant’s raw data was 99.62%. For Participant ◇, the unmodified peptide was not detected in the raw data as reported by the participant.

Using the recalculated pyro-Glu relative abundance value for Participant □ to determine interlaboratory variation brings the s value down to 0.30% (Supplemental Figure S8a).

NISTmAb Attribute Analytics: Lys-loss

Greater interlaboratory variability (s = 5.95%,CV = 6.60%) was observed in the values reported for the quantitation of heavy chain C-terminal Lys-loss as compared to pyro-Glu quantitation (Supplemental Figure S8, Supplemental Table S3), with the attribute relative abundance value averaging 90.2% (Figure , Figure b).

The unmodified SLSLSPGK peptide has previously been described as having greater abundance in the z = 2 charge state, while it is the z = 1 charge state that is more abundant for the Lys-loss SLSLSPG species during electrospray ionization.[26] The charge states included in the quantitation of the Lys-loss attribute relative abundance varied across participants (Supplemental Figure S10). All but two participants reported including the most abundant z = 1 Lys-loss species and z = 2 unmodified species, at a minimum, for their calculations. The exceptions to this trend were as follows: (1) the participant with the highest reported attribute relative abundance value (99.8% for ◨; note this participant was not an outlier), who used both charge states for the Lys-loss species and only the lower abundant z = 1 unmodified species for the calculation, and (2) the outlier participant (⬒) who used only the low abundant z = 2 Lys-loss species and the highly abundant z = 2 unmodified peptide to generate an attribute relative abundance value of 71.0%. Inclusion of the singly charged Lys-loss peptide for this outlier raised the attribute relative abundance value to 87.4%.

Recalculation of the interlaboratory standard deviation using relative abundance values for the Lys-loss SLSLSPG peptide that include all observed charge states does not lower the new s value to the same level as that of the other attributes. While the original value stands at s = 5.95%, the recalculated value only drops to 4.65% and the recalculated CV from 6.60% to 5.13% (Supplemental Figure S8). Thus, it is possible that not all sources of variation between participants have been identified for this attribute. The Lys-loss form of the peptide is likely to have lower ionization efficiency than the unmodified peptide due to the loss of the lysine residue.[26] Thus, the loss of ionization efficiency may be more dramatic under some source conditions than others, leading to greater variation in this peptide’s relative abundance attribute than in others.

NISTmAb Attribute Analytics: Met Oxidation

The average attribute relative abundance value reported for the oxidized DTLMISR peptide was 1.2%, with variation measured between laboratories as s = 0.48%,CV = 41.25% (Figure , Supplemental Figure S8, Supplemental Table S3) and two outlier values observed (Figure c). One outlier value (for Participant ◨) was attributed to the lack of detection of the oxidized species, though raw data were not available to allow further investigation. The other outlier value (for Participant ○) was higher than average at 2.4% attribute relative abundance of Met oxidation. Participant ○ reported the inclusion of only the unmodified DTLMISR peptide and its Met oxidized species in the attribute relative abundance calculation (Supplemental Figure S11). However, review of the raw data for this participant showed that ≈36% of the unmodified peptide underwent in-source fragmentation and ≈6% was subject to in-source dehydration. If the abundances of these in-source events are included when calculating the abundance of the unmodified peptide, the attribute relative abundance of DTLMISR Met oxidation is 1.4% for this participant. Using the recalculated DTLMISR oxidation relative abundance value for Participant ○ to determine interlaboratory variation brings the s value to 0.36% and CV to 30.94% (Supplemental Figure S8). The identification of this outlier highlights the vulnerability of this particular peptide to in-source degradation. Differing source conditions likely induced varying levels of in-source degradation for this peptide across participants and contributed to the higher CV value observed.

Some differences were observed in the peptide species that were included by participants when calculating DTLMISR oxidation. For example, many participants only included the unmodified and oxidized DTLMISR peptide when calculating the attribute relative abundance of Met oxidation. However, three participants included the missed cleavage peptide DTLMISRTPEVTCVVVDVSHEDPEVK when calculating the total abundance of the heavy chain M255 residue (Supplemental Figure S11). No effect on attribute relative abundance variation was observed due to these types of different species being included in (or excluded from) the calculation (data not shown). This lack of effect may be due to the low level at which these species exist in the sample, or that the number of data sets was too small to observe an effect.

NISTmAb Attribute Analytics: Asn Modification

Quantitation of the modification of heavy chain Asn387 and Asn392 residues (residing within the PENNY peptide, which spans heavy chain residues 374–395) was perhaps the most complex of this data set since both deamidation and ammonia-loss may be induced at these residues and produce isomeric species with varied retention times.

Participants were asked to report the attribute relative abundance of deamidation in the NISTmAb Reference Digest as a sum of all deamidation species that elute after the main unmodified peak and to calculate ammonia-loss using the sum of all species detected. All but two participants detected the first deamidation peak that elutes after the unmodified GFYPSDIAVEWESNGQPENNYK peptide (representing deamidation at the Asn392 residue), and nine participants also reported a second deamidation peak with a later elution time (representing deamidation at the Asn387 residue) (Supplemental Figure S12). All participants reported the most abundant ammonia-loss species at the Asn387 residue. Deamidation and ammonia-loss relative abundances averaged 1.8% and 2.0%, respectively (Figure , Supplemental Table S3). Interlaboratory variation was measured at s = 0.81% and CV = 45.47% for PENNY peptide deamidation and s = 0.73%,CV = 37.01 for ammonia-loss (Supplemental Figure S8), with no outlier values (Figure d).

The complexities of asparagine modifications may have contributed to relative abundance variability between laboratories due to differences in peak resolution and data analysis. For example, the monoisotopic mass of a deamidated peptide is within <8 ppm of the 13C1 isotope of the unmodified species. Accurate quantitation is thus made more difficult when deamidated and unmodified species are not well-resolved. Because different LC systems were used in this study, differences in resolution of the deamidated PENNY peptide from the unmodified peptide were a likely factor in the variability of relative abundance values measured.

NISTmAb Attribute Analytics: Glycosylation

Participants were asked to provide a list of all glycopeptides detected in the NISTmAb Reference Sample with an attribute relative abundance >0.1% of the total glycopeptide abundance. The majority of glycopeptide species reported represented glycans conjugated to the fully tryptic EEQYNSTYR peptide, while a few were observed at the Asn300 residue on peptides with up to three missed cleavages (Supplemental Figure S13). In addition to the aglycosylated peptide, a total of thirty-two different glycan species and forty-one different glycopeptide species (i.e., the same glycan species was sometimes identified on various missed cleavage species) were reported by the participants (Supplemental Table S4). Individually, participants reported between four and 25 different glycosylated (and aglycosylated) species, with an average of 11 glycopeptides (±6 glycopeptides) reported across participants (data not shown).

The wide range in the number the glycan species detected across participants may have resulted from their low abundance, the lack of harmonization of source conditions used for data acquisition or the unique challenges posed by positive ion MS analysis of glycopeptides. For example, the presence of a glycan moiety lowers the ionization efficiency of a glycopeptide as compared to aglycosylated peptides.[27] In addition, glycosidic linkages are labile and easily subject to in-source fragmentation[28] which artificially generates additional glycan species. Finally, glycopeptides are highly susceptible to the formation of metal adducts (e.g., sodiation) which can heavily partition the signal between the desired protonated species and the adducted species. Indeed, review of the available raw data shows that the level of metal adduction for the A2G0F glycopeptide ranged from 21% of the total glycopeptide signal to 1%.

The phenomena described above contribute to lowering the signal of the parent glycopeptide (i.e., the nonfragmented, nonadducted glycopeptide). For those species already low in abundance [i.e., near the limit of detection (LOD)], the decreased ionization efficiency and formation of in-source fragments or metal adducts will easily place the parent species below the LOD. Overall, ionization efficiency and the degree to which fragmentation and adduction occurs differs between source settings (e.g., different temperatures and voltages)[29,30] and likely impacted the ability to detect these species reliably.

All participants reported detection of the two most abundant glycopeptides (EEQYNSTYR+A2G0F, EEQYNSTYR+A2G1F), but only 93% of participants also reported the third most abundant EEQYNSTYR+A2G2F species (see Supplemental Table S4 for full list of detected glycopeptides and glycan compositions). The aglycosylated peptide was detected by 57% of participants, while the M5 glycopeptide was reported by 54%. Why Participants □ and ⬠ did not report the A2G2F species is not clear, as review of their raw data showed the presence of the A2G2F species detected at ≈8.5% relative abundance. Again, the software processing steps taken by these participants which ultimately produced their reported results were not available for review.

Because the number of detected glycopeptides varied widely across participants, the evaluation of attribute relative abundances was performed by considering only the top three most abundant glycopeptides. Here, attribute relative abundance was calculated as follows:where glycopeptide abundance is the summed XIC for each A2GXF species, which was calculated using methods specific to each participant (see Supplemental Figures S2 and S3) and included any missed cleavage glycopeptides detected.

The attribute relative abundance values of the A2G0F and A2G1F glycopeptides were the most variable between laboratories as compared to all other attribute relative abundances measured, with s = 8.1%,CV = 17.8% for A2G0F and s = 7.9%,CV = 17.9% for A2G1F (Supplemental Figure S8, Supplemental Table S3). Less interlaboratory variation was observed for the A2G2F species with s = 1.2%,CV = 11.6% (Supplemental Figure S8, Supplemental Table S3). One major outlier participant (◨) was noted as the main contributing factor to the high standard deviations observed for A2G0F and A2G1F, with A2G0F attribute relative abundance considerably higher than other participants’ values (80%, as compared to the average of 45%) and A2G1F attribute relative abundance considerably lower (8%, as compared to the average of 44%) (Figure , Figure e). This pattern was attributed to in-source loss of galactose from the A2G1F species, converting much of it to A2G0F, though raw data were not available for confirmation.

Two additional participant outliers (⌀ and ◇) reported attribute relative abundance values that were below the average for A2G0F (33.8% and 35.2%, respectively) and above the average for A2G1F (54.0% and 51.6%, respectively) (Figure e). This pattern was opposite to that of the high A2G0F and low A2G1F values that resulted from in-source fragmentation of the glycan. Rather, these outlier values resulted from the exclusion of some charge states from the attribute relative abundance calculation. For Participant ⌀ only the z = 3 charge state was included in the attribute relative abundance calculation for the A2G0F glycopeptide, while both z = 2 and z = 3 were included for the A2G1F and A2G2F species. If all detected charge states are used for the calculations, the attribute relative abundances of A2G0F, A2G1F and A2G2F are 44.6%, 45.2% and 10.3%, respectively for this participant. For Participant ◇, attribute relative abundances included the z = 3 charge state for the A2G0F and A2G2F glycopeptides, and the z = 3 and z = 4 species for A2G1F. Using the available raw data to recalculate these values with both charge states for all glycopeptides brought the attribute relative abundances of A2G0F, A2G1F, and A2G2F to 44.5%, 46.2%, and 9.3%, respectively.

Glycan relative abundance variation was determined using the recalculated values for Participants ⌀ and ◇ and by removing the in-source fragment outlier (◨) and the outliers for whom no raw data were available (⧖, ◎). Removing the outliers lowered s to 1.9% and 1.5% for A2G0F and A2G1F, respectively (Supplemental Figure S8, Supplemental Table S3). For A2G2F, the recalculated s = 0.82%.CV were reduced to <10% for all species.

Considerations for System Suitability Testing

The data gathered from the Calibration and Reference Peptides for retention time, mass accuracy and attribute relative abundance provide a survey of instrument performance across the industry. Reproducibility metrics for retention time, peptide relative abundance and mass accuracy were similar between the Calibration Peptide and the Reference Peptide data. Thus, for the parameters measured, either metric seemed appropriate for evaluating instrument performance pertaining to retention time, mass accuracy and peak quantitation. The lower levels of reproducibility observed for some attribute relative abundances, however, indicated the importance of utilizing system suitability tests that specifically evaluate the system with regard to measurement of the attributes to be monitored. For example, the source conditions for Participant ◨ appear to have caused higher-than-average in-source loss of galactose from the A2G1F glycopeptide (Figure e), yet the peptide relative abundance for the fragile DIPVPKPK peptide was 8.79% for this participant, indicating lower-than-average in-source degradation (per extrapolation of Supplemental Figure S4). Thus, the behavior of one peptide (e.g., an arbitrary Reference Peptide) in the MS source may not be predictive of the behavior of a different peptide (e.g., the attribute peptides). For this reason, incorporation of attributes of interest into the system suitability test when performing MAM with targeted attribute analytics is recommended. An attribute-specific system suitability test would not only include specifications for standard LC-MS parameters (e.g., retention time, mass accuracy and peak area), but also for the relative abundance of the attributes of interest. The attribute specificity would necessitate the use of a product-specific digest for system suitability testing.[31] A product-specific digest prepared in parallel with the test digest is optimal since this also informs on the suitability of the sample preparation itself.

Considerations for Improving Reproducibility

The data presented here indicate that across the community, the interlaboratory reproducibility of many attributes can be expected at <2% standard deviation in relative abundance (Supplemental Figure S8). Although multiple injections from each laboratory were not made, it is expected that repeatability within laboratories would show even less variability. Some individual attributes, such as the Lys-loss observed in this data set, may be inherently subject to higher levels of variation and thus require special consideration for specification setting (e.g., a larger range allowed to pass specification).

The sources of variation in peptide and attribute relative abundance followed the same theme for both Calibration and Reference Samples: in-source fragmentation and exclusion of abundant charge states. Participants were also surveyed as to the isotopes included in the relative abundance calculations. No relationship was seen between the isotope quantitation approach used and the resulting data (not shown), although the number of participants using each approach was not enough for a full evaluation.

There is no source parameter that will be optimal for every attribute which will be measured in a given MAM sample. However, care should be taken when developing the MS method to evaluate the behavior of the attributes of interest in the source to determine settings that are best suited for most peptides of interest. The detection of glycopeptides should be heavily considered when determining source conditions to reduce in-source fragmentation and metal adducts, and ultimately provide optimal signal from these low-abundant species. The ability to reproducibly detect and quantify low-abundant glycopeptide species is especially important for MAM if it is to be a viable replacement for orthogonal glycan analytical methods.

The exclusion of abundant charge states from relative abundance calculations had two sources. In the case of Participant ⬟, the abundant doubly charged species of the unmodified QVTLR peptide went undetected due to the m/z scan range set for that participant’s MS parameters. For other participants, abundant charge states were detected for the attributes of interest but were not included in the relative abundance calculation. These two sources of error may be remedied by setting software parameters to include all detected charge states when performing MAM attribute analytics and ensuring that the scan range does not exclude attributes when possible.

Comparison to Orthogonal Methods

The comparison of attribute relative abundances derived via MAM to those determined using alternate methods was not within the scope of data requested from study participants. However, the relative abundance of NISTmAb glycan and Lys-loss species have previously been reported and can be compared to the data collected here.

In one orthogonal method (typical of that used for process support), NISTmAb released glycans were labeled with 2-AB followed by hydrophilic interaction liquid chromatography (HILIC) separation and detection by fluorescence (FLD).[32] A dextran ladder was used for identification of each glycan species. From the reported data we calculated the intralaboratory average relative abundance of the A2G0F, A2G1F, and A2G2F species to be 46.5%, 44.6%, and 8.8%, respectively (Figure a; Supplemental Appendix S2). Note that one isomer of the 2-AB labeled A2G1F species coeluted with the A4F glycan, and the 2-AB labeled A2G2F species coeluted with the M5A1G1 species. Because glycan detection was performed using fluorescence, the unresolved glycans are included in the relative abundance values, however due to their low relative abundances they likely have only a minimal effect on the values.

Figure 8

Comparison of orthogonal methods for measuring relative abundance. (a) Glycopeptide relative abundance (RA) values derived from MAM are compared to glycan and glycopeptide RA values reported by Prien et al. (ref (32))* and De Leoz et al. (ref (33)).† MAM = interlaboratory average RA of the top three glycopeptides as reported by participants or with outliers recalculated from raw data. 2-AB = intralaboratory average RA of glycans released by peptide N-glycosidase F (PNGase F), labeled with 2-aminobenzamide, and analyzed by HILIC-FLD (as calculated from Prien et al.). 2-AA = RA of a single analysis of glycans released by peptide N-glycosidase F (PNGase F), labeled with 2-aminobenzoic acid, and analyzed by HILIC-FLD-MS (as calculated from Prien et al.). Multimethod = interlaboratory median RA values as measured from various glycan forms (i.e., released glycans, glycopeptides, intact molecule, etc.) using multiple analytical methods (as calculated from De Leoz et al.). (b) Lys-loss relative abundance (RA) values derived from MAM are compared to those calculated from various methods reported by Michels et al. (ref (34)).‡ MAM = interlaboratory average RA of Lys-loss as reported by participants or with outliers recalculated from raw data. CEX-HPLC = cation exchange-high performance liquid chromatography, CZE = capillary zone electrophoresis, cIEF = capillary isoelectric focusing, ICIEF = imaged capillary isoelectric focusing. See Table S3 for summarized values. See Supplemental Appendix S1 (Section B) and Supplemental Appendix S2 for quantitative and statistical equations. *Adapted with permission from ref (32). Copyright 2015 American Chemical Society. †Adapted with permission from ref (33). under the Creative Commons Attribution License CC BY (https://creativecommons.org/licenses/by/4.0/). Copyright 2020 NIST. ‡Adapted with permission from ref (34). Copyright 2015 American Chemical Society.

Released glycan relative abundance was also measured previously using 2-AA labeling followed by HILIC separation and detection by FLD and MS (typical for use in product characterization).[32] This method resulted in A2G0F, A2G1F and A2G2F relative abundance values of 44.5%, 45.0% and 10.5%, respectively (Figure a, Supplemental Appendix S2), for a single analysis. Here, relative abundance values for A2G1F and A2G2F include isobaric species as noted in the Experimental Section.

A large interlaboratory study focused exclusively on NISTmAb glycans collated quantitative data from multiple analytical methods (e.g., various labeling strategies for released glycans, glycopeptide analysis, intact mAb measurement).[33] The median interlaboratory relative abundance values for the A2G0F, A2G1F, and A2G2F species measured in this study were 45.5%, 44.6%, and 9.9%, respectively (Figure a, Supplemental Appendix S2).

The relative abundance values for the three most abundant glycan species reported using LC-MS analysis of glycopeptides in the interlaboratory MAM study were in good agreement with those derived from the orthogonal 2-AB, 2-AA methods and varied methods (Figure a). Values from the 2-AA and varied methods were aligned with the interlaboratory MAM averages. While the A2G1F value from the 2-AB method was also similar to that of the MAM method, the A2G0F was just above the upper quartile value and the A2G2F value was just below the lower quartile.

The MAM-derived s values [as recalculated to include all charge states for Participants ⌀ and ◇, and removing the in-source fragment outlier (Participant ◨)] were found to be somewhat higher than the intralaboratory standard deviation values reported for the 2-AB labeled glycans (e.g., s = 1.87 for MAM A2G0F and s = 0.31 for 2-AB A2G0F) (Supplemental Figure S14a). This difference was not surprising given that the MAM data were generated from interlaboratory results while the 2-AB data represent intralaboratory values. A comparison of the variation between the recalculated MAM data and the multiple methods used in ref (33) cannot truly be made because of the differences in statistical methods used for each study. A loose comparison was made, however, using the available measurements. Here, the recalculated MAM variability as determined by s (e.g., s = 1.87 for A2G0F) was lower than that of the multimethod data set as calculated using the median absolute deviation (e.g., MADE = 2.88 for A2G0F). Variability cannot be compared for the 2-AA values because replicate data were not collected.

CV values were below 20% for the MAM reported glycopeptide relative abundances and were less than 10% after recalculation of the outliers (Supplemental Figure S14b). Variability observed for the recalculated MAM values were well-aligned with those reported for the orthogonal methods (Supplemental Figure S14a,b).

A number of charge variant methods have been applied to the NISTmAb for quantitation of Lys-loss from the heavy chain C-terminus: CEX-high performance liquid chromatography (HPLC), CZE, cIEF and imaged capillary isoelectric focusing (ICIEF).[34] Each of these methods separates the main species in the sample from acid and basic species, followed by UV detection. The lysine-containing species are found within the basic peak along with species containing other modifications that render them more basic than the main peak. In the referenced study, lysine-containing antibody was quantified by measuring the basic peak before and after treatment with carboxypeptidase B, which removes the terminal lysine from the protein. The Lys-loss relative abundances as calculated from the CEX-HPLC, CZE, cIEF and ICIEF data were 91.9%, 90.8%, 91.2%, and 91.3%,[34] respectively, just above the MAM average of 90.2% (Figure b).

A comparison of the variability of Lys-loss measurements between MAM and orthogonal methods cannot be made here since replicate values were not acquired for the charge-based assays.

Conclusions

The study presented here represents the first industry-wide performance metric of the targeted attribute analytics component of MAM. Targeted analytics as performed under the MAM workflow leverages peptide mapping mass spectrometry which has been widely used across the industry for decades in the characterization space. Implementation of MAM as a QC assay would require method validation, among other considerations.[6] In practice, method validation includes evaluation of reproducibility across multiple sites belonging to the same company utilizing the same instrumentation and same instrument parameters. Because this interlaboratory study called upon participants from different companies, the instruments and parameters could not be harmonized. Thus, our study is not intended as a method validation, but rather to provide a metric of the state of performance of MAM across the industry. By making these results available we hope to engage discussion geared toward further enabling MAM implementation in QC.

The reproducibility of retention time, mass accuracy and relative abundance values measured in this study using Calibration Peptides was in good agreement with those data generated from the Reference Peptides. Several outlier data points stemmed from easily remedied causes. Optimization of source conditions and the inclusion of all observed charge states when calculating attribute relative abundances improved quantitation accuracy in the outlier examples that could be examined. The presence of these outliers highlights the need to not only carefully optimize data acquisition and software processing parameters but also to verify that the software processing workflow performs as expected (e.g., if two charge states are present in the raw data, is the processing software using both of them to quantify the desired attribute?). Once software performance is established through workflow development, the use of system suitability testing that incorporates product- and attribute- specific metrics will ensure the continued performance of data acquisition and software parameters and the overall success of attribute analytics platforms. Performance metrics should include both identification and consistent quantitation of the attributes of interest.

Most of the attributes measured in this study showed good agreement between the data reported across laboratories and would likely improve with harmonized instruments and acquisition parameters. Comparison of the interlaboratory data with orthogonal methods for Lys-loss are a hopeful indication of the readiness of the MAM platform to be considered as a replacement for some conventional QC methods. Previous intralaboratory studies have demonstrated comparability of glycan abundance measurements between the MAM approach and orthogonal methods.[7,35] Our retrospective comparison of the relative abundances of the top three most abundant glycans also showed good agreement with orthogonal methods. The variation in the total number of glycans detected by each participant may be reflective of the diversity of instruments and source parameters used for the study and exemplifies the need to consider limits of detection when MAM is to be considered as a replacement for traditional released glycan measurements. Studies specifically dedicated to evaluating the limits of detection of glycopeptides and optimizing parameters for reproducible detection and measurement would aid in this endeavor.

This study has provided a unique overview of the industry-wide performance capabilities of MAM. The results demonstrate that quantitative reproducibility and orthogonality can be achieved once appropriate controls and optimized parameters are employed. As with any method, continuous improvements brought to MAM will increase its prevalence of use and facilitate broader adoption into therapeutic protein development and QC pipelines.