Update on Standard Operating Procedures in Preclinical Research for DMD and SMA Report of TREAT-NMD Alliance Workshop, Schiphol Airport, 26 April 2015, The Netherlands.

A workshop took place in 2015 to follow up TREAT-NMD activities dedicated to improving quality in the preclinical phase of drug development for neuromuscular diseases. In particular, this workshop adressed necessary future steps regarding common standard experimental protocols and the issue of improving the translatability of preclinical efficacy studies.

BACKGROUND

Since 2007, TREAT-NMD has played a major role in accelerating research and moving more effective treatments towards the clinic for neuromuscular diseases. At the preclinical level, TREAT-NMD previously developed several standardized operating procedures (SOPs), in collaboration with the Wellstone Muscular Dystrophy Cooperative Research Centers and leading scientists, to improve reproducibility of research data [1, 2]. Several recommended guidelines were published to facilitate best practice in preclinical work with the mdx mouse model of Duchenne muscular dystrophy (DMD) [3-5]. Following this, a dedicated committee for expert evaluation of preclinical approaches and clinical plans (TREAT-NMD Advisory Committee for Therapeutics; TACT), established to ensure translation to clinical trials, was set up as an independent service for academic researchers, clinicians, patient groups and industry. At this stage, a specific workshop was required to (1) discuss the need for additional SOPs for preclinical research in relation to outcome measures used in clinical trials as well as the natural advancement of methodologies; (2) establish rules for the external submission and acceptance of SOPs; and (3) improve awareness concerning the importance and relevance of preclinical study quality for neuromuscular diseases. A small focus group of 11 participants met to discuss these issues and the main outcomes are presented herein. As more than 2 years have elapsed since this Workshop, some significant actions that have already resulted are also included in this report.

PARTICIPANT PRESENTATIONS

Raffaella Willmann welcomed participants and apologised on behalf of Annamaria De Luca and Markus Rüegg, who were unable to attend due to short-term commitments but were actively involved in the preparation of the workshop while ensuring their input in the follow up.

CLINICAL OUTCOME MEASURES

Anna Mayhew presented current clinician rated outcomes used in clinical trials. In ambulant DMD patients, the 6-minute-walk-test (6MWT) is still the most widely used primary endpoint [6]. Secondary outcomes include: myometry as a measure of muscle strength, North Star Ambulatory Assessment (NSAA) as a measure of motor performance [7, 8], timed tests such as rise from floor, 10 metre walk/run, climb and descend four stairs and respiratory measures such as forced vital capacity (FVC). Efforts have been made in the UK and Italy to correlate the total score on the NSAA with patient pathology and there is intent to do this in other databases too [9]. One of the challenges is to understand the nonlinear relationship between strength and function. For non-ambulatory DMD patients there is an increasing focus on assessing upper limb function. The Performance of Upper Limb scale (PUL 1.2 and PUL 2.0) has been developed to specifically assess this and is under further development [10]. Patient Reported Outcome Measures (PROMs) are also becoming increasingly important to regulatory authorities as they assess the impact of the disease on home and family life. A specific module has been designed for assessing arm function in DMD (PROM-Upper for DMD [11]). Quality controls of the assessments are carried out to ensure reliability across different centres. It is evident that reliable results can be obtained with all these measures. These efforts were done using an active dialogue with patients and regulators [12].

For spinal muscular atrophy (SMA), many clinician-rated outcomes exist for all levels of ability (Type I, II and III), although not all of these were specifically designed for SMA. The Children’s Hospital of Philadelphia Infant Test of Neuromuscular Disorders (CHOP-INTEND) is suitable for Type I infants [13]. There are international efforts in place to revise the Hammersmith Functional Motor Scale (HFMS – designed for use in Type II [14]) and its expanded counterpart for ambulant patients (HFMSE – for type II and III) into a more psychometrically robust measure – the Revised Hammersmith Scale (RHS) [15]. For arm function, there is a Revised Upper Limb Module (RULM) [16]. Other tests used are the Motor Function Measure (MFM), a more generic functional assessment for neuromuscular disease [17], and 6MWT which has been shown to measure fatigue in SMA [18]. Strength may be assessed in this group with very sensitive devices such as the myopinch. Activity monitors may also be useful and now exist for ambulatory and non-ambulatory patients. In this group, the relationship between function, fatigue and growth is an issue but ultimately patient reported measures are paramount.

ANIMAL MODELS

Joe Kornegay discussed available dog models for DMD and their mutations [19]. The golden retriever muscular dystrophy (GRMD) model remains the most used canine clinical model [20]: affected dogs tend to show a progressive phenotype but some have a relatively mild course more in keeping with Becker muscular dystrophy (BMD) [21]. The Labrador and Japanese Spitz show similar mild BMD syndromes and DMD gene mutations have more recently been described [22]. Interestingly, the German Shorthaired Pointer-Muscular Dystrophy Dog (GSHPMD) has essentially a spontaneous deletion of the entire DMD gene with no dystrophin and also no revertant fibers [23], offering a cleaner model for studying the immunologic response to dystrophin as a neoantigen. He emphasized the importance of the mdx mouse model as an initial screening tool to detect potential treatment efficacies, and the more pronounced phenotype and relevance of dog models to the DMD phenotype. As with DMD patients, phenotypic variation occurs in GRMD. This can potentially confound preclinical studies, necessitating larger group sizes to prove efficacy. Phenotypic variation can be countered by comparing longitudinal effects of treatment and by balancing disease severity between treatment and control groups. On the other hand, the presence of phenotypic variation allows for studies to identify genetic mechanisms that may contribute to so-called “secondary effects” of dystrophin deficiency.

Gender of GRMD dogs, e.g. homozygous females or heterozygous males, should also be considered in preclinical studies [24], given that mdx females can have a slightly less severe phenotype than males, presumably due to effects of estrogen [25]. However, for biomarkers commonly used in the Kornegay laboratory, there was no observed effect of gender [26]. It is important to bear in mind the man-dog age relationship, which differs among canine breeds. For example, larger breeds typically have shorter life spans so each year equates to a greater number of human years. For the golden retriever, the first year of life corresponds to the first 20 years in humans [27]. In this way, the course of GRMD over the first year and DMD over the first 20 years can be divided into quartiles, with 0–3, 3–6, 6–9, and 9–12 months corresponding to 0–5, 5–10, 10–15, and 15–20 years in DMD, respectively. This analogy is reasonably accurate over the first 6 months in GRMD (10 years of DMD), when both conditions progress at a similar rate. However, many GRMD dogs stabilize between 6 and 12 months of age, whereas DMD continues to progress. The rapid clinical progression of GRMD between 3 and 6 months of age (5–10 years of DMD) provides a relatively short window over which therapeutic efficacy can be tested. Joe mentioned the current SOPs available on the TREAT-NMD website (http://www.treat-nmd.eu/research/preclinical/dmd-sops/) and the opportunity to develop additional SOPs for tests such as the 6MWT, gait analysis, accelerometry, respiratory assessment (pneumotach, respiratory inductance plethysmography (RIP)), and cardiac assessment (electrocardiography, echo, magnetic resonance imaging).

TREAT-NMD ADVISORY COMMITTEE FOR THERAPEUTICS (TACT)

Kanneboyina Nagaraju presented the recent experience of TACT applications [28]. Many applicants were considered to have submitted very preliminary preclinical results with poor experimental design, but claimed efficacy as soon as a minimal significant difference was observed for a single outcome. The sample size was sometimes not considered to be adequate. This makes it difficult to interpret results. The majority of the TACT applicants (especially small companies) do not know about the existence of the TREAT-NMD SOPs (http://www.treat-nmd.eu/research/preclinical/dmd-sops/).

CURRENT CHALLENGES

Shin‘ichi Takeda noted that another important issue is the mouse age chosen for experimentations and suggested that tests in multiple domains (i.e. muscle function, muscle strength, histology etc.) should be required for each efficacy study on mdx mice, as described in earlier review articles [4, 29].

Importantly, the ongoing studies on natural history and clinician rated outcomes in DMD patients and the discussion around the canine models, corroborate the need to upgrade SOPs for mdx mice, in order to have more translatable readouts. Also, it is underlined that novel methodological approaches have been proposed by various groups to be included in SOPs for mdx mice as possibly important correlates of clinician rated outcomes. Consensus about which procedures to validate and include as new SOPs is then needed.

Annemieke Aartsma-Rus mentioned an open letter to journal editors available under http://www.treat-nmd.eu/resources/ethics/open-letter/ that emphasises the danger of highlighting potential efficacy of drugs using poor or not rigorously reproduced experimental evidence, because this leads some families to purchase products privately on the basis of such hope. She underlined the high need to distinguish between exploratory ‘proof of principle’ animal studies and confirmatory pre-clinical trials and to require stronger rigor for the latter.

Tom Gillingwater referred to a wide variability of pre-clinical results reported by the SMA research community, likely resulting from the use of several different mouse models presenting with different degrees of pathology. This also raised issues concerning the relevance of current SMA mouse models to different SMA types in patients. Based on the extensive variability observed between mouse models, he considered it unfeasible at present to propose a list of obligatory tests in preclinical studies for SMA. However, standards for reporting experimental details and conditions (including handling, feeding etc.) which could influence outcomes will be useful. In addition, the issue of treatments being efficacious in SMA models only if delivered pre-symptomatically, raises issues concerning applicability to patients.

DISCUSSIONS

There was extensive discussion on three main topics.

SOPs and guidelines for preclinical research

It was agreed that it is of key importance to increase awareness of quality controls required in preclinical research: a suggestion was made to include internal controls for drug experimentation in the form of a ‘standard’ drug for which the effect is already known. However, it was also recognized that this may not always be possible, due to restrictions of local ethical committees.

To increase the number of available TREAT-NMD SOPs, it was suggested that a Bio-NMD SOP be added for mouse serum sample collection and storage, to include the influence of freezing/thawing and other factors, and conditions in the assessment of biomarkers. For DMD dogs, the development of additional SOPs on functional tests like the 6MWT, gait analysis, accelerometry, respiratory and cardiac assessments was suggested.

It was also proposed to establish a curated database of natural history for mdx mice, to allow for meta-analysis of main mouse outcomes. The database should start by collecting data for mdx (considering both C57BL/10ScSnJ and C57BL/6J backgrounds; other genetic backgrounds could be added in a later phase) and mdx-utrn-/- mice. Key parameters to be measured (and replicated across multiple labs) include: survival (mdx-utrn-/- only), creatine kinase levels, grip strength, hanging test, locomotion assessment, force contraction, histology (quantification of centrally nucleated fibers, muscle fiber size, and fibrosis) as well as cardiac function. Measurement units, devices, mouse ages, gender, origins (commercial or in-house breeding) and sample sizes need be specified. Costs and personnel were briefly discussed and it was decided to explore funding opportunities. Work on collecting and collating data for this purpose has commenced.

Establish rules to accept SOPs submitted spontaneously to TREAT-NMD

Raffaella Willmann informed the group that the contact point for all requests about SOPs has been transferred to the TREAT-NMD secretariat and that a clear list for the acceptance procedure needs to be set.

The discussion suggested the following criteria need to be employed: use of a relevant model (DMD: mdx, GRMD/CXMDJ; SMA: all); data should have been reproduced several times in the originator laboratory and independently validated in at least two additional laboratories; information on sample size calculation for power analysis needs to be included; data on untreated animals (wild type and relevant disease model) should be reported for reference.

If criteria are met, the secretariat should contact the core committee (Miranda Grounds, Kanneboyina Nagaraju or Annemieke Aartsma-Rus for DMD mouse model assessments, Tom Gillingwater for SMA mouse model assessments) to nominate 2–3 adequate peer reviewers. Timeline: 2 weeks to check criteria, 1 week to identify reviewers, negotiation of 6–8 weeks to review protocol. Peer reviewers will appear on the SOP.

Organization of a stakeholder meeting to raise awareness of preclinical study quality in neuromuscular disease research

All participants agreed that the responsibility for increasing quality and reproducibility in preclinical research data cannot fall entirely on individual researchers. Funding organizations with no peer review, sometimes support studies without considering an appropriate project selection based on feasibility and on a strong rationale. Journals could play a key role by requiring stronger guidelines for publishing preclinical studies that aim at translation to clinical trials. The issue of extrapolation of mouse data to human outcomes should be discussed as well, especially for nutrition, standards of care, fatigue, strength, locomotion, and BMI through a retrospective evaluation of existing clinical data and animal model data. In addition, the need for preclinical research to connect and interact with the requirements of clinical studies was considered an important issue to address. Input on these matters is required from clinicians, academic researchers, funding agencies, Journal editors, and industry, as well as patient groups and representatives from regulatory agencies. It was decided to apply for a European Neuromuscular Center (ENMC) Workshop to address these important topics in a subsequent Workshop. This was achieved with a Workshop held in 2017 (see below).

CONCLUSIONS AND DELIVERABLES

The scope of this Schiphol workshop was to revise and review the efforts undertaken by the TREAT-NMD Alliance to improve the quality of preclinical studies for DMD and SMA, and to suggest feasible next steps for further developments.

Since the Workshop, the following main deliverables that have been achieved to date are (1) the establishment of a curated database of natural history for mdx, which allows for meta-analysis of main mouse outcomes, and (2) Guidelines and rules are now in place for the creation and submission of future SOPs. Finally, (3) a subsequent, larger ENMC Workshop was organized and took place in February 2017 in order to discuss preclinical study quality, with representatives from academic and clinical research, industry, patients, funding agencies, journals and regulatory agencies (Report manuscript in press).

Initiatives from this Workshop which are still pending are: finalisation of a SOP on serum collection for biomarker analyses in DMD mouse models, generation of more SOPs for DMD dog models, and agreements on standards of reporting for SMA preclinical research.