David J Jörg1, Doris H Fuertinger1, Alhaji Cherif2, David A Bushinsky3, Ariella Mermelstein2, Jochen G Raimann2, Peter Kotanko2,4. 1. Biomedical Modeling and Simulation Group, Global Research and Development, Fresenius Medical Care Germany, Bad Homburg, Germany. 2. Renal Research Institute, New York, United States. 3. Department of Medicine, University of Rochester School of Medicine and Dentistry, Rochester, United States. 4. Icahn School of Medicine at Mount Sinai, New York, United States.
Abstract
For the treatment of postmenopausal osteoporosis, several drug classes with different mechanisms of action are available. Since only a limited set of dosing regimens and drug combinations can be tested in clinical trials, it is currently unclear whether common medication strategies achieve optimal bone mineral density gains or are outperformed by alternative dosing schemes and combination therapies that have not been explored so far. Here, we develop a mathematical framework of drug interventions for postmenopausal osteoporosis that unifies fundamental mechanisms of bone remodeling and the mechanisms of action of four drug classes: bisphosphonates, parathyroid hormone analogs, sclerostin inhibitors, and receptor activator of NF-κB ligand inhibitors. Using data from several clinical trials, we calibrate and validate the model, demonstrating its predictive capacity for complex medication scenarios, including sequential and parallel drug combinations. Via simulations, we reveal that there is a large potential to improve gains in bone mineral density by exploiting synergistic interactions between different drug classes, without increasing the total amount of drug administered.
For the treatment of postmenopausal osteoporosis, several drug classes with different mechanisms of action are available. Since only a limited set of dosing regimens and drug combinations can be tested in clinical trials, it is currently unclear whether common medication strategies achieve optimal bone mineral density gains or are outperformed by alternative dosing schemes and combination therapies that have not been explored so far. Here, we develop a mathematical framework of drug interventions for postmenopausal osteoporosis that unifies fundamental mechanisms of bone remodeling and the mechanisms of action of four drug classes: bisphosphonates, parathyroid hormone analogs, sclerostin inhibitors, and receptor activator of NF-κB ligand inhibitors. Using data from several clinical trials, we calibrate and validate the model, demonstrating its predictive capacity for complex medication scenarios, including sequential and parallel drug combinations. Via simulations, we reveal that there is a large potential to improve gains in bone mineral density by exploiting synergistic interactions between different drug classes, without increasing the total amount of drug administered.
Osteoporosis, a disease characterized by porous bone prone to fractures, affects hundreds of millions of people worldwide (Cooper and Ferrari, 2019; Hernlund et al., 2013). Most recent estimates place the global annual incidence of bone fragility fractures at 9 million in the year 2000 (Cooper and Ferrari, 2019); projections for the year 2050 suggest between 7 and 21 million annual hip fractures (Gullberg et al., 1997). Osteoporosis-associated bone fractures lead to disabilities, pain, and increased mortality (Cooper and Ferrari, 2019). In the United States, medical cost for osteoporosis, including inpatient, outpatient, and long-term care costs, has been estimated at US$17 billion in 2005 (Burge et al., 2007); in the European Union, the total cost of osteoporosis, including pharmacological interventions and loss of quality-adjusted life-years (QALYs), is projected to rise from about €100 billion in 2010 to €120 billion in 2025 (Odén et al., 2013).Osteoporotic bone is the consequence of an imbalance of continuous bone resorption and bone formation, which—under close to homeostatic conditions—has the function to remove microfractures and renew the structural integrity of bone. Postmenopausal women are particularly at risk of osteoporosis: the rapid decline of systemic estrogen levels after menopause and other aging-related effects such as increased oxidative stress contribute to or drive the development of osteoporosis (Riggs et al., 1998; Manolagas, 2010). Moreover, osteoporosis can be a sequela of diseases affecting bone metabolism and remodeling such as primary hyperparathyroidism or gastrointestinal diseases (Painter et al., 2006). Osteoporosis can also be a side effect of treatments for other diseases; as a prime example, glucocorticoid administration is the most common cause of secondary osteoporosis (Weinstein, 2012). Over the last decades, an array of different osteoporosis treatments have emerged, from simple dietary supplementations such as calcium and vitamin D to specialized drugs targeting bone-forming and -resorbing cells and related signaling pathways (Tu et al., 2018). This entails a plethora of different medication options, including a large number of possible dosing schemes and combinations of drugs, administered in sequence or in parallel. Due to the huge number of such treatment schemes and the required time from study inception to completion, very few of them have been clinically tested so far when compared to the total number of available options.Concomitant with the development of new osteoporosis drugs, mathematical and biophysical modeling approaches capturing bone-related physiology have advanced our quantitative understanding of the biological principles governing bone mineral metabolism, bone turnover, and development of osteoporosis. Pioneering work by Lemaire et al., 2004 describes the dynamics of bone-forming and -resorbing cell populations coupled through signaling pathways and could qualitatively reproduce the effects of senescence, glucocorticoid excess, and estrogen and vitamin D deficiency on bone turnover. Since then, compartment-based descriptions of the mineral metabolism, bone-forming and -resorbing cell populations, and related signaling factors have elucidated the role of essential regulatory mechanisms underlying mineral balance and bone turnover (Komarova et al., 2003; Lemaire et al., 2004; Pivonka et al., 2008; Pivonka et al., 2010; Peterson and Riggs, 2010; Zumsande et al., 2011; Schmidt et al., 2011; Graham et al., 2013; Tanaka et al., 2014; Komarova et al., 2015; Berkhout et al., 2015). Coarse-grained as well as detailed spatially extended descriptions of bone geometry have also addressed the effects of mechanical forces and the propagation of the multicellular units responsible for bone turnover (Ryser et al., 2009; Buenzli et al., 2011; Scheiner et al., 2013; Buenzli et al., 2014; Pivonka et al., 2013), as well as the influence of secondary diseases such as multiple myeloma (Ayati et al., 2010). Detailed models of bone remodeling and calcium homeostasis have become versatile and widely used tools in hypothesis testing, such as the seminal model by Peterson and Riggs, 2010, which includes submodels for various organs such as gut, kidney, and the parathyroid gland. Pharmacokinetic and pharmacodynamic (PK/PD) models of therapeutic interventions have mostly focused on capturing the mechanisms of action of a single or a few drugs and testing their dosing regimens (Marathe et al., 2008; Marathe et al., 2011; Ross et al., 2012; Scheiner et al., 2014; Eudy et al., 2015; Lisberg et al., 2017; Martínez-Reina and Pivonka, 2019; Zhang and Mager, 2019). Recent modeling efforts have also started addressing the effects of drug combinations on bone-forming and -resorbing cells, pointing out the need for corresponding model frameworks to include clinically relevant variables like bone mineral density (BMD) and bone turnover biomarkers (BTMs) (Lemaire and Cox, 2019), as well as combination therapies of physical exercise and drug treatment (Lavaill et al., 2020). An integrated mathematical framework for multiple drugs, which can also be used to quantitatively predict the effects of drug combinations in sequence and in parallel is not yet available.Building on established mechanisms of bone turnover, we here present a quantitative model of bone turnover and postmenopausal osteoporosis treatment, unifying the description of multiple classes of drugs with different mechanisms of action, namely, bisphosphonates, parathyroid hormone (PTH) analogs, sclerostin antibodies, and receptor activator of NF-B ligand (RANKL) antibodies. We calibrate the model using published population-level data from several clinical trials and assess its ability to predict the outcome of previously conducted clinical studies based on the medication scheme alone. We then use the model to demonstrate how medication schemes involving drug combinations can be optimized for a given medication load and discuss future model extensions.
Mechanisms of bone turnover and its regulation
Our model is based on a small set of key principles of bone turnover, which we briefly recapitulate here (Figure 1). As a composite tissue comprising hydroxyapatite, collagen, other proteins, and water (Boskey, 2013), bone is constantly turned over to renew its integrity and remove microdamage, at an average rate of about 4% per year in cortical bone and about 30% per year in trabecular bone (Manolagas, 2000).
Figure 1.
Schematic of the osteoporosis model describing the cell dynamics and signaling pathways within a ‘representative bone remodeling unit (BRU)’.
Regulatory interactions between different model components are indicated by colored boxes (see legend). TGFβ, transforming growth factor beta; BMP, bone morphogenetic protein; PDGF, platelet-derived growth factor; IGF, insulin-like growth factor; FGF, fibroblast growth factor.
Schematic of the osteoporosis model describing the cell dynamics and signaling pathways within a ‘representative bone remodeling unit (BRU)’.
Regulatory interactions between different model components are indicated by colored boxes (see legend). TGFβ, transforming growth factor beta; BMP, bone morphogenetic protein; PDGF, platelet-derived growth factor; IGF, insulin-like growth factor; FGF, fibroblast growth factor.
Bone-resorbing and -forming cells
Bone resorption is performed by osteoclasts, multinucleated cells formed through the differentiation and fusion of their immediate precursors (pre-osteoclasts), which are derived from pluripotent hematopoietic stem cells via the myeloid lineage (Boyce and Xing, 2008). Osteoclasts attach to bone tissue and resorb it through the secretion of hydrogen ions and bone-degrading enzymes (Fuller and Chambers, 1995), which leads to the release of minerals and signaling factors stored in the bone matrix. New bone is formed by osteoblasts, a cell type derived from mesenchymal stem cells via several intermediate states that give rise to pre-osteoblasts and finally osteoblasts (Eriksen, 2010). Groups of osteoblasts organize into cell clusters (osteons) and collectively lay down an organic matrix (osteoid), which subsequently becomes mineralized over the course of months. Osteoblasts that are enclosed in the newly secreted bone matrix become osteocytes, nondividing cells with an average life span of up to several decades. Osteoclasts and osteoblasts organize into spatially defined local clusters termed ‘bone remodeling units’ (BRUs) (Figure 1), in which osteoblasts replenish the bone matrix previously resorbed by osteoclasts with a delay of several weeks. In cortical bone, the outer protective bone layer, BRUs migrate as a whole in ‘tunnels,’ whereas within the inner cancellous bone, BRUs propagate on the surfaces of the trabeculae, renewing the bone matrix in the process (Eriksen, 2010).
Signaling pathways
The differentiation and activity of osteoclasts and osteoblasts are regulated through several signaling pathways and hormones; recent reviews provide comprehensive descriptions of the various pathways (Siddiqui and Partridge, 2016). Osteoclast formation and activity are prominently regulated by RANKL and macrophage colony-stimulating factor (M-CSF) synthesized by bone marrow stromal cells. RANKL binds to receptor activator of NF-B (RANK) on osteoclast precursors and promotes their differentiation into mature osteoclasts; osteoprotegerin (OPG) acts as a decoy receptor for RANKL and thus inhibits bone resorption (Boyce and Xing, 2008; Clarke, 2008). When laying down new bone, osteoblasts store signaling factors in the bone matrix, including transforming growth factor beta (TGFβ), bone morphogenetic protein (BMP), insulin growth factors (IGFs), platelet-derived growth factor (PDGF), and fibroblast growth factors (FGFs) (Solheim, 1998). Upon bone resorption, these factors are released and regulate cell fates and activity of osteoblasts and osteoclasts, thereby coupling bone resorption and formation (Houde et al., 2009; Eriksen, 2010). Osteocytes secrete sclerostin, a Wnt inhibitor interfering with extracellular binding of Wnt ligands (Li et al., 2005). Sclerostin inhibits bone formation and promotes resorption via downregulation of osteoblastogenesis and upregulation of osteoclastogenesis (Delgado-Calle et al., 2017; Maré et al., 2020). Since bone also acts as a mineral reservoir for the body, regulators of calcium homeostasis such as PTH and vitamin D also strongly affect the balance of bone formation and resorption alongside the intestinal absorption and renal reabsorption of calcium (Mundy and Guise, 1999).
Estrogen
The sex hormone estrogen inhibits bone resorption by inducing apoptosis of osteoclasts (Kameda et al., 1997) and lowering circulating sclerostin levels (Mödder et al., 2011). The rapid decline of estrogen levels after menopause is one known cause of postmenopausal osteoporosis (Riggs et al., 1998).
Results
Model overview
The primary purpose of our model is to provide an efficient representation of bone turnover on multiple time scales from weeks to decades that allows for the quantitative description of drug interventions. Of particular interest are the consequences of pharmacological therapies on long-term dynamics of the BMD in specific bone sites and biochemical markers of bone formation and resorption. To this end, we considered a minimal set of physiologically relevant dynamic components (Figure 1) that are sufficient to capture a large range of clinically observed population-level data on drug interventions. Thus, our model describes a ‘representative BRU’ that abstracts from the vast set of intricate regulatory mechanisms underlying calcium homeostasis or the complex bone geometry.Our model comprises the following dynamic components to describe the bone turnover through a representative BRU: cell densities of (i) pre-osteoclasts, (ii) osteoclasts, (iii) pre-osteoblasts, (iv) osteoblasts, (v) osteocytes, (vi) sclerostin concentration, (vii) total bone density, and (viii) bone mineral content (BMC). The BMD is given by the product of bone density and BMC. Osteoblasts and osteoclasts can undergo apoptosis and are derived from pre-osteoblasts and pre-osteoclasts, respectively, with differentiation rates that depend on regulatory factors such as estrogen and sclerostin (Figure 1). Pre-osteoblasts and pre-osteoclasts are formed at constant rates and undergo apoptosis. These progenitor populations provide a dynamic reservoir for rapid differentiation and activation of osteoblasts and osteoclasts, respectively, which can be temporarily depleted if stimulated by a drug intervention. Osteocytes are derived from osteoblasts and provide a source of sclerostin, which has a regulatory effect on osteoblasts, osteoclasts, and thus, bone density change. The gain and loss rates of bone density are proportional to the density of osteoblasts and osteoclasts, respectively. The BMC has a steady state whose level can be temporarily shifted through drug administration, effectively accounting for more complex underlying dynamics such as promotion of secondary mineralization. All rates of cell formation, differentiation, apoptosis, and bone formation and resorption generally depend on the concentration of sclerostin, estrogen, and a ‘resorption signal.’ These dependencies also implicitly account for regulation of bone remodeling via other routes, for example, the RANK–RANKL–OPG pathway. The effects of aging and the onset of menopause are represented through an age-dependent serum estrogen concentration, which has been determined from the literature (Sowers et al., 2008; Appendix 1). The resorption signal corresponds to the melange of signaling factors stored in the bone matrix. Therefore, its release is proportional to the rate of bone resorption. The serum concentration of BTMs such as the resorption marker C-terminal telopeptide (CTX), the formation markers procollagen type 1 amino-terminal propeptide (P1NP), and bone-specific alkaline phosphatase (BSAP) were identified with elementary functions of the bone resorption and formation rates in the model (Appendix 1).We extended this core model of long-term bone turnover by a dynamic description of the mechanisms of action of several drug classes used in osteoporosis treatment: RANKL antibodies (denosumab), sclerostin antibodies (romosozumab), bisphosphonates (alendronate and others), and PTH analogs (teriparatide) (Appendix 2). We also included blosozumab, another sclerostin inhibitor, which was investigated in osteoporosis trials but not approved for osteoporosis treatment at the time the present work was conducted. PTH is known to exert anabolic or catabolic effects depending on whether administration is intermittent or continuous (Tam et al., 1982; Hock and Gera, 1992); PTH description in our model is restricted to the anabolic administration regimes relevant for osteoporosis treatment. A schematic overview of all model components, mechanisms, and regulatory interactions is given in Figure 1; a detailed formal description of the model and its extensions is provided in Appendix 1 and Appendix 2.
Capturing clinical study results with the model
The model and the corresponding medication modules rely on an array of physiological parameters (rates of cell formation, differentiation and death, concentration thresholds for signaling activity, medication efficacies and half-lives, etc.) many of which are not directly measureable. However, clinical measurements on physiological responses to medications with different mechanisms of action provide a wealth of indirect information about time scales of bone turnover and regulatory feedbacks. We calibrated the model using published clinical data from various seminal studies on both (i) long-term BMD age dependence and (ii) the response of BMD and BTMs to the administration of different drugs (see Appendix 3—table 2 for a comprehensive list of data sources). Although BMD constitutes the major target variable of our model, the dynamics of BTM concentrations carry important complementary information about the mode of action of the administered drugs (antiresorptive, anabolic, and combinations) that crucially informs the calibration procedure. To allow the model to capture the effects of medications as physiologically sensible modulations of the age-dependent bone mineral metabolism, we created hybrid datasets each of which comprised both aging-related BMD changes and the response to a treatment (see ‘Methods’ and Appendix 1—figure 1D).
Appendix 3—table 2.
Data sources used to calibrate and validate the model.
Columns titled ‘Figure(s)’ indicate the plot panels in the respective publication that were digitized. BMD always refers to total hip bone mineral density.
Publication
Medication(s)
Dosings
Figure(s)
Table(s)
BMD
CTX
P1NP
BSAP
BMD
Black et al., 2006
Alendronate
5–10 mg Q1D
2
3
3
—
—
Bone et al., 2011
Denosumab
60 mg Q6M
3b
4b
4a
—
—
Cosman et al., 2016
Romosozumab
210 mg Q1M
3b
3e
3d
—
—
Denosumab
60 mg Q6M
3b
3e
3d
—
—
Leder et al., 2014
Teriparatide
20 mcg Q1D
2d
4c,f
4b,e
—
—
Leder et al., 2015
Teriparatide
20 mcg Q1D
3
4
—
—
—
Denosumab
60 mg Q6M
3
4
—
—
—
Lewiecki et al., 2019
Romosozumab
210 mg Q1M
3b
—
—
—
—
Denosumab
60 mg Q6M
3b
—
—
—
—
Looker et al., 1998
[Age-dependent BMD]
—
—
—
—
—
7
McClung et al., 2006
Denosumab
6 mg Q3M, 14 mg Q6M, 210 mg Q6M
2b
2e
—
2f
—
McClung et al., 2017
Denosumab
6–14 mg Q3M, 14–210 mg Q6M
2b
—
—
—
—
McClung et al., 2018
Romosozumab
140 mg Q1M, 210 mg Q1M
3c
4b
4a
—
—
Denosumab
60 mg Q6M
3c,d
4b,d
4a,c
—
—
Alendronate
70 mg Q1W
3d
4d
4c
—
—
Recknor et al., 2015
Blosozumab
180 mg Q4W, 180 mg Q2W, 270 mg Q2W
3b
4d
4a
—
—
Saag et al., 2017
Alendronate
70 mg Q1W
3b
3d
3c
—
—
Q, every; M, month; D, day; W, week; CTX, C-terminal telopeptide; P1NP, procollagen type 1 amino-terminal propeptide; BSAP, bone-specific alkaline phosphatase.
Appendix 1—figure 1.
Parameterization of the aging behavior and creation of hybrid aging/treatment datasets for model calibration and validation.
(A) Age dependence of estradiol serum levels. Clinical data (dots) modified from Sowers et al., 2008. The curve shows a fit of the function given by Equation 18 to determine the parameter τe (Appendix 3—table 4). (B) Bone mineral density (BMD) age dependence for different ethnic groups as indicated. Data modified from Looker et al., 1998; reported age bin averages have been used to represent the center of the age bin. (C) BMD age dependence shown in panel (B), where all curves have been normalized to their earliest value (t0 = 25y). (D) Schematic of how hybrid aging/treatment datasets were generated by merging the same aging dataset with different treatment datasets; for details, see ‘Methods.’
We then determined a single set of model parameters through a simultaneous fit of the free 31 model parameters to capture a specified set of hybrid aging/treatment datasets containing different drug responses (Appendix 3). Without constraining the average rate of skeletal bone turnover, model calibration yielded an inferred value of about 6% per year on average, of the same order of values reported for cortical bone, which constitutes about 75% of the skeleton (Manolagas, 2000). The model was able to capture the BMD and BTM dynamics across all calibration datasets with remarkable accuracy (Appendix 3—figure 1), despite the model’s structural simplicity. To quantify the goodness of the fit, we computed the mean absolute percentage error (MAPE) between model simulations and clinical data; the MAPE for BMD was consistently below 1% for all calibration datasets (Appendix 3—table 3), indicating an excellent agreement between model and data. The qualitative behavior of BTMs (i.e., the direction of their excursions from baseline) was captured correctly in all calibration datasets, indicating an adequate description of the drugs’ mode of action in the model; relative deviations in the total magnitude of BTM excursions observed for some datasets were mostly due to slight offsets in the timing of peaks and troughs and low absolute values of the respective BTM concentrations, as highlighted by comparing different goodness measures (Appendix 3 and Appendix 3—table 3).
Appendix 3—figure 1.
Calibration datasets comparing model predictions and clinical data from various studies.
All conventions identical to Figure 2. Drug administrations are provided in the bottom row. See Appendix 3—table 2 for a list of data sources and Appendix 3—table 3 for goodness-of-fit measures. Dosing: mg, milligrams; mcg, micrograms; Q M, dose administered every months; Q W, every weeks; Q D, every days; B, blosozumab; A, alendronate; D, denosumab; T, teriparatide.
Appendix 3—table 3.
Goodness-of-fit measures for calibration and validation datasets.
Mean absolute percentage error (MAPE) and windowed minimal absolute percentage error (WMAPE) as defined in Equation 30 and Equation 31, respectively. The column ‘Shown in’ indicates the figure in this article that shows the respective simulation and data plot.
Q, every; M, month; D, day; W, week; BMD, bone mineral density; CTX, C-terminal telopeptide; P1NP, procollagen type 1 amino-terminal propeptide; BSAP, bone-specific alkaline phosphatase.
After obtaining the reference parameter set, we sought to validate the calibrated model by assessing its ability to predict the effects of drug dosing schemes that had not been used for calibration. Model validation included complex sequential and parallel drug combinations and therefore challenged the model to predict the effects of treatment schemes beyond those used in calibration (Appendix 3—table 2). To this end, the model received only drug dosing information used in the respective clinical trials but was not informed by BMD or BTM measurements, which instead it had to predict. With the single set of previously determined parameters, the model showed a remarkable capacity to quantitatively forecast the effects of a multitude of medication schemes, both during treatment and follow-up periods (Figure 2, Figure 2—figure supplement 1). Even in scenarios including sequential treatments with up to three different drug types and parallel treatments with two different drugs, respectively, the model was able to predict the complex progression of both BMD and biomarker levels with a high degree of accuracy (Figure 2). Across all validation datasets, MAPEs for BMD were consistently below 1.5% (Appendix 3—table 3), indicating an excellent predictive capacity of the model. In summary, this validation provided a strong corroboration of the model’s capacity to capture the physiological dynamics of bone turnover and the mechanisms of action of various drugs relevant to osteoporosis treatment using a single set of model parameters.
Figure 2.
With a single set of parameters, the calibrated model can quantitatively predict the effects of various drugs in different dosing regimens, alone and in combination.
(A) Comparison of simulated total hip bone mineral density (BMD, black curves) and clinical data (dots), including aging behavior (green dots) and treatment behavior (black dots) of various sequential drug treatments, including denosumab, romosozumab, alendronate, and teriparatide. Hybrid aging/treatment datasets were created combining data from Looker et al., 1998 (aging dataset, green dots in panel A; in total subjects 20 years and older), as well as Recknor et al., 2015 (blosozumab 180 mg Q2W: ), McClung et al., 2018 (placebo/deno.: , alendro./romo./deno.: ), and Leder et al., 2015 (deno./teri.: , teri. + deno./deno.: ) (treatment datasets, black dots in panels A and B) as indicated, see ‘Methods.’ (B) Zoom into the treatment regions shown in panel (A) including BMD (black) and baseline changes of the bone resorption marker C-terminal telopeptide (CTX, red) and the bone formation marker procollagen type 1 amino-terminal propeptide (P1NP, blue). Colored bars above the plots indicate the medication scheme (see legend). Data points show population averages; average types and error bar types as reported in the respective original publication. In both panels, BMD is displayed as a fraction of its value at years.
See Appendix 3—table 2 for a list of data sources and Appendix 3—table 3 for goodness-of-fit measures. Dosing: mg, milligrams; mcg, micrograms; Q M, dose administered every months; Q W, every weeks; Q D, every days; R, romosozumab; A, alendronate; D, denosumab; T, teriparatide.
Figure 2—figure supplement 1.
Continuation of Figure 2 comparing model predictions and clinical data from various studies, all conventions identical.
See Appendix 3—table 2 for a list of data sources and Appendix 3—table 3 for goodness-of-fit measures. Dosing: mg, milligrams; mcg, micrograms; Q M, dose administered every months; Q W, every weeks; Q D, every days; R, romosozumab; A, alendronate; D, denosumab; T, teriparatide.
With a single set of parameters, the calibrated model can quantitatively predict the effects of various drugs in different dosing regimens, alone and in combination.
(A) Comparison of simulated total hip bone mineral density (BMD, black curves) and clinical data (dots), including aging behavior (green dots) and treatment behavior (black dots) of various sequential drug treatments, including denosumab, romosozumab, alendronate, and teriparatide. Hybrid aging/treatment datasets were created combining data from Looker et al., 1998 (aging dataset, green dots in panel A; in total subjects 20 years and older), as well as Recknor et al., 2015 (blosozumab 180 mg Q2W: ), McClung et al., 2018 (placebo/deno.: , alendro./romo./deno.: ), and Leder et al., 2015 (deno./teri.: , teri. + deno./deno.: ) (treatment datasets, black dots in panels A and B) as indicated, see ‘Methods.’ (B) Zoom into the treatment regions shown in panel (A) including BMD (black) and baseline changes of the bone resorption marker C-terminal telopeptide (CTX, red) and the bone formation marker procollagen type 1 amino-terminal propeptide (P1NP, blue). Colored bars above the plots indicate the medication scheme (see legend). Data points show population averages; average types and error bar types as reported in the respective original publication. In both panels, BMD is displayed as a fraction of its value at years.
Continuation of Figure 2 comparing model predictions and clinical data from various studies, all conventions identical.
See Appendix 3—table 2 for a list of data sources and Appendix 3—table 3 for goodness-of-fit measures. Dosing: mg, milligrams; mcg, micrograms; Q M, dose administered every months; Q W, every weeks; Q D, every days; R, romosozumab; A, alendronate; D, denosumab; T, teriparatide.
Testing alternative treatment schemes
Having established the predictive capacity of the model for the considered medications, we aimed to utilize the model to study and optimize hypothetic drug dosing regimens. As an example, we considered a sequential treatment with three drugs of different types: the bisphosphonate alendronate, the sclerostin inhibitor romosozumab, and the RANKL inhibitor denosumab. In a clinical trial reported by McClung et al., 2018, the sequence alendronate (70 mg per week for 1 year), followed by romosozumab (140 mg per month for 1 year), followed by denosumab (60 mg per 6 months for 1 year) had been studied (Figure 2). However, in principle there are six different sequences in which these drugs can be administered: ARD, ADR, DAR, DRA, RAD, and RDA (A: alendronate; R: romosozumab; D: denosumab). A priori, it is not obvious whether synergistic or antagonistic interactions between these drugs and the physiological state in which they leave the patient may lead to a differential short- and long-term evolution of BMD and biomarkers between different medication sequences. Probing all six sequences in a clinical trial would present a time- and resource-consuming endeavor and inevitably expose part of the study population to suboptimal treatment schemes. Instead, we probed these different treatment options using the present model (Figure 3A). To assess the predicted clinical success of different sequences, we compared two clinically relevant outcomes across different schemes: (i) the maximum achieved BMD increase (as compared to baseline at treatment start) irrespective of when it occurred (Figure 3B) and (ii) the residual long-term effects of treatment on BMD as monitored by the relative BMD 10 years after treatment end (Figure 3C) .
Figure 3.
The model predicts differential outcomes for different sequences of the same drugs at constant total medication load.
(A) Simulated progression of bone mineral density (BMD) and C-terminal telopeptide (CTX) and procollagen type 1 amino-terminal propeptide (P1NP) concentrations for different sequences (columns) of the three drugs denosumab (D), alendronate (A), and romosozumab (R) as indicated. Simulated treatment starts at age 67. The total amount of drug administered is identical among columns. Clinical results on the sequence ARD (column 5) were reported in McClung et al., 2018, see also Figure 2. (B) Maximum simulated BMD (relative to baseline at treatment start) achieved during the course of treatment for different drug sequences. (C) Simulated BMD 10 years after treatment end (relative to baseline at treatment start) for different drug sequences.
The model predicts differential outcomes for different sequences of the same drugs at constant total medication load.
(A) Simulated progression of bone mineral density (BMD) and C-terminal telopeptide (CTX) and procollagen type 1 amino-terminal propeptide (P1NP) concentrations for different sequences (columns) of the three drugs denosumab (D), alendronate (A), and romosozumab (R) as indicated. Simulated treatment starts at age 67. The total amount of drug administered is identical among columns. Clinical results on the sequence ARD (column 5) were reported in McClung et al., 2018, see also Figure 2. (B) Maximum simulated BMD (relative to baseline at treatment start) achieved during the course of treatment for different drug sequences. (C) Simulated BMD 10 years after treatment end (relative to baseline at treatment start) for different drug sequences.Indeed, we found that the outcomes of different medication sequences were markedly different despite the same total amount of drug administered (Figure 3A). Some sequences (such as ARD and RAD) reached a considerably higher maximum BMD during the course of the simulated treatment, which allowed us to rank treatments according to maximum BMD gain (Figure 3B). Notably, while some sequences were superior to others as measured by the maximum BMD increase during treatment, they performed markedly worse (as compared to, e.g., DRA and RDA) with regard to long-term BMD evolution as predicted by model simulations (Figure 3C). This behavior suggests that short-term BMD gains may be limited as a proxy for the clinical benefit of a treatment as a whole. Within our modeling scheme, the explanation for this behavior is found in differing ‘rebound’ effects after treatment end: simulated drug-mediated inhibition of osteoclastogenesis leads to a build-up of an undifferentiated osteoclast precursor pool. After treatment end, this precursor pool becomes licensed to differentiate and rapidly gives rise to a large active osteoclast population, leading to accelerated resorption of the bone matrix that had been built up during treatment. In this paradigm, specific drug sequences lead to an attenuation of this effect, for example, by enhancing osteoclast apoptosis during such a ‘rebound’ phase, thereby modulating bone turnover in the long run.In summary, our model analysis suggests considerable potential in the improvement of dosing regimens and drug sequencing in osteoporosis treatment, especially combination therapies, to achieve an optimal effect for a given medication load. These improvements are possible because the mechanisms of action of one drug may act either favorably or adversely on the state of the bone mineral metabolism left behind by the preceding treatment with another drug.
Discussion
Treatment of osteoporosis is complex, expensive, and in many circumstances opinion-based. With bone physiology as our guiding principle, we have introduced a mathematical modeling framework that can quantitatively capture and predict the progression of osteoporosis in postmenopausal women with and without medical therapy. Our model is built on a small set of essential mechanisms of bone turnover. The effectivity of this approach suggests that—despite the complexity of the bone mineral metabolism—the dynamics relevant for osteoporosis medications can be condensed into only a few components. These components describe the biology of osteoblasts, osteoclasts, and osteocytes, as well as their precursor cell populations and a few essential regulatory feedbacks through hormones and signaling factors such as estrogen, sclerostin, and bone-matrix-derived factors.The general nature of the model allowed us to capture the BMD and BTMs of a multitude of clinical treatment studies. Notably, the model can also predict the effects of a broad range of drug dosing regimens and complex drug combination therapies beyond those used for model development. This corroborates the model’s predictive capacity, supporting its use for the design of future clinical trials. However, it is important to note that some parameters (e.g., concentration thresholds for signaling factors) were inferred through the model calibration procedure from BMD and BTM dynamics alone. Hence, when used as a predictive tool, general quantitative limitations of the model have to be considered, especially when extrapolating into extreme dosing regimens, dosing frequencies, or age regions beyond the validated ones.It is of clinical relevance that exemplary model predictions suggest a large potential for the development of optimized combination therapies involving different drug types and treatment schemes. These may range from a simple rearrangement of a sequence of drugs at given total drug doses (as shown in this article) to complex interwoven or cyclic administration schemes that exploit synergistic effects between different medication types. Notably, model simulations extrapolating the long-term BMD development after treatment end suggest that medication schemes eliciting a rapid BMD increase are not necessarily accompanied by a sustained elevation of the BMD. Instead, some initially successful treatment schemes may lead to a ‘rebound’ effect of accelerated bone loss after treatment end, a prediction that cautions against using short-term BMD increases as the sole proxy for treatment success. Such extrapolations into follow-up periods long after treatment end, which are mostly inaccessible to clinical studies, highlight the potential role of the model in considering long-term treatment success when optimizing treatment schemes.In our research, we have focused on postmenopausal osteoporosis, the most widespread type of osteoporosis. However, the generic manner in which the model represents bone remodeling and the effects of medications renders it a general platform for the study of treatments that can be adapted to other types of primary and secondary osteoporosis. The modular nature of the model enables future extensions; besides additional medication types, these may include the effects of comorbidities that elicit osteoporosis or interact with it (such as primary and secondary hyperparathyroidism), medications that contribute to osteoporosis (such as glucocorticoid therapy), lifestyle-dependent factors such as smoking and alcohol consumption, the effects of dietary supplementation of osteoporosis treatment through calcium and vitamin D and effects of microgravity on bone, as experienced by astronauts on extended missions in space. Physical activity is another important contributor to bone remodeling, which we have not considered here. Detailed modeling approaches involving biomechanical feedback suggest synergistic effects between drug treatment of osteoporosis and physical activity (Lavaill et al., 2020). Such results call for a further exploration of integrated approaches to osteoporosis therapy combining pharmacological treatment and lifestyle adjustments.Clearly, the goal of osteoporosis therapy is the reduction of fracture risk during and after therapy. While BMD has a prime role in the evaluation of osteoporosis therapies and can be measured rather easily using dual-energy x-ray absorptiometry (DXA), its relationship to fracture risk is complex. Fracture risk calculations used in clinical practice also involve demographic and lifestyle-related factors while mostly relying on BMD point measurements (Kanis et al., 2009). However, the quantitative associations between BMD, age, and fracture risk reported in many studies (Kanis et al., 2001; Berger et al., 2009; Austin et al., 2012; Krege et al., 2013; Black et al., 2018; Ensrud et al., 2022) can be used to create statistical models that may relate entire BMD time courses to a patient’s fracture risk. Combining such statistical models with the physiology-based model presented here would enable to optimize therapies directly for a minimized long-term fracture risk instead of maximized BMD gain. Thus, our model can serve as a quantitative starting point for the forecast of pharmacological therapies of osteoporosis but also highlights the role of mechanistic mathematical descriptions in understanding the biological principles of drug action.
Methods
Hybrid aging/treatment datasets
To create hybrid aging/treatment datasets, we merged a dataset comprising the BMD age dependence from Looker et al., 1998 with different clinical study datasets containing the BMD response to various medications (Appendix 3—table 2). The aging dataset from Looker et al., 1998 consisted of mean total femur BMD measurements in non-Hispanic white, non-Hispanic black, and Mexican American women, reported in 10-year age bins ranging from 20 to 80 years and older. We used bin averages as proxy BMD indicators for the center of the respective age window (Appendix 1—figure 1B). Rescaling the reported means for the three ethnic groups to their value for the earliest age bin revealed that relative changes in BMD were remarkably consistent among ethnic groups (Appendix 1—figure 1C) despite differing absolute baselines. Therefore, and since the model only addresses relative BMD changes, we resorted to the dataset with the largest underlying study population for calibration, which was the dataset comprising the non-Hispanic white female study population. Datasets on the response to medications from clinical trials on romosozumab, blosozumab, denosumab, alendronate, and teriparatide consisted of study population averages of total hip BMD and serum concentrations of one or more BTMs (CTX, P1NP, BSAP) during the treatment, and if available, during a follow-up period. Reported study population averages on the respective quantities were digitized directly from the data figures in the corresponding publications (Appendix 3—table 2).To merge aging and treatment datasets, the BMD from treatment datasets was rescaled such that the BMD baseline at treatment start corresponds to the linearly interpolated age-dependent BMD at treatment start. The treatment start was placed at the average age of the study population upon study start (rounded to full years) as reported in the respective publication (Appendix 1—figure 1D). BTM measurements were normalized to baseline values.The authors have developed a mathematical framework of drug interventions for postmenopausal osteoporosis using bisphosphonates, parathyroid hormone analogs, romosozumab, and denosumab. After calibrating and validating the model, authors demonstrated a predictive ability for complex clinical scenarios including sequential and parallel drug combinations. These data may be of great help in clinical practice.Our editorial process produces two outputs: (i) public reviews designed to be posted alongside the preprint for the benefit of readers; (ii) feedback on the manuscript for the authors, including requests for revisions, shown below. We also include an acceptance summary that explains what the editors found interesting or important about the work.Decision letter after peer review:Thank you for submitting your article "Modeling osteoporosis to design and optimize pharmacologic therapies comprising multiple drug types" for consideration by eLife. Your article has been reviewed by 1 peer reviewer, and the evaluation has been overseen by a Reviewing Editor and Mone Zaidi as the Senior Editor. The following individual involved in review of your submission has agreed to reveal their identity: Peter Pivonka (Reviewer #1).The reviewers have discussed their reviews with one another, and the Reviewing Editor has drafted this to help you prepare a revised submission.Essential revisions:(1) The discussion needs a lot of revision: it must be clear to authors that the manuscript will be read also by a lot of physicians and a clear, clinical language should be used.(2) In terms of bone density measurements authors should compare their data with those from Black et al., (JBMR 2018) from the SOF study and Dr Ensrud (JCEM 2022).The paper is very well written and presents promising results.The introduction is very comprehensive.The only additional reference I would suggest is:Lavaill et al., 2021 "Effects of PTH treatment in osteoporosis – insights from a mechanistic PK-PD model, BMMB, pp1-16, 2020." (https://doi.org/10.1007/s10237-020-01307-6)This paper explores the synergy between exercise and PTH treatment which is essentially a combined treatment. This would be one of the few papers that could be used in the Discussion section as comparison with the current results / model developments.Sentence containing "(BMD) in specific bone types";I suggest substituting "types" with "site" as you refer to hip, wrist, vertebra BMD.Essential revisions:(1) The discussion needs a lot of revision: it must be clear to authors that the manuscript will be read also by a lot of physicians and a clear, clinical language should be used.We appreciate the comment as it is our goal to make the manuscript as readable as possible for a large interdisciplinary audience. In response, in the revised manuscript, we have extensively revised the Discussion section, placing special emphasis on the clinical relevance of our findings and avoiding modelling-related jargon as much as possible. Also, in response to point 2, we have added a paragraph on how our findings can be related to fracture risk reduction, the prime clinical goal of osteoporosis therapy (see response below). Since there are many changes which affect the Discussion section as a whole, we refrain from listing each individual change here.(2) In terms of bone density measurements authors should compare their data with those from Black et al., (JBMR 2018) from the SOF study and Dr Ensrud (JCEM 2022).We thank the Reviewer for pointing us to the papers by Black et al., (JBMR 33, 389, 2018) and Ensrud et al., (JCEM, dgac324, 2022).First, we would like to recall that our model predicts the time course of the bone mineral density (BMD) for a given drug treatment or ageing scenario. In contrast, the publications by Black et al., and Ensrud et al., report fracture predictions based on BMD measurements. Relating BMD time courses generated by our model to fracture risk would require an additional (statistical) model that depends on details of the target demographic (as does the well-known FRAX tool, which is region-dependent and takes into account a variety of non-BMD related risk factors). This however is a major independent modelling effort and goes beyond the scope of the current approach. However, we believe that the above references (as well as other seminal studies relating BMD and fracture risk) are a perfect occasion to discuss the possibility of creating such a statistical model. In the Discussion section of the revised manuscript, we have therefore included the following new part:“Clearly, the goal of osteoporosis therapy is the reduction of fracture risk during and after therapy. While BMD has a prime role in the evaluation of osteoporosis therapies and can be measured rather easily using dual-energy x-ray absorptiometry (DXA), its relationship to fracture risk is complex. Fracture risk calculations used in clinical practice also involve demographic and lifestyle-related factors while mostly relying on BMD point measurements (Kanis et al., Bone 44, 2009). However, the quantitative associations between BMD, age and fracture risk reported in many studies (Kanis et al., Osteoporos. Int. 12, 2001; Berger et al., JBMR 24, 2009; Austin et al., JBMR 27, 2012; Krege et al., BoneKEy Rep. 2, 2013; Black et al., JBMR 33, 2018; Ensrud et al., JCEM 2022) can be used to create statistical models that may relate entire BMD time courses to a patient’s fracture risk. Combining such statistical models with the physiology-based model presented here would enable to optimize therapies directly for a minimized long-term fracture risk instead of maximized BMD gain.”The paper is very well written and presents promising results.The introduction is very comprehensive.We thank the Reviewer for the positive assessment of our manuscript.The only additional reference I would suggest is:Lavaill et al., 2021 “Effects of PTH treatment in osteoporosis – insights from a mechanistic PK-PD model, BMMB, pp1-16, 2020.” (https://doi.org/10.1007/s10237-020-01307-6)This paper explores the synergy between exercise and PTH treatment which is essentially a combined treatment. This would be one of the few papers that could be used in the Discussion section as comparison with the current results / model developments.We thank the Reviewer for pointing us towards this paper. Indeed, physical activity is an important contributor to bone remodelling. As our current model focuses exclusively on pharmacologic treatments and does not capture biomechanical feedback, a direct comparison of results with those by Lavaill et al., is not possible. Nevertheless, the perspective of including the effects of physical activity in an integrated modelling framework is attractive both from a modelling and a therapy standpoint. In the revised discussion, we have therefore added a part highlighting the possible role of physical activity on osteoporosis therapy:“Physical activity is another important contributor to bone remodeling, which we have not considered here. Detailed modeling approaches involving biomechanical feedback suggest synergistic effects between drug treatment of osteoporosis and physical activity (Lavaill et al., BMMB 19, 2020). Such results call for a further exploration of integrated approaches to osteoporosis treatment combining pharmacologic therapy and lifestyle adjustments.”We also mention the work in the introduction, where we discuss previous modelling of combination therapies.Sentence containing "(BMD) in specific bone types";I suggest substituting "types" with "site" as you refer to hip, wrist, vertebra BMD.We agree with the suggestion and have made the corresponding change in the revised manuscript.In our initial response to the Public Review, we also promised to describe better the rationale behind the simplified pharmacokinetic description of the various drug classes captured by the model. In the revised manuscript, we have modified and extended the following part in the beginning of Appendix B:“We include a dynamic description of several drug classes through separate model extensions, which depend on the functional drug concentration. The pharmacodynamic description is drug-specific and represents the individual mechanism of action of the respective drug class. For the pharmacokinetic description of each drug, we resort to simple first-order kinetics with drug-specific half-lives, which reduces the amount of model parameters. More detailed pharmacokinetic descriptions involve drug absorption and transfer between different body compartments, depending on the route of administration (oral, intravenous or subcutaneous). However, simulations of the calibrated model demonstrate that first-order kinetics yields an effective approximation of the pharmacokinetic features essential to capture a drug's long-term effects on bone remodeling, as suggested by comparisons of simulated and measured BTM concentrations (Supplementary Figures 2 and 3).”
Appendix 3—table 1.
Values of fit weights used in Equation 29.
Weight
Value
W(BMD)
300
W(CTX)
1
W(P1NP)
1
W(BSAP)
1
BMD, bone mineral density; CTX, C-terminal telopeptide; P1NP, procollagen type 1 amino-terminal propeptide; BSAP, bone-specific alkaline phosphatase.
Appendix 3—table 4.
Full list of parameters of the core model and the medication extensions.
Parameter
Description
Value
Unit
Origin
Model equation
Core model
ωC*
Reference pre-osteoclast to osteoclast differentiation rate
0.93
d-1
Calibration
Equation 10
eC*
Estrogen threshold for downregulation of pre-osteoclast to osteoclast differentiation
0.94
1
Calibration
Equation 10
sC*
Sclerostin threshold for upregulation of pre-osteoclast to osteoclast differentiation
8.60×106
1
Calibration
Equation 10
ηC
Reference osteoclast apoptosis rate
0.02
d-1
Calibration
Equation 13
eC
Estrogen threshold for upregulation of osteoclast apoptosis
0.99
1
Calibration
Equation 13
rC
Resorption signal threshold for upregulation of osteoclast apoptosis
10.10
1
Calibration
Equation 13
νC
Max. rel. effect of regulatory factors on osteoclast apoptosis
1.23×10-4
1
Calibration
Equation 13
ωB*
Reference pre-osteoblast to osteoblast differentiation rate
0.32
d-1
Calibration
Equation 10
sB*
Sclerostin threshold for downregulation of pre-osteoblast to osteoblast differentiation
1.63×102
1
Calibration
Equation 10
ηB
Reference osteoblast apoptosis rate
8.68×10-3
d-1
Calibration
Equation 13
ωB
Reference osteoblast to osteocyte conversion rate
6.24×10-4
d-1
Calibration
Equation 11
ηY
osteocyte apoptosis rate
1.10×10-4
d-1
Estimate
Equation 14
κs
Sclerostin degradation rate
0.05
d-1
Estimate; see Suen et al., 2015; Ominsky et al., 2015.
Equation 15
es
Estrogen threshold for downregulation of sclerostin secretion
9.60
1
Calibration
Equation 15
λC
Reference bone resorption rate per unit density osteoclast
3.82×10-6
d-1
Calibration
Equation 16
λB
Reference bone formation rate per unit density osteoblast
1.29×10-6
d-1
Calibration
Equation 16
sΩ
Sclerostin threshold for downregulation of bone formation
3.04×103
1
Calibration
Equation 16
rΩ
Resorption signal threshold for upregulation of bone formation
1.02×103
1
Calibration
Equation 16
νΩ
Max. rel. effect of the resorption signal on bone formation
1.08×102
1
Calibration
Equation 16
γ
Equilibration rate of the bone mineral content
6.65×10-3
d-1
Calibration
Equation 17
c0
Reference bone mineral content
0.80
1
Estimate
Equation 17
te
Onset of estrogen decline
50.00
y
Estimate
Equation 18
τe
Time scale of estrogen decline
2.60
y
Indep. fit (Appendix 1—figure 1A)
Equation 18
Bone turnover markers
qCTX
Exponent relating the bone resorption rate to CTX levels
1.16
1
Calibration
Equation 15
qP1NP
Exponent relating the bone formation rate to P1NP levels
1.45
1
Calibration
Equation 15
qBSAP
Exponent relating the bone formation rate to BSAP levels
0.92
1
Calibration
Equation 15
Medication extension: sclerostin antibodies
Eblosozumab
Efficacy: blosozumab
0.01
1
Calibration
Equation 21
Tblosozumab
Effective half-life: blosozumab
7.00
d
Tromosozumab
Equation 21
Eromosozumab
Efficacy: romosozumab
0.01
1
Eblosozumab
Equation 21
Tromosozumab
Effective half-life: romosozumab
7.00
d
Solling et al., 2018
Equation 21
δs
Sclerostin/antibody unbinding rate
0.05
d-1
κs
Equation 25
Medication extension: RANKL antibodies
Edenosumab
Efficacy: denosumab
4.34×103
1
Calibration
Equation 21
Tdenosumab
Effective half-life: denosumab
10.00
d
Bekker et al., 2004
Equation 21
βC*rAb
Max. rel. effect of RANKL antibodies on pre-osteoclast to osteoclast differentiation
0.87
1
Calibration
Equation 24
βbrAb
Max. rel. effect of RANKL antibodies on mineralization
0.02
1
Calibration
Equation 24
Medication extension: bisphosphonates
Ealendronate
Efficacy: alendronate
2.97×10-5
1
Calibration
Equation 22
Talendronate
Effective half-life: alendronate
1.53×102
d
Calibration
Equation 22
ηCbp
Max. contribution of bisphosphonates to osteoclast apoptosis rate
1.00
d-1
Calibration
Equation 26
Medication extension: PTH analogs
Eteriparatide
Efficacy: teriparatide
0.27
1
Calibration
Equation 22
Tteriparatide
Effective half-life: teriparatide
0.04
d
Satterwhite et al., 2010
Equation 22
βBpth
Max. rel. effect of PTH analogs on osteoblast apoptosis
1.31
1
Calibration
Equation 27
βC*pth
Max. rel. effect of PTH analogs on pre-osteoclast to osteoclast differentiation
Authors: Kristine E Ensrud; Li-Yung Lui; Carolyn J Crandall; Eric S Orwoll; Lisa Langsetmo; John T Schousboe; Howard A Fink; Nancy E Lane; Deborah M Kado; Jane A Cauley; Marcia L Stefanick; Peggy M Cawthon Journal: J Clin Endocrinol Metab Date: 2022-08-18 Impact factor: 6.134
Authors: Michael R McClung; E Michael Lewiecki; Stanley B Cohen; Michael A Bolognese; Grattan C Woodson; Alfred H Moffett; Munro Peacock; Paul D Miller; Samuel N Lederman; Charles H Chesnut; Douglas Lain; Alan J Kivitz; Donna L Holloway; Charlie Zhang; Mark C Peterson; Pirow J Bekker Journal: N Engl J Med Date: 2006-02-23 Impact factor: 91.245
Authors: Claudie Berger; Lisa Langsetmo; Lawrence Joseph; David A Hanley; K Shawn Davison; Robert G Josse; Jerilynn C Prior; Nancy Kreiger; Alan Tenenhouse; David Goltzman Journal: J Bone Miner Res Date: 2009-02 Impact factor: 6.741
Authors: T Kameda; H Mano; T Yuasa; Y Mori; K Miyazawa; M Shiokawa; Y Nakamaru; E Hiroi; K Hiura; A Kameda; N N Yang; Y Hakeda; M Kumegawa Journal: J Exp Med Date: 1997-08-18 Impact factor: 14.307