A change-point regression approach for efficacy evaluation of dietary supplements.

In clinical trials for dietary supplements and functional foods, the study population tends to be a mixture of healthy subjects and those who are not so healthy but are not definitely diseased (called "borderline subjects"). For such heterogeneous populations, the t-test and ANCOVA method often fail to provide the desired treatment efficacy. We propose an alternative approach for the efficacy evaluation of dietary supplements and functional foods based on a change-point linear regression model. The model does not require the assumption of a constant treatment effect and provides clinically interpretable results. By employing the AIC-based profile likelihood method, inferences can be made easily using standard statistical software. The proposed method was applied to the Garcinia study data, and the merit of the method was demonstrated by comparing it with traditional methods.

Introduction

Dietary supplements and functional foods have been popular and are widely used. Just as with drugs, the efficacy and safety of dietary supplements and functional foods should be evaluated from a scientific viewpoint. One of the most desirable ways is to evaluate them based on data from randomized clinical trials. Various approaches such as randomization procedures, blinding and so on which are used in evaluating drugs in addition to most statistical methods that are used in drug development are quite useful for evaluating dietary supplements and functional foods. Among these approaches, the t-test and the analysis of covariance method (ANCOVA) are often applied for the efficacy evaluation of dietary supplements and functional foods.

Nagano et al. [1] and Tsuji et al. [2] applied the t-test in evaluating the efficacy, respectively, of diacylglycerol and medium-chain triacylglycerols in anti-obesity remedies; the change from a baseline for obesity-related parameters such as visceral fat area (VFA), body weight and body mass index were compared between two experimental groups. Nagano et al. [1] also applied ANCOVA for testing the decreasing effect of diacylglycerol on VFA. In these papers, the t test and ANCOVA were successfully applied, indicating the usefulness of these methods for evaluating dietary supplements and functional foods. However, these methods assume the effect of constant treatment on a study population. While an experimental drug may be expected to be effective regardless of baseline observations, this is not always true in evaluating dietary supplements and functional foods. The main reason for this difference is the fact that when a mixture of healthy subjects and those who not so healthy but not definitely diseased (called “borderline “subjects) are enrolled in a study, the magnitude of the efficacy among healthy subjects tends to be less than that with borderline subjects. This heterogeneity may cause a loss in the effectiveness of t tests and ANCOVA, and therefore the estimate of the treatment effect by assuming a homogenous study population may be misleading.

In this article, we propose an alternative approach to the t test and ANCOVA for evaluating dietary supplements and functional foods. Our method is based on a change-point linear regression, which does not require the assumption of a constant treatment effect. Our method assumes that the treatment effect is zero for subjects whose baseline value are less than the change-point and varies monotonically as the baseline value more than the change-point increase. Thus the change-point has a good interpretation that subjects with baseline value more than the change-point is potential candidate to have a benefit. We present a method to estimate regression parameters in a change-point regression analysis and propose a way to determine the change-point based on Akaike Information Criterion (AIC) based profile likelihood approach.

Materials and Methods

Suppose we are interested in analyzing data from a randomized and controlled clinical trial of dietary supplements or functional foods which was conducted under what is called pre-post design. More precisely, we consider a two-armed comparative study consisting of a control arm and an experimental arm with the endpoint being measured at the baseline and at the study’s end. Since the study endpoint may be strongly influenced by the values measured at the baseline, the treatment effect may depend on the level of baseline values shown in Fig. 1 (Panel A).

Fig. 1

Fitted regression lines for three models. Each panel shows measurements of subjects in the placebo (open circle) and the experimental groups (closed circle) with regression lines for the placebo (dashed line) and experimental groups (solid line). Regression lines in Panel A are by the Change-point regression model (Eq 1), those in Panel B are by ANCOVA (Eq 2) and those in Panel C are by ANCOVA with interaction (Eq 3).

We propose the following regression model, which takes into account differential baseline effects:

yi = β0 + β1xi + β2I (x>x)(x – x) g + ε (i = 1, 2, 3, …, n)    (Eq 1)

where n is the number of subjects, y is the observation at the study’s end of subject i, x is the observation at the baseline of subject i, x is a constant, g is the group indicator (control; 0, treatment; 1) and I (·) is an indicator function defined as I (x>x) = 1 if x>x and 0 is otherwise.

The regression lines for the two groups are the same for x less than x and different for x equal to or more than x. Then x is regarded as a change-point of the relationship of observations at the baseline and those at the study’s end; we call this model the change-point regression model (CPRM). If the CPRM fits the data well, and β2 is not equal to zero, the treatment is effective for subjects with observations at baseline greater than x. Thus, x is a useful indicator of whether the treatment provides benefits.

To estimate the regression coefficients and the change-point, we employ the maximum profile likelihood approach. For a fixed x, the CPRM is regarded as a special case of ordinal linear regression models. This implies that the regression coefficients can be easily estimated by the standard maximum likelihood methods for linear regression models and that one can obtain estimators with any standard software package which can handle linear regression models. We propose to apply the CPRM with various change-points x and select the optimal change-point by using AIC:

AIC = −2 maximum log-likelihood +2 p, where p is the number of unknown parameters of the model [3]. We select the model which minimizes AIC among candidate models.

The ANCOVA models

y = β0 + β1x + β2g + ε (i = 1, 2, 3, …, n)    (Eq 2)

or

y = β0 + β1x + β2g + β3x*g + ε (i = 1, 2, 3, …, n) (Eq 3)

are often applied to dietary supplements and functional foods data. The ANCOVA model (Eq 2) assumes that the effect of the treatment is homogenous over entire baseline observations and may be less appropriate for dietary supplements and functional foods data since healthy subjects may be influenced by the treatment (Fig. 1, Panel B). The ANCOVA model (Eq 3) has an interaction term that makes it impossible to summarize the effect of the treatment in a simple way (Fig. 1, Panel C). A frequently used t-test for measurement at the study’s end minus the baseline value is represented as

y – x = β0 + β1g + ε (i = 1, 2, 3, …, n)    (Eq 4)

, which becomes a special case of (Eq 2) by setting β1 = 1. Thus, ANCOVA adjusts the baseline more flexibly. AIC enables us to compare these analyses of the covariance models and the CPRM.

Application data

Garcinia (Garcinia cambogia), a plant native to Southeastern Asia, includes (−)-hydroxycitric acid [4]. HC (−)-HCA is has been shown to inhibit ATP-citrate lyase, blocking the conversion of citrate to acetyl-CoA, the first step in fatty acid synthesis [5]. Recently, Garcinia extracts containing HCA have been commonly marketed as dietary supplements for weight management [6-9].

We consider two anti-obesity studies of Garcinia [10, 11]. Both studies were double-blind, placebo-controlled trials conducted in Japan in which the efficacy was evaluated by VFA at the study’s end. The first study (called Study 1) was conducted to evaluate the efficacy and safety of Garcinia for subjects with Class 1 Obesity (BMI, 25–35 kg/m2). Note that in Study 1, VFA was not accounted for as inclusion criteria. The average of VFA at the baseline was 83.0 cm2 (range; 26.2–143.0 cm2). There were no differences between the two groups, but there was higher VFA in the subjects who had an initial VFA that was >90 cm2, the VFA in the Garcinia-treated group significantly decreased compared to placebo group [10].

The other study (called Study 2) was conducted to evaluate the efficacy and safety of Garcinia for subjects with a higher VFA at the baseline. In Study 2, VFA was used to define inclusion criteria; subjects with VFA greater than 90 cm2 were enrolled. The average of VFA at the baseline was145.5 cm2 (range; 90.4–244.3 cm2), and the Garcinia group had a significantly reduced VFA (without subgroup analysis) [11]. Details of Studies 1 and 2 are summarized in Table 1.

Table 1

Summary of Garcinia studies

	Study design	Subject diagnosis	Number of subject	Dosage in mg/day (duration)	VFA at baseline (range) (cm2)
Study 1	Randomized, placebo-controlled, double-blind	Overweight	40 (20 male, 20 female)	1000 mg (−)-HCA (8 weeks)	83.0 (26.2–143.0)
Study 2	Randomized, placebo-controlled, double-blind	Overweight or obese	39 (18 male, 21 female)	1000 mg (−)-HCA (12 weeks)	145.5 (90.4–244.3)

Except for the inclusion criteria regarding VFA, Studies 1 and 2 were conducted under a similar study procedure. In both studies, VFA at the study’s end was strongly influenced by that at the baseline. With the pooled dataset from Studies 1 and 2 (which included subjects with wider VFA at the baseline), we evaluated the efficacy of Garcinia with regard to VFA at the baseline. The relationship between VFA at the baseline and that at the study’s end was expected to be examined more precisely with the pooled dataset. Thus, we applied CPRM, which we proposed, to the pooled dataset in order to evaluate whether Garcinia is effective or not and with what VFA at the baseline subjects would be expected to respond to Garcinia.

Results

Fig. 2 (Panel A) shows AICs for various change-point regression models in the Garcinia study. The model with the change-point of 62.4 cm2 has minimum AIC among the CPRM and therefore is the most preferable. In Table 2, AICs for the linear regression model with only a group indicator as explanatory variables (t-test) and the ANCOVA model (Eq 2) and (Eq 3), as well as CPRM with the change-point of 62.4 cm2, are presented. Table 2 indicates that the CPRM fits better than the ANCOVA models. Thus, we selected the change-point regression model with the change-point of 62.4 cm2 as the final model.

Fig. 2

Result of Change-point regression model (CPRM) for Garcinia study (combined data). The open and closed circles mean placebo and treatment groups, respectively. Panel A is profile of AICs of CPRM with a change point of x: minimum AIC is attained at x is 62.4 (cm2). Panel B is Scatter plot of VFA at the study end versus Pre-VFA with regression lines for placebo (dashed line) and experimental groups (solid line) by CPRM with a change-point of 62.4 cm2 in Garcinia study: the treatment group was effective only for subjects with Pre-VFA greater than 62.4 cm2.

Table 2

AIC of CPRM and ANCOVA models for Gracinia study

Model	AIC
CPRM with xcp of 62.4 (Eq 1)	667.184
ANCOVA (Eq 2)	685.930
ANCOVA with interaction (Eq 3)	667.605
T-test (Eq 4)	684.969

Estimated regression coefficients are presented in Table 3. The interaction term is statistically significantly for values far from 0 (p<0.0001). It indicates that the CPRM fits well with the data. For values below 62.4, both groups showed the same slope (1.10506), meaning that for subjects with baseline measurement less than 62.4 cm2 the placebo and Garcinia groups did not change from their baseline measurements (Fig. 2, Panel B). For values over 62.4, only for the placebo groups did the slope not change. The slope of the Garcinia group declined to 0.78292, a decrease of approximately 32% compared to that of the placebo groups. This result implies that Garcinia is effective for subjects with baseline measurements greater than 62.4 cm2 and that the magnitude of its effectiveness is proportional to the baseline measurements.

Table 3

ANOVA table of CPRM with x of 62.4 in Garcinia study

	Estimate Coefficient	SE	t value	p value
β0	−6.884	4.827	−1.426	0.158
β1	1.105	0.044	25.032	<0.0001
β2	−0.322	0.051	−6.262	<0.0001

Discussion

In the nutrient field, CPRM has already been applied in previous studies. For example, Marini et al. applied it in vivo urea kinetic studies [12]. Other examples can be found in studies on amino acid in animal or human subjects [12-16]. Here CPRM was used to estimate nutrient requirements. Robbins et al. reported the growth response of young chicks to graded additions of L-histidine; they used CPRM (they called it the “broken-line model”) to estimate the L-histidine requirement [13]. Zello et al. used CPRM (they called it the “two-phase linear regression model”) and reported the dietary lysine requirement of young adult males by oxidation of L-[1-13C] phenylalanine [14].

In this paper, we proposed the use of CPRM to evaluate the efficacy of dietary supplements and functional foods data, especially in a two-armed comparative study. The CPRM is a natural approach for dietary supplements and functional foods in the sense that since these are used by both healthy people and “borderline” people, their efficacy may not be homogeneous among this entire population. We think that desirable dietary supplements have efficacy in mild ways and are unable to bring about a change that is equal to maintaining the body condition of healthy men. Their efficacy can be examined by applying CPRM and checking whether the change-point is within the normal range of the endpoint at the baseline.

While there are estimation methods available to estimate the change-point directly [17], they are quite complicated. We employed the profile likelihood approach and proposed using AIC to determine the change-point. The AIC-based profile likelihood approach is much simpler and can be easily conducted by any standard statistical software that can handle multiple regression models. Thus, our method has great potential for wide use in practice. In addition, CPRM also can provide onset information about the effects of the x value. This information cannot be provided by ANCOVA, t-test and other usual methods, and its great usefulness in planning future studies provides a further advantage for CPRM.

We mention the range of the change-points that are provided by the AIC. In Fig. 3 (Panel A), profiles of AICs are shown that were obtained by applying our method only to Study 1. Recall that the range of VFA at the baseline was 26.2–143.0 (cm2), which was much narrower than that for the combined data. AIC profiles for the change-points shown in Fig. 3 (Panel A) have an unusual profile with two minimals. The minimal around 120 cm2 may be due to a sample size that was too small for a baseline VFA greater than 120 cm2; in Study 1 there were only 8 subjects in both groups with such a baseline VFA. Thus, when CPRM is applied, subjects with baseline observations ranging sufficiently wide should be enrolled. The models with the change-points near the boundary should be excluded from candidate models. In other words, the range of the change-points should be determined in advance of the model selection. Future research should examine how to determine this range.

Fig. 3

Result of Change-point regression model (CPRM) for Garcinia study. Panel A is profile of AICs of CPRM with a change-point of x: minimum AIC is attained at x and is 68.7 (cm2). Panel B is Scatter plot of VFA at the study end versus Pre-VFA with regression lines for placebo (dashed line) and experimental groups (solid line) by CPRM with a change-point of 68.7 cm2 for Garcinia study: the treatment group is effective only for subjects with Pre-VFA greater than 68.7 cm2.

Finally we provide an interpretation of CPRM as a varying-coefficient model [18]. Consider a varying-coefficient model defined as

y = γ0 + γ1x + γ2 (x)g + ε (i = 1, 2, 3, …, n)   (Eq 5)

, where γ2 (x) is a function of x, representing the covariate-varying treatment effect. When γ2 (x) = β2, (Eq 4) reduces to ANCOVA (Eq 2) and when γ2 (x) = β2 + β3x, it reduces to (Eq 3). CPRM corresponds to γ2 (x) = β2I (x>x)(x – x). This is one of the simplest forms of the covariate –varying treatment effect and has a nice interpretation. By applying spline-based regression models or local-polynomial regression techniques, one can handle the varying coefficient models with general functions γ2 (x). However, in clinical trials for dietary supplements and functional foods, sample size is not always large, and simpler statistical models are preferable. Thus, our approach is especially attractive when the sample size is not necessarily large.

Conclusion

We propose an alternative approach for the efficacy evaluation of dietary supplements and functional foods based on a change-point linear regression model. By employing the AIC-based profile likelihood method, inferences can be made easily using standard statistical software. The proposed method was applied to the Garcinia study data, and the merit of the method was demonstrated by comparing it with the ANCOVA models.