Jacob Jeries Abou-Hanna1,2, Chris A Andrews3,4, Naheed W Khan3, David C Musch3,4,5, K Thiran Jayasundera3. 1. Department of Ophthalmology and Visual Sciences, School of Medicine and Public Health, University of Wisconsin-Madison, Madison, Wisconsin, USA. 2. University of Michigan Medical School, University of Michigan, Ann Arbor, Michigan, USA. 3. Department of Ophthalmology and Visual Sciences, Center for Eye Policy and Innovation, University of Michigan, Ann Arbor, Michigan, USA. 4. Institute for Healthcare Policy and Innovation, University of Michigan, Ann Arbor, Michigan, USA. 5. Department of Epidemiology, School of Public Health, University of Michigan, Ann Arbor, Michigan, USA.
Abstract
Background: Several novel treatments of inherited retinal degenerations have undergone phase I/IIa clinical trials with limited sample size, yet investigators must still determine if toxicity or an efficacy signal occurred or if the change was due to test-retest variability (TRV) of the measurement tool.Materials and Methods: Synthetic datasets were used to compare three types of TRV estimators under different sample sizes, mean drift, skewness, and number of baseline measurements. Results: Mixed effects models underestimated the standard deviation of measurement error (SDEM); the unbiased change score estimator method (UBS) was more accurate. The fixed effect model had less bias and smaller standard deviation than UBS if >2 baseline measurements. The change score estimator had no bias; other estimators introduced bias for lower variability. With sample size <10, all estimators had high variance. With sample size ≥10, the differences between methods were often minimal. The pooled estimator model did not capture drift, whereas a fixed effect regression or mixed effects models accounted for drift while maintaining an accurate measure of variance. With small sample sizes, the bootstrap estimates of SDEM were severe underestimates, while the jackknife estimates were mildly low but much better. The jackknife was more accurate for the unbiased change score method than for the pooled estimator.Conclusions: The ideal phase I/IIa study has ≥20 subjects and uses UBS or its fixed effect model generalization if >2 baseline measurements. With non-ideal study parameters, investigators should at least quantify the error estimate present in their data analysis.
Background: Several novel treatments of inherited retinal degenerations have undergone phase I/IIa clinical trials with limited sample size, yet investigators must still determine if toxicity or an efficacy signal occurred or if the change was due to test-retest variability (TRV) of the measurement tool.Materials and Methods: Synthetic datasets were used to compare three types of TRV estimators under different sample sizes, mean drift, skewness, and number of baseline measurements. Results: Mixed effects models underestimated the standard deviation of measurement error (SDEM); the unbiased change score estimator method (UBS) was more accurate. The fixed effect model had less bias and smaller standard deviation than UBS if >2 baseline measurements. The change score estimator had no bias; other estimators introduced bias for lower variability. With sample size <10, all estimators had high variance. With sample size ≥10, the differences between methods were often minimal. The pooled estimator model did not capture drift, whereas a fixed effect regression or mixed effects models accounted for drift while maintaining an accurate measure of variance. With small sample sizes, the bootstrap estimates of SDEM were severe underestimates, while the jackknife estimates were mildly low but much better. The jackknife was more accurate for the unbiased change score method than for the pooled estimator.Conclusions: The ideal phase I/IIa study has ≥20 subjects and uses UBS or its fixed effect model generalization if >2 baseline measurements. With non-ideal study parameters, investigators should at least quantify the error estimate present in their data analysis.
Authors: Vaidehi S Dedania; Jerry Y Liu; Dana Schlegel; Chris A Andrews; Kari Branham; Naheed W Khan; David C Musch; John R Heckenlively; K Thiran Jayasundera Journal: Ophthalmic Genet Date: 2017-07-13 Impact factor: 1.803
Authors: Catherine Cukras; Henry E Wiley; Brett G Jeffrey; H Nida Sen; Amy Turriff; Yong Zeng; Camasamudram Vijayasarathy; Dario Marangoni; Lucia Ziccardi; Sten Kjellstrom; Tae Kwon Park; Suja Hiriyanna; J Fraser Wright; Peter Colosi; Zhijian Wu; Ronald A Bush; Lisa L Wei; Paul A Sieving Journal: Mol Ther Date: 2018-07-07 Impact factor: 11.454
Authors: Brett G Jeffrey; Catherine A Cukras; Susan Vitale; Amy Turriff; Kristin Bowles; Paul A Sieving Journal: Transl Vis Sci Technol Date: 2014-10-03 Impact factor: 3.283
Authors: Julia S Steinberg; Marlene Saßmannshausen; Maximilian Pfau; Monika Fleckenstein; Robert P Finger; Frank G Holz; Steffen Schmitz-Valckenberg Journal: Transl Vis Sci Technol Date: 2017-07-13 Impact factor: 3.283