BACKGROUND: New health status survey instruments are often described by their psychometric (measurement) properties, such as Validity, Reliability, Effect Size, and Responsiveness. For cluster-randomized trials, another important statistic is the Intraclass Correlation (ICC) for the instrument within clusters. Studies using better instruments can be performed with smaller sample sizes, but better instruments may be more expensive in terms of dollars, opportunity cost, or poorer data quality due to the response burden of longer instruments. METHODS: We defined the psychometric statistics in terms of a mathematical model, and examined the power of a two-sample test as a function of the test-retest Reliability, Effect Size, Responsiveness, and Intraclass Correlation of the instrument. We examined the "cost-effectiveness" of using a one-item versus a five-item measure of mental health status. FINDINGS: Under the standard model for measurement error, the psychometric statistics are all functions of the same error term. They are also functions of the setting in which they were estimated. In randomized trials, power is a function of Reliability and sample size, and a less reliable instrument can achieve the desired power if N is increased. In cluster-randomized trials, adequate power may be obtained by increasing the number of clusters per treatment group (and often the number of persons per cluster), as well as by choosing a more reliable instrument. The one-item measure of mental health status may be more cost-effective than the five-item measure in some situations. CONCLUSION: If the goal is to diagnose or refer individual patients, an instrument with high Validity and Reliability is needed. In settings where the sample sizes are large or can be increased easily, any valid instrument may be cost-effective. It is likely that many published values of psychometric statistics are accurate only in settings similar to that in which they were estimated.
BACKGROUND: New health status survey instruments are often described by their psychometric (measurement) properties, such as Validity, Reliability, Effect Size, and Responsiveness. For cluster-randomized trials, another important statistic is the Intraclass Correlation (ICC) for the instrument within clusters. Studies using better instruments can be performed with smaller sample sizes, but better instruments may be more expensive in terms of dollars, opportunity cost, or poorer data quality due to the response burden of longer instruments. METHODS: We defined the psychometric statistics in terms of a mathematical model, and examined the power of a two-sample test as a function of the test-retest Reliability, Effect Size, Responsiveness, and Intraclass Correlation of the instrument. We examined the "cost-effectiveness" of using a one-item versus a five-item measure of mental health status. FINDINGS: Under the standard model for measurement error, the psychometric statistics are all functions of the same error term. They are also functions of the setting in which they were estimated. In randomized trials, power is a function of Reliability and sample size, and a less reliable instrument can achieve the desired power if N is increased. In cluster-randomized trials, adequate power may be obtained by increasing the number of clusters per treatment group (and often the number of persons per cluster), as well as by choosing a more reliable instrument. The one-item measure of mental health status may be more cost-effective than the five-item measure in some situations. CONCLUSION: If the goal is to diagnose or refer individual patients, an instrument with high Validity and Reliability is needed. In settings where the sample sizes are large or can be increased easily, any valid instrument may be cost-effective. It is likely that many published values of psychometric statistics are accurate only in settings similar to that in which they were estimated.
Authors: Karen B DeSalvo; William P Fisher; Ky Tran; Nicole Bloser; William Merrill; John Peabody Journal: Qual Life Res Date: 2006-03 Impact factor: 4.147
Authors: Michael J Zvolensky; Lorra Garey; Thomas A Fergus; Matthew W Gallagher; Andres G Viana; Justin M Shepherd; Nubia A Mayorga; Lance P Kelley; Jackson O Griggs; Norman B Schmidt Journal: Psychiatry Res Date: 2018-08-29 Impact factor: 3.222
Authors: Jill Dawson; Helen Doll; Irene Boller; Ray Fitzpatrick; Christopher Little; Jonathan Rees; Andrew Carr Journal: Qual Life Res Date: 2008-10-29 Impact factor: 4.147
Authors: Paula H Diehr; Ann M Derleth; Stephen P McKenna; Mona L Martin; Donald M Bushnell; Gregory Simon; Donald L Patrick Journal: Health Qual Life Outcomes Date: 2006-04-25 Impact factor: 3.186
Authors: Ole M Ekeberg; Erik Bautz-Holter; Einar K Tveitå; Anne Keller; Niels G Juel; Jens I Brox Journal: BMC Musculoskelet Disord Date: 2008-05-15 Impact factor: 2.362
Authors: A Barnadas; M Muñoz; M Margelí; J I Chacón; J Cassinello; S Antolin; E Adrover; M Ramos; E Carrasco; M A Jimeno; B Ojeda; X González; S González; M Constenla; J Florián; A Miguel; A Llombart; A Lluch; M Ruiz-Borrego; R Colomer; S Del Barco Journal: J Patient Rep Outcomes Date: 2019-12-21