Seung W Choi1, Wim J van der Linden2. 1. Pacific Metrics Corporation, 1 Lower Ragsdale Drive, Monterey, CA, 93940, USA. choi.phd@gmail.com. 2. Pacific Metrics Corporation, 1 Lower Ragsdale Drive, Monterey, CA, 93940, USA.
Abstract
PURPOSE: Most computerized adaptive testing (CAT) applications in patient-reported outcomes (PRO) measurement to date are reliability-centric, with a primary objective of maximizing measurement efficiency. A key concern and a potential threat to validity is that, when left unconstrained, individual CAT administrations could have items with systematically different attributes, e.g., sub-domain coverage. This paper aims to provide a solution to the problem from an optimal test design framework using the shadow-test approach to CAT. METHODS: Following the approach, a case study was conducted using the PROMIS® (Patient-Reported Outcomes Measurement Information System) fatigue item bank both with empirical and simulated response data. Comparisons between CAT administrations without and with the enforcement of content and item pool usage constraints were examined. RESULTS: The unconstrained CAT exhibited a high degree of variation in items selected from different substrata of the item bank. Contrastingly, the shadow-test approach delivered CAT administrations conforming to all specifications with a minimal loss in measurement efficiency. CONCLUSIONS: The optimal test design and shadow-test approach to CAT provide a flexible framework for solving complex test-assembly problems with better control of their domain coverage than for the conventional use of CAT in PRO measurement. Applications in a wide array of PRO domains are expected to lead to more controlled and balanced use of CAT in the field.
PURPOSE: Most computerized adaptive testing (CAT) applications in patient-reported outcomes (PRO) measurement to date are reliability-centric, with a primary objective of maximizing measurement efficiency. A key concern and a potential threat to validity is that, when left unconstrained, individual CAT administrations could have items with systematically different attributes, e.g., sub-domain coverage. This paper aims to provide a solution to the problem from an optimal test design framework using the shadow-test approach to CAT. METHODS: Following the approach, a case study was conducted using the PROMIS® (Patient-Reported Outcomes Measurement Information System) fatigue item bank both with empirical and simulated response data. Comparisons between CAT administrations without and with the enforcement of content and item pool usage constraints were examined. RESULTS: The unconstrained CAT exhibited a high degree of variation in items selected from different substrata of the item bank. Contrastingly, the shadow-test approach delivered CAT administrations conforming to all specifications with a minimal loss in measurement efficiency. CONCLUSIONS: The optimal test design and shadow-test approach to CAT provide a flexible framework for solving complex test-assembly problems with better control of their domain coverage than for the conventional use of CAT in PRO measurement. Applications in a wide array of PRO domains are expected to lead to more controlled and balanced use of CAT in the field.
Entities:
Keywords:
Content validity; Mixed-integer programming; Optimal test design; Patient-reported outcomes measurement information system (PROMIS); Shadow-test approach to CAT
Authors: Jin-Shei Lai; David Cella; Seung Choi; Doerte U Junghaenel; Christopher Christodoulou; Richard Gershon; Arthur Stone Journal: Arch Phys Med Rehabil Date: 2011-10 Impact factor: 3.966
Authors: Heleen Beckerman; Isaline Cjm Eijssen; Jetty van Meeteren; Marion C Verhulsdonck; Vincent de Groot Journal: Sci Rep Date: 2020-03-05 Impact factor: 4.379
Authors: Inger L Abma; Bas J D Butje; Peter M Ten Klooster; Philip J van der Wees Journal: Health Qual Life Outcomes Date: 2021-02-24 Impact factor: 3.186