Jeffrey S Mohlman1, Jonathan R Genzen1, Ronald L Weiss1, Robert L Schmidt1.
Abstract
OBJECTIVES: To determine whether different laboratory developed test (LDT) risk stratification proposals would assign differing levels of risk to selected LDTs as a measure of the validity of those proposals, and whether there would be differing interrater agreement rates as a measure of the reliability of those proposals.
METHODS: A total of 4 reviewers applied 6 proposals for risk stratification of 4 LDTs. Interrater agreement was calculated as a measure of the reliability of the proposals. Also, a consensus risk categorization and concordance rate for each LDT was developed as a measure of the validity of the proposals.
RESULTS: Interrater agreement rates (reliability) ranged from 38% to 100%, and concordance rates (validity) ranged from 20% to 100%.
CONCLUSIONS: A spectrum of reliability and validity was observed depending on the policy used and the LDT categorized. Before implementation or legislation of risk-stratification methods, large evaluations of reliability and validity should be conducted on any proposed method. © American Society for Clinical Pathology 2018. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com.
OBJECTIVES: To determine whether different laboratory developed test (LDT) risk stratification proposals would assign differing levels of risk to selected LDTs as a measure of the validity of those proposals, and whether there would be differing interrater agreement rates as a measure of the reliability of those proposals.
METHODS: A total of 4 reviewers applied 6 proposals for risk stratification of 4 LDTs. Interrater agreement was calculated as a measure of the reliability of the proposals. Also, a consensus risk categorization and concordance rate for each LDT was developed as a measure of the validity of the proposals.
RESULTS: Interrater agreement rates (reliability) ranged from 38% to 100%, and concordance rates (validity) ranged from 20% to 100%.
CONCLUSIONS: A spectrum of reliability and validity was observed depending on the policy used and the LDT categorized. Before implementation or legislation of risk-stratification methods, large evaluations of reliability and validity should be conducted on any proposed method. © American Society for Clinical Pathology 2018. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com.
Keywords:
classification; laboratory developed test; measurement; reliability; risk assessment; validity
Mesh：
Year: 2019
PMID： 30169875 DOI： 10.1093/labmed/lmy052
Source DB: PubMed Journal: Lab Med ISSN： 0007-5027