Analysis of agreement between measurements of continuous variables: general principles and lessons from studies of imaging of carotid stenosis.

Clinical research often involves measurement of continuous variables. However, clinical measurements are seldom precise. It is frequently necessary, therefore, either for the reproducibility of measurements to be assessed (observer agreement studies), or for measurements made by different techniques to be compared (method comparison studies). There are numerous ways in which data can be analysed and reported in such studies, and several pitfalls. In order to determine which methods are commonly used in the medical literature, a systematic review of studies involving measurement of carotid stenosis was performed. A random sample of 40 studies were selected for detailed assessment. The methods of analysis of reproducibility of measurement of stenosis and/or comparison of two alternative techniques of measurement were recorded. Ten different methods were identified. The advantages and disadvantages of each of the methods are discussed in a non-technical and non-mathematical manner, and illustrated using data from a study of measurement of carotid stenosis by two observers on 1001 carotid angiograms.