Peter Hobson1, Brian C Lovell2, Gennaro Percannella3, Mario Vento4, Arnold Wiliem5. 1. Sullivan Nicolaides Pathology, 134 Whitmore street, Taringa, Queensland 4068, Australia. Electronic address: peter_hobson@snp.com.au. 2. School of Information Technology and Electrical Engineering, The University of Queensland, St Lucia, Queensland 4072, Australia. Electronic address: lovell@itee.uq.edu.au. 3. Department of Information Engineering, Electrical Engineering and Applied Mathematics, University of Salerno, Via Giovanni Paolo II, 132, Fisciano, SA I-84084, Italy. Electronic address: pergen@unisa.it. 4. Department of Information Engineering, Electrical Engineering and Applied Mathematics, University of Salerno, Via Giovanni Paolo II, 132, Fisciano, SA I-84084, Italy. Electronic address: mvento@unisa.it. 5. School of Information Technology and Electrical Engineering, The University of Queensland, St Lucia, Queensland 4072, Australia. Electronic address: a.wiliem@uq.edu.au.
Abstract
OBJECTIVE: This paper presents benchmarking results of human epithelial type 2 (HEp-2) interphase cell image classification methods on a very large dataset. The indirect immunofluorescence method applied on HEp-2 cells has been the gold standard to identify connective tissue diseases such as systemic lupus erythematosus and Sjögren's syndrome. However, the method suffers from numerous issues such as being subjective, time consuming and labor intensive. This has been the main motivation for the development of various computer-aided diagnosis systems whose main task is to automatically classify a given cell image into one of the predefined classes. METHODS AND MATERIAL: The benchmarking was performed in the form of an international competition held in conjunction with the International Conference of Image Processing in 2013: fourteen teams, composed of practitioners and researchers in this area, took part in the initiative. The system developed by each team was trained and tested on a very large HEp-2 cell dataset comprising over 68,000 images of HEp-2 cell. The dataset contains cells with six different staining patterns and two levels of fluorescence intensity. For each method we provide a brief description highlighting the design choices and an in-depth analysis on the benchmarking results. RESULTS: The staining pattern recognition accuracy attained by the methods varies between 47.91% and slightly above 83.65%. However, the difference between the top performing method and the seventh ranked method is only 5%. In the paper, we also study the performance achieved by fusing the best methods, finding that a recognition rate of 85.60% is reached when the top seven methods are employed. CONCLUSIONS: We found that highest performance is obtained when using a strong classifier (typically a kernelised support vector machine) in conjunction with features extracted from local statistics. Furthermore, the misclassification profiles of the different methods highlight that some staining patterns are intrinsically more difficult to recognize. We also noted that performance is strongly affected by the fluorescence intensity level. Thus, low accuracy is to be expected when analyzing low contrasted images.
OBJECTIVE: This paper presents benchmarking results of human epithelial type 2 (HEp-2) interphase cell image classification methods on a very large dataset. The indirect immunofluorescence method applied on HEp-2 cells has been the gold standard to identify connective tissue diseases such as systemic lupus erythematosus and Sjögren's syndrome. However, the method suffers from numerous issues such as being subjective, time consuming and labor intensive. This has been the main motivation for the development of various computer-aided diagnosis systems whose main task is to automatically classify a given cell image into one of the predefined classes. METHODS AND MATERIAL: The benchmarking was performed in the form of an international competition held in conjunction with the International Conference of Image Processing in 2013: fourteen teams, composed of practitioners and researchers in this area, took part in the initiative. The system developed by each team was trained and tested on a very large HEp-2 cell dataset comprising over 68,000 images of HEp-2 cell. The dataset contains cells with six different staining patterns and two levels of fluorescence intensity. For each method we provide a brief description highlighting the design choices and an in-depth analysis on the benchmarking results. RESULTS: The staining pattern recognition accuracy attained by the methods varies between 47.91% and slightly above 83.65%. However, the difference between the top performing method and the seventh ranked method is only 5%. In the paper, we also study the performance achieved by fusing the best methods, finding that a recognition rate of 85.60% is reached when the top seven methods are employed. CONCLUSIONS: We found that highest performance is obtained when using a strong classifier (typically a kernelised support vector machine) in conjunction with features extracted from local statistics. Furthermore, the misclassification profiles of the different methods highlight that some staining patterns are intrinsically more difficult to recognize. We also noted that performance is strongly affected by the fluorescence intensity level. Thus, low accuracy is to be expected when analyzing low contrasted images.
Authors: Jan Damoiseaux; Luis Eduardo Coelho Andrade; Orlando Gabriel Carballo; Karsten Conrad; Paulo Luiz Carvalho Francescantonio; Marvin J Fritzler; Ignacio Garcia de la Torre; Manfred Herold; Werner Klotz; Wilson de Melo Cruvinel; Tsuneyo Mimori; Carlos von Muhlen; Minoru Satoh; Edward K Chan Journal: Ann Rheum Dis Date: 2019-03-12 Impact factor: 19.103