Rieko Sakurai1,2, Masao Ueki1,2, Satoshi Makino1,3, Atsushi Hozawa2,3, Shinichi Kuriyama2,3,4, Takako Takai-Igarashi2,3, Kengo Kinoshita2,5, Masayuki Yamamoto2,3, Gen Tamiya1,2. 1. Statistical Genetics Team, RIKEN Center for Advanced Intelligence Project, Tokyo, Japan. 2. Tohoku Medical Megabank Organization, Tohoku University, Sendai, Japan. 3. Graduate School of Medicine, Tohoku University, Sendai, Japan. 4. International Research Institute of Disaster Science (IRIDeS), Tohoku University, Sendai, Japan. 5. Graduate School of Information Sciences, Tohoku University, Sendai, Japan.
Abstract
BACKGROUND: Biobanks increasingly collect, process and store omics with more conventional epidemiologic information necessitating considerable effort in data cleaning. An efficient outlier detection method that reduces manual labour is highly desirable. METHOD: We develop an unsupervised machine-learning method for outlier detection, namely kurPCA, that uses principal component analysis combined with kurtosis to ascertain the existence of outliers. In addition, we propose a novel regression adjustment approach to improve detection, namely the regression adjustment for data by systematic missing patterns (RAMP). RESULT: Application to epidemiological record data in a large-scale biobank (Tohoku Medical Megabank Organization, Japan) shows that a combination of kurPCA and RAMP effectively detects known errors or inconsistent patterns. CONCLUSIONS: We confirm through the results of the simulation and the application that our methods showed good performance. The proposed methods are useful for many practical analysis scenarios.
BACKGROUND: Biobanks increasingly collect, process and store omics with more conventional epidemiologic information necessitating considerable effort in data cleaning. An efficient outlier detection method that reduces manual labour is highly desirable. METHOD: We develop an unsupervised machine-learning method for outlier detection, namely kurPCA, that uses principal component analysis combined with kurtosis to ascertain the existence of outliers. In addition, we propose a novel regression adjustment approach to improve detection, namely the regression adjustment for data by systematic missing patterns (RAMP). RESULT: Application to epidemiological record data in a large-scale biobank (Tohoku Medical Megabank Organization, Japan) shows that a combination of kurPCA and RAMP effectively detects known errors or inconsistent patterns. CONCLUSIONS: We confirm through the results of the simulation and the application that our methods showed good performance. The proposed methods are useful for many practical analysis scenarios.
Authors: Sara Muhammadullah; Amena Urooj; Muhammad Hashim Mengal; Shahzad Ali Khan; Fereshteh Khalaj Journal: Comput Math Methods Med Date: 2022-05-06 Impact factor: 2.809
Authors: Justin H Davies; Sarah Ennis; Hang T T Phan; Florina Borca; David Cable; James Batchelor Journal: Sci Rep Date: 2020-06-23 Impact factor: 4.379