Harmonization, data management, and statistical issues related to prospective multicenter studies in Ankylosing spondylitis (AS): Experience from the Prospective Study Of Ankylosing Spondylitis (PSOAS) cohort.

Ankylosing spondylitis (AS) is characterized by inflammation of the spine and sacroiliac joints causing pain and stiffness and, in some patients, ultimately new bone formation, and progressive joint ankyloses. The classical definition of AS is based on the modified New York (mNY) criteria. Limited data have been reported regarding data quality assurance procedure for multicenter or multisite prospective cohort of patients with AS. Since 2002, 1272 qualified AS patients have been enrolled from five sites (4 US sites and 1 Australian site) in the Prospective Study Of Ankylosing Spondylitis (PSOAS). In 2012, a Data Management and Statistical Core (DMSC) was added to the PSOAS team to assist in study design, establish a systematic approach to data management and data quality, and develop and apply appropriate statistical analysis of data. With assistance from the PSOAS investigators, DMSC modified Case Report Forms and developed database in Research Electronic Data Capture (REDCap). DMSC also developed additional data quality assurance procedure to assure data quality. The error rate for various forms in PSOAS databases ranged from 0.07% for medications data to 1.1% for arthritis activity questionnaire-Global pain. Furthermore, based on data from a sub study of 48 patients with AS, we showed a strong level (90.0%) of agreement between the two readers of X-rays with respect to modified Stoke Ankylosing Spondylitis Spine Score (mSASSS). This paper not only could serve as reference for future publications from PSOAS cohort but also could serve as a basic guide to ensuring data quality for multicenter clinical studies.

Introduction

Ankylosing spondylitis (AS) is characterized by inflammation of the spine and sacroiliac joints causing pain and stiffness and, in some patients, ultimately new bone formation and progressive joint ankylosis [1]. AS may also affect the hips and peripheral joints, as well as extra-articular sites such as the uveal tract, tendon insertions, proximal aorta and, rarely, the lungs and kidneys [2].

Multicenter or multisite prospective cohort studies for uncommon diseases such as AS provide an opportunity to increase enrollment [[3], [4], [5], [6]], improve the generalizability of the findings to the target population [[7], [8], [9], [10]], and promote a closer collaboration among a large group of investigators with diverse expertise [8,11,12]. The Prospective Study Of Ankylosing Spondylitis (PSOAS) cohort supported by the National Institute of Arthritis and Musculoskeletal and Skin disease (NIAMS), has been running since 2002, and includes five sites (4 US sites and 1 Australian site) with the overall aim of characterizing the role of genetic and non-genetic factors to susceptibility and outcome in AS. The Data Management and Statistical Core (DMSC) joined PSOAS cohort team in 2012 to assist in study design, provide an accurate, and a secure approach for data management, and appropriate statistical analysis of data.

Despite the aforementioned benefits of multicenter or multisite studies, there are challenges in harmonization, management, quality assurance, and statistical analysis of data from these studies [[13], [14], [15]]. The main objective of this article is to describe innovations in harmonizing, data quality assurance, and other key components involved in the coordination and management of data for a prospective multicenter cohort study of patients with AS.

Methods

Participating clinical sites, administrative and data management and Statistical Cores

The PSOAS cohort involves four clinical centers in the US and one in Australia, an Administrative Core, and a DMSC. The clinical sites in the US include the University of Texas McGovern Medical School at Houston (UTH), Cedars-Sinai Medical Center in Los Angeles, California (CSMC), The National Institutes of Health (NIH) Clinical Center in Bethesda, and University of California San Francisco (UCSF) Medical Center. The clinical site in Australia is housed at the Princess Alexandra Hospital (PAH) in Brisbane. The DMSC is housed in the Biostatistics/Epidemiology/Research Design (BERD) component of Center for Clinical and Translational Sciences (CCTS) at UTH. The DMSC assists in study design issues and in developing, maintaining, and a web-based information system for standardized data collection and management across the consortium including the training of personnel, data entry, quality assurance procedures, and data security. The DMSC is also responsible for study design and statistical analysis of data. The Administrative Core is housed in UTH, and works very closely with the clinical sites and the DMSC and coordinates various administrative responsibilities including communications with the sites and the DMSC. The complexity and varied needs of different study sites in the PSOAS cohort necessitated the need for a coordinating center for these sites. The Administrative Core addresses these needs and serve all of the projects in PSOAS cohort. This core administers and integrates the overall research program and coordinates various tasks among all projects. These tasks include handling the administrative oversight of contractual, budgetary, annual reporting and institutional review board (IRB) issues regarding the PSOAS cohort collaborating sites. It will also entail coordinating the collection of clinical data from the collaborating sites. Furthermore, it ensures regular communication among PIs of the projects and the cores, the co-investigators, other study personnel, and the internal and external scientific advisory board. The Administrative Core processes and stores biological samples and extracts DNA from different sites for shipment to the laboratories where the genotyping, biomarker, and cytokine analysis will be performed. Organization structure of the PSOAS cohort management is displayed in Fig. 1.

Fig. 1

P01-Genetics and Ankylosing Spondylitis (AS) pathogenesis.

Inclusion and exclusion criteria for PSOAS

Eligibility criteria for being considered for the PSOAS cohort is based on meeting the Modified New York criteria (mNY criteria) are described in (Table 1). The only exclusion criteria are not meeting mNY criteria [1,16].

Table 1

Eligibility criteria for PSOAS cohort.

❖Met Modified New York criteria (mNY criteria) for ankylosing spondylitis: ❖Clinical criteria1. Low back pain for at least 3 months duration improved by exercise and not relieved by rest 2. Limitation of lumbar spine motion in sagittal and frontal planes 3. Chest expansion decreased relative to normal values for age and sex ❖Radiological criteriaa. Unilateral sacroiliitis grade 3–4 b. Bilateral sacroiliitis grade 2–4 Definite ankylosing spondylitis if any radiological criteria (a or b) and any clinical criterion (1–3).

❖Clinical criteria

❖Radiological criteria

Modified New York criteria (mNY criteria)

The diagnosis of sacroiliitis in radiographs for the sacroiliac joint is the most important factor for diagnosis, classification, and monitoring of patients with Spondyloarthropathies (SpA) particularly for AS [1]. Among items listed in the mNY criteria (Table 1), two radiographic measurements: 1) unilateral sacroiliitis grade 3–4, and 2) bilateral sacroiliitis grade 2–4 are considered critical for the diagnosis of AS. The dynamic nature of AS status as it may be influenced by differences in the reading of the pelvic radiographs from each set of films, which can be affected by differences in angulation of the X-ray cone or the patient's posture, bowel gas, etc., adds to the complexity of managing data related to AS.

Harmonization

Harmonization is a systematic approach, which allows integration of data collected in multiple studies. Multicenter studies necessitate adherence to a standardized protocol with adequate quality assurance (QA) procedures that ensure integrity and quality of the data [17,18]. Although several reports provided important information regarding the type of variables that should be included in AS registries [[19], [20], [21], [22], [23], [24], [25], [26], [27], [28]], and data collected for single or multisite clinical trials or studies on AS [29], very few have discussed systematic approach to ensure data quality for their studies [21]. Furthermore, limited information exists regarding the challenges in harmonization and management of data from prospective multicenter studies in AS. As mentioned earlier, data from different studies are harmonized to develop the PSOAS cohort. The PSOAS cohort initially (2002–2006) comprised two different studies. 1) A cross-sectional study that focused on enrolling AS patients that had a long disease duration (>20 years from onset of inflammatory back pain (IBP)). Patients in the cross-sectional PSOAS study were initially enrolled beginning in 2002 for one visit where radiographs, limited clinical information and DNA were obtained until the protocol was amended to include a second follow up visit about 2–3 years after their initial enrollment. On their second visit radiographic, medications, and metrology data were captured. 2) In the longitudinal PSOAS study, the focus was on enrolling AS patients with disease symptom duration of <20 years. Local IRBs approved the protocols and consent forms for cross-sectional and longitudinal PSOAS studies.

Four centers (UTH, CSMC, Stanford, and UCSF) were initially involved with the cross-sectional PSOAS but the NIH site was added when the investigator at Stanford moved to the NIH clinical center in 2003. Enrollment of AS patients commenced in February 2003 and follow up visits were conducted every 4 months for a maximum of eight visits over 2–3 years in different sites until 2006. In 2007, the longitudinal PSOAS study was expanded to include an Australian center (the Princess Alexandra Hospital in Brisbane) and certain psychological instrument forms were added that were administered between 2007 and 2012. In addition, the interval between two consecutive follow up visits was changed from 4 months to 6 months. In 2012, the DMSC was added to the previous data management system and some study forms were updated, removed or changed. Details regarding the timeline, number of sites, and type of data pulled from different studies for the PSOAS cohort are displayed in Fig. 2.

Fig. 2

Timeline, sites, and type of data pulled from different PSOAS studies for the PSOAS cohort.

Data management

Data management involves development of processes to ensure systematic and efficient handling of research information as well as data quality assurance [3,30]. These include making improvements to the study forms, herein called Case Report Forms (CRFs), in consultation with the investigators, and developing procedures for data entry, quality assurance, and data security with the development of a web-based data collection in Research Electronic Data Capture (REDCap) [31] that ensures uniformity in methodology for collection of data by study sites [32,33]. REDCap maintains an audit trail that logs all activities by users which allows DMSC to monitor all the modifications made to data by any user. DMSC team developed a comprehensive database with all necessary fields for the various types of data to be longitudinally collected for the PSOAS cohort.

Standardization of data collection forms

As the first step in standardizing data collection forms members of the DMSC discussed the need for reviewing and improving the data collection forms. Based on prior changes in the forms during the PSOAS study (2003–2011), DMSC developed revised CRFs for the PSOAS cohort in 2012. For example, in some forms an option for identifying missing data was added to some variables in the revised CRFs to distinguish the difference between a response that was considered as real missing from a situation in that data were missing due being not applicable. The DMSC team circulated the revised CRFs, which were reviewed by the PSOAS cohort investigators and their feedback was incorporated in the final revised CRFs. The final copy of the revised CRFs was sent to the sites for IRB approval in each site. After IRB approval, each site started to capture data in the revised forms. In the following, we describe the type of data collected for the PSOAS cohort.

The revised CRFs comprised several forms including: i) background information questionnaire (BIQ), ii) proband (question related to symptoms and diagnosis for the AS patient and family members), iii) medical conditions or comorbidity (MDC), iv) radiographic, v) arthritis activity questionnaire (AAQ), vi) metrology, vii) laboratory, viii) medications, and ix) psychosocial questionnaire (PSY). However, in 2012 the site investigators decided to stop administering the psychosocial questionnaire to participants in PSOAS cohort. Details regarding the timelines for administering the aforementioned questionnaires for collection of data in the PSOAS cohort are provided in Table 2 and Fig. 2.

Table 2

Revised forms used for data collection and the timelines for administering the forms in the Prospective Study of Ankylosing Spondylitis (PSOAS) cohort.

Name of the forms	Changes in the revised CRFs	Time for collection data	Forms include the following information
I) Background Information Questionnaire (BIQ)	• Four different versions of BIQ form, with different variables in each study in the PSOAS cohort were used to revise to BIQ form as described in the following:I. Spondylitis information questionnaire (SIQ) (PSOAS cross sectional 2002–2005), which assessed the background information. II. Spondylitis assessment questionnaire (SAQ) (PSOAS cross sectional-follow up), which included the BASFI and BASDAI, comorbidities (Medical condition), surgeries in the past 2 years. III. Background information questionnaire (BIQ) (PSOAS longitudinal, before 2012) and the CES-D, BASFI and BASDAI were moved to AAQ. IV. Background information questionnaire (BIQ) (PSOAS longitudinal, after 2012)	Every 2 Years	• Captures background information • Years of education • Marital status • Employment status • Affected joints • History of surgery • List of medication used for treatment of problems related to AS • Biological drugs • Smoking data
II) Proband	Questions related to symptoms and diagnosis for AS patients (the probands) and family members that were collected in the PSOAS (cross-sectional and longitudinal cohort) study, were moved to BIQ in the revised CRFs.	Every 2 Years	• Health conditions related to AS for the affected patient (proband) • Health conditions for biological families (father, mother, children and siblings) that includes; iritis, psoriasis, ulcerative, Crohn's disease and back pain
III) Medical conditions or comorbidity (MDC)	Replaced the ICD codes with the name of diseases	Every 2 Years	• Medical problems: cardiovascular, lung, skeletal, gastrointestinal, nervous system diseases, and cancer
IV) Radiographic	No changes were made in the revised form	Every 2 Years	• Grading the lateral lumbar and lateral cervical spines using mSASSS [34] • Grading the sacroiliac (SI) and hip joints by Bath Ankylosing Spondylitis Radiology Index (BASRI) [35]
V) Arthritis Activity Questionnaire (AAQ)	Added AAQ measurement by two different methods:• Numeric Rating Scale (NRS) [36] • Visual Analogue Scale (VAS) [37]	Every 6 months	• Bath Ankylosing Spondylitis Functional Index (BASFI) [38] • Bath Ankylosing Spondylitis Disease Activity Index (BASDAI) [39] • Center for Epidemiologic Studies Depression Scale (CES-D) [40] • Pain, and exercise
VI) Metrology	Added Hip tender/swollen in the revised form.	Every 6 months	• Joint tenderness • Joint swelling • Tenderness at entheses and joint flexibility
VII) Laboratory	No changes were made in the revised form	Every 6 months	CRP and ESR
VIII) Medications	Classified into several categories [e.g., Non-steroidal anti-inflammatory drugs (NSAIDS), TNF Blockers, DMARD (Disease Modifying Anti-Rheumatic Drugs), etc.].	Every 6 months	• Information using of medication classified into several categories [e.g., Non-steroidal anti-inflammatory drugs (NSAIDS), TNF Blockers, DMARD (Disease Modifying Anti-Rheumatic Drugs), etc.]
IX) Psychosocial questionnaire (PSY)	Removed from the CRFs	Not in the revised CRFs	• Vanderbilt Pain Management Inventory (VPMI) [41] • Arthritis Self Efficacy Scales (ASES) [42] • Arthritis Helplessness Index (AHI) [43] • Brief Resilient Coping Scale (BRCS) [44] • Patient Health Questionnaire 9 (PHQ-9) [45] • International Physical Activity Questionnaire (IPAQ) [46]

Four different versions of BIQ form, with different variables in each study in the PSOAS cohort were used to revise to BIQ form as described in the following:

Development of databases

DMSC team developed several databases in REDCap [31], and is responsible for data entry, data quality assurance, security, and maintaining REDCap databases for the PSOAS cohort. The REDCap database was based on the approved CRFs by all the site investigators. Members of DMSC tested the database using real data before the PSOAS cohort data were entered in the REDCap. The medications form in REDCap was tested by Dr. Reveille. DMSC trained three Graduate Research Assistants before allowing them to enter data into the REDCap databases.

Data quality assurance

Data quality assurance involves a series of procedures that ensure the reliability and quality of data collected. These procedures include establishing intra- and inter-rater reliability of data collected. In addition, initial inspection of data in the PSOAS cohort prior to 2012, the period that the DMSC was not involved in the PSOAS study, revealed the need for establishing a systematic approach for data quality assurance procedures of all data in the PSOAS cohort.

Data cleaning

Data cleaning deals with identification and correction of erroneous data in databases. For example, missing, out of range values, and impossible data values can easily be identified using descriptive statistics, including frequencies, means, medians and box plots with interquartile ranges. However, even the most careful on-site review cannot identify unusual data patterns that are readily detected even by simple statistical methods. In some instances more sophisticated statistical methods will be required. For the PSOAS study, after uploading and entering data in the REDCap, DMSC developed and implemented program for data querying, reporting, and cleaning based on univariable and multivariable rules that detect potential data inconsistencies, missing, out of range values, and impossible data with SAS 9.4 [47]. For example, we developed a multivariable rule to detect unusual pattern in the sacroiliac joint scores (left and right) that were used for evaluating the mNY criteria. Specifically, any unusual fluctuations of sacroiliac joint scores either left or right between 3 consecutive radiographs visits triggered the need for review by the clinical investigators and its resolution. Since we did not expect this type of unusual fluctuations in radiographic data, this was considered as a new situation that arose during the study and our SAS program was updated to address this particular data quality issue.

Central statistical monitoring and communication with the study sites to resolve potential discrepancies in data

Central statistical monitoring is an alternative way for data monitoring in multisite studies that help to improve quality of data, while keeping costs under control compared with on-site source data verification [[48], [49], [50]]. However, central statistical monitoring has some limitations. For example; Central, central statistical monitoring may lack specificity and detect inconsequential data issues [50]. For the PSOAS cohort, DMSC team works very closely with the members of the Administrative Core to coordinate communications with the study sites and seek resolutions for potential data discrepancies that are identified during the process of ensuring data quality. Members of the DMSC have held weekly conference calls and regular e-mail communications with the project staff and investigators. DMSC also participated in the annual Investigators' meeting and provided information regarding data management and analysis issues in the PSOAS cohort.

Quality assurance (QA)

QA steps include the development of processes that ensure reliability of data. DMSC conducted comprehensive quality control (QC) for all data in the REDCap. It is important to emphasize that every piece of information collected in the CRF during the conduct of the clinical studies, and every variable coded in the clinical database is potentially indicative of data quality, not just those that may be associated with a set of indicators predefined to reflect site performance in terms of data quality. Therefore, this approach requires a large number of statistical tests to be performed.

For assessing the quality of data in REDCap database, DMSC developed and implemented rigorous procedure to identify potentially erroneous data and resolved potential discrepancies. These include identification of missing values, incorrect data type, out of range values, and outliers. In addition, DMSC developed data cleaning programs based on univariable and multivariable rules that detect potential data inconsistencies, missing, out of range values, and impossible data. DMSC conducted data quality checks by selecting random subsamples of different sizes ranging from 5 to 10% of the forms per visit and per site periodically. DMSC used double data entry procedure for these data, and identified discrepancies between the two entries, which were reviewed and adjudicated by the data manager at DMSC and project coordinators at study sites. Then DMSC randomly selected other subsamples that involved 5–10% of forms for various type of data and compared data between the source documents and data in REDCap database and calculated the error rate for various types of data in REDCap. The findings are reported in the Results section.

Reproducibility of radiographic data

The radiographic form contains one of the most important outcomes in this cohort study. First, these data are used to identify whether patients are qualified to be included in the cohort of AS patients based on disease progression and severity defined by mNY criteria. The radiographic data form includes grading of the lateral lumbar and lateral cervical spines on the score of 0–3 (no abnormality, erosion or sclerosis or squaring, syndesmophyte, and total bony bridging) and grading the sacroiliac (SI) and hip joints on the scale of 0–4 (for normal, suspicious, mild, moderate, and severe). Lateral lumbar and lateral cervical spine scores are captured in the modified Stoke Ankylosing Spondylitis Spine Score (mSASSS) form, whereas hip and SI joint involvement as well as BASRI lumbar and cervical scores are captured in the BASRI form. The mSASSS is an important score to determine severity and longitudinal progression of spinal disease and the BASRI score is a reliable method for grading radiographic changes in patients with AS, especially in the hips and SI joints, not evaluated by the mSASSS [35].

Since one of the primary outcomes in Project 2 is based on progression of radiographic damage and diagnosis of radiographic sacroiliitis is essential for study inclusion [34], the PSOAS cohort investigators designed a sub-study to establish the agreement of X-rays interpretations for this important variable. The X-rays of patients from the US sites were evaluated by Dr. Thomas J. Learch at CSMC site and, because of the lack of feasibility of transferring DICOM images due to cost, the radiographs from the Australian site were evaluated by Dr. Matthew Brown at PAH site. DMSC assessed the inter- and intra-rater reliability (IRR) based on data from a sub-study that was conducted to assess inter-rater reliability for two readers (Drs. Brown and Learch) and intra-rater reliability of two consecutive readings for the same patients by each reader separately. For a sample of 48 patients with AS, independent evaluation of radiographs was made by two readers at baseline and follow-up (3 or 5 years). A mixed effect negative binomial (NB) regression model [51] was used to estimate the degree of agreements for mSASSS total. The findings from this sub-study are reported in the Results section.

Utilization of data and requests for data analysis by DMSC

Guidelines for utilization of data and requests for analysis of multisite data managed by the DMSC are described in the Data Sharing and Publication Committee Guidelines. All participating site PIs as well as the PI of the DMSC serves as voting members of the Data Sharing and Publication Committee. The publication committee guidelines require that for utilizing the multisite PSOAS cohort data a two-page request for analysis of data to be circulated among the members of the Publication Committee. After receiving approval from a majority of members of the Publication Committee, the DMSC performs data analysis.

Statistical analysis

The statistical analyses are based on research objectives or hypotheses. However, most of these requests are expected to be based on longitudinal analysis of the cohort data. For example, for assessing the factors associated with longitudinal outcome (e.g., Opioid usage [52]), the DMSC statistician applied univariable [52] and multivariable longitudinal analyses using generalized estimating equation (GEE) methods or mixed effect regression models to account for the serial correlations in measurements within a patient [53]. In addition, there were challenges with handling missing data. For example, when the role of medications (e.g., Tumor Necrosis Factors inhibitors (TNFi) and Nonsteroidal Anti-Inflammatory Drugs (NSAIDs)) in longitudinal radiographic progression is assessed, missing medication data may lead to biased results. In this study, during 6-month follow-up visits, patients were asked to provide information about the last 6 months medications usage while their X-rays were collected every 2 years. Since radiographic progression is determined based on those two consecutive sets of X-rays, complete data for every medication that was taken between two X-ray visits is considered important but it was not available for some patients in this study.

The aforementioned statistical challenges provided great opportunities to develop and test innovative methods for analyses of complex data in this project. For example, we found that about 69% of patients had missing NSAID index data for at least one time point during their follow up visits (as of March 2017). Various imputation methods have been proposed and used to handle specific types of missing data, but many of those are likelihood-based methods that rely on estimation of means assuming normality of the data distribution. We found that NSAID index data in PSOAS cohort, which are defined by dose and frequency, contain many zero values, and are not normally distributed. The DMSC statisticians thus proposed a statistical method to deal with missing values in non-normal medication usage data (e.g., NSAID index) based on longitudinal Bayesian quantile regression (BQR) under latent class framework that properly incorporates unobserved heterogeneity in longitudinal medication usage data into the imputation processes. An application of the proposed imputation method to PSOAS data is illustrated by examining the longitudinal association between NSAID usage and radiographic damage for AS patients [54].

Another opportunity to develop innovative statistical methods in PSOAS arose when laboratory data (e.g., C-reactive protein data) were pulled together from different PSOAS studies (PSOAS-cross sectional and PSOAS longitudinal), for assessing their association with certain longitudinal outcomes over the follow-up time points. For example, we found that C-reactive protein (CRP), one of the primary biomarkers of disease activity, not only was censored due to limits of detection (LoD) but also missing for some patients during early study visits (i.e., PSOAS-cross sectional study) because blood sample collection was not required in the PSOAS-cross sectional study. This issue motivated development of a new approach to handle missing data while controlling for censoring due to LoD simultaneously. The DMSC statisticians proposed to develop a multiple imputation (MI) approach that is based on censored quantile regression (CQR) that accounts for censored data while applying the inverse probability weighting technique to properly deal with missing data in early visits. Details regarding this innovative approach are described elsewhere [55].

Results

Since 2002, 1690 participants have been screened for inclusion in the PSOAS cohort. Of these, 1272 (75.3%) met mNY criteria for study inclusion after review of the pelvic radiographs by the study radiologist. As expected, about 74% of qualified AS patients were male and 81.2% were white (Caucasian). Details regarding demographic and other characteristics of all participants in the PSOAS cohort are displayed in Table 3.

Table 3

Demographic and characteristics of all participants in PSOAS cohort, from 2002 to June 2017.

Study site	CSMC	NIH	UCSF	PAH	UTH	Total
Total No. of screened patients, n (%)	477 (28.2)	272 (16.1)	342 (20.2)	96 (5.7)	503 (29.8)	1690
Met mNY criteria (Qualified)c, n (%)	349 (73.2)	233 (85.7)	283 (82.7)	90 (93.8)	317 (63.0)	1272 (75.3)c
Did not qualify (DNQ), n (%)	125 (26.2)	34 (12.5)	49 (14.3)	6 (6.2)	180 (35.8)	394 (23.3)
Unknown status, n (%)	3 (0.6)	5 (1.8)	10 (2.9)	0 (0.0)	6 (1.2)	24 (1.4)
Sex, nd	349	233	283	90	317	1272
Female, n (%)	93 (26.6)	60 (25.8)	70 (24.7)	16 (17.8)	87 (27.4)	326 (25.6)
Male, n (%)	256 (73.4)	173 (74.2)	213 (75.3)	74 (82.2)	230 (72.6)	946 (74.4)
Ethnicity, nd	349	233	283	90	317	1272
Hispanic or Latino, n (%)	28 (8.0)	19 (8.2)	18 (6.4)	0 (0.0)	29 (9.0)	94 (7.4)
Not Hispanic or Latino, n (%)	321 (92.0)	214 (91.8)	265 (93.6)	90 (100.0)	288 (90.9)	1178 (92.6)
Racea, nd	349	233	283	90b	317	1272
White (Caucasian), n (%)	290 (83.1)	181 (77.7)	216 (76.3)	90 (100.0)	256 (80.8)	1033 (81.2)
Black or African-American, n (%)	12 (3.4)	20 (8.6)	2 (0.7)	0 (0.0)	15 (4.7)	49 (3.9)
Asian, n (%)	15 (4.3)	14 (6.0)	40 (14.1)	0 (00.0)	17 (5.4)	86 (6.8)
Native American Indian or Alaskan Native (Hispanic), n (%)	25 (7.2)	17 (7.3)	11 (3.9)	0 (0.0)	22 (6.9)	75 (5.9)
Native Hawaiian or Other Pacific Islander, n (%)	0 (0.0)	0 (0.0)	0 (0.0)	0 (0.0)	0 (0.0)	0 (0.0)
More Than One Race (Mixed), n (%)	7 (2.0)	1 (0.4)	13 (4.6)	0 (0.0)	7 (2.2)	28 (2.2)
Unknown or Not Reported, n (%)	0 (0.0)	0 (0.0)	1 (0.4)	0 (0.0)	0 (0.0)	1 (0.1)

Sites: UTH: University of Texas McGovern Medical School at Houston; CSMC: Cedar Sini Medical Center in Los Angeles, California; NIH: The National Institutes of Health Hospital; UCSF: University of California San Francisco Medical Center; PAH: Princess Alexandra Hospital in Brisbane, Australia.

Racial categories definition in this study is based on that of NIH in the US [56]; (Racial categories was missing for one patient from UCSF).

In order to reduce ethnic heterogeneity in genetic data, the Australian site (PAH) selected only white Europeans.

SAA (Spondylitis Association of America) and PA (Palo Alto University (Stanford), California) patients are not included.

All result are based on number of participates that meeting the Modified New York (mNY) criteria.

With assistance from the site investigators DMSC conducted a comprehensive QC for all data in REDCap. For example, DMSC checked 80,340 medication data values for QC and found that error rate in these data was 0.07%. Details regarding the error rate for other variables in REDCap are reported in Table 4.

Table 4

Error rate for each form used in PSOAS cohort.

Questionnaires	Total number of data points checked	Error rate (%)
BIQ	32,565	0.52
MDC (Comorbidity)	8612	0.29
Proband	29,180	0.30
HLA	3976	0.13
AAQ-BASFI (VAS)	1060	0.85
AAQ-BASFI (NRS)	1180	0.08
AAQ-BASDAI (VAS)	651	0.61
AAQ-BASDAI (NRS)	809	0.62
AAQ-CES-D (VAS)	3511	1.00
AAQ-Global and pain	2372	1.10
AAQ-Exercise (VAS)	3055	0.72
Lab (CRP, ESR)	2496	0.44
MED (Medication)	80,340	0.07
Radiographic	15,240	0.37
Psycho-social-PHQ	1210	0.74
Psycho-social-IPAQ	1710	0.53
Metrology	2250	0.71

The findings from the reproducibility of radiographic data sub-study based on 48 patients with AS indicated a strong level of agreement between the two readers with respect to mSASSS. Specifically, based on liner mixed effects model, the inter-rater reliability for mSASSS total was IRR = 0.90 (95% confidence interval (CI) = [0.82, 0.94]), indicating a strong level of agreement between the readings from Drs. Brown and Learch. In addition, inter-rater reliability for the change in mSASSS total (between two time points for each patient) showed a high level of agreement of two readers (IRR = 0.83, 95% CI = [0.72, 0.90]). For each reader (Dr. Learch and Dr. Brown) the intra-rater reliability (or intra-class correlation (ICC)) and 95% CIs for mSASSS total based on the repeated data within the same patients at baseline were 0.83 [0.72, 0.90] and 0.93 [0.88, 0.96], respectively. These data indicate an excellent level of intra-reader reliability (or consistency) for each of the two readers.

Discussion

The first step towards data integration and harmonization is usage of a data standard [57]. National Institute of Health (NIH) recommends the use of common data elements (CDEs) [58], from which CRFs can be developed. Investigators can standardize data collection, follow-up comparison of results, and can combine sets of CDEs consisting of individual or more complex questionnaires, across multiple studies [59]. In our PSOAS cohort, where we had multiple sites from two countries (4 US sites and 1 Australian site) and a follow up of more than a decade, some of the items in some forms were changed over time as described in Table 2. The study team reviewed all the forms and developed a CRF that allowed addition, elimination, or partial changes to the questions. The revised CRF was reviewed by all site investigators and DMSC before it was finalized and sent to the sites for implementation. The revised CRF was used as the basis for developing a REDCap database. Whenever necessary, special features were added in REDCap to facilitate data entry from the old and the new forms. For example, due to differences in the interval between the two consecutive visits in the PSOAS cohort (before 2007 every 4 months) and after 2007 (every 6 months), we harmonized the timing of data collected in different studies. Although for statistical analysis we mainly focused on a common set of variables, the aforementioned arrangement allowed additional flexibility for utilizing various subsets of our data set from different periods of follow up in our cohort.

QA steps include the development of processes that ensure reliability of data, which include establishment, and implementation of procedures that ensure the quality of data [30]. Minimizing errors is an important objective of QA processes [60,61]. In the following, we discuss quality assurance of PSOAS cohort data with a particular focus on data quality and reliability of reading radiographs.

In this study, we used available features in REDCap for assuring data quality and validation of data. These include identification of missing values, incorrect data type, out of range values, and potential outliers. In addition, we developed programs for data querying, reporting and cleaning, based on univariable and multivariable rules that detect potential data inconsistencies, missing, out of range values, and impossible data. We conducted data quality checks by selecting random subsamples of different sizes ranging from 5 to 10% of the forms per visit and per site, and used double data entry procedures for these data to identify discrepancies between the two entries, which were reviewed and adjudicated by the data manager at DMSC and project coordinators at study sites. We calculated the error rate by comparing the source documents and data in REDCap database based on randomly selected subsamples that involved 5–10% of forms for various types of data. We found an overall error rate of 0.07% for medication data. In the PSOAS cohort, the PI of the study accepted the responsibility for data entry and quality assurance of medication data, which required someone with a medical background to interpret what was written down on the patient generated forms. Although some studies suggested an acceptable error rate of 0.1%–0.5% for clinical trials [62,63], for observational studies a higher estimate for the error rate has been reported, ranging from 1% to 5% for general databases [64].

Reliability of radiographic readings

In our sub study, data for forty-eight patients indicated a strong level (89.9%) of agreement between the two readers. Similarly, a study in Italy showed that inter-observer reliability for mSASSS scores were more reliable for intra- and inter-observer, with ICC of 0.87 and 0.94, respectively [65]. A study in Turkey, reported a high agreement for mSASSS intra and inter-rater reliability with ICC between 0.86 and 0.99 [66]. However, Cortes et al. (2015) assessed the inter-reader reliability for radiographs on Australo-Anglo-American Spondyloarthritis Consortium (TASC) data by 22 radiographs selected that included a cross-sectional set (from 10 patients) and a longitudinal set (two time points) of radiographs (from 6 patients), which were evaluated by four readers that included Drs. Brown and Learch. They modified mSASSS using two different ways of scoring. In “version A” of scoring, they modified classical mSASSS score of 3 to 2, scores of 2 to 1 and scores of 1 to 0. In “version B”, they modified classical mSASSS of 3 and 2 to 1 and scores of 1 to 0, and compared with mSASSS prior to making modifications and showed that the inter-reader agreement improved from 69.7 to 81.4% [67].

Standardizing medication data

The medications form provided two different major issues. First, because the forms were patient-generated there were occasional missing data, particularly information about the doses. DMSC created a comprehensive database for medication used by participants in this cohort study by classifying medications used (e.g., TNF, NSAIDS, etc.), dosage, and route for each medication. Since interpretation of these data required clinical expertise (such as knowledge of correct medication spelling, pill size and FDA approved dosing intervals), Dr. Reveille (PI of the project) accepted the responsibility of data entry and adjudication of these data based on a set of rules which were reviewed and approved by all PIs and co-investigators in the PSOAS cohort. Dr. Reveille entered all medication data in the study based on these approved rules. A list of rules used for adjudication of medication data in REDCap as well as the data quality assurance procedures for these data are provided as “Supplementary Information”. As an example of these rules, if the patient did not provide information about the number of months a drug has been taken in the past 6 months, Dr. Reveille imputed this information based on the following rule: a) entered 6 months, how and if the drug was used previously as indicated in the prior visit; b) entered 3 months, if the drug is newly reported by the patient.

The second important challenge with some medication data was that the names and doses of some medications from the Australian site were different from names of similar medications used in the US sites. For standardizing medication data across the sites, DMSC reclassified medication names in categories recommended by the investigators. For example, all medications used as nonsteroidal anti-inflammatory drugs (NSAIDs) were classified under NSAID category. In addition, for AS patients the amount of NSAIDs taken (NSAID Index) is considered important in relation to certain outcomes [68]. For AS patients who had complete medication history, DMSC used the formula published by Dougados et al. (2011) [68] for calculation of NSAID index, which is based on: 1) the type of NSAID, 2) the dose, and 3) the percentage of days with intake over a certain period of time [68]. For AS patients who did not provide complete data for their medication use, Dr. Reveille adjudicated missing information on the patient generated forms for calculation of the NSAID index based on a set of rules that were reviewed and approved by the investigators.

Standardizing metrology data

In each visit, patients underwent a physical exam to check for joint tenderness, joint swelling, tenderness at specific enthuses, and joint flexibility. This examination was performed by the study investigators (at three sites) or trained metrologists (at two sites). Two standardization exercises were conducted with all the investigators examining the same patients after reviewing the appropriate method to carry them out in 2003 and 2005. Differences between measurements were reviewed and discussed to minimize inter-observer variability.

Conclusions

In this study, we described the PSOAS cohort with a particular focus on the responsibilities of DMSC including data management and statistical analysis. Specifically, we have described harmonization of data from different phases of PSOAS cohort. In addition, we provided information regarding our systematic approach for ensuring data quality. Our quality assurance procedures comprised improving the CRFs, developing a database using REDCap, establishing reliability of assessment of X-rays as well as development and implementation of univariable and multivariable data monitoring rules to ensure data quality. We demonstrated an excellent reliability and excellent data quality in PSOAS cohort. The error rate for data in various forms of PSOAS ranged from 0.07% -1.10%, and was in the acceptable range of 1–5% for observational studies. Central statistical monitoring can both optimize on-site monitoring and improve data quality, and as such provides a cost-effective way of meeting regulatory requirements for multicenter clinical studies. We also introduced methodologic developments that were motivated by statistical challenges raised in PSOAS study. This paper not only could serve as a reference for future publications from PSOAS cohort, but also could serve as a basic guide to ensuring data quality for other multicenter clinical studies.