Development and validation of a clinical diagnostic model for pregnant women with renal colic in the emergency department in China: a protocol for a retrospective cohort study.

INTRODUCTION: Urolithiasis affects many people throughout their lives. Among the maternal population, although the morbidity of acute urolithiasis in pregnant women is unremarkable, it is the leading cause of hospitalisation during pregnancy. There is no effective clinical diagnostic tool to help doctors diagnose diseases. Our primary aim was to develop and validate a clinical prediction model based on statistical methods to predict the probability of having disease in pregnant women who visited the emergency department because of urolithiasis-induced colic. METHODS AND ANALYSIS: We will use multivariate logistic regression analysis to build a multivariate regression linear model. A receiver operating characteristic curve plot and calibration plot will be used to measure the discrimination value and calibration value of the model, respectively. We will also use least absolute shrinkage and selection operator regression analysis combined with logistic regression analysis to select predictors and construct the multivariate regression model. The model will be simplified to an application that has been reported before, and users will only need to enter their clinical parameters so that risk probability is automatically derived. ETHICS AND DISSEMINATION: The review and approval documents of the clinical research ethics committee have been received from the ethics committee of our hospital (The Third Affiliated Hospital of Wenzhou Medical University). We will disseminate research findings through presentations at scientific conferences and publication in peer-reviewed journals. © Author(s) (or their employer(s)) 2022. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

This statistical model will be the first prediction model based on data from pregnant women with symptomatic stones, and this protocol will use the PROGRESS framework and follow Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis guideline.

This prediction model will be developed by systematically reviewing existing literature and referring to reported prediction model literature to meet clinical needs and to ensure consistency with existing clinical guidelines.

This study will be based on relevant literature, including a systematic review of existing prediction modelling and will select a broad series of relevant candidate predictors.

We will use least absolute shrinkage and selection operator regression analysis combined with logistic regression analysis to select predictors and construct the multivariate regression model.

It will lose partial useful data that can adversely affecting the results of the study due to such retrospective cohort study as clinical data were collected from previous electronic medical records.

Introduction

Urolithiasis affects many people throughout their lives. Recently, the trend of urolithiasis has increased, and there has been corresponding increases in outpatient visits and the financial cost of treating the disease.1 2 Diet, climate changes and some metabolic diseases such as diabetes and hypertension will lead to urolithiasis.3 Especially, this trend of urolithiasis has become more prominent in women than in men, whereas previously it was more frequent in men. The National Health and Nutrition Examination Survey database has reported that urolithiasis rates in men and women were 10.6% and 7.1%, respectively, and it was a prominent disease in middle-aged men.1 4 In the pregnant population, although the morbidity of acute urolithiasis is unremarkable, it still is the major reason for non-obstetric hospital admissions.3

Urolithiasis in pregnancy is an infrequent disease, but it can lead to many poor deliveries. Urolithiasis can cause premature rupture of the membranes, preterm labour and preterm birth in pregnant women who have symptomatic urolithiasis.3 5 6 The limitations of imaging methods, laboratory tests and clinical treatments make it difficult for emergency physicians to determine the treatment options, which will lead to misdiagnosis and potential risks of both the mother and fetus.5 7 8

In recent years, the multivariable clinical prediction model (CPM) has become increasingly popular in diseases diagnosis.9–12 As a single predictor is always insufficient to measure a valuable HR of disease, especially in some complicated situations, the CPM can combine multiple predictors to predict the incidence of related diseases or to classify the level of diseases, and it can indicate effective treatment for patients.12 13

The current diagnostic methods of urolithiasis in pregnant women are based on the subjective judgement of the physician/internist when ultrasonography cannot clearly show urolithiasis and radiological imaging methods are limited, because urolithiasis can lead to pregnancy loss or foetal anomalies.14 There is no effective clinical diagnostic tool to help doctors diagnose diseases. Even so, the CPM is still uncommonly used in clinical diagnosis because of its complicated operation or lack of credibility in diagnosing diseases.9 To resolve these issues, we will simplify the CPM by referring to the method published by Okita et al,12 verifying the sensitivity and specificity of the model, and conducting internal validation of the model.

Objectives

Primary objective

The primary objective of this study was to develop and internally validate a clinical diagnostic model that can be used not only to detect potential urolithiasis in pregnant women on a daily basis but also to help diagnose flank colic in pregnant women suspected of having urolithiasis in the emergency department.

Secondary objectives

We plan to conduct an epidemiological survey to analyse the incidence of urolithiasis in pregnant women in recent years. The epidemiological investigation will include the use of statistical methods to describe the demographic, clinical signs, imaging data and laboratory data of patients. We will compare the clinical data of healthy pregnant women with those of urolithiasis-positive pregnant women to identify additional risk factors that are strongly associated with urolithiasis.

Methods and analyses

Ethics statements

This study will be reported according to the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) statement, which is used to present studies concisely and transparently.15 We will use novel measurement methods introduced by Steyerberg et al and cited in many studies to verify the performance of the prediction model.9 16

Source of data and patient selection

This study will be a retrospective cohort study. The baseline data of patients will come from the database of HaiTai electronic medical records (Nanjing Haitai Medical Information System Co, Nanjing, China) at our hospital. First, we will screen patients who have been treated in the emergency department of our hospital from January 2009 to December 2019 for lateral abdominal pain or colic. Patients who have undergone renal and ureteral ultrasonography bilaterally, low-dose CT and MRI at our hospital will be included in our database. Second, we will exclude female patients who do not have results of a prenatal ultrasound examination in the same year. We will obtain all imaging records and allow a radiologist with national certification in health technology and a gynaecologist with a senior professional title access them without knowing the patient information and previous diagnosis; the two experts will not know each other’s diagnoses. Every pregnant woman’s electronic medical record will be reviewed to determine if symptomatic stones developed during pregnancy, and the integrity of the data will be reviewed. The positive cases will be used to establish the model. Meanwhile, the clinical data collected from negative and positive cases will be compared using t-test/χ2 test to identify novel predictors. Diagnostic criteria for urolithiasis will be based on the Chinese Guidelines for the Diagnosis and Treatment of Urological Diseases (Edited by Chinese Urology Surgery Branch, published by People’s Medical Publishing House V.2014). The name of the diagnosed disease will be based on the International Classification of Diseases, 10th Revision.

The inclusion criteria for positive patients will be as follows: (1) pregnant woman who were admitted to the hospital because of urinary tract obstruction or renal colic and diagnosed with urolithiasis after examination by two experts (a radiologist and a gynaecologist); and (2) pregnant women with documented clinical signs and laboratory data (all the clinical signs and laboratory data that will be used to diagnose urolithiasis are discussed in the Candidate predictors subsection). The exclusion criterion will be patients who do not meet the aforementioned inclusion criteria.

Sample size

To achieve lower bias and better performance of the prediction model, we will refer to the conclusion that Moons and Wolff drew in their paper published in 201917: the larger the sample sizes, the better the results (smaller SEs and narrower CIs). This is more important in prediction models than in other models, and we will also consider the actual number of patients we have received every year. We will set the events per variable to approximately 20, as Ogundimu et al recommended in their article.18 Based on the data we have collected from 2018 to 2019 (60 cases of symptomatic urolithiasis), we conservatively estimate that we will collect more than 500 clinical cases of symptomatic urolithiasis in pregnant women plus 500 confirmed cases of non-urolithiasis or symptomatic urolithiasis during pregnancy, and 1000 eases will be used for model training and validation. The number of patients used for training and validation will be shown in a flowchart.

Candidate predictors

We conducted a systematic review to find identified predictors that have been previously reported as risk predictors to assess the potential patients with urolithiasis or to diagnose the probability of urolithiasis when the patient visits the emergency department because of flank pain.5 12 19–21 We have consulted specialists in obstetrics and gynaecology, the imaging department and clinical laboratory for all predictors. Subsequently, we concluded that the following would be candidate predictors:

Presence of gestational diabetes.

Presence of gestational hypertension.

Gestational age.

Presence of urinary irritation symptoms.

Presence of nausea and/or vomiting.

Haematocrit level.

Albumin level.

Uric acid level.

Hypersensitive C reactive protein level.

Presence of haematuria.

Electrolyte levels (including potassium, sodium, chlorine, calcium, phosphorus and magnesium).

Presence of hydronephrosis and side of hydronephrosis.

Ultrasonic examination results and stone location.

History of calculi.

Presence of knocking pain in the area of the kidneys.

Type of pain and Numeric Rating Scale score for pain.

Smoking habit.

Alcoholism.

History of drug use.

Selection of predictors

The number of risk predictors needs to be appropriate, and multicollinearity among the predictors, as well as the applicability and ease of use of the final prediction model, needs to be considered. For these reasons, we will use least absolute shrinkage and selection operator (LASSO) regression analysis combined with logistic regression analysis to select predictors and construct the multivariate regression model. The LASSO method has superiority of selection of predictors, and it can manage collinear partial estimation as it adds a penalty term to the least square method to determine which predictors are most relevant to the outcome.22 The outcome of LASSO regression analysis will be presented in a graph. All clinical predictors will be collected from patients’ electronic medical records.

Data management

We have developed the following set of rules to improve the effectiveness of the patient data collected.

Positive cases should occur during gestation, neither before nor after gestation.

If patients have several records of visits due to symptomatic urolithiasis, the one with the most quantitative and qualitative data will be selected.

All clinical data during pregnancy will be collected, with variability of the clinical indicators considered.

Any positive signs that are present will be recorded as positive, even if the positive signs disappear later because of medication or other reasons.

Pregnant women with gestational hypertension and gestational diabetes mellitus should have corresponding case histories and be treated with relevant drugs.

All clinical data will be collected before the intervention or medication.

Imaging data of the affected side will be collected.

Continuous variables will be transformed into category variables according to statistical analysis if necessary. Both types of data will be presented in tables (continuous variables will be presented as mean±SD, and category variables will be presented as number and percentage). To ensure the integrity of the data of patients for the training and validating prediction models and to avoid coding errors, overall clinical data that are relevant to patients will be reviewed by two experienced clinical researchers in a password-protected medical data system. Both researchers have more than 4 years of clinical working experience.

Missing data

To solve the inevitable problem of missing values in the collection of patients’ data, we will use multiple imputation, which was based on the chained equation and recommended in many studies9 13 23 24 to resolve the problem of missing data at random. Additionally, we will exclude the data missing more than 80% for effectiveness of multiple imputation.9

Statistical analysis

We will use multivariate logistic regression analysis9 24 25 to establish a multivariate regression linear model to predict the risk of symptomatic urolithiasis during pregnancy. The discrimination value and calibration value of the model will be used to evaluate the performance of the model and will be presented as a receiver operating characteristic (ROC) curve plot and calibration plot, respectively. At the same time, we will use decision curve analysis (DCA) and combine the cut-off value obtained from the ROC curve to calculate the clinical net benefit value to analyse the clinical validity of the prediction model and solve the problems caused by misdiagnosis and missed diagnosis.

Construction of the risk prediction nomogram and model simplification

We will apply the covariable of each identified risk predictor obtained via logistic regression analysis to establish a diagnostic prediction model. We will refer to Okita et al’s previous idea12 to simplify the model into a mobile phone application. Users will only need to input several required clinical parameters, such as duration of pregnancy (in weeks), laboratory test data, etc, to automatically generate the probability of disease.

Patient and public involvement

Neither patients nor the public was involved in setting the research question or the outcome measures, designing the investigation or interpreting the data. There are no plans to involve patients in the dissemination of the results.

Discussion

The primary purpose of this study was to construct and validate a prediction model that can be used not only to detect potential urolithiasis in pregnant women on a daily basis but also to help diagnose flank colic in pregnant women suspected of having urolithiasis in the emergency department. Therefore, the positive population must be patients diagnosed with urinary tract stones and with obstructive symptoms. If a patient does not have symptoms of urinary tract obstruction, she will be classified as negative to distinguish whether the cause of renal colic in this patient is urinary tract calculi or other reasons. Although there is no direct relationship between urinary calculi and pregnancy, the model population that will be used in this study is pregnant women, which is of great significance in establishing a diagnostic model.

To our knowledge, this statistical model will be the first prediction model based on data from pregnant women with symptomatic stones, and this protocol will use the PROGRESS framework and follow the TRIPOD guideline. We will use some common laboratory data and easily observed clinical symptoms as risk predictors to predict disease risk. All candidate predictors used for risk prediction were previously reported in the existing literature3 5 12 19–21 26 to ensure the effectiveness of the model.

The calculating result of the model will be presented as a percentage, and the corresponding level of risk (low, medium or high) will be presented too. We will use more modern statistical methods (LASSO regression analysis) to screen for risk predictors. Regarding model performance validation, the ROC curve and calibration plot will be used to calculate the model’s discrimination value and calibration value, respectively. Moreover, DCA will be used to estimate the clinical validity of the prediction model and to resolve questions of misdiagnosis and missed diagnosis. The prediction model will be simplified as a mobile application for easy use and popularisation.

We will lose partial useful data that can adversely affect the study’s results due to the retrospective cohort study design, as the clinical data will be collected from patients’ electronic medical records. Problems in retrospective studies also include recall bias, coding errors and missing data. It is very important to use effective methods to solve the problems of missing values and recall bias to build prediction models.27 In this study, we will use multiple interpolation to solve the problem of missing data. To solve recall bias, we will obtain all the imaging data, and two qualified doctors will review the imaging results and make a blinded diagnosis. There are still too few patient-related clinical indicators included in our study, and more specific maternal molecular markers and imaging-related parameters can be added in the future.

This prediction model cannot be applied to people from other regions or other races because the training data used in this study was from a single centre. This weakness will need to be compensated for in the future by multicentre, multiracial data.

In the absence of imaging evidence, the diagnosis of urolithiasis in pregnant women remains controversial. Therefore, the diagnosis and treatment of disease can only rely on the subjective judgement and empirical treatment of clinicians. In this study, we will collect all imaging records and have radiologists with national certification in health technology and gynaecologists with a senior professional title access them and make a final diagnosis without knowing the patient’s previous diagnosis and the diagnosis made by another person.

The CPM can provide a relatively accurate prediction value for diagnosing or evaluating the stratification of diseases; however, many questions still remain, and all of them should be answered before the CPM can be used in clinical practice.14 The primary question is the determination of ending events. Emergency physicians should pay more attention to the consequences of making a diagnosis of urolithiasis in pregnant women with flank pain when radiology cannot be used.14 28 Additionally, the CPM should not only be confirmed by the statistical assessments but should also be useful in clinical application. Ultimately, the clinician needs to use the CPM and trust it.14 29

Ethics and dissemination

The examination and approval documents of the clinical research ethics committee have been received from the ethics committee of our hospital (The Third Affiliated Hospital of Wenzhou Medical University). We will disseminate research findings through presentations at scientific conferences and publication in peer-reviewed journals.