Nicotine and cotinine in oral fluid: Passive exposure vs active smoking.

Scheidweiler and colleagues have clinically tested and identified a reporting cutoff (10 ng/mL) of nicotine and cotinine in oral fluid that could reliably determine active smoking in patients. The results from that study were reevaluated using a large data set of oral fluid nicotine and cotinine results available from pain medication monitoring. Additionally, test results from patients using a nicotine transdermal patch delivery device are compared with those from smokers. Finally, oral fluid test results collected over a 2-year period were normalized and transformed to yield a near Gaussian distribution for nicotine. The normalized and transformed data reveal the presence of two independent populations: a larger population consistent with active smokers and a smaller population consistent with those passively exposed to smoke. Furthermore, application of this model to patients prescribed transdermal nicotine reveals oral fluid levels consistent with those of active smokers. The clinical delineation of smokers from non-smokers reported earlier is supported by the oral fluid nicotine data modelling presented herein. These data indicate that oral fluid is an acceptable sample matrix for determining the smoking status of patients. Further, these data indicate that oral fluid test results are indistinguishable between patients prescribed transdermal patches and active smokers; however, oral fluid testing can determine absence of patches or smoking.

Introduction

Nicotine and its metabolites can be tested in oral fluids, urine, and blood to discern the use of tobacco products [1]. Nicotine is also used as a pharmacological aid to smoking cessation [2] often via transdermal delivery. As reported in earlier publications [3], [4], oral fluid testing is gaining prevalence because of the ease with which samples can be collected without the need for specialized professionals or dedicated facilities.

Apart from identifying active smokers, clinicians periodically request nicotine testing to monitor patients on transdermal nicotine cessation therapies. Cutoff levels for cotinine in plasma, saliva, and urine have been proposed [5], [6]. However, similar information for nicotine has not been documented in urine or plasma. The clinical trial reported by Scheidweiler, et al. [1] in oral fluids has, to our knowledge, been the only successful attempt at benchmarking distinct cutoff concentrations of nicotine and cotinine in a biological fluid to differentiate between smokers and non-smokers. That study examined four groups of patients for oral fluid levels of nicotine and several of its metabolites. The groups in question included: Group A—46 nonsmokers, Group B—36 nonsmokers exposed to secondhand smoke, Group C—44 light smokers (≤ 10 cigarettes/day) and Group D—46 heavy smokers (>10 cigarettes/day). Their results indicated that a reporting cutoff of 10 ng/mL for both nicotine and cotinine concentrations would capture 88% and 98% of active smokers, respectively [1].

Nicotine and cotinine test results from oral fluid samples are presented in this report. Values are testing results from patients who either smoked or used transdermal products. Reporting cutoffs from the earlier report [1] are applied to these data sets and compared to assess relative levels from these separate delivery mechanisms.

In an effort to define “normal” populations of smokers vs passive nicotine exposure, nicotine smoking data were modelled as per the report of Cummings, et al. [7] and compared with the reporting cutoffs from the earlier clinical study [1]. Finally, using this modelling approach, test results from patients who smoked are compared with those from patients who used transdermal nicotine patches.

Materials and methods

All specimens used in this analysis were de-identified. Ameritox, LLC, is accredited by the College of American Pathologists (CAP) and abides by CAP, Clinical Laboratory Improvement Amendments (CLIA), and Health Insurance Portability and Accountability Act (HIPAA) requirements. Due to the secondary analysis nature of this work and the absence of clinical conclusions, neither U.S. Food and Drug Administration (FDA) nor other clinical trial review/approval was obtained by Ameritox, LLC. Writing this manuscript did not involve human subjects as defined by the U.S. Code of Federal Regulations (45 CFR 46.102); thus, an IRB approval of these specific research activities was not necessary.

Reagents

All reference standards were purchased from Cerilliant (Round Rock, TX, USA). All solvents, including methanol (optima grade), acetonitrile (optima grade), isopropanol (optima grade) and formic acid (88%), were purchased from VWR (Radnor, PA, USA). Negative synthetic saliva was obtained from Immunalysis (Pomona, CA, USA).

Sample collection and preparation

Patient samples were collected at various clinics in the course of pain medication management testing. Some information about the sample groups is given in Table 1. Patient samples were collected, diluted, and prepared as per a standard protocol. These samples were collected using a Quantisal™ sampling device (Alere, Waltham, MA) which uses a collection pad to gather 1 mL (±10%) of sample before being submerged in 3 mL of buffer to ensure stability. Once accessioned in the laboratory, the pad is wrung out to yield approximately 4 mL of total 4 × diluted sample. 500 μL of patient specimen is pipetted into labeled tubes containing 50 μL internal standard (amphetamine D5 at 500 ng/mL). These diluted samples are vortexed and centrifuged for 5 min at ~8000 g. The samples are then extracted using a Phenomenex Trace B solid phase extraction cartridge (Phenomenex, Torrance, CA). Reference standards were diluted to appropriate calibrator level concentrations (0.25, 0.5, 2, 10, 50, and 250 ng/mL) in negative synthetic saliva. The curve points were then prepared by adding 500 μL of standard into 50 μL internal standard. It is notable that results on these samples must be multiplied by 4 in order to account for the upfront 4× dilution in buffer to obtain the true “in mouth” concentration [8]. Samples with a concentration higher than the upper limit of linearity (ULOL) were further diluted and re-analyzed.

Table 1

Drug and Demographic Information Summary of Patient Population Used to Develop the Oral Fluid Nicotine Model and Nicotine Transdermal Patch Users.

Population			Total Nicotine Model	Nicotine Patch Users
			6178 subjects	49 subjects
Data collection Period			2.4 years	1.7 years
Gender Distribution			Female: 3164 (51.2%)	Female: 26 (53.1%)
			Male: 3014 (48.8%)	Male: 23 (46.9%)
Patient Population Averages	Age		50 years	51 years
	Weight		84.1 kg	83.6 kg
	Height		1.7 m	1.69 m
	Nicotine Concentration (ng/mL)	Min	10.0	10.0
		Avg	677.4	245.5
		Max	43896	2278
	Cotinine Concentration (ng/mL)	Min	10.0	10.0
		Avg	307.7	216.1
		Max	8138	823
	Body Surface Area (BSA)		1.97 m2	1.96 m2
	Lean Body Weight (LBW)		55 kg	54.4 kg
	Calculated Blood Volume (CBV)		5.3 L	5.2 L

LC-MS/MS conditions

Samples were analyzed by LC-MS/MS on an AB Sciex 4500 platform (AB SCIEX, Foster City, CA, USA) using an Agilent 1290 chromatographic system (Agilent Technologies, Waldbroon, Germany), a Phenomenex Kinetex 2.6 µm Phenyl-Hexyl 100 Å, 50 × 4.6 mm UHPLC column, and a gradient of 0.1% formic acid aqueous (solvent A) and 0.1% formic acid in methanol (solvent B). Column temperature was maintained at 40 °C. Elution started with a linear gradient of 5–40% solvent B in the first 2.2 min, and 40–98% in the following 2.3 min. Solvent B was held at 98% for 1 min and then returned to starting conditions, 5%, for a 1.1 min hold which allowed for column equilibration. The run time for this method is 6.5 min. Method flow rate was set to 700 μL/min with a 10 μL injection volume. The multiple-reaction monitoring transitions used are detailed in Table 2. The validation of this method which included 28 other analytes with five internal standards followed CLIA guidelines as described in detail by Enders and McIntire [8].

Table 2

Validation results for LC-MS/MS analysis of nicotine and cotininea.

		Linearitya(all units ng/mL)			Carryoverb	Precision and accuracy
	MS/MS transitionsc	LLOQ / LOD	ULOL	r2	Avg. Conc. (ng/mL) (N = 5)b	Avg. % Target (N = 30)			Avg. % CV (N = 30)			Matrix Effect (%)
						5 ng/mL	25 ng/mL	100 ng/mL	5 ng/mL	25 ng/mL	100 ng/mL
Nicotine	163.1→132.1	2	1000	1.000	0.93	88.8	102.2	109.7	8.1	13.2	11.3	−85.2
	163.1→130.1
Cotinine	176.9→80.1	1	1000	0.9997	0.12	96.8	107.5	108.3	7.1	3.6	3.5	−1.3
	176.9→97.8

In mouth concentrations, actual method LLOQ/LOD values are 0.5 for nicotine and 0.25 for cotinine, and ULOL values are 250 ng/mL.

Average in mouth concentration after a 1000 ng/mL injection.

First transition is the quantifier and the second is qualifier. Internal Standard, amphetamine D5, transitions were 141.2→93.0 (Quantifier), 141.2→67.2 (Qualifier).

Mathematical modelling

The data analysis (Fig. 1) and model development (Fig. 2, Fig. 3, Fig. 4) were conducted using R version 3.3, a language and environment for statistical computing and graphing [9]. Data smoothing was conducted by kernel density estimation, which is a well-accepted mathematical tool to smooth continuous data (e.g., histograms) [10].

Fig. 1

Box and Whiskers plot of oral fluid nicotine and cotinine concentrations (in mouth) from smokers and transdermal delivery.

Fig. 2

(a) Histogram of nicotine and cotinine concentrations. (b) Kernel density estimation nicotine and cotinine concentrations.

Fig. 3

(a) Histogram of the transformed, normalized, and standardized raw nicotine data (≥ 10 ng/mL). The least squares best fit normal distribution is also shown. (b) Kernel density estimation plot derived from the transformed, normalized, and standardized raw nicotine data (≥ 10 ng/mL) overlaid with the least squares minimized best fit Gaussian distribution curve.

Fig. 4

Transformed, normalized, and standardized raw nicotine data (≥ 2 ng/mL) showing a bimodal distribution. The deconvolution of the nicotine data generated two populations as shown in blue and black curves, potentially representing passive exposed population (blue) and positive population (black), respectively. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

Although the quantitative method has lower limits of quantification (LLOQs) of 2 ng/mL for nicotine and 1 ng/mL for cotinine, the reporting cutoff for both analytes was set to 10 ng/mL. Only test results from specimens with desired demographic information (gender, weight, and height), and positive test results (reporting cutoff = 10 ng/mL) were used to develop the model. These data are summarized in Table 1.

The raw nicotine concentration measured in oral fluid of a patient is normalized and transformed as a function of patient lean body weight (LBW), body surface area (BSA), and calculated blood volume (CBV) as described in Eq. (1) which is similar but not identical to the earlier report [7]:

In the equation, ln is the natural log, N is the concentration of nicotine in kg/L, LBW is the lean body weight of the patient in kg, BSA is the body surface area of the patient in meters squared, and CBV is the calculated blood volume in liters. The resulting value is then transformed into its corresponding value on the standard normal distribution using Eq. (2):where Z is the standardized normal value, referred to as the z-score for simplicity; μ and σ are the mean and the standard deviation (SD) of the population calculated by Eq. (1). The values of μ and σ in this model are −13.729 and 1.465, respectively. Eq. (2) adjusts the Gaussian distribution to center it at 0 on the X-axis and with a standard deviation of “1”. The resulting mean and standard deviation of Z are thus 0 and 1, respectively.

The parameters used in Eq. (1) are either calculated directly from the raw data or derived using relevant equations. The LBW accounts for the weight of human body with the exception of the fat including but not limited to bones, muscles, and organs. The LBW is calculated using the James Formula described in Eq. (3):where fact equals 1.1 for male and 1.07 for female and, fact equals 128 for male and 148 for female, respectively [11], [12].

The BSA is the calculated surface area of the patient. The BSA is calculated using the Mosteller Method [13] as described in Eq. (4):

The CBV parameter accounts for the volume of blood (both red blood cells and plasma) in the circulatory system of a patient. The CBV of each patient is estimated using Eq. (5):where AVG_BV is the estimated average blood volume in L/kg of each patient, which is determined using a modified version of Gilcher's Rule of Five and the BMI chart classification of weight categories. The BMI is calculated using Eq. (6) [14]:

The BMI is then used to assess the body fatness of patients and place them into weight categories (Table 3). In the modified version of Gilcher's Rule of Five [15], the specimens were classified into six categories: male obese (BMI ≥ 30 kg/m2), male normal (18.5 ≤ BMI<30 kg/m2), male underweight (BMI<18.5 kg/m2), female obese (BMI ≥ 30 kg/m2), female normal (18.5 ≤ BMI<30 kg/m2), and female underweight (BMI<18.5 kg/m2). The results of these calculations are shown in Table 3.

Table 3

The association of the BMI chart and a modified version Gilcher's Rule of Five utilized in the development of the nicotine model.

BMI index chart		Average blood volume (mL/kg of body weight)
BMI (kg/m2)	Category	Classification	Male	Female
<18.5	Underweight	Thin	65	60
18.5–24.9	Normal	Normal	70	65
≥25	Overweight-obese	Obese	60	55

Results

Table 2 shows a summary of the validation results for this method which is an abbreviated version of the results of the larger overall method. The validation process itself is documented by Enders et al. [8]. Overall, r2 values for validation curves were acceptable at>0.99, the carryover was less than 50% of the LLOQ for both analytes, and precision and accuracy were within 15% for both analytes. While nicotine exhibited a rather large negative matrix effect, this did not affect linearity or LLOQ / ULOL levels of this analyte.

Fig. 1 shows the raw nicotine and cotinine data from smoking as well as the raw data for transdermal patients as “box and whisker” plots. While the absolute number of “positive” samples was 8173 (i.e., the sum of test results above the respective LLOQ for nicotine or cotinine), the number of data points given in the figure reflects all test results>10 ng/mL for nicotine (the reporting cutoff). As shown in Fig. 1, this results in 6178 results for nicotine. Likewise, this process results in 47 test results for transdermal samples. A two tailed Mann-Whitney test indicates that neither the nicotine nor cotinine concentrations are different between smokers and transdermal patch patients.

These data can be normalized and transformed to yield a near normal distribution [7]. Fig. 2 shows the histogram (2.a) and kernel density distribution (2.b) of the raw nicotine and cotinine data from smoking which resemble exponential decays as might be seen in a pharmacokinetic response to a single oral dose. To complete the normalization and transformation of these data, patient data points missing needed demographic information or with nicotine or cotinine concentrations< LLOQs, see Table 2, were excluded, which resulted in a data set that showed 76% (6178/8173) nicotine positivity and 77% (6256/8173) cotinine positivity.

Normalization and transformation of the positive nicotine patient data set (N = 6178, Table 1) using LBW, BSA and CBV yields a histogram that closely resembles a Gaussian distribution, where characteristics of mean and standard deviation have meaning (Fig. 3.a). The kernel density estimation plot using Z from the normalized, transformed, and standardized raw nicotine data overlays closely with the standard normal distribution derived from a least squares optimized fitting algorithm (Fig. 3.b). While a goodness of fit was not determined, visual examination suggests that the normalized nicotine data, after natural logarithm transformation are consistent with a Gaussian distribution.

Fig. 4 shows the results of including nicotine data above the LLOQ (nicotine concentration ≥ 2 ng/mL) and below the reporting cutoff of 10 ng/mL in the transformation and normalization process. The normalized and transformed histogram of these data exhibits a bimodal distribution which was deconvoluted consistent with two distinct bell-shaped curves annotated with solid lines in Fig. 4. This is clearly different from the single Gaussian distribution generated by using only data above the 10 ng/mL reporting cutoff [1].

Normalization and transformation of cotinine data did not yield a useful distribution for further analysis.

Test results for 63 patients prescribed 7 mg, 14 mg, and 21 mg transdermal nicotine patches are summarized in Table 4. There is a hint of increasing median concentrations with increasing dose (Table 4), but the data set is too small to draw adequate conclusions. Nicotine concentrations for transdermal patient samples with adequate demographic information (N = 49 with demographic information summarized in Table 1) were normalized and transformed. The results are plotted as individual points over the N = 6178 nicotine model in Fig. 3b. Notably, the Zscores for over 94% of the transdermal patients are consistent with active smokers (>10 ng/mL).

Table 4

Summary of results from patients prescribed transdermal nicotine patches.

Dose (mg)	Number of specimens	Number of positive results		Minimum (ng/mL)		Maximum (ng/mL)		Median (ng/mL)
		Nicotine	Cotinine	Nicotine	Cotinine	Nicotine	Cotinine	Nicotine	Cotinine
7	6	4	6	<1	10	472	150	57	140
14	22	22	21	24	5	2278	823	129	201.5
21	35	33	34	6	3	1016	823	151	199
Total	63	59	61

Discussion

Fig. 1 illustrates the raw data for nicotine and cotinine. These data are consistent with published results for cotinine [16]. They also demonstrate that nicotine data from smoking and from transdermal patches are not visually or statistically different. This is true for cotinine as well. This data presentation allows rapid comparison of a single data point to the population without sophisticated mathematical processing of the data and thus, may be of value to physicians. This representation will not afford estimates of normal distribution with symmetric variance and a true average value. As demonstrated below, mathematical normalization to calculated blood volume and logarithmic transformation of these data afford a close approximation to a Gaussian distribution.

The normalization and transformation of the raw nicotine data above the suggested reporting cutoff of 10 ng/mL results in a symmetric normal distribution (Fig. 3). Using this process, a “normal” population is defined where the status of individual patient results can easily be interpreted as “consistent” or “inconsistent” with 95% (e.g., 2 standard deviations) of that population [7]. Interestingly, not having the individual doses of nicotine and thus not being able to normalize to dose as per reference [6] does not seem to hamper the modelling process. It is possible that this results from the small volume of distribution of nicotine (1–3 L/Kg) and rapid adsorption of nicotine via the lungs. However, it may also reflect the “distribution” of dosages observed in this population. Unlike pharmaceutical preparations, the “dose” of nicotine represents a “continuous function” across the population which is reflected in the resulting transformation.

The smaller population at lower nicotine concentrations in Fig. 4 (blue trace) is consistent with those patients in the previous clinical trial [1] who were exposed nonsmokers, Group B, while the larger population (black trace) is consistent with active smokers, both light and heavy. The intersection of the two groups, exposed and active smokers, is approximately at a Z of −2. Taking only data at and above 10 ng/mL, the transformation in Fig. 3 closely approximates a Gaussian distribution without any evidence of the population below 10 ng/mL. These results are interesting but don’t contradict the observations from the initial clinical trial [1]. They are consistent with the conclusion of the earlier work [1] that a cutoff of 10 ng/mL for nicotine is appropriate to identify active smoking via oral fluid samples and that oral fluid data can be normalized via calculated blood volume [7].

Test results for patients prescribed transdermal nicotine patches are shown in summary in Table 4. These data suggest that overall, patch doses between 7 and 21 mg do not show a clear “dose response” in oral fluid test results. Rather, this data set is consistent with the Gaussian distribution of light and heavy smokers in Fig. 3 for nicotine. This particular data set is small with no more than 35 patients in each dose group. Further, these data are from “steady state” patch users rather than a “single dose”. Nevertheless, these data are consistent with earlier reports of transdermal patch levels of cotinine and nicotine in oral fluid. For example, Miller et al. [17], tested oral fluid for nicotine and cotinine after removal of a 7 mg transdermal patch from “non-smoker” volunteers at 30 min and 45 min post patch removal. They found median values from 10 patients of 15.5 ng/mL of nicotine and 17.5 ng/mL of cotinine. Albeit low, these data suggest that transdermal patches are consistent with the normalized and transformed data from active smokers (light + heavy). Indeed, work by Kataoka et al. [18] seems to indicate that oral delivery via chewing gum is also consistent with these findings as well.

The use of population statistics to model the distribution of nicotine in oral fluid is uniquely suited to this large amount of data. That is to say, small variations including any genetic variations, different routes of administration, and time of last dose are averaged into the overall population. Nicotine may well be unique inasmuch as its low volume of distribution [19] and rapid half life tend to evenly distribute the drug throughout the body making delivery route less important. As noted in Scheidweiler's original clinical study [1], firm cutoffs in natural systems are statistical estimates and there will be cases that do not meet the described model from the statistical approach to what is a continuous distribution. Physicians should be mindful that values near any reported cutoff; either above or below, should be viewed with caution.

Conclusion

In summary, nicotine and cotinine levels from transdermal application do not differ significantly from those observed from smoking. Thus, oral fluid nicotine testing cannot differentiate between patch application and active smoking indicating that oral fluid testing for nicotine is inadequate for determining patient adherence (to their prescribed transdermal patch). The normalization and transformation of nicotine data from oral fluid samples is consistent with earlier clinical results [1] which indicated that a reporting cutoff of 10 ng/mL is appropriate to differentiate active smoking from passive exposure.