Sensitivity, Specificity and Accuracy of a Novel EEG-Based Objective Test, the Cognalyzer®, in Detecting Cannabis Psychoactive Effects.

INTRODUCTION: Current standards for identifying recent cannabis use are based on body fluid testing. The Cognalyzer® is a novel electroencephalography (EEG) measurement device and algorithm designed to objectively characterize brainwave alterations associated with cannabis. The objective of this study was to assess the accuracy, sensitivity and specificity levels of the Cognalyzer® to characterize brainwave alterations following cannabis inhalation.
METHODS: Seventy-five participants, aged 19-55 years, were enrolled, and oral fluid samples were collected pre-cannabis inhalation. EEG and subjective drug effects questionnaire (DEQ) were administered pre- and post-ad libitum cannabis inhalation. Fifty participants remained in the clinic for 4 h post-inhalation. Blinded analyses of the EEG files were conducted by Zentrela Inc. using two versions (V1 and V2) of the Cognalyzer® algorithm. Pre- vs. post-inhalation comparison status was characterized by the Cognalyzer® and summarized for: sensitivity, specificity, accuracy, percent false positive, percent false negative and positive and negative predictive value. The null hypothesis was tested using McNemar's test. Cognalyzer® results pre- and post-inhalation were combined with the oral fluid tetrahydrocannabinol (THC) concentration to evaluate potential to improve current drug testing.
RESULTS: The two versions of the Cognalyzer® algorithm had similar diagnostic results. Diagnostic outcomes were improved when participants with missing EEG recordings or electrode placement errors were removed. The Cognalyzer® accuracy was 85.5% and 83.9%, sensitivity was 87.1% and 88.7%, and specificity was 83.9% and 79.0% for algorithm V1 and V2, respectively. Combining Cognalyzer® results with oral fluid concentrations reduced false-positive oral fluid test results by up to 49%.
CONCLUSION: The Cognalyzer® characterized brainwave alterations associated with cannabis inhalation with high levels of accuracy in a population of participants with varied cannabis inhalation histories, relative to the comparison standard of pre- vs. post-inhalation status. The Cognalyzer® allows the results to be generalized to the larger population addressing a limitation in currently accepted standards.

Key Summary Points

Digital Features

This article is published with digital features, including a summary slide, to facilitate understanding of the article. To view digital features for this article go to https://doi.org/10.6084/m9.figshare.14233043.

Introduction

Federal legalization of cannabis in Canada in 2018 has allowed for investigation and promotion of its therapeutic potential for various indications. Since legalization, there has been a statistically significant increase in cannabis use between January–March 2018 and January–March 2019 in males aged 18–64 years and females aged 45–64 [1]. A recent report by Statistics Canada during the months of August through October of 2019 revealed that more than 5 million Canadians had used cannabis in the past 3 months [2]. As legalization allows for increased cannabis use, determining cannabis impairment with accuracy and objectivity is increasingly important. It is essential to recognize the differences between the detection of impairment versus recent use. Current tests for detection and categorization of impairment include the Standard Field Sobriety Test (SFST), 12-step evaluation of Drug Recognition Expert (DRE) Evaluation and oral fluid sampling for recent use [3]. Technology that can objectively and accurately determine cannabis impairment must be integrated into the current investigation process in the workplace or while driving.

Cannabinoids and metabolites remain in the body, after intake, for extended periods, and correlations between physiological drug levels and impairment have not been clearly characterized [4-6]. Acute cannabis use may impair cognition and psychomotor function, but there is significant variability with dose, route of administration and individual cannabis tolerance [5, 7, 8]. Cannabinoids and metabolites are often measured in urine, blood and oral fluid to detect recent use. Drug testing is commonly done on urine samples; however, detectable levels of cannabinoids and their metabolites remain in urine the longest compared to the other two matrices [6, 9]. In participants who do not regularly use cannabis and had not used cannabis for 3 months prior to study screening, 11-nor-9-carboxy-tetrahydrocannabinol (THC–COOH), a tetrahydrocannabinol (THC) metabolite, was present in urine 93.3 h [cut-off = 15 nanograms/millilitre (ng/ml)] after ingestion of an edible dose [10]. Additionally, last positive detection ranges vary between people and are affected by cannabis use history [10, 11]. In some instances, detection of cannabinoids in urine and blood may also occur after only passive exposure [12, 13]. THC can be detectable in blood for days after cannabis use, with substantial variability between people [5, 14, 15]. Cannabis use history has an impact on both the time course and concentration of THC in blood. In frequent cannabis users who reported daily or near-daily use over the preceding 14 days, 6 of 25 participants had detectable THC in whole blood after 6 days of monitored abstinence [14]. Route of administration affects the time course and concentration levels of THC in blood. When THC is inhaled it bypasses the gut, and peak concentrations are reached within 0.17 (0.15–0.33) h and decrease rapidly [16-18]. Peak concentration occurs later with ingestion making it challenging to conclusively determine the time of last use [5, 16, 17, 19, 20]. Different cannabis use histories and dosages further confound this determination. Oral fluid is subject to similar limitations that impact the ability to identify recent use. Oral fluid THC and THC-COOH levels are shown to be detectable up to a mean range of 0.7–37.3 h following edible consumption [5]. Further variability can be added by possible dilution or contamination by foods, drinks and chewing gum [4, 5] or with differences in the degree of cannabis deposited in the oral cavity [21, 22].

Currently, roadside tests in Canada include the SFST, or oral fluid sampling [3]. The SFST was created for the detection of alcohol intoxication in drivers and is currently also used for drug intoxication screening [23]. When applied to acute cannabis intoxication, the accuracy of classification ranges from 65.8% to 76.3% [23, 24]. The sensitivity is further reduced in chronic cannabis users [25], which may be a result of tolerance to drug effects [26]. When impairment is suspected, a DRE is employed to conduct a 12-step evaluation to determine the drug category that may be causing the impairment [16]. Many of the components of these assessments are arguably subjective in nature, are time-consuming, may be confounded by cannabis use history, are not performed roadside and require an expensive training program to qualify as an administrator. As a result, law enforcers and employers require a complementary, rapid and cost-effective method to provide evidence of cannabis impairment, allowing them to improve safety while reducing the risk of legal challenges resulting from false-positive test results or accusations of subjective bias.

The most typical physiological effect of acute cannabis intake is increased heart rate [27], however; several studies have found significant effects on the electroencephalogram (EEG). For example, visually evoked error-related negativities were reduced in magnitude after cannabis consumption, and short episodes of euphoria induced by cannabis were accompanied by 70% increases in alpha band power in parietal electrodes [28]. Brown et al. have demonstrated that theta band EEG power decreases significantly after cannabis dosage and that this decrease was significantly correlated with poorer simulated driving performance [29, 30]. Thus, it seems evident that recent cannabis intake can be detected through changes in EEG and that to some extent the magnitude of these changes may reflect the amount of impairment currently being experienced.

Zentrela Inc. (Hamilton, Ontario, Canada) developed a new EEG-based method, the Cognalyzer®, for assessing cannabis’ psychoactive effects and detecting brainwave alterations associated with its inhalation. The Cognalyzer® is a novel EEG device and patent-pending algorithm that uses brain signal processing techniques and machine learning to objectively detect and assess potential impairment through brainwave analyses. Instead of measuring levels of THC in bodily fluids, the Cognalyzer® detects the presence of abnormal brainwaves that are associated with THC psychoactive effects and measures their consistency over a 2.5-min data collection period. It is a portable and objective solution that could potentially detect impairment and may be applied to the workplace or roadside testing in conjunction with the currently established and accepted methods. The objective of this study was to assess the accuracy, sensitivity and specificity levels of the Cognalyzer® to characterize brainwave alterations associated with cannabis inhalation using EEG signals. For purposes of the study, the participants were required to arrive at the clinic having refrained from cannabis use for 3 days, and it was assumed they had ‘normal’ brainwaves initially and should have exhibited ‘altered’ brainwaves as detected by the Cognalyzer® algorithm immediately after cannabis inhalation. The potential improvement in determining cannabis impairment when the Cognalyzer® was combined with an oral fluid test for THC was also evaluated. It was hypothesized that administering the Cognalyzer® in conjunction with the results of the oral fluid test, commonly administered roadside or in the workplace, could provide complementary evidence of current impairment and recent use while reducing the false-positive rate.

Methods

Study Design

This study was conducted at the KGK Science Inc. clinic site, London, ON, Canada, from February 28, 2020, to August 29, 2020. The trial received research ethics board approval on February 14, 2020, from the Institutional Review Board (IRB) Services, Aurora, Ontario (Pro00041616). All participants provided written informed consent to participate in the study prior to initiation of study procedures. All participants with identifying information included in the manuscript provided written consent to the use of their photograph. The study was conducted in compliance with the International Council for Harmonization of Technical Requirements for Pharmaceuticals for Human Use (ICH) Guideline for Good Clinical Practice (GCP) and in accordance with the Declaration of Helsinki guidelines and its subsequent amendments.

Participants met the following inclusion criteria: were male or female between the ages of 19–65; provided two adult contacts to ensure transportation to and from the clinic; were willing to comply with all study requirements, questionnaires, records and assessments. Cannabis-specific inclusion criteria included: abstinence from cannabis for 3 days prior to visit, brought a cannabis product purchased from a legal medical or recreational source, in its sealed, original packaging, for use via an inhalation route of administration during the visit and had the required cannabis use history. Cannabis use history was assessed with a seven-item Cannabis Use Questionnaire (CUQ). Participants reported using cannabis at least a few times/month and no more than 2–3 times/week, usually via an inhalation route of administration (vape, smoke); that they do not struggle to control their high or get dizzy, vomit or become paranoid and can handle a self-reported 7/10 level of high; and that they had not used recreational drugs other than alcohol or cannabis within the last few weeks.

Individuals were excluded if they had participated in any previous Zentrela Inc. EEG data collection; had an allergy, sensitivity or stated that they had religious practices that precluded the application of the EEG headset; had a history of a clinically significant adverse event (AE) associated with cannabis, brain injury, epilepsy, seizures, concussion, stroke, anxiety, depression, dementia, mood disorder, schizophrenia, psychotic disorder or familial history of psychosis or current migraine; had a history of alcohol or drug abuse in the last 12 months; self-reported use of cannabinoid products within 3 days of their study visit; impairment from illicit drugs or alcohol during the study visit as assessed by the medical director; had unstable chronic disease; had current or history of kidney or liver disease; had unstable hypertension; had a significant cardiovascular event within the last 6 months; had a self-reported HIV, hepatitis B or hepatitis C diagnosis. Participants on stable medications for at least 3 months, those with a history of surgery within the last 3 months as well as any other condition which may have adversely affected the participants’ ability to complete the study or its measures, or which may have posed significant risk to them, were considered case by case by the medical director.

Investigational Device

The Cognalyzer® is a new technology to detect the presence of cannabis’ psychoactive effects based on brain signal analysis and machine learning. It utilizes a proprietary battery-powered data collection device that streams data via Bluetooth to a recording system on a nearby laptop. Once data were collected, raw continuous data files were retained and sent with blinded filenames to Zentrela Inc. for evaluation. The algorithm could alternatively be operated on the laptop for near real-time evaluations. The EEG device was an 8-channel system with a 250-Hz sampling rate and 24-bit resolution. Conductive gel was applied to reduce electrode impedance below 35 kΩ. Electrodes were placed on the left and right side of a participant’s head to coincide with their frontal, temporal, occipital and parietal lobe positions. Ground and driven right leg (DRL) electrodes were placed on the forehead, for a total of ten electrodes. Electrodes were held in position using a proprietary ten-electrode headband. Once the collection began, data were segmented into 10-s segments with 5-s overlap. Unreliable segments such as signals with high peak voltage, very high variance, very high or low root of mean square (RMS) voltage, and unusual spectral properties caused by eye or body movements were rejected in real time, and data collection continued until a total of 30 artifact-free segments were collected (Fig. 1).

Fig. 1

(i) Photograph of a model participant undergoing a Cognalyzer® test. (ii) Photograph of Cognalyzer® electrode placements on a study participant. The study participant has given us permission for their photo to be used in the publication

Algorithm Description

The patent-pending Cognalyzer® algorithm works directly with the segmented EEG data collected from the 8-channel EEG device. During data collection, each segment is analyzed for artifact as described above, and a set of 30 artifact-free segments is submitted to the algorithm. Each segment is independently analysed for features including power spectral density, cross power spectral density, coherence and RMS power. Based on these data, the algorithm produces a classification for each segment, either ‘normal’ or ‘abnormal.’ With this set of classifications, the strength of the psychoactive effect for a given data file is determined by calculating the percentage of total segments (0–100%) that are classified as ‘abnormal.’ Two such recordings were made for each participant’s data collection episode, one immediately after the other. The psychoactive effect of an episode is the average of the percentage of total ‘abnormal’ segments of the two EEG files in that episode. An EEG episode was predicted to be post-inhalation when the percentage psychoactive effect was higher than 38%, or otherwise predicted as being pre-inhalation. Two candidate algorithms using slightly different parameter weightings were evaluated, Cognalyzer® version 1 (V1) and version 2 (V2). Cognalyzer® V2 was designed to be more sensitive than V1 in detecting a cannabis effect and V1 to be more balanced in sensitivity vs. specificity.

Study Procedures and Assessments

Screening was conducted via telephone. Prior to enrollment of participants, ten mock participant visits were completed to ensure accurate administration of the Cognalyzer® and maintain the scientific rigor and reproducibility of results. At the 1-day, in-clinic visit, participants signed an informed consent form, and medical history and eligibility criteria were reviewed. Seventy-five participants were enrolled in the study. To evaluate primary and secondary objectives, study assessments were conducted pre- and post-cannabis inhalation. For all enrolled participants, pre-cannabis inhalation, seated blood pressure (BP) and heart rate (HR) were measured, oral fluid was collected (Quantisal™ collection device, Immunalysis Corp., limit of detection: 0.5 ng/ml) to measure THC concentration, and two 2.5-min baseline EEG recordings were completed with the Cognalyzer® investigational device. Oral fluid was collected from under the participant's tongue and in accordance with the collection device instructions. For each participant, photographs of the EEG headset applied on their head were captured to document that the ten EEG electrodes were placed in the correct locations. Photos captured the frontal, posterior and lateral left and right views of the electrode placements, and all identifying participant features were removed. Participants were allowed to smoke or vape the cannabis products they brought to the clinic ad libitum. They were required to do so outside of the clinic building, while monitored by clinic staff. Participants were instructed to inhale cannabis until a self-reported 7/10 feeling of high was reached. Immediately after cannabis inhalation, a 5-item Drug Effects Questionnaire (DEQ-5) was administered, and two 2.5-min post-inhalation Cognalyzer® EEG readings were collected [5, 31]. The five questions on the DEQ-5 included: (1) Do you feel the drug effect right now? (2) Are you high right now: (3) Do you dislike any of the effects you are feeling right now? (4) Do you like any of the effects you are feeling right now? (5) Would you like more of the drug you took right now? The last 50 participants enrolled in the study remained in the clinic for 4 h post-inhalation, and the Cognalyzer® and DEQ were repeated at 30 min, 1, 1.5, 2, 2.5, 3, 3.5 and 4 h. These data will be presented elsewhere [32]. For these participants, a second oral fluid sample was collected 4 h post-inhalation. Participants’ vital signs were measured, and they were assessed for safety prior to leaving the clinic at the end of the study visit.

Prior to sending the participants’ blinded EEG data to Zentrela Inc. for analysis, each participant’s EEG data were saved as both blinded and unblinded files for a total of eight files (four blinded and four unblinded) per participant. Files were blinded by replacing the filename with a random nonrepeating integer (http://www.randomization.com), and within each participant and time point (pre and post), the two files collected during one episode were grouped together. To further ensure blinding, each file was saved as a new document to remove the original timestamp and date. A password-protected master list with the identified random number assigned to each of the raw files was generated and retained at KGK Science Inc. The files were clear text CSV data that contained only the eight channels of raw EEG data with no other identifying information. This structure was verified by KGK Science Inc. personnel before submission to Zentrela Inc. for analyses. Zentrela Inc. personnel were not present during data collection, and Zentrela Inc. personnel who conducted the initial (primary outcome) data analyses were only unblinded after results were returned to KGK Science Inc. for tabulation and the database locked.

During the study visit, AEs were recorded and documented in the study record. All AEs were classified as per the description, duration, intensity, frequency and outcome. The medical director determined the causality and intensity of all reported AEs, as appropriate. AEs were coded according to the Medical Dictionary for Regulatory Activities (MEDRA) terminology System Organ Class, version 23.0.

Data and Statistical Analysis

The planned sample size for this study was 150 participants. A previous pilot study estimated a sensitivity of 83.3% for the Cognalyzer®. With a sample size of 150 participants, it was possible to obtain sensitivity of between 75.2% and 90% with at least 80% power and 5% significance level, given 20% attrition. Due to the impact of the global coronavirus disease 2019 (COVID-19) pandemic on timelines and project funding, the number of participants was reduced [33]. A sample size of 75 participants was enrolled in the study. The sensitivity obtained from the study was within the planned range on the sample size calculation.

Two per protocol (PP1 and PP2) populations were assessed in this study. The PP populations consisted of all participants who completed the study visit and procedures connected with the Cognalyzer® EEG measurement. The PP1 population consisted of participants with two EEG files for both the pre-inhalation and the first post-inhalation time points. The PP2 population consisted of participants with two EEG files for pre-inhalation and all post-inhalation time points.

This study used a comparison standard of pre- vs. post-inhalation status. Self-reported, subjective impairment was assessed with the DEQ-5 and was used to classify subjective impairment at 4 h post-inhalation.

The pre- vs. post-inhalation comparison status characterized by Cognalyzer® was summarized for the following parameters: sensitivity, specificity, accuracy, percent false positive, percent false negative, positive predictive value (PPV) and negative predictive value (NPV) of the instrument. The PPV is the proportion of positive test results which are true positives. The NPV is the proportion of negative test results which are true negatives. The diagnostic performance of Quantisal™, Immunalysis oral fluid testing relative to pre- vs. post-inhalation was evaluated with THC concentration cut-offs of 0.5 ng/ml. Since there are different guidelines for determining the cut-off and this study was primarily designed to evaluate the Cognalyzer® and its potential application in synergy with oral fluid, it was decided to include all concentrations > 0.5 ng/ml as positive samples. To calculate a false-positive rate at 4 h post-inhalation, it was arbitrarily defined that an answer to Q1 (“Do you feel the drug effect right now?”) of the DEQ-5 of < 4 indicated not being subjectively impaired; if participants answered with a level ≥ 4 they were subjectively considered impaired. This level was chosen arbitrarily to align with the threshold of 38% used for the Cognalyzer® test. By using the DEQ-5, the participant's own subjective self-reported intoxication level was considered as the comparison standard of intoxication. Participants were encouraged to answer the DEQ-5 questions honestly and were not incentivized to report any specific DEQ-5 values.

The ability to improve current drug testing procedures was evaluated by combining the Cognalyzer® results pre-inhalation and 4 h post-inhalation with oral fluid THC concentration assays.

The null hypotheses were tested using McNemar’s test. p values ≤ 0.05 were considered statistically significant. All statistical analyses were completed using R Statistical Software Package version 3.6.3 or newer for Microsoft Windows [34].

Results

Participants

Participants enrolled in this study ranged from 19 to 55 years of age and were 64% male and 36% female. Participants had variable cannabis use histories and reported having used cannabis for 9 years on average, with a range of 3 months to 29 years and one report of the age of first use as early as 12 years. Frequency of use varied. Seventy-four participants reported only using cannabis recreationally; however, one participant reported use of both recreational and medicinal cannabis for sciatic nerve pain. Of the 104 participants who were screened for this study, 75 were enrolled and 72 completed the full study visit (Fig. 2). Three participants dropped out before completing the full study visit, but all participants provided both EEG recordings before cannabis inhalation and at least one EEG recording post-inhalation. All participants provided cannabis products for inhalation in this study that were purchased from a legal source, were in sealed packaging and contained 14.0–85.9% THC.

Fig. 2

Disposition of study participants

Diagnostic Performance of Cognalyzer®

The two versions of the Cognalyzer® algorithm evaluated had similar diagnostic results. The accuracy of the device on the full study population was 81.3% and 80.0%, sensitivity was 80.0% and 81.3% and specificity 82.7% and 78.7% for algorithm V1 and V2, respectively (Table 1). There were no significant differences between the proportions of positive and negative tests between both Cognalyzer® versions and the comparison standard of pre- vs. post-inhalation status. The PPVs were 82.2% and 79.2% and NPVs were 80.5% and 80.8% for V1 and V2, respectively.

Table 1

Diagnostic performance of two versions of the Cognalyzer® algorithm in the full study population (n = 75)

	Diagnostic performance of Cognalyzer®—full population
	Post-inhalation	Pre-inhalation	Total	Sensitivity (95% CI)	Specificity (95% CI)	Accuracy (95% CI)	Percent false positive (95% CI)	Percent false negative (95% CI)	PPV (95% CI)	NPV (95% CI)	p value
Cognalyzer®—algorithm version 1
Cognalyzer® version 1—post-inhalation	60	13	73	80.0 (69.2–88.4)	82.7 (72.2–90.4)	81.3 (74.2–87.2)	17.3 (9.6–27.8)	20 (11.6–30.8)	82.2% (71.5- 90.2)	80.5% (69.9–88.7)	0.705
Cognalyzer® version 1—pre-inhalation	15	62	77	–	–	–	–	–			–
Total	75	75	150	–	–	–	–	–			–
Cognalyzer®—algorithm version 2
Cognalyzer® version 2—post-inhalation	61	16	77	81.3 (70.7–89.4)	78.7 (67.7–87.3)	80.0 (72.7–86.1)	21.3 (12.7–32.3)	18.7 (10.6–29.3)	79.2% (68.5–87.6)	80.8% (69.9–89.1)	0.715
Cognalyzer® version 2—pre-inhalation	14	59	73	–	–	–	–	–			–
Total	75	75	150	–	–	–	–	–			–

McNemar’s chi-squared test was used to compare the difference in performance between the diagnostic test being evaluated and the comparison standard

To further refine the Cognalyzer® algorithms, eight participants were removed from the analysis for missing or late EEG recordings or electrode placement errors for the PP1 population and 13 participants for the PP2 population. Diagnostic outcomes were improved when these participants were removed. In the PP2 population, the Cognalyzer® accuracy was 85.5% and 83.9%, sensitivity was 87.1% and 88.7% and specificity was 83.9% and 79.0% for algorithm V1 and V2, respectively (Table 2). There were no significant differences between the proportions of positive and negative tests between both Cognalyzer® versions and the pre- vs. post-inhalation comparison standard. The PPVs were 84.4% and 80.9% and NPVs were 86.7% and 87.5% for V1 and V2, respectively.

Table 2

Diagnostic performance of two versions of the Cognalyzer® algorithm after participants with missing EEG recordings or who had electrode placement errors and were removed from the population (PP2, n = 62)

Diagnostic performance of cognalyzer®
	Post-inhalation	Pre-inhalation	Total	Sensitivity (95% CI)		Specificity (95% CI)	Accuracy (95% CI)	Percent false positive (95% CI)	Percent false negative (95% CI)	PPV (95% CI)	NPV (95% CI)	p value
Cognalyzer®—algorithm version 1
Cognalyzer® version 1—post-inhalation	54	10	64		87.1 (76.1–94.3)	83.9 (72.3–92)	85.5 (78–91.2)	16.1 (8–27.7)	12.9 (5.7–23.9)	84.4% (73.1–92.2)	86.7% (75.4–94.1)	0.637
Cognalyzer® version 1—pre-inhalation	8	52	60		–	–	–	–	–			–
Total	62	62	124		–	–	–	–	–			–
Cognalyzer®—algorithm version 2
Cognalyzer® version 2—post-inhalation	55	13	68		88.7 (78.1–95.3)	79.0 (66.8–88.3)	83.9 (76.2–89.9)	21 (11.7–33.2)	11.3 (4.7–21.9)	80.9% (69.5–89.4)	87.5% (75.9–94.8)	0.180
Cognalyzer® version 2—pre-inhalation	7	49	56		–	–	–	–	–			–
Total	62	62	124		–	–	–	–	–			–

McNemar’s chi-squared test was used to compare the difference in performance between the diagnostic test being evaluated and the comparison standard

Diagnostic Performance of Oral Fluid THC Concentration

The average pre-inhalation oral fluid THC concentrations were 6.5 ± 24.1 ng/ml (0.5–178.0 ng/ml); one participant who dropped off early (1-h post-inhalation) from the study had missing oral fluid data pre-inhalation. There was a significant increase in the average THC concentration to 57.2 ± 66.1 ng/ml (2.0–276.0 ng/ml) at 4 h post-inhalation (p < 0.001). The diagnostic performance of the oral fluid testing was significantly different than the comparison standard of pre- vs. post- inhalation status in the full study population (Table 3). The PPV for this test was 65.7% and the NPV was 100%. A table with the complete oral fluid results for each participant is available on the Zenodo database (10.5281/zenodo.4647066) or upon request to  the corresponding author from interested researchers.

Table 3

Diagnostic performance of oral fluid THC concentration in the full study population (n = 74)

Comparison standard vs. oral fluid THC test—ITT population (positive = saliva THC > 0.5 ng/ml)
	4-h post-inhalation	Pre-inhalation	Total	Sensitivity (95% CI)	Specificity (95% CI)	Accuracy (95% CI)	Percent false positive (95% CI)	Percent false negative (95% CI)	PPV (95% CI)	NPV (95% CI)	p value
Oral fluid THC positive	48	25	73	100 (92.6–100)	66.2 (54.3–76.8)	79.5 (71.3–86.3)	33.8 (23.2–45.7)	0 (0–7.4)	65.7% (53.7–76.5)	100.0% (92.7–100.0)	< 0.001
Oral fluid THC negative	0	49	49	–	–	–	–	–			–
Total	48	74	122	–	–	–	–	–			–

McNemar’s chi-squared test was used to compare the difference in performance between the diagnostic test being evaluated and the comparison standard

Four hours post-inhalation, each participant’s self-reported DEQ-5 intoxication level was considered as the ‘true’ state of intoxication. Using oral fluid THC concentration cut-offs of 10 and 25 ng/ml, the diagnostic performance was significantly different than the comparison standard in the PP2 population (Table 4). The PPVs were 5.1%% and 0.0% and NPVs were 95.3% and 93.0% for the 10 and 25 ng/ml cut-offs, respectively.

Table 4

Diagnostic performance of oral fluid THC test in detecting a DEQ response of < 4 or ≥ 4 on question 1: Do you feel a drug effect right now? (PP2, N = 41)

Before tests are not impaired; final tests (T = 4 h) are subjectively not impaired if DEQ < 4
	DEQ ≥ 4	DEQ < 4	Total	Sensitivity (95% CI)	Specificity (95% CI)	Accuracy (95% CI)	Percent false positive (95% CI)	Percent false negative (95% CI)	PPV (95% CI)	NPV (95% CI)	p value
Oral fluid test is positive if THC ≥ 10 ng/ml
Oral fluid THC positive	2	37	39	50.0 (6.7–93.2)	52.6 (40.9–64.0)	52.4 (41.1–63.6)	47.4 (36.0–59.1)	50.0 (6.8–93.2)	5.1% (0.6–17.3)	95.3% (84.2–99.4)	< 0.001
Oral fluid THC negative	2	41	43	–	–	–	–	–			–
Total	4	78	82	–	–	–	–	–			–
Oral fluid test is positive if THC ≥ 25 ng/ml
Oral fluid THC positive	0	25	25	0.0 (0.0–60.2)	67.9 (56.4–78.1)	64.6 (53.3–74.9)	32.1 (21.9–43.6)	100.0 (39.8–100.0)	0.0% (0.0–13.7)	93.0% (83.0–98.1)	< 0.001
Oral fluid THC negative	4	53	57	–	–	–	–	–			–
Total	4	78	82	–	–	–	–	–			–

McNemar’s chi-squared test was used to compare the difference in performance between the diagnostic test being evaluated and the comparison standard

Diagnostic Performance of Cognalyzer® combined with Oral Fluid THC

Combining the Cognalyzer® results with oral fluid THC concentration reduced false-positive oral fluid test results in participants who remained in the clinic for 4 h post-inhalation. Prior to cannabis inhalation, there were 14.6% false positives for the oral fluid test at 10 ng/ml and 9.8% at 25 ng/ml. For the Cognalyzer® test alone, there were 19.5% false positives. When the Cognalyzer® and oral fluid were combined, the false positive rate was 0% for both oral fluid thresholds. Four hours post-inhalation, the false-positive rate for the oral fluid test was 75.6% at 10 ng/ml and 51.2% at 25 ng/ml. Adding the requirement for a positive Cognalyzer® result reduced these rates to 26.8% and 14.6%, respectively. When pre- and post-inhalation time points were analyzed together, false-positive rates were 14.1% and 7.7% for the 10 and 25 ng/ml cut-offs, respectively (Table 5). The PPVs were 15.4% and 0.0% and NPVs were 97.1% and 94.7% for the 10 and 25 ng/ml cut-offs, respectively.

Table 5

Diagnostic performance of oral fluid THC test plus Cognalyzer® test in detecting pre- and post-inhalation (PP2, N = 41)

	Post-inhalation	Pre-inhalation	Total	Sensitivity (95% CI)	Specificity (95% CI)	Accuracy (95% CI)	Percent false positive (95% CI)	Percent false negative (95% CI)	PPV (95% CI)	NPV (95% CI)	p value
Oral fluid test is positive if THC ≥ 10 ng/ml and Cognalyzer test is positive
Positive	2	11	13	50.0 (6.7–93.2)	85.9 (76.2–92.7)	84.1 (74.4–91.3)	14.1 (7.3–23.8)	50.0 (6.8–93.2)	15.4% (1.9–45.4)	97.1% (89.9–99.6)	0.013
Negative	2	67	69	–	–	–	–	–			–
Total	4	78	82	–	–	–	–	–			–
Oral fluid test is positive if THC ≥ 25 ng/ml and Cognalyzer test is positive
Positive	0	6	6	0.0 (0.0–60.2)	92.3 (84.0–97.1)	87.8 (78.7–94.0)	7.7 (2.9–16.0)	100.0 (39.8–100.0)	0.0% (0.0–45.9)	94.7% (78.1–98.5)	0.527
Negative	4	72	76	–	–	–	–	–			–
Total	4	78	82	–	–	–	–	–			–

McNemar’s chi-squared test was used to compare the difference in performance between the diagnostic test being evaluated and the comparison standard

Safety Outcomes

There were 12 adverse events reported by 11 participants in this study. The adverse events included six reports of headache or migraine, one of anxiety, two of dizziness and three of nausea. One of the AEs was categorized as “most probable” and four as “possible” in terms of relation to the investigational device. The “most probable” AE was classified as a headache and the four possible were classified as two headaches and one each of nausea and anxiety. All AEs were resolved by the end of the in-clinic study visit or upon telephone follow-up within 24 h.

Discussion

The sensitivity, specificity and accuracy of the Cognalyzer® were assessed relative to the comparison standard of pre- vs. post-inhalation status that was used in this study. Accuracy, sensitivity and specificity for Cognalyzer® algorithm V1 for the PP2 population were 85.5%, 87.1% and 83.9% and for algorithm 2 were 83.9%, 88.7% and 79.0%, respectively.

The sensitivity of the device, representing the proportion of true-positive tests relative to the comparison standard [35], was 87.1–88.7% when participants with missing EEG data or electrode placement errors were removed. The specificity of the device, representing the proportion of true negative tests relative to the comparison standard [35], was 79.0–83.9%. Specificity and sensitivity are extremely important in a tool for detecting both recent cannabis use and impairment, as false negatives and positives have consequences for diagnostic devices. Biological fluids, blood, oral fluid and urine, have been used as diagnostic matrices for detecting recent cannabis use despite known limitations. Detectable amounts of THC or metabolites are present after intoxication has subsided or through passive exposure and there are often time lags and large variability between the time cannabis was used and fluid collection [5, 8, 9, 11, 36, 37], affecting accuracy of detection. Blood concentrations of THC have better associations with cannabis impairment [5] however decrease rapidly within minutes of cannabis inhalation [19]. Samples are often taken hours later, or long after THC levels are detectable [37]. SFSTs and behavior examinations are common tools for detecting impairment but are arguably limited by subjectivity and potential for bias from examiners, lack of evidence supporting use with cannabis, sensitivity to cannabis use history [5, 24] and lack of consistent agreement with body fluid concentrations [38, 39].

Accuracy, the ability of the Cognalyzer® to distinguish between pre- and post-inhalation [35], ranged from 83.9% to 85.5% for the two algorithm versions. There were no significant differences between the proportions of false-positive and -negative results between the Cognalyzer® and the comparison standard. These findings are superior to previously reported assessments of the SFST of up to 73.9% at correctly classifying cannabis intoxication [23]. The participants in the study had variable cannabis use histories and brought a variety of products to the clinic for use via different inhalation methods (smoking, vaping). These factors all impact THC concentrations in body fluids [4, 6, 10, 40]. The diagnostic performance of the Cognalyzer®, despite participant and product variability, highlights its potential for distinguishing recent cannabis inhalation accurately in a real-world setting.

Behavioral assessments have been used in conjunction with measurements of body fluid to determine cannabis impairment [41]. In the current study, the objective Cognalyzer® assessment was combined with oral fluid THC measurement. As a single test, the Cognalyzer® performed better than the oral fluid THC test to accurately distinguish between pre- and post-inhalation, relative to the comparison standard. Compared to the pre- vs. post-inhalation comparison standard, there were significant differences between the proportions of false-positive and -negative results with the oral fluid test (p ≤ 0.001) that were not observed with the Cognalyzer®. Previous work has shown that oral fluid THC levels are detectable for up to 0.5–37.3 h after cannabis [4], and linear relationships with behavioral performance impairment are weak [37]. Furthermore, the efficacy and sensitivity of these tests can change over time, decreasing from 90% 5 min after smoking to 50% by 80 min [42]. Cannabis use history impacts the pharmacokinetics of oral fluid response to THC [19]. Concentrations reach higher peaks soon after cannabis use in chronic smokers but return to similar magnitudes for occasional users within 15 min [43]. Combining the Cognalyzer® and oral fluid THC tests yielded superior results to either test alone for establishing recent cannabis use that might affect human performance. When the pre-inhalation assessments were combined, the false-positive rate was 0%. False-positive rates in cannabis drug testing can lead to questioning of test conclusions, particularly when there are legal and personal consequences to outcomes. The combination of these test modalities shows promise in mitigating the risks from false-positive assessments. Post-cannabis inhalation, our study relied on the DEQ-5 to classify subjective impairment status at the 4-h time point. Reliance on participants’ subjective self-assessment of impairment status was the only available standard for comparison at the 4 h time point. In double-blind placebo-controlled studies, self-report of intoxication level has been shown to have reliably similar dose-response functions to physiological and pharmacokinetic effects [44, 45].

The current study provides evidence for the safety and tolerability of the use of Cognalyzer® investigational device in adults. Prior to enrollment participants were free of comorbidities that may have affected their response to cannabis and other drugs. There were only five AEs that were determined to be possibly or most probably related to the Cognalyzer®. Four were classified as possibly related and one as probably related. These AEs included headache reported by three participants, feeling anxious by one participant and nausea by one participant.  All participants recovered by the end of the in-clinic study visit or upon telephone follow-up within 24 h. None of the participants indicated that they were unwilling to undergo the EEG collection procedure, although they were free to do so, suggesting that the instrument is not particularly intimidating. Collecting bodily fluid such as oral fluid, blood and urine may expose the investigator to health risks and these are eliminated with the use of the Cognalyzer®. Based on the findings of this study, using the Cognalyzer® EEG device did not pose risk or cause harm to the participant, supporting its continued evaluation in more participants and other populations.

There were limitations to the current study that should be considered in the interpretation of the results. It was assumed that all participants should have exhibited ‘normal’ brainwaves for the pre-inhalation time point and should have exhibited sufficient altered brainwaves to trigger the algorithm at the initial post-inhalation time point. Although all participants agreed to abstain from cannabis for 3 days prior to their study visit, 25 participants had detectable levels of THC in their oral fluid pre-inhalation that ranged from 0.6 to 178 ng/ml. Twenty of these participants had concentrations ≤ 19 ng/ml. Despite this limitation, the Cognalyzer® had high levels of accuracy in detecting brainwave alterations associated with recent cannabis inhalation. Participants reported using cannabis up to three times per week and had used cannabis via an inhalation route of administration, which may explain the pre-inhalation oral fluid THC levels. The lack of a positive or negative control limits the ability of this study to definitively state that this algorithm detects only EEG changes due to THC; brain state changes caused by other drugs, medical conditions or abnormal emotional states might activate an ‘abnormal’ result for the algorithm. In the confines of the design of this study the assessment by the medical director of each participant based on the exclusions, their medical histories and comorbidities precluded the inadvertent enrollment of those that may have confounded the results of the study. Future work should address this limitation by expanding the population to those who use cannabis more and less frequently as well as evaluate the diagnostic potential with additional routes of administration. Certainly, the addition of both positive and negative controls to the design in future studies should be considered. Participants inhaled different cannabis products ad libitum and therefore the dose was variable between participants. This was reflective of real-world use; however, future studies are needed to expand this work with more controlled dosing regimens. In the context of a clinical study, there were no negative consequences to the participant for ‘failing’ the test. Some circumstances would certainly be different due to the substantial pressure that would be present in the circumstance facing an employee or driver being tested for potential recent cannabis use.

Conclusions

The Cognalyzer® detected brainwave alterations with high levels of accuracy relative to the comparison standard of pre- vs. post-inhalation status in a population of male and female adults between the ages of 19–55 years. Current available methods for detecting impairment or recent use are limited based on the influence of cannabis use history on their results. Combining the Cognalyzer® test with readily accepted oral fluid testing for THC improved the diagnostic performance of each test used on its own. The variability in cannabis use history and the plethora of available strains with varying levels of THC are certainly real-world situations that require assessment by any diagnostic tool if it is to be considered reliable and accurate. The robustness of the Cognalyzer® test is evident in its ability to maintain high levels of accuracy in a population of participants with heterogeneity of exposure, history and frequency of use. It is noteworthy that despite these variations the Cognalyzer® was capable of inhalation detection with high accuracy, sensitivity and specificity and is a novel and reliable cannabis detection tool that has potential to be administered both roadside and in the workplace.

Why carry out this study?

Current standards for identifying impairment are based on body fluid testing to detect recent cannabis use and behavioral assessments; therefore, determining cannabis impairment with accuracy and objectivity is crucial as legalization has increased use

The objective of this study was to assess the accuracy, sensitivity and specificity levels of a new electroencephalography (EEG)-based technology, the Cognalyzer®, for assessing cannabis’ psychoactive effects and characterize brainwave alterations associated with cannabis inhalation

What was learned from the study?

The accuracy of the Cognalyzer® was 83.6–85.5%, sensitivity was 85.1–88.7% and specificity was 79.0–85.1%

Combining the Cognalyzer® test with readily accepted oral fluid testing for THC improved the diagnostic performance of each test

The robustness of the Cognalyzer® test is evident in its ability to maintain high levels of accuracy in a population of participants with variable cannabis use histories, addressing a significant limitation of currently accepted methods