Consumer perceptions of telehealth for mental health or substance abuse: a Twitter-based topic modeling analysis.

Objective: The objective of this study is to understand the primary topics of consumer discussion on Twitter associated with telehealth for mental health or substance abuse for prepandemic versus during-pandemic time-periods, using a state-of-the-art machine learning (ML) natural language processing (NLP) method. Materials and
Methods: The primary methodological phases of this project were: (1) collecting, cleaning, and filtering data (tweets) from January 2014 to June 2021, (2) describing the final corpus, (3) running and optimizing Bidirectional Encoder Representations from Transformers (BERT; using BERTopic in Python) models, and (4) human refinement of topic model results and thematic classification of topics.
Results: The number of tweets in this context increased by 4 times during the pandemic (2017 tweets prepandemic vs 8672 tweets during the pandemic). During the pandemic topics were more frequently mental health related than substance abuse related. Top during-pandemic topics were therapy, suicide, pain (associated with burnout and drinking), and mental health diagnoses such as ADHD and autism. Anxiety was a key topic of discussion both pre- and during the pandemic. Discussion: Telehealth for mental health and substance abuse is being discussed more frequently online, which implies growing demand. Given the topics extracted as proxies for demand, the most demand is currently for telehealth for mental health primarily, especially for children, parents, and therapy for those with anxiety or depression, and substance abuse secondarily. Conclusions: Scarce telehealth resources can be allocated more efficiently if topics of consumer discussion are included in resource allocation decision- and policy-making processes.

INTRODUCTION

Background and significance

Telehealth for mental health or substance abuse treatment is the use of virtual services, such as 2-way video applications (apps), for mental health and/or addiction treatment and support. Given that mental health and substance abuse are significant societal challenges and that access to help and support in these areas is vital, telehealth offers an excellent way to increase access and improve population health. However, mental health and substance abuse treatment was previously considered as most helpful when used in rural areas or by underserved populations, prior to the pandemic. While we know that such virtual services can be quite effective, less than half of mental health providers offered telehealth services prepandemic. Since the pandemic, however, “68% of outpatient mental health facilities and 57% of substance use disorder treatment facilities” now offer telehealth services,, Encouragingly, many barriers, such as reimbursement issues and questions about effectiveness in a virtual environment versus face-to-face, are being overcome as telehealth usage rates increase. However, telehealth usage rates peaked during the pandemic and are currently plateauing or declining. Thus, it is vital to understand whether consumers want such virtual services to be offered in the future, or even expanded. Such understandings are important for providers and policy makers making telehealth investment (and subsidization or reimbursement) decisions.

We know little, though, about how consumers view telehealth for mental health and/or substance abuse. It is important to understand whether consumers will continue to view telehealth as a viable and effective option for treatment. This is an important area of research as perceptions of such virtual services will play a significant role in whether such services continue to be invested in, incorporated into provider workflows, and utilized by those who need them most. Fortunately, social media data sources can provide excellent insights into consumer perceptions. Data sources such as Twitter are especially valuable for learning more about topical discourse and perceptions. Twitter has been used in notable studies to understand more about discourse in areas such as consumer perceptions of health, mental health,, the pandemic,, and vaccinations. However, a study has yet to be conducted on perceptions of telehealth for mental health or substance abuse. Further, we do not yet know how perceptions have changed during the pandemic. Even further, the use of state-of-the-art topic modeling can more accurately identify topics than prior methods, even if word meanings are ambiguous. Thus, we have an opportunity to apply state-of-the-art topic model to revealed preference data by consumers toward a better understanding of where current interests (and thus demand) lie for telehealth for mental health and/or substance abuse. Findings can guide how scare clinical resources are allocated and policies are proposed in the future.

Objective

The objective of this study is to understand the primary topics of discussion on Twitter associated with telehealth for mental health and/or substance abuse. We specifically compare topics within and between pre- and during-pandemic time periods, extracted through application of an machine learning (ML) natural language processing (NLP) method. While topic modeling is not new to the literature,,,, methodological advances in analysis of unstructured data, particularly using bidirectional transformers, offers an opportunity to provide more accurate, meaningful, and specific topic results. Methodologically, we seek to contribute through application of Bidirectional Encoder Representations from Transformers, BERT, topic modeling, applied to consumer discussions of telehealth for mental health and/or substance abuse on Twitter. We further contribute by proposing a series of steps for human evaluation and refinement of topic model results and classification into themes.

MATERIALS AND METHODS

The primary methodological phases of this project were: (1) collecting, cleaning, and filtering data (tweets) from Twitter at the intersection of telehealth and mental health and/or substance abuse for January 2014 through June 2021, (2) describing final corpus, (3) running and optimizing BERT topic models for the pre- and during-pandemic corpora, and (4) human evaluation and refinement of topic model results and thematic classification of topics.

Data collection and cleaning

Data was collected from Twitter using the Twitter Application Programming Interface (API) and the “twint” package in Python for the period of January 2014 to June 2021 by using keywords related to telehealth (eg, telehealth, telemedicine, virtual care, etc.; please see the Supplementary Appendix for a full list of keywords). Keywords were selected by searching on the web, MeSH, and UMLS for articles and term lists related to telehealth, mental health, and substance abuse. We conducted several searches, extracted keywords from relevant lists and articles, and then ran a deduplication method in Python and finalized. Our first phase of raw data collection resulted in an extract of 353 554 tweets related to telehealth for this period (see Figure 1). A time plot (available in the Supplementary Appendix) shows that most tweets per month in our corpus occurred during the pandemic (from about March 2020 to the end of our data collection period, with a peak in May of 2020).

Figure 1.

Data collection, cleaning, and topic modeling process.

The second step was to apply a keyword list specific to mental health and substance abuse to this original, raw Twitter extract, given that the raw tweets gathered were already related to telehealth (and virtual health services, telemedicine, etc.). Using keywords such as abuse, addict, grief, isolation, mental health, and narcotics (see the Supplementary Appendix for the full list of keywords used), we identified 100 460 tweets at the intersection of telehealth and mental health and/or substance abuse.

The final data cleaning challenge was that many of the 100 460 tweets came from organizations, such as hospitals, providers, or vendors, that advertised telehealth services. While such tweets provide evidence of service availability, they do not reflect consumer perceptions of telehealth. Thus, our final steps were to remove as many advertising (eg, telehealth services available in your area), promotional (eg, telehealth conference announcements), and bot-related tweets (eg, automatic retweets of articles or links to generate traffic) as possible. This filtering process required a time-intensive manual reading of tweets and development of a list of keywords associated with tweets that were deemed to be advertising, promotion, or auto generated (a full list of these keywords is available in the Supplemental Appendix). The result was a final corpus of 10 689 tweets, most relevant to consumer perceptions of telehealth for mental health and/or substance abuse.

Data analysis

To analyze our final corpus of 10 689 (cleaned and filtered) tweets, we: (1) developed keyword frequency charts (Figures 2 and 3), and (2) ran optimized BERT models (using BERTopic) to extract the primary topics of discussion (Tables 1–3). We ran the BERT models on the corpora with stop words removed and also without stop words removed. The results were similar. Thus, the results without stop words removed are presented here.

Figure 2.

Top word frequencies for the prepandemic tweet corpus (January 2014 to January 20, 2021).

Figure 3.

Top word frequencies for during-pandemic tweet corpus (January 21, 2020 to June 2021).

Table 1.

Top 10 topics in the prepandemic corpus

Rank	Topic title/label (topic no.)	Mental health specific?	Sub. abuse specific?	No. of tweets
1	Abuse—Telemedicine—Regulation—Addict.—Prescribing (3)	Yes	Yes	44
2	App—Pocket—Replace—PTSD—Therapy (4)	Yes	—	44
3	Telemental—Medical—Survey (5)	Yes	—	43
4	UFB—Lancet—Psychotherapy (7)	Yes	—	40
5	Shortage—MD—Duckworth—Psychiatrist (11)	Yes	—	32
6	Austin—Raise—Aetna—Addiction—Treatment (14)	—	Yes	30
7	Smoking—Cessation—Smoke (15)	—	Yes	30
8	LGBT—Panic—Homeless (16)	Yes	—	30
9	Attack—Disruption—Ashburner—Anxiety—Stress (24)	Yes	—	26
10	Spending—Initiative—Fund—Healthcare—Opioid (26)	—	Yes	24

Note: Topic titles/labels include the first 3 keywords assigned to the topic by the model. If additional context was needed, additional keywords were added to the title. The selection criteria for the topic being in the top 10 was the greatest number of associated tweets for topics selected as retained for consistency by the authors and the topic had to be mental health and/or substance related. Ranking was determined by the number of tweets, with Rank 1 being the topic assigned to the most tweets.

Once the corpus was finalized, we developed keyword frequency charts to visualize the most used words., This approach provides a broad overview of keyword “popularity” in the tweets extracted as well as differences in most frequently used words between pre- versus during-pandemic corpora. While this approach can provide a broad overview of the words most frequently observed in a corpus of data, this approach does not provide more fine-grained information about relationships between words and thus cannot be used to identify specific topics. Thus, the next step in our analysis was topic modeling.

Topic modeling has become a popular way to analyze unstructured data, such as tweets, to provide more fine-grained insights. One method frequently applied in prior studies is latent Dirichlet allocation (LDA). LDA assumes that each latent topic can be represented as a word distribution. To uncover latent topics, word co-occurrence is determined and then similarly co-occurring words across a corpus are grouped together, forming topics. While LDA is effective at identifying topics, especially when the corpus of data includes longer and larger documents, it is not as effective at inferring meaning of ambiguous words in context. More recent methods, including BERT, read sentences in both directions (ie, bidirectional instead of only left to right), apply multiple layers of analysis (eg, deep learning method pretrained on English language data, in this case, by Google from Wikipedia), and dynamically determine word embeddings depending on the surrounding words. Thus, BERT models are able to provide more contextually specific topics while understanding some of the nuances of language that LDA might miss. Thus, we opted to leverage this state-of-the-art topic NLP approach.

An additional choice was the type of BERT module to use to develop the topics, as BERT on its own derives embeddings but not topics. One choice was BERTopic, a Python package, which uses embeddings to generate clusters of keywords that form topics. Another option was to use BERTweet, which is similar to BERTopic, but is trained instead on English language tweets. To determine which approach would be best, we ran multiple BERTopic models and compared the results to BERTweet models. The BERTweet models returned many topics with low coherence. Further, many of the topics returned did not make sense to a human evaluator (ie, the keywords in many topics seemed very different from each other rather than similar). Therefore, we used the BERTopic package for our topic modeling results.

Finally, BERT on its own generates document embeddings, but not topics. The BERTopic package generates the document embeddings and then uses the embeddings to formulate topics. Specifically, the BERTopic package starts with generating document embeddings through BERT; after that, the model clusters topics into semantically similar clusters in 2 steps: (1) it uses UMAP to reduce the dimensionality of embeddings, (2) it uses HDBSCAN to cluster the reduced embeddings. In the last step, the BERTopic model creates topics and generates keywords for each topic by extracting class-specific words (see https://hackernoon.com/nlp-tutorial-topic-modeling-in-python-with-bertopic-372w35l9).

RESULTS

Data description

Within the final corpus of 10 689 tweets, 4589 unique usernames (ie, Twitter accounts) were represented with an average number of 2 tweets per account. Each of the tweets in this corpus had an average of 1.91 sentences per tweet (with a minimum of 1 and a maximum of 10) and an average of 21.82 words per tweet (with a minimum of 1 of and a maximum of 55).

We divided the final corpus of tweets into 2 time periods: (1) prepandemic (January 2014 to January 20, 2020), and (2) during-pandemic (January 21, 2020 through June 2021). The prepandemic corpus contained 2017 tweets. The during-pandemic corpus contained 8672 tweets.

Keyword frequencies

The results of the keyword frequencies are depicted in Figures 2 and 3. In the prepandemic corpus (Figure 2), the top 10 relevant keywords were: telehealth, health, mental, care, treatment, use, help, patient, abuse, and pain. In the during-pandemic corpus (Figure 3), the top 10 relevant keywords were: telehealth, health, mental, care, therapy, therapist, get, appointment, help, and treatment. From this general analysis, we can conclude: (1) telehealth was discussed more frequently on Twitter during the pandemic (6330 relevant tweets with “telehealth” as a keyword) than before the pandemic (922 relevant tweets with “telehealth” as a keyword) and (2) mental health and therapy are most frequently discussed, whereas substance abuse and treatment related keywords do not appear near as frequently, especially during the pandemic.

Topic modeling, selection, and thematic classification process

BERT models (BERTopic) were run on the prepandemic and during-pandemic corpora. A particular challenge with any topic model is to determine the optimal number of topics. We determined the optimal number of topics by sequentially running each BERT model on progressively larger numbers of topics, ranging from 5 topics to 80 topics, with 5 as the gap. We also ran each model using unigrams, bigrams, and trigrams (ie, for every number of topics from 5 to 80 with 5 as the gap, we ran each the model 3 times, once using unigrams, once using bigrams, and once using trigrams). We calculated coherence scores for all topics for every model run, specifically using CV coherence score based on cosine similarity. Coherence scores measure the degree of similarity among words within a particular topic. Higher coherence means that the keywords selected by the model for a topic are similar within the identified topic. The coherence score plots are available in the Supplementary Appendix. From this coherence score analysis, we determined that: (1) the optimal number of topics to be 70 via a unigram analysis for the prepandemic corpus and (2) the optimal number of topics to be 65 via a unigram analysis for the during-pandemic corpus.

Once the topics were generated by the models, not all topics contained entirely consistent keywords. This is a well-known issue in topic modeling. Following other ML work that has combined ML-based topic modeling with human evaluation and refinement,,, we applied the following steps to determine if topics should be retained as well as to assign titles/labels and primary themes:

We titled/labeled all topics using the first 3 keywords output by the model and, where additional context seemed necessary by the authors, we added additional keywords to the title.

We asked 18 Masters of Data Science and Analytics students to evaluate each topic for keyword consistency and also to classify each topic by primary theme. The final list of themes, used to group topics together, after refinement is: Funding/Coverage, Modalities, Needs/Conditions, Population Segments, Provider Types/Issues, Regulation, and Research.

Following guidelines for Fleiss’ Kappa calculation and evaluation, we dropped all topics with a interrater reliability (Fleiss’ Kappa) of less than or equal to 0.40, which the is upper bound of fair agreement. We retained all topics with a interrater reliabilty of greater than 0.80, which is the lower bound of almost perfect agreement.

All topics with interrater reliability ratings of between 0.41 and 0.80 were further evaluated by the 3 authors of this study. We discussed each topic in this range and made decisions by consensus as to whether to retain or drop the topic as well as to which theme to classify the topic as.

The authors further examined all the keywords in all of the topics and manually classified each topic as containing keywords specific to mental health, substance abuse, both, or neither. For mental health classification, keywords associated with treatment (eg, therapy, physiatrist, psychotherapy), emotions (eg, anxiety, depression, stress), and conditions (eg, bipolar, ADHD) were used to classify the topic as associated with mental health. For classification of topics to be associated with substance abuse or treatment, keywords associated with substances (eg, smoking, opioid), treatment (eg, rehabilitation), or post-treatment (eg, recovery, relapse) were used to classify the topic as associated with substance abuse or treatment. Topics that were not classified as specific to mental health or substance abuse, such as topics associated with pain being discussed regarding physical rather than emotional pain and rehabilitation topics specific to surgery rather than addiction rehabilitation, were dropped from the final set of topics.

As a result of this process, for the topic model run on the prepandemic corpus, 70 topics were generated and 33 were retained by the authors. For the topic model run on the during-pandemic corpus, 65 topics were generated and 32 were retained by the authors. Then, we performed 2 types of analysis: (1) identification and summary of the top topics for pre- versus during-pandemic corpus (Tables 1 and 2), and (2) summary and comparison of the topics and themes between the pre- and during-pandemic corpora (Table 3).

Table 2.

Top 10 topics in the during-pandemic corpus

Rank	Topic title/label (topic no.)	Mental health specific?	Sub. abuse specific?	No. of tweets
1	Therapist—Therapy—Session (1)	Yes	—	384
2	Bill—2112—3242—CA—Suicide—Firearms (2)	Yes	—	266
3	Pain—Burnout—Drink (3)	—	Yes	254
4	Autism—ADHD—Diagnosis (4)	Yes	—	214
5	Business—Chance—Plan—Suicide (5)	Yes	—	207
6	Pandemic—Depression—Rock—Stigma—Psychologist (7)	Yes	—	174
7	Stress—Exacerbate—Anxiety—Pandemic (8)	Yes	—	158
8	Insurance—Coverage—Copay—Psychiatrist (10)	Yes	—	155
9	Anxiety—Med—Xanax (15)	Yes	Yes	115
10	Teletherapy—Trans—Psychologist (17)	Yes	—	110

Note: Topic titles/labels include the first 3 keywords assigned to the topic by the model. If additional context was needed, additional keywords were added to the title. The selection criteria for the topic being in the top 10 was the greatest number of associated tweets for topics selected as retained for consistency by the authors and the topic had to be mental health and/or substance related. Ranking was determined by the number of tweets, with Rank 1 being the topic assigned to the most tweets.

Table 3.

Topic thematic classification results for pre- versus during pandemic comparison

Primary theme	Topics from the prepandemic corpus of tweets (Jan 2014—1/20/2020)a	Topics from the during-pandemic corpus of tweets (1/21/2020—June 2021)a
Funding/coverage	Spending—Initiative—Fund—Healthcare—Opioid (26) AHIP—Embrace—Nation—Abuse—Addiction (39) CMS—Code—Codes (41) Opioid—Nev—Coverage (47) Cloud—Insurer—Therapist—Coverage (50)	Insurance—Coverage—Copay—Psychiatrist (10)
Modalities	Telemental—Medical—Survey (5) App—Pocket—Replace—PTSD—Therapy (4)	Teletherapy—Trans—Psychologist (17) iPad—Depression—Sleep (34) Doctorcare247—App—Download (65)
Needs/conditions	Smoking—Cessation—Smoke (15) Alone—Annually—Employer—Hyperactivity—Bipolar (30) Stroke—Cognitive—Rehab (35) Stress—Tech—Paulsonnier (46) Addiction—T2—Sobriety (53) Cause—Effect—Depression (54) Telephysiotherapy—Rehabilitation—Kinect (64) Std—Grief—Stress (70)	Therapist—Therapy—Session (1) Pain—Burnout—Drink (3) Autism—ADHD—Diagnosis (4) Business—Chance—Plan—Suicide (5) Pandemic—Depression—Rock—Stigma—Psychologist (7) Stress—Exacerbate—Anxiety—Pandemic (8) Anxiety—Med—Xanax (15) Substance—Treatment—Relapse (21) Counseling—AI—Emotions (24) Isolation—Awareness—Calm—Anxiety (28) Cessation—Smoke—Smoking (38) Humor—Vern—Anger—Sadness—Depression (39) Emma—Lifeline—Isolation—Trauma—Psychological (42) Anxiety—Depression—Covid19 (44) Anxiety—CHCS—CHC—Shelter (52) Anxiety—Refill—Parenthood—Stress (55) Genie—Recovery—Rehab (56) Opioid—Oud—Treatment (60) Rehab—View—Episode—Therapy (62)
Population segments	Shortage—MD—Duckworth—Psychiatrist (11) Austin—Raise—Aetna—Addiction—Treatment (14) LGBT—Panic—Homeless (16) Percent—Bridge—Experience—Veteran—Psychiatry (40) Missouri—Rural—Minnesota (48) Visit—Virtual—Young (49) North—Dakota—Psychiatrist (60) Veteran—Psychotherapy—Exposure (62) Old—Adult—Visit—Anxiety—Depression (65)	Bill—2112—3242—CA—Suicide—Firearms (2) Perinatal—Pain—Stay—Depression (19) Cover—Insurance—NJ—Counselor (30) Child—Kid—Pediatrician—Emotionally (31) Counseling—Her—Counsel—Daughter—Psychologist (32) Counselor—Depression—County (OH)—Anxious (61)
Provider types/issues	Major—Space—Think—Psychiatrist (55) Illness—Appeal—Drought—Psychiatrist (56) Psychological—Technology—Remote (68)	Monday—Holiday—Therapist—Appointment (53) Naturopathic—Bhavna—Psychiatrist (63)
Regulation	Abuse—Telemedicine—Regulation—Addiction—Prescribing (3) Prescribe—Pass—Legislation—Addict (58)	—
Research	UFB—Lancet—Psychotherapy (7) Attack—Disruption—Ashburner—Anxiety—Stress (24) Depression—RCT—Healthline (37) Revolutionize—Mariea—Snell—Recovery—Treatment (44)	Patient—Therapy—Treatment—RCT (59)

The topics are sorted by topic number (in parenthesis). Lower topic numbers represent more keyword frequency in the tweets. In other words, the keywords in topic 1 appeared in the most tweets, the keywords in topic 2 were the second most frequently occurring in the tweets, etc. Thus, lower topic numbers represent the most “popular” topics.

Topic modeling results

The top prepandemic topics (Table 1) related to mental health include: abuse and addiction, apps for therapy and PTSD, telehealth for mental health (telemental), and a specific topic of psychiatrist shortages in Maryland (MD). The top topics related to substance abuse include: addiction related prescribing, addiction treatment, smoking cessation, and opioid funding.

The top during-pandemic topics (Table 2) related to mental health include: therapists and therapy, suicide, autism and ADHD, and business ventures related to novel ways of offering telehealth. Interestingly, only 2 topics in the top 10—related to drinking and the use of medications—were substance related. Out of the 32 topics retained for the during-pandemic corpus, only 7 were substance abuse related (as opposed to 26 specific to mental health). This was a surprising trend and suggests that telehealth services during the pandemic were mostly discussed regarding mental health as opposed to substance abuse. We return to this point later.

Further comparison of all the topics, rather than just the top 10, across both corpora is reported in Table 3.

The most notable differences in topics between pre- and during-pandemic topics are the shift in needs and conditions from relatively specific needs prepandemic (eg, smoking cessation, stroke rehab, addiction, depression) to a much broader set of primarily mental health issues during the pandemic, such as anxiety, depression, isolation, recovery, and therapy for mental health conditions, with therapy being the most frequently discussed topic. We also observed shift in discussion of funding and support for opioid support, coverage, and prescribing via telehealth prior to the pandemic to more of a focus on telehealth for mental health support and coverage during the pandemic. We further observe a population segment topic shift from a number of state specific issues and veterans to more of a focus on children and peri/postnatal depression, in addition to some state specific issues, during the pandemic. We also note that anxiety was a consistent theme pre- and during the pandemic, suggesting that telehealth for anxiety support remains in high demand. Also notable, as mentioned above, is that while substance abuse and rehabilitation related topics were certainly present during the pandemic, particularly for addiction, opioids, and alcohol/drinking, the number of mental health topics is much larger.

DISCUSSION

In sum, this study has examined tweets and overall topics of discussion on Twitter by consumers at the intersection of telehealth and mental health and/or substance abuse for pre- versus during-pandemic time periods.

Principal findings and contributions

First, we find BERT ML modeling for NLP to be effective at extracting relevant topics from Twitter data, which can be difficult to examine using ML methods due to the variety of discussion styles and topics in consumer discussions. We further find, though, that human evaluation is a key component of this process, as has also been found in other, recent ML studies focused on topic modeling. We contribute a specific method for addressing potential keyword inconsistencies in BERT topic models using 2 stages of human evaluation: (1) initial human evaluation of keywords and thematic classification with a relatively large number of assistants (18, in our case) and (2) a second stage of topic retention and thematic classification decision-making made by a smaller team of domain experts (the 3 authors of this study, in our case).

In regard to tweet frequency, we find that the number of tweets related to telehealth for mental health and/or substance abuse increased by 4-fold during the pandemic. We also find that during the pandemic, telehealth for mental health topics were more frequent than telehealth for substance abuse. If we consider these topical discussions as proxies for demand, these findings suggest significant demand for telehealth for mental health primarily, and telehealth for substance abuse secondarily.

Primary topics overall were found to be in the areas of anxiety, depression, addiction, and rehabilitation. While these topics and closely related topics remained consistent from prepandemic to during the pandemic, the most notable shifts were: (1) much more content and frequency of telehealth for mental health topics during the pandemic, especially in the areas of therapy and anxiety, (2) a shift in subpopulation topics from military, veterans, and PTSD prepandemic to more of a subpopulation focus on children and parent issues during the pandemic, (3) less modality and state specific discussions during the pandemic, suggesting that the technology and state-related regulations were less of an issue, and (4) a consistent set of topics focused on anxiety both pre- and during the pandemic. These findings suggest that resource allocations need to be made to and within pediatric providers, those providing services to parents, and anxiety support for all population segments. Much prior work has focused on telehealth for veterans,,,, which is commendable, but these findings suggest that services for children, parents, and those with anxiety are also population segments with high demand for these services. If we interpret these findings as revealed preferences by consumers, the top topics identified here can inform telehealth resource allocation decision-making processes.

Limitations

This study is limited by: (1) our focus on Twitter and English language tweets, (2) our focus on Twitter as our sole social media source, and (3) the keywords used. First, we focus on Twitter due to its broad use, but future inclusion of additional social media sites, including those used in other countries and by users who speak other languages, may enhance the results reported here. Additionally, the Twitter population of users has been shown to be younger and more Democrat, which means that studies based on tweets may not fully represent the US population. Further, while we did our best to select a comprehensive and accurate set of keywords, use of additional keywords may result in additional findings.

Implications

Telehealth for mental health and/or substance abuse should not be limited to only those in with specialized needs, in rural settings, in underserved areas, or even only veterans. Our findings suggest that telehealth for mental health discussion topics dominated during the pandemic, especially for anxiety and related issues, and frequently were related to children and their parents, which are good indicators of where scarce mental health resources should be invested if they are to meet current demand. In regard to virtual services for substance abuse, it is not clear if the lower frequency of substance abuse related tweets and topics is due to lack of availability of such services virtually or an indication of a preference for in-person availability of such services. This would be an excellent area for future research, as addiction, rehabilitation, alcohol, and opioid use were top telehealth for substance abuse related topics during the pandemic, but it is unclear as to how the demand for virtual services in this area would compare to virtual services for mental health, if virtual services were equally available in both areas.

AUTHOR CONTRIBUTIONS

All authors who contributed to this article, and who are listed meet all 4 criteria for authorship according to the ICMJE guidelines for authorship.

SUPPLEMENTARY MATERIAL

Supplementary material is available at JAMIA Open online.

CONFLICT OF INTEREST STATEMENT

None declared.

DATA AVAILABILITY

Data were derived from a source in the public domain: Twitter.com (using the Twitter API). Data can be downloaded using the date range specified in the article and the keywords specified in the Supplementary Materials.

Click here for additional data file.