Literature DB >> 35493820

Association Between Concussions and Suicidality in High School Students in the United States.

Grant L Iverson1,2,3,4,5, Justin E Karr6.   

Abstract

Importance: Prior research has shown a statistically significant association between sustaining a concussion and suicidality in adolescents, but this prior research controlled for relatively few variables predictive of suicidality. Objective: To examine whether sustaining a concussion remained a significant predictor of suicidality after controlling for relevant covariates (e.g., sexual abuse/assault, bullying, substance use, depression), hypothesizing that the relationship between concussion and suicidality would become non-significant after controlling for these variables. Design: This study involved secondary data analysis of the 2019 Youth Risk Behavior Surveillance (YRBS) System, a national cross-sectional study of adolescents. Analyses were stratified by gender. Setting: A national sampling of U.S. high school students. Participants: Eleven thousand two hundred sixty-two students in the YRBS database, including 5,483 boys and 5,779 girls. Exposures: Participants included in the analyses reported whether, in the last year, they experienced a concussion and/or suicidality. Main Outcomes and Measures: The main outcome was suicidality (i.e., ideation, planning, attempt), which was predicted by concussion in an unadjusted analysis and by concussion along with other risk factors in a multivariable analysis.
Results: The final sample included 11,262 participants with available data on concussion and suicidality in the last year (14-18 years-old; 51.3% girls; 49.0% White). Per unadjusted odds ratios with 95% confidence intervals, there was a relationship between concussion and suicidal ideation [girls: OR = 1.46 (1.24, 1.73); boys: OR = 1.69 (1.41, 2.03)], planning (girls: OR = 1.39 [1.16, 1.66]; boys: OR = 1.76 [1.44, 2.14]), and attempt [girls: OR = 1.70 (1.32, 2.19); boys: OR = 3.13, (2.37, 4.15)]. These relationships became mostly non-significant after controlling for relevant risk factors for suicidality. The adjusted odds ratios showed no relationship between concussion and suicidal ideation [girls: OR = 1.11 (0.86, 1.44); boys: OR = 1.24 (0.92, 1.69)] or planning (girls: OR = 1.07 [0.82, 1.40]; boys: OR = 1.12 [0.82, 1.55]); but a significant relationship with suicide attempts in boys [OR = 1.98 (1.28, 3.04)], but not girls [OR = 1.05 (0.74, 1.49)]. Conclusions and Relevance: There was an association between concussion and suicidality in U.S. high school students; however, after controlling for other variables (e.g., depression, sexual abuse/assault, illicit drug use), there was no association between concussion and suicidality aside from a significant relationship between concussion and attempts in boys.
Copyright © 2022 Iverson and Karr.

Entities:  

Keywords:  concussion; head injury; sports; suicide; traumatic brain injury

Year:  2022        PMID: 35493820      PMCID: PMC9039310          DOI: 10.3389/fneur.2022.810361

Source DB:  PubMed          Journal:  Front Neurol        ISSN: 1664-2295            Impact factor:   4.003


Introduction

Suicidal thoughts and behaviors during adolescence are considered to be a public health problem, and many factors are known to be associated with increased risk for suicidality including genetics (1); female sex (2–4); mental disorders (5), such as depression (5, 6) and anxiety (6); impulsivity (7); attention-deficit hyperactivity disorder (8, 9); chronic illness (10); and sleep problems (11, 12). Substance use is associated with suicidality in adolescents (10), including illicit drugs (13, 14), marijuana (14, 15), alcohol (13, 15), and tobacco (13, 15). Psychosocial stressors and adverse childhood experiences also are associated with suicidality, including cyberbullying (16–18); not feeling safe at school (19); and sexual abuse, physical abuse, emotional abuse, and neglect (20–23). Parental psychological distress (24), marital problems (25, 26), academic problems (27), and being bullied (27, 28) at school are risk factors for mental health difficulties in youth. In contrast, a high level of physical activity, and participation in team sports, are associated with higher self-esteem and life satisfaction, and lower risk for psychological distress (29–32). Adolescents who are physically active have a lower risk for experiencing a mood disorder (33) and suicidality (34). Adolescents who participate in sports are less likely to report depression (35) or suicidal ideation (35–37). Involvement in school sports during adolescence is associated with better mental health in young adulthood (38), and adolescents who are physically active in general, and maintain that behavior into early adulthood, experience better mental health (39). Factors that are associated with lower risk for suicidality in youth include regular exercise (40) and if they perceive that their parents understand their problems, monitor their academic and leisure time activities, and respect their privacy (3). In recent years, epidemiological survey studies have found an association between sustaining a concussion, mostly during sports or recreational activities, and suicidality in high school students (41–43)—even after controlling for some factors known to be associated with suicidality, such as sex (2, 42, 43), depression (41), and bullying (42). However, none of these studies controlled for sexual abuse or assault, and none of the studies statistically modeled a more comprehensive set of factors known to be associated with suicidality in youth. Examining these associations is complex because some associations are small, and depression can be both a direct risk factor (i.e., a determinant) and a mediating variable (e.g., bullying leads to depression and suicidality). This gap in the literature was addressed, in the present study, by examining data from the 2019 “Youth Risk Behavior Survey” (YRBS) (44). Every 2 years, the Center for Disease Control and Prevention in the United States conducts a national survey of high school students (i.e., the YRBS). The purpose of this study is to examine the association between concussion and suicidality in high school students in the United States. It was hypothesized that in univariate analyses concussion would be associated with suicidality, similar to prior studies, but when controlling for other variables the association would be greatly attenuated or become non-significant.

Materials and Methods

Survey Methodology

The national survey is conducted every 2 years, during odd-numbered years, among students in grades 9–12 who attend a public and private schools in the United States. The Institutional Review Board of the CDC approved the protocol for the survey. The survey procedures were designed to allow for voluntary and anonymous participation. The students completed the survey in school during a single class period (~45 min). The total national sample included 13,872 students from 136 schools (44). The response rate for the schools was 75.1%, the response rate for the students was 80.3%, yielding an overall response rate of 60.3% [i.e., (student response rate) × (school response rate)]. A questionnaire failed quality control if fewer than 20 responses remained after editing or if the student marked the same answer for 15 or more consecutive questions—yielding 195 questionnaires that failed quality control. The survey data from 1991 through 2019 are publicly available (https://www.cdc.gov/healthyyouth/data/yrbs/).

Survey Questions and Combined Variables

There were 99 questions, 89 of which were included in the standard questionnaire and 10 additional questions were added to address areas of interest for CDC and other stakeholders. The survey questions, recall periods, response options, and definitions of each variable are provided in the 2019 YRBS questionnaire and data user's guide available on the website (https://www.cdc.gov/healthyyouth/data/yrbs/). The survey questions used in this study are reprinted in Table 1. In addition, variables were combined based on two or more survey questions, and those variables are also presented in Table 1.
Table 1

Survey questions and combined variables used in this study.

Q15. During the past 30 days, on how many days did you not go to school because you felt you would be unsafe at school or on your way to or from school? Endorsing 1 or more was coded as “Felt Unsafe at School.”
Q16. During the past 12 months, how many times has someone threatened or injured you with a weapon such as a gun, knife, or club on school property? Endorsing 1 or more was coded as “Threatened on School Property.”
Q19. Have you ever been physically forced to have sexual intercourse when you did not want to? Endorsing this question as “yes” was coded as “Sexual Abuse/Assault.”
QN17. During the past 12 months, how many times were you in a physical fight? Coded “yes” if endorsed as 1 or more.
QN20. During the past 12 months, how many times did anyone force you to do sexual things that you did not want to do? (Count such things as kissing, touching, or being physically forced to have sexual intercourse.)
QN21. During the past 12 months, how many times did someone you were dating or going out with force you to do sexual things that you did not want to do? (Count such things as kissing, touching, or being physically forced to have sexual intercourse.) (Endorsed as yes if the person had dated and this happened 1 or more times)
QN22: During the past 12 months, how many times did someone you were dating or going out with physically hurt you on purpose? (Count such things as being hit, slammed into something, or injured with an object or weapon.) (Endorsed as yes if the person had dated and this happened 1 or more times)
Q23. During the past 12 months, have you ever been bullied on school property?
Q24. During the past 12 months, have you ever been electronically bullied? (Count being bullied through texting, Instagram, Facebook, or other social media.) Endorsing either Q23 or Q24 as “yes” was coded as “Bullied.”
Q25. During the past 12 months, did you ever feel so sad or hopeless almost every day for two weeks or more in a row that you stopped doing some usual activities? Endorsing this question as “yes” was coded as “Depression in Past Year.”
Q26. During the past 12 months, did you ever seriously consider attempting suicide?
Q27. During the past 12 months, did you make a plan about how you would attempt suicide?
QN28. During the past 12 months, how many times did you actually attempt suicide? This variable was coded as “yes” if the student reported one or more attempts. Coded “yes” if 1 or more.
QN32. During the past 30 days, on how many days did you smoke cigarettes? Coded yes if 1 or more.
QN42. During the past 30 days, on how many days did you have 4 or more drinks of alcohol in a row, that is, within a couple of hours (if you are female) or 5 or more drinks of alcohol in a row, that is, within a couple of hours (if you are male)? Coded as “yes” if 1 or more for the variable “Current Binge Drinking.”
During your life, how many times have you (i) taken prescription pain medicine without a doctor's prescription or differently than how a doctor told you to use it (Q49); (ii) used any form of cocaine, including powder, crack, or freebase (Q50); (iii) sniffed glue, breathed the contents of aerosol spray cans, or inhaled any paints or sprays to get high (Q51); (iv) used heroin (also called smack, junk, or China White) (Q52); (v) used methamphetamines (also called speed, crystal meth, crank, ice, or meth) (Q53); (vi) used ecstasy (also called MDMA or Molly) (Q54); or (vii) used a needle to inject any illegal drug into your body (Q56)? The QN variables were used for “yes” or “no” and the regular variables were used to classify as 3 or more times.
Used Illicit Drugs 3 or More Times: Reported using one or more of the following drugs 3 or more times during lifetime: (i) prescription pain medication with a prescription; (ii) cocaine; (iii) inhalants; (iv) heroin; (v) methamphetamines; (vi) ecstasy/MDMA; or (vii) used a needle to inject any illegal drug into your body.
Q78. During the past 7 days, on how many days were you physically active for a total of at least 60 min per day? (Add up all the time you spent in any kind of physical activity that increased your heart rate and made you breathe hard some of the time.) Endorsing 5 or more days was coded as “Physically Active 5+ Days.”.
QN82. During the past 12 months, on how many sports teams did you play? (Count any teams run by your school or community groups.) Coded as “yes” if 1 or more teams.
Q83. During the past 12 months, how many times did you have a concussion from playing a sport or being physically active? Endorsing one or more was coded as “yes.”
Q88. On an average school night, how many hours of sleep do you get? Those who endorsed 5 or fewer hours were coded as “Insufficient Sleep.”
Q89. During the past 12 months, how would you describe your grades in school? Those who endorsed “Mostly D's” or “Mostly F's” were classified as “Low Grades.”
Q90. During the past 30 days, how many times have you taken prescription pain medicine without a doctor's prescription or differently than how a doctor told you to use it? Endorsing 1 or more was coded as “yes” for “Opiate Use in Past 30 Days.”
No Adversity or Depression: Responded “no” to the following questions: felt unsafe at school; threatened on school property; being in a physical fight; ever forced to have sexual intercourse; forced sexual things in past year; forced sexual things while dating (if the person dated in the past year); being physically hurt by someone while dating (if the person dated in the past year); bullied on school property; bullied electronically; and being depressed in the past year.

These questions were derived from the 2019 YRBS Data User's Guide (.

Survey questions and combined variables used in this study. These questions were derived from the 2019 YRBS Data User's Guide (.

Statistical Analyses

The frequency of suicidal ideation, making a suicide plan, and making a suicide attempt was compared between boys and girls using χ2 tests, with p < 0.05 indicating a significant difference in frequency on these variables between gender groups. Three sets of binary logistic regressions were conducted separately in boys and girls, with (1) suicidal ideation, (2) making a suicide plan, and (3) making a suicide attempt serving as dichotomous dependent variables. Each suicide variable was predicted by each of the following variables in bivariate models, with the unadjusted odds ratio (OR) reported for each analysis: (1) being physically active for 5 or more days, (2) history of sexual abuse or assault, (3) feeling unsafe at school, (4) being threatened at school, (5) being bullied electronically or at school, (6) having on average D or F grades, (7) having insufficient sleep (i.e., 5 or fewer h on school nights), (8) current binge drinking, (9) current cigarette use, (10) lifetime use of 3 or more illicit drugs, (11) depression (i.e., feeling sad or hopeless), and (12) having a concussion from playing a sport or physical activity. Table 1 provides the specific question text of each of these variables. An OR above 1.00 with a 95% confidence interval (CI) not including 1.00 indicated the predictor was associated with greater odds of endorsing the dependent variable, whereas an OR below 1.00 with a 95% CI not including 1.00 indicated the predictor was associated with reduced odds of endorsing the dependent variable. A multivariable logistic regression was conducted including all predictors in the same model to examine the association between concussion and the suicide-related variables controlling for other variables protective against or increasing risk for suicide. The ORs can be interpreted in the same manner, but they reflect the increase or decrease in odds of suicidal ideation, suicide planning, and suicide attempt among adolescents with a concussion in the prior year, controlling for all other variables in the model.

Results

There are 13,677 students in the database, of whom 151 had missing data on their sex and 72 had missing data on their age. Of those 13,442 students between the ages of 14 and 18, 13,243 (98.5%) answered the question about suicidal ideation, and of those there were 11,262 who answered the question about concussion as well. For the final sample of 11,262, there were 5,779 girls (51.3%) and 5,483 boys (48.7%). In regard to self-identified race and ethnicity, the sample had the following composition: 49.0% White, 15.8% Multiracial and Hispanic/Latinx, 13.6% Black/African American, 8.3% Hispanic/Latinx, 5.0% Asian, 4.8% Multiracial and non-Hispanic/Latinx, 0.8% American Indian or Alaskan Native, 0.4% Native Hawaiian or Pacific Islander, and 2.2% with no reported race/ethnicity. In the total sample, 19.3% endorsed having suicidal ideation in the past year, 15.7% endorsed having a plan for suicide, and 6.6% endorsed having attempted suicide. Girls endorsed suicidal ideation (24.7%) more frequently than boys [13.6%; χ2(1) = 221.49, p < 0.001; OR = 2.08 (95% CI: 1.89, 2.29)]; girls reported making a suicide plan (20.0%) more frequently than boys [11.3%; χ2(1) = 158.63, p < 0.001; OR = 1.96 (1.76, 2.18)]; and girls reported attempting suicide (11.4%) more frequently than boys [6.0%; χ2(1) = 75.63, p < 0.001; OR = 2.00 (95% CI: 1.71, 2.35)]. The percentages of youth endorsing suicidal ideation, plans, and attempts, stratified by sex and subgroups, are presented in Table 2. See Figure 1 for the percentages of youth, in specific subgroups, who endorsed suicidal ideation in the past year.
Table 2

Percentages of high school students endorsing suicidality.

Group Girls Boys Total
Suicidal ideation Suicide plan Suicide attempt Suicidal ideation Suicide plan Suicide attempt Suicidal ideation Suicide plan Suicide attempt
n % n % n % n % n % n % n % n % n %
Total sample5,77924.7%5,75120.0%4,42711.4%5,48313.6%5,46411.3%4,0746.0%11,26219.3%11,21515.8%8,5018.8%
Age 1479925.5%79221.5%54312.9%60413.7%5999.7%4035.2%1,40320.5%1,39116.4%9469.6%
Age 151,46324.9%1,45920.2%1,14312.3%1,37212.8%1,36911.5%9855.8%2,83519.0%2,82816.0%2,1289.3%
Age 161,56125.1%1,55520.2%1,20610.7%1,47314.3%1,46711.0%1,0876.6%3,03419.9%3,02215.7%2,2938.8%
Age 171,32524.5%1,31619.4%1,02511.6%1,30713.1%1,30411.6%1,0225.2%2,63218.8%2,62015.5%2,0478.4%
Age 18+63122.3%62918.4%5108.6%72714.4%72512.4%5777.3%1,35818.1%1,35415.2%1,0877.9%
Ethnicity
Hispanic/Latinx1,40222.8%1,39520.0%1,14012.1%1,31112.4%1,30810.4%1,0076.1%2,71317.8%2,70315.4%2,1479.3%
Non-Hispanic/Latinx4,33025.3%4,30920.0%3,25811.1%4,08613.8%4,07311.6%3,0146.0%8,41619.7%8,38215.9%6,2728.6%
Race
White2,85224.6%2,84118.8%2,2959.5%2,67113.7%2,66711.4%2,1195.2%5,52319.3%5,50815.2%4,4147.4%
Black/African American77124.8%76521.0%43716.2%75810.7%7519.7%4148.2%1,52917.8%1,51615.4%85112.3%
Hispanic/Latinx48720.3%48316.8%3739.9%44611.0%44610.3%3165.7%93315.9%92913.7%6898.0%
Asian28322.6%28118.1%2206.8%28216.7%28214.2%2196.4%56519.6%56316.2%4396.6%
American Indian/Alaskan Native4040.0%3933.3%2718.5%5527.3%5518.2%3627.8%9532.6%9424.5%6323.8%
Multiracial and Hispanic/Latinx91424.1%91121.7%76713.2%86513.2%86210.4%6916.2%1,77918.8%1,77316.2%1,4589.9%
Multiracial and Non-Hispanic/Latinx30635.6%30627.5%22120.4%23819.3%23815.1%1745.2%54428.5%54422.1%39513.7%
Played on one or more sports teams2,42022.7%2,41218.9%2,21010.6%2,50711.8%2,50310.1%2,2135.6%4,92717.2%4,91514.4%4,4238.1%
Physically active 5+ days2,07220.0%2,06416.8%1,6399.1%2,86110.6%2,8568.9%2,2304.4%4,93314.5%4,92012.2%3,8696.4%
Sexual Abuse/Assault49254.5%49248.4%44137.2%15644.9%15639.7%13432.8%64852.2%64846.3%57536.2%
Bullied electronically or at school1,72941.2%1,71534.4%1,34821.5%1,01430.6%1,00825.0%79415.1%2,74337.3%2,72330.9%2,14219.1%
Felt unsafe at school53644.0%52835.2%38325.1%38431.5%37824.6%26520.8%92038.8%90630.8%64823.3%
Threatened on School Property33552.2%33239.8%24831.5%42832.2%42427.6%31820.8%76341.0%75632.9%56625.4%
Currently smoke cigarettes25455.5%25048.0%19937.7%34429.4%34123.8%24816.9%59840.5%59134.0%44726.2%
Currently using marijuana1,16839.8%1,16233.0%91321.8%1,15421.1%1,14916.4%8759.7%2,32230.5%2,31124.7%1,78815.9%
Current binge drinking77435.9%77029.2%58516.6%63019.4%62716.9%46810.9%1,40428.5%1,39723.7%1,05314.1%
Opiate use in past 30 days35050.0%34643.1%30730.3%25635.9%25535.3%21031.0%60644.1%60139.8%51730.6%
Illicit drugs 3 or more times46454.3%45745.1%34133.7%47334.7%46629.8%32224.5%93744.4%92337.4%66329.3%
Sleeping 5 or fewer hours nightly1,42237.6%1,40831.4%1,10018.8%1,26921.4%1,25819.2%89511.2%2,69130.0%2,66625.7%1,99515.4%
Low grades (Ds and Fs)17346.8%17345.1%12628.6%31827.7%31326.8%22317.9%49134.4%48633.3%34921.8%
Depression in past year2,65946.4%2,65237.5%2,09522.0%1,45939.3%1,45331.8%1,15217.3%4,11843.9%4,10535.5%3,24720.3%
Concussion75331.2%74524.8%51716.8%93019.4%92416.7%62113.4%1,68324.7%1,66920.3%1,13814.9%
No adversity or depression1,4773.7%1,4733.3%1,3350.6%1,8372.3%1,8352.0%1,5840.4%3,3142.9%3,3082.5%2,9190.5%

Columns labeled n reflect the subgroup sample size that responded to the question on suicidal ideation, planning, or attempt and columns labeled % reflect the percent of that subgroup who endorsed suicidal ideation, planning, or making a suicide attempt. See .

Figure 1

Percentages of high school students endorsing suicidal ideation over the past year. See Table 1 for the definitions of the variables. See Table 2 for the sample sizes. The no adversity or depression group responded “no” to all of the questions relating to sexual abuse or assault, feeling unsafe at school, being bullied, and being depressed in the past year. The physically active group reported being active for 60 min or more for at least 5 of the past 7 days. The illicit drug use groups reported a lifetime history of using drugs 3 or more times.

Percentages of high school students endorsing suicidality. Columns labeled n reflect the subgroup sample size that responded to the question on suicidal ideation, planning, or attempt and columns labeled % reflect the percent of that subgroup who endorsed suicidal ideation, planning, or making a suicide attempt. See . Percentages of high school students endorsing suicidal ideation over the past year. See Table 1 for the definitions of the variables. See Table 2 for the sample sizes. The no adversity or depression group responded “no” to all of the questions relating to sexual abuse or assault, feeling unsafe at school, being bullied, and being depressed in the past year. The physically active group reported being active for 60 min or more for at least 5 of the past 7 days. The illicit drug use groups reported a lifetime history of using drugs 3 or more times. In the total sample, 12.8% reported sustaining a concussion in the past year (11.2% of girls and 14.4% of boys). Students who reported sustaining a concussion in the past year endorsed suicidal ideation more frequently (24.7%) than students who did not sustain a concussion in the past year [18.3%; χ2(1) = 36.69, p < 0.001; OR = 1.46 (1.29, 1.65)]. Students who reported sustaining a concussion endorsed a plan for suicide more frequently (20.3%) than students who did not sustain a concussion in the past year [15.0%; χ2(1) = 30.66, p < 0.001; OR = 1.45 (1.27, 1.65)]. Students who reported sustaining a concussion endorsed a suicide attempt more frequently (14.9%) than students who did not sustain a concussion in the past year [7.9%; χ2(1) = 61.72, p < 0.001; OR = 2.06 (1.72, 2.48)]. Multivariable binary logistic regression was used to examine the association between sustaining a concussion and suicidality after controlling for other known risk factors and correlates of suicidality. The results of the unadjusted and multivariable adjusted analyses for the girls are presented in Table 3 and for boys in Table 4. For girls, there was no significant association between sustaining a concussion and suicidal ideation, plans, or attempts after controlling for other risk factors. For boys, there was no significant association between sustaining a concussion and suicidal ideation or plans after controlling for other risk factors. However, there was a significant independent association between sustaining a concussion and suicide attempts.
Table 3

Logistic regressions predicting suicidality in girls.

Suicidal ideation
B SE Wald df p Adjusted odds ratio 95% Confidence interval Unadjusted odds ratio 95% Confidence interval
Lower Upper Lower Upper
Physically Active 5 + days−0.150.102.3310.1270.860.711.040.660.580.75
Sexual abuse/assault0.680.1424.991 <0.001 1.97 1.512.564.463.685.40
Felt unsafe at school−0.010.150.0110.9330.990.741.322.682.233.22
Threatened at school0.390.184.751 0.029 1.47 1.042.093.652.924.57
Bullied0.590.1037.491 <0.001 1.80 1.492.183.292.903.73
Low grades0.310.251.5410.2151.370.832.242.842.093.85
Insufficient sleep0.360.1013.471 <0.001 1.44 1.181.742.342.052.67
Current binge drinking0.140.131.3010.2541.150.901.481.941.652.28
Cigarettes (Current)0.470.224.691 0.030 1.60 1.052.464.163.225.37
Illicit drugs0.880.1631.631 <0.001 2.40 1.773.254.133.405.02
Depression2.380.12428.601 <0.001 10.84 8.6513.5813.3611.3215.76
Concussion history0.110.130.6410.4231.110.861.441.461.241.73
χ2(12) = 1,101.09, p < 0.001, Cox & Snell R2 = 0.258, Nagelkerke R2 = 0.385.
Suicide plan
Physically Active 5 + days−0.020.100.0410.8370.980.801.190.720.630.83
Sexual abuse/assault0.800.1336.201 <0.001 2.24 1.722.914.613.805.60
Felt unsafe at school−0.080.150.2510.6190.930.691.252.411.992.91
Threatened at school0.110.180.3410.5581.110.781.582.862.273.60
Bullied0.600.1036.831 <0.001 1.83 1.512.233.272.863.74
Low grades0.630.256.401 0.011 1.88 1.153.063.552.614.82
Insufficient sleep0.430.1018.221 <0.001 1.54 1.261.882.362.062.71
Current binge drinking−0.020.130.0310.8600.980.761.261.861.572.21
Cigarettes (Current)0.490.225.151 0.023 1.63 1.072.494.043.125.22
Illicit drugs0.870.1532.301 <0.001 2.38 1.763.203.753.074.57
Depression2.180.12309.521 <0.001 8.83 6.9311.2611.419.5313.67
Concussion history0.070.140.2510.6201.070.821.401.391.161.66
χ2(12) = 881.38, p < 0.001, Cox & Snell R2 = 0.213, Nagelkerke R2 = 0.337
Suicide attempt
Physically Active 5+ days−0.080.140.3810.5390.920.711.200.540.420.70
Sexual abuse/assault1.240.1567.441 <0.001 3.46 2.574.656.545.228.19
Felt unsafe at school−0.180.190.8610.3540.840.571.222.992.323.85
Threatened at school0.260.211.5510.2131.300.861.975.383.947.34
Bullied0.720.1330.731 <0.001 2.06 1.602.673.733.084.52
Low grades0.410.301.8610.1721.510.842.723.482.335.18
Insufficient sleep0.320.135.871 0.015 1.37 1.061.782.371.952.87
Current binge drinking−0.100.170.3710.5430.900.651.251.791.402.28
Cigarettes (Current)0.410.252.7510.0971.510.932.475.484.057.43
Illicit drugs0.900.1727.021 <0.001 2.46 1.753.454.733.696.07
Depression2.360.21125.391 <0.001 10.55 6.9815.9314.9310.8520.54
Concussion history0.050.180.0910.7681.050.741.491.701.322.19
χ2(12) = 570.75, p < 0.001. Cox & Snell R2 = 0.156; Nagelkerke R2 = 0.317

See .

Table 4

Logistic regressions predicting suicidality in boys.

Suicidal ideation
B SE Wald df p Adjusted odds ratio 95% Confidence interval Unadjusted odds ratio 95% Confidence interval
Lower Upper Lower Upper
Physically Active 5+ days−0.260.124.651 0.031 0.77 0.610.980.580.490.67
Sexual abuse/assault0.900.2711.411 0.001 2.46 1.464.155.794.178.04
Felt unsafe at school0.180.200.7510.3871.190.801.773.302.624.16
Threatened at school0.510.197.451 0.006 1.66 1.152.383.502.814.37
Bullied0.610.1321.831 <0.001 1.84 1.432.384.143.504.89
Low grades0.370.213.2010.0741.450.962.192.592.003.36
Insufficient sleep0.240.133.5510.0591.270.991.642.141.822.53
Current binge drinking−0.150.190.6510.4220.860.601.241.721.392.14
Cigarettes (Current)0.220.240.8910.3451.250.791.982.982.333.81
Illicit drugs0.720.1815.351 <0.001 2.06 1.442.964.053.285.00
Depression2.460.13356.971 <0.001 11.73 9.0915.1515.1312.5318.26
Concussion history0.220.161.9310.1641.240.921.691.691.412.03
χ2(12) = 744.88, p < 0.001, Cox & Snell R2 = 0.193, Nagelkerke R2 = 0.358
Suicide plan
Physically active 5+ Days−0.190.132.2610.1330.830.651.060.590.500.70
Sexual abuse/assault1.250.2622.481 <0.001 3.49 2.085.845.904.228.26
Felt unsafe at school−0.260.221.3810.2400.770.501.192.832.203.64
Threatened at school0.540.198.071 0.004 1.71 1.182.493.492.774.40
Bullied0.450.1410.631 0.001 1.57 1.202.063.763.144.49
Low grades0.480.215.221 0.022 1.62 1.072.463.152.414.11
Insufficient sleep0.380.138.151 0.004 1.46 1.131.892.432.042.90
Current binge drinking−0.030.190.0310.8740.970.671.411.771.402.22
Cigarettes (Current)0.080.240.1110.7441.080.671.752.702.073.52
Illicit drugs0.810.1918.511 <0.001 2.24 1.553.243.893.124.85
Depression2.370.14292.431 <0.001 10.75 8.1914.1111.949.8014.54
Concussion history0.120.160.5110.4741.120.821.551.761.442.14
χ2(12) = 629.70, p < 0.001, Cox & Snell R2 = 0.166, Nagelkerke R2 = 0.330
Suicidal attempt
B SE Wald df p Adjusted odds ratio 95% Confidence interval Unadjusted odds ratio 95% Confidence interval
Lower Upper Lower Upper
Physically Active 5 + days−0.290.192.3910.1220.740.511.080.540.420.70
Sexual abuse/assault1.240.3116.201 <0.001 3.47 1.896.369.496.4314.00
Felt unsafe at school0.150.280.2910.5921.160.672.035.003.596.97
Threatened at school0.510.254.031 0.045 1.66 1.012.745.383.947.34
Bullied0.550.207.931 0.005 1.74 1.182.564.523.475.88
Low grades0.740.287.051 0.008 2.10 1.213.624.082.815.92
Insufficient sleep0.250.191.6510.1991.280.881.882.682.053.50
Current binge drinking0.000.270.0010.9871.000.591.712.451.763.41
Cigarettes (Current)0.090.330.0710.7911.090.572.104.092.845.88
Illicit drugs0.950.2514.541 <0.001 2.59 1.594.237.065.229.56
Depression2.830.28104.461 <0.001 16.92 9.8329.1013.919.9319.50
Concussion history0.680.229.621 0.002 1.98 1.283.043.132.374.15
χ2(12) = 378.51, p < 0.001, Cox & Snell R2 = 0.116, Nagelkerke R2 = 0.351

See .

Logistic regressions predicting suicidality in girls. See . Logistic regressions predicting suicidality in boys. See . Exploratory logistic regressions were conducted without including depression as a variable, given its strong association with suicidality, as both a direct determinant and a mediator variable between the other covariates, and also how it might arise as a consequence of concussion. In addition, other variables were removed to examine whether concussion was associated with suicidality after removing additional variance associated with those variables, and to reduce the large number of predictors from 12 to 8 (i.e., Tables 5, 6). Feeling unsafe at school was removed because it was not a significant predictor in the larger model and due to some conceptual redundancy (shared variance) with bullying. Being threatened at school was also removed due to some potential redundancy with bullying. Current cigarette use was removed because it was a low frequency behavior relating to substance use (Table 2), and we wanted to keep higher frequency substance use behaviors such as binge drinking and illicit substance use in the model.
Table 5

Logistic regressions predicting suicidality in girls (reduced set of predictors).

Suicidal ideation
B SE Wald df p Adjusted odds ratio 95% Confidence interval Unadjusted odds ratio 95% Confidence interval
Lower Upper Lower Upper
Physically Active 5 + days−0.310.0912.781 <0.001 0.73 0.620.870.660.580.75
Sexual abuse/assault1.010.1269.451 <0.001 2.75 2.173.504.463.685.40
Bullied1.060.08156.941 <0.001 2.89 2.453.413.292.903.73
Low grades0.460.234.141 0.042 1.59 1.022.472.842.093.85
Insufficient sleep0.690.0959.461 <0.001 1.99 1.672.372.342.052.67
Current binge drinking0.380.1111.541 0.001 1.46 1.181.831.941.652.28
Illicit drugs1.170.1470.851 <0.001 3.22 2.454.234.133.405.02
Concussion history0.140.121.4110.2351.150.911.461.461.241.73
χ2(8) = 523.96, p < 0.001, Cox & Snell R2 = 0.130, Nagelkerke R2 = 0.194
Suicide plan
Physically Active 5+ Days−0.180.093.7810.0520.830.691.000.720.630.83
Sexual abuse/assault1.050.1272.521 <0.001 2.85 2.243.632.862.273.60
Bullied1.000.09124.581 <0.001 2.73 2.293.253.272.863.74
Low grades0.690.239.141 0.002 1.99 1.273.113.552.614.82
Insufficient sleep0.710.0957.311 <0.001 2.04 1.692.452.362.062.71
Current binge drinking0.250.124.511 0.034 1.29 1.021.631.861.572.21
Illicit drugs1.160.1468.631 <0.001 3.19 2.434.203.753.074.57
Concussion history0.120.130.9110.3411.130.881.451.391.161.66
χ2(8) = 464.32, p < 0.001, Cox & Snell R2 = 0.116, Nagelkerke R2 = 0.184
Suicide attempt
Physically Active 5+ days−0.200.132.4010.1210.820.641.050.540.420.70
Sexual abuse/assault1.430.14100.311 <0.001 4.20 3.175.566.545.228.19
Bullied1.100.1282.151 <0.001 3.01 2.373.823.733.084.52
Low grades0.540.293.4410.0641.710.973.033.482.335.18
Insufficient sleep0.570.1320.121 <0.001 1.76 1.372.252.371.952.87
Current binge drinking0.080.160.2510.6141.080.791.481.791.402.28
Illicit drugs1.210.1653.481 <0.001 3.34 2.424.624.733.696.07
Concussion history0.110.170.3810.5371.110.791.551.701.322.19
χ2(8) = 361.54, p < 0.001, Cox & Snell R2 = 0.100, Nagelkerke R2 = 0.203

See .

Table 6

Logistic regressions predicting suicidality in boys (reduced set of predictors).

Suicidal ideation
B SE Wald df p Adjusted odds ratio 95% Confidence interval Unadjusted odds ratio 95% Confidence interval
Lower Upper Lower Upper
Physically Active 5 + days−0.430.1116.461 <0.001 0.65 0.530.800.580.490.67
Sexual abuse/assault1.220.2328.151 <0.001 3.38 2.165.315.794.178.04
Bullied1.270.11129.541 <0.001 3.56 2.864.434.143.504.89
Low grades0.540.188.661 0.003 1.71 1.202.452.592.003.36
Insufficient sleep0.510.1219.931 <0.001 1.67 1.332.092.141.822.53
Current binge drinking0.020.160.0110.9201.020.741.391.721.392.14
Illicit drugs0.940.1635.111 <0.001 2.55 1.873.484.053.285.00
Concussion history0.300.144.651 0.031 1.35 1.031.771.691.412.03
χ2(8) = 305.61, p < 0.001, Cox & Snell R2 = 0 082, Nagelkerke R2 = 0.152
Suicide plan
Physically Active 5+ days−0.340.119.091 0.003 0.71 0.570.890.590.500.70
Sexual abuse/assault1.450.2339.631 <0.001 4.25 2.716.675.904.228.26
Bullied1.080.1280.981 <0.001 2.95 2.333.733.763.144.49
Low grades0.660.1912.261 <0.001 1.93 1.342.793.152.414.11
Insufficient sleep0.620.1226.491 <0.001 1.86 1.472.362.432.042.90
Current binge drinking0.070.170.2010.6561.080.781.491.771.402.22
Illicit drugs1.030.1640.531 <0.001 2.81 2.053.873.893.124.85
Concussion history0.230.152.4610.1171.260.941.681.761.442.14
χ2(8) = 278.91, p < 0.001, Cox & Snell R2 = 0.075, Nagelkerke R2 = 0.149
Suicide attempt
Physically Active 5 + days−0.470.177.081 0.008 0.63 0.450.880.540.420.70
Sexual abuse/assault1.600.2831.951 <0.001 4.95 2.848.629.496.4314.00
Bullied1.290.1755.501 <0.001 3.65 2.605.134.523.475.88
Low grades1.030.2516.631 <0.001 2.80 1.714.604.082.815.92
Insufficient Sleep0.500.187.451 0.006 1.64 1.152.342.682.053.50
Current binge drinking0.020.240.0110.9361.020.641.632.451.763.41
Illicit drugs1.210.2230.461 <0.001 3.34 2.185.137.065.229.56
Concussion history0.820.2017.041 <0.001 2.27 1.543.343.132.374.15
χ2(8) = 228.73, p < 0.001, Cox & Snell R2 = 0.070, Nagelkerke R2 = 0.206

See .

Logistic regressions predicting suicidality in girls (reduced set of predictors). See . Logistic regressions predicting suicidality in boys (reduced set of predictors). See . Again, for girls, there was no significant association between sustaining a concussion and suicidal ideation, plans, or attempts after controlling for other risk factors. For boys, there was a modest association between sustaining a concussion and suicidal ideation with the reduced set of predictors that was not observed when including the larger set of predictors in the model [p = 0.031, OR = 1.35 (95% CI: 1.03, 1.77)], meaning variance in suicidal ideation was explained by the removed predictors (e.g., current cigarette use and feeling unsafe or being threatened at school). The significant independent association between sustaining a concussion and suicide attempts [p < 0.001, OR = 2.27 (1.54, 3.34)] was observed in the reduced model, as was observed in the model with the full set of predictors. There was no significant association between sustaining a concussion and suicide planning after controlling for other risk factors.

Discussion

There was an association between sustaining a concussion in sports or recreational activity and suicidality in high school students surveyed in the United States in 2019. This is consistent with prior research using epidemiological survey data collected during 2013 (45) and 2017 (41–43, 46). After controlling for other variables, however, the associations between concussion and suicidality were greatly attenuated or became non-significant. For high school girls, there were no statistically significant associations in the adjusted analysis between concussion and suicidal thoughts, plans, or attempts. For girls, significant predictors of suicidal thoughts, in decreasing order of magnitude, were depression, sexual abuse/assault, current cigarette use, lifetime illicit drug use, being threatened at school, being bullied at school or electronically, low grades, feeling unsafe at school, insufficient sleep, and current binge drinking. For high school boys, there were no statistically significant associations in the adjusted analysis between concussion and suicidal thoughts or plans, but there was an association between concussion and suicide attempts. For boys, significant predictors of suicidal thoughts, in decreasing order of magnitude, were depression, sexual abuse/assault, being bullied at school or electronically, lifetime illicit drug use, being threatened at school, feeling unsafe at school, current cigarette use, low grades, insufficient sleep, and current binge drinking. Being physically active in the last 5 days was protective against suicidal ideation in both boys and girls, reducing the odds of suicidal thoughts by about half. Past researchers using data from the YRBS have reported that suicidality in high school students is associated with female sex (2, 4), sexual minority status (47–49), adverse childhood experiences (50, 51), physical teen dating violence (52), forced sexual intercourse (18, 53), body weight perceptions [both overweight (54, 55) and underweight (55)], illicit drug use (13, 18), prescription opioid misuse (56–58) [especially in girls (59)], alcohol use (15, 18, 60, 61), marijuana use (13, 15, 62), bullying (18, 55, 63, 64) and cyberbullying (18, 55, 63–65), frequent physical fighting (66), excessive television/video game/internet use (67, 68), insufficient sleep (69), and even soft drink consumption (70). With so many variables being associated with suicidality in youth—especially given the very large national sample allowing the identification of statistically significant findings with small effect sizes—it is not particularly surprising that concussion was also associated in the present study, in univariate analyses, and in prior studies using the 2013 (45) and 2017 YRBS (41–43, 46). These studies found associations between concussion and suicidal ideation (41, 42, 45), planning (41, 45), and attempts (41–43, 45) through unadjusted analyses and multivariable analyses adjusting for some relevant variables. These prior studies controlled for demographic variables (e.g., age, gender, race/ethnicity, sexual orientation) and some other variables associated with mental health problems, such as alcohol use (45); bullying (42, 46); grades (41, 43); and physical activity, grades, sports participation, poor sleep, alcohol use, marijuana use, and depression (41). In general, the previous studies controlled for a relatively small number of variables, however, and none controlled for a history of sexual abuse or assault. The current study included all of these variables and more in a single multivariable logistic regression model, observing only an increased risk of suicide attempt in boys associated with concussion. When including multiple variables in the model, most predictors have a large reduction in effect size because the predictors are interrelated (i.e., they share variance), and the predictors with the smallest univariate odd ratios (including concussion) become non-significant (see Tables 3, 4). As prior authors using YRBS data have noted, we cannot infer a causal association between concussion and suicidality given this is a cross-sectional survey study (41–43, 45, 46). That said, the association may be due, at least in part, to concussion being a major life stressor in boys, especially those with high athletic identity in their sport. Sustaining an injury in sports can be a major life stressor, and it can trigger or unmask difficulties with depression and suicidality (71). Concussions can remove athletes from their peer group and regular physical exercise, disrupt their academic performances, and lead to uncertainty about their futures. Such a disruption may have substantial psychological consequences for some adolescent boys. Among samples of predominantly male athletes, greater athletic identity has been associated with greater symptoms of depression following sport-related injuries (72), which may explain, in part, the current finding. The array of vulnerability and risk factors for suicidality likely co-occur as precipitants to, and/or consequences of, psychological distress. Adolescents who engage in substance use may do so to cope with underlying emotional problems, or they may do so as recreation with their peers, with increased or riskier use leading to psychological distress. Adolescents who have experienced sexual violence may reside in environments with greater risk for being bullied, threatened, or further victimized, contributing to mental health problems and increasing risk for suicidality. A student-athlete who experiences a concussion may be removed from the social support of a team setting and may exercise less frequently when removed from practice or play. Considering the relationship between physical activity, mental health, and cognitive functioning (73), student-athletes removed from sport may have worse mental health, struggle more in school, and engage in high-risk activities to cope with their distress (e.g., substance use), which may contribute to greater distress.

Limitations

This study has several important limitations. The methodology, of course, was a survey—and this cross-sectional survey study does not allow us to draw causal inferences. The survey was administered during a single class period and the students were surrounded by their peers. The CDC implemented a system check that attempted to identify surveys that reflected mischievous responding by excluding those who responded the same way 15 or more times in a row. There are other possible response biases and styles that could have been operative among some students, such as social desirability, under or over-reporting, extreme response styles, acquiescence response styles, or yea-saying and nay-saying. Attempting to study different types of response styles and response biases is beyond the scope of this study; it is possible that some students' survey responses were influenced by certain bias, styles, or mischievous responding. The students were not asked any details about their concussions, such as the severity of their symptoms, the extent to which they received treatment or rehabilitation, or their recovery time—and there is no way to determine, from this cross-sectional survey, how many students developed depression and/or suicidality after sustaining a concussion, or whether depression or suicidality appeared to be causally related to sustaining a concussion. There are also important statistical considerations. It is important to appreciate that we did not attempt to Model suicidality in youth, per se, but rather we aimed to examine the association of concussion with suicidality while considering multiple other variables of interest. There would be, as expected, some modest overfitting of the statistical model given the large number of variables included, although this overfitting is mitigated by the very large sample size. However, the most notable statistical issue relates to confounding and mediation. Depression is by far the strongest correlate with suicidality (i.e., greatest effect size). However, several of the other variables, for example sexual abuse/assault, illicit drug use, and bullying, are determinants of, or presumably causally linked to, depression. Thus, depression is a likely mediator between the predictors and suicidality. By including depression in the statistical model there is an introduction of bias due to either confounding (i.e., bidirectional relationship between depression and covariates) or mediation (i.e., a directional relationship). Therefore, the models are also presented without including depression (Tables 5, 6). It is also important to appreciate that other variables might contribute to confounding, mediation, or both. For example, bullying might also be linked to substance use. These interrelationships contribute to the large reductions in effect sizes, in the analyses, between the unadjusted (single variables) and adjusted (multiple variables) effect sizes.

Conclusions

Suicidality in high school students is a public health problem. There are many known predictors and correlates of suicidality in youth, with depression having, by far, the strongest association with suicidality in the present study. Other significant predictors, in both boys and girls, included a history of sexual abuse or assault and the use of illicit drugs. Concussion was not a significant predictor of ideation, plans, or attempts in girls after controlling for other variables, and it was not a significant predictor of ideation or plans in boys in multivariable analyses. Concussion remained a significant predictor in boys for suicide attempts after controlling for other variables. That said, children and adolescents presenting to specialty clinics for treatment and rehabilitation following concussion often experience emotional health difficulties, such as anxiety and depression (74, 75). Having pre-injury mental health problems is a risk factor for experiencing post-injury anxiety and depression (74, 76, 77)—and these are risk factors for suicidality. Because concussion can disrupt a young person's lifestyle in fairly dramatic ways, there are multiple potential reasons why some students who sustain concussions might evolve into experiencing depression and suicidality, and health care providers are encouraged to be mindful of this possibility and to provide swift, evidence-informed mental health care for youth who are suffering.

Data Availability Statement

Publicly available datasets were analyzed in this study. This data can be found here: Centers for Disease Control and Prevention (CDC) Youth Risk Behavior Surveillance System (YRBSS), https://www.cdc.gov/healthyyouth/data/yrbs/data.htm.

Ethics Statement

The Institutional Review Board of the CDC approved the protocol for the YRBS. Survey procedures were designed to protect students' privacy by allowing for anonymous and voluntary participation. Before survey administration, local parental permission procedures were followed.

Author Contributions

GI: conceptualized the study, conducted the literature review, conceptualized the statistical analyses, conducted the analyses, and drafted portions of the manuscript. JK: assisted with the literature review, conducted the analyses, drafted portions of the manuscript, and reviewed and approved the final manuscript. All authors contributed to the article and approved the submitted version.

Funding

GI has received unrestricted philanthropic support from ImPACT Applications, Inc., the Mooney-Reed Charitable Foundation, the National Rugby League, the Boston Bolts, and the Spaulding Research Institute. None of the above entities were involved in the study design, collection, analysis, interpretation of data, the writing of this article, or the decision to submit it for publication.

Conflict of Interest

GI serves as a scientific advisor for NanoDx® Inc., Sway Operations, LLC, and Highmark, Inc. He has a clinical and consulting practice in forensic neuropsychology, including expert testimony, involving individuals who have sustained mild TBIs (including athletes). He has received research funding from several test publishing companies, including ImPACT Applications, Inc., CNS Vital Signs, and Psychological Assessment Resources (PAR, Inc.). He has received research funding as a principal investigator from the National Football League, and salary support as a collaborator from the Harvard Integrated Program to Protect and Improve the Health of National Football League Players Association Members. The remaining author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
  75 in total

Review 1.  Annual Research Review: Interparental conflict and youth psychopathology: an evidence review and practice focused update.

Authors:  Gordon T Harold; Ruth Sellers
Journal:  J Child Psychol Psychiatry       Date:  2018-04       Impact factor: 8.982

2.  Investigating the association between bullying victimization and suicidal ideation among adolescents: Evidence from the 2017 Youth Risk Behavior Survey.

Authors:  Philip Baiden; Savarra K Tadeo
Journal:  Child Abuse Negl       Date:  2020-02-26

3.  School sport participation during adolescence and mental health in early adulthood.

Authors:  Rachel Jewett; Catherine M Sabiston; Jennifer Brunet; Erin K O'Loughlin; Tanya Scarapicchia; Jennifer O'Loughlin
Journal:  J Adolesc Health       Date:  2014-06-10       Impact factor: 5.012

4.  Physical Activity and Mental Disorder Among Adolescents in the United States.

Authors:  Jian-Ping He; Diana Paksarian; Kathleen R Merikangas
Journal:  J Adolesc Health       Date:  2018-08-29       Impact factor: 5.012

5.  Predicting Psychological Distress after Pediatric Concussion.

Authors:  Brian L Brooks; Vickie Plourde; Miriam H Beauchamp; Ken Tang; Keith Owen Yeates; Michelle Keightley; Peter Anderson; Naddley Désiré; Nick Barrowman; Roger Zemek
Journal:  J Neurotrauma       Date:  2018-10-10       Impact factor: 5.269

Review 6.  The relation between child maltreatment and adolescent suicidal behavior: a systematic review and critical examination of the literature.

Authors:  Adam B Miller; Christianne Esposito-Smythers; Julie T Weismoore; Keith D Renshaw
Journal:  Clin Child Fam Psychol Rev       Date:  2013-06

7.  Children at Risk for Suicide Attempt and Attempt-related Injuries: Findings from the 2007 Youth Risk Behavior Survey.

Authors:  Bethany A West; Monica H Swahn; Frances McCarty
Journal:  West J Emerg Med       Date:  2010-08

8.  Physical activity and sport participation among adolescents: associations with mental health in different age groups. Results from the Young-HUNT study: a cross-sectional survey.

Authors:  Maren Hjelle Guddal; Synne Øien Stensland; Milada Cvancarova Småstuen; Marianne Bakke Johnsen; John-Anker Zwart; Kjersti Storheim
Journal:  BMJ Open       Date:  2019-09-04       Impact factor: 2.692

Review 9.  Investigating correlates of athletic identity and sport-related injury outcomes: a scoping review.

Authors:  Tian Renton; Brian Petersen; Sidney Kennedy
Journal:  BMJ Open       Date:  2021-04-09       Impact factor: 2.692

Review 10.  A systematic review of the psychological and social benefits of participation in sport for children and adolescents: informing development of a conceptual model of health through sport.

Authors:  Rochelle M Eime; Janet A Young; Jack T Harvey; Melanie J Charity; Warren R Payne
Journal:  Int J Behav Nutr Phys Act       Date:  2013-08-15       Impact factor: 6.457

View more

北京卡尤迪生物科技股份有限公司 © 2022-2023.