Literature DB >> 34729796

Exploring the perceived negative and positive long-term impact of adolescent bullying victimization: A cross-national investigation.

Sara Pabian1,2, Francine Dehue3, Trijntje Völlink3, Heidi Vandebosch2.   

Abstract

Adolescent bullying victimization can have long-term mental health and well-being outcomes. This study focused on the potential mediating role of the perceived long-term negative and positive impact of adolescent bullying victimization to understand its relations with mental health and well-being problems during emerging adulthood. A retrospective study consisting of closed and open-ended questions was conducted among 1010 Flemish and 650 Dutch emerging adults aged 18-26. Path analyses among early victims (NFlemish  = 644; NDutch  = 217) demonstrated that the relations between bullying victimization intensity and current self-esteem, social interaction anxiety, and life satisfaction were all mediated by the perceived negative long-term impact of adolescent bullying victimization. Additionally, the open-ended questions provided a better understanding of the perceived negative and positive impact of adolescent bullying victimization. The present study offers insights into the importance of considering subjective perceptions of the overall impact of being bullied to understand long-term bullying victimization outcomes. Implications and limitations are discussed, including the need for longitudinal data-for instance, a three-wave panel study-to establish the temporal order of effects.
© 2021 The Authors. Aggressive Behavior published by Wiley Periodicals LLC.

Entities:  

Keywords:  bullying; impact; long-term; mental health; victimization; well-being

Mesh:

Year:  2021        PMID: 34729796      PMCID: PMC9298246          DOI: 10.1002/ab.22006

Source DB:  PubMed          Journal:  Aggress Behav        ISSN: 0096-140X            Impact factor:   3.047


INTRODUCTION

Both traditional bullying and cyberbullying have attracted the attention of many researchers in the last decade. Bullying is defined as an aggressive, intentional act or behavior that is carried out by a group or an individual repeatedly and over time against a victim who cannot easily defend himself or herself (Olweus, 1993). Whereas the majority of traditional bullying occurs at school, on school grounds, and on school buses, cyberbullying can occur anywhere, via information and communication technologies (ICT). Researchers have indicated unique features of cyberbullying, such as the possibility to behave anonymously and without face‐to‐face confrontation, fewer temporal and spatial limitations (i.e., it can happen any time, any place), less adult supervision, and the potential to be permanently recorded and publicly visible (Brewer & Kerslake, 2015; Slonje & Smith, 2008; Slonje et al., 2013; Sticca & Perren, 2013). Both traditional bullying and cyberbullying are pervasive problems among children and adolescents, with widely variable victimization rates, between 5.5% and 72% (e.g., Kann et al., 2016; Patchin & Hinduja, 2012; Tippett & Wolke, 2014), depending on the scope of the study (ranging from one form of cyberbullying and/or traditional bullying to many specific traditional and/or cyberbullying behaviors), the time frame and frequency being assessed (ranging from recently to ever and from once to several times a week), and the population under study. These prevalence rates are alarming, especially if the negative outcomes that are linked to bullying victimization are taken into account. Traditional bullying victimization among children and adolescents has been linked to mental health and well‐being problems in the short run, that is, a few days, weeks, or months after being bullied (e.g., Brunstein Klomek et al., 2007), but also in the long run, that is, years or even decades after victimization (e.g., Brunstein Klomek et al., 2015; Stapinski et al., 2014). In recent years, studies investigating the associations between recent cyberbullying victimization and mental health and well‐being have also been published (e.g., Kim et al., 2018; Palermiti et al., 2017), as have a few short‐term longitudinal studies in which longitudinal relationships between cyberbullying victimization and mental health and well‐being problems were found 6 months to 2 years after being cyberbullied (e.g., Bannink et al., 2014; Foody et al., 2015; Nicolai et al., 2018). There are initial indications in the literature that early cyberbullying experiences (during adolescence) are also related to mental health and well‐being problems during adulthood (Nicolai et al., 2018; Pabian & Vandebosch, 2019). Most studies on the long‐term outcomes of traditional bullying victimization or cyberbullying victimization have focused on how frequencies of past victimization and/or the seriousness of these bullying acts are associated with mental health and well‐being during adulthood. However, it is still unclear whether the perceived long‐term impact of these bullying experiences, that is, the perception of the magnitude of the negative/positive impact of adolescent victimization, plays an important role in the relation between former victimization and adult self‐esteem, social behavior, and the extent to which formerly victimized adults are satisfied with their lives. Cognitive theorists have indicated that subjective experiences of traumatic events—“the meaning the event has to the individual” (Allen, 2001, p. 5)—might be a better predictor of psychological outcomes than objective indicators like frequencies and the seriousness of bullying events. The literature has indicated that the subjective or perceived impact of traumatic experiences that occurred during adolescence is related to mental health and well‐being problems in adults, such as depression and anxiety (e.g., Giant & Vartanian, 2003; Riggs & Han, 2009). Until now, no research has provided empirical evidence on whether this relationship between the perceived impact of adolescent traumatic events and aversive psychological outcomes in adulthood also holds true for bullying victimization. Today, bullying victimization can include traditional bullying victimization, cyberbullying victimization, or both traditional and cyberbullying victimization. Toward this aim, the present study investigated, by means of quantitative survey data, the relationships between adolescent bullying victimization and three indicators of mental health and well‐being—namely, self‐esteem, social interaction anxiety, and life satisfaction—and sought to explain these relationships by focusing on the perceived long‐term negative and positive impact or meaning of former victimization. In addition, the study aimed to provide a better understanding of the perceived negative and positive long‐term impact of former victimization by analyzing qualitative data gathered via open‐ended survey questions. The study was conducted among two populations, Flemish and Dutch emerging adults between the age of 18 and 26, who were the victims of bullying between the age of 10 and 18. By employing a cross‐national design, the generalizability of the findings can be more thoroughly discussed.

BULLYING VICTIMIZATION AND SELF‐ESTEEM, SOCIAL INTERACTION ANXIETY, AND LIFE SATISFACTION

One of the mental health and well‐being problems related to bullying victimization concerns self‐esteem. Several studies have shown that victimization from traditional bullying and cyberbullying corresponds with reduced self‐esteem (e.g., Na et al., 2015; Wild et al., 2004). Although most of these studies were cross‐sectional and hence did not provide information on the direction of the relationship between victimization and self‐esteem, similar associations were found in retrospective and longitudinal studies. These studies have shown that being bullied in childhood is related to lower self‐esteem in early adolescence (e.g., Blood & Blood, 2016; Olweus, 1993; Pollastri et al., 2010) and to even lower levels of self‐esteem 40 years later (Takizawa et al., 2014). Thus, victimization from bullying appears to have an impact on self‐esteem in the long run. Self‐esteem is not stable but develops from childhood to early adulthood (Trzesniewski et al., 2003) through the feedback of others (Lodge & Feldman, 2007). Being bullied can be perceived by the victim as a negative judgment. The fear of being judged negatively is one of the components of social interaction anxiety (Pabian & Vandebosch, 2016). The relation between bullying victimization and social interaction anxiety has been demonstrated not only in several cross‐sectional studies (e.g., Coelho & Romāo, 2018; Ranta et al., 2009) but also in retrospective studies (e.g., McCabe et al., 2010) and longitudinal studies (e.g., Blood & Blood, 2016; Fahy et al., 2016). Being a victim of bullying may also result in decreased levels of life satisfaction. Several studies have found that high scores on victimization are associated with low scores on life satisfaction (e.g., Arnarsson et al., 2019; Varela et al., 2019). There are also initial indications of a long‐term association between bullying victimization and life satisfaction. In their longitudinal study consisting of a 1‐year time lag between two measurement points, Martin et al. (2008) found a bidirectional relation. However, in other longitudinal studies, a low level of life satisfaction was found to be only a consequence of being bullied, not a cause (Blood & Blood, 2016; Takizawa et al., 2014).

PROCESSES THAT EXPLAIN THE RELATIONSHIPS BETWEEN BULLYING VICTIMIZATION AND NEGATIVE MENTAL HEALTH AND WELL‐BEING OUTCOMES

Research has suggested some factors that can help explain the relationships between bullying victimization during adolescence and negative mental health and well‐being outcomes during adulthood. One of these factors is social support or the degree to which the social network of an individual supports coping with stressful events and/or achieving personal goals (C. Du et al., 2018; Hellfeldt et al., 2020; Lin et al., 2020). For instance, in a cross‐sectional study by C. Du et al. (2018) among adolescents, peer support partially mediated the relationship between bullying victimization and depression: (1) bullying victimization was positively associated with depression symptoms, with those with higher victimization scores reporting worse depression symptoms; (2) bullying victimization was negatively associated with peer support, with those with higher victimization scores reporting less peer support; and (3) peer support was negatively related to depression symptoms, with those who experienced more peer support reporting less severe depression symptoms. Evidence for the (partial) mediating role of family and teacher support in the relationship between cyberbullying victimization and depression and anxiety symptoms was also found in a cross‐sectional study by Hellfeldt et al. (2020) among adolescents. Also, Lin et al. (2020) found similar relationships in their retrospective study among university students. Two other factors that have been found to explain the relationship between bullying victimization during adolescence and negative mental health and well‐being outcomes during adulthood are personal resilience (Lin et al., 2020; Zhou et al., 2017) and self‐efficacy (Lin et al., 2020). Resilience can be broadly defined as the capacity of a person to adapt successfully to challenges that threaten his or her functioning, survival, or future development (Masten & Barnes, 2018). Self‐efficacy refers to the capacity to perform difficult tasks and attain desired outcomes (Bandura, 1997). In their retrospective study among Chinese and German university students, Lin et al. (2020) found that higher victimization frequency during adolescence was associated with lower levels of personal resilience and self‐efficacy, which in turn predicted poorer mental health. Taken together, there are indications in the literature that interpersonal positive resources (e.g., social support) and intrapersonal characteristics (e.g., personal resilience and self‐efficacy) can help explain the relationship between bullying victimization experiences and mental health and well‐being outcomes. These can serve as protective factors that mitigate the negative impact of bullying experiences on mental health, but they can also be influenced by the bullying experiences (Lin et al., 2020). However, these previously mentioned factors are all “objective” measures and as such fail to consider subjective experiences or the perceived impact of traumatic experiences. Within the field of trauma research, researchers have highlighted the importance of the perceived impact (e.g., Allen, 2001). Research suggests that the perceived impact of adolescent bullying victimization seems to be associated with negative mental health and well‐being outcomes (Allen, 2001; Riggs & Han, 2009). Those who perceive a large negative impact of a traumatic experience during adolescence have a higher probability of having mental health and well‐being problems during emerging adulthood (e.g., Giant & Vartanian, 2003; Riggs & Han, 2009). Besides the perceived negative impact, victims can also experience a positive impact from traumatic events, also called posttraumatic growth (e.g., Kou et al., 2021). The perceived positive impact of a traumatic event might be related to mental health and well‐being as well, but in a negative way: A higher perceived positive impact might be related to lower scores on negative mental health and well‐being outcomes. However, until now, research on the long‐term outcomes of bullying experiences during adolescence and emerging adulthood in terms of mental health and well‐being has not focused on the perceived positive or negative impact or meaning of past bullying experiences. Like the indirect relationships that have been found for the “objective” factors, it can be expected that the perceived impact of bullying experiences not only aggravates (negative impact) or mitigates (positive impact) negative mental health and well‐being outcomes but can also be influenced by the bullying experiences themselves. It is likely that a more intense negative bullying experience influences one's perception of its negative impact (in a positive way) and one's perception of its positive impact (in a negative way).

THE PRESENT STUDY

The literature review showed that (1) lower levels of self‐esteem, as well as higher levels of social interaction anxiety and lower levels of life satisfaction, have been found to be a long‐term outcome of bullying victimization; (2) “objective” factors have been identified that explain the relationships between bullying victimization and negative mental health and well‐being outcomes; and (3) research on traumatic experiences has suggested that the long‐term subjective impact or meaning of these experiences might be an explanatory factor of negative mental health and well‐being outcomes. The present study tested whether bullying victimization, both offline and online, during adolescence is associated with mental health and well‐being indicators among emerging adults, namely self‐esteem, social interaction anxiety, and life satisfaction. The study considered whether adults' perception of the negative and positive long‐term impact or meaning of adolescent bullying victimization could explain the relationship between the intensity of bullying victimization and indicators of mental health and well‐being. The following hypotheses were formulated based on the literature review: There is a relation between Flemish and Dutch emerging adults' former victimization experiences and their current self‐esteem, social interaction anxiety, and life satisfaction: (a) the longer and more severely former victims were bullied, the lower their self‐esteem, (b) the higher their anxiety, and (c) the lower their life satisfaction. There is a relation between the perceived impact of former victimization experiences and current self‐esteem, social interaction anxiety, and life satisfaction: the more negative the current perception of the impact of former victimization experiences, (a) the lower the self‐esteem, (b) the higher the social interaction anxiety, and (c) the lower the life satisfaction; and the more positive the current perception of the impact of former victimization experiences, (d) the higher the self‐esteem, (e) the lower the social interaction anxiety, and (f) the higher the life satisfaction. Emerging adults' former victimization experiences are negatively and indirectly related to self‐esteem and life satisfaction in adulthood, as well as positively and indirectly related to social interaction anxiety in adulthood, through a positive relation with the current perceived negative impact of former victimization experiences and a negative relation with the current perceived positive impact of former victimization experiences. These hypotheses are displayed in Figure 1 and were investigated with a retrospective survey questionnaire among a sample of Flemish and Dutch emerging adults between the age of 18 and 26 who were bullied during adolescence. In addition, to reflect the quantitative data, the present study also reported qualitative data, gathered with an open‐ended question on the retrospective survey questionnaire, to gain a better understanding of how Flemish and Dutch former victims of adolescent bullying perceive its impact.
Figure 1

Hypothesized model

Hypothesized model

METHODS

Procedure and participants

A large, quantitative, retrospective study was conducted among two samples: Flemish and Dutch emerging adults. APA Ethical Guidelines for Research with Human Subjects were followed during data collection. The first sample was representative in terms of gender and age and comprised 1010 Flemish (Dutch‐speaking Belgians) emerging adults (57.8% female) aged 18–26 (M age = 21.73, SD = 2.24). In this sample, 95.50% of respondents were Belgian nationals. The respondents were randomly selected from a representative panel by a market‐research facilitator and were asked to complete an online survey. Data gathering for the Flemish sample took place in September and October 2017. The respondents received credits from the market‐research facilitator that could be exchanged for vouchers for leisure activities. This study was approved by the Ethics Committee for the Social Sciences and Humanities of the University of Antwerp. The second sample consisted of a convenience sample of 650 Dutch emerging adults (78.61% female) aged 18–26 (M age = 21.61, SD = 2.33). In this sample, 98.84% of respondents were Dutch nationals. The respondents were recruited in two rounds that took place in June 2017 and June 2018. In both rounds, the respondents were recruited via social media (Facebook and Instagram) and individually by e‐mail and WhatsApp. In the second round, the respondents were also recruited via a school for higher education. This study was approved by the cETO (Commission Ethical Testing Research) of Open University Heerlen. The analytic samples consisted of emerging adults who indicated having been victimized between the age of 10 and 18 to investigate the long‐term impact of these experiences and to test the proposed model. In the Flemish sample, 644 emerging adults (63.76% of the total sample) indicated having been victimized between the age of 10 and 18. In the Dutch sample, 317 emerging adults (48.77% of the total sample) indicated having been victimized during the same age bracket. A two sample t‐test showed that a significantly larger share of the Flemish sample were victimized between the age of 10 and 18 compared to the Dutch sample (t(959) = 4.439, p < .001). Of the Dutch sample, 100 respondents did not provide answers to the main variables of the study and, therefore, the analytic Dutch sample comprised 217 emerging adults.

Instruments

The questionnaire consisted of a mix of validated scales and newly developed measures. The existing scales were translated from English to Dutch by the authors. The translated versions were translated back into English by an independent professional translator, and this English version was compared with the original English version to detect possible language issues. This resulted in small adjustments to the Dutch version. Preceding the data collection, a pilot study was conducted among 12 emerging adults between 18 and 20 years old (M age = 18.73; SDage = 0.65) who were recruited via a school within the personal network of the first author. Informed, active consent was provided by the school head and the students. The pilot study indicated the need for minor revisions to improve the readability of the survey. The measures of the present study are presented below in the same sequence as their appearance in the online survey. Reliability scores (internal consistency, Cronbach's α) and mean scores for each scale for the Flemish and Dutch samples are presented in Table 1.
Table 1

Descriptive statistics and zero‐order correlation coefficients

1. Bullying victimization2. Perceived negative impact3. Perceived positive impact4. Self‐esteem5. Social interaction anxiety6. Life satisfaction
CorrelationFlemish sampleDutch sampleFlemish sampleDutch sampleFlemish sampleDutch sampleFlemish sampleDutch sampleFlemish sampleDutch sampleFlemish sampleDutch sample
1.//
2.0.40*** 0.44*** //
3.−0.010.04−0.10* 0.12//
4.−0.17*** −0.23** −0.36*** −0.41*** 0.27*** 0.16* //
5.0.13** 0.21** 0.35*** 0.32*** −0.20*** −0.23** −0.62*** −0.53*** //
6.−0.20*** −0.25** −0.33*** −0.29** 0.23*** 0.19* 0.66*** 0.61*** −0.44*** −0.50*** //
M 1.621.615.034.305.244.892.702.871.651.283.183.48
SD0.720.712.362.632.392.720.510.560.790.780.89.70
Range1–31–31–101–101–101–101–41–40–40–41–51–5
Cronbach's α)//////.85.89.93.93.83.70

p < .05.

p < .01

p < .001.

Descriptive statistics and zero‐order correlation coefficients p < .05. p < .01 p < .001.

Bullying victimization intensity

First, Olweus's (1993) widely cited definition of bullying was presented to the respondents. According to this definition, bullying is an aggressive, intentional act or behavior that is carried out by a group or an individual repeatedly and over time against a victim who cannot easily defend himself or herself. After presenting this definition, respondents were asked to indicate whether they were bullied between the age of 10 and 18. The answer options were (1) “no,” (2) “lightly bullied for a short period of time,” (3) “lightly bullied for a long period of time,” (4) “severely bullied for a short period of time,” and (5) “severely bullied for a long period of time.” This single‐item measure has already been used in previous retrospective research on bullying (Lund et al., 2009; Olweus, 2007). Respondents who answered “no” to this question were not included in the analytic samples. For further analyses, bullying victimization intensity was recoded into an ordinal variable such that a score of “1” indicated “lightly bullied for a short period of time,” “2” indicated “lightly bullied for a long period of time” or “severely bullied for a short period of time,” and, finally, “3” indicated “severely bullied for a long period of time.”

Bullying victimization environment

Next, the respondents were asked to indicate the environment in which they were bullied. This question was only presented to those who indicated being a victim of bullying between the age of 10 and 18. The following answer options were provided: “only online,” “only offline,” and “both online and offline.”

Perceived impact

The perceived impact of bullying victimization was measured with two closed questions and two open questions. These questions were only presented to respondents who indicated being a victim of bullying between the age of 10 and 18. First, the respondents were asked to indicate the perceived general negative impact of their experiences: “How large do you perceive the negative impact of your bullying victimization experiences between the age of 10 and 18 on your current life?” Respondents could answer by means of a slider bar ranging from 1 (no impact at all) to 10 (strong negative impact). Next, the respondents were asked to describe how their bullying victimization experiences between the age of 10 and 18 had a negative impact on their current life (open question). The perceived positive impact of bullying victimization experiences was measured in the same way. First, a closed question was presented to measure the perceived positive impact of these experiences. A slider bar ranging from 1 (no impact at all) to 10 (strong positive impact) was provided. Next, an open‐ended question was provided in which respondents were asked to describe the perceived positive impact of their bullying victimization experiences on their current life.

Self‐esteem

All respondents completed the validated version of the Dutch Rosenberg's self‐esteem scale (Franck et al., 2008). The scale consists of 10 items, including “I feel that I have a number of good qualities” (Rosenberg, 1965). All of the items were assessed using 4‐point Likert scales with item responses ranging from 1 (strongly disagree) to 4 (strongly agree). Five items were recoded in such a way that higher values indicated higher self‐esteem. A mean score for self‐esteem was calculated for each respondent.

Social interaction anxiety

All respondents completed the validated Dutch Social Interaction Anxiety Scale (de Beurs et al., 2014), which is based on the validated Social Interaction Anxiety Scale by Mattick and Clarke (1998). This scale consists of 20 items, including “I find myself worrying that I won't know what to say in social situations.” For all items, 5‐point Likert scales with item responses ranging from 0 (not at all true) to 4 (extremely true) were provided. Three items were reverse‐coded so that a higher score indicated a stronger fear of social interaction. A mean score for social interaction anxiety was calculated for each respondent.

Life satisfaction

Overall life satisfaction was measured with four items that were constructed by Przybylski et al. (2013). All respondents were asked to think about their own life and to indicate how satisfied they were with the following aspects of their current life: physical health, emotional health (mental and emotional well‐being), personal relationships, and life as a whole. Five‐point Likert scales with item responses ranging from 1 (not satisfied at all) to 5 (very satisfied) were provided. For each respondent, a mean score for life satisfaction was calculated.

Data analysis and coding

The quantitative data were tested in two steps. In the first step, descriptive statistics were calculated. In the next step, the hypothesized model was tested using path modeling in Mplus 8.2 (Muthén & Muthén, 2018). Two path models were calculated, one for the Flemish sample and one for the Dutch sample (see Figures 2 and 3). Indirect effects were tested via bootstrap analyses with 1000 samples, generating a 95% confidence interval of the indirect effect. When the interval did not include 0, a significant indirect effect was present. The path model was performed with all variables regressed on the sociodemographic variables gender and age. Both path models had an acceptable fit: Flemish sample, CFI = 0.994; RMSEA = 0.097 (90% CI: 0.040–0.170); χ²(1) = 7.102, p = .01; Dutch sample, CFI = 0.993; RMSEA = 0.087 (90% CI: 0.000–0.222); χ²(1) = 2.63, p = .11.
Figure 2

Path model representing significant standardized coefficients for the Flemish sample. *p < .05, **p < .01, ***p < .001

Figure 3

Path model representing significant standardized coefficients for the Dutch sample. **p < .01, ***p < .001

Path model representing significant standardized coefficients for the Flemish sample. *p < .05, **p < .01, ***p < .001 Path model representing significant standardized coefficients for the Dutch sample. **p < .01, ***p < .001 The following procedures were used for cleaning and coding the qualitative data in both the Flemish and Dutch data sets. First, the answers in both data sets were screened to check whether the content matched the questions. More specifically, responses that referred to the negative or positive impact of bullying experiences during adolescence were omitted from the data sets, as the questions were intended to measure the perceived negative or positive impact of these experiences on current adult life. Second, the first author open‐coded the answers in the Flemish data set. In the next step, the second and third authors applied these codes to the Dutch data set. Thereafter, the authors compared and discussed the coding of both data sets and refined the codes. This resulted in three codes or three impact domains that were applicable to both the answers regarding the perceived negative impact of bullying experiences and the answers regarding the perceived positive impact of bullying experiences. In the final step, the first author recoded the data in both the Flemish and Dutch data sets. To each answer, at least one of the three codes was assigned. To some answers, two or three codes were assigned if the described impact matched more than one impact domain. The three impact domains that emerged from the data were perceived impact on social functioning (impact domain 1), perceived impact on one's personality or self‐image (impact domain 2), and perceived impact on one's health (impact domain 3).

RESULTS

General results

About one half of the respondents, across the two analytic samples, were bullied lightly for a short period of time during adolescence (n = 447; 51.9%). One out of three were bullied lightly for a longer period of time or severely for a short period of time (n = 299; 34.7%). A minority of the respondents were bullied severely for a long period of time (n = 115; 13.4%). A χ² test revealed no difference between the Flemish and Dutch samples regarding the intensity of the bullying (χ²(2) = 0.31, p = .856). Respondents also indicated the environment in which they were bullied between the age of 10 and 18. Across the samples, three out of four respondents were bullied exclusively offline (n = 616; 71.5%). One out of four respondents were bullied both offline and online (n = 224; 26%). Only a few respondents were bullied exclusively online (n = 21; 2.4%). A χ² test revealed no difference between the two samples regarding the environment in which the bullying took place (χ²(2) = 3.56, p = .169). Across the two analytic samples, the mean perceived negative impact score was M = 4.84 (SD = 2.45, range: 1–10) and the perceived positive impact score was somewhat higher, M = 5.15 (SD = 2.48, range: 1–10). Furthermore, the average scores for the other three main variables self‐esteem, social interaction anxiety, and life satisfaction were, respectively, M self‐esteem = 2.74 (SD = 0.52, range: 1–4), M social interaction anxiety = 1.58 (SD = 0.81, range: 0–4), and M life satisfaction = 3.24 (SD = 0.86, range: 1–5). The mean scores for each sample are displayed in Table 1. A set of t‐tests revealed that both samples differed from each other on the mean score on perceived negative impact (t(340.26) = 3.60, p < .001), self‐esteem (t(812) = −3.69, p < .001), social interaction anxiety (t(802) = 5.47, p < .001), and life satisfaction (t(257.72) = −4.32, p < .001), but not for bullying victimization (intensity of the bullying, t(859) = 0.15, p = .884) or for the perceived positive impact of bullying experiences (t(335.05) = 1.67, p = .095). More precisely, Flemish emerging adults scored significantly higher on the perceived negative impact of adolescent bullying victimization and social interaction anxiety and significantly lower on self‐esteem and life satisfaction compared to the Dutch emerging adults. Besides a set of t‐tests performed to determine whether differences existed for the outcome variables between the two samples, two MANOVA analyses were performed on the full samples, containing both victims and nonvictims, to test whether differences existed between those who were and were not victimized between the age of 10 and 18 with different levels of intensity. The results indicated that there were significant multivariate differences between bullying victimization intensity in a model containing the three outcome variables in both samples: Flemish sample, Pillai's Trace = 0.103, F(12,3015) = 8.936, p < .001, partial η 2 = 0.034; Dutch sample, Pillai's Trace = 0.144, F(12,915) = 3.85, p < .001, partial η 2 = 0.048. The between‐subjects test showed there was a significant difference between those with different levels of victimization intensity for self‐esteem (Flemish sample: F(4,1005) = 18.80, p < .001, partial η 2 = 0.07; Dutch sample: F(4,305) = 4.42, p < .01, partial η 2 = 0.055), social interaction anxiety (Flemish sample: F(41,005) = 16.52, p < .001, partial η 2 = 0.062; Dutch sample: F(4,305) = 8.09, p < .001, partial η 2 = 0.096), and life satisfaction (Flemish sample: F(4,1005) = 20.58, p < .001, partial η 2 = 0.076; Dutch sample: F(4,305) = 4.72, p < .01, partial η 2 = 0.058). The means for the three outcomes for each category in each sample are displayed in Table 2, as are the results of the post hoc tests. In sum, Bonferroni pairwise comparison showed that, in the Flemish sample, those who were not victimized had a significantly higher level of self‐esteem and a significantly lower level of social interaction anxiety compared to those who were victimized. Their level of life satisfaction was significantly higher than it was for those who were bullied, except of those who were lightly bullied for a short period of time. In the Dutch sample, there were less clear differences between those who were not victimized and those who were, as nonvictims did not differ with regard to some of the victimization intensity categories for self‐esteem, social interaction anxiety, and/or life satisfaction.
Table 2

Means and SDs for outcome variables, sorted by level of bullying victimization intensity

Outcome variableBullying victimization intensityFlemish sampleDutch sample
N Mean (SD) N Mean (SD)
Self‐esteemNot victimized3662.91 (0.46)1663.001 (0.45)
Lightly bullied for a short period of time3352.80a (0.50)732.991,2 (0.55)
Lightly bullied for a long period of time1672.58b (0.45)432.722 (0.53)
Severely bullied for a short period of time542.56b (0.55)102.741,2 (0.51)
Severely bullied for a long period of time882.64a,b (0.53)182.671,2 (0.62)
Social interaction anxietyNot victimized3661.31 (0.74)1660.981 (0.57)
Lightly bullied for a short period of time3351.54a (0.81)731.151,2 (0.80)
Lightly bullied for a long period of time1671.84b (0.73)431.352,3 (0.72)
Severely bullied for a short period of time541.70a,b (0.79)101.111,2,3 (0.66)
Severely bullied for a long period of time881.73a,b (0.80)181.813 (0.74)
Life satisfactionNot victimized3663.52a (0.86)1663.681 (0.68)
Lightly bullied for a short period of time3353.40a (0.82)733.611 (0.68)
Lightly bullied for a long period of time1672.91b (0.86)433.491,2 (0.66)
Severely bullied for a short period of time542.87b (0.91)103.181,2 (0.69)
Severely bullied for a long period of time883.07b (0.95)183.062 (0.69)

Note: Different alphabetical superscripts indicate that means for the outcome variable differed significantly between individuals with different levels of bullying victimization intensity at p < .05 in the Flemish sample. Different numeric superscripts indicate that means for the outcome variable differed significantly between individuals with different levels of bullying victimization intensity at p < .05 in the Dutch sample.

Means and SDs for outcome variables, sorted by level of bullying victimization intensity Note: Different alphabetical superscripts indicate that means for the outcome variable differed significantly between individuals with different levels of bullying victimization intensity at p < .05 in the Flemish sample. Different numeric superscripts indicate that means for the outcome variable differed significantly between individuals with different levels of bullying victimization intensity at p < .05 in the Dutch sample. For the two analytic samples, consisting of Flemish and Dutch emerging adults who were victimized during adolescence, the correlation coefficients between the main variables indicated that bullying victimization correlated, in both samples, significantly positively with the perceived negative impact of bullying experiences and with social interaction anxiety (see Table 1). In other words, a more intense bullying experience during adolescence was associated with a stronger perceived negative impact of adolescent bullying victimization and higher levels of social interaction anxiety during emerging adulthood. Furthermore, bullying victimization was, in both samples, significantly negatively associated with self‐esteem and life satisfaction: A more intense bullying experience during adolescence was related to lower levels of self‐esteem and life satisfaction. In both samples, the intensity of bullying experiences was not related to its perceived positive impact. Furthermore, the perceived negative impact of bullying experiences correlated, in both samples, with self‐esteem, social interaction anxiety, and life satisfaction. In other words, a greater perceived negative impact of adolescent bullying victimization was associated with lower levels of self‐esteem, higher levels of social interaction anxiety, and lower levels of life satisfaction. In contrast, a greater perceived positive impact of these experiences was, in both samples, related to higher levels of self‐esteem, lower levels of social interaction anxiety, and higher levels of life satisfaction.

Part 1: Path models investigating the associations between adolescent bullying victimization, perceived impact of adolescent bullying victimization, self‐esteem, social interaction anxiety, and life satisfaction

The path model of the Flemish sample (Figure 2) revealed that bullying victimization intensity during adolescence predicted the perceived negative impact of these experiences on current emerging adult life (β = .40, p < .001) but not their perceived positive impact (β = −.01, p = .736). Both the perceived negative impact and the perceived positive impact of adolescent bullying victimization on current life were associated with current levels of self‐esteem, social interaction anxiety, and life satisfaction. More precisely, a greater perceived negative impact of adolescent bullying victimization was related to lower levels of self‐esteem (β = −.33, p < .001), higher levels of social interaction anxiety (β = .34, p < .001), and lower levels of life satisfaction (β = −.27, p < .001). A greater perceived positive impact of these experiences was related to higher levels of self‐esteem (β = .24, p < .001), lower levels of social interaction anxiety (β = −.16, p < .001), and higher levels of life satisfaction (β = .21, p < .001). Indirect associations provided evidence for the mediating role of the perceived negative impact of adolescent bullying victimization in the relationships between bullying victimization intensity and self‐esteem, social interaction anxiety, and life satisfaction. Three indirect effects were found. First, there appeared to be a significant indirect, fully mediated relationship between bullying victimization intensity and self‐esteem via the perceived negative impact of adolescent bullying victimization (β = −.13, p < .001). The results of the bootstrap analysis confirmed this, as indicated by the 95% CI, which did not include zero [−0.17 to −0.09]. Second, the perceived negative impact of adolescent bullying victimization seemed to fully mediate the relationship between bullying victimization intensity and social interaction anxiety (β = .13, p < .001). The results of the bootstrap analysis also confirmed this, as indicated by the 95% CI, which did not include zero [0.10–0.17]. Finally, the perceived negative impact of adolescent bullying victimization seemed to mediate the relationship between bullying victimization intensity and life satisfaction (β = −.11, p < .001). The results of the bootstrap analysis also confirmed this, as indicated by the 95% CI, which did not include zero [−0.14 to −0.07]. This relationship was only partially mediated by the perceived negative impact of adolescent bullying victimization, as the model also provided evidence for a direct relationship between bullying victimization intensity and life satisfaction (β = −.10, p < .05). No evidence was found for the mediating role of the perceived positive impact of adolescent bullying victimization. The total explained variances (R²) of self‐esteem, social interaction anxiety, and life satisfaction were, respectively, 0.17, 0.14, and 0.15. The path model of the Dutch sample also indicated that bullying victimization intensity during adolescence predicted the perceived negative impact of these experiences on current emerging adult life (β = .44, p < .001) but not their perceived positive impact (β = .04, p = .518). In accordance with the Flemish sample, both the perceived negative impact and the perceived positive impact of adolescent bullying victimization on current life were associated with current levels of self‐esteem, social interaction anxiety, and life satisfaction. More precisely, a greater perceived negative impact of adolescent bullying victimization was related to lower levels of self‐esteem (β = −.42, p < .001), higher levels of social interaction anxiety (β = .32, p < .001), and lower levels of life satisfaction (β = −.25, p < .01). A greater perceived positive impact of adolescent bullying victimization was related to higher levels of self‐esteem (β = .24, p < .001), lower levels of social interaction anxiety (β = −.29, p < .001), and higher levels of life satisfaction (β = .27, p < .001). The Dutch sample also provided evidence for the mediating role of the perceived negative impact of adolescent bullying victimization in the relationships between bullying victimization intensity and self‐esteem, social interaction anxiety, and life satisfaction. Again, three indirect effects were found. First, there appeared to be a significant indirect, fully mediated relationship between bullying victimization intensity and self‐esteem via the perceived negative impact of adolescent bullying victimization (β = −.18, p < .001). The results of the bootstrap analysis confirmed this, as indicated by the 95% CI, which did not include zero [−0.26  to −0.11]. Second, the perceived negative impact of adolescent bullying victimization seemed to fully mediate the relationship between bullying victimization intensity and social interaction anxiety (β = .14, p < .001). The results of the bootstrap analysis also confirmed this, as indicated by the 95% CI, which did not include zero [0.07–0.20]. Finally, the perceived negative impact of adolescent bullying victimization seemed to mediate the relationship between bullying victimization intensity and life satisfaction (β = −.11, p < .01). The results of the bootstrap analysis also confirmed this, as indicated by the 95% CI, which did not include zero [−0.19 to −0.04]. In contrast to the Flemish sample, in which a partial mediation was found, in the Dutch sample this relationship was fully mediated by the perceived negative impact of adolescent bullying victimization, as the model provided no evidence for a direct relationship between bullying victimization intensity and life satisfaction (β = −.15, p = .055). Also, for the Dutch sample, no evidence was found for the mediating role of the perceived positive impact of adolescent bullying victimization. The total explained variances (R²) of self‐esteem, social interaction anxiety, and life satisfaction were, respectively, 0.26, 0.22, and 0.19. In sum, both path models were almost identical. For all tested relationships, except for the direct relationship between bullying victimization intensity and life satisfaction, the sizes of the regression coefficient were more or less equal. In contrast to the path model of the Dutch sample, the path model of the Flemish sample indicated a direct negative relationship between bullying victimization and life satisfaction. It should be noted, however, that the p‐value of this relationship in the Dutch sample was borderline insignificant (p = .055).

Part 2: Understanding the perceived impact

Emerging adults who were victimized during adolescence described how their bullying experiences between the age of 10 and 18 still had a negative/positive impact on their current life in response to the two open‐ended questions. Three impact domains resulted from the data: perceived impact on social functioning (impact domain 1), perceived impact on one's personality or self‐image (impact domain 2), and perceived impact on one's health (impact domain 3).

Perceived negative impact

About one out of five respondents in the Flemish sample and one out of four in the Dutch sample explained how their bullying experiences still had a negative impact on how they interacted socially (impact domain 1: Flemish sample, n = 118, 18.32%; Dutch sample, n = 58, 26.73%). This was reflected in statements such as “high levels of distrust towards (new) others,” “avoiding group interactions,” “having more difficulties with forming new friendships,” “having doubts about friendships,” “being afraid that others exclude me,” “being afraid that others will laugh at me,” “being afraid that others will gossip about me,” and “having more difficulties dealing with critique.” In both the Flemish and Dutch samples, about one out of three respondents gave examples of how adolescent bullying victimization still had a negative impact on their personality or self‐image (impact domain 2: Flemish sample, n = 195, 30.28%; Dutch sample, n = 70, 32.26%). The perceived negative impact on their personality or self‐image was described in many different ways, including “being more insecure,” “being more shy,” “being more aggressive,” “lacking confidence,” “having lower levels of self‐esteem,” and “being unsatisfied with myself.” Finally, fewer than 1 out of 10 Flemish and Dutch respondents described how adolescent bullying victimization had impacted their current health in a negative way (impact domain 3: Flemish sample, n = 38, 15.90%; Dutch sample, n = 17, 7.83%). The perceived negative impact on this domain was explained by examples such as “having more stress,” “being anxious,” “being lonely, being depressive,” “being obsessive,” “having an eating disorder,” and “having suicidal thoughts.”

Perceived positive impact

Whereas in the Flemish sample the largest share of respondents recounted how their bullying experiences had a positive impact on their current personality and self‐image (impact domain 2), the largest share of Dutch respondents provided examples for the perceived positive impact of these experiences on current social functioning (impact domain 1: Flemish sample, n = 97, 15.06%; Dutch sample, n = 58, 26.73%. Impact domain 2: Flemish sample, n = 129, 20.03%; Dutch sample, n = 54, 24.89%). The perceived positive impact of adolescent bullying victimization on current social functioning (impact domain 1) was described by examples such as “paying more attention to respecting others and treating them properly,” “valuing friendships more,” “having strong friendships with those who supported me when I was bullied,” “being better at defending myself,” “being more assertive,” and “being better in recognizing bullying and standing up for others who are bullied.” The perceived positive impact of adolescent bullying victimization on current personality or self‐image (impact domain 2) was described as, for instance, “being more self‐assured,” “having higher levels of self‐esteem,” “being more resilient,” “being stronger,” and “having a thicker skin.” Finally, in both samples, only a very small portion of respondents provided examples of how adolescent bullying victimization impacted current health (impact domain 3: Flemish sample, n = 8, 1.24%; Dutch sample, n = 3, 1.38%). Statements related to the positive impact on this domain were “being mentally stronger” and “being physically stronger (negative experiences are a motivator for keeping in shape).”

DISCUSSION

The aim of the present study was to investigate the long‐term impact of early bullying victimization (during adolescence) among emerging adults. In contrast to earlier research, the present study focused on a “subjective” factor, namely current perceived negative and positive impact or meaning of adolescent bullying victimization on self‐esteem, social interaction anxiety, and life satisfaction. For this study, quantitative and qualitative data were gathered among two samples of Flemish and Dutch emerging adults who were bullied between the age of 10 and 18, to discuss more thoroughly the generalizability of the findings. The same path model was tested for each sample. First, only limited evidence was found for the expected direct relationship between adolescent bullying victimization intensity and indicators of current mental health and well‐being (H1). More precisely, bullying victimization intensity was not directly associated with self‐esteem (H1a) or social interaction anxiety (H1b). A small direct relationship between bullying victimization intensity and life satisfaction was found in the Flemish sample, but not in the Dutch sample (H1c). These mixed results warrant further investigation. Nonetheless, it seems that all of the measured mental health and well‐being indicators were related to adolescent bullying victimization intensity, but in an indirect way. As expected, in both the Flemish and Dutch samples, a higher perceived negative impact of adolescent bullying victimization was associated with lower levels of self‐esteem (H2a), higher levels of social interaction anxiety (H2b), and lower levels of life satisfaction (H2c). Furthermore, in both samples, the relationships between bullying victimization intensity and self‐esteem, bullying victimization intensity and social interaction anxiety, and bullying victimization intensity and life satisfaction were all mediated by the perceived negative impact of adolescent bullying victimization, as hypothesized (H3). Longer and more severe bullying victimization was associated with a higher perceived negative impact. Taken together, in contrast to previous research (Arnarsson et al., 2019; Takizawa et al., 2014), the present study did not provide (or provided only very limited) evidence for direct relationships between bullying victimization intensity and mental health and well‐being outcomes. It seems that these relationships can be fully (for self‐esteem and social interaction anxiety) or partially (for life satisfaction) explained by the perceived negative long‐term impact of adolescent bullying victimization on current life. The importance of the perceived impact of adolescent bullying victimization in explaining long‐term mental health and well‐being outcomes has been highlighted in earlier research on traumatic events (e.g., Giant & Vartanian, 2003; Riggs & Han, 2009). The results of the present study seem to indicate that understanding impact perceptions can also help clarify the long‐term mental health and well‐being outcomes of bullying experiences (offline, online, or both offline and online) during adolescence. As expected, both path models indicated that a higher perceived positive impact of adolescent bullying victimization was related to higher levels of self‐esteem (H2d), lower levels of social interaction anxiety (H2e), and higher levels of life satisfaction (H2f). This corresponds with earlier findings. The higher perceived positive impact of these experiences might be related to the presence of protective factors (i.e., good performance at school, good social skills, coming from a stable family, being attached to parents, and having prosocial friends) during the bullying episodes as described by Ttofi et al. (2014). Contrary to our expectations, the perceived positive impact of these experiences did not mediate the relationship between bullying victimization intensity and self‐esteem, social interaction anxiety, and life satisfaction (H3). This can be explained by the absence of a relationship between bullying victimization intensity and the perceived positive impact of adolescent bullying victimization. It seems that being bullied more lightly and/or for a shorter period of time does not lead to a stronger belief that these experiences have a positive impact on current life. In other words, it seems that those who were victimized heavily and/or for longer periods of time perceived some positive impact of these experiences. Finally, although not the main focus of the present study, both path models showed significant correlations between the three indicators of mental health and well‐being: self‐esteem, social interaction, and life satisfaction. A relation was found between the current self‐esteem of former Flemish and Dutch bullying victims and their social interaction anxiety and life satisfaction at present: the lower their self‐esteem, (a) the higher their anxiety and (b) the lower their life satisfaction. These findings are in accordance with those generated in earlier research among general populations (e.g., Berger et al., 2017; H. Du et al., 2015; Moksnes & Espnes, 2013) but also seem to hold true for former victims of adolescent bullying. The present study generated qualitative data resulting from open‐ended question to explore in more depth how emerging adults describe the perceived negative and positive impact of adolescent bullying victimization on their current life. Flemish and Dutch emerging adults described positive and negative effects of adolescent bullying victimization along three domains, namely perceived impact on social functioning, perceived impact on personality or self‐image, and perceived impact on health. The qualitative data therefore confirmed the findings of the path models. More precisely, while the statistical analyses provided evidence for a relationship between adolescent bullying victimization and mental health and well‐being during early adulthood, the respondents qualitatively described the negative and positive impact of adolescent bullying victimization on their self‐esteem, social interaction anxiety, and life satisfaction. Their answers were not triggered by the quantitative measurements of the mental health and well‐being indicators, as these were assessed after the data for the open‐ended questions were collected. The present study has theoretical and practical implications. The results highlight the importance of measuring subjective perceptions of the overall impact of being bullied to understanding the long‐term impact of bullying victimization, in addition to measuring bullying victimization frequency and specific negative mental health and well‐being outcomes. Therefore, the perceived impact of bullying victimization should be added to theoretical models aimed at predicting its outcomes. In terms of practical implications, our results showed that emerging adults who perceived a negative impact of adolescent bullying victimization reported poorer mental health and well‐being; conversely, emerging adults who perceived a positive impact of adolescent bullying victimization reported better mental health and well‐being. Given these results, supporting former victims of bullying to think about the potential positive outcomes of such traumatic experiences may assist their mental health (Facchin et al., 2013). Some methods might help former victims of bullying overcome its negative impact on self‐esteem, life satisfaction, and social interaction. First, motivational interviewing, developed by Miller and Rollnick (1991), has been shown to be a useful intervention strategy in the treatment of lifestyle problems. This method helps users to focus on positive lifestyle experiences and to frame negative experiences as opportunities for learning. Additionally, the method can be used to improve self‐efficacy and take positive steps toward improving (social) skills. Focusing on strengths instead of shortcomings can help users build a more positive self‐image, strengthen self‐esteem, enhance life satisfaction, and improve social skills, which can in turn bolster social interactions. Second, expressive writing might have beneficial effects on the mental health and well‐being of adults who experienced traumatic events during adolescence (Facchin et al., 2013). Third, a trauma‐informed framework can be adopted by clinicians working with former victims. Harris and Fallot (2001) outlined the principles of a trauma‐informed framework and emphasized the need to focus on what happened to a person instead of what is wrong with a person. Harris and Fallot highlighted the need to move beyond simply asking about victimization. Instead, victimization and its impact should be viewed and understood from a holistic perspective—that is, in the context of the specific lives and traumatic experiences of former victims. The current study had some limitations that could serve as avenues for future research. First, as the study was based on self‐reported retrospective data, no causal conclusions can be made. Longitudinal research is necessary to more closely investigate changes over time and to establish the temporal order of effects. Several large research projects have already gathered longitudinal data to investigate the long‐term outcomes of traditional bullying victimization during adolescence among adults (e.g., Ttofi et al., 2014); however, similar data are not yet available for cyberbullying. In the present study, bullying victimization intensity and its perceived impact, as well as mental health and well‐being outcomes, were all measured at the same time point. For this reason, it cannot be determined whether one's perception of the impact of adolescent bullying victimization also influenced his or her estimation of bullying victimization intensity. The reported associations must be interpreted with caution and require replication in future studies. Second, the present study consisted of two samples: a representative sample of Flemish emerging adults and a convenience sample of Dutch emerging adults. Due to the limited amount of available resources, it was not possible to draw a representative sample of Dutch emerging adults, which resulted in a nonrepresentative sample with more female (78%) than male respondents. This might limit the generalizability of the Dutch path model. However, many of the relationships revealed in the present study were found in both the Flemish and Dutch samples, which does strengthen the generalizability of these relationships.
  30 in total

1.  Stability of self-esteem across the life span.

Authors:  Kali H Trzesniewski; M Brent Donnellan; Richard W Robins
Journal:  J Pers Soc Psychol       Date:  2003-01

2.  Cyberbullying: another main type of bullying?

Authors:  Robert Slonje; Peter K Smith
Journal:  Scand J Psychol       Date:  2008-04

Review 3.  Socioeconomic status and bullying: a meta-analysis.

Authors:  Neil Tippett; Dieter Wolke
Journal:  Am J Public Health       Date:  2014-04-17       Impact factor: 9.308

4.  Bullying, depression, and suicidality in adolescents.

Authors:  Anat Brunstein Klomek; Frank Marrocco; Marjorie Kleinman; Irvin S Schonfeld; Madelyn S Gould
Journal:  J Am Acad Child Adolesc Psychiatry       Date:  2007-01       Impact factor: 8.829

5.  Envisioning a trauma-informed service system: a vital paradigm shift.

Authors:  M Harris; R D Fallot
Journal:  New Dir Ment Health Serv       Date:  2001

6.  Adult health outcomes of childhood bullying victimization: evidence from a five-decade longitudinal British birth cohort.

Authors:  Ryu Takizawa; Barbara Maughan; Louise Arseneault
Journal:  Am J Psychiatry       Date:  2014-07       Impact factor: 18.112

7.  Self-esteem in pure bullies and bully/victims: a longitudinal analysis.

Authors:  Alisha R Pollastri; Esteban V Cardemil; Ellen H O'Donnell
Journal:  J Interpers Violence       Date:  2009-12-29

8.  Self-esteem and life satisfaction in adolescents-gender and age as potential moderators.

Authors:  Unni K Moksnes; Geir A Espnes
Journal:  Qual Life Res       Date:  2013-05-10       Impact factor: 4.147

9.  Exposure to bullying at school and depression in adulthood: a study of Danish men born in 1953.

Authors:  Rikke Lund; Karoline Kragelund Nielsen; Ditte Hjorth Hansen; Margit Kriegbaum; Drude Molbo; Pernille Due; Ulla Christensen
Journal:  Eur J Public Health       Date:  2008-11-12       Impact factor: 3.367

10.  Cyberbullying and Psychological Well-being in Young Adolescence: The Potential Protective Mediation Effects of Social Support from Family, Friends, and Teachers.

Authors:  Karin Hellfeldt; Laura López-Romero; Henrik Andershed
Journal:  Int J Environ Res Public Health       Date:  2019-12-19       Impact factor: 3.390

View more
  2 in total

1.  Correlational study on cyberbullying and social abilities in intercultural teenagers.

Authors:  María Tomé-Fernández; José Manuel Ortiz-Marcos; Christian Fernández-Leyva
Journal:  Front Psychol       Date:  2022-07-29

2.  Exploring the perceived negative and positive long-term impact of adolescent bullying victimization: A cross-national investigation.

Authors:  Sara Pabian; Francine Dehue; Trijntje Völlink; Heidi Vandebosch
Journal:  Aggress Behav       Date:  2021-11-02       Impact factor: 3.047

  2 in total

北京卡尤迪生物科技股份有限公司 © 2022-2023.