Who benefits and appreciates more? An evaluation of Online Service-Learning Projects in Mainland China during the COVID-19 pandemic.

Previous studies have reported the feasibility and benefits of online service learning, but little is known about who benefits more from online SL and who is more satisfied. This study addressed these questions based on an evaluation of online service learning projects implemented in Xi'an and Chengdu, China, during the COVID-19 pandemic. Pretest-posttest comparison showed significant positive changes in two intended learning outcomes (i.e., positive youth development qualities and service leadership qualities) and life satisfaction before and after the service among participating students in the Xi'an project. Cross-lagged modeling based on Xi'an and Chengdu data revealed that students with better initial positive youth development qualities tended to show increased service leadership qualities and life satisfaction after the service, while initial service leadership qualities did not predict positive youth development qualities and life satisfaction after the service. However, the two-line test indicated that some of these relationships might be curvilinear. Finally, Pearson correlation analyses demonstrated that students who experienced greater changes in positive youth development qualities and service leadership qualities reported better appraisal of course qualities, teacher performance, and course effectiveness, while multiple regression analyses showed the unique effects of change in service leadership qualities (but not change in positive youth development qualities) on the perception of teacher performance and course effectiveness. Altogether, this study not only showcases the potential benefits of online SL, but also provides initial evidence suggesting the variation in (perceived) benefits by students' psychosocial competencies and learning experiences. © The International Society for Quality-of-Life Studies (ISQOLS) and Springer Nature B.V. 2022.

Introduction

Service learning (hereafter “SL”), an experiential learning pedagogy that integrates academic learning with community service (Celio et al., 2011), is related to many beneficial outcomes among participating students, such as improving civic development, communication skills, academic performance, leadership development, positive youth development, and life satisfaction (Celio et al., 2011; Lin & Shek, 2021; Salam et al., 2019). SL is grounded in John Dewey’s (1986) experiential learning theory characterized by “learning by doing” and aims to both enhance students’ understanding of theories and fulfill community needs (Salam et al., 2019). Students are provided opportunities to apply their course knowledge and skills to projects that benefit a certain group of people in need or the community (Waldner et al., 2012). Such experiential learning allows students to apply theoretical knowledge in a real-world setting and solve real-life problems, strengthening their understanding of theoretical concepts and fostering multiple competencies and personal growth (Salam et al., 2019). In this way, both students and the community can benefit (Henry & Breyfogle, 2006).

Unfortunately, since 2020, the outbreak of the COVID-19 pandemic and related quarantine restrictions have created significant barriers to face-to-face teaching and learning, including the implementation of SL projects (Shek, 2021). Similar to other teaching and learning activities (Marinoni et al., 2020), some SL educators have sought to transform the SL into online activities (e.g., Lin & Shek 2021; Tian & Noel Jr,, 2020). However, the limited face-to-face human contact during the learning and serving processes presents two new questions: Will online SL projects remain beneficial for student learning and well-being? Are students satisfied with the online SL?

This study thus aims to illuminate understanding of the student outcomes of online or virtual SL by examining a credit-bearing course titled “Service Leadership Through Serving Children and Families with Special Needs (hereafter “Service Leadership”). Going beyond previous studies that merely reported students’ changes (i.e., pretest-posttest change) in learning outcomes or personal qualities (e.g., Lin & Shek 2021) and those that documented students’ subjective appraisal (i.e., course evaluation; Marcus et al., 2020), this study attempts to address two more questions: Who benefits more from the online SL (i.e., in terms of higher levels of learning outcomes) and who appreciates the online SL more (i.e., in terms of higher positive evaluation)? Overall, the findings from this study would advance our understanding of the effects of online SL, and bring implications for higher education and youth programs aimed at promoting leadership and well-being.

Online Service Learning and its benefits

The majority of existing SL projects use a direct mode, in which students provide on-site service and build a relationship with the people they serve (Gelmon et al., 2018). This type of SL allows close interaction and communication between service providers and recipients, which presumably has a direct impact on the service recipients (Tian & Noel Jr., 2020). In contrast, online SL incorporates virtual components into at least one aspect of the SL process (i.e., learning and/or service; Waldner et al., 2012). The online SL projects appeared to emerge during the COVID-19 pandemic (e.g., Burton & Winter 2021; Schmidt, 2021), although this concept has been proposed for years with an objective to engage students receiving distance education or service recipients having difficulties in visiting service sites (Chen et al., 2011; Waldner et al., 2012; Strait & Sauer, 2004).

Arguably, students who join online SL cannot experience “spontaneity and excitement by not being physically onsite” (Malvey et al., 2006, p. 191). Nevertheless, the rapid development of video conferencing tools, such as Zoom, has made synchronous learning and service feasible and effective. Online SL is particularly applicable to the young generation, who grow up in an Internet world and have greater readiness to learn electronic technology than previous generations (Marcus et al., 2020). Although studies of online SL projects (particularly evaluation studies) are sparse, several benefits have been observed from previous projects, including academic improvement (e.g., content knowledge mastery), skill enhancement (e.g., workplace skills and teamwork skills), increase in professional opportunities (e.g., networking), and personal growth (e.g., self-efficacy) (for a review, see Faulconer 2021). A few studies have also indicated that online SL yielded similar positive outcomes in students compared with face-to-face SL (Lin & Shek, 2021; McGorry, 2012).

However, compared with direct SL (for meta-analysis reviews, see Celio et al., 2011; Salam et al., 2019; Yori & Ye, 2012), evidence of the effectiveness of online SL is far from sufficient. Additionally, most of the outcomes under study are limited to knowledge, attitudes, competencies, and skills, with rare evidence showing its impact on student well-being. Fulfilling the learning objectives of SL is challenging and even painstaking, especially when it is implemented online amid the pandemic. Such a learning process might improve students’ positive feelings regarding life because of self-improvement or contribution to community, or it may make them feel even worse about life due to stress and uncertainty. It is thus crucial to understand whether online SL will relate to better or worse student well-being. Taken together, we need more scientific studies to examine online SL’s outcomes in learning and well-being.

Who benefits more from Online SL?

Despite emerging evidence about the learning benefits of online SL, to the best of our knowledge, little is known about who obtains more benefits from online SL. Considering that it concerns online learning and service, two competing perspectives regarding the beneficiaries of social media use may provide insights into this issue: “social compensation hypothesis” and “social enhancement hypothesis” (Valkenburg & Peter, 2007). The social compensation hypothesis (i.e., the “poor-get-richer hypothesis”) contends that individuals who are weak in offline social interaction engage more actively in social media and may obtain more benefits from social media use because it provides an alternative networking vehicle to compensate for their deficits in offline interaction (Poley & Luo, 2012). Accordingly, students who have a lower level of psychosocial competencies (such as communication skills) may possibly gain more from online SL because they could make use of web conferencing instruments that provide both verbal and written ways of spontaneous communication and allow non-face-to-face interaction. Such an online platform possibly facilitates those “shy” students to present themselves and communicate with others with minimized social anxiety (Appana, 2008; Shek et al., 2021a, b, c), which in turn improves their learning and service. By contrast, the social enhancement hypothesis (i.e., the “rich-get-richer hypothesis”) predicts that socially competent individuals may reap more benefits because it serves as an additional platform for them to apply their existing competencies (Cheng et al., 2019). Accordingly, students with a higher level of psychosocial competencies are more likely to benefit from the online learning and service because their preexisting competencies probably enable them to make better use of the online channel to learn new knowledge and practice new skills.

Indeed, in the literature on social media use, the findings regarding these two hypotheses are mixed (e.g., Bonetti et al., 2010; Danielsbacka et al., 2019; Lee, 2009). For example, supporting the social compensation hypothesis, Bonetti et al., (2010) found that people with higher levels of loneliness reported more online communication on personal and intimate topics. Supporting the social enhancement hypothesis, Lee (2019) indicated that adolescents with better social relationships used more online communication and then had more cohesive friends and stronger connectedness to school. Moreover, a few studies did not find that competence was related to the frequency and benefits of social media use (e.g., Ross et al., 2009), supporting neither of the hypotheses. In a meta-analysis of 161 studies, Cheng et al., (2019) found that the frequency of social media use was related to higher social anxiety and higher extraversion, supporting both hypotheses. Additionally, they found that higher extraversion and lower loneliness were related to more online social capital, supporting only the rich-get-richer effect. However, loneliness was not associated with frequency of social media use, and social anxiety was not associated with online social capital. In view of these inconsistent findings, a few scholars (e.g., Pouwels et al., 2021; Wang et al., 2018) contended that these two hypotheses were not necessarily mutually exclusive. It is possible that both high-competent and low-competent individuals (relative to medium–competent ones) receive the most benefits from social media use. Such a possibility could not be tested if only the linear relationship between competence attributes and benefits is investigated. For example, null relationships might occur when a positive relationship and a negative relationship between competence attributes and benefits counteract each other. By testing a quadratic relationship, Wang et al. (2018) demonstrated a U-shaped curvilinear relationship between extroversion and number of friends on social media, suggesting that the high extroverts and low extroverts obtained the most benefits. Therefore, to better test these two hypotheses, it is necessary to consider the curvilinear relationship between the competent attributes and beneficial outcomes in the context of SL.

In the educational field, based on a course on service leadership without the SL component, two studies (Lin & Shek, 2019; Zhu & Shek, 2021) using cross-lagged panel modeling found that students with higher initial positive youth development (PYD) qualities tend to improve their service leadership qualities at the end of the course, controlling for autoregressive effects (i.e., temporal stabilities from the pretest to the posttest) of the variables, thus supporting the rich-get-richer effect. However, similar to most prior studies, these two studies did not consider a curvilinear relationship, thus overlooking the possibility of the coexistence of social compensation and social enhancement. Altogether, it is important to test a curvilinear relationship between students’ initial competence and their learning outcomes.

Who appreciates Online SL more?

Despite the documented feasibility and benefits of online SL, students may have different views toward the online SL’s curriculum and design, teacher’s performance and its effectiveness in helping students achieve the learning outcomes (Shek et al., 2020), as well as benefits to the service recipients (Shek et al., 2021a, b, c). Students’ subjective evaluation serves as a crucial indicator of course effectiveness (Marsh & Roche, 1997), further guiding action to improve the project (Shek, 2010). Given its importance, it is valuable to understand what contributes to a positive evaluation. Theoretically, such a subjective evaluation should reflect the quality and effectiveness of the course and correspond to students’ actual learning progress and outcomes (Marsh, 1994). Accordingly, it is possible that students’ appraisals of the online SL would be related to how much they have gained from this project. If students reap more gains from the project, they will appreciate it more.

The above conjecture requires empirical testing because of a lack of direct evidence. Previous studies usually reported students’ views on online SL via surveys or interviews without explicating what contributes to these views (Marcus et al., 2020). Even studies that examined both students’ changes in learning outcomes and subjective evaluation did not attempt to relate them to each other (e.g., Lin & Shek 2021). In the field of health and social services, although there are studies relating clients or patients’ subjective evaluation and their experience of change, the findings are not entirely consistent, with some studies revealing positive relationships (e.g., Shek 2010; 2014) while others showing non-significant relationships (e.g., Walsh & Lord 2004). Thus, it is essential to empirically test whether students’ experiences of change in learning outcomes influence their subjective evaluations of online SL.

The current study

Given the aforementioned research gaps, this study aims to advance understanding of the student outcomes of online SL based on the course “Service Leadership.” This course is built on the Service Leadership Model developed by Chung (2012) in response to the transformation of leadership qualities in service economies. First, the Service Leadership Model redefines leadership as a service provided to satisfy a wide range of needs, including needs of self, groups, community, and society (Shek & Lin, 2015). Second, it upholds three attributes that are intrinsic to effective service leadership (i.e., moral character, caring disposition, and competencies), and it emphasizes everyone’s potential and capacity to lead himself/herself and others. Accordingly, the Service Leadership course is designed to nurture students’ attitudes and values of service leadership, cultivate their self-leadership, and promote moral character, caring disposition, and competencies in terms of a set of PYD qualities (e.g., social competence, emotional competence, and resilience) through the integration of class learning and community service (Shek et al., 2020; Zhu & Shek, 2021).

Two signature projects under this course are designed to serve children in need, such as migrant children and left-behind children in Chengdu and Xi’an of mainland China (Lin & Shek, 2021). In China, while migrant children usually refer to children who move from rural regions to urban regions along with their parents for better job opportunities, left-behind children refer to those who stay in rural hometowns, with their parents leaving to work in cities (Zhao & Yu, 2016). These groups of children have been found to show a higher risk of mental health and behavioral problems compared with their urban counterparts (Hu et al., 2014; Zhang et al., 2019). Students taking this course are required to acquire knowledge of service leadership and practical skills of service delivery through lectures and workshops before the service and then are offered a summer camp. After the service, they are also required to present their experiences and reflections on the whole SL in the last workshop.

Since 2020, this summer camp has been moved online due to the COVID-19 pandemic with the technical support of VooV Meeting, a video conferencing tool developed by Tencent. In 2021, the Chengdu project offered a five-day online summer camp, while the Xi’an project offered a four-day online summer camp for one primary school. Approximately 480 children in Chengdu attended the summer camp at home during their summer vacation, while nearly 389 children in Xi’an attended it at school together with their classmates after their final examination. University students provided online services either at home or on campus. With an overall goal of fostering children’s competences, resilience, and aspiration, university students provided children with four categories of lessons—Living English, Science, Health and Personal Growth, and organized opening and closing ceremonies. The Chengdu project provided 15 lessons, each lasting for 40 minutes, and the Xi’an project provided 12 lessons, each lasting for 30 minutes. Students applied different tools such as PowerPoint, animation, and videos for the teaching. Due to the limitations of the online context, no outdoor activities (e.g., physical exercises) were held during the summer camp. Instead, multiple in-class activities, such as drawing, games, and handcraft making, were frequently used to facilitate the teaching. Lin & Shek (2021) investigated the effectiveness of this online SL course with three service sites (Hong Kong, Chengdu, and Xi’an) in the first offering, and they found significant improvement of students in PYD qualities, service leadership qualities, and life satisfaction as well as favorable evaluation toward the projects. However, this study did not investigate the factors related to the improvement and the positive evaluation.

The current study further addresses these two issues. It adopts two approaches to understanding student outcomes: pretest-posttest and post-course evaluations. The pretest-posttest evaluation is used to investigate the changes in intended learning qualities (i.e., PYD qualities and service leadership qualities) before and after the service via the one-group pretest-posttest design. Moreover, although well-being is not an intended learning quality, previous studies have suggested that leadership courses and SL projects might help improve students’ well-being (Lin & Shek, 2019, 2021). Thus, we included a well-established indicator of well-being—life satisfaction—as a student outcome. The post-course evaluation was used to solicit students’ subjective perceptions of course effectiveness, including course quality, teacher performance, and perceived course effectiveness (Shek, 2010). Based on previous evaluation studies on this course (e.g., Lin & Shek 2021), we expected that students would show positive changes in PYD qualities (Hypothesis 1a), service leadership qualities (Hypothesis 1b), and life satisfaction (Hypothesis 1c) from the pretest to the posttest, and report positive evaluation on the course (Hypothesis 2).

Furthermore, as research is scarce regarding who gains more from the online SL, this study attempts to address one more question: How do students’ baseline qualities influence their growth in other intended learning qualities and well-being? We aim to test the following three hypotheses. First, following previous studies on service leadership courses (Lin & Shek, 2019; Zhu & Shek, 2021), we use cross-lagged panel modeling to test the linear relationship between initial competence and other learning outcomes. For the social compensation hypothesis, we expected that students’ PYD qualities at the pretest would negatively predict service leadership qualities and life satisfaction at the posttest and that students’ service leadership qualities at the pretest would negatively predict PYD qualities and life satisfaction at the posttest, controlling for temporal stabilities of related variables (Hypothesis 3a). For the social enhancement hypothesis, we expect that students’ PYD qualities at the pretest would positively predict service leadership qualities and life satisfaction at the posttest, and students’ service leadership qualities at the pretest would positively predict PYD qualities and life satisfaction at the posttest, controlling for temporal stabilities of related variables (Hypothesis 3b). Moreover, to test the possibility of coexistence of both hypotheses, we further examined whether a U-shape quadratic relationship would exist, such that a negative relationship would emerge when students’ initial qualities are at relatively low levels and a positive relationship would emerge when students’ initial qualities are at relatively high levels (Hypothesis 3c).

Finally, although little is known about who appreciates online SL more, we derived a hypothesis according to previous studies of youth social services in Hong Kong (e.g., Shek 2010; 2014). Specifically, we expected that students’ pretest-posttest changes in intended learning outcomes (i.e., PYD qualities and service leadership qualities) would be positively related to their subjective course evaluation (Hypothesis 4).

Methods

Participants and Procedure

Participants were students who attended the SL courses in Chengdu (N = 85) and Xi’an (N = 88) in the summer of 2021. A total of 173 students completed the same battery of questionnaires before the first workshop (i.e., pretest) and after the last workshop (i.e., posttest). By removing three pretest cases and four pretest cases with careless responses, the dataset included 170 students who completed pretest questionnaires (Chengdu: N = 83; Xi’an: N = 87), and 166 who completed posttest questionnaires (Chengdu: N = 82; Xi’an: N = 84). A total of 166 valid cases of pretest-posttest evaluation were matched with 82 cases from the Chengdu site (mean age = 20.27 ± 1.35 years; female: 63.9%) and 84 cases from the Xi’an site (mean age = 19.23 ± 1.13 years; female: 62.1%). There was no significant difference in gender ratio (χ2(1) = .316, p > .05) between the two projects, but students in the Chengdu project were older than those in the Xi’an project (t(159) = 5.284, p < .001). In addition, 169 students completed the post-course evaluation questionnaire (Chengdu: N = 82; Xi’an: N = 87). Lastly, 160 cases of pretest-posttest evaluation and post-course evaluation were successfully matched. Given that the sample size was subject to the course enrollment quota and students’ voluntary participation, we performed sensitivity power analyses to test statistical power. Upon the assumption of power (1-β) = 0.8 and α = 0.05, we found that the current sample size enabled us to detect the effect size Cohen’s d = .31, |ρ| = 0.22, ƒ2 = 0.06 in paired-sample t-test, correlations, and multiple regression, respectively.

Measures

Pretest–Posttest evaluation

The students responded to a questionnaire that assessed PYD qualities, service leadership qualities, and life satisfaction (for sample items, see Table S1 in the supplementary materials). First, PYD qualities were assessed using a brief version of the Chinese Positive Youth Development Scale developed by Shek et al., (2007). This scale contains 36 items that assess 10 dimensions of PYD qualities, including emotional competence, cognitive competence, social competence, moral competence, behavioral competence, self-determination, clear and positive identity, belief in the future, resilience, and spirituality on a 6-point Likert scale (1 = strongly disagree; 6 = strongly agree). All the subscales and the whole scale demonstrated satisfactory internal consistencies, except moral competence and spirituality (see Table 1). We computed a mean score for each subscale, as well as an overall mean score, to indicate the level of PYD qualities, with higher scores reflecting better qualities. Second, students’ service leadership qualities were measured using the Service Leadership Scale developed by Shek et al., (2014). This scale assesses four key components of service leadership qualities: self-leadership, character strength, caring disposition, and service leadership values. All the subscales and the whole scale yielded satisfactory internal consistencies (see Table 1). We computed a mean score for each dimension and an overall score to indicate the general level of service leadership qualities, with higher scores reflecting better service leadership qualities. Finally, the students reported their life satisfaction using the Chinese version of the Satisfaction with Life Scale (Diener et al., 1985) adapted by Shek (1992). A mean score was computed, with higher scores reflecting higher life satisfaction.

Table 1

Means, standard deviation, reliabilities, and mean differences of study variables in pretest–posttest evaluation

	Number of Items	Pretest		Posttest			Paired-Sample T-test (Xi’an)
		Mean (SD)	α	Mean (SD)	α		Mean Difference (Posttest-Pretest)	t	Effect size
Positive youth development qualities (overall level)		4.53(0.56)	0.94	4.87(0.58)	0.95		0.36	7.32a	0.80
Social competence	3	4.75(0.69)	0.90	5.15(0.63)	0.91		0.41	6.06a	0.66
Emotional competence	3	4.59(0.74)	0.82	4.95(0.68)	0.81		0.39	4.62a	0.50
Cognitive competence	4	4.67(0.63)	0.84	5.06(0.62)	0.86		0.37	6.40a	0.70
Behavioral competence	2	4.62(0.74)	0.79	5.00(0.67)	0.79		0.40	5.51a	0.61
Moral competence	4	4.64(0.64)	0.57	4.79(0.65)	0.45		0.09	1.35	0.15
	3#	4.82 (0.63)	0.74	5.13(0.67)	0.72		0.31	4.85a	0.53
Self-determination	3	4.49(0.71)	0.75	4.88(0.66)	0.79		0.41	5.21a	0.57
Clear and positive identity	2	4.07 (0.94)	0.83	4.56 (0.93)	0.83		0.51	5.07a	0.56
Belief in the future	3	4.72(0.68)	0.78	5.02(0.72)	0.81		0.33	4.32a	0.48
Spirituality	4	4.26(0.71)	0.46	4.49(0.79)	0.61		0.59	6.76a	0.41
Resilience	3	4.45(0.75)	0.83	4.76(0.85)	0.88		0.27	3.69a	0.50
Service leadership qualities (overall level)		4.66(0.56)	0.96	5.02(0.59)	0.97		0.38	7.32a	0.80
Self-leadership	5	4.53(0.65)	0.85	4.83(0.74)	0.91		0.32	4.17a	0.46
Caring disposition	8	4.76(0.64)	0.93	5.10(0.66)	0.94		0.35	5.73a	0.63
Character strengths	15	4.50(0.56)	0.91	4.92(0.59)	0.93		0.45	7.50a	0.83
Service leadership values	6	4.85(0.62)	0.92	5.21(0.64)	0.95		0.41	7.40a	0.81
Life satisfaction	5	3.99(0.83)	0.86	4.39(0.94)	0.89		0.40	3.43a	0.38

Note. The paired-sample t-test was based on Xi’an data collected in the summer of 2021; effect size was measured using Cohen’s d. ap < .003 (using Bonferroni correction). # One items was removed from the subscale of moral competence (“It is no big deal to give up some of our moral principles in exchange for money.”)

Post-course evaluation

The present study used a post-course questionnaire to measure the participants’ subjective evaluations of the course. Three aspects of the course were evaluated, including perceived course qualities (10 items; 1 = strongly disagree, 5 = strongly agree), perceived teacher performance (10 items; 1 = strongly disagree, 5 = strongly agree), and perceived course effectiveness (18 items; 1 = not helpful at all, 5 = very helpful). These three subscales had good internal consistencies (αs = 0.91, 0.96, and.98). A mean score was taken for each dimension, with higher scores representing a better evaluation.

Results

Students’ changes in Pretest–Posttest evaluation (pretest–Posttest changes)

We used a series of paired-sample t-test analyses to investigate whether students experienced any changes from the pretest to the posttest. We included the Xi’an data only to test the pretest-posttest changes, as findings from the Chengdu site were reported in another study (Shek et al., 2022). There were no significant differences between the two service sites in all the pretest variables and posttest variables except posttest emotional competence (t(164) = 2.341, p < .05). As 17 comparisons were tested, we adjusted the statistical inference using Bonferroni correction. Accordingly, we applied a p-value of 0.003 as the cutoff for statistical significance. As shown in Table 1, significant increases between the pretest and the posttest were found in PYD qualities, service leadership qualities, and life satisfaction. Students demonstrated significant positive changes in all the aspects of the PYD qualities except moral competence, with the largest change (in terms of effect size) in cognitive competence (Cohen’s d = 0.70), followed by social competence (Cohen’s d = 0.66), behavioral competence (Cohen’s d = 0.61), and spirituality (Cohen’s d = 0.41). As the reliability of the moral competence subscale was not good, we re-calculated the test by removing one reverse-keyed item. The pretest–posttest change became significant (Cohen’s d = 0.53). Thus, we should be cautious when interpreting results concerning moral competence. Similarly, they experienced significant positive changes in all the variables of service leadership qualities, including character strengths (Cohen’s d = 0.83), caring disposition (Cohen’s d = 0.63), self-leadership (Cohen’s d = 0.46), and service leadership values (Cohen’s d = 0.81). Finally, a significant positive change was observed in life satisfaction (Cohen’s d = 0.38). These changes were considered medium to large (Cohen, 1992).

Students’ responses to post-course evaluation

Most of the students who served children in Chengdu and Xi’an evaluated the course in a positive light (see Table S2 in the supplementary materials). Regarding course qualities, the greatest satisfaction came from class participation (87.6%), followed by the objectives of curriculum (85.9%), class atmosphere (84.1%), and receipt of encouragement (84.1%). Despite being ranked lowest among 10 items, more than two-thirds of students endorsed the good arrangement of activities (78.2%) and peer interaction (77.1%). Regarding teacher performance, the best evaluation rested in the lecturers’ involvement (92.4%), followed by lecturers’ readiness to offer help (91.2%), a good mastery of the curriculum (90.6%), and interaction with students (90.6%). Despite being rated the lowest, 87.1% of the students considered the lecturers’ teaching skills to be good. Lastly, regarding course effectiveness, most of the students perceived SL courses to be effective in enhancing their PYD qualities and service leadership qualities, with the frequencies of positive responses ranging from 90.6% (i.e., understanding of importance of service leadership, sense of responsibility, and theories, research, and concepts on service leadership) to 77.1% (i.e., improving meaning in life).

Predictors of Pretest–Posttest Changes in Learning Outcomes

We conducted cross-lagged modeling to examine how students’ baseline qualities assessed at the pretest were related to their changes in PYD qualities, service leadership qualities, and life satisfaction, controlling for temporal stabilities of the variables (i.e., cross-lagged effects). Both Chengdu and Xi’an data were used. Although not the major focus, we also tested whether the students’ change in one attribute was related to the change in another (i.e., correlated change). The model is depicted in Fig. 1. Specifically, we tested whether the model applied to students of both genders and students of two service sites using a two-group cross-lagged panel model. First, we estimated a model with free paths (Model 0). Second, we estimated a model with temporal stabilities and cross-lagged effects constrained to be equal between the two groups (Model 1). Finally, we estimated a model with additional constraints on the correlated changes (Model 2). The constraints on Models 1 and 2 did not worsen the model fit, as compared with Model 0 (gender-invariant models: Δχ2(9) = 10.88, Δχ2(12) = 11.79, ps > 0.05; service site-invariant models: Δχ2(9) = 16.32, Δχ2(12) = 17.77, ps > 0.05). Thus, we chose Model 2, which assumed invariance between males and females (CFI = 1.000; TLI = 1.001; RMSEA = 0.000) or that between the Chengdu site and the Xi’an site (CFI = 0.992; TLI = 0.971; RMSEA = 0.054) as the final model for interpretation of the results.

Fig. 1

Illustration of the cross-lagged model

Note. r indicates residual, and the correlations among residuals indicate correlated change

As presented in Table 2, the results indicated that baseline PYD qualities were related to increased service leadership and life satisfaction at the posttest, controlling for temporal stabilities in service leadership qualities, PYD qualities, and life satisfaction. However, baseline service leadership qualities were unrelated to PYD qualities and life satisfaction at the posttest. These findings indicated that students with a higher initial level of PYD qualities had better service leadership qualities and life satisfaction at the end of the SL project. The results partially support the social enhancement hypothesis. Additionally, we also found that the correlations among residuals were all positive and significant, controlling for temporal stabilities and cross-lagged effects. These results suggested that the change in one attribute of service leadership qualities, PYD qualities, and life satisfaction was related to the change in another. In other words, these changes went hand-in-hand during the online SL process.

Table 2

Coefficients of cross-lagged models

			Male	Female				Chengdu	Xi’an
	β	SE	b	b		β	SE	b	b
Temperal stabilities
SLQ	0.39***	0.11	0.36	0.38		0.41***	0.11	0.38	0.38
PYDQ	0.58***	0.12	0.62	0.53		0.58***	0.12	0.57	0.56
LS	0.21*	0.10	0.19	0.19		0.25*	0.09	0.23	0.22
Cross-lagged effects
Pretest_SLQ → Posttest_PYDQ	0.12	0.11	0.11	0.12		0.12	0.11	0.11	0.12
Pretest_ SLQ → Posttest_LS	− 0.06	0.21	− 0.03	− 0.04		0.03	0.20	0.02	0.02
Pretest_PYDQ → Posttest_ SLQ	0.33*	0.12	0.36	0.29		0.32**	0.12	0.31	0.29
Pretest_PYDQ → Posttest_LS	0.61**	0.23	0.38	0.35		0.47*	0.21	0.32	0.26
Pretest_LS → Posttest_ SLQ	0.03	0.05	0.05	0.04		0.03	0.05	0.04	0.04
Pretest_ LS → Posttest_PYDQ	0.01	0.05	0.01	0.01		− 0.01	0.05	− 0.01	− 0.01
Correlated Changes
SLQ ~ PYDQ	0.31***	0.06	0.84	0.83		0.15***	0.02	0.80	0.82
SLQ ~ LS	0.22***	0.07	0.43	0.41		0.18***	0.03	0.53	0.46
PYDQ ~ LS	0.34***	0.09	0.55	0.56		0.23***	0.03	0.67	0.60

Note. SLQ = service leadership qualities; PYDQ = positive youth development qualities; LS = life satisfaction. The coefficients were derived from the final models that constrained the two gender groups or the two service sites to be equal.

To further test the curvilinear relationship between pretest qualities and posttest outcomes, we adopted a novel statistical approach—two-line test (Simonsohn, 2018). This test estimates two regression lines of a predictor simultaneously—one for high level and one for low level—and it provides the estimation of the exact break point based on the Robin Hood algorithm. In other words, it provides evidence of varied relationships between pretest qualities and posttest outcomes at different levels of initial qualities. Simulations and empirical analyses suggest that it outperforms traditional quadratic-regression-based tests to fit a quadratic relationship (Dorfman et al., 2019; Simonsohn 2018).

We conducted the two-line test via the online APP (http://webstimate.org/twolines) developed by Simonsohn (2018). First, we regressed pretest PYD qualities on posttest service leadership qualities by controlling for the pretest service leadership qualities. The Robin Hood algorithm placed the break points at a medium-to-high level out of a 6-point Likert scale. As shown in Fig. 2, there was a positive and significant relationship between initial PYD qualities and posttest service leadership qualities (b = 0.37, z = 3.13, p < .01) before the break point (4.94), whereas there was a negative yet non-significant relationship after the break point (b = − 0.28, z = − 0.089, p = .37). Second, we regressed pretest PYD qualities on posttest life satisfaction by controlling for pretest life satisfaction. Similarly, the results revealed a positive and significant relationship (b = 0.47, z = 2.31, p < .05) before the break point (4.95), whereas there was a negative yet non-significant relationship after the break point (b = − 0.62, z = -1.26, p = .21). The findings are more aligned with the social enhancement hypothesis than the social compensation hypothesis, although the “rich-get-richer” effect appeared more salient at the medium levels of initial PYD qualities.

Fig. 2

Scatter plots of the relationship between pretest qualities and posttest outcomes

Note. The scatter plots were generated by the two-line test APP (Simonsohn, 2018); PYDQ = positive youth development qualities; SLQ = service leadership qualities. # The outcome variable assessed at the pretest was controlled in the two-line tests

Third, we regressed pretest service leadership qualities on posttest PYD qualities by controlling for pretest PYD qualities. The results showed a negative and significant relationship between initial service leadership qualities and posttest PYD qualities (b = − 0.46, z = -2.12, p < .05) before the break point (4.40) but a positive and significant relationship after the break point (b = 0.33, z = 2.16, p < .05). This finding showed a U-shaped relationship between initial service leadership qualities and outcome of PYD qualities, providing support for the coexistence of social compensation and social enhancement effects. In other words, students with relatively low and high initial levels of service leadership qualities probably gained the most in PYD qualities from the online SL. Fourth, we regressed pretest service leadership qualities on posttest life satisfaction by controlling for pretest life satisfaction. The result indicated a positive but non-significant relationship (b = 0.23, z = 0.61, p = .54) before the break point (4.35) and a positive and significant relationship after the break point (b = 0.79, z = 3.5, p < .001). It generally supports the social enhancement hypothesis.

Factors related to course evaluation

Finally, we were interested in how the pretest–posttest changes in the intended learning outcomes (i.e., PYD qualities and service leadership qualities) related to students’ subjective course evaluation. Suggested by previous studies (e.g., Rowan et al., 2017; Shek, 2014), we used residual change scores to represent the change of the variable from the pretest to the posttest. Residual change scores are computed by the discrepancies between the observed posttest scores and the predicted posttest scores, which are estimated by regressing the posttest score on the pretest score (Kisbu-Sakarya et al., 2013). First, we conducted zero-order correlations among the study variables. The results (see Table 3a) showed that the change of PYD qualities and change of service leadership qualities were positively associated with perceived course qualities, perceived teacher performance, and perceived course effectiveness. These results suggest that positive changes in intended learning outcomes possibly contribute to students’ stronger positive evaluation of the course.

Table 3

b. Multiple regression results

Variables	Course qualities				Teacher performance				Course effectiveness
	β	SE	b		β	SE	b		β	SE	b
Step 1
Age	− 0.027	0.043	− 0.061		0.010	0.039	0.024		− 0.060	0.044	− 0.131
Gender	− 0.017	0.112	− 0.013		0.057	0.101	0.049		0.015	0.114	0.011
Service site	0.001	0.118	0.001		− 0.123	0.106	− 0.110		0.004	0.119	0.003
Step 2	ΔR2 = 0.128, p < .001				ΔR2 = 0.107, p < .001				ΔR2 = 0.171, p < .001
RCS of PYDQ	0.103	0.077	0.166		− 0.019	0.070	− 0.033		0.010	0.076	0.016
RCS of SLQ	0.133	0.076	0.219		0.195**	0.069	0.355		0.250**	0.075	0.403

Note. PYDQ = positive youth development qualities; SLQ = service leadership qualities; RCS = residual change score; **p < .01

Furthermore, to explore the relative contribution of PYD qualities and service leadership qualities, we performed three multiple regression analyses by simultaneously including both residual change scores. We controlled for age, gender, and location of service targets in the first step and entered the residual change scores of PYD qualities and service leadership qualities in the second step to predict perceived course qualities, perceived teacher performance, and perceived course effectiveness, respectively. As shown in Table 3b, regarding perceived course qualities, the change of PYD qualities and change of SL qualities did not significantly predict students’ evaluation of the course when they were investigated simultaneously, although the inclusion of these two predictors significantly increased the R2. Regarding perceived teacher performance and perceived course effectiveness, the change in SL qualities (but not the change in PYD qualities) significantly predicted better evaluation. On the one hand, considering the large correlation between these two residual change scores, regression analyses possibly partialled out their shared variance—overall growth in qualities, which possibly contributed to their evaluation of the course. On the other hand, the findings suggested that the enhancement in service leadership qualities appeared to contribute more than that in PYD qualities to the positive appraisal of teacher performance and course effectiveness.

Discussion

Although the online mode has been suggested as an alternative approach to practice SL for more than a decade, evidence about its effectiveness and factors contributing to its outcomes is still scarce. The literature on online SL is mainly anecdotal and mostly relies on case studies (Salam et al., 2019; Waldner, 2012). To fill the gaps, this study systematically examined online SL’s effectiveness from participating students’ perspectives based on an online SL course offered amid the COVID-19 pandemic. The findings of this study not only provide evidence showing the positive outcomes of online SL projects but also further demonstrate what may contribute to these outcomes.

First, the study demonstrated positive changes in PYD qualities, service leadership, and life satisfaction among the students who served Xi’an, as well as substantial favorable evaluation regarding course qualities, teacher performance, and course effectiveness among students who served children in Chengdu and Xi’an (see also Shek et al., 2022). The literature suggests that adequate teacher support, adjusting teaching content to an online setting, effective delivery of teaching information, and well-prepared contingency plans are possible factors that may contribute to the success of online learning (Bao, 2020). Our online service-learning projects included these factors. Additionally, these projects provided students with multiple opportunities for reflection, which was deemed an effective component of service learning (Celio et al., 2011). Together with studies of online SL projects in our previous offerings (Leung et al., 2021; Lin & Shek, 2021) and current offerings in the Chengdu site (Shek et al., 2022), as well as studies of other researchers’ projects (e.g., Burton & Winter, 2021; Harris, 2017), these results indicate the effectiveness of practicing SL in a virtual manner. Despite the need for student training in digital technology and online communication, the online approach to learning and service can connect students with teachers and service recipients who are located in different places. Even when the pandemic ends, it could be considered an additional educational approach to promote students’ leadership and well-being in higher education. It also outweighs other asynchronous indirect service modes, such as making masks or videos (Tian & Noel Jr., 2020), because it enables instant interaction and spontaneous communication, which possibly strengthens the relationship among people involved. Admittedly, the success of the online projects relies on the strong support of digital devices and Internet connections, which are not available in some disadvantaged communities. Educators and program implementers still need to consider the circumstances of service recipients in the design and implementation of online SL. Moreover, the finding regarding students’ positive change in life satisfaction also echoes previous studies suggesting that leadership programs (Lin & Shek, 2019) and SL programs (Leung et al., 2021) may contribute to students’ well-being. Admittedly, a control group was needed to exclude the possibility that students’ well-being might naturally improve when their study was finally accomplished. However, the current finding suggests that researchers and practitioners may expand the scope of SL by considering it as a vehicle to promote the well-being of young people in the future and further examine what factors contribute to the increase of well-being. Additionally, for a more comprehensive understanding, practitioners are advised to go beyond life satisfaction by including other well-established well-being indicators, such as meaning in life (Zhu et al., 2022).

Second, the current study yields more evidence supporting the social enhancement hypothesis compared to the social compensation hypothesis, which is consistent with the studies on social media use (Cheng et al., 2019). We found that students with better initial PYD qualities tended to demonstrate better service leadership qualities and life satisfaction in the posttest. Additionally, students with better initial service leadership qualities tended to demonstrate better PYD qualities and life satisfaction at the posttest when their initial service leadership qualities were relatively high. These results echo previous studies on non-SL courses about service leadership (Lin & Shek, 2019; Zhu & Shek, 2021). PYD qualities denote optimal functioning in social, emotional, cognitive, behavioral, and moral domains (Shek & Sun, 2009). These competencies probably serve as psychological resources that facilitate students’ acquisition of new knowledge and skills about service leadership, and enable them to overcome the difficulties better and derive pleasure and contentment from online learning and service. Similarly, strong service leadership qualities that include ability of self-leadership, caring disposition, moral character, and service values probably help students improve their PYD qualities and increase overall satisfaction with life. On the other hand, advanced web-conferencing tools with strong audio and video functions have made online communication akin to face-to-face communication. However, it might constrain the improvement of students with relatively lower competence. Therefore, it reminds teachers or practitioners that extra support might be needed for these students.

Meanwhile, we found support for the coexistence of social compensation and social enhancement effects in the relationship between initial service leadership qualities and posttest PYD qualities. This observation echoes a prior proposal of the coexistence of these two effects in social media use (Pouwels et al., 2021; Wang et al., 2018). It is possible that online SL provides students who are relatively left behind in the initial service leadership qualities with opportunities for meaningful social interactions with teachers, classmates, and particularly service recipients. The support, encouragement, and positive reinforcement experienced through such social interactions, which might be rarely received from their previous lives, may help them more in building PYD qualities. This finding suggests that researchers should fine-tune the theory of service learning by considering both social compensation and social enhancement. Moreover, educators and practitioners will benefit from considering individual differences in online learning and exploring how to improve learning outcomes among students with differential competence levels. However, the current findings are tentative because they are based on a single study. In addition, it is noted that our sample was dominated by students with medium-to-high levels of qualities (see Fig. 2). The lack of observed cases at low levels of quality hinders us from inferring whether social compensation occurs among these “incompetent” students. Therefore, future studies would benefit from including both at-risk students and typically developing students in the investigation.

Finally, we found that the change in PYD qualities and the change in service leadership qualities were related to students’ appraisal of course effectiveness when they were examined individually. The growth in intended learning outcomes probably makes students feel more satisfied with the online SL course. Regarding the debate on the validity of subjective evaluation on program benefits (see Shek, 2010; 2014), this study also provides evidence supporting that subjective evaluation is congruent with actual pretest-posttest change and can serve as a valid indicator of course effectiveness. Multiple regression analyses that examined both changes further showed that the change in service leadership qualities (vs. the change in PYD qualities) had a unique effect on perceived teacher performance and course effectiveness, suggesting an additional contribution of the change in service leadership qualities. Although the change in PYD qualities did not show a unique contribution due to the large shared variance between the changes in these two kinds of qualities, we could not exclude the possibility that the overall personal growth from the pretest to the posttest contributes to a better appraisal.

Altogether, this study provides an additional perspective for understanding individual differences in SL outcomes. Previous studies that investigated the variance in the outcomes of SL often focused on program characteristics, such as whether it is mandatory or voluntary (Yorio & Ye, 2012) and how many recommended practices (e.g., reflection) are used (Celio et al., 2011). Our study further suggests that students’ prior characteristics and learning experiences also influence SL outcomes. We look forward to mediation studies that explain how such characteristics may translate into positive learning outcomes of SL. In view of the need to conduct more research studies on children and adolescents in the context of mainland China (Shek et al., 2021), this study is an important addition to the literature on online SL in mainland China.

Limitations and future directions

Several cautions are warranted in this study. First, we could not infer a causal relationship between SL projects and student outcomes, as this study did not involve a comparable control group. Moreover, it is unknown how long the positive changes will persist, as only two occasions (i.e., pretest and posttest) were assessed. Thus, future studies would benefit from a quasi-experimental design with a control group and follow-up assessments. With additional time points, future studies could explore how the initial level of competences influences the growth rate of students’ learning outcomes by using growth curve modeling. Second, all the assessments relied on students’ self-reports, and thus, the relationships between variables might be inflated due to common measure variance. Harman’s single-factor test of all the measures revealed one factor that accounted for 34.31% of the variance, indicating that common measure bias in this study was not severe (see Tehseen et al., 2017). Moreover, due to the correlational design and self-report, the current study could not infer whether actual changes in student outcomes led to positive course evaluations or vice versa. It is possible that when positive feel good about the course, they would report higher levels of learning outcomes and well-being at the posttest, which makes the pretest–posttest change seemingly larger. To provide a more comprehensive understanding of student outcomes, other informants such as teachers, classmates, and service partners are needed in future studies. Finally, this study focused on students’ intended learning outcomes and well-being, while leaving a question unexplored, namely, whether online SL will improve their knowledge and skills relevant to virtual settings such as media literacy and skills. It is worth studying the unique benefits of online SL in the future.

Conclusions

Despite the aforementioned limitations, this study suggests that online SL could serve as an effective educational approach to nurturing students’ leadership and well-being. Although the benefits of online SL depend on the design of the program and curriculum as well as the involvement of the teachers, it enlightens educators and other related practitioners to reimagine and innovate pedagogy that helps promote holistic youth development. More importantly, it provides initial evidence showing that initial student qualities are related to the gains from online SL and how the changes in these qualities relate to students’ satisfaction with the projects. These findings offer insights for educators and practitioners to better refine their SL practices to benefit the well-being of a broader range of students, particularly in a Chinese context experiencing the COVID-19 pandemic.

Below is the link to the electronic supplementary material.

Supplementary Material 1

Table 3

a. Correlations between pretest–posttest change in PYDQ and SLQ (residual change scores) and subjective evaluation

Variables	1.	2.	3.	4.	5.	6.	7.	8.
1. RCS of PYDQ	1
2. RCS of SLQ	0.766***	1
3. Perceived course qualities	0.349***	0.358***	1
4. Perceived teacher performance	0.296***	0.355***	0.745***	1
5. Perceived course effectiveness	0.380***	0.437***	0.826***	0.742***	1
6. Age	− 0.093	− 0.105	− 0.068	− 0.026	− 0.122	1
7. Gender	0.061	− 0.014	0.040	0.081	0.059	0.117	1
8. Service site	− 0.058	− 0.091	− 0.041	− 0.051	− 0.021	0.387***	− 0.044	1

Note. PYDQ = positive youth development qualities; SLQ = service leadership qualities; RCS = residual change score; ***p < .001