Development and factor analysis of an instrument to measure service-learning management.

Service-learning activities are rapidly expanding worldwide. Several instruments are available for measuring these activities. However, none of these are suitable for measuring management issues and challenges in service-learning implementation. This study reports on the development and factor analysis of an instrument to measure service-learning management. In total, 315 teachers from vocational training institutes of Punjab, Pakistan participated in the study. The three stage scale approach of Fraser (1998) was used for construction of the scale. Items were selected based on deductive method using the existing theory. Exploratory Factor Analysis (EFA) was used to explore the factor structure of the scale. A four factor structure having 21 items was indentified. The factorial validation based on initial assessment of the scale provides initial evidences for reliability and validity of the scale. The newly developed instrument may be used for measuring challenges and issues in service-learning management. The limitations of the study are discussed and some recommendations are made for future practice.

Introduction

Service-learning activities are getting rapid popularity worldwide. As a community based learning approach it offers numerous promises for students and communities. It has proved its worth as a civic learning pedagogy. It provides wider opportunities for social, intellectual and moral development of students in a community based environment where they interact with people in a real life context on hand and prepare for the course work on the other. Thus it creates a blending of classroom objectives with life experiences in the society. However, researchers have hinted at several issues and challenges by students, teachers and communities during the implementation of the service-learning activities. Less attention has been paid towards this problem in the current scenario. There is no valid instrument for measuring the service-learning management activities. This situation warrants for a valid and reliable scale for measuring issues and challenges to service-learning management. Developing such a valid and reliable scale would give promising results to researchers, administrators and practitioners. The existing service-learning instruments do not effectively meet the requirements of current service-learning implementation. The present study aimed to carry out an initial assessment and validation of a scale for measuring service-learning management activities.

Literature review

Service-learning is an experiential strategy that connects community service with academic goals enriching students’ learning on one hand and strengthening communities on the other ( Bringle and Hatcher, 1996; Jacoby, 2014). Lu and Lambright (2010) developed a scale to examine the views of public administration students about the factors affecting their service-learning outcomes. In another study, Hase and Goldberg (1967) explored the perspectives of students and teachers about service-learning practice. Hair et al. (2011) explored factors influencing service-learning in economic education from the perspectives of community and students. Most of these studies are reported from developed world context. Only one study was reported from developing world context in which (Delve et al., 1990) studied challenges to service-learning implementation in ESL classroom in Oman. Recently, Blakey et al. (2016) studied the general issues in service-learning from the perspectives of faculty members. It also did not explore factors influencing service-learning implementation.

Despite different theoretical and operational challenges for service-learning research, many scales have been used to document the outcomes of students' service-learning participation. For example, Eyler and Giles (1999) have used a mixed method study to examine students' service-learning participation. The results of the study showed an improvement in different areas of student outcomes such as civic responsibility, critical thinking and interpersonal skills. However, all these quantitative assessments were limited to single item survey responses with few multiple items. They did not capture the complete picture of the situation. There is a need for further research to provide firm foundations to these previous findings. Other writers argue that rather than limiting the construct to a single behaviour and attitude, a broader explanation may help in better understanding of students’ service-learning outcomes.

The existing research has provided some foundations for quantitative assessment of service-learning outcomes in different education settings (Rue, 1996). These studies also provided evidences for students' outcomes in terms of increased communication skills, knowledge, belief in one's abilities to do service for public good and commitment to community work. Furthermore, these existing studies have used single item and non-validated measures for assessing service-learning outcomes Given the methodological issues involved in the measurement of service-learning management, the directional changes over time have not been noticed about the factors affecting management of projects both within and outside of the service-learning experiences.

Majority of the research in service-learning has been limited to assessing service-learning outcomes and courses of academia and less work has focused on management issues involved in program design and implementation of such projects For example, in one study, views of social work educators were measured on the factors influencing service-learning utilization in social work using an online survey which also lacks clear validity and theoretical justification (Gul et al., 2021c). In another study, Lu and Lambright (2010) assessed the views of public administration students about the factors affecting their learning outcomes. The results showed that instructional guidance, integration of material, reflection, student influence and their contact with service beneficiaries were some of the influencing factors. However, the author has suggested for further research due to the dearth of literature in this area. Researchers have explored the perspectives of students and teachers about service-learning practice and its outcomes, but this there is a lack of clear theoretical and strong empirical support to prove the findings. The study was conducted using an interview protocol that focuses how intentions and outcomes are transformed into service-learning experience. This study lacked other means such as exploratory and confirmatory factor analysis to verify the interview data. Another study explored factors influencing service-learning in economics education from the perspectives of community and students by using a service-learning inventory (Ayub et al., 2021).

However, this inventory too did not clearly indicate the different influencing aspects of service-learning which were clearly affected in economic education as a result of participating in service-learning. The measure only inquired into the views of students and community members. It did not cover the perspectives of teachers. Furthermore, majority of the existing service-learning measures have less efficacy for effectively covering the issues of management rather cover alternative instructional delivery (Bringle and Clayton, 2012, Gul and Khilji, 2021, Butin, 2006, DeVellis, 1991). This study contributes to this gap in the current literature by measuring perspectives of teachers for assessing the challenges and issues in service-learning management. The results of this study contribute to the development of new knowledge base for effective implementation of service-learning activities.

Method

A quantitative descriptive research method based on survey approach was used to collect data. A total of 315 teachers from vocational training institutes of Punjab province of Pakistan participated in the study. Permission was granted from Departmental Research Ethical Committee and the willingness and consent of the participants was obtained before sending questionnaires for data collection. The selected sample consisted of those teachers who were part of service-learning projects in these institutes. The three-stage scale approach of Fraser (1998) was used for construction of the scale. A deductive approach was used for the development of the scale consisting of the following stages:

Stage 1- literature review

A survey of literature was carried out on managing service-learning activities. The literature review consisted of service-learning books, published research papers, magazines and other related materials. Although, several key components of service-learning activities were focused during the survey of literature. However, more specifically, the focused areas for the review included issues and challenges in the implementation of service-learning activities. A number of key statements related to different key issues were identified. Based on the statements four major themes were extracted such as planning and collaboration, evaluation and need analysis, training and orientation and student placement.

Stage 2- item generation

A total of 25 items were generated based on the review of literature. Each theme is represented by different items in the scale.

Table 1 shows that the important and prominent statements were grouped together based on the themes that were identified during the review process. These themes were checked for repetitions. After preparing the initial draft having all the selected items, the items were checked and approved by a panel of experts. The items were further narrowed down and simplified. The double barrel items were removed and finally 21 items were finalized.

Table 1

Subscale, items and statements.

Subscale	# of Items	Statements
Planning and Collaboration	06	1. Management prepares the service-learning plans2. Faculty is involved in the service-learning planning3. Students contribute to the service-learning planning4. Community participates in service-learning planning5. Community facilitates the service-learning activity6. Students and community work closely
Evaluation and need analysis	07	1. Variety of tools are used for assessment2. Assessment is used to improve the courses3. Assessment is done through paper pencil test4. Community is involved in the assessment process5. Community is involved in need analysis6. Need assessment is done only by management7. Students are assigned location based on their need
Student Placement	04	1. Student placed is decided without their consent2. Students decide service site placement on their own3. Placement is on mutual choice of all partners4. Placement is an essential component of service-learning
Training and Orientation	04	1. Students are trained for service activity2. Community is given orientation on service-learning3. Faculty gets regular training on service-learning4. Training is useful for service-learning implementation

Stage 3- pilot testing

For reliability assessment, the newly prepared instrument was pilot tested for collecting sufficient response to carry out the statistical analysis.

Table 2 shows that the Cronbach's alpha was used to determine the internal consistency of the scale items. The Cronbach's alpha for the service-learning management scale was.96 being at an acceptable range.

Table 2

Reliability statistics.

S. No	Subscale	No of Items	Cronbach's Alpha Based on Standardized Items
1	Planning and Collaboration	06	.683
2	Evaluation and Need Analysis	07	.780
3	Student Placement	04	.703
4	Training and Orientation	04	.783
Overall Reliability		21	.965

Table 3 indicates the mean score and standard deviation for all items in the scale. The mean and standard deviation for each of the variables included in the scale were calculated. All items in the scale have mean score ranging from 2.13 to 2.84 and standard deviation is within the acceptable range.

Table 3

Statements, mean and standard deviation (SD).

S.#	Variable	Min	Max	Mean	SD
1	Management prepares the service-learning plans	1	5	2.35	.972
2	Faculty is involved in the service-learning planning	1	5	2.55	1.058
3	Students contribute to the service-learning planning	1	5	2.38	.971
4	Community participates in service-learning planning	1	5	2.99	1.228
5	Community facilitates the service-learning activity	1	5	2.30	.941
6	Students and community work closely	1	5	3.69	.927
7	Variety of tools are used for assessment	1	5	3.78	.972
8	Assessment is used to improve the courses	1	5	2.40	1.121
9	Assessment is done through paper pencil test	1	5	2.55	1.079
10	Community is involved in the assessment process	1	5	3.33	1.052
11	Community is involved in need analysis	1	5	2.40	.978
12	Need assessment is done only by management	1	5	2.44	.966
13	Students are assigned location based on their need	1	5	2.33	.983
14	Student placed is decided without their consent	1	5	2.13	.864
15	Students decide service site placement on their own	1	5	3.77	.995
16	Placement is on mutual choice of all partners	1	5	2.74	1.040
17	Placement is an essential component of service-learning	1	5	2.43	1.009
18	Students are trained for service activity	1	5	2.45	.937
19	Community is given orientation on service-learning	1	5	2.30	.892
20	Faculty gets regular training on service-learning	1	5	2.72	1.136
21	Training is useful for service-learning implementation	1	5	2.84	1.129

Exploratory factor analysis

As a next step, the data and sample size adequacy for conducting EFA was checked. This was achieved by measuring the Kaier-Meyer-Olkin (KMO) of sample adequacy and Bartlett's test for sphericity.

Table 4 indicates that the KMO measure of sample adequacy was .910 and Bartlett's sphercitiy was significant at .000. This allowed factor analysis to measure the service-learning management construct. To investigate about the appropriateness of the data, Bartlett test of sphericity was used.

Table 4

KMO and Bartlett's test of Sphericity

Kaiser-Meyer-Olkin Measure of Sampling Adequacy.		.910
Bartlett's Test of Sphericity	Approx. Chi-Square	1735.612
	Df	210
	Sig.	.000

Figure 1 indicates that for analyzing the data on management of service-learning activities, principal component analysis was applied as a method using varimax rotation method and the scree plot.

Figure 1

Scree plot.

Table 5 shows that four components were extracted from the 21 items scale and component loadings. The scree plot also suggested a four factor solution as indicated in Figure 1. The total variance explained is 65.395 % for four factor solution based on eigenvalue more than 1. The factor analysis revealed that the first factor included 6 items and it explained 42.971 % of the total variance in the four factor structure. Based on the nature of the items, the component was named planning and collaboration. The second factor included 7 items and it explained 11.828 % of the total variance. This factor was named evaluation and need analysis. The third factor included 4 items and it explained 5.730 % of the total variance. This component was named student placement. The last factor included 4 items and it explained 4.865 % of the total variance. This factor was named training and orientation.

Table 5

Total variance explained.

Comp.	Initial Eigenvalues			Extraction sums of squared loadings		Rotation sums of squared loadings
	Total	% of Variance	Cumulative %	Total	% of Variance	Cumulative %	Total	% of Variance	Cumulative %
1	9.024	42.971	42.971	9.024	42.971	42.971	5.161	24.578	24.578
2	2.484	11.828	54.799	2.484	11.828	54.799	4.720	22.476	47.054
3	1.203	5.730	60.530	1.203	5.730	60.530	2.558	12.179	59.233
4	1.022	4.865	65.395	1.022	4.865	65.395	1.294	6.162	65.395
5	.923	4.396	69.791
6	.842	4.008	73.799
7	.661	3.150	76.949
8	.570	2.716	79.665
9	.508	2.420	82.085
10	.473	2.254	84.339
11	.458	2.180	86.519
12	.438	2.087	88.606
13	.406	1.933	90.539
14	.351	1.673	92.212
15	.317	1.511	93.722
16	.267	1.270	94.992
17	.261	1.244	96.236
18	.235	1.121	97.356
19	.215	1.023	98.379
20	.180	.859	99.238
21	.160	.762	100.000

Table 6 indicates that the assessment of the four factor structure indicated that all items strongly loaded on the four components being above 0.5 and were at acceptable range.

Table 6

Rotated component matrix.

	Statements	Component
		1	2	3	4
1	Management prepares the service-learning plans	.744
2	Faculty is involved in the service-learning planning	.801
3	Students contribute to the service-learning planning	.599
4	Community participates in service-learning planning	.744
5	Community facilitates the service-learning activity	.761
6	Students and community work closely	.785
7	Variety of tools are used for assessment		.685
8	Assessment is used to improve the courses		.773
9	Assessment is done through paper pencil test		.765
10	Community is involved in the assessment process		.749
11	Community is involved in need analysis		.694
12	Need assessment is done only by management		.737
13	Students are assigned location based on their need		.626
14	Student placed is decided without their consent			.557
15	Students decide service site placement on their own			-.686
16	Placement is on mutual choice of all partners			.580
17	Placement is an essential component of service-learning			.518
18	Students are trained for service activity				.872
19	Community is given orientation on service-learning				.632
20	Faculty gets regular training on service-learning				.526
21	Training is useful for service-learning implementation				.610

Extraction Method: Principal Component Analysis. Rotation Method: Varimax with Kaiser Normalization.

a. Rotation converged in 7 iterations (Bernaards & Jennrich, 2008).

Table 7 shows that the item means and standard deviation indicated for each item in the scale have an acceptable rage based on descriptive statistical analysis.

Table 7

Item means and standard deviation, skewness and kurtosis.

S.#	Statements	Mean	SD	Skewness		Kurtosis
				Statistic	Std. Error	Statistic	Std. Error
1	Management prepares the service-learning plans	2.35	.972	.797	.204	-.353	.406
2	Faculty is involved in the service-learning planning	2.55	1.058	.610	.204	-.773	.406
3	Students contribute to the service-learning planning	2.38	.971	.756	.205	-.220	.407
4	Community participates in service-learning planning	2.99	1.228	.084	.204	-1.229	.406
5	Community facilitates the service-learning activity	2.30	.941	.972	.204	.500	.406
6	Students and community work closely	3.69	.927	-.920	.204	.533	.406
7	Variety of tools are used for assessment	3.78	.972	-1.013	.204	.945	.406
8	Assessment is used to improve the courses	2.40	1.121	.721	.204	-.461	.406
9	Assessment is done through paper pencil test	2.55	1.079	.484	.204	-.706	.406
10	Community is involved in the assessment process	3.33	1.052	-.203	.204	-.705	.406
11	Community is involved in need analysis	2.40	.978	.923	.204	.136	.406
12	Need assessment is done only by management	2.44	.966	.751	.204	-.359	.406
13	Students are assigned location based on their need	2.33	.983	.705	.204	-.418	.406
14	Student placed is decided without their consent	2.13	.864	1.017	.204	.938	.406
15	Students decide service site placement on their own	3.77	.995	-.983	.204	.527	.406
16	Placement is on mutual choice of all partners	2.74	1.040	.275	.204	-.853	.406
17	Placement is an essential component of service-learning	2.43	1.009	.736	.204	-.198	.406
18	Students are trained for service activity	2.45	.937	.580	.204	-.270	.406
19	Community is given orientation on service-learning	2.30	.892	.843	.204	.474	.406
20	Faculty gets regular training on service-learning	2.72	1.136	.371	.204	-1.091	.406
21	Training is useful for service-learning implementation	2.84	1.129	.252	.204	-1.218	.406

After factor analysis and descriptive statistical analysis of all the variables in the instrument, it was important to examine the correlation matrix of all the items constituting the four constructs or subscales. A strong correlation was found among all the subscales in the instrument as shown in Table 8.

Table 8

Correlations among subscales.

Subscales	Placola	EvaNa	StuPla	TraOre
Placola	1.000
EvaNa	.549∗∗	1.000
StuPla	.476∗∗	.586∗∗	1.000
TraOre	716∗∗	458∗∗	.499∗∗	1.000

(Placola = planning and collaboration, EvaNa = evaluation and need analysis, StuPla = student placement, TraOre = training and orientation).

Correlation is significant at the 0.01 level (2-tailed).

Discussion

Due to the worldwide popularity of service-learning, it was essential for developing and validating a scale to measure the numerous issues involved in its implementation. Previous research studies have investigated the service-learning outcomes from the perspectives of students. However, this study examined the perspectives of teachers about the management issues in service-learning implementation. Based on the data, a four factor structure was identified using varimax rotation. This study was an attempt to develop and validate a scale for measuring issues in service-learning management. The construct was developed based on literature review related managing the process of service-learning implementation in Pakistan. Previous studies have focused on service-learning outcomes (Ahmad and Gul, 2021: Butin, 2010; Gul et al., 2021; Jacoby, 2014). and little studies have considered the challenges involved in management of the implementation of service-learning activities. The construct was mainly developed keeping in view the scarcity of valid scales for measuring service-learning management issues and careful examination of important service-learning management concepts on the subject (Blakey et al., 2016; Bukhari et al., 2021; Gul et al., 2021).

The scale incorporates many key elements of service-learning management such as planning, evaluation, cooperation and reflection. To fully discern the psychometric properties of the scale, different analysis were conducted as part of reliability and validity determination such as descriptive statistics and correlations among the items and sub-dimensions of the scale. The current study used exploratory factor analysis to identify the underlying factor structure based on the 21 items. The present study is unique in two ways. First, currently, there is dearth of valid and strong instruments in the field of service-learning which could be used for measuring issues and problems in service-learning management. Secondly, little studies are available from vocational education field especially from the context of a developing country. Although, some studies have used scales for measuring (Gelmon et al., 2018), however, the measuring tools used by these studies have not sufficiently addressed the issues related to service-learning management. Therefore, this current study provides strong theoratical and empricial evidence based on its results to the solution of service-learning management issues. For strengthening the theoratical soundness of the construct, correlation analysis was carried out to assess the degree of relationship among the sub-dimensions of the scale (Bukhari et al., 2021). No viable studies were found in the context of the current study for developing the items in the scale, hence, mostly studies wer consulted which have been conducted in Asian and European contexts. Therefore, this may affect the efficacy of the scale in addressing the needs of service-learning management in those contexts. Therefore, further investigations may help in strengthening the findings of the current study in other contexts.

Literature suggests that service-learning is a challenging activity (Battistoni, 2017). Its implementation has different dimensions and phases which may have different impacts in different settings (Gul, R., Khilji, G. 2021). Hence it is important to find out the typical cultural issues involved in managing its implementation. Therefore, this current scale may provide a partial outlook towards the management of service-learning. The reliability analysis of the scale and its sub-scales showed that the various measures have good internal consistency and revealed in the form of coefficient alpha and mean inter-item correlation coefficient.This shows that the items within the different factor dimension are measuring the same factors which were important to establish the validity of the construct.

The current study is a pioneering effort and the results of the study are robust. However, there are several limitations. The sample of the study was less and mostly the participants came from the same geographical region and institutions of Punjab province of Pakistan. The method of sampling was not random in nature. As this study was the first of its nature in Pakistani vocational educational context for developing a measure in service-learning management, so more validation work needs to be done for replication of the study in other disciplines such as engineering, medical, teacher training for testing the factor structure of the scale and its universal application as a valid and relaible measure of service-learning management. The reliability of the scale was checked on the basis of assessing internal consistency or alpha, however, other types of tests such as test-retest reliability may also be used in future research studies for further verifying the stability of the construct. Despite the limitations, the present study broadly contributes towards the development of valid and robust measures in service-learning management in Pakistan.

Conclusion

The study aimed to develop and validate an instrument for measuring service-learning management. The application of factor analysis showed a four factor structure for the service-learning management scale. The validation analysis provided an evidence for the scale to be a reliable and valid for measuring challenges and issues in service-learning management. The initial assessment of the measure showed the instrument having above 70% reliability. The overall correlations among the all four sub-scales were above 0.70 which was at acceptable range (Hinkin, 1995). The overall instrument reliability was .965 which was an acceptable range for internal consistency. The newly constructed scale may be used in different cultural and social context to measure service-learning management issues and challenges in educational institutions. Further research is recommended to compare the findings of this study in other contexts. The results of the current study were based on data collected from vocational education of Pakistan. The study used only exploratory factor analysis for identifying the factor structure. This approach has its own limitations. Therefore, future studies may consider confirmatory factor analysis to test the hypothesized four factor model using more robust data. This would provide better understanding of the model and also help in further refinement of the scale.

Declarations

Author contribution statement

Dr. Rani Gul: Conceived and designed the experiments; Wrote the paper.

Dr. Iqbal Ahmad: Analyzed and interpreted the data.

Dr Tehseen Tahir: Performed the experiments.

Dr Umbreen Ishfaq: Contributed reagents, materials, analysis tools or data.

Funding statement

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Data availability statement

Data included in article/supplementary material/referenced in article.

Declaration of interests statement

The authors declare no conflict of interest.

Additional information

No additional information is available for this paper.