Examining the validity and reliability of the revised developmental work personality scale (RDWPS)-traditional Chinese version.

BACKGROUND: Work personality was found to relate to successful work adjustment, job readiness, work motivation and job maintenance. The revised developmental work personality scale (RDWPS) is a self-reported assessment to evaluate the work personality of the examinee which further psychometric study is required and needs to be applied to different culture. The aim of this study was to examine the theoretical structure, validity, and reliability of the traditional Chinese version of the RDWPS.
METHODS: The subjects were 113 university students with no known physical or mental illness who aged between 19 and 22. Cluster analysis was used to examine the theoretical structure. Rasch analysis was applied to examine the psychometric properties.
RESULTS: We dropped the unfit three items and found the 11 items of traditional Chinese version of the RDWPS was adequate to fit the theoretical construct. The results of the Rasch analysis showed that the 11 items with a three-point rating scale of the traditional Chinese version of the RDWPS had acceptable internal consistency (α = 0.76), moderate reliability, and met the criteria of unidimensionality.
CONCLUSIONS: The results of this study provided initial evidence that the traditional Chinese version of the RDWPS can be readily applied to research related to work behavior for those whose primary language is traditional Chinese, with adequate reliability and validity.

Introduction

Work Personality is a construct developed in the field of rehabilitation psychology and has been found to play a critical role in developing the foundation for effective vocational and career behavior and has been linked to the development of effective work behaviors in the work environment (Bolton, 1992; Hershenson, 1996a, 1996b; Strauser et al., 1999). Given the theoretical origins and current research in the area of work personality, researchers have advocated applying a developmental conceptualization of work personality based on the integration of Erikson’s theory of human development, Neff’s theory of work adjustment, and Bandura’s social cognitive theory (Strauser et al., 1999). Applying a developmental perspective emphasizes the importance of home and school environments, social skills, and role models embedded in each individual’s environment as it is related the development of appropriate and effective adult work behaviors (Strauser et al., 1999). Research in the area of developmental work personality has consistently found that higher levels of developmental work personality have been related to successful work adjustment, job readiness, work motivation and job maintenance across various populations including college students, adults, and individuals with disabilities (O'Sullivan & Strauser, 2010; Strauser et al., 2012; Strauser et al., 1999; Strauser et al. , 2013).

Applying the developmental approach to work personality in vocational and occupational settings it is important to have a psychometrically sound measure that provides a valid and reliable assessment of work personality (Strauser & Keim, 2002). Initially, applying developmental work personality theory, Strauser and Keim (2002) created a 26-item Developmental Work Personality Scale (DWPS), which claimed to measure the social behaviors, role model and tasks that individuals encountered during the Eriksonian development period of Industry versus Inferiority. Construct validity for the initial scale was demonstrated with the sample of participants in a major urban area in the United States and the initial instrument was identified as the Developmental Work Personality Scale (DWPS). O'Sullivan and Strauser (2010) investigated the DWPS in a three-phase study, focusing on factor structure, convergent validity and stability, demonstrating the robust psychometric properties of the scale related to persons with disabilities. Three strong factors (i.e., Work Tasks, Social Skills and Role Model) emerged that were theoretically consistent and had positive significant correlations with other measures of work behavior and personality (Work Personality Profile-Self-Report) and the NEO Five-Factor Inventory subscales of Agreeableness and Conscientiousness. Given the very low test-retest reliability coefficient (r2 = .29) associated with the Role Model subscale, it was determined that further examination with a more balanced gender-represented sample of participants was needed. Strauser et al. (2013) used the DWPS to examine differences in developmental work personality in a sample of young adult CNS (central nervous system cancer) cancer survivors and a group of young adult college students without disabilities, again finding a low reliability for the role model dimension (α = .27). Strauser et al. (2012) investigated the relationship between the variables of work engagement, development work personality and academic effort in a small sample of 65 college undergraduate students. Using Hierarchical Linear Modeling, results revealed a positive relationship between the three variables and identified gender differences regarding work personality with female students scoring higher than male students across all subscales.

Given the continued difficulty with the Role Model subscale, Wong et al. (2012) revised the Role Model subscale by adding 9 items to the original 26 items resulting in a 35-item self-response measure to better reflect role model types. Through additional validity analysis the 35-item scale was reduced to 14-items with the same theoretically relevant subscales being identified. The 14-item scale was labeled the Revised Developmental Work Personality Scale (RDWPS). To date, a review of the literature has not revealed any additional investigations into the psychometric qualities and dimensionality of the RDWPS. In addition, a review of the literature indicates that to date no research has been conducted to examine if the cross-cultural application of work personality as it relates to vocational and occupational outcomes. One major issue that may be impacting cross-cultural research is the lack of a linguistically appropriate, reliable, and valid version of the RDWPS that can be used in research.

As a result of the limited investigation into the psychometric qualities of the RDWPS, and the lack of a non-English version of the RDWPS that can be used in cross-cultural research, there appears to be a significant need to further examine the RDWPS from a cross cultural perspective. Therefore, the purpose of this study is to conduct a comprehensive psychometric analysis of the RDWPS that has been translated to Traditional Chinese with the goal of providing practitioners and researchers with a scale that can be used in both clinical and research settings to measure the important construct of work personality. Specifically, the goal of this study is to examine the theoretical structure, validity, and reliability of the Traditional Chinese version of the RDWPS. We have the following hypotheses:

Through cluster analysis, the traditional Chinese version of the RDWPS would be found to possess subscales which are theoretically consistent with the developmental model of work personality.

The identical subscales of the traditional Chinese version of the RDWPS would show adequate validity and reliability in the Rasch measurement model.

Methods

Translation

We obtained the translation permit. One of the authors translated the original version into Traditional Chinese; then the other author back translated the Traditional Chinese version into English. Afterwards the original developer compared two versions and discussed with both authors whether they were the same as the original expression. The final version was decided based on culturally relevance and consistency with original idea. The agreement rate for the first phase translation is 87%. We made slight modification on two items (item 9 &10), other items were considered clear. The final version of the RDWPS was determined after experts (including original developer and two bilingual occupational therapists) agreed on its readability, comprehensibility and congruence with the original version.

Participants & procedures

The study was approved by Institutional Review Board (201911082 W). We sent out 130 questionnaires to university students who are at least 18 years old and received 113 questionnaires in return. Participants consented to participate in the study and filled up the questionnaire. The overall response rate was 86.9%. The participants were university students with no known physical or mental illness which aged from 19 to 22, lived in Taiwan, and composed by 74% female and 26% male and it is a convenience sample. Please refer to Table 1 for details of the sample. The inclusion criteria was: above 18 years old; willingness to participate in the study; and were cognitively competent to fill out the self-reported questionnaire (Table 1).

Table 1.

Demographic characteristics of the participants (n = 113).

variable	n	%
Gender
Male	26	25.66
Female	84	74.33
Variable	Mean	SD (range)
Age,	20.04	0.81 (19–22)
RDWPS score	54.57	7.82 (31–65)

RDWPS: The Revised Developmental Work Personality Scale.

Instrument

The revised developmental work personality scale (RDWPS) (Table 2)

The RDWPS was designed to evaluate the status of the developmental working personality based on 14 items (See Table 2). Each item is rated on a 6-point rating scale representing from 0 representing “not like me at all” to 5 representing “very much like me”. The examinees were asked to recall their childhood experience and choose the most similar answer to each question. Example of the question will be “I was in trouble a lot with my teachers”.

Table 2.

The revised developmental work personality scale.Please reflect on your childhood experiences. Answer the following questions according to how much the experience was like you. A score of 0 indicates the behavior was not at all like you, a score of 5 indicates the behavior was very much like you.

Not at all like me Very much like me
0	1	2	3	4		5
1.	I completed school through the following grade: 5 6 7 8 9 10 11 12 12+
2.	I was in trouble a lot with my teachers.
3.	When I was in school I got in trouble a lot.
4.	When I was in school, I had problems getting along with classmates.
5.	In school I completed my work on time.
6.	In school I tried my best even if I didn’t like what I was doing.
7.	It was important for me to complete all my school work
8.	It made me feel good when I completed all my school work
9.	I complete all my assignments in school
10.	When I needed help with my homework, one of my parents was available to help.
11.	Growing up, I was responsible for chores at home.
12.	Growing up, I had someone who inspired me.
13.	There was someone in my life whom I admired.
14.	If I did not do my homework or chores, I got into trouble.
15.	I got in arguments a lot with classmates when I was in school.

Statistical analysis

Cluster analysis

Hierarchical cluster analysis using Ward’s method applying squared Euclidean Distance as the distance measure was applied to identify latent clusters, grouping of scale items which were related to the items making up the RDWPS. Specifically, the analysis of variance approach was used to evaluate the distances between clusters with cluster membership being assessed by calculating the total sum of the squared deviations from the mean cluster. That is, the fusion criterion coefficient produced is the smallest possible increase in the error of the sum of squares. After the jump in criterion coefficient is identified, examination of the dendrogram allowed for further examination of the cluster structure. These clusters of items can be conceptualized as instrument subscales. Upon identification of an acceptable number of clusters (subscales), the cluster analysis will be rerun specifying the chosen optimum number of clusters to determine if the cluster structure is sound. As with the first cluster analysis, coefficients and the examination of the dendrogram will be used to examine if there is confirmation to the cluster structure. Cluster analyses were conducted by SPSS 22 version.

Rasch analysis

Rasch analyses were performed based on the result from cluster analysis using the WINSTEPS Rasch computer program (version 3.63.2, Winsteps.com, Chicago, IL, USA). We applied partial credit model in the analysis. The Rasch analysis was proceeded with the following 5 phases: (Chen et al., 2015).

Phase 1: Rating scale analysis

This is to examine whether the rating scale structure of each item was adapted appropriately. (Amin et al., 2012) It was examined based on the following criteria set out by Linacre (2002): (1) at least 10 observations in each category and a regular observation distribution across categories; (2) monotonically increasing average measures across categories; (3) monotonically increasing step calibrations. If a threshold disordering appeared in the rating scale and the average measure did not advance with category, we intended to collapse the adjacent categories. If items failed to meet the assertions of the Rasch model, the plan was to omit them one at a time until all items met the criteria. Prior to the removal of an item, we considered the theoretical importance of the item and tried to retain as many items as possible.(Chien & Bond, 2009; Forkmann et al., 2009).

Phase 2: Unidimensionality

Infit mean square statistics (MnSq) and t standardized statistics (Zstd) were used to examine the fitness of the Rasch model of the scale items. Infit is sensitive to ratings on the items located close to the participants’ latent trait. The goodness of fit was set at MnSq = 0.6–1.4 with Zstd = −2 to 2 (Amin et al., 2012; Kim & Park, 2011). When items demonstrate statistical goodness of fit with the Rasch model, the scale can be said to be unidimensional, thus supporting the scale’s construct validity (Walker et al., 2012).

Phase 3: Targeting

Targeting refers to the extent that items are of an appropriate difficulty level for the participants. We adapted Test Information Function (TIF) as an index for accuracy. The TIF is defined as the reciprocal of the radical of the standard error measurement (SEM) function (TIF = 1/√SEM) and it estimates the degree of accuracy with which an individual’s work personality can be measured across the total latent continuum. The typical shape of an TIF is a bell-shaped curve with high information functions corresponding to high precision of the work personality estimates (i.e., low standard error). Item measure estimates with an SEM < 0.5 were considered sufficiently precise (Lai et al., 2007; Lai et al., 2005).

Phase 4: Reliability analysis

The reliability information is revealed by two indices of a separation reliability coefficient and a separation index. The person separation index indicates how well the RDWPS items separate the participants into statistically distinct levels of severity. The item separation index indicates how well the participants separate the items into different levels of difficulty. The separation index should be at least 2.0 to obtain the desired reliability coefficient of 0.8, indicating that the sample/items can be separated into at least three distinct groups/levels. The formula used to calculate the strata is Strata = (4 G + 1)/3, where G is the separation index (Arnadottir & Fisher, 2008). In addition, the internal consistency of the scale was examined by Cronbach a (criterion: >0.7).

Phase 5: Rasch cut-off score

Traditional analysis treats raw scores as interval data. Rasch measurement model provides a way to transform the raw score of the item to an interval score, which is more appropriate to be used as a cut-off point. In order to differentiate statistically the levels of work personality level among the participants, we divided the estimates (person measure) into distinct levels based on the method suggested by Wright (Wright, 2001). We started at the lowest end of the raw score range and worked toward the highest. We advanced each time by twice the joint standard error of the current starting and ending measures until there was no room for another level.

Result

Cluster analysis

Examination of the coefficients and dendrogram (Figure 1(a)) revealed an acceptable three cluster solution that made sense of the theoretical concept and matched the subscales identified in prior research examining the structure of the RDWPS. Specifically, items 2, 3, 4, and 15 were consistent with the previously identified Social subscale, items 5, 6, 7, 8, and 9 were consistent with the Task Orientation subscale, and finally items 11, 12, and 13 were resembled the Role Model subscale. However, items 10 and 14 were not assigned to cluster, and item 11 was identified as theoretically identifying with the wrong cluster. As a result, these items were dropped due to their perceived lack of fit. Finally, the cluster analysis confirmed the results of three subscales of Social (items 2, 3, 4, and 15), Role Models (items 12 & 13), and Task Orientation (items 5, 6, 7, 8, and 9) (Figure 1(b)). The subscale items in the original version and the Traditional Chinese version and the reduced item version of RDWPS can be seen in Table 3.

Figure 1.

(a) First cluster analysis with 14 items. (b) 2nd cluster analysis with 11 items.

Table 3.

The subscale items in three versions of the RDWPS.

subscales	RDWPS-original version		RDWPS-Traditional Chinese version		RDWPS- reduced item version
Social Skills	Item2	I was in trouble a lot with my teachers.	Item2	我和老師之間有很多麻煩。	Item2	我和老師之間有很多麻煩。
	Item3	When I was in school I got in trouble a lot.	Item3	我在學校時我惹上很多麻煩。	Item3	我在學校時我惹上很多麻煩。
	Item4	When I was in school, I had problems getting along with classmates.	Item4	我在學校時和同學相處有問題。	Item4	我在學校時和同學相處有問題。
	Item15	I got in arguments a lot with classmates when I was in school.	Item15	在學校時我和同學會起很多爭執。	Item15	在學校時我和同學會起很多爭執。
Work Tasks	Item5	In school I completed my work on time.	Item5	我準時完成我的作業。	Item5	我準時完成我的作業。
	Item6	In school I tried my best even if I didn’t like what I was doing.	Item6	我在學校時有盡力去做事, 就算是我不喜歡的事情。	Item6	我在學校時有盡力去做事, 就算是我不喜歡的事情。
	Item7	It was important for me to complete all my school work	Item7	完成我所有學校作業對我來說是重要的。	Item7	完成我所有學校作業對我來說是重要的。
	Item8	It made me feel good when I completed all my school work.	Item8	完成所有學校作業讓我感覺良好。	Item8	完成所有學校作業讓我感覺良好。
	Item9	I complete all my assignments in school.	Item9	我能完成老師指派需於課堂中完成的作業。	Item9	我能完成老師指派需於課堂中完成的作業。.
	Item11	Growing up, I was responsible for chores at home.	Item11	長大後, 我需負責做家事。	Item11	Deleted
	Item14	If I did not do my homework or chores, I got into trouble.	Item14	如果我沒做作業或家事, 就會陷入麻煩之中。	Item14	Deleted
Role Model	Item10	When I needed help with my homework, one of my parents was available to help.	Item10	當我小學寫作業需要幫忙時, 父親或母親其中一人會來幫我。	Item10	Deleted
	Item12	Growing up, I had someone who inspired me.	Item12	長大後, 我遇見激勵我的人。	Item12	長大後, 我遇見激勵我的人。
	Item13	There was someone in my life whom I admired.	Item13	在我生命中有一位我敬愛的人。	Item13	在我生命中有一位我敬愛的人。

RDWPS: The Revised Developmental Work Personality Scale.

Rasch analysis

Based on the criteria of rating scale analysis as mentioned in the rating scale part of the methodology session, we collapsed the adjacent categories of 11 items. The results showed that item 6 was changed to 4-point rating scale; item 2–5, 7–9, and 12–13 were changed to 3-point rating scale. Since item 13 still showed sign of displacement of categories, we decided to collapse rating scale into 2-point rating scale. The final rating scale structure of the remaining items were appropriate.

All items fit the criteria except item 12, which did not meet the criteria of unidimensionality (Infit MMSQ = 1.53, ZSTD = 4.1) (Table 4). However, due to the theoretical necessity, we decided to keep it in the scale (Forkmann et al., 2009). The principal component analysis of the residual showed that 46% of the variance were explained by 11 items of RSWPS.

Table 4.

The Rasch Item Statistics of the RDWPS.

Entry number	Measure	Infit		PTMEA CORR.
		MNSQ	ZSTD
12	0.18	1.53	4.1	0.25
15	0.11	1.17	1.5	0.45
4	0.56	1.11	1.1	0.48
13	–0.34	1.07	0.7	0.33
6	0.23	1.05	0.4	0.57
8	0.11	0.96	–0.3	0.57
3	–0.23	0.95	–0.4	0.56
5	–0.01	0.82	–1.6	0.63
9	–0.26	0.77	–2.0	0.65
2	–0.60	0.77	–1.8	0.64
7	0.23	0.75	–2.4	0.67

RDWPS: The Revised Developmental Work Personality Scale.

The TIF values ranged from 0.05 to 5.56 with the highest TIF (also lowest SEM = 0.42) occurring in the middle range of the work personality continuum. The 11 items reliably (SEM < 0.5) measured participants who reported their work personality with scale scores between −1.14 and 1.14, which captured 61.9% of the sample, indicating that the RDWPS provided an acceptable estimate for most participants in this study (Lai et al., 2007; Lai et al., 2005). Figure 2 shows the relationship of TIF, SEM, and the distribution of scores on the same continuum.

Figure 2.

The TIF and SEM of the RDWPS.

Person separation, Item separation (1.68), and item reliability (0.74) were acceptable and clearly exceeded critical values indicating that the RDWPS defined at least two strata of items. The RDWPS showed acceptable internal consistency(α = 0.76).

According to the Rasch analysis, the range of person measures based on the 11 items was from −3.01 to 4.33 logits. According to Wright’s method, the measures were divided into four statistically distinct levels of work personality disturbances (Table 5).

Table 5.

Work personality disturbances.

Group (number of people)	Score range
1 (8)	1–6
2 (37)	7–13
3 (58)	14–19
4 (10)	20–22

Discussion

The primary purpose of this study was to examine the theoretical structure, validity, and reliability of the Chinese version of the RDWPS. The results of the study provide support for the traditional Chinese version of the RDWPS as being a reliable and valid measure of the construct of developmental work personality. It also confirms that there are three factors contributing to the development of work personality and the concept can be utilized as intervention strategy to foster clients’ positive work personality. Our results suggest that the RDWPS 11 items (The final Traditional Chinese version see in appendix) have a unidimensional construct and can be measured on a continuum.

The result of the targeting analysis showed that the distribution of the level of item difficulty of the RDWPS-11 was lower than average persons measured (accuracy rate as 61.9%). Since our participants were college students without known illness, their score distribution is skewed and lack of variance, therefore there is low accuracy rate. It is suggested that in the future, a more diverse sample can be collected to ensure the scale targeting.

We dropped three items based on the result of cluster analysis, including item 10” when I needed help with my homework, one of my parents was available to help”, item 11” growing up, I was responsible for chores at home”, and item 14” if I did not do my homework or chores, I got into trouble”. We argue that there are cultural differences in terms of the degree of assistance parents offer when the kids meet difficulty of homework, and if parents would let their kids perform chores independently as well as the demand of task done such as homework and chores. Previous study showed that the Caucasian families involved their kids in personal independence activities (such as household chores) more often than Asian families (Zhang, 2005) might be a proof of it. Furthermore, there might be differences of the parenting skills and value differences across cultures (Wu et al., 2002), so that these items did not fit. Lastly, the differences might be due to the issues of translations (Fisher et al., 1992).

In Rasch analysis, item 12 did not meet the criteria of unidimensionality. However, item 12 belongs to domain of role model, if it is deleted, there will be one item left for role model domain. Therefore, we decided to keep it in the scale.

Limitations

The limitations of this study include the small sample size and that the sample is homogeneous. And our sample’s ability of work personality is high. We suggest that future study can recruit samples with diverse employment status and disease situations. With a bigger and more diverse samples, we can identify if the 11 item RDWPS traditional Chinese version is of sufficient level of difficulty to measure the status of the work personality.

Besides, we did not perform differential item functioning (DIF) in Rasch analysis, so we didn’t know if participants of different subgroups have the same probability of endorsing the item.

Conclusion

In conclusion, the results of this study provided initial evidence that the RDWPS can be readily applied to research related to work behavior for those whose primary language is Traditional Chinese, with adequate reliability and validity. Occupational therapy and rehabilitation researchers can use the scale to measure the developmental aspects of work personality to gain a better understanding of how personality impacts work performance and occupational outcomes.

Click here for additional data file.

Supplemental material, sj-pdf-1-hjo-10.1177_15691861211021926 for Examining the validity and reliability of the revised developmental work personality scale (RDWPS)-traditional Chinese version by De-Chang Liu, David Strauser and Ay-Woan Pan in Hong Kong Journal of Occupational Therapy