Only When the Societal Impact Potential Is High? A Panel Study of the Relationship Between Public Service Motivation and Perceived Performance.

Many studies find positive associations between public service motivation (PSM) and performance, but much of this literature is based on cross-sectional data prone to endogeneity and common method bias. Moreover, we know little about potential moderators. In this study, we test the moderating role of societal impact potential (SIP)-the degree to which the job is perceived to provide opportunities to contribute to society. We use cross-sectional data from 13,967 employees in 2010 and 2012 aggregated to construct longitudinal data for 42 organizations. As expected, the association between PSM and individual perceived performance is positive when SIP is high. However, when SIP is low, PSM is only weakly or not at all related to performance. This is an important insight for organizations that try to enhance performance through PSM. Our findings suggest that this can only be done when the employees think that their jobs allow them to contribute to society.

Introduction

How can we explain why some public organizations perform better than others? The public administration literature indicates that public service motivation (PSM) can be an important element in explaining high performance, because it can make employees go “above and beyond the call of duty” (DiIulio, 1994; Perry & Wise, 1990). PSM represents an autonomous type of motivation, stemming from a sense of duty and identification with public service provision (Houston, 2011; Vandenabeele, 2013). Individuals with high PSM therefore feel an internal drive to work hard and do well, if they work on a public task (Vandenabeele, 2007). Several studies have documented a positive association between PSM and individual performance (e.g., Andersen, Heinesen, & Pedersen, 2014; Bellé, 2013; Bright, 2007; Vandenabeele, 2009), but we know less about this relationship at the organizational level. Some studies conclude that PSM is positively associated with organizational performance (Brewer & Selden, 2000; Kim, 2005), but most of these studies only look at individual employees as the unit of analysis.

This study reexamines the hypothesis about a positive PSM–performance relationship. We seek to make two significant contributions. Utilizing a large representative data set of Dutch public-sector employees in 42 organizations collected as two independent, representative samples in 2010 and 2012, we test the hypothesized positive association using the individual-level pooled data set (n = 13,697) and provide a conservative test of the PSM–(perceived) performance relationship over time at the organizational level (n = 42)—testing the robustness of the individual-level findings in a panel setup where our data allow us to use a fixed effects regression analysis. In addition to looking at organizations as relevant units of analysis, our study also asks whether the relationship only exists when there is a good fit.

Our contribution regarding whether the PSM–performance relationship is context-dependent is especially relevant for the generalizability of the PSM–performance relationship. Studies of the PSM–performance relationship usually assume that it is possible for all employees in public organizations to contribute to society. We know from previous studies that person–environment fit (P-E)—the fit between person and work environment in values or needs (Edwards & Shipp, 2007; Kristof-Brown, Zimmerman, & Johnson, 2005)—matters for the relationship between PSM and work outcomes (Bright, 2007; Leisink & Steijn, 2009; Taylor, 2008; Wright & Pandey, 2008). It may thus be essential whether employees actually perceive their job to have a high societal impact potential (SIP; that is, they perceive their jobs as providing opportunities to make a meaningful contribution to society). This study begins to fill the gap in the literature by including the SIP of a job. This subjective (PSM-) fit can be seen as a needs–supply equilibrium, a specific type of person–job fit (Kristof-Brown et al., 2005). We expect that the PSM–performance relationship will be strongest when SIP is high.

Regarding our second contribution, there are theoretical as well as methodological reasons for supplementing existing individual-level, cross-sectional analyses of the PSM–performance relationship with a study over time at the organizational level. Theoretically, Brewer and Selden (2000) argued that employees with high PSM reach beyond their own work tasks and help colleagues through “extra-role” behaviors (Smith, Organ, & Near, 1983; Williams & Anderson, 1991). Methodologically, aggregated organizational data of individual PSM and performance over time at the organizational level make it possible to overcome previous studies’ endogeneity problems and reduce common method bias (Favero & Bullock, 2014; Jakobsen & Jensen, 2015).

This study provides insight in whether enhancing PSM through selection or socialization can be a fruitful way to enhance performance in all public organizations or only those with a high SIP. We thus analyze whether PSM is always positively related to performance or if this is only the case in some situations. Moreover, the study provides information on whether SIP perceptions change over time, which might mean that it can be influenced by managers to enhance the impact of PSM on performance.

In the following, we discuss the relationship between PSM and performance. Based on these contributions, we present our theoretical model which is then tested in the empirical results section after a presentation and discussion of data and methods. We conclude with a discussion of the findings and points for future research and practice.

Theoretical Framework

The Relationship Between PSM and Performance

According to Perry and Wise (1990), PSM can be seen as an individual drive toward providing public services stemming from rational (wanting to participate in and enhance public services), normative (a commitment to the public interest and a feeling of duty), and affective (compassion and identification with others) motives. Vandenabeele (2007) defined PSM as “the belief, the values and attitudes that go beyond self-interest and organizational interest, that concern the interest of a larger political entity and that motivate individuals to act accordingly whenever appropriate” (p. 549). PSM is not a stable trait but is shaped and formed throughout life by the institutions with which the individual interacts (Perry & Vandenabeele, 2008).

Perry and Wise (1990) proposed a positive relationship between PSM and individual performance, and this spurred empirical research on the relationship. Performance of a public organization is seen as the degree it achieves its goals (as specified in its mission), and individual performance is seen as the individual’s contribution to achieving this mission. However, to study how well public organizations and those within public organizations are doing their job is complex: A single measure can seldom accurately and fully capture performance of public organizations (Andrews, Boyne, & Walker, 2006; Boyne, 2002; Brewer & Selden, 2000). For public organizations, it is difficult to measure their performance through financial targets because they have multiple (oftentimes conflicting) goals (Behn, 2003; Boyne, 2002; Brewer, 2006).

One approach is to distinguish between different aspects of performance, analyzing how these different performance criteria are influenced. Another approach (used in this study) is to find a general, overall measure of performance. Given that this study compares many different organizations, employee self-reported general performance is a useful measure because it allows for a comparison across organizations and jobs as opposed to more context-specific measures. By focusing on employee self-perceived performance, we thus show one (comparable) piece of the puzzle.

Given these observations, a positive relationship between PSM and individual performance is likely, and it is theoretically assumed to be based on various mechanisms, among which are identification and commitment mechanisms (Brewer, 2008). Those who feel a strong drive to contribute to society are expected to identify strongly with the work they are doing in the public sector, adhering to a logic of appropriateness rather than a logic of consequence (March & Olsen, 1989). Doing something because you think it is important corresponds to a more autonomous type of regulation (Deci & Ryan, 2004); hence, PSM can qualify as an autonomous yet extrinsic type of motivation (Perry, 2000; Vandenabeele, 2007). Having this kind of motivation for the work means the individual will exert more effort, and it is consequently expected that those who are highly public service motivated will perform better than those who do not have high PSM (Andersen et al., 2014; Brewer, 2008; Vandenabeele, 2009).

Moreover, a high commitment to public service may drive employees to place their work above their own interests and do more than they are asked to do (Brewer, 2008; DiIulio, 1994). Empirical studies on PSM have mainly focused on the individual level because PSM is an individual-level construct, and the most likely effect of PSM is on individual performance (Bellé, 2013; Bright, 2007; Perry & Wise, 1990; Vandenabeele, 2009). Almost all studies found a significant positive relationship between PSM and individual performance. Our first expectation, regarding the individual level, is therefore formulated as follows:

Hypothesis 1: Individual PSM is positively related to self-perceived individual performance in public organizations.

In addition to PSM, other types of motivation (such as intrinsic task motivation) can be relevant for performance, and this is also the case for variables such as employee abilities and managerial practices (e.g., Walker & Andrews, 2015). This may mean that the association between PSM and performance is expected to be relatively weak, as also implied by Warren and Chen’s (2013) meta-analysis. Still, given the arguments presented above, we expect the association to be positive and statistically significant.

The Relationship Between PSM and Performance at the Organizational Level

Conceptualizing individual performance as the individual’s contribution to achieving the organizations’ mission emphasizes that organizational performance is often more important than individual performance in public organizations as the work of all individuals together creates the quality of the service. If one nurse performs excellent but his or her colleagues perform poorly, the quality of health care is at risk. Thus, it is often the organizational performance that matters. Goals are regularly formulated at the organizational level, making it very important to address how the characteristics of the employees affect the performance of the entire organization. Similar to individual performance, organizational performance can have many dimensions, and it might look differently seen from different positions within and outside the organization. Ideally, organizational performance would be conceptualized as a weighted sum of the legitimate stakeholders’ understandings of performance, but a less complex conceptualization can also contribute to our knowledge about the association between PSM and organizational performance. This section will discuss our theoretical arguments concerning this association and show how it supplements studies of individual performance.

One of the reasons behind the relevance of analyzing PSM and performance at the organizational level is that public managers can work at the organizational level to support PSM. The average organizational PSM is a result of selection and attrition as well as changes in the motivation among the individuals remaining in the organization. Organizations that are able to attract, socialize, and retain employees with higher levels of PSM are expected to have higher levels of performance, because their employees perform better, work harder, and help each other (Bellé, 2013; Bright, 2007; Gould-Williams, Mostafa, & Bottomley, 2015; Leisink & Steijn, 2009; Vandenabeele, 2009). Organizational performance is seldom just the sum of the individual performance on the task (Brewer & Selden, 2000). Individuals can reach beyond their own tasks through their “extra-role” behaviors to help colleagues and the organization in general (Smith et al., 1983; Williams & Anderson, 1991). Likewise, Petrovsky and Ritz (2014) also argued that a “highly PSM infused culture due to high levels of individuals’ PSM and leadership based on PSM values is expected to positively affect organizational performance” (p. 60). Furthermore, having an organization with high average PSM might also mean that the organization is collectively better able to deal with more complex bureaucratic processes and red tape which may add to foster and retain a higher organizational performance (Petrovsky & Ritz, 2014).

Still, the key mechanism behind the potential association between PSM and performance at the organizational level is that PSM contributes to organizational performance through higher individual performance (Perry & Wise, 1990). We therefore focus on the average individual performance in the organization. It is a conservative test because the performance measure used here only includes an evaluation of the individuals’ performance on their tasks, not of their extra-role performance, on which PSM has been found to have a substantial effect and which also increases organizational performance (Gould-Williams et al., 2015; Kim, 2006; Podsakoff, Ahearne, & MacKenzie, 1997). Still, aggregated individual performance provides information on how the individuals in the organization on average are doing their work.

Performing an analysis at the organizational level also helps alleviate a methodological problem. Using cross-sectional data, some previous studies have, as mentioned, analyzed the relationship between PSM and perceived organizational performance at the individual level and found a positive association (Brewer & Selden, 2000; Kim, 2006). By testing the PSM–performance relationship with cross-sectional data and self-perceived performance at the individual level, such analyses are, however, very likely to suffer from endogeneity problems and/or common method bias leading to a potential overestimation of the hypothesized association or even worse—a false positive. To alleviate some of this potential bias, the data used in our study consist of different respondents in both years and are aggregated to the organizational level to be able to analyze the relationships over time. Compared with the previous studies, this enables us to control for time-invariant variables causing bias at the individual level while separating measurement of independent and dependent variables, and we thus overcome some of the potential endogeneity problems.

To understand why aggregating of PSM to the organizational level helps us test our theoretical expectation of a positive PSM–performance association and which way is appropriate, the nature of the group and the concepts of interest are important (Bell, 2007). First, the level on which the relationship is analyzed—organizations—is in this study heterogeneous and does not bear the characteristics of an interdependent team. On a lower level, the employees may be part of a team—for instance, a chirurgical team in which the members depend on each other to succeed—but the organization as a whole is not interdependent on each and every individual. This can be described as an additive model, where each member contributes to the organization’s performance (Bell, 2007; Chan, 1998). Moreover, PSM is an individual attribute (Perry & Wise, 1990) and the aim is not to test for the effect of similarities in PSM on performance, but rather whether an organization with a high proportion of public service motivated employees also has a higher average performance. Consistency or within-group agreement is less relevant in such a situation (Bell, 2007; Glew, 2009; Meyer & Schermuly, 2012). Due to these two characteristics of the analysis, aggregating to organizational means is the recommended procedure (Bell, 2007; Chan, 1998; Glew, 2009; Meyer & Schermuly, 2012).

A central question is here whether PSM can theoretically be expected to change over time. Average PSM in an organization can, as mentioned, change for two reasons: The employees can change (e.g., new employees with more PSM can be hired), and original employees’ PSM can change. Wright and Grant (2010) asked whether individual PSM is a dynamic state or a static trait, and recent research (Brænder & Andersen, 2013; Georgellis & Tabvuma, 2010; Kjeldsen & Jacobsen, 2012; Kroll & Vogel, 2014) indicates that it is dynamic and can change under the influence of, for instance, dramatic circumstances (war), education, and starting a job. PSM is therefore—especially at the organizational level—expected to be able to change over time, enabling us to test our second hypothesis:

Hypothesis 2: The average PSM of employees in an organization is related to the average individual performance in the organization over time.

The Context Dependency of the PSM–Performance Relationship

Although most empirical studies of employees have shown that PSM is positively related to performance (e.g., Andersen et al., 2014; Bright, 2007; Ritz, 2009; Vandenabeele, 2009), some studies did not find a significant relationship (Alonso & Lewis, 2001; Ritz, 2009). This may be due to differences in the institutional context related to whether the employee feels able to contribute to society, in which the PSM–performance relationship has been studied. Several studies have therefore included a measure of the fit between PSM and the environment (Bellé, 2013; Bright, 2007; Leisink & Steijn, 2009).

The institutional context can be important for the PSM–performance relationship, because highly public service motivated employees are expected to perform better due to their internal, autonomous drive to contribute to society, but only if they perceive they can do exactly that (Deci & Ryan, 2004; Vandenabeele, 2013). Institutional context can influence employees by creating formal and informal rules and defining the dominant logic (March & Olsen, 1989; Scott, 2001). Structural, normative, and cultural-cognitive elements are carried by organizational missions, rules, tasks, job design, symbols, and everyday practices (Scott, 2001). Perry and Wise (1990) already argued that being public service motivated means the individual is likely to respond to motives grounded in public institutions. Following this, several authors have pointed toward the institutional embeddedness of PSM (Vandenabeele, 2007). For instance, Wright and Pandey (2008) argued that the organization’s mission matters in linking PSM to behavior. Bellé and Ongaro (2014) argued that differences in institutional settings over time and between countries may influence the impact of reforms on PSM. These elements can differ between public organizations and may influence the relationship between PSM and performance.

It is thus potentially important that the work context provides opportunities to actually contribute to society (Brewer, 2008; Leisink & Steijn, 2009). Following PSM theory, a positive association between PSM and outcomes can be expected if—and only if—the work context (through the job, organization, team, etc.) provides opportunities to contribute to society or aims to fulfill a public mission (Bright, 2008; Kjeldsen & Andersen, 2012; Leisink & Steijn, 2009). Contributing to society can be seen as part of publicness, which is defined as organizational attachment to public values (Antonsen & Jørgensen, 1997).

Although publicness has mostly been linked to authority or ownership (Bozeman, 1987; Rainey, 2003), using an institutional approach to publicness means that not only structural elements of the environment but also normative and cultural-cognitive elements are seen as important for publicness, and are thus expected to play a role for the relationship between PSM and performance. Structural elements such as ownership sector and working conditions may matter (Bellé, 2013; Brewer & Brewer, 2011). Bellé (2013), for instance, found that the effect of PSM on effort, persistence, quality, and effectiveness was stronger under work conditions in which the prosocial impact of the job was clearer. Normative elements such as values and expectations of individuals regarding their job can also matter. For instance, several studies have found that the congruence between PSM and the organization’s values matters for performance (Bright, 2007; Gould-Williams et al., 2015).

P-E fit theory (Edwards & Shipp, 2007; Kristof-Brown et al., 2005) argues that the institutional context interacts with individual characteristics in determining performance. P-E fit theory distinguishes different forms of fit, such as needs–supply, demands–abilities, and congruence fit, and types, such as person–job fit and person–organization fit (Kristof-Brown et al., 2005). Contributing to society can be seen as an individual “need” that public service motivated employees strive to fulfill through their work, which searches for a “supply” of opportunities to do so. This needs–supply fit is mostly studied on the job level, studying person–job fit (Kristof-Brown et al., 2005). Empirical evidence suggests that both actual work settings and perceived fit matter for the relationship between PSM and work outcomes (Bright, 2007; Leisink & Steijn, 2009; Stritch & Christensen, 2014; Taylor, 2014; Wright & Pandey, 2008). Most PSM studies have, however, focused on perceived fit, studying direct perceptions of the fit with the job or organization (Bright, 2007; Gould-Williams et al., 2015; Wright & Pandey, 2008). Studies on perceived fit are useful for showing whether a fit with the job or organization matters but are less able to show in which specific contexts PSM has a strong positive relationship with individual performance and in which contexts this is not the case.

This study focuses on the SIP of a job, that is, employee’s perception of a job characteristic indicating to what extent the job provides opportunities to contribute to society (previously used by Leisink & Steijn, 2009). We focus on the SIP as moderator on the job level because previous studies on the importance of a fit with the job and organization have found that person–job fit was more important than that with the organization for employees with high PSM (Christensen & Wright, 2011). The perceived SIP can be seen as one of the normative elements of the publicness of the institutional context—which is also determined by organizational values, authority, ownership, type of users, job characteristics, and so on (Antonsen & Jørgensen, 1997; Bozeman, 1987; Rainey, 2003). As opposed to prosocial impact that shows the impact on others, SIP refers to the potential impact on society.[1] The interaction between PSM and SIP has been labeled PSM fit, a special type of (needs–supply) person–job fit measuring how well a public service motivated individual perceives to fit the opportunities provided by the job (Leisink & Steijn, 2009). Leisink and Steijn (2009), however, used the direct score of high SIP in combination with high PSM and named that PSM fit. Here, we study the moderation of the original variable, which we call societal impact potential.

The key claim in the PSM-fit argument is that individual performance is the result of an interaction between PSM and the publicness of the institutional context. Here, we focus on the perceptions of publicness by the employee—the degree of SIP of the job. If employees feel a drive to contribute to society through their work, but are not in a position to do so, this may lead to frustration, resigned satisfaction, or even burnout (Giauque, Anderfuhren-Biget, & Varone, 2013). We expect differences in the SIP perceived by employees between job types (e.g., administration, teaching, and social work), between organizations with the same job types, and over time. Some organizations are better able to provide or create jobs in which the perceived SIP is high (Moynihan & Pandey, 2007; Perry, 2000). If employees are highly public service motivated, but do not perceive a high SIP, their fit, and with that their performance, is expected to be lower.

Regarding the analysis over time, the change in average SIP per organization is included as moderator. With this average organizational SIP, we capture to what degree individuals in the organization on average see opportunities in their job to contribute to society. Although the SIP may differ between jobs within the organization, and some (such as supportive staff) may see less potential for having a meaningful impact on society in their jobs, the average SIP within the organization measures to what extent employees within the organization generally perceive that they can have an impact on society through their specific jobs. Management and leadership, but also the organizational rules and way of doing things, may have an important influence on the perceived SIP. If a supervisor highlights the problems in contributing to society, or protocols all processes, the perception of employees of opportunities to contribute may decrease. Following budget cuts, employees may experience they have to say “no” to citizens in need and feel less able to contribute to society. Importantly, we therefore argue that perceived SIP can change over time, and we utilize both the variation over time and the variation at a given time between individuals to test whether the association between PSM and self-perceived performance depends on SIP.

Hypothesis 3: The SIP of the job moderates the relationship between PSM and individual performance in such a way that PSM is more positively related to individual performance if the job has a SIP.

Research Model

Figure 1 shows the hypothesized models. First, the relationship between PSM and individual performance is analyzed (cross-sectional) on the individual level while controlling for the organization in which the individual works. Next, a panel analysis is carried out. In this analysis, the relationship between the changes in average individual PSM and SIP within the organization from 2010 to 2012, and the potentially corresponding changes in average individual performance in the organization from 2010 to 2012, is analyzed. Through this aggregation to the organizational level, a panel analysis can be done, which allows us to test whether the relationship between PSM, SIP, and performance remains significant when controlling for potential omitted variables and common method bias that do not change over time (Wooldridge, 2013). This design increases the robustness of the findings regarding the relationship between PSM and performance but still does not allow causal inference. How the relationships are analyzed will be explained in the next section.

Figure 1.

Theoretical model for relationship between PSM, SIP, and performance on the individual level and, over time, on the organizational level.

Note. PSM = public service motivation; SIP = societal impact potential.

Methods

Data collection

Every 2 years, the Dutch Ministry of Interior and Kingdom Relations conducts a “Personnel and Mobility Survey” (Personeels en Mobiliteitsonderzoek; POMO). The definition used to define “public” is the legal status of the organization. This article utilizes the data collected in 2010 (n = 26,830) and 2012 (n = 22,446) as the government decided to include multiple items on PSM from 2010 onward. Moreover, these years’ surveys allow identification of a large part of the sample respondents’ organizations, which makes it possible to analyze the same organizations over time. The data are collected by drawing representative samples from each organization. This is done by the Ministry itself. As the samples are representative, the average organizational PSM, SIP, and performance can be seen as comparable over years. Unfortunately, it is not possible to link a given individual’s answers over time—which is also the case with many similar data sets, for example, the yearly American Federal Employee Viewpoint survey, but the fact that new samples were drawn from the organizations each year limits the number of individuals appearing in both years.[2]

Employees for whom we could not identify the specific organization (such as municipality employees) are excluded, and so are organizations with less than 20 respondents in one of the years (seven organizations) and organizations that have been merged or split up from 2010 to 2012 (three organizations). The analysis includes respondents with all types of jobs as there is no reason why we should not expect PSM to also matter, for instance, for those in supportive functions. This left us with a pooled data set for both years of 13,967 employees in 42 organizations. The average age of the respondents in the pooled data set was 46 years and the average tenure was almost 8 years. Forty-nine percent of the sample was male, 58% of employees followed training, and 26% of the employees held a supervisory position. On average, the number of respondents per organization was 285.

Measures

The measurement model of PSM, SIP, and individual performance is analyzed with confirmatory factor analysis (CFA), which provides insight into how well the items represent the underlying construct (Kline, 2010).[3] Reliability is assessed using Raykov’s rho, which bases its calculation on the factor loadings of each item (Bacon, Sauer, & Young, 1995; Raykov, 2009). A Raykov’s rho of ρ = .70 or higher indicates good reliability.[4]

With respect to PSM, the survey contains several items measuring PSM on a scale from completely agree (5) to completely disagree (1) from Perry’s (1996) original list of items. This includes the dimensions of attraction to public policy (APP), compassion (COM), and commitment to the public interest (CPI). A CFA testing a second-order, three-dimensional model proved a good fit with the pooled data set (comparative fit index [CFI] = .963, Tucker–Lewis index [TLI] = .946, root mean square error of approximation [RMSEA] = .046). A reliability test indicates the scale to be reliable as it is above the recommended threshold (ρ = .783). To create one score on PSM, we first calculated scores for each dimension, going from 0 to 100, adding the scores on the relevant items. Then we combined the three dimensions to a composite measure of PSM, which was also rescaled to go from 0 to 100. Thus, all items have equal weight in the measures of the dimensions, and the three dimensions have equal weight in the PSM measure. This means that zero PSM is scoring low on all PSM items, while 100 oppositely is obtained by agreeing to all statements expressing high PSM (and disagreeing on statements expressing low PSM). Calculating the measure this way eases interpretation and relates our findings to define minimum and maximum scores while not changing the relative distances between units on the scale.

SIP is measured with three items, previously used by Leisink and Steijn (2009) measuring the opportunity to contribute to society. The overall question was to what degree the following statements are applicable to the employee’s job, that is, not referring to their opinion on specific issues, but how they perceive their job to be characterized. The scale ranges from not applicable at all (1) to completely applicable (5). A CFA with these three items does not lead to fit indices due to low degrees of freedom. However, factor loadings ranged from .53 to .86, and the reliability was high (ρ = .798). Therefore, we continue with the measure. The appendix contains a table with the average scores on PSM and SIP per organization in 2010 and 2012 (Table A1). It shows that most agencies improved their average SIP score from 2010 to 2012, but PSM decreased or increased depending on the organization. The lowest score on PSM was recorded in the Navy (at the time facing major budget cuts).

Table A1.

Average PSM and SIP per Organization in 2010 and 2012.

Organization	2010			2012
	n	PSM	SIP	n	PSM	SIP
R General Affairs	18	69.10	53.13	31	70.91	57.26
R Internal Affairs	155	69.58	56.45	248	67.29	56.17
R External Affairs	76	71.13	58.47	40	64.63	64.22
Taxing Agency	935	66.03	57.53	751	65.62	60.74
R Finance	94	65.91	54.85	79	67.74	58.54
Agency Judicial Facilities	402	67.36	59.93	307	66.41	62.95
Agency Immigration and Naturalization	84	66.69	57.07	29	66.59	63.58
R Justice	534	67.54	57.26	485	67.26	60.13
R Education, Culture, and Science	210	69.64	53.30	228	67.42	54.50
R Social Affairs and Employment	73	69.29	58.65	77	69.81	60.06
Rijkswaterstaat (highway/waterway)	316	68.53	58.47	319	66.73	61.07
R Health, Welfare, and Sport	139	67.33	53.64	135	67.55	58.84
U Erasmus University Rotterdam	70	68.24	48.93	58	65.98	60.54
U University of Leiden	126	67.16	44.99	68	65.32	50.46
U Radboud University Nijmegen	138	67.96	41.39	94	65.60	45.21
U Delft	180	64.34	44.48	126	64.77	51.79
U Technical University Eindhoven	86	64.61	44.40	70	67.20	51.70
U University of Maastricht	107	66.34	47.72	56	66.74	46.65
U University of Twente	92	64.50	47.76	79	65.03	50.40
U Utrecht University	215	68.44	47.99	141	66.50	52.48
U University of Amsterdam	152	69.93	48.07	113	68.46	53.48
U University of Tilburg	56	66.70	47.43	35	68.21	49.29
U VU University Amsterdam	119	68.75	50.05	86	68.85	50.73
U Wageningen University	89	70.39	47.61	59	68.57	50.00
U Open University	31	65.32	50.40	22	65.81	54.26
SFOM (research institute)	118	64.34	39.19	72	65.39	41.75
Royal Library	71	66.11	46.21	48	66.02	45.57
Dutch Association for Scientific Research	151	66.97	42.88	107	64.47	41.82
H Amsterdam Medical Center	135	65.63	49.54	96	65.86	52.99
H Leiden University Medical Center	171	65.07	47.48	107	64.56	49.77
H Vrije Universiteit Medical Center	124	65.03	50.25	122	64.82	51.08
H Academic Hospital Maastricht	119	66.11	47.49	72	64.55	51.13
H Erasmus Medical Center	243	66.45	49.43	194	65.66	55.00
H University Medica Center Utrecht	213	66.90	50.15	194	64.83	50.77
H University Medical Center Groningen	217	67.29	47.84	158	65.68	50.00
H Radboud Nijmegen	206	66.06	48.36	163	65.26	51.57
D Navy	202	63.35	54.95	256	63.11	58.79
D Land	433	64.55	53.74	519	63.85	58.56
D Air	223	66.38	53.62	270	63.99	58.15
D Marechaussee	107	66.45	63.96	175	66.56	63.68
P KLPD (police service for severe crimes)	152	67.49	59.05	154	65.91	60.11
P Police Academy (education)	74	66.08	54.98	68	69.33	64.34

Note. PSM = public service motivation; SIP = societal impact potential; R = Ministry; U = University; H = Hospital; D = Defense; P = Police; SFOM = Stichting voor Fundamenteel Onderzoek der Materie; KLPD=Korps Landelijke Politiediensten.

Finally, the survey contained several items which ask for a self-report of the evaluation the employee receives from colleagues and their supervisor. In this study, we use three items that refer to how the employee is appreciated in the organization as a proxy of his or her performance. Performance measures that are established impartially and without influence of the object being rated are sometimes seen as the golden standard (Andrews et al., 2006). For instance, Meier and O’Toole (2013) showed how subjective data can lead to biased results. However, in their comparison of subjective and objective data, they analyze different aspects of performance, whereas other studies with more comparable subjective and objective measures find higher correlations between the two types of measures (Andrews et al., 2006; Andrews, Boyne, & Walker, 2011; Bommer, Johnson, Rich, Podsakoff, & Mackenzie, 1995; Hoffman, Nathan, & Holden, 1991).

We argue that both objective and subjective performance measures have benefits. Although subjective measures are more comparable between different types of organizations and tend to capture a broader part of employees’ multiple goals (Lazear & Gibbs, 2009), these measures are more prone to common method bias and social desirability bias than more objective measures (Meier & O’Toole, 2013; Podsakoff, MacKenzie, Lee, & Podsakoff, 2003). Different types of performance measures (subjective, objective, external, internal) can thus be seen as different pieces of the public performance puzzle. Each piece provides important—but limited—information about performance; and different pieces serve different purposes (Andrews et al., 2006; Andrews et al., 2011; Behn, 2003; Brewer, 2006), indicating that both relatively subjective and relatively objective measures are useful for gaining insight into how organizations function. Our measure thus provides a limited view on performance—that view reported by the employee.

Self-reported performance measures may be flawed by various mechanisms (Podsakoff et al., 2003). However, this bias is not always in the same direction. Effect sizes can be inflated if both independent and dependent variables have the same social desirability bias or if the variables share other types of common method bias. Petrovsky and Ritz (2014) and Wright and Grant (2010) argued that many PSM–performance studies can have spurious results due to a social desirability component in both dependent and independent variables. Nevertheless, the results of a meta-analysis (Warren & Chen, 2013) on the PSM–performance relation indicate that studies with objective data actually demonstrate larger effect sizes than those with self-reported data. With regard to self-reported performance, Warren and Chen also noted that studies using control variables and studies not being carried out in the United States show higher effect sizes in the PSM–performance relationship.

When evaluating the validity of our self-perceived performance measure, one of the three items had a low loading; yet, the performance scale’s reliability was good (ρ = .730).[5] To test the fit of all constructs together, we ran a CFA including, PSM, SIP, and performance. This model fitted well (CFI = .944, TLI = .930, RMSEA = .042). The calculation of SIP and individual performance was done similar to the PSM measure, ensuring that it goes from zero (answers indicating low SIP or performance in all three questions) to 100 (answers indicating high SIP or performance in all three questions). Table 1 shows the items, factor loadings, average scores, and reliabilities.

Table 1.

Descriptive Statistics of Measures in Pooled Cross-Sectional Data Set (Individual Level).

Measures		FL	Minimum	Maximum	SD	M
PSM (ρ = .783; α = .690)			0	100	11.60	66.38
Public policy making			0	100	20.53	68.62
PSM0	Politics is a dirty word. (R)	.585
PSM1	I have little interest in politics. (R)	.901
Public interest/civic duty			0	100	15.36	60.09
PSM2	I unselfishly contribute to my community.	.507
PSM3	Providing meaningful public service is very important to me.	.693
PSM4	I find it more important to contribute to the public good than having personal success.	.592
PSM5	The general interest is a key driver in my daily life.	.732
Compassion			0	100	13.51	70.44
PSM6	It is difficult for me to contain my feelings when I see people in distress.	.615
PSM7	I think the welfare of fellow citizens is very important.	.740
PSM8	If we do not show more solidarity, our society will fall apart.	.516
SIP (ρ = .798, α = .753)			0	100	21.29	54.39
SIP0	I contribute to the development or execution of public policy in my job.	.692
SIP1	I contribute to the public interest through my job.	.851
SIP2	I contribute to achieving a greater degree of solidarity in our society through my job.	.537
Performance (ρ = .730, α = .550)			0	100	11.85	70.10
E3	Compared with people who do the same work as I do, I am highly appreciated by my organization.	.245
E4	In my work, colleagues ask me for advice if things get complicated.	.768
E5	In my work, I am given the more difficult jobs.	.738

Note. Both Raykov’s rho (ρ) and Cronbach’s alpha (α) are displayed. FL = factor loading; PSM = public service motivation; SIP = societal impact potential. (R) indicates reversed item.

Next to these measures, we control for several other factors. On the individual level, gender, age, tenure, salary, whether employees received training, and whether they held a supervisory position can all matter for their performance. On the organizational level, it is only relevant to control for factors that can change over time as the organizational-level fixed effects in the panel analysis rule out influence by time-invariant factors. Given that the composition of employees varies a bit over time, we therefore control for the percentage of women and supervisors, the average age, and the number of employees.

Statistical Models

To examine the association between individual PSM and performance depending on the experienced SIP of the respondents’ jobs, we use a pooled, cross-sectional POMO data set including all respondents (n = 13,967) who answered the surveys in 2010 and 2012. We estimate a Generalizaed Least Squares random effects regression which accounts for the organization in which the respondent works by including an organization-specific random intercept in the prediction of performance, that is, a multilevel regression that deals with the dependence among groups of individuals belonging to the same organization (Cameron & Trivedi, 2009; Rabe-Hesketh & Skrondal, 2008). In this way, the multilevel model allows us to estimate how much variation in performance is explained by PSM and SIP between different individuals within organizations and across organizations (R2 within and between organizations, respectively; cf. Table 2). More specifically, the model is hierarchically organized where the direct associations between PSM, SIP, and individual performance are analyzed first. Next, the possible moderation effect of SIP is tested by including the interaction term, PSM × SIP, created by multiplying the summative individual scores on PSM and SIP. Finally, the control variables are added to the analysis to see whether the relationship is robust when we control for potential confounders.

Table 2.

Individual-Level Cross-Sectional Analysis.

Dependent variable: Individual performance
	Model 1	Model 2	Model 3	Model 4	Model 5	Model 6
Constant	54.87*** (0.590)	65.75*** (1.404)	59.01*** (2.056)	63.74*** (2.074)	64.97*** (2.243)	64.18*** (2.298)
PSM	0.148*** (0.00867)	−0.0187 (0.0213)	−0.0149 (0.0213)	−0.00530 (0.0215)	−0.00902 (0.0227)	−0.00976 (0.0227)
SIP	0.104*** (0.00480)	−0.100*** (0.0244)	−0.103*** (0.0243)	−0.0919*** (0.0246)	−0.0936*** (0.0259)	−0.0927*** (0.0259)
PSM × SIP		0.00307*** (0.000359)	0.00307*** (0.000359)	0.00261*** (0.000363)	0.00255*** (0.000382)	0.00256*** (0.000382)
Gender (male = 1)			0.922*** (0.212)	−0.353 (0.219)	−0.437[†] (0.230)	−0.446[†] (0.234)
Age			0.365*** (0.0720)	0.0604 (0.0736)	−0.0153 (0.0804)	0.00466 (0.0808)
Age²			−0.00469*** (0.000807)	−0.00195* (0.000821)	−0.00104 (0.000894)	−0.00122 (0.000897)
Year (2012 = 1)			−0.484* (0.195)	−0.414* (0.197)	−0.219 (0.213)	−0.124 (0.223)
Salary				0.822*** (0.0394)	0.727*** (0.0438)	0.730*** (0.0441)
Tenure					0.00705 (0.0146)	0.00711 (0.0146)
Supervisory (y = 1)					1.943*** (0.271)	1.809*** (0.279)
Training (y = 1)					1.261*** (0.212)	1.278 *** (0.212)
No. of employees						−0.0000208 (0.0000147)
% supervisor						1.746 (1.266)
n (respondents)	13,967	13,967	13,967	13,264	11,925	11,925
r2 within	.0707	.0756	.0810	.110	.115	.115
r2 overall	.0643	.0691	.0745	.105	.112	.114
r2 between	.0474	.0438	.0332	.000110	.0128	.0298
sigma_u	0.792	0.811	0.654	0.443	0.481	0.547
sigma_e	11.40	11.37	11.34	11.16	11.20	11.20
rho	.00480	.00506	.00332	.00157	.00184	.00238
n (organizations)	42	42	42	42	40	40

Note. PSM = public service motivation; SIP = societal impact potential. Standard errors in parentheses.

p < .1. *p < .05. **p < .01. ***p < .001.

Moving on to test the proposed associations at the organizational level, we perform an organizational fixed effects panel regression allowing us to test how the changes in PSM and PSM × SIP from 2010 to 2012 are related to employee performance within the organizations. To do this, the average PSM and SIP scores as well as average performance scores are calculated for both survey years in each organization. Thus, the data are aggregated to the organizational level by means of saving the average organizational PSM, SIP, and performance per organization. This reduces the n to 42 organizations. Using a fixed effects panel regression on aggregated data at the organizational level reduces endogeneity issues and common method bias, which includes at least parts of omitted variable and social desirability bias (Verbeek, 2008). Because the data are analyzed over time, this time variation can be used to control for factors that do not change over time, leaving only the variation that does change left to be explained by the included variables.[6]

Looking at recent methodological articles (Favero & Bullock, 2014; Jakobsen & Jensen, 2015), they suggest that panel analysis is a very effective means to limit the common method bias typically associated with cross-sectional studies using self-perceived performance measures, because social desirability bias can be assumed to be relatively stable across time. An aggregated panel analysis at the organizational level is a very conservative test of the PSM–performance relationship and (especially) of the moderating role of SIP. Empirical findings thus suggest that common method bias will normally not produce false positives in interaction models, because common method bias attenuates the interaction effect rather than boosting it when a true interaction effect exists (Jakobsen & Jensen, 2015).

Results

This section first discusses the individual-level analysis followed by the organizational-level panel analysis.

The Relationship Between PSM, SIP, and Performance—Individual Level

In the first analysis, a GLS regression on the pooled cross-sectional data was conducted on whether PSM and SIP are significantly related to individual performance. Table 2 shows the results of the analysis. In Model 1, we see that PSM is significantly and positively related to performance, which supports Hypothesis 1. In Model 2, the interaction between PSM and SIP is added. In Models 3 to 6, various control variables are added. This does not change the relationships between PSM, SIP, and performance, which are still highly significant. This speaks against positive serial autocorrelation being a problem for the strength of our results and conclusions (cf. Note 1).

Likewise, the interaction, PSM × SIP, is significant and positive regardless of the model specification, and this supports Hypothesis 3. The magnitude of the interaction is substantially interesting as illustrated in Figure 2, which shows the estimated associations for three levels of SIP (minimum, average, and maximum). The figure illustrates that PSM and performance are more strongly associated for higher levels of SIP. The estimated performance difference between employees with very low and very high PSM is thus 26 points on a 0 to 100 point performance scale for employees with very high SIP, while the corresponding difference is very close to zero (−1) for employees with very low SIP. Figure 2 also illustrates that if PSM is very low, performance is estimated to be higher for lower levels of SIP. The main effects for PSM and SIP in Table 2 mean that for employees without PSM, the association between the job’s SIP and performance is negative, and for employees who see no potential for a societal impact in their jobs, the association between PSM and performance is very close to zero. Models 5 and 6 show that supervisory position, higher salary, and received training are positively related to higher performance, while there is a borderline significant, weak association between gender and performance (males score a little lower).

Figure 2.

Illustration of estimated individual-level associations between PSM and performance for minimum, average, and maximum levels of SIP (illustrating Model 6 in Table 2).

Note. PSM = public service motivation; SIP = societal impact potential.

The final model explains 11.5% of the within-organizational variance in performance whereas it explains around 3% of the individual performance variance across organizations.

The Relationship Between Average PSM, SIP, and Individual Performance—Over Time

Table 3 shows the results of the organizational-level analysis. All scores from employees are now aggregated to the organizational level, meaning that the average organizational scores on PSM, SIP, and individual performance are used. The sample is reduced to n = 42 on the group level as this is the number of investigated organizations.

Table 3.

Organizational-Level (Panel Regression, Fixed Effects) Analysis.

Dependent variable: Aggregated average performance
	Model 1	Model 2	Model 3	Model 4
Constant	46.08** (12.36)	83.00** (23.89)	81.27** (24.91)	62.11* (25.53)
PSM	0.307[†] (0.166)	−0.276 (0.364)	−0.274 (0.371)	−0.219 (0.358)
SIP	0.0694 (0.0763)	−0.755 (0.467)	−0.809 (0.484)	−0.519 (0.478)
PSM × SIP		0.0127[†] (0.00709)	0.0136[†] (0.00733)	0.0121[†] (0.00698)
% men			−3.022 (4.203)	−5.024 (4.013)
Average age			0.0624 (0.174)	0.133 (0.163)
% supervisor				7.836** (2.631)
No. of employees				0.000200 (0.000213)
n (observations)	84	84	84	80
r2 within	.0869	.156	.170	.364
r2 overall	.00868	.00317	.00535	.00165
r2 between	.0542	.0424	.0422	.00298
sigma_u	1.513	1.700	1.927	3.007
sigma_e	1.344	1.308	1.332	1.208
rho	.559	.628	.677	.861
n (organizations)	42	42	42	40

Note. PSM = public service motivation; SIP = societal impact potential.

p < .1. *p < .05. **p < .01. ***p < .001.

Model 1 shows that also at the organizational level, average PSM is positively related to the average performance over time. This is in line with Hypothesis 2 and adds to confirm the robustness of our individual-level results presented above. Note, however, that the association is only significant at the .1 level (at least partly due to the low n). The effect size is, however, larger—perhaps as a result of controlling for omitted variables. In Model 2, the main effects are negative (similar to individual-level analysis). Given that no organization had an average PSM or SIP of zero, these coefficients are not relevant in themselves. Most importantly, the interaction between PSM and SIP is significant at the .1 level. This does not change when we control for gender, average age, percentage of supervisors, and number of employees in Models 3 and 4. The results thus show that the combination of high PSM and high SIP is positively related to individual performance in an organization over time. Figure 3 illustrates the findings (calculated for Model 4), and it shows that the association between PSM and performance is substantially stronger for higher levels of SIP. It also shows that (within the observed range of average organizational PSM) the estimated aggregated performance is higher for higher values of organizational SIP.

Figure 3.

Illustration of estimated aggregated PSM–performance associations for minimum, average, and maximum levels of SIP (illustrating Model 4 in Table 3).

Note. PSM = public service motivation; SIP = societal impact potential.

Model 4 shows that the percentage of supervisors is highly significant, indicating that the higher the proportion of supervisors in the organization, the higher the average performance score. This final model explains 36.4% of the change in performance from 2010 to 2012 within the organizations (the high percentage reflects the fact that there is less total variation, given the control for time-invariant factors). In sum, we find support for Hypothesis 3 that the SIP moderates the relationship between PSM and performance over time.

Discussion and Conclusion

The findings on both the individual level and the organizational level provide several new insights. First, the panel data analysis on the organizational level provided a robust test of the PSM–performance relationship and the moderating effect of the job’s SIP. Our use of fixed effects panel regression reduces the risks regarding endogeneity issues such as omitted variable or social desirability bias, and this makes us more confident in our results and supports the theoretical argument that having highly public service motivated employees in an organization, who believe that they can actually contribute to society in their jobs, can be considered an organizational resource.

Another key finding is that the job’s SIP moderates the PSM–performance association, indicating more generally that the association between PSM and performance is dependent on the context in which the work is done. Our estimates indicate that for individuals who perceive their jobs as lacking a SIP, PSM is not related to performance. This is highly understandable, given that it can be hard to feel motivated to strive to do good for society without perceiving that the job gives the opportunity to have a societal impact. For employees with high PSM, it might even be impossible to feel successful according to the individual’s internal standards in terms of making a societal difference. Our findings thus concur with and supplement previous findings showing that employees with high PSM are more committed to the organization, put in more effort, and perform better (Bellé, 2013; Brewer, 2008; Brewer & Selden, 2000; Bright, 2007; Kim, 2005; Leisink & Steijn, 2009; Vandenabeele, 2009).

Our findings also suggest that it can be important to study the role of publicness as referring to institutional characteristics that create or limit opportunities to contribute to society through the job in determining the relationship between PSM and performance. Jacobsen, Hvitved, and Andersen (2014) and Moynihan and Pandey (2007) showed how institutional settings and the perceptions of them (such as red tape and command systems) can influence PSM. We add to this knowledge, showing that the SIP of the job matters for the relationship between PSM and performance. Whereas Bright (2007), Gould-Williams et al. (2015), and Kim (2012) found that person–organization fit was relevant for the PSM–performance relationship, we found a specific type of subjective person–job fit to be relevant.

Whether person–job fit or person–organization fit is more important can, however, not be concluded from this study. The findings on the role of PSM and perceived SIP for performance are important because they indicate that organizations benefit more from high levels of PSM if they are able to offer jobs with a high SIP. PSM is thus not a quick-fix instrument to increase performance in all contexts as it may even be negatively related to performance when there is no SIP of the job. This is bad news for managers if the jobs do not have the potential to impact society, but it is good news for managers where this is the case.

There are some limitations to this study. First, on the individual level, we used a cross-sectional design which potentially suffers from common method bias. Still, we find the same patterns when using the average PSM, SIP, and performance on the organizational level over time while controlling for time-invariant factors. Although this organizational-level analysis is a step forward, it would also have been highly useful to have individual panel data. The fact that the relationship was only significant at the p < .1 level is most likely due to the very small sample size (n = 42). Although other studies have used similar strategies (Brewer & Selden, 2000; Kim, 2005), there may be more variation within an organization than between them. On the contrary, this study uses more enhanced techniques such as fixed effects regression which capture (and control for) organizational levels of social desirability bias unless this changed over time (Cameron & Trivedi, 2009; Jakobsen & Jensen, 2015). Each method thus has both drawbacks and advantages.

Second, the explained variance of PSM was limited, but other studies also suggest that PSM explains some, but not all, variation in performance. In the panel regression, which partly controls for this at the organizational level, the explained variance of performance by PSM was much higher. We have also focused on perceptions of individual performance. As such, we have probably not captured the full extent of organizational performance, which is a multidimensional concept where individual self-reported performance can only be expected to capture the dimensions seen as salient by individual employees. Moreover, we have analyzed the relationship in one context. The use of a large representative data set provides robustness to the findings. Still, it would be useful if future studies could replicate this in different countries to see how specific country characteristics are of influence (Kjeldsen & Andersen, 2012) and maybe also go into more detail with the varying degrees of perceived SIP between different types of public service organizations.

The findings have several implications for practice. First, the results indicate that if one is to utilize PSM in terms of performance improvements, good job design and communication are essential. Public organizations rely on the PSM of employees due to fixed, often below-market-standard salaries. It is, however, too simple to say that employees will perform well if they are public service motivated. Organizations can still benefit from the selection, socialization, and retention of employees with PSM, but they also need to pay attention to job design in terms of making the employees see that they can have an impact on society through their jobs. This point has also been emphasized by Hackman and Oldham (1976), who with their job characteristics model of work motivation highlighted that employee experience of task identity and task significance is important for creating a feeling of meaningfulness in the job. None of the organizations did, however, have an average SIP of zero. The lowest score on SIP was 39 on a scale from 0 to 100 and was recorded in a research institute for “Fundamental Research of Matter,” a score well below the middle point. Ministries and security agencies such as police and army divisions, which can be seen as classic government tasks, had the highest scores on SIP.

Some aspects of the institutional context which define the SIP may be hard to change such as the organization’s mission, changes in wider society, and political support (budget and personnel). Others can be influenced by the organization. Internal factors such as leadership capacity (communication, structure, culture) and job design can be important. Through, for instance, transformational leadership styles (Bellé, 2014) and communicating the prosocial impact (Grant, 2008), perceptions of SIP can be enhanced. Other leadership styles such as more distributive and participatory leadership styles in which the leader aims to empower the employees may also interact positively with public service motives of employees (Jakobsen, Kjeldsen, & Pallesen, 2017). Most importantly, through good job design, providing interesting, meaningful jobs, agencies can create optimal circumstances for their public service motivated employees (Hackman & Oldham, 1976; Perry, Mesch, & Paarlberg, 2006).

This study shows that if public organizations want to improve performance, enhancing PSM through selection or socialization can be a useful strategy—if the organization has a high SIP. Furthermore, to utilize PSM in terms of performance improvements, it is important that the job design accounts for the fact that public service motivated employees want opportunities to have a positive impact on society. Finally, we found that PSM and SIP within organizations change over time, which means that managers might be able to influence these factors.

This study, in conclusion, shows that although PSM can contribute to higher performance, managers should be aware that this can only be expected to happen when employees feel their work can have a positive impact on society.