Changes in perceived productivity of software engineers during COVID-19 pandemic: The voice of evidence.

BACKGROUND: The COVID-19 pandemic triggered a natural experiment of an unprecedented scale as companies closed their offices and sent employees to work from home. Many managers were concerned that their engineers would not be able to work effectively from home, or lack the motivation to do so, and that they would lose control and not even notice when things go wrong. As many companies announced their post-COVID permanent remote-work or hybrid home/office policies, the question of what can be expected from software engineers who work from home becomes more and more relevant. AIMS: To understand the nature of home telework we analyze the evidence of perceived changes in productivity comparing office work before the pandemic with the work from home during the pandemic from thirteen empirical surveys of practitioners.
METHOD: We analyzed data from six corporate surveys conducted in four Scandinavian companies combined with the results of seven published surveys studying the perceived changes in productivity in industrial settings. In addition, we sought explanations for the variation in perceived productivity among the engineers from the studied companies through the qualitative analysis of open-ended questions and interviews.
RESULTS: Combined results of 7686 data points suggest that though on average perceived productivity has not changed significantly, there are developers who report being more productive, and developers being less productive when working from home. Positively affected individuals in some surveys form large groups of respondents (up to 50%) and mention benefiting from a better organization of work, increased flexibility and focus. Yet, there are equally large groups of negatively affected respondents (up to 51%) who complain about the challenges related to remote teamwork and collaboration, as well as emotional issues, distractions and poor home office environment and equipment. Finally, positive trends are found in longitudinal surveys, i.e., developers' productivity in the later months of the pandemic show better results than those in the earlier months.
CONCLUSIONS: We conclude that behind the average "no change" lays a large variation of experiences, which means that the work from home might not be for everyone. Yet, a longitudinal analysis of the surveys is encouraging, as it shows that the more pessimistic results might be influenced by the initial experiences of an unprecedented crisis. At the end, we put forward the lessons learned during the pandemic that can inspire the new post-pandemic work policies.

Introduction

As the novel coronavirus SARS-CoV-2 started to spread globally in late 2019 and early 2020, many technology companies closed their offices and sent employees to work from home (WFH), marking the turn of the history in the magnitude of experience and perception of telework. While some warn that quarantine work is not normal working from home due to the experiences of an unprecedented crisis (Ralph et al., 2020), many agree that this event will lead to significant changes in the post-pandemic workplace and is an opportunity to derive the important lessons learned regarding development productivity (Ford et al., 2020, Bao et al., 2020, Miller et al., 2021, Smite et al., 2021b).

One of the first rigorous studies quantifying the changes in developer activity brought by the transition into the work-from-home mode is made available by the GitHub research team (Forsgren, 2020). The analysis of all GitHub projects (open source and corporate) concludes that the developer activity in terms of the number of pushes, pull requests, code reviews and commented issues remained similar or slightly increased compared to the pre-pandemic year. This finding is consonant with numerous other studies that conclude that software companies have nothing to worry about since working from home is per se not a significant challenge for software engineers (Smite et al., 2021b, Russo et al., 2021, Bezerra et al., 2020). In fact, earlier studies included in the state-of-the-art of telework by Fløvik et al. (2021) suggest that most related studies report higher productivity and somewhat higher job satisfaction for those working from home in comparison with those working from the office (Fløvik et al., 2021).

At the same time, there is little knowledge about the long-term effects of WFH, because studies related to pre-pandemic telework included only part-time voluntary teleworkers. However, some researchers focusing on the experiences from the first months of the pandemic already warn about the possible burnout caused by an increased activity during the non-core hours, such as early mornings, late evenings and weekends (DeFilippis et al., 2020, Forsgren, 2020, Nolan et al., 2020, de Mendonça et al., 2020, Miller et al., 2021), which is also known from earlier studies of telework (Fløvik et al., 2021). If developers put more time into achieving the same result, this means that their productivity has decreased. Yet, artifact-based analyses fall short to provide convincing evidence or a valid explanation, since it is difficult to know whether registered activity reflects the true work activity, i.e., whether the time between the first and the last commit or email sent does not contain longer breaks or periods of inactivity.

To complement the quantitative studies referred to above and the studies focusing solely on a short WFH time period, the current study is set out to summarize the evidence from perception-based studies of developer productivity over one year’s time. This research is, thus, driven to address the following research question:

How did the perceived productivity change when software engineers worked from home during the COVID-19 pandemic?

Furthermore, as many companies announce their post-COVID permanent remote-work or hybrid home/office policies, we are motivated to better understand what hinders and enables productivity of software engineers when working from home. Thus, we pose the second research question:

What factors predict changes in perceived productivity?

The remainder of the paper is organized as follows. In the next section we outline the background related to the productivity measurement in software engineering, as well as refer to the studies of pandemic productivity. In Section 3, we describe the thirteen surveys of the changes of perceived productivity during the COVID-19 pandemic included in our study. Section 4 is dedicated to the results, which are discussed in Section 5. Finally, Section 6 concludes our paper with a short summary of the key findings.

Background

Researchers and practitioners have been interested in productivity of software engineers for many decades. Yet, the questions of how to measure or even define developer productivity remain elusive (Forsgren et al., 2021). Studying productivity in software engineering is inseparable from the discussion about the degree to which the productivity metrics reflect the actual productivity, in other words, validity of productivity as a construct (Ralph and Tempero, 2018). Productivity generally refers to the rate of produced output per unit of time (Meyer et al., 2014, de Lemos Meira et al., 2010). Examples of metrics typically used for monitoring productivity in software engineering include a number of tasks per month (Zhou and Mockus, 2010), deployment frequency (Forsgren et al., 2021), lines of code per person–month (Nguyen et al., 2011), time to resolve a modification request (Cataldo et al., 2008), a number of pull requests or a number of builds per engineer per hour (Spataro, 2020), a number of bugs found and fixed, a number of emails sent (Meyer et al., 2014), and function point productivity (Petersen, 2011). However, there is no metric that has not been criticized. For example, measuring productivity with lines of code can be a matter of preference among different developers (Ralph et al., 2020, Ralph and Tempero, 2018) and can depend on the choice of the programming language (Petersen, 2011). Similarly, function point productivity has been criticized because not all development efforts are counted as function points (Petersen, 2011). Such considerations discourage many researchers from studying productivity (Ko, 2019), while others look at developers’ productivity as a complex and nuanced phenomenon (Forsgren et al., 2021), affected by technical aspects (e.g., tools), as much as social aspects (e.g., team organization and work environment) (de Lemos Meira et al., 2010). Thus, it is not surprising that subjective measures of productivity become popular instruments in the productivity research.

Perceived productivity can be viewed as an alternative self-reported indicator which allows to evaluate how productive developers feel. A typical method is asking developers to self-assess productivity on a Likert scale from low (e.g., very unsatisfied with productivity) to high (very satisfied) (Meyer et al., 2014). Just as objective metrics, self-reported productivity represents a threat for the construct validity. Belonging to the toolbox of survey research, assessment of self-reported productivity requires rigorous design and validation of the questionnaires (Molléri et al., 2020). A more advanced and well-established tool for measuring perceived productivity (although not specific to software engineers as a profession) is the WHO’s Health and Work Performance Questionnaire (HPQ)1 .

By the time we write this paper, a number of research studies have reported the experiences with pandemic programming, including survey studies that report practitioners’ perceived changes in productivity when comparing the office times with the WFH period (Ralph et al., 2020, Ford et al., 2020, Russo et al., 2021, Oliveira et al., 2020) (see our detailed description in Section 3.2 and a summary of the papers in Table 2). Additionally, we found three surveys of perceived productivity (Miller et al., 2021, Bezerra et al., 2020, Neto et al., 2020) that are related but do not match our research questions for different reasons, which we summarize in the following.

Table 2

Overview of related surveys from previous studies reporting the changes in perceived productivity in WFH.

	Surveys	N	Time period	Population
7	Ralph et al. (2020)	1940	Mar 27–Apr 29, 2020	Predominantly software developers from 53 countries
8	Ford et al. (2020) 1	1369	Mar 16–20, 2020	Microsoft developers and program managers from the Washington State, US
9	Ford et al. (2020) 2	2078	Apr 22–May 9, 2020	Microsoft developers, program managers and data scientists from US
10	Russo et al. (2021) 1	192	Apr 20–26, 2020	Professionals from the UK, US and Portugal, and 38 other countries, primarily in Europe.
11	Russo et al. (2021) 2	182	May 4–10, 2020
12	Oliveira et al. (2020) 1	184	Apr 2020	Professionals working in software companies in Brazil.
13	Oliveira et al. (2020) 2	229	May 2020
	Total:	6499

The first study by Miller et al. (2021) reports a continuation of the work conducted by Bao et al. (2020) which is a set of surveys conducted in Microsoft. The study was excluded from our analysis because the authors focused on team productivity, while we are interested in individual productivity. The study reports results from the second survey included in Bao et al. (2020) combined with an additional team-productivity survey, ran in July 2020. The new analysis concludes that 20% of respondents report their team productivity being positively affected by WFH, 56% remaining the same, and 23% being negatively affected, followed by a rich insight into various aspects of communication and collaboration that explain the results.

Neto et al. (2020) surveyed 279 professionals from 32 countries and concluded that the overall productivity and task completion time did not decrease during the pandemic period compared to the usual time. We do not include this study in our analysis, because the authors only examine the negative changes or the absence of such, which made it impossible to assess whether respondents have been affected positively (which is important for our analysis).

Finally, Bezerra et al. (2020) assessed the state of practice of WFH in Brazilian software companies in May 2020 on a scale of unsatisfactory, regular, good and excellent, i.e., the current productivity during the pandemic and not on the changes. Hence, this study was excluded from our analysis as well. The authors found that the majority reported good or excellent productivity and that productivity was influenced by external interruptions, adaptation and emotional well-being.

Included surveys and research methodology

In this paper, we offer the results of thirteen surveys reporting the changes in perceived productivity in WFH when compared to the pre-pandemic times, and the key reasons behind these changes. Hereby, we first describe data from six surveys that we collected (secondary data) from four software-intensive companies (see a summary in Table 1), and then seven related surveys from previous studies which we found in the literature with similar measures of perceived productivity (see a summary in Table 2). We conclude the section with the data analysis approach.

Table 1

Overview of the case-based corporate surveys exploring the changes in perceived productivity in WFH (secondary data).

	Case id	N	Response rate	Time period	Population
1	GlobCo	458	75%	Mar 2021	Software engineers from one company site in Sweden
2	NorBank	87	80%	Jun 2020	Software engineers from Norway, representing the whole company
3	SavingsBank 1	268	67%	May 2020	Software engineers from Norway, representing the whole company
4	SavingsBank 2	225	61%	Apr 2021	Software engineers from Norway, representing the whole company
5	NorIT 1	131	75%	May–Jun 2020	Software engineers from Norway, representing one department
6	NorIT 2	18	90%	Jan 2021	Software engineers from Norway, representing another department
	Total:	1187

Empirical studies and the first six surveys

The cases included in this paper are GlobCo, Norbank, SavingsBank, and NorIT — pseudonyms of four companies, the identities of which are preserved for confidentiality reasons. In all cases, we relied on company-internal surveys that were designed and ran independently from our study, which qualifies our actions as secondary analysis of the secondary data (Robson, 2002). In three of four cases we did not have access to the raw survey data. For this reason and due to confidentiality, we are unable to share the data openly.

GlobCo

GlobCo is a large international company headquartered in Northern Europe. GlobCo develops a wide range of software-intensive products and solutions, including generic software products offered to an open market and complex compound systems with customized versions. In this study, we examine one geographic site of GlobCo in Sweden. GlobCo was very early to start preparations for the pandemic, as the company limited the business travel in early 2020 and closed their sites in March. All employees were allowed to borrow office equipment and directed to work from home. During the summer 2020 and episodically throughout the pandemic, employees were allowed to visit the offices for important meetings, strictly limiting the number of simultaneous visits and the number of people in the meetings rooms and offices. However, in practice, many chose to remain working from home full time. In November 2020, the company announced the program for reimbursing home office equipment and furniture, with the intention to improve the WFH conditions.

Survey 1: We analyzed the results from an internal WFH survey of employees in one department at GlobCo. The survey was run in March 2021, and had 458 respondents, representing 75% of employees in that location (which is above the acceptable level (i.e., 60%) (Robson, 2002)). 94% of respondents have been working from home full time. In the survey, respondents were asked to rate their perceived productivity on a 3-point Likert scale (less than normal, as normal, more than normal).

Qualitative data: To complement the survey findings and find explanations for the results, the first and the third author conducted qualitative interviews with 11 representatives from the studied GlobCo location (project leaders, developers, architects/designers, testers, and DevOps specialists). Interviewees were selected by convenience sampling, at the same time aiming at having representatives from different age groups and with different family situations. The interviews were 30–60 min long and focused on the rhythm of a typical day under WFH, reflections on the changes in the daily routines (schedule, tasks, meetings, teamwork, ceremonies, including pair programming), on what works and what does not work in the WFH mode, home office setup, and hopes for the future. Interviews were held in English via MS Teams, video- and audio-recorded and later transcribed.

Norbank

Norbank is a financial services company headquartered in Norway, which operates in the Nordic markets and offers pension, savings, insurance and banking products to both the private and the business market. While the company employs more than 2,000 people in total, this study includes software developers working in software teams from different parts of the company. Following the lockdown in March 2020, a vast majority of Norbank employees moved from working in an open office and applying agile practices and ceremonies (for over 10 years) to working from home. Those with business-critical tasks could use the office, and during the fall those not having a good home office environment could even send a request to return to the office. Further, team members could borrow whatever technical equipment they needed from the office, and each person received 500€ for buying equipment. Finally, management arranged frequent meetings to make sure everyone was informed regarding the current situation.

Survey 2: We analyzed the results of an all-company internal WFH survey run in June 2020. After removing non-developers, 87 responses were included in this study (80% response rate among developers, which is well above the acceptable level (Robson, 2002)). In the survey, respondents were asked to rate how their productivity was affected by the COVID-19 and the WFH on a 5-point Likert scale ranging from significantly decreased (1) to significantly increased (5).

Qualitative data: In the survey, respondents were also asked to describe the main challenges and benefits associated with WFH in a free-text form. We received 60 responses, which were used for qualitative analysis. Further, the third and the sixth author presented the results back to the company to capture their opinions and understanding of the results.

SavingsBank

SavingsBank is a Norwegian software development company owned by an alliance of banks. Before the pandemic, 24 development teams were situated in an open office, and most use a Kanban variant with elements of Scrum as their way of working. In February 2020, one team member contracted the virus and the whole team started working from home. In March 2020, the rest of the employees went from predominantly on-site work to fully distributed work from home as the company sites closed. Everyone was offered to take the equipment they needed from the office. Further, everyone could get a reimbursement on their internet cost (up to 60€ per month), and those who needed a new chair, or a table could get a 300€ support. The company frequently communicated strategy updates, good team practices and how to work from home. During the course of the pandemic, employees were episodically allowed to be at the office when the spread of the virus decreased, and sent home again, when the restrictions worsened. The number of simultaneous employees at the office and in meeting rooms, and the distance between employees were controlled.

Surveys 3–4: In SavingsBank, we based our findings on the results from two waves of an internal survey of employees with 268 respondents (response rate 67%, which is acceptable (Robson, 2002)) in the first wave (completed in May 2020) and 225 respondents (response rate 61%, which is acceptable (Robson, 2002)) in the second wave (April 2021). In both surveys, respondents were asked to quantify their productivity in comparison with the times when they were working in the office by judging their output: “I have produced/delivered…” with the response options on the 5-point Likert scale (much more from home than from the office, more from home than from the office, equally much from home as from the office, slightly less from home as from the office, significantly less from home than from the office). The survey was designed and collected by the SavingsBank and analyzed by the three first authors.

Qualitative data: To complement the survey findings and find explanations for the results, the second and the third authors analyzed the free text responses (150 comments in the first wave and 151 comments in the second wave). We also interviewed 7 company representatives. The interviews were transcribed and used in the qualitative analysis. Finally, we presented the results back to the company to elicit their opinions.

NorIT

NorITdevelops products consisting of hardware and software components. The company combines traditional project management with agile development methods. Before the pandemic, the teams were either situated in an open office, in own offices or shared the office with one or two teammates. Notably, 8% worked regularly from home already prior to the pandemic due to a long commute time (2–3 hours/day). In March 2020, 91% went into the work from home mode as the company sites closed. Many of those that needed access to a restricted network had to work from the office. Everyone was offered to take the equipment they needed from the office. During the course of the pandemic, employees were allowed back to the NorIT office when the spread of the virus decreased. This allowed the company to start testing how to organize a hybrid workplace for the future.

Surveys 5–6: In NorIT, we based our findings on two surveys in two different departments conducted at two different points of time. The first survey was ran in May 2020 and gathered responses from 395 respondents working on one specific product, among which 131 were working on software development (75% response rate among software developers, which is above the acceptable level (Robson, 2002)). The second survey was performed in January 2021 and gathered 45 responses from developers working on another product, with 18 being involved in the software development (90% response rate among the software developers, which is well above the acceptable level (Robson, 2002)). Most developers worked from home all days during the pandemic. Perceived productivity was assessed through an item used in both surveys: “When I work from home I am:…” with the response options: “more productive than at the office, have the same productivity home and at the office, less productive than at the office”.

Qualitative data: As in other corporate surveys from Norwegian cases, respondents from NorIT were asked to describe the main challenges and benefits associated with WFH in a free-text form. We received 71 comments in the first NorIT survey, and 13 comments in the second survey, which were all used in the qualitative analysis.

Seven surveys from related literature

To answer our research questions, we supplemented the perceived productivity data from the empirical cases with the data and/or results available from earlier related studies that compare the pre-pandemic office productivity of software engineers with that during the pandemic (Ralph et al., 2020, Ford et al., 2020, Russo et al., 2021, Oliveira et al., 2020). In the following, we describe the survey instruments employed in each of the included cases.

Ralph et al.

Ralph et al. (2020) performed an extensive study of the pandemic impact on programming, including productivity in the early months of WFH. The authors also assessed how perceived productivity is affected by disaster preparedness, change in well-being and home office ergonomics. They conclude that perceived productivity has declined (admitting a marginal effect size) as a result of negatively affected wellbeing and that organizations need to accept that expecting normal productivity under the crisis circumstances is unrealistic.

Survey 7: Ralph et al. run their survey in March–April 2020 gathering a large number of responses from 2078 professionals from 53 countries (a subset of those who responded to the perceived productivity question). The analysis included respondents who switched from working in the office to working from home. With respect to the profession and affiliation, the respondents are very diverse, representing organizations ranging from 0–9 employees to more than 100,000, with 80% being software developers or equivalent and 20% being other kinds of software professionals (e.g., project manager, quality assurance analyst). To elicit perceived productivity, the authors applied the WHO’s Health and Work Performance Questionnaire (HPQ) and report average total scores for both before and after the transition to WFH. The participants of a single survey were asked to evaluate their own productivity for the period before the switch (HPQB) and the period since the switch (HPQS), where each period was assessed through 9 items. The total perceived productivity scores were calculated by deriving a total of several items from the HPQ separately for before and since the pandemic. The items were standardized, and two items were omitted from the traditional HPQ to increase the convergent and discriminant validity of the factor analysis. Furthermore, the authors assessed disaster preparedness using Yong et al.’s disaster preparedness scale (Yong et al., 2017), emotional well-being using WHO’s five-item wellbeing index and ergonomics through a self-created six-item 6-point Likert scale concerning distractions, noise, lighting, temperature, chair comfort and overall ergonomics.

Ford et al.

Ford et al. (2020) conducted a two-wave study with the first wave including Microsoft engineers from one location in the US, and the second — Microsoft engineers from all locations in the US two surveys in Microsoft. Both surveys indicated that the productivity increased among some participants and stayed the same or decreased among the others, which led the authors to conclude that the productivity during the COVID-19 pandemic was dichotomous.

Surveys 8–9: Data from the first survey was collected among 1369 developers and program managers in March 2020, whereas the second wave was run in April–May 2020 among 2265 developers, program managers and data scientists. In both surveys the respondents were asked, “Compared to working from home, how has your productivity changed?” and gathered responses on a 4-point Likert scale: Less productive, About the same, More productive, and Significantly more productive. The results were summarized by waves and compared across the waves. Additionally, Survey 9 elicited experiences with the challenges and benefits when working from home, as well as their impact (“What work-related challenges have you experienced working from home and how impactful are these challenges?”, “What work-related challenges have you experienced working from home and how impactful are these challenges?”).

Qualitative data: Survey 8 collected qualitative insights to better understand the change in perceived productivity. The participants were asked to follow-up on their response on perceived productivity: “Please share details about your answer to the previous question on how your productivity has changed”. The answers were used to design a list of challenges/benefits that the participants could choose from when answering the matching questions in Survey 9 (see above).

Russo et al.

Russo et al. (2021) performed a two-wave longitudinal study with a diverse group of professionals, diving into the impact of over 50 psychological, social, situational, and physiological factors and their ability to predict the variance in well-being and productivity. The study concludes with a few associations between the studied factors and perceived productivity and puts forward actionable recommendations. In particular, the authors emphasize boredom as an important factor associated with productivity, which was measured using the 8-item from the Boredom Proneness Scale (Farmer and Sundberg, 1986), including items like “It is easy for me to concentrate on my activities” and “Many things I have to do are repetitive and monotonous” assessed on a 4-point Likert scale. Another important factor found by Russo et al. was distraction at home, which was measured with two items: “I am often distracted from my work (e.g., noisy neighbors, children who need my attention)” and “I am able to focus on my work for longer time periods” (recoded) with responses given on a 5-point scale. Regarding the changes in productivity, the authors conclude that WFH does not per se present a challenge for either organizations or developers.

Survey 10–11: In their survey, Russo et al. conducted two wave data collection from 192 professionals in April and May 2020. They asked the respondents to rate their productivity by comparing it to their own normal. The change in productivity ( Productivity) was operationalized through three variables: P1 (“How many hours have you been working approximately in the past week?”, P2 (“How many hours were you expecting to work over the past week assuming there would be no global pandemic and lockdown?” and P3 “If you rate your productivity (i.e., outcome) per hour, has it been more or less over the past week compared to a normal week?”. The total Productivity was calculated for each wave using the formula (P1/P2) ((P3 ＋ 100)/100), where the values between 0 and .99 would reflect that people were less productive than normal, and values above 1 would indicate that they were more productive than normal.

Oliveira et al.

Oliveira et al. (2020) gathered data from two online surveys of Brazilian professionals. The authors found that perceived productivity in WFH when comparing with the office times has increased. Another important finding made regarding the changes in perceived productivity during the pandemic was that the number of positively affected respondents grew from 40% in the first wave to 60% in the second wave.

Survey 12–13: The two surveys were run in April and May 2020 and gathered responses from a diverse set of professionals, 75% being software developers or equivalent. Most of respondents work in companies with more than 100 employees (63%), others work in companies with 10–50 (18%), 51–100 employees (9%) and less than 10 employees (9%). All respondents work 100% remotely, although they started remote working at different points of time. Perceived productivity data was gathered by inquiring, “Do you consider that your productivity in this period: Continued the same, Increased, and Decreased”. Interestingly, the order of the response options in this study is different from the other studies that offered the response options from negative to neutral and then to positive options. Furthermore, the authors have made stratified analysis on different groups of respondents to understand if any of the groups is particularly affected.

Data analysis approach

To answer our first research question (RQ1: How did the perceived productivity change when software engineers worked from home during the COVID-19 pandemic?), we summarized the results of thirteen surveys after standardizing the scales across all surveys (see Fig. 1). In the previous section, we described the different scales used in each study. Most of the studies used a Likert scale of three, four or five points. Two studies (Ralph et al., 2020, Russo et al., 2021) used continuous scales, which we have standardized according to the original operationalization. Since the actual distributions of the perceived productivity responses are not disclosed in either of the papers, we replicated the calculations of the productivity scores in these studies using the publicly available replication packages to estimate the number of participants with scores indicating decreased, same or increased productivity after the switch to WFH (frequency distributions). See the mapping of the variables in Table 3. Proceeding in this way allowed us to transform the data in scales comparable to the Likert scales used by the majority of the studies. The frequency distributions from the study by Ralph et al. (2020) were estimated by comparing the total scores before the switch (coded as HPQB in the survey) with those since the switch (coded as HPQS in the survey). Similarly, we estimated the frequency distributions from Russo et al. (2021) by comparing the total Productivity scores to 1 (according to the authors’ guidelines). The calculations lead to reduction of the data points when division with zero values occurred in the process. In Table 3, we provide a summary of the translation of the scales.

Fig. 1

Perceived productivity distributions in the empirical cases and related survey studies sorted chronologically. Note: due to the differences in the times considered in the survey, survey instruments and samples the results cannot be used for direct comparison.

Table 3

Translation of the different scales of the perceived productivity across studies.

	Survey	Perceived productivity question	Decreased	Same	Increased
1	GlobCo	I am as productive/effective as I would like during the current situation. Choose one option.	Less than normal	As normal	More than normal
2	Norbank	The productivity of my work has been…	Significantly decreased; Decreased	Same as before COVID-19	Increased; Significantly increased
3–4	SavingsBank	Quantify your productivity: I have produced/delivered…	Slightly less from home as from the office; Significantly less from home than from the office	Equally much from home as from the office	More from home than from the office; Much more from home than from the office
5–6	NorIT	When I work from home I am:…	Less productive than at the office	As productive at home as at the office	More productive than at the office
7	Ralph et al.	Multiple questions from the WHO’s Health and Work Performance Questionnaire	HPQB > HPQS	HPQB = HPQS	HPQB < HPQS
8–9	Ford et al.	Compared to working from home, how has your productivity changed?	Significantly less productive; Less productive	About the same	More productive; Significantly more productive
10–11	Russo et al.	P1: How many hours have you been working approximately in the past week?”, P2: “How many hours were you expecting to work over the past week assuming there would be no global pandemic and lockdown?”, P3: If you rate your productivity (i.e., outcome) per hour, has it been more or less over the past week compared to a normal week?	Δ Productivity <1, where Δ Productivity = (P1/P2) × ((P3 ＋ 100)/100)	Δ Productivity = 1, where Δ Productivity = (P1/P2) × ((P3 ＋ 100)/100)	Δ Productivity > 1, where Δ Productivity = (P1/P2) × ((P3 ＋ 100)/100)
12–13	Oliveira et al.	Do you consider that your productivity in this period:…	Decreased	Continued the same	Increased

To address our second research question (RQ2: What factors predict changes in perceived productivity?), we synthesized the findings from multiple data sources. First, we extracted the reasons for lowered and increased productivity as reported in the related studies, included in our analysis (Ralph et al., 2020, Ford et al., 2020, Russo et al., 2021, Oliveira et al., 2020). Secondly, we coded the free-text responses in the available case surveys that explained the changes in perceived productivity, which we labeled positive and negative changes with respect to the WFH situation compared to the pre-COVID one. This included the analysis of 445 free-text comments from NorBank, SavingsBank 1 and 2 and NorIT 1 and 2. Similar coding of the reasons for lowered and increased productivity was performed for the GlobCo and SavingsBank cases using the transcripts of interviews (eleven interviews in GlobCo and seven interviews in SavingsBank). The following codes exemplifies our coding process: “More difficult to take the quick informal conversations with colleagues - especially across teams and departments” was coded as “Greater difficulty communicating with peers”; “Better concentration - no disturbances” coded as “Fewer interruptions from colleagues”; “Less stress around delivery and picking up my kids in the kindergarten” was coded as “Better work-life-balance” and “Less stress”. Finally, we grouped the emerging codes into four higher level categories, including Emotional issues factors, Organization of work and time, Home office environment and equipment, Team and work processes and collaboration (see Table 4, Table 5 for the summary of all resulting codes and categories). We used the coded excerpts from the interviews and survey responses as quotes to illustrate the factors described in Section 4.2.

Table 4

Barriers to perceived productivity.

Barriers	Studies	GlobCo	NorBank	SavingsBank	NorIT1
Emotional factors
Blurred work-life balance	(Ford et al., 2020)	X	X	X	X
Lack of positive peer pressure/stimulus			X	X	X
Isolation		X	X	X	X
Boredom	(Russo et al., 2021)	X	X	X	X
Disaster preparednessa	(Ralph et al., 2020)	X
Emotional issues, pandemic-related fearsa	(Ralph et al., 2020, Oliveira et al., 2020)		X	X	X
Organization of work and time
Distractions, difficulties to focus	(Russo et al., 2021, Oliveira et al., 2020)		X	X	X
Interruptions from family members	(Ford et al., 2020, Oliveira et al., 2020)	X	X	X	X
Increased number of meetings		X	X	X	X
More exhausting meetings		X	X	X	X
Home office environment and equipment
Poor home office ergonomics	(Ralph et al., 2020)		X	X	X
Poor workspace equipment	(Ford et al., 2020, Oliveira et al., 2020)		X	X	X
Connectivity problems	(Ford et al., 2020, Oliveira et al., 2020)	X			X
Unavailability of all resources and tools	(Oliveira et al., 2020)		X	X	X
Teamwork and collaboration
Greater difficulty communicating with peers	(Ford et al., 2020, Oliveira et al., 2020)	X	X
Socialization decrease in the team	(Oliveira et al., 2020)	X	X	X	X
Not matching working hours in the team
Poor awareness of what is going on		X	X	X	X
Greater difficulty to run brainstorming sessions		X	X	X	X
Greater difficulty to run problem solving sess.		X	X	X	X
Loosening social ties across the organization		X	X	X	X
Greater difficulty to grow a contact network		X	X	X	X

Barriers directly related to the pandemic.

Table 5

Drivers of perceived productivity.

Drivers	Studies	GlobCo	NorBank	SavingsBank	NorIT1
Emotional factors
Better work-life balance	(Ford et al., 2020)	X	X	X	X
Less stress	(Oliveira et al., 2020)		X	X	X
Organization of work and time
Better focus time	(Ford et al., 2020)	X	X	X	X
Fewer interruptions from colleagues	(Ford et al., 2020, Oliveira et al., 2020)	X	X	X	X
Shorter meetings			X	X	X
More efficient meetings	(Ford et al., 2020)	X	X	X	X
Increased flexibility for planning the work hours	(Ford et al., 2020)	X	X	X	X
Less time spent on commuting	(Oliveira et al., 2020)	X	X	X	X
Less time spent on communication with managers			X	X	X
Home office environment and equipment
Better/more comfortable work environment	(Ford et al., 2020, Oliveira et al., 2020)	X	X	X	X
Teamwork and collaboration
Increased accessibility of peers, especially in teams distributed before the pandemic		X	X	X	X

Validity of the results

Our results, namely the changes in perceived productivity and the list of explaining factors, are subjects to validity threats. In the following, we describe the factors that impact the reliability and generalizability of our results.

Validity of the survey results: Our study is descriptive in nature and the purpose of our summary is to illustrate the variability in the results of individual surveys (Fig. 1). The studies included in our analysis are performed at different points in times using different survey instruments based on samples drawn from different populations and can therefore not be used for direct comparison. The reliability of the results of each individual survey, in their turn, highly depends on the survey instrument. The fact that we used data collected through both Likert and non-Likert scale instruments, is therefore one of the validity threats in our study.

Belonging to the toolbox of survey research, assessment of self-reported productivity requires rigorous design and validation of the questionnaires (Molléri et al., 2020). We thus start by bringing forward the differences in operationalizing the survey instruments to study the perceived productivity of software engineers as a potential validity threat in our study. All surveys assessing perceived productivity with a single Likert-scale item, as in Ford et al., 2020, Oliveira et al., 2020 and all our own cases report the trend that productivity did not change or improved. Yet, approaching perceived productivity measurement with the Likert-scale approach is potentially less reliable comparing to applying more complex measures such as the ones in Ralph et al., 2020, Russo et al., 2021. The results from Ralph et al. (2020) and Russo et al. (2021) are evidently most distinct (see Fig. 1) with half of the respondents reporting being negatively impacted. Ralph et al. (2020) used the modified WHO’s Health at Work Performance Questionnaire that computes a perceived productivity score based on a series of questions. Interestingly, the use of the traditional version of the WHO instrument (without any further changes) has been regarded as unfit for the purpose by Russo et al. (2021) who state that knowing other workers productivity (which is used for comparison with own productivity) might be problematic when working remotely. In their turn, Russo et al. (2021) have measured productivity relative to the expected productivity by contrasting productivity in the past week with the participant’s expected productivity level before the lockdown with response options on a bipolar slider measure. The authors reported the test-retest reliability of their instrument to be good. Although the results are not much different from those in the studies using the 3-, 4- or 5-point Likert scales, the mentioned studies based on a more advanced measurement report a much lower number of respondents reporting the same perception of productivity (i.e., no changes) than found in the studies using a Likert-scale. This is understandable, because the distance between “the same” and “decreased” or “increased” on a Likert-scale might be perceived larger than the distance between 1 (the same) and 1,05 or 0.95.

The validity of the self-assessed productivity by responding to a Likert-scale item has been questioned before also by pointing out that the respondents, when asked to rate their productivity from very unsatisfied to very satisfied, most participants will typically report good (satisfied) or neutral productivity (Meyer et al., 2014). Likert-scale surveys included in our analysis might suffer from this bias, since most of them demonstrate bias towards the neutral and positive responses, except for Ford et al. (2020) who found that a larger group of 38% reported being negatively affected in their second survey.

Validity of the predictors of productivity:Table 4, Table 5 list the factors predicting changes in perceived productivity based on the qualitative analysis of the interviews, survey comments from the industrial cases and the articles reporting the related surveys. The qualitative analysis results could be subject to researcher bias. To mitigate this threat, several researchers have been involved in each case. First, two researchers were involved in each interview. Second, two researchers were involved in the data analysis — one performed the coding, and the other validated the emerging list of factors from the case. Finally, we do not claim that the resulting list of factors predicting changes in perceived productivity is exhaustive. While the factor categories identified in the cases of GlobCo, NorBank, SavingsBank and NorIT have been saturated and confirmed by all four cases (see the factors unique to our cases in Table 4, Table 5), adding new company cases with differing characteristics might lead to new factors. However, we believe that the factors are representative for at least NorBank, SavingsBank and NorIT, in which we have analyzed large samples of responses”.

Generalizability of the findings: One important question when it comes to the validity of the research results is whether they are applicable in, i.e., generalizable to other contexts and situations. In our researcher, this concerns the ability to predict the perceived productivity of software engineers working from home in a given company based on our findings. Although the cumulative number of data points in our research is high, the survey responses show a large variation of experiences and the diverse nature of the included surveys does not allow for statistical generalizability, leaving us with a possibility to only perform analytical generalizability. The list of the factors (in Table 4, Table 5) can help predicting the changes in perceived productivity. However, our findings are biased by the time when the surveys were executed, by the locations of the survey respondents and by the pandemic situations/conditions in the given time and location. With respect to time, our findings are likely to be biased towards the experiences from the first three months of the pandemic. With respect to geographic locations, the surveys in our study include both single-company single-region surveys and surveys from diverse multi-country samples (41 and 53 country representatives). This is likely to increase the generalizability of our results. However, to predict the changes in perceived productivity in a particular geographic location, one needs to consider the particular lockdown measures and that people from different cultural environments might differently react to the crisis, to sharing of household and parenting duties, to connecting to the internet from home or to working in solitude. When comparing the trends in perceived productivity, we notice the datasets that we collected from the Scandinavian companies (from Norway and Sweden) appear to report the most positive experiences comparing to studies performed in the USA and Brazil. This could be attributed to the organizational culture of trust, autonomy and self-management routed in extensive experiences with agile ways of working (Smite et al., 2021b), as opposed to more traditional management of control in USA and Brazil, or the relatively relaxed lockdown measures and early reopening of the society. However, more research is needed to determine the true reasons.

Results

Changes in perceived productivity (RQ1)

Fig. 1 summarizes the evidence from the thirteen surveys on the state of the changes in perceived productivity of software engineers while working from home during the COVID-19 pandemic. The data behind the summary comes from the total of 7686 data points from the six corporate surveys ran in our case companies (1187 respondents in total, min 18 in NorIT 2 and max 458 in GlobCo) and the seven related survey studies (Ralph et al., 2020, Ford et al., 2020, Russo et al., 2021, Oliveira et al., 2020) (6499 respondents in total, min 182 in the second wave of Russo et al. (2021), and max 2265 in the second wave of Ford et al. (2020)). The main findings based on the analysis of the data are twofold: all surveys are dichotomous and latter surveys report more optimistic results than the surveys conducted in the early months of the pandemic.

Dichotomous trends: Among the 13 surveys we find those reporting half of respondents negatively affected (Ralph et al., 2020, Russo et al., 2021) and surveys that report predominantly no change or positive affect of WFH on perceived productivity (Ford et al., 2020, Oliveira et al., 2020 and all our company surveys).

No changes in productivity: The results show that relatively many employees in ten out of 13 datasets reported no changes (38%–52%).

Negative changes in productivity: The groups of negatively affected developers range from 11% in the second wave of NorIT to 51% in the study by Ralph et al. (2020). The studies reporting the most pessimistic results include the surveys by Russo et al. (2021) and Ralph et al. (2020), whose data indicates that around half of the respondents experience a decline in perceived productivity (50% in the first wave and 46% in the second wave in Russo et al. (2021) and 51% in Ralph et al. (2020)). The amount of negatively affected respondents in all six corporate case surveys was not high. Roughly every fifth respondent in SavingsBank 1 (21%), NorBank (19%) and GlobCo (17%) as well as roughly every tenth employee in NorIT 1 (12%), NorIT 2 (11%) and SavingsBank 2 (12%) reported that their productivity was lower than normal.

Positive changes in productivity: Considerably many employees have reported an increase in productivity (ranging between 24% in the first wave of Oliveira et al. (2020) to 50% in the second wave of SavingsBank). Further, several studies have performed two-wave analysis of the changes in perceived productivity (Ford et al., 2020, Russo et al., 2021, Oliveira et al., 2020), and our two cases — SavingsBank and NorIT. In all longitudinal studies, the second wave responses were more positive than those in the first wave. The positive changes over the course of the pandemic are also consonant with our findings from studying remote pair programming during WFH, in which we found that pairing from almost an extinct practice in the first months of the work from home, became increasingly practiced in early 2021 (Smite et al., 2021a).

Evidently, the corporate case company surveys emerge as a block of studies with a more optimistic results, while the variation in responses across the published surveys is larger and they do not report the same positive trend. But the results might be also attributed to the time of inquiry.

Changes over time: To see the perceived productivity trends over time, we sorted the surveys in Fig. 1 chronologically. The time when each survey was executed is marked on the timeline in the right side of the figure. Evidently, time seems to be an important predictor, as the surveys at the bottom of the page (later months of the pandemic) tend to be more positive than those on the top (early months of the pandemic). We also find that subsequent runs of the surveys in all cases report improved perceptions of productivity, which is perhaps the most interesting finding. The positive changes over time were also emphasized by our interviewees and in the free-text survey responses. These changes varied from those relating to the changing lockdown measures to those relating to the adjustments that people and organizations have made over time. For example, in Norway schools reopened after the strict spring lockdown in 2020 and the experiences in the first few months were drastically different from the later ones. Significant improvements have been also made by many in the home office equipment. As someone from GlobCo explained the effect of the reimbursement bonus introduced in the beginning of October 2020: “Earlier I was sitting in the kitchen and, you know, a kitchen table is not the best work area for sure. That has improved”. Similarly, the distributed collaboration tools and familiarity with those were said to considerably improve few months into the working from home mode. In fact, in some companies like SavingsBank approval of the new tools took a while. As an interviewee from SavingsBank explained: “We have increased our competence when it comes to digital collaboration, getting new tools also improved my productivity”. Participants from NorBank also mentioned the exceptional change in the meeting frequency in the early months of the pandemic, which has returned to the norm after some time, as someone explained: “After a couple of weeks the meeting frequency increased significantly, but it calmed down after a while”. Many interviewees also commented on the way teams have experimented and adjusted their ways of working, someone even referred to this painful period as a “rollercoaster”. Many interviewees from GlobCo and SavingsBank mentioned improving their ways of working, restructuring the daily routines and ceremonies, and improving the meeting culture both in terms of communication and preparations. As two representatives from GlobCo explained: “We set up some action points to improve what people did not like about the current situation” and “That’s all been an ongoing conversation ever since March – […] how we collaborate to make it better. I would say we talk about that a lot. All the time”. All these adjustments might explain why perceptions of productivity in the later months tends to be more optimistic than in the early months of the pandemic, especially when comparing the number of respondents reporting negative changes.

Factors affecting perceived productivity (RQ2)

Table 4 and Table 5 summarize the factors that predicted the changes in perceived productivity. The analysis is based on the associations reported in related studies and our own qualitative analysis of the case survey free-text responses that explain the changes in perceived productivity.

Reasons for productivity decrease: The authors of the studies showing the most pessimistic productivity change do not report the perceived productivity but study the variables that explain the variance in well-being and productivity. Russo et al. suggest that stress predicted negative well-being, while boredom and distractions predicted negative productivity (Russo et al., 2021), while Ralph et al. found that the pandemic has had a negative effect on developers’ wellbeing and productivity due to disaster preparedness, fear related to the pandemic and home office ergonomics (Ralph et al., 2020). Another interesting aspect of these studies is that they both operationalize the perceived productivity measure differently than the other studies. The values in both survey instruments seem to be biased towards either decline or increase and are less sensitive to those whose productivity is about the same. The explanations for why the productivity decreased in the other studies include emotional issues related to COVID-19 (Oliveira et al., 2020), difficulties to focus (Oliveira et al., 2020), interruptions from family members (Ford et al., 2020, Oliveira et al., 2020), connectivity problems (Ford et al., 2020, Oliveira et al., 2020) and poor workspace equipment (Ford et al., 2020, Oliveira et al., 2020), not matching working hours in the team (Oliveira et al., 2020), unavailability of all resources (Oliveira et al., 2020) and a greater difficulty communicating with people (Ford et al., 2020, Oliveira et al., 2020).

Similarly, respondents from the case companies shared their thoughts about the feeling of isolation, boredom and fears related to the pandemic. Work from home was said to fail to provide the so needed energy, as an engineer from GlobCo explained: “It’s a bit boring to be home … It’s a different environment, of course, when you’re in the office and some of the noises,.. it’s like energy – in the way people laugh in the corridor and offices next door and so on. That gives some energy, I guess, as well.” Further, we learned that stimulus and motivation to work were also negatively impacted by the blurred work-life balance and the lack of positive peer pressure when working from home. As another engineer from GlobCo explained: “I’m working by myself now in silence here. So it’s like, … you don’t get the same stimulus as you usually would, and I think that’s not ideal. I think that [positive peer pressure] helps a lot.” With respect to work and time organization, we found that working from home is associated with distractions and interruptions from family members, especially in families with children on home schooling. In NorBank, participants also complained about increased workload combined with lower efficiency. A developer commented: ”Everything is more difficult. Must work more, but complete less, but in practice same requirements, as before”. Increased number of meetings and worse understanding of what is going on in the team were also reported by those who complained about the decreased productivity. As someone from SavingsBank reported –“With so many team meetings, you do not have time to produce anything”. For others, it was not about the number of meetings as much as the exhaustion from running meetings virtually. Many respondents from Norway and interviewees from GlobCo revealed that employees have been challenged by technical problems, such as connectivity and bandwidth, especially in the beginning of the WFH. Negatively affected respondents from SavingsBank and NorIT further complained about a lack of information and awareness of what is going on, and a lack of socialization with colleagues. Interestingly, as much as engineers enjoyed the focused time and the lack of interruptions from the peers, we received numerous complaints about the negative impact of not overhearing the office conversations. Those suffering the most were teams that were used to highly collaborative work style. These teams required considerable time for adjustment. As a manager from GlobCo explained, “I think that now the teams have adapted to this way of working. OK… now I feel that it’s not a blocker for them. […] that it works pretty good for many of the teams”. Further, running sessions with open content, such as brainstorming sessions and problem-solving sessions, was mentioned to be more challenging and to require more time due to the lack of accustomed whiteboards, possibility to spontaneously connect to the needed people, and requiring considerably more time to prepare for. Isolation from colleagues has also been reported to loosen the ties with colleagues beyond the teammates, which means that the information is not circulating in the same way as it did in the office. Finally, an emerging vulnerable group in our study is recently onboarded individuals. As an engineer from SavingsBank explains that he is significantly less productive: “Because I cannot observe what others do and have not learned any tasks I can do independently”, and another reports, “[I am] brand new to the job and need a lot of follow-ups”. Similarly, a recent hire at GlobCo explains that getting help remotely requires more time: “There’s just a slight barrier compared to just asking a question out in the room. Then you have to maybe ping several people and it takes longer time to get an answer as well, compared to just when you’re sitting in the same room with someone.”

Reasons for productivity increase: The qualitative explanations for why perceived productivity increases are similar across our sources. Many respondents claim to have less stress (Oliveira et al., 2020), less time to commute (Ford et al., 2020, Oliveira et al., 2020), better focus time (Ford et al., 2020), fewer interruptions (Ford et al., 2020, Oliveira et al., 2020), more time to complete work (Ford et al., 2020), more efficient meetings (Ford et al., 2020), better/more comfortable work environment (Ford et al., 2020, Oliveira et al., 2020), better work life balance (Ford et al., 2020). Many people we interviewed were surprised to deliver good results despite the sudden shift from the office to the WFH setup. A developer from NorBank expressed: ”Didn’t expect to function so well at the home office. The team functioned from day one, and we delivered good results”. The benefits reported in our case companies include an increased ability to focus, fewer distractions, increased flexibility to organize ones work hours, less time spent on commuting, as well as more efficient and shorter meetings mentioned by many respondents in all companies. A participant from NorIT commented: ”I have more free time saving about two hours driving time to/from work”. Another participant from NorIT was also satisfied with the efficiency and availability of the colleagues: ”I think I work more effectively. Use less time between the meetings. It seems that those I depend on for completing the job, became more available”. In all our Norwegian case companies many have moved to having 25 min or 50 min long meetings instead of 30 min and 60 min. People who are more productive also reported having more control over their schedule. As one from SavingsBank wrote: “There are two things that decrease efficiency [at the work office]: a lot of noise in the open landscape, and frequent interruptions from people. At the home office, I can largely decide when I will be available”. The reopening of schools, i.e., kids not being around was also frequently mentioned as a prerequisite for being efficient when working from home. Finally, respondents from Norway reported spending less time communicating with managers when comparing with the office times, when managers could closely supervise members of their teams.

Interesting in our study was that even people who reported increased productivity were still dissatisfied with other aspects of working from home (e.g., having distractions from kids at home or a lack of contact with colleagues). This highlights that high productivity in the home-office does not always coincide with the satisfaction with WFH.

Respondents reporting no change: Interestingly, we found that the drivers and barriers to productivity while working from home can have a canceling effect as a respondent from SavingsBank noted: “It depends on what task you are doing. Some tasks are easier to do because you are less interrupted, but some take longer time if you are solving complex problems with others”. As a result, there were groups of engineers who perceived their productivity to be comparable with the pre-pandemic productivity in the office.

Discussion

In this paper, we have presented our findings from summarizing six corporate surveys in four case companies and seven surveys reported by other authors diving into the changes of perceived productivity when moving from the office work to work from home during the COVID-19 pandemic. Combining the results of 7686 data points we learned that there are developers who report being more productive, developers being less productive and those maintaining their usual productivity when working from home. The decreased productivity when moving from collocated to fully distributed setup is not that surprising. Distributed work is infamous for being considerably less productive than collocated work due to impaired teamwork (Herbsleb and Mockus, 2003). What is in fact surprising, is that some WFH developers perceive being more productive. Understanding the latter is therefore important for the future of telework. Furthermore, we evidenced positive trends in the later months of the pandemic with regards to the fewer complains of decreased productivity, supported by the reports of numerous adjustments in the way remote work is carried out from home. In this section, we discuss the key implications of our findings for the future and important research topics as the companies transit to the post-COVID permanent remote-work or hybrid home/office policies. We then discuss the validity of our findings and try to understand why some surveys convey a more pessimistic view of the WFH while others convey a more optimistic view.

Expectations for the post-pandemic future

In Table 4, Table 5, we summarized the factors that predicted or explained the changes in perceived productivity in the included surveys as productivity barriers and drivers. These include factors grouped into the categories of Emotional issues, Organization of work and time, Home office environment and equipment, Teamwork and work processes, and Commuting to work.

The first evident reflection is that the number of barriers (these are 22, see Table 4) is twice as high as the number of drivers (these are 11, see Table 5). Yet, combining these findings with the overall perceived productivity changes from the surveys suggests that the impact of the drivers might be much more profound that the listed challenges. Alternatively, items on both sides might have a canceling effect, in case they are experienced by the same people.

Deeper analysis shows that productivity barriers mainly concern the emotional issues and team and work processes, while the drivers relate to the organization of work from home. The most common barriers included emotional issues and pandemic-related fears, distractions and interruptions, poor workplace equipment and ergonomics, and teamwork-related issues such as socialization decrease, poor awareness of what is going on, greater difficulty to communicate with peers, grow a contact network or run brainstorming and problem-solving sessions. Among the most common benefits of working from home were better work-life balance, fewer interruptions from colleagues, better focus time, more time to complete work and increased flexibility in planning the work hours, as well as more efficient meetings and less time spent on commuting and communicating with the management.

Interestingly, similarly to the perceived productivity results that demonstrate polarity of experiences, some of the issues included in the barriers and drivers are contradictory. This means that what one considers a benefit, for another might be a barrier. For example, some complained about the difficulties to focus (Russo et al., 2021, Oliveira et al., 2020), while others reported a better focus time when working from home [10, GlobCo, NorBank, SavingsBank, NorIT]. This highlights once again the fact that the work from home mode is not for everybody.

The next important question relates to whether the identified factors can be attributed to the actual working from home or to pandemic circumstances. Our analysis shows that a few barriers can be attributed to the pandemic — Disaster preparedness and Emotional issues, pandemic-related fears (see Table 4), while most factors are related to working from home. Evidently, none of the drivers of perceived productivity are attributed to the pandemic. This means that in the likely post-pandemic scenario, in which people will continue working from home, there will be fewer barriers while drivers will remain the same.

We also see positive trends over time — there are fewer negatively impacted respondents in the later months of the pandemic. Many of the factors reported as barriers to productivity have been also in the focus of experimentation and improvement initiatives. This means that the impact of these barriers might have been more profound in the early months.

At the same time, it is important to note that the surveys included in our research are conducted within the first 13 months of working from home and the long-term effects of some of the barriers, such as the loosening ties which is compared to the melting iceberg of relationships (Clear, 2021), might not have yet surfaced. Table 4 exemplifies the barriers associated with decreased teamwork and networking the true long-term effects of which shall be monitored, including a greater difficulty communicating with people or growing a contact network, poor awareness of what is going on, loosening social ties across the organization, and decreased socialization in the team.

Notably, some of the barriers and drivers that we found have been studied before. For example, earlier studies included in the state-of-the-art of telework by Fløvik et al. (2021) suggest that the balance between work and leisure was found to be a challenge for those working from home, while job overall productivity and job satisfaction tended to benefit from the home environment. Our findings provide further details about the problems and benefits associated with working remotely from home, and perhaps more clearly demonstrate the polarity of experiences.

Future research agenda

In our study, we identified several important research directions.

First, it is paramount to better understand the different groups of engineers or teams that either benefit or are hindered by the work from home. We suggest analyzing the productivity in different contexts. Our study pointed to the possible differences in the WFH experiences based on the geographical locations, but more research is needed to better understand the differences with respect to locations (e.g., larger cities with longer commute times versus smaller towns), and national and organizational cultural impacts. Other related surveys have highlighted the vulnerability of different gender groups, people with disabilities and groups with varying family conditions (Ralph et al., 2020). Other context factors such as the nature of tasks (routine work versus complex or innovative tasks), learning opportunities, levels of interdependence and coordination needs, leadership styles and many others could also contribute to the success or failure of for remote work. More research is needed to understand which of these contexts are favorable and which are unfavorable for WFH to inform the future remote work policies.

Second, we suggest exploring the circumstances that destine the same characteristics of the working from home to play in advantage or disadvantage under the work-from-home conditions. In particular, a better understanding of the work-life balance, focus and virtual meetings is needed.

Third, our results identified a number of factors affecting well-being and mental health of individuals working from home, which, in our opinion, deserve extra attention.

Fourth, we believe that it is crucial to continue monitoring the work from home experiences to better understand the long-term effects of remote work that might not have yet surfaced (Clear, 2021).

Finally, in the light of different future work policies, it is important to dedicate research efforts into the understanding of the impact of hybrid work practices (working partially in the office and partially from home) on well-being, productivity, and teamwork.

Implications for practice

The key implications of our findings are twofold. Overall, our results indicate that there is no reason to be concerned about the productivity of software engineers working from home. Many engineers benefit from working from home, individual engineers and teams have managed to adjust their home environments and ways of working to the requirements of remote work, and the number of negatively affected engineers has decreased over time. Yet, the factors summarized in Table 4 highlight important issues such as the negative impact of WFH on mental health and relationships inside the team and across the organization. In the following, we put forward several recommendations emerging from our findings.

Individual employees seem to respond to home office situation very differently, depending on their personal characteristics, and family situation. For example, some report better work-life balance at the home office, while others report the opposite. Further, we know little about the favorable and unfavorable circumstances for remote work with respect to the nature of tasks (routine work/complex/innovative), levels of interdependence and coordination needs, and organizational culture. Therefore, there will likely be no one-fits-all future work strategy. Yet, it is fair to assume that while some employees will rush into the office when the pandemic is over, there will be employees willing to remain working fully or partially at home. Thus, we recommend forming future work policies with respect to the needs of different individuals and groups.

Notably, working from home as reported in our study is influenced by the disaster of the COVID-19 pandemic, while the future working from home is likely to be very different. One may therefore believe that in the absence of interruptions from home schooling children and pandemic fears we might evidence a further increase in productivity. Indeed, our findings show that engineers adopt to the new circumstances and report increased productivity 13 months into the pandemic when comparing with the results from the early months. At the same time, the true long-term effects of remote teamwork and loosening ties is yet to be understood (Clear, 2021). Besides, the identified emotional and well-being issues that are associated with remote work are worrisome and might signal about the possible mental health problems or burnouts as also suggested in related studies (Forsgren et al., 2021). Thus, we recommend monitoring the long-term effects of remote work, and experimenting and adjusting the work policies in response to the gained experience.

Finally, as the COVID-19 pandemic continues, we recommend companies to urgently address the needs of those employees who struggle when working from home. Our results show that employees will benefit from improved home office ergonomics and equipment, and internet connectivity. Such issues are relatively easy to address through the reimbursement programs, as the ones described in the case companies. Similarly, we can recommend improving the meeting culture. There are also barriers that are more challenging, such as distributed teamwork and collaboration. A related study shows that companies that had a strong collaborative culture coupled with rich tools supporting remote work had a relatively easy transition to working from home (Smite et al., 2021b). For other companies, we therefore recommend continuously supporting distributed collaboration processes and tools, especially because remote work is likely to stay even after the pandemic.

Conclusion

On the one hand, our findings confirm that software intensive companies have moved rather smoothly into the WFH mode during the COVID-19 pandemic and that on average, the organizational productivity has not significantly changed (Smite et al., 2021b, Forsgren, 2020). On the other hand, and more importantly, our analysis of individual productivity clearly shows that there are sufficiently large groups of developers who report being more productive, and groups of developers reporting being less productive as also found in a commit-based productivity study (Forsgren et al., 2021), meaning that on an individual level, things have changed. We also evidenced that things changed over time, as individual engineers and teams managed to adjust their home environments and ways of working to the requirements of remote work. And although “it depends” type of finding is not very useful in predicting the changes in perceived productivity in a random company, it is an important observation with important practical implications. First, companies shall find ways to support those developers who do not enjoy working from home and teams that could not reach the satisfactory remote operation, especially because the return to the normal operation appears to be much slower than expected and the periods of forced WFH might reappear periodically in the future. Second, it is fair to assume that the likely future workplace after the world returns to the normal operation, will include the elements of working-from-home, as developers who have been highly productive might want to continue working from home at least parttime. The companies, therefore, shall be prepared to facilitate the hybrid work environment in the future.

What we learned about pandemic programming by no means is complete or reflects the normal work-from-home experience. However, it gives hope that the positive factors and useful adjustments relating to the work-from-home are likely to be lasting beyond the pandemic would the companies decide to grant their employees with flexibility, while the issues of concern provide sufficient ground to plan future research. Specifically, we emphasize the importance of addressing the issues related to individual well-being and mental health as well as issues associated with remote teamwork and collaboration, whose direct impact might not have yet surfaced.

CRediT authorship contribution statement

Darja Smite: Conceptualization, Investigation, Visualization, Writing – original draft, Funding acquisition. Anastasiia Tkalich: Conceptualization, Investigation, Data curation, Writing – original draft. Nils Brede Moe: Investigation, Writing – original draft, Funding acquisition. Efi Papatheocharous: Data curation, Writing – review & editing. Eriks Klotins: Methodology, Data curation, Writing – review & editing. Marte Pettersen Buvik: Investigation, Writing – review & editing.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.