Pairwise difference regressions are just weighted averages.

: R. F. Savaris et al.; Scientific Reports 10.1038/s41598-021-84092-1 (2021).

Savaris et al.[1] aim at “verifying if staying at home had an impact on mortality rates.” This short note shows that the methodology they have applied in their paper does not allow them to do so. An estimated coefficient does not imply that there is no association between the variables in either country. Rather, their pairwise difference regressions are computing coefficients that are weighted-averages of region-specific time series regressions, such that it is possible that the association is significant in both regions but their weighted-average is close to zero. Therefore, the results do not back up the conclusions of the paper.

Consider two regions: A and B. Suppose that the true relationships between the change in deaths per million and the change in an index of staying at home at epidemiological week t in countries are the following:

For simplicity in exposition, assume that are all zero mean, iid processes. By subtracting the second equation from the first and defining and , we can write:where . It is easy to see that, for , estimation of will not be consistent, since, by construction, .

If nonetheless one estimates (1) by ordinary least squares, what does the regression coefficient converge to? It turns out that it converges to a variance-weighted average of , , as summarized in the following proposition.

Proposition 1

Let , be all as above. Then , the ordinary least squares coefficient of regressing on , converges in probability to:with .

Proof

Under the stated assumptions, . One can calculate the population parameter analytically:Note that and . Using that and the definition of w we arrive at the desired result.

The intuition regarding the (2) in the Proposition is simple. Whenever the variance of is large relative to country B, and . Similarly, if the variance of is large relative to country A, and .

What does this mean for the analysis of Savaris et al.[1]? In general, it means that one cannot interpret their estimated without knowing the underlying relative variances. Additionally, one cannot infer that an insignificant (or even numerically zero) implies absence of association in either country.

To see that, suppose countries A and B have identical variance in their independent variables, but , are different. In country A, the policymaker adjusts stay-at-home orders in response to the increase in deaths, such that the change in the percentage of the public staying at home is positively correlated with the change in deaths. In country B, the policymaker does not act, such that the change in share of population staying at home is negatively correlated with contacts, infections, and deaths.

Consider the case in which . Then, since the regions have identical variance, and even though the true association is nonzero in both countries. The regression coefficients in Savaris et al.[1] should not lead one to conclude that, in either country, there is no association between the change in mobility and the change in deaths per million. Figure 1 shows the result of 10,000 simulated in which and . In this case, and variables are iid and normally distributed. As expected, sample estimates are distributed around the population value of .

Figure 1

In-sample simulated for 10,000 random draws with , , and , for ; ; and , . As expected the sample values are distributed around the true population value of .

For , then, region-specific dynamics are heterogeneous and, as shown by Pesaran & Smith[2], aggregating or pooling slopes can lead to biased estimates, making individual regressions for each group member preferable. If authors assume that for each pair in their sample – i.e., homogeneous –, then dynamic panels would have many advantages in terms of efficiency and use of instruments to circumvent endogeneity. In either case, their pairwise approach would not be appropriate.

In order to verify if “staying at home had an impact on mortality rates,” it would be necessary to address many other issues in the analysis, including, but not limited to, omitted variable bias, measurement error, and endogeneity of the regressors. However, as shown above, even in a purely correlational analysis, with no causality claims, the applied methodology will simply deliver a weighted-average of coefficients across the two regions. An estimated coefficient does not imply that there is no association between the variables in either country. Therefore, their conclusion does not follow from their regressions.

Supplementary Information.