Backtesting VaR under the COVID-19 sudden changes in volatility.

We analyze the impact of the COVID-19 pandemic on the conditional variance of stock returns. We look at this effect from a global perspective, so we employ series of major stock market and sector indices. We use the Hansen's Skewed-t distribution with EGARCH extended to control for sudden changes in volatility. We oversee the COVID-19 effect on measures of downside risk such as the Value-at-Risk. Our results show that there is a significant sudden shift up in the return distribution variance post the announcement of the pandemic, which must be explained properly to obtain reliable measures for financial risk management. Crown

Introduction

The COVID-19 outbreak has caused the most severe economic crisis since The Great Depression (Caggiano et al., 2020). In particular, global financial markets have experienced extensive and massive uncertainty with volatility at record levels in recent history (Baker et al., 2012, Shehzad, Xiaoxing, Kazouz, 2020, Goodell, 2020, Zhang, Hu, Ji, 2020, and Baig et al., 2021). New research has shown that connectedness in financial markets has increased during this period of due-to-COVID-19 uncertainty; see, among others, (Bouri, Cepni, Gabauer, Gupta, 2020a, Bouri, Demirer, Gupta, Pierdzioch, 2020b); (Izzeldin, Muradoglu, Pappas, Sivaprasad, 2021, Bouri, Gkillas, Gupta, Pierdzioch, 2021, Gupta, Subramaniam, Bouri, Ji, 2021); and (Shahzad et al., 2021). In this paper we contribute to this literature with an investigation on the impact of the COVID-19 crisis on the time-varying variances of major stock market and sector index returns. We follow the methodology based on GARCH models with shift dummy variables of Lamoureux and Lastrapes (1990); Aggarwal et al. (1999) and (Mikosch and Stărică, 2004); see also Malik et al. (2005); Kang et al. (2009); Ewing and Malik (2017) and Anjum and Malik (2020) for recent empirical studies.

For this purpose we use the exponential GARCH (EGARCH) model of Nelson (1991) augmented with a sudden shift dummy variable to incorporate the COVID-19 effect on volatility. For the skewed and heavy-tailed distribution of the standardized returns, we employ the popular Skewed-t (ST) of Hansen (1994). Hereafter, this model is referred to as EGARCH-D-ST. In our empirical exercise, we show evidence that incorporating the COVID-19 abrupt shift has an important impact on the accuracy of estimating volatility dynamics and forecasting future Value-at-Risk (VaR). In line with previous results, we also find clear evidence on that accounting for the sudden change reduces the persistence in the EGARCH model. The performance of the previous model is compared with that of the model without the dummy variable through the unconditional backtesting procedure of Kupiec (1995) for the pandemic period. Since the asymptotic distribution of the Kupiec’s backtesting test is not adequate for our small sample size, we have obtained Monte Carlo p-values according to Christoffersen (2011).

The remainder of the paper is organized as follows. In Section 2 we present the EGARCH-D-ST model for asset returns. Section 3 provides an empirical application to forecast the VaR of major stock and sector returns with a backtesting analysis. Section 4 gathers the conclusions.

Modeling asset returns

Let the asset return be a process characterized by the sequence of conditional densities where denotes the information set available prior to the realization of is the vector of unknown parameters such that is the subset characterizing both the conditional mean and variance of i.e. and and finally, is the subset characterizing the shape of the distribution of the innovations, . Thus, we assume thatSo, Eq. (1) decomposes the return at time into a conditional mean which is assumed to be constant, and the term defined as the product between the conditional standard deviation, and the innovation (or standardized return), with zero mean and unit variance. It is assumed that is a sequence of independent identically distributed (iid) random variables driven by the ST distribution with parameter set where and control, respectively, for skewness and kurtosis, and denoted as . Let be the EGARCH (1,1) conditional variance model augmented with an intercept dummy variable to account for changes due the COVID-19 pandemic. Thus,where if the return observation belongs to after the 31th of December 2019 as the starting date of the COVID-19 period when the first case was reported to the World Health Organization (2020) by China.

Empirical application

Dataset and estimation

We analyze the time-series behavior of 17 major stock market and 27 world sector indices. The data employed were daily percentage log returns, which were computed as from daily closing prices (in $). The time period for series used comprises from January 2, 2017 to May 25, 2020, for a total number of observations. Table 1 provides the list of the series. All data series were downloaded from Datastream. The world sector indices data are supplied by Morgan Stanley Capital International (MSCI) Barra. The MSCI world sector indices capture the large and mid-cap companies across 23 developed markets countries around the world. All securities in each index are classified in the corresponding sector as per the Global Industry Classification Standard. The stock market indices analyzed are selected to represent major stock markets across the world. Table 1 also reports the standard deviations of daily returns before and after December 31, 2019. These statistics confirm that the pandemic has had a great influence on the stock markets and as a result, an increase in the volatility in all cases. This evidence suggests a possible structural change in the unconditional volatility that should be considered in modeling the conditional variance in the spirit of Lamoureux and Lastrapes (1990).

Table 1

Stock market and sector indices used in the empirical analysis

Name	s_b	s_a	Name	s_b	s_a	Name	s_b	s_a
Stock market indices
ASX 200	0.65	2.59	MIB	0.98	3.02	S&P 500	0.72	3.06
AEX	0.72	2.41	HANG SENG	0.99	1.86	NASDAQ	1.07	3.05
CAC 40	0.79	2.79	IBEX 35	0.81	2.73	SMI	0.72	2.03
BOVESPA	1.22	4.03	KOSPI	0.75	2.29	TSX	0.88	2.23
DAX 30	0.84	2.71	FTSE 100	0.69	2.43	MOEX	0.53	3.07
EUROSTOXX 50	0.77	2.66	MEXICO IPC	0.84	1.91
Sector indices
Banks	0.80	3.46	Communication services	0.73	2.40	Hotels	0.66	3.35
Materials	0.77	2.59	Transportation	0.72	2.43	Insurance	0.64	3.00
Aerospace and defense	0.91	3.83	Media	0.83	2.80	IT services	0.98	3.31
Oil and gas	0.95	4.36	Health Care	0.70	2.20	Airlines	0.96	4.01
Utilities	0.58	2.82	Biotec	1.04	2.42	Pharmaceuticals	0.65	1.81
Financials	0.74	3.25	Chemicals	0.78	2.56	Retail	1.02	2.50
Industrials	0.69	2.65	Consumer services	0.66	3.34	Software	1.12	3.25
Real State	0.57	2.90	Food/beverage/tobacco	0.58	2.04	Tobacco	1.04	2.44
Information technology	1.04	3.19	Gas utilities	0.56	1.85	Water utilities	0.88	3.32

This table presents the names and sample standard deviations of the stock market and sector indices used in the empirical analysis of this article. Both and denote the sample standard deviations of the series before and after 31/12/2019, respectively.

The parameters of our EGARCH-D-ST model were estimated using maximum likelihood (ML). Table 2 presents the estimation results. The unconditional mean parameter, is not significant for many return series. The parameter estimates of the conditional variance Eq. (2) show that, for all series, the model correctly captures the asset returns stylized features of (i) clustering and high persistence in volatility, and (ii) asymmetric response of volatility to positive and negative shocks. Indeed, the parameter which is related to the persistence for the EGARCH, is rather high for all series with mean estimates of 0.944 and 0.958 for stock market and world sector indices, respectively. Also, asymmetric response, is significant for all series. The ST asymmetry parameter, is significant for 13 out the 17 stock market indices, and 19 out of the 27 world sector indices. So, there is evidence for asymmetry for most standardized returns series. Note also that the ST degrees of freedom parameter, estimates indicate that the cross-sectional means for the stock market and sector indices exhibit kurtosis levels of 6.2 and 7.1, which are far away from the Normal distribution (i.e., large value of ). In short, the previous results suggest that the standardized returns are not normally distributed.

Table 2

Estimation results

Cross-sectional distribution
Panel 1: sample period 02/01/2017-25/05/2020
	μ	ω	δ	α	γ	β	υ	λ
Stock market indices
Mean	0.020	-0.095	0.093	0.095	-0.171	0.944	6.190	-0.141
Q1	0.002	-0.111	0.072	0.058	-0.220	0.941	4.907	-0.175
Median	0.010	-0.095	0.083	0.094	-0.174	0.946	5.705	-0.142
Q3	0.038	-0.064	0.107	0.115	-0.128	0.958	6.370	-0.093
M	2	14	15	17	17	17	17	13
Sector indices
Mean	0.034	-0.136	0.082	0.147	-0.112	0.958	7.105	-0.115
Q1	0.018	-0.078	0.062	0.118	-0.127	0.953	5.275	-0.016
Median	0.030	-0.132	0.071	0.149	-0.106	0.955	6.704	-0.133
Q3	0.062	-0.104	0.096	0.177	-0.087	0.964	8.918	-0.078
M	8	26	25	25	27	27	27	19
Panel 2: EGARCH Dummy significance over OOS subperiods
Sample ends	31/01/2020	28/02/2020		30/03/2020	31/04/2020		25/05/2020
Stock market	4 [5]	3 [7]		16 [17]	15 [16]		15 [16]
Sector	3 [6]	10 [15]		26 [26]	25 [27]		25 [26]

The rows present the mean, median, 25 and 75 percentiles (Q1 and Q3, respectively) from the cross-sectional distribution of the parameter estimates listed in the columns. denotes the number of series with significant parameter at 5% level. There are 17 stock market and 27 sector indices. Panel 2 reports number of series for which the dummy variable parameter is significant at 5% (10% in brackets) for the several samples ending on 31/01/2020, 28/02/2020, 30/03/2020, 31/04/2020 and 25/05/2020.

The dummy parameter, denoted as is significant for 15 and 25 stock market and world sector indices, respectively, indicating an important due-to-COVID sudden change in volatility across these indices. In order to analyse more precisely when the shift in volatility starts to become relevant, we estimate our model for four different subsamples across the whole sample period. The results, presented in Table 2 (Panel 2), indicate that for most of the series the sudden-change dummy variable effect kicks in March 2020 as becomes statistically significant. The sector index volatilities that reacted faster to the COVID shock seem to be those related to online activity. Namely, IT technology, IT services and Software series showed the dummy parameter is significant already in January 2020. As regards the stock market indices, predominantly those from less developed markets picked up the shock faster. The dummy parameter is significant in February 2020 for the Chinese, Mexican and Russian series, although it is also for NASDAQ. From our results we can infer high connectedness among the majority of sector and stock index volatilities in the response to the COVID crisis, as most series showed a shift in volatility in March 2020. We find that the magnitudes of the dummy coefficients become larger as well as significant at lower levels as we move through the out-of-sample (OOS) period from February 3, 2020 to May 25, 2020. Fig. 1 shows the plots of coefficient estimates and t-statistics over the OOS period for NASDAQ and Banks return series as representative examples.1

Fig. 1

Dummy coefficient over the out-of-sample period This figure presents the dummy variable coefficient estimates together with their t-statistics over the OOS period: February 3, 2020 to May 25, 2020. Series: NASDAQ, Banks. Observations 81.

Backtesting VaR

For the OOS analysis, we are interested in the VaR-backtesting performance comparison between the EGARCH-D-ST model, which does consider the sudden change in volatility due to the COVID-19 effect, and the EGARCH-ST model which is nested in the former when in (2) and does not account for the previous effect.

The backtest implementation involves the first - observations for the first in-sample window and the OOS period of length from February 3, 2020 to May 25, 2020, using a constant-sized rolling window. For every window we estimate the model parameters by ML and obtain a one-day-ahead forecast of the conditional variance, . We have done this for all return series presented above under several coverage levels (denoted as ): 1%, 2.5%, 5%. The one-day-ahead VaR for the -quantile is given by where represents the -quantile of the distribution for the random variable obtained through the inverse of its cumulative distribution function (cdf), and denoted as . Letdenote the violation or hit variable. We obtain the quadratic loss function, which incorporates the exception magnitude and provides useful information to discriminate among similar models in terms of the unconditional coverage criterion. Thus,We estimate the sample averages for the daily estimations of (3) and (4) corresponding to the daily violations in (3) and the daily quadratic losses in (4) for the OOS period of days. The probability suggests that violations are Bernoulli variables with mean . The null hypothesis for the unconditional backtest, corresponds to the following likelihood ratio (LR) test statistic initially proposed by Kupiec (1995):where is the likelihood of an Bernoulli hit sequence, i.e. such that and are the number of zeroes and ones (or hits) in the sample, and is the sample average of the hit sequence in (3) for the whole OOS period such that with and as the estimations of and . Hence, we can easily obtain .

Finally, as our OOS period is short, we perform a simulation exercise to check the robustness of our number of violations respecting the sample size. For this purpose, we follow Christoffersen (2011) so as to obtain the Monte Carlo simulated p-values since they are more reliable than those under the distribution for small sample sizes. The simulated p-values are obtained as follows. First, we generate 9999 samples of random Bernoulli variables with sample size . Second, we calculate 9999 simulated test statistics according to (5) and denoted as where corresponds to the simulated -th sample. Finally, the simulated p-value is given by

Table 3 exhibits a descriptive analysis of VaR average violations (VIOL) and quadratic losses (MSE) obtained from EGARCH-D-ST and EGARCH-ST models through the OOS period. As a way to summarize the results across all indices, we report the mean, median, 25% and 75% percentiles (Q1 and Q3, respectively) of the cross-sectional distribution of each return index type. Our results clearly show that the EGARCH-D-ST delivers a number of violations closer to the theoretical ones, according to the unconditional backtest test results by using (6), as well as lower MSE values, for all three confidence levels and both index sets. As an example, Fig. 2 exhibits, for the KOSPI series at the top left, daily 1% VaR forecasts, . This plot shows that the number of violations over are 0 and 3 for the EGARCH-D-ST and EGARCH-ST, respectively. The figure also exhibits series plots related to the computation of the VaR series. The top-right plot shows that the one-period-ahead conditional volatility forecasts are higher under the EGARCH-D-ST. The plots at the bottom are for the parameters implied in the ST distribution for the standardized returns. The parameter, plots at the bottom right, and the parameter, bottom left, both control predominantly for the skewness and kurtosis, respectively. It is observed that the estimates are higher under the EGARCH-D-ST, and the series are negative and verify that the size of is higher under the EGARCH-ST. The corresponding skewness and kurtosis series under the ST distribution, which are obtained by plugging and into the higher-order moment closed-form expressions in Jondeau and Rockinger (2003), show higher daily levels of both negative skewness and kurtosis under the EGARCH-ST model.2 This evidence of higher kurtosis levels due to not considering shift dummies are, for instance, in line with that in Ewing and Malik (2017) and Anjum and Malik (2020). Fig. 3 clearly illustrates, for the KOSPI series, the relation between persistence in volatility and sudden changes. The left plot shows that daily volatility autocorrelations (using the absolute return as a proxy for the volatility) are much higher for the whole sample, which includes the COVID-19 sudden change in volatility. The right plot shows that including sudden shift dummies reduces the persistence in the EGARCH-D-ST model.

Table 3

Descriptive analysis of violations and MSE

	VIOL		MSE
α	EGARCH-D-ST	EGARCH-ST	EGARCH-D-ST	EGARCH-ST
Stock market indices
0.01	Mean	2.9	4.1	0.268	0.405
	Q1	2	3	0.024	0.083
	Median	3	4	0.167	0.262
	Q3	4	5	0.382	0.518
	m	2 [12]	7 [15]
0.025	Mean 4.8	7.5	0.585	0.787
	Q1	4	7	0.091	0.184
	Median	5	8	0.418	0.705
	Q3	7	8	0.772	0.942
	m	4 [6]	14 [16]
0.05	Mean	8.4	11.8	0.974	1.250
	Q1	7	10	0.264	0.550
	Median	8	11	0.767	1.085
	Q3	10	13	1.210	1.442
	m	6 [8]	17 [17]
Sector indices
0.01	Mean 2.7	4.0	0.240	0.354
	Q1	2	3	0.011	0.056
	Median	3	4	0.069	0.175
	Q3	4	5	0.309	0.490
	m	3 [15]	11 [24]
0.025	Mean	4.7	7.0	0.481	0.681
	Q1	4	6	0.086	0.258
	Median	5	7	0.342	0.507
	Q3	6	8	0.657	0.823
	m	3 [9]	17 [25]
0.05	Mean	7.3	9.6	0.847	1.152
	Q1	6	8	0.305	0.600
	Median	7	10	0.672	0.847
	Q3	9	11	0.942	1.312
	m	3 [7]	15 [18]

This table presents a descriptive analysis of one-day-ahead VaR forecasting performance from EGARCH-D-ST and EGARCH-ST models. Both VIOL and MSE denote, respectively, average violations and quadratic losses. The coverage level is . For each we present the mean, median, 25 and 75 percentiles (Q1 and Q3, respectively) for VIOL and MSE across the out-of-sample period. denotes the number of times the null of the unconditional backtest is rejected according to equation (6) at 1% and (in brackets) at 5% levels. The data consists of daily return series from stock market and sector indices. Total sample: 887 observations from January 2, 2017 to May 25, 2020. OOS period: February 3, 2020 to May 25, 2020. Predictions: 81.

Fig. 2

VaR and volatility forecasts and Skewed-t parameter estimates. Series: KOSPI This figure presents 1% VaR and volatility forecasts, as well as Skewed-t parameter estimates over the OOS period: February 3, 2020 to May 25, 2020. Series: KOSPI. Observations 81.

Fig. 3

Volatility autocorrelation and persistence. Series: KOSPI The left plot of this figure exhibits the autocorrelation of absolute value returns for both the whole sample and subsample up to 31/01/2020. The right plot presents the beta parameter estimates from EGARCH-ST and EGARCH-D-ST models for the OOS period: February 3, 2020 to May 25, 2020. Series: KOSPI. Observations 81.

Conclusions

In this paper we have investigated the sudden change in volatility of major stock and sector indices caused by the COVID-19 pandemic. Using the popular EGARCH with Hansen’s Skewed-t distribution augmented with a sudden change dummy variable, we show the importance of incorporating the abrupt volatility shift for explaining volatility dynamics, forecasting VaR and backtesting. In addition, we confirm that when these changes are accounted for the persistence in volatility diminishes considerably.

Our findings have valuable implications for portfolio risk managers and policymakers. For the former, the proposed model ensures accuracy in the measurement of risk in the event of large shocks to volatility such as those created by the COVID-19 outbreak. These risk measures are critical in portfolio management for hedging and diversification. For policymakers who seek financial stability in financial markets, if necessary, they can intervene to control uncertainties from highly stressful periods such as the current pandemic one. Our study based on a univariate framework can be extended to tackle the financial markets connectedness through the tail dependency by combining the copula approach and quantile regression; see e.g. Bouri and Jalkh (2019), as well as the networks of spillovers in low and high volatility regimes, see e.g. Shahzad et al. (2021) for different volatility regimes. We deem these two avenues very interesting for future research.

CRediT authorship contribution statement

Brenda Castillo: Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Software, Validation, Visualization, Writing - original draft, Writing - review & editing. Ángel León: Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Software, Validation, Visualization, Writing - original draft, Writing - review & editing. Trino-Manuel Ñíguez: Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Software, Validation, Visualization, Writing - original draft, Writing - review & editing.