A residual driven ensemble machine learning approach for forecasting natural gas prices: analyses for pre-and during-COVID-19 phases.

The natural gas price is an essential financial variable that needs periodic modeling and predictive analysis for many practical implications. Macroeconomic euphoria and external uncertainty make its evolutionary patterns highly complex. We propose a two-stage granular framework to perform predictive analysis of the natural gas futures for the USA (NGF-USA) and the UK natural gas futures for the EU (NGF-UK) for pre-and during COVID-19 phases. The residuals of the previous stage are introduced as a new explanatory feature along with standard technical indicators to perform predictive tasks. The importance of the new feature is explained through the Boruta feature evaluation methodology. Maximal Overlap Discrete Wavelet Transformation (MODWT) is applied to decompose the original time-series observations of the natural gas prices to enable granular level forecasting. Random Forest is invoked on each component to fetch the respective predictions. The aggregated component-wise sums lead to final predictions. A rigorous performance assessment signifies the efficacy of the proposed framework. The results show the effectiveness of the residual as a feature in deriving accurate forecasts. The framework is highly efficient in analyzing patterns in the presence of a limited number of data points during the uncertain COVID-19 phase covering the first and second waves of the pandemic. Our findings reveal that the prediction accuracy is the best for the NGF-UK in the pre-COVID-19 period. Also, the prediction accuracy of the NGF-USA is better in the COVID-19 period than the pre-COVID-19 period.

Introduction

Natural gas usage has seen a continuous surge in the world energy systems due to sustainability challenges and to mitigate hazardous environmental impacts (Avraam et al., 2021; Zhou et al., 2021). Economic activities are closely linked with natural gas prices. As a result, predicting natural gas prices becomes a highly arduous task of paramount relevance in the planning and operations of various key assets, including public utilities (Goncharuk & Cirella, 2020). Natural gas price predictions assist in regulatory decision-making regarding electric and gas utilities (Costello, 2010). Despite an uninterrupted growth of natural gas trade, bubbles in natural gas prices in the European Union, Asia, and the US markets have been documented (Li et al., 2020). Economic euphoria, geopolitical events, fluctuations of crude oil prices largely account for the same. Unavoidable lockdowns and curbs owing to the COVID-19 pandemic have further triggered uncertainty in the overall consumption of natural gas and (Cihan, 2022).

The literature reported the rambling price movements of natural gas played a pivotal role in political chaos in Venezuela, public protests in Iran, and political turmoil in Kazakhstan (Rostan & Rostan, 2021). Precise predictions of natural gas prices can help optimize resource utilization, reserves management, and drawing decisions on economic reforms. Natural gas price movements are susceptible to natural calamities and external economic volatility (Huang & Etienne, 2021; Shi & Shen, 2021). Its co-movement and strong nexus with the crude oil price and stock market across different economies are also apparent (Ftiti et al., 2020; Mensi et al., 2021). Fluctuations of natural gas prices induce volatility towards gas-and-coal-fired electricity generation (Linn and Muehlenbachs, 2018). The potential implications of natural gas price predictions motivate us to develop a robust framework that yields highly accurate forecasts during the normal and new-normal timelines.

Natural gas futures price forecasting has received less attention than crude oil. The literature is more focused on predictive analysis of natural gas consumption at industrial and household levels (Gao & Shao, 2021; Hu et al., 2020; Xiao et al., 2020). We retrieve some literature that deals with natural gas futures price forecasting. Variants of regression and artificial neural network (ANN) models have been used for predictive modeling of natural gas prices (Salehnia et al., 2013). The results revealed that the accuracy of the ANN models was the best. Historical prices coupled with news on the internet and search data are used to forecast natural gas prices (Tang et al., 2019). The latter two components impacted positively to achieve better predictions. A Variational mode decomposition-based approach is used to estimate the point and interval forecasts for natural gas prices (Jiang et al., 2021). The obtained forecasts were more reliable and stable than the other models. Some hybrid approaches are also proposed to forecast daily natural gas prices (Wang et al., 2020). Unfortunately, the dearth of predictive modeling work to predict natural gas prices during the COVID-19 pandemic is pretty apparent. It is also important to acknowledge that the previous studies have considered the larger sample sizes for model building. They are not tailor-made to identify the patterns for a shorter sample size, which imposes several practical decision-making restrictions.

This paper contributes in the following two ways: First, it proposes a unique residual driven ensemble machine learning approach. The residuals of the previous stage have been utilized as a technical indicator to perform predictive tasks. The inclusion of residuals of predictive modeling of the previous stage has transpired to be of utmost importance in drawing forecasts of supreme precision in the subsequent stage. Second, explicitly carrying out forecasting during the timeline of the pandemic covering the first and second waves is of enormous practical implications and acts as an acid test to check the robustness and capacity of any predictive structure. The predictive modeling approach proposed in this study helps to produce quality forecasts during said periods. Also, it uses relatively small segments of data of the first and second waves of pandemics exhibiting a high degree of turbulence owing to apparent implications.

The rest of the paper is organized as follows: In Sect. 2, we discuss the pertinent literature in brevity to comprehend the trend and highlight the existing gaps to justify the positioning of current work. Data descriptions alongside the settings to ascertain the prediction quality during Pre COVID-19 and COVID-19 regimes have been elaborated in Sect. 3. The research methodology has been elucidated in Sect. 4. The outcome of predictive exercises has been presented and analyzed in Sect. 5. Finally, the paper is concluded in Sect. 6 by mentioning the key findings, limitations, and future research agendas.

Literature review

Owing to its contribution to the energy market, different aspects of natural gas modeling have received considerable attention from researchers. The co-movement patterns of natural gas and crude oil prices were studied (Mensi et al., 2021). Natural gas prices offered diversification benefits. Scholars also studied the interplay of natural gas prices with other commodity and financial assets (Ameur et al., 2020; Kumar et al., 2021; Liao et al., 2021). On a similar note, the natural gas consumption prediction has garnered attention, as reported in the literature (Liu et al., 2021; Lu et al., 2020; Qiao et al., 2020; Wei et al., 2021; Zhou et al., 2020). Unfortunately, few studies are available so far in the literature for forecasting natural gas prices using machine and deep learning techniques despite being a widely used source of clean energy.

An ensemble predictive structure was proposed to analyze Henry Hub's natural gas spot prices using the least-squares boosting technique (Ali, 2020). The predictive performance was satisfactory. A similar framework was constructed to estimate the daily US natural gas price (Wang et al., 2020). Long short-term memory network (LSTM), Support vector regression (SVR), and improved pattern sequence similarity search (IPSS) models were used for generating initial forecasts that were combined using variance reciprocal framework to obtain the final forecasts. The approach was found to be extremely proficient in yielding high-quality predictions.

A granular predictive model was proposed to predict monthly spot prices of natural gas (Li et al., 2021) of Henry Hub. The methodological framework incorporated variational mode decomposition (VMD) to disentangle the original series into granular components and combined particle swarm optimization (PSO) and deep belief network (DBN) to draw forecasts. The findings revealed that WTI crude oil spot prices and natural gas consumption largely governed the evolution of Henry Hub's natural gas spot price. A neoteric short-term forecasting structure was proposed for natural gas price prediction (Wang et al., 2021). The granular framework utilized ensemble empirical mode decomposition to decompose the original series into subseries wherein an RNN model optimized through PSO was applied to fetch component-wise forecasts to draw the final aggregate forecasts. The framework performed better than the other benchmark models.

Most of these studies rely on sophisticated mathematical modeling in conjunction with machine learning and computational intelligence techniques. Unlike natural gas price modeling, the evolutionary pattern of natural gas consumption has been critically delved into during the COVID-19 pandemic (Cihan, 2022). Although natural gas prices and consumption are not entirely independent and exclusive, the evolutionary patterns and overall dynamics of both variables are influenced by different macroeconomic and financial conditions. Therefore, extending models built for consumption patterns recognition to predict price movement may not be ideal in challenging circumstances. Hence, the development of a dedicated framework to estimate futuristic figures of natural gas prices becomes essential.

The review of related literature demonstrates an inclination towards granular and artificial intelligence-driven methodologies for predictive analytics of natural gas prices. Despite being an influential energy commodity with substantial nexus with other financial assets, modeling futures prices has seen meager attention. To the best of our knowledge, no attempt has been made to date to specifically estimate forecast in the COVID-19 period encompassing both first and second waves of infection. Unlike Pre COVID-19 phase, performing prediction in the COVID-19 regime is extremely difficult irrespective of the high-end model used for drawing predictions due to profound uncertainty, disruptions, and anticipation of the market crash. Thus, barring the contribution towards the research void, the current work also propounds a neoteric feature engineering framework to accomplish the objective that can be leveraged for predicting extremely chaotic and uncertain events. Precise predictions of the UK and the US natural gas futures can directly be utilized for risk mitigation through portfolio realignment. Therefore, the apparent absence of research and profound practical applications justify the underlying research significantly.

Data description

We investigate the predictability of natural gas prices for the pre-and during-COVID-19 periods and see whether the current pandemic leads to more uncertainty in the pricing structure. Accordingly, we collect daily data for the NGF-UK and NGF-USA from Bloomberg. The ‘During Covid-19’ period is taken from December 31, 2019, the day of receiving the first confirmed COVID-19 case in Wuhan, China, to May 20, 2021. The span of the ‘During COVID-19’ period for this study is 506 days containing 367 observations each for both the considered series. In the ‘Pre COVID-19’ period, we consider data from August 09, 2018, to December 30, 2019, to keep the uniformity in the number of observations for both series.

In summary, we have only 367 observations in each of the pre-and during-COVID-19 periods. We present the temporal patterns of Natural gas futures for the UK and USA during the investigation periods in Fig. 1. The visual inspection implies that all four price movements are highly nonlinear. In the rest of the paper, we use ‘UK-Pre’, ‘UK-During’, ‘US-Pre’, and ‘US-During’ to denote the time series for the NGF-UK in the ‘Pre COVID-19’ period, NGF-UK prices in the ‘During COVID-19’ period, NGF-USA in the ‘Pre COVID-19’ period, and NGF-USA ‘During COVID-19’ period, respectively.

Fig. 1

Evolutionary patterns of natural gas futures. Figure 1 represents the temporal patterns on Natural gas futures for the UK and USA during the mentioned periods. The x-axis shows the duration and the y-axis of the figures (a) and (b) shows the natural gas futures for the UK in US$ per MMBtu. The y-axis of figures (c) and (d) shows the NGF-USA in US$ per MMBtu

Table 1 provides the summary statistics and statistical tests results to establish the behavioral patterns of the datasets.

Table 1

Descriptive statistics

	UK-Pre	UK-During	US-Pre	US-During
Mean	0.60	0.44	2.79	2.30
SD	0.21	0.22	0.56	0.48
Median	0.53	0.40	2.65	2.40
Min	0.31	0.10	2.07	1.48
Max	1.03	1.05	4.84	3.35
MAD	0.22	0.30	0.46	0.71
Skewness	0.46	0.34	1.53	0.05
Kurtosis	− 1.28	− 1.04	2.25	− 1.42
N.valid	367	367	367	367
Jarque–Bera Test	37.55***	23.05***	228.99***	30.72***
Shapiro–Wilk Test	0.89***	0.94***	0.85***	0.92***
Kolmogorov–Smirnov Test	0.62***	0.54***	0.98***	0.94***
Anderson–Darling Test	15.22***	6.74***	14.45***	11.44***
Augmented Dickey-Fuller Test	− 1.18#	− 2.39#	− 2.60#	− 3.24#
Ljung–Box Test	361.79***	359.78***	352.36***	354.17***
Terasvirta’s Neural Network Test	8.07**	23.41***	30.54***	6.96**
White’s Neural Network Test	8.15**	17.22***	8.76***	6.32**
Hurst Exponent	0.86	0.86	0.83	0.86

***Significant at 1%, **Significant at 5%, *significant at 10% levels of significance, # Not Significant. US-Pre and UK-Pre denote NGF-USA and NGF-UK from August 09, 2018 to December 30, 2019, respectively. Similarly, US-During and UK-During denote NGF-USA and NGF-UK from December 31, 2019 to May 20, 2021, respectively

From Table 1, we find that the test statistic values of Jarque–Bera, Shapiro Wilk, Kolmogorov–Smirnov, and Anderson–Darling are highly significant for all the time series. So, the natural gas futures exhibit non-parametric behavior. The ADF test statistic values appear insignificant for all the series in pre-and during COVID-19 periods. Therefore, we conclude that no series is stationary. A highly significant test statistic value of the Ljung-Box test at lag 10 confirms the presence of a strong autocorrelation structure in all the time series in pre-and during COVID-19 periods. The test statistic values of both neural network tests substantiate the nonlinear behavior of all four time series. The Hurst exponent values are significantly higher than 0.5 for all the time series. So, we find the existence of persistent trends in the evolutionary patterns of natural gas futures of the UK and the USA.

Several statistical tests on the time series of natural gas futures of the UK and the USA reveal the presence of long memory and autocorrelation structure in pre-and during COVID-19 periods. In such situations, a predictive framework developed based on technical indicators is proven to provide superior forecasts (Ghosh et al., 2019). This encourages us to use a technical indicator-based predictive modeling framework. We select 30 technical indicators for predicting natural gas prices (Ghosh et al., 2019; Hsu, 2011; Huang & Tsai, 2009; Jana et al., 2020), following the previous works. The brief description of the technical indicators used in this work are as follows: MA5, MA10, and MA20 denote 5-day, 10-day, and 20-day moving average, respectively; B5, B10, and B20 denote 5-day, 10-day, and 20-day bias, respectively; MTM5, MTM10, and MTM20 denote 5-day, 10-day, and 20-day momentum, respectively; ROC5, ROC10, and ROC20 denote 5-day, 10-day, and 20-day rate of change, respectively; LAG1, LAG2, LAG3, LAG4, and LAG5 denote 1-day, 2-day, 3-day, 4-day, and 5-day back prices, respectively; EMA5, EMA10, EMA12, EMA20, and EMA26 denote 5-day, 10-day, 12-day, 20-day, and 26-day exponential moving average, respectively; UB, and LB denote upper and lower Bollinger band, respectively; LOSS denotes the difference between today’s and next day’s prices; DIFF denotes the difference between 12-day and 26-day moving average; MACD denotes the moving average convergence divergence; HP signifies the highest price of the day; LP represents the lowest price of the day; and WMS denotes the Williams overbought/oversold index.

The proposed approach uses residual (Res) of predictive modeling in the previous stage as an input feature for predicting the movements in the next stage. So, the forecasts are obtained in the first stage by considering the above 30 features as independent input variables. In the second stage, we propose to include as an explanatory feature in addition to the previous 30 features.

Methodology

In this section, we enunciate the components of the adopted integrated framework to carry out natural gas futures price forecasting in the considered regime. The construction of the predictive architecture commences with the feature engineering process. The most important contribution of the predictive structure lies in the selection of features that can compensate for the volatile impact of macroeconomic and other external shocks in recognizing the hidden pattern. Boruta feature screening algorithm has been utilized for accomplishing the task. MODWT and RF are the other two principal components of the proposed methodology.

BORUTA feature selection algorithm

Boruta (Kursa et al., 2010) is an ensemble feature selection algorithm that is in many ways similar to the RF algorithm with marginal changes. It depends on bootstrapped samples that augment the induced randomness in the underlying information system. Interested readers may follow Kursa and Rudnicki (2010) for details about this algorithm. The fundamental steps of the algorithm are outlined below. In our work, Boruta has been used to evaluate the importance of residual as an explanatory feature and identify the top three features across the granular components on which actual predictive exercises are carried out to obtain the final forecasts. We now discuss the MODWT technique, which is used for the granular decomposition of considered time series.

Maximal Overlap Discrete Wavelet Transformation (MODWT)

Wavelet decomposition is a widely used technique ideal for the multi-resolution analysis of complex time series, allowing simultaneous decomposition in time and frequency space (Das et al., 2018; Ghosh et al., 2021). We have invoked MODWT to execute the said decomposition for component-wise predictive analysis. MODWT has been preferred over conventional discrete wavelet transformation (DWT) as dyadic restriction and sensitivity to the circular shift of the latter are not applicable to the former.

The technique transforms a given function into a mother and father wavelets at a defined scale. If is the i-th time series value at time t, then the detail () and approximation () can be written as follows:where , , , and .

The inverse wavelet transform iswhere is the discrete wavelet transform of .

The decomposition can swiftly disentangle the linear and nonlinear parts of original time series observations into manageable subseries, preserving the fundamental features of financial time series such as spillovers, heteroscedasticity, volatility clustering, etc. (Das et al., 2018; Jana et al., 2021; Singh et al., 2019). The ‘haar’ wavelet transform has been used to implement MODWT to carry out four decomposition levels as it is memory efficient, fast, and precisely reversible with no edge effects. As a result, one approximation (S) and four detailed components (D, D, D, and D) are generated. The RF is invoked to perform a forecasting task at each granular component to derive the final forecasts. Mathematically, the final forecast is derived using (3).where represents the final forecast obtained from RF for a particular variable i; accounts for the obtained forecasts by RF on detailed components, and reflects the forecasted figure at approximation component by RF.

Random forest (RF)

The RF (Breiman, 2001) is an ensemble machine learning algorithm capable of handling complex nonlinear and volatile patterns to carry out classification and regression tasks. It has been found to be extremely resilient to outliers and overfitting. The algorithm is relatively faster than classical bagging-based ensemble modeling and better than the adaptive boosting procedure in terms of accuracy (Breiman, 2001). It relies upon the bootstrapping process to construct decision trees as a base learner to classify regression tasks. Classical regression trees are used as base learners in this algorithm. RF utilizes a random sub-spacing principle in growing the trees wherein the best feature to perform branching operations is selected from a random subset of features. The aforesaid setup of randomization eventually results in an unbiased estimation. The operational steps can be found in Breiman (2001).

Performance evaluation

We have resorted to five distinct metrics to ascertain the quality of predictions as briefly mentioned next:

Mean Absolute Error (MAE) , where and are predicted and actual values for the i-th observation. A lower MAE value implies a superior prediction.

Root Mean Squared Error (RMSE) , where and are predicted and actual values for the i-th observation. A lower RMSE value implies a superior prediction.

Nash–Sutcliffe Efficiency (NSE) , where represents the mean of actual observations. NSE values lie between to 1. The NSE value should be close to 1 for superior forecast accuracy.

Index of Agreement (IA) . The IA value should be close to 1 for superior forecast accuracy.

Theil Index (TI) . The TI value should be close to zero for superior forecast accuracy.

Results and discussion

This section systematically presents the main findings of MODWT decomposition of the original time series corresponding to the natural gas futures of the UK and the USA, feature importance and impact of Error as a feature, predictive modeling exercise, and a comparative performance assessment.

MODWT decomposition

We decompose the natural gas futures time series in pre-and during COVID-19 periods using the MODWT model into four components. The “haar” filter is selected for proving the best decomposed components. The decomposition results are presented in Fig. 2a–d. In each figure, observations are shown along the horizontal axis, and absolute values are given along the vertical axis of respective decomposed components.

Fig. 2

MODWT decomposition of natural gas futures

Feature importance and impact of as a feature

We extensively study the feature importance and the impact of the newly introduced residual () in the second stage of forecasting the natural gas prices. For this purpose, the Boruta feature selection algorithm is run for 100 iterations only as we do not find significant changes in the importance of features and their rankings. Tables 2, 3, 4, 5 denote the key findings of the feature selection experiment that include mean of squared residuals and percentage variance explained top three features with their importance and execution time. For each decomposed component of the time series representing the NGF-UK and the NGF-USA in the pre-and during COVI-19 periods, we compare the performance of the of by considering the as not a feature (NF) and as a feature (F).

Table 2

Features importance of the decomposed components of UK-Pre (Pre COVID-19 NGF-UK)

Series	Mean of squared residuals	% Var explained	Feature Importance			Execution Time (sec)
			Feature #1	Feature #2	Feature #3
D1 (F)	0.000045	61.33	Res(42.71)	EMA12 (8.15)	MTM5 (7.82)	0.41
D1 (NF)	0.000114	3.04	LAG1 (8.91)	MTMS (7.84)	WMS (6.56)	0.52
D2 (F)	0.000041	75.9	Res(37.2)	LOSS (20.29)	WMS (16.71)	0.43
D2 (NF)	0.000085	50.58	LOSS (21.62)	WMS (17.33)	LAG1 (13.89)	0.36
D3 (F)	0.000031	87.03	Res(24.18)	LAG1 (20.48)	WMS (15.48)	0.49
D3 (NF)	0.000044	81.69	LAG1 (21.90)	LOSS (16.61)	WMS (15.88)	0.37
D4 (F)	0.000017	95.74	LAG1 (19.76)	Res(17.77)	B5 (16.46)	0.36
D4 (NF)	0.000020	94.73	LAG1 (19.23)	LOSS (16.06)	B5 (14.61)	0.36
S4 (F)	0.000054	99.88	MTM10 (10.53)	B10 (9.63)	LAG1 (7.87)	0.37
S4 (NF)	0.000056	99.87	MTM20 (9.47)	MTM5 (9.33)	LAG1 (8.86)	0.34

Table 3

Features importance of the decomposed components of UK-During (During COVID-19 NGF-UK)

Series	Mean of squared residuals	% Var explained	Feature Importance			Execution Time (sec)
			Feature #1	Feature #2	Feature #3
D1 (F)	0.000049	56.29	Res(43.39)	UB (7.86)	EMA10 (6.11)	0.47
D1 (NF)	0.000133	18.06	UB (10.29)	EMA10 (7.48)	EMA12 (7.35)	0.50
D2 (F)	0.000091	51.93	Res(27.57)	WMS (11.96)	LAG1 (11.07)	0.36
D2 (NF)	0.000144	23.71	LAG1 (14.05)	WMS (12.04)	EMA5 (8.75)	0.38
D3 (F)	0.000062	75.20	Res(20.98)	LAG1 (17.64)	MTM5 (12.46)	0.44
D3 (NF)	0.000078	68.73	LAG1 (19.87)	EMA5 (12.50)	B5 (12.18)	0.42
D4 (F)	0.000017	97.20	LAG1 (18.18)	Res(17.87)	MTM5 (13.49)	0.36
D4 (NF)	0.000019	96.72	LAG1 (18.37)	MTM5 (14.74)	LAG2 (12.85)	0.36
S4 (F)	0.000101	99.78	LAG1 (12.68)	LAG2 (12.21)	ROC20 (11.28)	0.33
S4 (NF)	0.000101	99.78	LAG1 (12.20)	LAG2 (11.75)	EMA5 (11.11)	0.34

Table 4

Features importance of the decomposed components of US-Pre (Pre COVID-19 NGF-USA)

Series	Mean of squared residuals	% Var explained	Feature Importance			Execution Time (sec)
			Feature #1	Feature #2	Feature #3
D1 (F)	0.003222	33.03	Res(20.76)	MACD (5.35)	MA20 (4.81)	0.52
D1 (NF)	0.003923	25.46	LAG5 (5.81)	EMA12 (4.76)	MACD (4.44)	0.62
D2 (F)	0.001165	68.57	Res(27.77)	LAG1 (13.89)	EMA5 (10.94)	0.42
D2 (NF)	0.002380	35.79	LAG1 (15.37)	EMA5 (12.82)	WMS (12.27)	0.48
D3 (F)	0.000822	85.56	LAG1 (19.39)	Res(18.46)	EMA5 (12.47)	0.39
D3 (NF)	0.001133	80.10	LAG1 (20.72)	EMA5 (13.52)	MTM10 (12.64)	0.52
D4 (F)	0.000661	94.74	LAG1 (19.39)	LAG2 (13.81)	EMA5 (12.50)	0.37
D4 (NF)	0.000796	93.67	LAG1 (19.98)	LAG2 (15.33)	EMA5 (12.69)	0.37
S4 (F)	0.001700	99.43	LAG1 (13.24)	Res(13.14)	LAG2 (11.82)	0.34
S4 (NF)	0.001964	99.34	LAG1 (13.20)	LAG2 (13.01)	EMA5 (11.21)	0.34

Table 5

Features importance of the decomposed components of US-During (During COVID-19 NGF-USA)

Series	Mean of squared residuals	% Var explained	Feature importance			Execution Time (sec)
			Feature #1	Feature #2	Feature #3
D1 (F)	0.000575	64.74	Res(40.61)	LAG1 (6.60)	LAG2 (6.56)	0.44
D1 (NF)	0.001687	23.43	LAG2 (7.74)	LAG5 (7.46)	LAG1 (7.04)	0.45
D2 (F)	0.000596	77.62	Res(35.80)	LOSS (22.42)	WMS (15.07)	0.39
D2 (NF)	0.001317	50.53	LOSS (23.58)	LAG1 (15.17)	WMS (13.90)	0.39
D3 (F)	0.000448	88.69	Res(23.54)	LAG1 (21.49)	LOSS (17.89)	0.36
D3 (NF)	0.000695	82.44	LAG1 (22.52)	LOSS (20.67)	WMS (15.29)	0.36
D4 (F)	0.000289	95.52	LAG1 (20.56)	Res(15.82)	LAG2 (14.27)	0.36
D4 (NF)	0.000341	94.71	LAG1 (20.10)	LOSS (15.39)	MTM5 (14.00)	0.36
S4 (F)	0.000649	99.69	LAG1 (12.24)	LAG2 (11.53)	B20 (11.19)	0.34
S4 (NF)	0.000730	99.65	LAG1 (12.11)	LAG2 (11.26)	EMA5 (11.15)	0.34

For the UK-Pre, has transpired to be extremely important features for detailed components. The turned out to be the most important feature for D1, D2, and D3 and occupied the second spot for D4. These detailed components reflect the high-frequency nonlinear parts of the original series. Thus, the inclusion of is extremely useful for nonlinear modeling. The high quantum of actual variation can be explained by solely. The degree of variation explained by is substantially higher than any other feature. When has not been used, other technical indicators have taken the spot. Among the technical indicators, LAG1 has appeared to be very influential that signifies the dependence of natural gas prices on the immediate past information. On the approximation component, has not turned out to be lying on the top three spots in terms of feature importance.

The features importance UK-During are almost like that of UK-Pre. has appeared to possess significant explanatory power for modeling nonlinear components. It would be interesting to compare whether the actual predictive performance reflects the same phenomenon or not. Among the technical indicators, LAG1 has turned out to be highly important as well.

For the US-Pre, has appeared to be crucial as well. Unlike the UK counterparts, has not appeared to be important for the D4 component. Nevertheless, it has transpired to play a critical role in explaining the variation for the approximation component. Its inclusion increases the overall explained variation component. LAG1 has emerged to be important among the set of technical indicators implying that natural gas futures for the US depend on its immediate past behavior.

Features evaluation of US-During justifies the inclusion of . It is amply apparent that the said feature has appeared to be an extremely impactful feature that largely expounds the overall variability on all high-frequency detailed components. Likewise, in earlier instances, LAG1 has turned out to be a critical technical indicator also. Hence, it is evident that plays a pivotal role in explaining highly nonlinear trends that may be beneficial for prediction exercise in the pandemic time horizon. It becomes imperative to evaluate how the previous stage unexplained behavior can explain the future volatile traits. Natural gas futures of both the UK and USA across the regimes have significantly depended on immediate historical information.

The decomposed series obtained through MODWT contains one approximation (S) and four detailed components (D, D, D, and D). Out of the four detailed components, D1 accounts for the most volatile part (high frequency). It contains the maximum unexplained component compared to the other three detailed components. So, the corresponding to D1 has the maximum importance as a feature used in the second stage and plays the most crucial role in the forecasting process.

Predictive analytics

The proposed two-stage predictive framework utilizes the MODWT, Boruta, and RF algorithms. Time series corresponding to the NGF-UK and the NGF-USA is initially decomposed using MODWT. In the first stage, 30 independent features are used, whereas, in the second stage is added as the 31st independent feature. In each stage, the final forecast is obtained by summing the component-wise forecasts.

The original daily time series corresponding to all the natural gas futures are randomly portioned into training (70%), testing (15%), and validation (15%) datasets to conduct the predictive exercise. It may be noted here that there are only 367 observations for each time series. Therefore, the training dataset contains 256 observations to build the random forest model. The entire predictive performance is done using the ‘rattle()’ package in R. The number of trees in the model is set to 500, a default value in the package. In this study, we extensively investigate the predictability the natural gas prices in the pre-and during COVID-19 periods. The investigation roadmap is as follows:

We partition each of the NGF-UK and the NGF-USA into two time zones—pre (August 09, 2018, to December 30, 2019) and during (December 31, 2019, to May 20, 2021) COVID-19 periods. So, there are four time series that are denoted by UK-Pre, UK-During, US-Pre, and US-During. We then decompose each of them into five granular components using MODWT. So, we have a total of 20-time series.

We apply the FR algorithm on the mentioned 20-time series to obtain the stage 1 forecasts for UK-Pre, UK-During, US-Pre, and US-During based on the 30 selected features. The final forecast of this stage is obtained by aggregating the forecasts obtained from the granular decomposed components.

We calculate the for all 20-time series and update the feature lists.

Then, we calculate the stage 1 forecasts for all 20 decomposed time series based on initially selected 30 features and the as the additional feature. Following a similar process, as followed in stage 1, the final forecast of the second stage is obtained.

To compare the performance of the proposed two-stage framework, we obtain forecasts from the original UK-Pre, UK-During, US-Pre, and US-During time series without decomposing them and applying the RF algorithm.

Finally, we calculate forecast errors and access the predictive performances for the training and the whole datasets for all the decomposed components, aggregate series, and the original series without decomposition considering. For the first two cases, we compute forecast errors and predictive performances for stage 1 ( not as a feature) and stage 2 ( as a feature). These results are presented in Tables 6, 7, 8. Also, the forecast errors and predictive performances based on the training dataset on MODWT-decomposed individual components for both the stages are shown in the "Appendix" to obtain more insights on the overall predictive performance of the proposed two-stage ensemble machine learning approach.

Table 6

Forecast error and predictive performance based on the training dataset on MODWT-decomposed series

Series	Res as a feature					Res not as a feature
	MAE	RMSE	IA	NSE	TI	MAE	RMSE	IA	NSE	TI
UK-Pre	0.001732	0.002878	0.987615	0.961127	0.005087	0.002518	0.005111	0.968857	0.917194	0.005092
UK-During	0.001943	0.003367	0.986098	0.955955	0.005776	0.002778	0.004201	0.966141	0.910027	0.005665
US-Pre	0.009599	0.018212	0.971798	0.924649	0.01896	0.011984	0.021464	0.957751	0.892595	0.020434
US-During	0.006545	0.010399	0.989794	0.967014	0.013529	0.009634	0.013870	0.97179	0.923665	0.014333

Table 7

Forecast error and predictive performance based on the whole dataset on MODWT-decomposed series

Series	Res as a feature					Res not as a feature
	MAE	RMSE	IA	NSE	TI	MAE	RMSE	IA	NSE	TI
UK-Pre	0.00779	0.01129	0.99927	0.99710	0.01500	0.01244	0.01699	0.99835	0.99343	0.02167
UK-During	0.00842	0.01360	0.99908	0.99633	0.02140	0.01190	0.01814	0.99835	0.99347	0.02721
US-Pre	0.03723	0.06953	0.99610	0.98412	0.02348	0.04691	0.08409	0.99422	0.97766	0.02832
US-During	0.02972	0.04699	0.99760	0.99055	0.01888	0.04580	0.06650	0.99508	0.98075	0.02664

Table 8

Forecast error and predictive performance based on the whole dataset without MODWT-decomposition

Series	MAE	RMSE	IA	NSE	TI
UK-Pre	0.01076	0.01500	0.99871	0.99488	0.01938
UK-During	0.01093	0.01747	0.99848	0.99394	0.02649
US-Pre	0.04339	0.07496	0.99551	0.98225	0.02529
US-During	0.04109	0.06108	0.99585	0.98376	0.02453

We discuss the forecasting performance of the proposed approach based on the forecast errors and predictive performances presented in Tables 6, 7, 8. As expected, the MAE and RMSE values for UK-Pre, UK-During, US-Pre, and US-During are superior for the training dataset with as a feature compared to the MODWT-decomposed series with not as a feature and without MODWT-decomposition. The forecasting errors are minimum for the NGF-UK in the pre-COVID-19 period. The corresponding MAE and RMSE values are 0.001732 and 0.002878, respectively. On the other hand, the forecasting errors are worst for the NGF-USA in the pre-COVID-19 period. Surprisingly, the forecast errors are smaller for the NGF-USA during the during-COVID-19 period compared to the pre-COVID-19 period. However, the forecast errors are smaller for the NGF-UK in the pre-COVID-19 period compared to the during-COVID-19 period.

The predictive performances manifested through IA, NSF, and TI values indicate that the proposed approach is the best. The IA and NSE values are close to 1 for the whole series. For the training dataset, these values are slightly inferior as there are only 256 observations. Smaller TI values indicate superior predictive performance. The best TI values have been obtained for the whole series. As before, these values are marginally less for the training dataset. The predictive performances of the ensemble approach without MODWT decomposition are second best (Table 8). However, both the error components are relatively higher compared to the proposed approach under training and whole datasets.

In terms of the methodology, the forecast errors are least for the proposed MODWT-based ensemble technique with as a feature, followed by the MODWT-based ensemble machine learning technique with not as a feature as evident from Table 6 and 7 on both training and the whole datasets. From Table 6 and Table 8, we find that the proposed approach outperforms the ensemble machine learning technique without integrating MODWT applied to all the four original series in pre-and during-COVID-19 periods. Even the MODWT-based ensemble technique with not as a feature also outperforms the ensemble technique. Another interesting fact is coming out from the comparison of the second part of Tables 7 and 8. The MODWT-based ensemble machine learning technique with not as a feature offers no better performance than the ensemble machine learning technique. The MAE and RMSE values are higher in this case. In summary, we conclude that residual-driven MODWT-based ensemble machine learning approach for forecasting natural gas prices appeared to be the best compared to all other scenarios. This was possible because of the inclusion of of the first stage as a feature in the second stage for forecasting the future movements of natural gas prices.

Interested readers may please refer to the "Appendix" for finding the forecast error and predictive performance based on the training dataset on decomposed components with as a feature and not as a feature (Fig. 3).

Fig. 3

Prediction of natural gas futures. Note. Figure 3 represents the comparison of prediction of Natural gas futures for the UK and USA during the mentioned periods. The x-axis shows the duration, and the y-axis of the figures (a) and (b) shows the natural gas futures for the UK in US$ per MMBtu. The y-axis of figures (c) and (d) shows the NGF-USA in US$ per MMBtu

Table 9 presents the correlation between the predicted and actual values. These values should ideally be close to 1 for a better fit model. In all three cases, the correlation values are very high and almost approach 1. This confirms the appropriateness of the predictive frameworks used for forecasting future values of natural gas future prices. For the proposed approach with as a factor, the correlation value is maximum for the UK-Pre, followed by UK-During, US-During, and US-Pre. Correlation values obtained for the proposed approach are higher than the values obtained for the other two approaches. In particular, the correlation values are slightly inferior for the decomposed series with not as a feature than the series without decomposition. So, the correlation test indicates that the proposed residual-based predictive framework is a better fit than the other two models.

Table 9

Correlation between the actual and predicted values

Series	Series with decomposition and Res as a feature	Series with decomposition and Res not as a feature	Series without decomposition
	Correlation	Correlation	Correlation
UK-Pre	0.99855	0.99671	0.99745
UK-During	0.99817	0.99673	0.99696
US-Pre	0.99252	0.98897	0.99112
US-During	0.99527	0.99042	0.99192

Conclusion

This research proposes a residual-driven ensemble machine learning approach for forecasting natural gas prices. We combine the MODWT decomposition technique and an ensemble machine learning algorithm to achieve the research endeavors. The introduction of the unexplained components in terms of the residuals of the previous stage to build a granular forecasting structure for the next stage is a substantial contribution that resulted in highly accurate predictions. The significant findings of this research are summarized next.

The natural gas futures markets of the UK and US have exhibited persistent trends that suggested the deployment of technical indicators as explanatory features. The feature, has enhanced the quality of forecasts by largely explaining the nonlinear granular components of respective original series. The efficacy of the two-stage forecasting approach is amply evident through the incorporation of as a feature. Despite being developed on a relatively small chunk of samples, the predictive architecture demonstrates the ability to yield high-quality predictions in challenging circumstances. The role of MODWT in improving the accuracy of obtained forecasts is imminent that conforms to previous cognate literature of MODWT based complex financial time series prediction (Jana et al., 2021). The prediction accuracy of NGF-UK in Pre COVID-19 phase has appeared to be better than the COVID-19 phase. On the other hand, the prediction accuracy of the NGF-USA has turned out to be better in the COVID-19 phase than Pre COVID-19 regime. So, there is no clear evidence of the impact of COVID-19 on the natural gas future prices was established through this study.

The most challenging part of this research was to obtain a superior forecast for the natural gas prices in the presence of a limited number of observations using machine learning. The availability of more observations will undoubtedly lead to superior predictive performance. The scope of the present work is limited to the UK and US natural gas futures markets. However, this research opens up the opportunities of applying the proposed residual-driven framework for predictive modeling of any economic and financial time series. The approach will offer more benefits for the time series that are more volatile and complex. The scalability of the presented framework makes it ideal for carrying out the predictive task for time series representing high-frequency intraday data as well. Real-life portfolio management and realignment can be conducted based on future figures. The exercise can separately be performed during normal and new normal timelines using predictive residuals of the previous stage as a key component. The unique feature processing utility described here needs further exploration to predict extremely volatile events, including market crashes.

Table 10

Forecast error and predictive performance based on the training dataset on decomposed components with as a feature

	UK-Pre					UK-During
	D1	D2	D3	D4	S4	D1	D2	D3	D4	S4
MAE	0.001688	0.001639	0.001589	0.001283	0.002458	0.001352	0.001872	0.001718	0.00137	0.003405
RMES	0.003479	0.003045	0.002545	0.001945	0.003377	0.002947	0.004168	0.003346	0.001898	0.004478
IA	0.964902	0.983439	0.992352	0.997445	0.999935	0.974904	0.970036	0.987213	0.998446	0.999891
NSC	0.896923	0.945863	0.972806	0.990303	0.99974	0.922702	0.908185	0.955372	0.993954	0.999564
TI	0.005995	0.005181	0.004815	0.004648	0.004796	0.005409	0.006917	0.005526	0.003323	0.007707

Table 11

Forecast error and predictive performance based on the training dataset on decomposed components with not as a feature

	UK-Pre					UK-During
	D1	D2	D3	D4	S4	D1	D2	D3	D4	S4
MAE	0.003464	0.003016	0.002118	0.00149	0.002502	0.003447	0.003197	0.002234	0.001561	0.003451
RMES	0.005337	0.004522	0.003089	0.002152	0.003463	0.005111	0.005293	0.003932	0.002076	0.004592
IA	0.898186	0.960623	0.988674	0.996868	0.999932	0.902358	0.9482	0.982125	0.998136	0.999885
NSC	0.757503	0.880691	0.959922	0.988126	0.999727	0.767536	0.851947	0.938352	0.992761	0.999541
TI	0.005644	0.005045	0.00476	0.005099	0.004914	0.005701	0.00633	0.005159	0.003237	0.0079