Min-max approach for comparison of univariate normality tests.

Comparison of normality tests based on absolute or average powers are bound to give ambiguous results, since these statistics critically depend upon the alternative distribution which cannot be specified. A test which is optimal against a certain type of alternatives may perform poorly against other alternative distributions. Thus, an invariant benchmark is proposed in the recent normality literature by computing Neyman-Pearson tests against each alternative distribution. However, the computational cost of this benchmark is significantly high, therefore, this study proposes an alternative approach for computing the benchmark. The proposed min-max approach reduces the calculation cost in terms of computing and estimating the Neyman-Pearson tests against each alternative distribution. An extensive simulation study is conducted to evaluate the selected normality tests using the proposed methodology. The proposed min-max method produces similar results in comparison with the benchmark based on Neyman-Pearson tests but at a low computational cost.

1. Introduction

Normality of the data is the underlying distributional assumption of multitude of statistical procedures and estimation techniques. In both cross-sectional and time series data, assuming the data normality without testing may affect the accuracy of the econometric inference [1]. Statistical inference from regression models applied to time series [2], categorical [3] and count data [4] depends crucially on the assumption of normal errors. The experimental data sets generated in clinical chemistry for the construction of population reference ranges require the assumption of normality [5]. In short, normality assumption of the given data is the key to validate the inferences made from regression models and other statistical procedures. Diagnostic tests for normality are important as Blanca et al. [6], find only 5.5 percent of the 693 real data distributions close to normality while considering skewness and kurtosis together.

Given the importance of the subject, literature has produced a plethora of goodness-of-fit tests to detect departures from normality [7-13]. With the development of several normality tests over the decades, power comparison of these statistics has been given the due consideration in literature in search of the best test thus helping the researchers in the choice of suitable normality test [14-19]. Different characteristics of normal distribution are exploited while developing normality statistics consequently the power of normality tests varies, depending upon the nature of non-normality [19]. Thus, one normality statistic may perform well for one alternative distribution and another for another alternative non-normal distribution [18]. Comparison of normality tests via simulations are bound to give ambiguous results, since these statistics critically depend upon the alternative distribution which cannot be specified.

This study rests on the finding that one normality test is optimal against one alternative and another for another alternative distribution [20]. The best test’s performance against each alternative distribution provides us the benchmark for comparison of normality tests by using the max-min criterion. Maximum deviations of all selected tests from the benchmark is computed and the test with minimum deviation is ranked as best. This method reduces the calculation burden in terms of computing and estimating the Neyman-Pearson test against each alternative distribution for the benchmark as proposed in [18]. Another problem is that the alternative space is infinite dimensional. Since we plan to use numerical methods, we must narrow this space down to something sufficiently small to permit exploration by numerical methods. At the same time, the space should be large enough to provide a good approximation to the full space of alternatives–failing that, it should be large enough to approximate the distributions conventionally used in simulations studies. First and second order departures from normality depend on the skewness and kurtosis of the distribution, we have used 72 alternatives with wider ranges of these parameters. This alternative space includes mixture of uniform distributions, mixture of t-distributions and the distributions used in the literature [14, 16–18, 21].

2. Normality tests

This section deals with the background and technical details of the selected normality tests. Each of these tests belongs to a different class of normality tests e.g. ECDF, moments, regression and correlation based tests etc.

In the following literature review, we consider x1, x2,….,x as a random sample of size n. Then are the sample mean, variance, skewness and kurtosis respectively, defined as

Where the rth central sample moment is defined as

2.1. Moment tests

2.1.1 The Bowman-Shenton K2-test

Skewness refers to the symmetry of a distribution and Kurtosis refers to the flatness or ‘peakedness’ of a distribution. These two statistics have been widely used to differentiate between distributions.

The distributions of and b2 have been approximated by using the Pearson curves. D’Agostino [22] and Anscombe and Glynn [23] have found the normalizing transformations for and b2 respectively. and Z2(b2) denote the resulting approximate standardized normal variables. These can be computed by the following algorithm provided in [24].

Computational Algorithm for :

Compute as defined in (1) above

Compute

Let k4 be the fourth central moment of then

Compute

Computational Algorithm for Z(b2):

Compute from the sample data.

Compute the mean and variance of b2.

Compute the standardized version of b2,

Compute the third standardized moment of b2 [24],

Compute

Compute

D’Agostino and Pearson [8] proposed a test statistic for testing normality that combines and b2 in the following way: where K is distributed as chi-square with two degrees of freedom. The normality hypothesis is rejected for large values of the test statistic.

2.1.2 The Jarque-Bera test

In the field of economics, the most widely used statistics for normality testing is introduced by Jarque and Bera [10, 11]. It is based on the standardized third and fourth moments: where n is the number of observations, and m is the ith central moment of the observations (i.e. ). Asymptotically, the JB-statistic is distributed as chi-square with two degrees of freedom. The hypothesis of normality is rejected for large values of the test statistic.

2.1.3 The Robust Jarque-Bera test

Gel and Gastwirth [13] introduced robust measures of sample skewness and kurtosis by utilizing a robust measure of dispersion which is less sensitive to outliers, the average absolute deviation from the sample median, and leads to the following robust JB-statistic. where M is the sample median. The RJB statistic asymptotically follows the Chi-square distribution with two degrees of freedom.

2.1.4 The Bonett-Seier test

An alternative measure of kurtosis (G-kurtosis) based on Geary’s [25] test for normality is defined by Bonett and Seier [12] as where σ and τ is the population standard deviation and mean absolute deviation respectively. The factor of 13.29 is used to scale it up to 3 so that it matches standard measure of kurtosis and the . To test G-kurtosis = 3, Bonett & Seier [12] used the following statistic

The above Zw-statistic is approximately distributed as standard normal.

2.2. Distance/ECDF tests

This class of tests deals with the comparison of the empirical cumulative distribution function (ECDF), , which is estimated based on data with the cumulative distribution function of normal distribution, Z. Stephens [26] provided versions of the ECDF tests with unknown mean and variance. ECDF tests can be further classified into those involving either the supremum or the square of the discrepancies, F(x()−Z.

ECDF tests involving the square of the discrepancies are known as those from the Cramér-von Mises family.

2.2.1 The Anderson-Darling A2-test

Anderson-Darling test is, in fact, a modified form of Cramér-von Mises test. It gives more weight to tails of the distribution than does the Cramér-von Mises test. The computational form of the Anderson-Darling statistic is:

A2-test is the most familiar among all ECDF tests. The asymptotic distribution is known and it was found that the critical values for finite samples quickly converge to their asymptotic values for n ≥ 5.

2.2.2 The Kolmogorov-Smirnov test

In the ECDF class of tests, involving the supremum, a well-known statistic is the Kolmogorov-Smirnov test.

The null hypothesis of normality is rejected for large values of the test statistic.

2.2.3 The Za and Zc tests

Zhang & Wu [15] proposed two more likelihood ratio statistics of normality testing to the class of EDF tests. The proposed statistics can be defined as follows:

Let X(1), X(2),.. ..., X(n) are the ordered statistics from a continuous random variable X with distribution function F(x) to be used for the following hypothesis testing setup. where F0(x) = ∅(x)−the cumulative distribution function of the normal distribution.

Null is rejected for large values of the test statistics.

2.3. Regression/correlation tests

2.3.1 The Shapiro-Wilk and Shapiro-Francia tests

Graphically determining the linearity between the ordered observations x( and the expected values of the standard normal ordered statistics, m is known as normal probability plotting. The main idea behind these tests is normal probability plotting. Formally, regression or correlation techniques are used to determine the linearity, hence the name of this group of tests.

The Shapiro and Wilk [27] W statistic is defined as the ratio of two estimates of variance of a normal distribution and can be calculated by

The vector of weights can be computed by where m and V are the mean vector and covariance matrix of the ordered statistics of the standard normal distribution [28]. If the distribution of x is normal, the W-statistic is close to unity otherwise less the unity. The critical values of W are tabulated up to sample sizes of 50. However, Shapiro and Francia [29] noted that as the sample size increases, the ordered observations tends to be independent (i.e. v = 0 for i≠j). Treating V as an identity matrix, W can be extended for n larger than 50 by

Values of {m} are available in [30] up to sample sizes of 400. However, Weisberg and Bingham [31] suggested the following approximation to compute the values of {m}.

It was shown that the approximation works even for the small samples as there is no significant difference between the null distributions of W and W′ statistics. This simplifies the computation of the test statistics.

2.3.2 The Chen-Shapiro test

Chen and Shapiro [32] proposed another competitor of Shapiro-Wilk test based upon the normalized spacing which can be defined as where . ∅−1(.) is the inverse of standard normal distribution. Since the authors have shown a close relationship between the Chen-Shapiro (CS) and the Shapiro-Wilk (W) test, it is therefore expected that the performance of the CS test would be comparable with the W test. The normality hypothesis is rejected for small values of the test statistic.

2.3.3 The COIN test

Coin [33] has proposed a normality test especially for the symmetric non-normal alternatives based on polynomial regression. Let x( be a vector of ordered observations drawn from a normal population with unknown mean, μ and variance, σ2 then it is possible to write where μ and σ are the parameters of the best fit line of a normal Q-Q plot and ε is a vector of errors which are assumed to be homoscedastic. The above two parameters may be estimated by using the Least Square method. Instead of using the above model, COIN proposed the following polynomial model where the vector of ordered observations, x( has been replaced by z( −a vector of ordered standard normal statistics. where β are the fitting parameters and α represents the expected values of standard normal ordered statistics. The estimated value of β3 significantly different from zero implies that the sample is drawn from a symmetric non-normal distribution. However, Coin suggests the use of as a statistic for testing the null hypothesis of normality. Hypothesis of normality is rejected for large values of the test statistic.

2.3.4 The BCMR test

Del Barrio, Cuesta-Albertos, Matrán and Rodríguez-Rodríguez [34] proposed the following test statistic for testing normality based on the L-Wasserstein distance between a sample distribution and the set of normal distributions.

Let x1, x2,… …,x be a random sample drawn from a distribution with the distribution function F. Let Fn denotes the empirical distribution function, ∅ the distribution function of the standard normal law and s2 the sample variance.

This statistic is asymptotically equivalent to Shapiro-Wilk and Shapiro Francia statistics [34]. The normality hypothesis is rejected for large values of the statistic.

2.4. Other tests

2.4.1 The Gel-Miao-Gastwirth test

Recently, Gel and Gastwirth [13] have contributed to the literature of directed tests of normality by proposing a statistic which focuses on detecting heavy tails and outliers of symmetric distributions. The test statistic is simply the ratio of standard deviation to the robust measure of dispersion which should tend to unity under normality of data. where M is the median of the sample data. Normality hypothesis is rejected for large values of the statistic. However, the statistic, √n(R-1) is asymptotically distributed as normal with zero mean and standard deviation equal to . The applications of this test can be extended to light tailed distributions as well by using two-sided test for rejecting the null hypothesis of normality.

3. Alternative distributions

As already stated, to permit exploration by numerical methods we must narrow down the infinite dimensional alternative space to a space large enough to approximate the distributions conventionally used in simulations studies. We have used 72 alternative distributions with wide ranges of skewness and kurtosis as the first and second order departures from normality depend on these parameters. The simulation study considers the distributions used in the literature (Table 1), mixture of uniform distributions, and mixture of t-distributions (Table 2). The alternate space includes all kind of (a)symmetric, short- and long-tailed distributions.

Table 1

General distributions.

Sr. No.	Distribution	Skewness	Kurtosis	Sr. No.	Distribution	Skewness	Kurtosis
1	Beta(4,0.5)	-1.79	6.35	18	Weibull (2,3.4)	0.05	2.71
2	Beta(5,1)	-1.18	4.20	19	Gamma(100,1)	0.20	3.06
3	Beta(2,1)	-0.57	2.40	20	Gamma(15,1)	0.52	3.40
4	Weibull (3,4)	-0.09	2.75	21	Beta (2,5)	0.60	2.88
5	Beta(0.5,0.5)	0.00	1.50	22	Weibull (1,2)	0.63	3.25
6	Beta(1,1)	0.00	1.80	23	Gamma(9,1)	0.67	3.67
7	Tukey(2)	0.00	1.80	24	Chi2 (10)	0.89	4.20
8	Tukey(0.5)	0.00	2.08	25	Gamma (5,1)	0.89	4.20
9	Beta (2,2)	0.00	2.14	26	Gumbel (1,2)	1.14	5.40
10	Tukey(5)	0.00	2.90	27	Chi2 (4)	1.14	6.00
11	Tukey(0.14)	0.00	2.97	28	Gamma (3,2)	1.15	5.00
12	t(10)	0.00	4.00	29	Gamma (2,2)	1.41	6.00
13	Logistic (0,2)	0.00	4.20	30	Chi2 (2)	2.00	9.00
14	Tukey (10)	0.00	5.38	31	Weibull (0.5,1)	2.00	9.00
15	Laplace (0,1)	0.00	6.00	32	Chi 2 (1)	2.83	15.00
16	t(4)	0.00	--	33	LN (0,1)	6.18	113.90
17	t(2)	0.00	--	34	Cauchy (0,1)	--	--

Table 2

Mixture of uniform & t-distributions.

	Uniform Distributions					t-distributions
Sr.	U1	U2	λ	Skew	Kurt	t1	t2	λ	Skew	Kurt
1	(-8, -2)	(0, 4)	0.2	-1.31	3.66	(10, 3)	(5, 50)	0.5	0.00	1.01
2	(1, 2)	(3, 5)	0.1	-1.18	4.11	(100, -4)	(75, 4)	0.5	0.00	1.23
3	(-2, 1)	(0, 2)	0.1	-0.94	4.42	(100, 4)	(75, 6)	0.5	0.00	2.53
4	(-8, -2)	(0, 4)	0.3	-0.86	2.37	(8, 5)	(10, 3)	0.5	0.04	3.02
5	(-2, 1)	(0, 2)	0.3	-0.75	3.01	(5, 2)	(7, 4)	0.7	0.09	4.95
6	(1, 2)	(3, 5)	0.3	-0.47	1.79	(10, 5)	(5, 7)	0.5	0.16	4.20
7	(-2, 1)	(0, 2)	0.5	-0.40	2.26	(100, 4)	(75, 6)	0.7	0.27	2.77
8	(-2, -4)	(-1, 4)	0.37	0.00	1.62	(8, 5)	(10, 3)	0.1	0.30	3.95
9	(-2, -1)	(-1, 2)	0.25	0.00	1.80	(8, 5)	(10, 3)	0.2	0.32	3.57
10	(2, 5)	(4, 10)	0.229	0.00	1.93	(100, -4)	(75, 4)	0.9	0.36	3.39
11	(2, 5)	(4, 8)	0.36	0.00	2.05	(10, 5)	(5, 7)	0.9	0.38	4.65
12	(-10, 0)	(-5, 8)	0.296	0.00	2.13	(100, -4)	(75, 4)	0.7	0.78	1.93
13	(-2, 1)	(0, 2)	0.1	0.09	2.07	(5, 10)	(7, 25)	0.7	0.82	1.83
14	(1, 2)	(3, 5)	0.5	0.20	1.44	(10, 3)	(5, 50)	0.7	0.87	1.77
15	(1, 2)	(3, 5)	0.7	0.96	2.38	(8, 0)	(12, 5)	0.9	1.31	5.11
16	(-8, -2)	(0, 4)	0.9	1.11	4.05	(8, 0)	(12, 5)	0.95	1.32	6.63
17	(1, 2)	(3, 5)	0.9	2.35	8.08	(8, -1)	(12, 5)	0.9	1.58	5.60
18	(1, 2)	(3, 5)	0.95	3.08	14.16	(8, -10)	(12, 5)	0.9	1.78	6.02
19	--	--	--	--	--	(5, 10)	(7, 25)	0.9	2.36	7.35
20	--	--	--	--	--	(10, 3)	(5, 50)	0.9	2.64	8.06

The mixtures of t- & uniform distributions are generated by the following rules. where, ν & η (i = 1,2) are the degrees of freedom and the means of the respective t-distributions and a & b (i = 1,2) are the bounds of uniform distributions.

4. Simulation study

An extensive simulation study is conducted in the following to estimate the size and power of the selected normality tests. First, exact critical values are obtained for samples of size 25, 50 & 75 from normal distribution, at 0.05 percent level of significance on the basis of 100, 000 Monte Carlo simulations in MATLAB R2013a. Second, powers of the fourteen normality statistics are computed against general distributions, mixture of uniform and t- distributions.

As stated earlier, no normality test can be uniformly most powerful against all alternative distributions, one test is optimal for one alternative and another is optional for another alternative. The trajectory of maximum power obtained by any test against each alternative distribution provides us the benchmark against which all tests can be compared. Deviations for each test are computed with reference to the benchmark.

Any function, T(x), which takes values {0. 1} is called hypothesis test. The size of the test is defined as where φ belongs to null space, Φ. The power of the test is the probability of not committing type-II error i.e.,

For any test, maximum achievable power for a given alternative is defined as

For different values of φ, we get different optimal tests statistics. The locus of the powers of these statistics provides us the benchmark. Following loss function is computed to evaluate each normality test in terms of its deviation from the benchmark.

A test with minimum loss or deviation is defined as the best test. The most stringent test will have zero percent loss or deviation from the benchmark. This allows us to rank the normality tests in a unique manner.

5. Results & discussion

Normality tests are evaluated against 72 alternative distributions including mixture of uniform distributions (18 distributions), mixture of t-distributions (20 distributions) and the distributions used in the literature (34 distributions) with wide ranges of skewness and kurtosis. The alternate space includes all kind of (a)symmetric, short (long)-tailed distributions. The most stringent test is the one with minimum deviation from the benchmark.

While evaluating the losses or deviations of normality statistics against the selected alternative space, CS test outperforms the remaining tests for small (n = 25) and medium (n = 50) sample sizes at 5 percent level of significance with 12.3 & 17.3 percent respective deviations from the benchmark (Table 3). These results corroborate with the findings in [17, 18]. Shapiro-Wilk’s W-test is the first ranked statistic for large sample size (n = 75) with 27.4 percent deviation closely followed by CS, Z, & Z statistics. For third rank, this study recommends BCMR, A, & Z, tests for small and BCMR for medium and large sample sizes. The JB and RJB tests perform poorly with more than 90 percent losses for all sample sizes which is in line with the findings in [18].

Table 3

Ranking of tests at α = 0.05.

n = 25			n = 50			n = 75
Test	Rank	Loss	Test	Rank	Loss	Test	Rank	Loss
CS	1	12.3%	CS	1	17.3%	W	1	27.4%
W	2	15.5%	W	2	20.5%	Zc	2	30.0%
BCMR	3	20.6%	Zc	2	20.5%	CS	2	30.6%
A2	3	21.5%	Za	2	21.0%	Za	2	32.5%
Za	3	22.0%	BCMR	3	24.3%	BCMR	3	34.0%
Zc	4	24.8%	A2	4	33.4%	A2	4	38.5%
W′	5	32.8%	W′	5	44.7%	W′	5	49.8%
K2	6	59.9%	K2	6	67.8%	R	6	85.2%
R	7	76.5%	R	7	75.7%	JB	6	85.9%
Zw	7	78.3%	Zw	8	88.3%	Zw	7	88.3%
COIN	8	85.5%	JB	8	89.9%	K2	8	96.3%
KS	9	95.4%	COIN	9	96.2%	KS	8	97.1%
JB	9	97.3%	KS	9	96.8%	COIN	8	97.4%
RJB	9	98.2%	RJB	10	100.0%	RJB	9	100.0%

These results clearly indicate that the min-max strategy adopted in this study produces similar results as achieved with Neyman-Pearson benchmark in Islam (2017). Furthermore, the computational cost reduces significantly.

It is interesting to note that symmetric short- and long-tailed alternative distributions are the worst alternatives for both the top ranked statistics, CS and W, in terms of maximum deviations from the benchmark for all sample sizes (Table 4). The W-test also outperforms the A-test with significant lesser deviations from the benchmark which is in line with the findings in [35].

Table 4

Worst alternatives for CS & W test.

Distribution	Skewness	Kurtosis	CS-Loss	W-Loss
	n = 25
U(-2,-1)*0.25+t(-1,2)*0.75	0.00	1.80	11.3%	15.0%
Tukey(2)	0.00	1.80	11.5%	15.5%
Beta(1,1)	0.00	1.80	11.6%	15.5%
Laplace (0,1)	0.00	6.00	12.3%	10.8%
	n = 50
U(-2,-1)*0.25+t(-1,2)*0.75	0.00	1.80	10.2%	16.3%
Tukey(2)	0.00	1.80	10.0%	16.1%
Beta(1,1)	0.00	1.80	10.1%	16.4%
U(2,5)*0.229+t(4,10)*0.771	0.00	1.93	10.2%	17.4%
Beta (2,2)	0.00	2.14	12.2%	16.4%
Tukey(5)	0.00	2.90	12.9%	11.8%
t(4)	0.00	--	13.0%	9.9%
Tukey(0.5)	0.00	2.08	15.2%	20.5%
Laplace (0,1)	0.00	6.00	17.3%	13.5%
	n = 75
Logistic (0,2)	0.00	4.20	13.2%	10.0%
t(4)	0.00	--	13.7%	10.1%
Tukey(5)	0.00	2.90	15.2%	14.2%
Laplace (0,1)	0.00	6.00	17.2%	12.5%
Beta (2,2)	0.00	2.14	19.3%	25.5%
Tukey(0.5)	0.00	2.08	19.8%	27.4%
U(-10,0)*0.296+U(-5,8)*0.704	0.00	2.13	30.6%	7.60%

These two statistics along with the W′, BCMR & COIN tests belongs to the ‘regression & correlation’ class of normality tests. The worst alternatives for the rest of the members of this class are test dependent e.g., the worst alternatives for the COIN tests are skewed and the near normal distributions. On balance, when considering the performance of the regression and correlation-based group of normality statistics, CS is the best test (rank#1) for small and medium sample size closely followed by the W test at rank two position. For large samples, the W test outperforms the CS statistics by a margin of 3.2 percent and occupies the first rank position. These two statistics are closely followed by the BCMR test which is placed at rank three position for all sample sizes. The Shapiro-Francia’s test (W′) shows consistent performance by occupying the fifth position for all sample sizes with maximum deviations ranging from 33 to 50 percent.

Moment based JB & RJB tests perform poorly against the short-tailed symmetric and slightly skewed alternatives (Figs 1 and 2). It is pertinent to mention that the performance of JB & RJB is same at medium sample size (n = 50). Other moment based tests under consideration in this study are K & Z. The K statistic is ranked at 6 for small and medium and 8 for large samples sizes with deviations ranging from 60 to 96 percent. The Z test is ranked at 7th position for small and large and at 8th for medium sample sizes with maximum deviations from the benchmark range from 78 to 88 percent.

Fig 1

Worst alternatives for JB & RJB (n = 25) in terms of deviations.

Fig 2

Worst alternatives for JB & RJB (n = 75) in terms of deviations.

Among the normality tests based on empirical cumulative distribution function (ECDF), A, Z and Z occupy the third and fourth rank respectively for small samples. The normality tests proposed by Zhang & Wu (2005), Z & Z, performed well by occupying the second rank with respective maximum losses of 21.0 & 20.5 percent for small and 32.5 & 30.0 percent for large sample sizes. The Anderson-Darling’s statistic (A) is ranked at fourth position for medium and large sample sizes with 33.4 and 38.5 percent losses. Our results corroborate with the findings in Zhang & Wu (2005). Among the ECDF group of normality tests, the KS and COIN tests did not perform well with above 85 percent deviations from the benchmark for all sample sizes. The COIN test has slight edge to the KS statistic in small sample sizes. For medium and large sample sizes, both share the same ranks with losses above 95 percent.

The R statistic introduced by Gel and Gastwirth [13] for symmetric distributions occupies rank 7 for small and medium sample sizes and rank 6 for large sample sizes. Range of the deviation from the benchmark is 76–85 percent when evaluated against the entire class of alternatives. The worst distributions for the R test belong to asymmetric alternative space for the obvious reasons. Interestingly, the R test occupies sixth and fifth ranks for small and medium to large sample sizes respectively when evaluated against the symmetric alternatives (Table 5). The worst distributions for the R test belongs to symmetric short-tailed alnternative space (Fig 3) for all sample sizes. Therefore, R test is not recommended for symmetric short-tailed alternatives. The COIN test perform relatively much better than the R test which is inline with the findings in [17, 33].

Table 5

Ranking of tests against symmetric alternatives.

n = 25			n = 50			n = 75
Test	Rank	Loss	Test	Rank	Loss	Test	Rank	Loss
CS	1	12.3%	CS	1	17.3%	W	1	27.4%
COIN	1	14.1%	COIN	1	19.0%	Zc	2	30.0%
W	2	15.5%	W	2	20.5%	CS	2	30.6%
BCMR	3	20.6%	Zc	2	20.5%	Za	3	32.5%
A2	3	21.5%	Za	2	21.0%	COIN	3	33.9%
Za	3	22.0%	BCMR	3	24.3%	BCMR	3	34.0%
Zw	4	23.1%	Zw	4	26.5%	A2	4	38.5%
Zc	4	24.8%	R	5	28.7%	Zw	4	40.0%
W′	5	32.8%	A2	6	33.4%	R	5	45.8%
R	6	36.1%	W′	7	44.7%	W′	6	49.8%
K2	7	43.9%	K2	8	67.8%	KS	7	85.3%
KS	8	91.6%	JB	9	89.9%	JB	7	85.9%
JB	9	97.2%	KS	9	90.9%	K2	8	96.3%
RJB	9	98.2%	RJB	10	100.0%	RJB	9	100.0%

Fig 3

Worst symmetric alternatives for R test.

While considering symmetric alternative space, the CS & COIN are the best options for testing normality for small to medium sample sizes, the Shapiro-Wilk’s W test is recommended for large sample sizes (Table 5). The W-test occupies second rank for small and medium sample sizes. The moment-based JB & RJB tests performed poorly against the symmetric class of alternatives as well. The worst distributions for these statistics belongs to symmetric and short-tailed class of alterntaives. These results corroborate with the findings in [16, 35, 36]. The Z test perfoms relatively well among the moment-based normality tests and occupies fourth rank for all sample sizes with maximum deviation from the benchmark ranging between 23–40 percent. The K test is ranked at 7th & 8th positions for small and medium to large samples respectively with losses range of 44–96 percent.

When considering the regression and correlation based group of normality tests, the CS, COIN, W & BCMR are the best options against the symmetric alternatives and occupy top three ranks in the table. Romão et al. [17], recommend the CS & W statistics for asymmetric group of alternatives by comparing the absolute powers. However, when these statistics are evaluated against a benchmark instead of absolute powers, these statistics turn out to be best for symmetric alternative distributions as well. The Shapiro-Francia’s (W′) test does not perform well against symmetric alternaives and occupies ranks 5, 7 & 6 for small, medium and large smaple sizes respectively.

Among the ECDF class of normality tests, A & Z occupy the third rank for small samples with 21.5 & 22.0 percent deviations from the benchmark. The Z & Z are recommended for medium and large sample sizes as Anderson-Darling’s statistic (A) occupies sixth and fourth positions for medium and large sample sizes respectively. In terms of maximum deviatoins, the Z has slight edge to Z test for medium and large sample sizes which does not corroborate with the findings in [15]. The KS test does not perform well against the symmetric alternatives with more than 85 percent losses for all sample sizes.

When the selected normality tests are evaluated against the asymmetric class of alternatives, W, Z, & CS tests occupy the rank one position for small, CS for medium, and CS, W, Z & Z for large sample sizes (Table 6). On balance, the CS and W tests from regression and correlation based group of normality tests is recommended for all sample sizes whereas the COIN test did not perform well with very high range of deviations from the benchmark when the alternative distribution is drawn from the asymetric distributional space due to obvious reasons. These findings are corroborate with the findings in [17, 18, 35].

Table 6

Ranking of tests against asymmetric alternatives.

n = 25			n = 50			n = 75
Test	Rank	Loss	Test	Rank	Loss	Test	Rank	Loss
W	1	5.0%	CS	1	6.9%	CS	1	9.0%
Zc	1	5.3%	W	2	9.5%	W	1	10.2%
CS	1	5.4%	Zc	2	10.2%	Zc	1	10.3%
BCMR	2	7.7%	Za	3	12.2%	Za	1	10.5%
Za	2	9.0%	BCMR	4	15.3%	BCMR	2	13.6%
W′	3	15.8%	A2	5	30.7%	A2	3	26.5%
A2	4	21.0%	W′	5	31.4%	W′	4	33.4%
KS	5	49.0%	K2	6	64.9%	K2	5	50.6%
K2	6	59.9%	KS	6	65.5%	KS	6	63.4%
R	7	76.5%	JB	7	72.6%	JB	7	76.2%
Zw	7	78.3%	R	8	75.7%	R	8	85.2%
COIN	8	85.5%	Zw	9	88.3%	Zw	9	88.3%
JB	9	97.3%	COIN	10	96.2%	COIN	10	97.4%
RJB	9	97.7%	RJB	11	99.0%	RJB	11	98.4%

Among the ECDF class of tests, the Z is ranked as number one statistic for small & large sample sizes and number two for medium sample size against the selected asymmetric distributional space closely followed by Z test. Maximum deviations of these tests range from 5 to 12 percent. TheAnderson-Darling test, A, is placed at fourth, fifth, and third positions for small, medium, and large sample sizes, respectively with a range of 21–31 percent maximum deviations from the benchmark. Moment-based tests did not perform well with more than 50 percent maximum deviations from the benchmark for all sample sizes against the asymmetric distributional space.

There is no significant difference between the performances of BCMR and W tests of normality in terms of discriminating the long-tailed distributions (β2 > 3). Both the statistics share first rank when evaluated against the selected class of heavy tailed distributional space (Table 7) closely followed by the CS test. The Shapiro-Francia’s W′ test performed well for small samples and occupies third rank with power loss of 15.1 percent however, from regression and correlation class, the COIN test peforms poorly and occupies the last rank with more than 85 percent power losses at all sample sizes. On balance, in terms of maximum deviations from the benchmark, moment-based normality tests do not perform well (Table 7).

Table 7

Ranking of the tests against long-tailed alternatives.

n = 25			n = 50			n = 75
Test	Rank	Loss	Test	Rank	Loss	Test	Rank	Loss
BCMR	1	8.9%	W	1	13.5%	W	1	12.5%
W	1	10.8%	BCMR	1	15.3%	BCMR	1	13.6%
CS	2	12.3%	CS	2	17.3%	CS	2	17.2%
W′	3	15.1%	A2	2	17.3%	Zc	3	22.0%
Za	3	15.4%	Za	3	19.6%	Za	3	22.9%
A2	3	16.5%	Zc	3	19.7%	A2	4	26.5%
Zc	4	24.8%	W′	4	31.4%	W′	5	33.4%
JB	5	37.8%	K2	4	32.3%	K2	5	35.7%
K2	6	43.9%	KS	5	34.9%	KS	6	43.0%
RJB	6	44.4%	JB	6	51.5%	R	7	70.6%
KS	7	49.0%	RJB	6	51.8%	JB	8	76.2%
R	8	66.4%	R	7	75.7%	RJB	9	78.7%
Zw	9	78.3%	Zw	8	88.3%	Zw	10	88.3%
COIN	10	85.5%	COIN	9	96.2%	COIN	11	97.4%

However, these statistics really performed well when evaluated against the symmetric long-tailed distributions with clear dominance of the RJB test (Table 8). These results are in line with the findings in [18, 33, 36]. Overall, the JB & RJB tests perform well against the long-tailed distributional space except for the alternatives listed in Table 9. Among the ECDF class of normality tests, all the statistics except for KS performed well and are listed among the top four tests for all sample sizes. The R test when evaluated against the thick-tailed alternatives do not perform well and power deviations vary from 66–76 percent (Table 7).

Table 8

Powers of moment-based tests against symmetric long-tailed alternatives.

Distribution	Skewness	Kurtosis	K2	JB	RJB	Zw	Best Test
t(10)	0.00	4.00	0.23	0.26	0.27	0.17	0.27
Logistic (0,2)	0.00	4.20	0.29	0.33	0.35	0.25	0.35
Tukey (10)	0.00	5.38	0.81	0.91	1.00	1.00	1.00
Laplace (0,1)	0.00	6.00	0.63	0.70	0.80	0.80	0.81
t(4)	0.00	…	0.63	0.68	0.71	0.62	0.71
t(2)	0.00	…	0.95	0.96	0.98	0.97	0.98

Table 9

Worse long-tailed alternatives for JB & RJB (deviations in percentages).

Distribution	Skewness	Kurtosis	JB	RJB
	n = 25
Beta(5,1)	-1.18	4.20	35.20%	44.38%
Tukey (10)	0.00	5.38	37.85%	--
U(1,2)*0.9+t(3,5)*0.1	2.35	8.08	24.23%	32.60%
	n = 50
U(-2,1)*0.3+t(0,2)*0.7	-0.75	3.01	50.53%	50.62%
U(-2,1)*0.1+t(0,2)*0.9	-0.94	4.42	51.49%	51.81%
	n = 75
U(-2,1)*0.3+t(0,2)*0.7	-0.75	3.01	76.22%	78.74%
U(-2,1)*0.1+t(0,2)*0.9	-0.94	4.42	72.65%	78.06%

For distributions from the short-tailed alternative space (β2 < 3), we recommend CS & Z for small, CS for medium and W test for large sample sizes (Table 10). Romão et al. [17], also recommends the use of CS & W tests for small and large sample sizes. The W-test is also ranked second for small and medium sample sizes with respective maximum deviations of 15.5 & 20.5 percent from the benchmark. Performance of the W test is much better than the KS test irrespective of the fact that the alternative belongs to short- or long-tailed distributional space which corroborates with the findings in [18, 35]. Both the Z & Z statistics are among the top three positions with Z having a slight edge to Z against the short-tailed alternatives which is in line with the findings in [18]. Anderson-Darling test statistic (A) also performs well and occupies third & fourth ranks for small and medium & large samples, respectively. Based on the maximum deviations from the benchmark, BCMR test is placed at rank three with respective power losses of 20.6, 24.3, & 34.0 percent. Among the correlation and regression-based normality tests, the COIN test could not perform well when evaluated separately both for short- and long-tailed alternatives. Performance of the JB, RJB, K, & Z is not up to the mark with very high-power deviations which corroborates with the findings in [36].

Table 10

Ranking of the tests against short-tailed alternatives.

n = 25			n = 50			n = 75
Test	Rank	Loss	Test	Rank	Loss	Test	Rank	Loss
CS	1	11.6%	CS	1	15.2%	W	1	27.4%
Zc	1	12.1%	W	2	20.5%	Zc	2	30.0%
W	2	15.5%	Zc	2	20.5%	CS	2	30.6%
BCMR	3	20.6%	Za	2	21.0%	Za	2	32.5%
A2	3	21.5%	BCMR	3	24.3%	BCMR	3	34.0%
Za	3	22.0%	A2	4	33.4%	A2	4	38.5%
W′	4	32.8%	W′	5	44.7%	W′	5	49.8%
KS	5	48.9%	K2	6	67.8%	KS	6	63.4%
K2	6	59.9%	Zw	7	73.1%	Zw	7	82.2%
Zw	7	62.3%	R	7	74.9%	R	8	85.2%
COIN	8	71.4%	KS	8	78.4%	JB	8	85.9%
R	9	76.5%	COIN	8	80.0%	COIN	9	92.7%
JB	10	97.3%	JB	9	89.9%	K2	10	96.3%
RJB	10	98.2%	RJB	10	100.0%	RJB	11	100.0%

Table 11 presents the top five damaging distributions for each normality test at samples of size 25, 50, & 75. It is evident from the results that ECDF based normality tests suffer more against the symmetric short-tailed and symmetric long-tailed distributions with significant outliers. Symmetric short-tailed and skewed distributions affect the performance of normality tests belong to regression and correlation class. However, the most damaging distributions for the moment based normality tests are specific to individual test in this class. For example, the JB & RJB tests suffer greater power loss against the long-tailed alternatives at small, negatively skewed alternatives at medium and large sample sizes.

Table 11

Top five damaging distributions for normality tests.

Sr.	Skew	Kurt	KS	CS	A2	Za	Zc	K2	JB	RJB	Zw	W	W’	COIN	BCMR	R
D1	0.00	1.62							β	βγ			α		α
D2	0.00	1.80	Βγ	α	αβ	α	α		β	γ		αβ	αβ		αβ
D3	0.00	1.93				β	β		γ			β	βγ		αβ
D4	0.00	2.05	Γ		βγ				γ				βγ	α	γ
D5	0.00	2.13		γ									γ		γ
D6	-1.31	3.66												βγ
D8	-0.94	4.42							γ
D9	-0.86	2.37						α								αβ
D10	-0.75	3.01							γ
D11	-0.47	1.79						α
D12	-0.40	2.26						βγ	γ							γ
D13	0.09	2.07	Αβγ		αβ						βγ			γ		βγ
D14	0.20	1.44							α	αβ
D15	0.96	2.38						α	α	α						αβγ
D17	2.35	8.08	Α		α						αβγ			αβγ		α
D19	0.00	1.50	Αβ						α	αβγ			α
D20	0.00	1.80	Αβγ	α	αβ	α	α		β	γ		αβ	αβ		αβ
D21	0.00	1.80	Γ	α	αβ	α	α		β	γ		α	αβ		αβ
D22	0.00	2.08		βγ	γ	βγ	βγ					βγ	γ		γ
D23	0.00	2.14		βγ	γ	βγ	βγ					βγ	γ		βγ
D24	0.00	2.90		βγ		βγ	βγ					γ			γ
D26	0.00	4.00						α
D28	0.00	5.38				α	α	β
D29	0.00	6.00		αβγ		βγ	αβγ					αγ
D30	0.00	..		β		γ
D31	0.00	..		α			γ					α
D32	..	..						α
Sr.	Skew	Kurt	KS	CS	A2	Za	Zc	K2	JB	RJB	Zw	W	W’	COIN	BCMR	R
D33	-1.79	6.35									α			αβ
D34	-1.18	4.20									αβγ			βγ		αβ
D35	-0.57	2.40		β					βγ	β						βγ
D40	0.60	2.88			γ			βγ				γ				γ
D41	0.63	3.25			γ
D46	1.14	6.00									βγ			β
D47	1.15	5.00												γ
D48	1.41	6.00									βγ
D49	2.00	9.00									α			α
D50	2.00	9.00									α			α
D53	0.00	1.01						βγ	α	α
D54	0.00	1.23						γ		β
D55	0.87	1.77														α

Note: α, β, and γ represent the damaging distributions at samples of size 25, 50, and 75 respectively. Di represent the distributions presented in Tables 1 and 2.

6. Conclusion

Comparison of normality test without having an invariant benchmark has not been proven fruitful in the normality literature. This study proposes an alternative way to compute the benchmark instead of the Neyman-Pearson test-based benchmark proposed in literature. The proposed benchmark is based on the min-max approach which reduces the calculation cost in terms of computing and estimating the Neyman-Pearson tests against each alternative from the selected distributional space. The min-max approach is based on the finding that one test is best against one alternative and another for another alternative [20]. Thus, against each alternative distribution, we get different optimal normality tests. The locus of these statistics provides us the benchmark. Maximum deviations from the benchmark are computed for the selected normality statistics. A test with minimum loss or deviation is defined as the most stringent test. An extensive simulation study is conducted to rank the selected normality tests against a vast distributional space consisting of mixture of uniform distributions, mixture of t-distributions, and distributions used in literature.

General recommendations derived from the analysis of maximum deviations from the benchmark indicate the most stringent normality test is CS for small (n = 25), medium (n = 50), and Shapiro-Wilk’s W-test for large sample size (n = 75) closely followed by CS, Z, & Z statistics against the entire alternative space. While considering symmetric alternative space, the CS & COIN are the best options for testing normality for small to medium sample sizes, and the Shapiro-Wilk’s W test for large sample sizes. When the selected normality tests are evaluated against the asymmetric class of alternatives, W, Z, & CS tests occupy the rank one position for small, CS for medium, and CS, W, Z & Z for large sample sizes.

There is no significant difference between the performances of BCMR and W tests of normality in terms of discriminating the long-tailed distributions (β2 > 3). Both the statistics share first rank when evaluated against the selected class of heavy tailed distributional space closely followed by the CS test. For distributions from the short-tailed alternative space (β2 < 3), we recommend CS & Z for small, CS for medium and W test for large sample sizes.

16 Jun 2021

PONE-D-21-14331

An Alternative Approach for Comparison of Univariate Normality Tests

PLOS ONE

Dear Dr. Islam,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Jul 31 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Saeed Mian Qaisar, Ph.D.

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1.Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Upon re-submitting your revised manuscript, please upload your study’s minimal underlying data set as either Supporting Information files or to a stable, public repository and include the relevant URLs, DOIs, or accession numbers within your revised cover letter. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories. Any potentially identifying patient information must be fully anonymized.

Important: If there are ethical or legal restrictions to sharing your data publicly, please explain these restrictions in detail. Please see our guidelines for more information on what we consider unacceptable restrictions to publicly sharing data: http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions. Note that it is not acceptable for the authors to be the sole named individuals responsible for ensuring data access.

We will update your Data Availability statement to reflect the information you provide in your cover letter

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: No

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: I found the paper very interesting and opening up a new debate on Normality testing. On balance, this is a technically sound piece of research which explores the power properties of normality tests using a diversified alternative distributional space. The data generation, simulation design, and finite sample properties are handled well. The proposed min-max approach definitely reduces the calculation cost in terms of computing power envelope which is the contribution of this paper. However, I have the following minor comments:

1. In section 2, distribution of the RJB test should be mentioned as well.

2. In table 3 & 5, how the ranks are defined for tests with small differences in losses?

3. Fig. 1 & 2 highlight the deviations of JB and RJB test against the worst alternatives at sample of sizes 25 and 75. Do the tests behave alike at sample of size 50? If yes, add this information as footnote or in the main text.

4. Table numbers are not aligned, e.g., there are two tables titled as table 5.

5. It is evident from literature that the JB and RJB tests behave well against long-tailed alternatives. This study should acknowledge the same (as it is evident from this study results as well) and particularly mention that there are only few long-tailed worst alternatives which should be taken care of while testing normality assumption by applying the JB and RJB tests.

Reviewer #2: The paper presents an extensive simulation study on normality tests. It could be useful for researchers in order to select the most appropriate normality test. I have two suggestions to the authors:

1. More details on the simulations should be disclosed. For example, which software do you use? Additionally, in order to facilitate the reproduction of the study you should provide data. If you generate your distributions with Python, R, Matlab, etc, you could include the code as a supplementary material or upload at a repository such as Github.

2. Probably you could complement your results using a classification or a clustering algorithm. In other words, which features of the distribution influences the loss function the most? Do some tests perform better at detecting non normality against a Beta rather than against a Laplace distribution? You could use Self Organizing Maps or K-means or many other techniques in order to produce such representation. IN this way instead of a ranking such as in tables 5,6,7, you could produce a ranking that informs on the different dimensions that affect the normality detection.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Asad ul Islam KHAN

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

26 Jun 2021

Saeed Mian Qaisar, Ph.D.

Academic Editor

PLOS ONE

Manuscript ID: PONE-D-21-14331

Dear Dr. Saeed Mian Qaisar,

Thank you for inviting me to submit a revised draft of our manuscript entitled, "Min-Max Approach for Comparison of Univariate Normality Tests" to PLOS ONE. I also appreciate the time and effort you and each of the reviewers have dedicated to providing insightful feedback on ways to strengthen my paper. Thus, it is with great pleasure that I resubmit my article for further consideration. I have incorporated changes that reflect the detailed suggestions you have graciously provided. I also hope that my edits and the responses I provide below satisfactorily address all the issues and concerns you and the reviewers have noted.

To facilitate your review of my revisions, the following is a point-by-point response to the questions and comments delivered in your letter dated June 16 2021.

Journal Requirements:

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming.

Response: I have followed all the style requirements including the file names.

2. Upon re-submitting your revised manuscript, please upload your study’s minimal underlying data set as either Supporting Information files or to a stable, public repository and include the relevant URLs, DOIs, or accession numbers within your revised cover letter.

Response: This is a simulation based study and all the codes required to generate data or replicate the study are deposited on the Github as suggested by one of the reviewer. The URL of the file is included in the revised cover letter.

Reviewer # 1 Comments:

I appreciate the time and energy the anonymous reviewer committed and provided me the valuable feedback for the improvement of the manuscript.

1. In section 2, distribution of the RJB test should be mentioned as well.

Response: The distribution is mentioned on pg#7, line# 132-33.

2. In table 3 & 5, how the ranks are defined for tests with small differences in losses?

Response: The ranking changes if the difference in losses is more than 2%.

3. Fig. 1 & 2 highlight the deviations of JB and RJB test against the worst alternatives at sample of sizes 25 and 75. Do the tests behave alike at sample of size 50? If yes, add this information as footnote or in the main text.

Response: Thanks for highlighting the missing information. Yes, the tests behave alike at sample of size 50 as well and this information is added in main text at pg#21, line# 335-36.

4. Table numbers are not aligned, e.g., there are two tables titled as table 5.

Response: This typo is corrected on pg# 25, line# 397.

5. It is evident from literature that the JB and RJB tests behave well against long-tailed alternatives. This study should acknowledge the same (as it is evident from this study results as well) and particularly mention that there are only few long-tailed worst alternatives which should be taken care of while testing normality assumption by applying the JB and RJB tests.

Response: Thank for the suggestion! I have acknowledged the same on pg# 32, line# 463-64.

Reviewer # 2 Comments:

I am thankful to you for providing me the valuable comments for the value addition to my work.

1. More details on the simulations should be disclosed. For example, which software do you use? Additionally, in order to facilitate the reproduction of the study you should provide data. If you generate your distributions with Python, R, Matlab, etc, you could include the code as a supplementary material or upload at a repository such as Github.

Response: Thanks for providing guidance regarding the repository like Github. I have uploaded the codes required to generate the data for simulations and code for the replication of the study as well. The URL of the file is included in the revised cover letter. The software information is added to the manuscript at pg# 16, line# 275.

2. Probably you could complement your results using a classification or a clustering algorithm. In other words, which features of the distribution influences the loss function the most? Do some tests perform better at detecting non normality against a Beta rather than against a Laplace distribution? You could use Self Organizing Maps or K-means or many other techniques in order to produce such representation. In this way instead of a ranking such as in tables 5,6,7, you could produce a ranking that informs on the different dimensions that affect the normality detection.

Response: Thanks again for the valuable comment, it helped me to improve my analysis of results. I have highlighted the top-five damaging distributions for each selected normality test at all sample sizes in table 8. This would help researchers to choose the best normality test based on the data distribution in hand.

Again, thank you for giving me the opportunity to strengthen my manuscript with your valuable comments and queries. I have worked hard to incorporate your feedback and hope that these revisions persuade you to accept my submission.

Your Sincerely,

Tanweer Ul Islam,

National University of Sciences & Technology Islamabad, Pakistan, 44000.

Email: tanweer@s3h.nust.edu.pk ; Tel: +92-51-9085 3567

Submitted filename: Response to Reviewers.docx

Click here for additional data file.

9 Jul 2021

Min-Max Approach for Comparison of Univariate Normality Tests

PONE-D-21-14331R1

Dear Dr. Islam,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Saeed Mian Qaisar, Ph.D.

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

I am pleased to tell you that your work has now been accepted for publication in the PLOS ONE Journal.

Thank you for submitting your work to this journal.

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: All of the comments have been addressed by the author. My recommendation is "accept" without further comments.

Reviewer #2: The author correctly addressed all the previous comments of this reviewer.

Just please check the URL of github you provided, as I could not reach it, and include it in a footnote in the paper.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Asad ul Islam KHAN

Reviewer #2: No

21 Jul 2021

PONE-D-21-14331R1

Min-max approach for comparison of univariate normality tests

Dear Dr. Islam:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Saeed Mian Qaisar

Academic Editor

PLOS ONE