Factor Analysis Procedures Revisited from the Comprehensive Model with Unique Factors Decomposed into Specific Factors and Errors.

Factor analysis (FA) procedures can be classified into three types (Adachi in WIREs Comput Stat https://onlinelibrary.wiley.com/doi/abs/10.1002/wics.1458 , 2019): latent variable FA (LVFA), matrix decomposition FA (MDFA), and its variant (Stegeman in Comput Stat Data Anal 99: 189-203, 2016) named completely decomposed FA (CDFA) through the theorems proved in this paper. We revisit those procedures from the Comprehensive FA (CompFA) model, in which a multivariate observation is decomposed into common factor, specific factor, and error parts. These three parts are separated in MDFA and CDFA, while the specific factor and error parts are not separated, but their sum, called a unique factor, is considered in LVFA. We show that the assumptions in the CompFA model are satisfied by the CDFA solution, but not completely by the MDFA one. Then, how the CompFA model parameters are estimated in the FA procedures is examined. The study shows that all parameters can be recovered well in CDFA, while the sum of the parameters for the specific factor and error parts is approximated by the LVFA estimate of the unique factor parameter and by the MDFA estimate of the specific factor parameter. More detailed results are given through our subdivision of the CompFA model according to whether the error part is uncorrelated among variables or not.

Factor analysis (FA) was originally conceived of by Spearman (1904) and developed toward its modern form by Thurstone (1947). FA is performed for a multivariate data set in order to extract two types of mutually uncorrelated factors: common factors and all others. The common factors, whose number is much less than that of observed variables, serve to explain the variation of all variables. On the other hand, the other factors explain the variations in the variables that remain unaccounted for by the common factors. The latter factors are referred to as unique factors in a prevalent FA model, while those factors are called specific factors in another FA model. Those two FA models are introduced in the following two paragraphs.

The prevalent FA model is expressed asfor a p-variate observed vector x(p 1), with and the expectations of the elements in x being zero (e.g., Bartholomew, Knott, & Moustaki, 2011; Mulaik, 2010; Yanai & Ichikawa, 2007). Here, vectors and contain the common and unique factor scores, respectively, with m < p. The scores in f and u are treated as random latent variables, while ( and are the nonrandom parameter matrices to be estimated. The elements of are called factor loadings, as they describe how the p variables in x load the m common factors in f. In contrast to being unconstrained, is restricted to a diagonal matrix. This implies that the p variables in x have one-to-one correspondences to the p unique factors in u: its jth element uniquely affects the jth one of x with the jth diagonal element of being a coefficient. We refer to the FA procedure based on (1) as latent variable FA (LVFA) following Adachi (2019), in order to distinguish it from a procedure to be introduced later.

The vector in (1) is sometimes referred to as an error vector; however, the classic literature such as Harman (1976), Reyment & Jöreskog (1993), and Thurstone (1947) includes the description that is divided into two vectors asHere, e (rather than is referred to as an error vector, and the elements of s (p 1) are called specific factor scores, with (p p) being diagonal. That is, the FA model is also introduced by incorporating (2) into (1), i.e.,Here, is found to perform a role similar to the unique factor part in (1): is diagonal as is , which implies that the jth specific factor score in s specifically (i.e., uniquely) affects the jth variable of x, with the jth diagonal element of a coefficient. As found in this sentence, the adjective "specific" used for s in (3) has the same implication as the "unique" for u in (1): "specific" and "unique" merely serve to avoid the confusion between in (1) and in (3). However, the error vector e is included in (3) in addition to , while the error vector in (1) equals u. That is, each error in is assumed to uniquely affect the corresponding variable in (1), but is separated from e. In this point, (3) is more generalized or comprehensive than (1). We thus refer to (3) as a comprehensive FA (CompFA) model. This model can also be considered compatible with Spearman’s (1904) original conception of FA (Anderson & Rubin, 1956, p. 112; Yanai, Shegemasu, Mayekawa, & Ichikawa, 1990, p. 4).

However, the CompFA model has been left out of consideration in recent FA studies, as found in the fact that (3) is not treated in the major introductions to FA published in this century (e.g., Bartholomew, Knott, & Moustaki, 2011; Mulaik, 2010). That can be attributed to the following two points seen in the above classic literature: First, the CompFA model has been very briefly mentioned on only a couple of pages in Harman (1976, pp. 19–20), Reyment and Jöreskog, (1993, pp. 75–76), and Thurstone (1947, pp. 74–75); thus, model (3) was not impressed as fulfilling a certain role in FA. Second, in that literature, (3) has been introduced merely as a model; no procedure is described for estimating parameters based on (3).

However, a recently proposed FA procedure, which is called matrix decomposition FA (MDFA) to distinguish it from LVFA (Adachi, 2019; Adachi & Trendafilov, 2018), can be viewed as a parameter estimation procedure for the CompFA model. In more exactness, MDFA can be modeled as a nonrandom matrix version of (3):Here, X is an n-observations p-variables column-centered data matrix, F is the n m matrix of common factor scores, S is the n p matrix of specific factor scores, and E is an n p error matrix, with and being the same as those in (3). The rows of X, F, S, and E in (4) correspond to the transposes of x, f, s, and e in (3), respectively. However, F and S in (4) are treated as nonrandom parameter matrices, while f and s in (3) are random variable vectors. To the best of our knowledge, MDFA was first proposed by Professor Henk A. L. Kiers at the University of Groningen in 2001, as described in Sočan (2003, pp. 19–20). Then, the properties of the MDFA solutions were studied by Adachi and Trendafilov (2018) and Stegeman (2016). The last author has also proposed a restrictive variant of MDFA. The factor vectors f and u in LVFA model (1) are random, as are f and s in (3), but F and S in MDFA model (4) are not random, as described above. This difference is not crucial in this paper, but rather we note that the unique factor part in (1) is decomposed into the specific factor and error parts in (3), whose nonrandom matrix version is (4).

In the above MDFA papers, the elements in the matrix corresponding to S have been referred to as unique factor scores, but they must be called specific factor scores according to the terminology in CompFA model (4). Such a confusing reference to S in the MDFA papers is due to the fact that the CompFA model was not related to MDFA there. This fact provides a major motivation for this paper: we study the relationships of the CompFA model to MDFA and its restrictive variant. This study includes reformulating those MDFA procedures to elucidate whether their parameter estimates are matched to the CompFA model assumptions for the parameters, with those assumptions to be introduced in Sect. 1. Here, we must inform beforehand that the reformulation (in Sect. 4) would allow Stegeman’s (2016) restrictive variant of MDFA to be rephrased as completely decomposed FA (CDFA). We thus call the variant CDFA hereafter. The relationships of LVFA to the CompFA model are also studied in this paper, as model (1) for LVFA is linked to the CompFA model through (2). The goal of our studies is to theoretically and empirically show how LVFA, MDFA, and CDFA behave for CompFA data, i.e., how the parameters in the CompFA model are recovered in the FA procedures, where the CompFA data refer to the data underlaid by the CompFA model. The goal would include showing that all parameters in that model can be recovered fairly well in CDFA but cannot be recovered as well in LVFA and MDFA.

The remaining sections in this paper are organized as follows: In Sect. 1, we specify the CompFA model by introducing its assumptions. Then, LVFA, MDFA, and CDFA are treated in Sects. 2,  3, and 4, respectively; we theoretically discuss how each procedure is related to the CompFA model and behaves for the CompFA data. The discussions for the behaviors are numerically assessed in Sect. 5 and illustrated in Sect. 6. Throughout this paper, we suppose n > p > m.

A more detailed prospect for the remaining sections will be presented in the final part of the next section, as that presentation is possible only after the CompFA model is fully specified. This specification also includes our subdivision of the CompFA model along whether the errors are assumed to be uncorrelated among variables or not. The subdivision is made, because the behaviors of the FA procedures differ between the CompFA data with uncorrelated errors and those with correlated ones, as discussed in Sects. 2, 3, 4 and shown numerically in Sect. 5.

Comprehensive Factor Analysis Model

In Sects. 1.1 and 1.2, we review the standard assumptions for CompFA model (3) and its nonrandom matrix version (4), respectively. Those standard assumptions are not involved with the inter-variable correlations of errors. We discuss in Sect. 1.3 that the consideration of the error correlations allows the CompFA model to be subdivided. The prospects for the following sections are given in the final subsection.

Random Version of Standard Assumptions

The CompFA model is expressed as (3), i.e., , for the random observation vector x, whose expectation E[x] is the zero vector 0. We review the assumptions for the expectations and covariances of f, s, and e in the classic literature (Harman, 1976; Reyment & Jöreskog, 1993; Thurstone, 1947).

In line with , the expectations of the common factor, specific factor, and error vectors are supposed asThe covariance matrices for the factor score vectors are assumed to satisfyHere, C[f, s] E[(f[f])(s [s])] denotes the m p covariance matrix between f and s, I is the m m identity matrix, and O expresses the m p zero matrix. The factor score vectors are assumed to be uncorrelated to the error vector:The standard constraints for model (3) consist of (5)–(7).

From (6), the covariance matrix of and that of s are found to be and , respectively. Here, the diagonal elements of are called specific variances, as they stand for the variances of the specific factor part s. Further, (6) and (7) lead to and . Using these results, the inter-variable covariance matrix for (3) is found to be expressed as

Nonrandom Matrix Version of Standard Assumptions

The nonrandom matrix version of the CompFA model can be expressed as (4), i.e., for the data matrix X. Here, X is column-centered with 1 and supposed to have full column rank with rank(X) , with 1 and rank(X) denoting the n 1 vector of ones and the rank of X, respectively. We summarize the assumptions for (4), i.e., the matrix versions of the assumptions in Sect. 1.1.

The versions of (5) and (6) are expressed asrespectively. The two equations in (7) can be changed into the matrix forms and ,which are equivalent torespectively. The standard assumptions for model (4) consist of (9)–(12).

The inter-variable matrices for X and E can be expressed as and , because of and (9). Then, (9)–(12) lead to the nonrandom matrix version of (8):where F S E is column-centered because of (9).

Uncorrelated Error and Correlated Error Assumptions

In the classic literature (Harman, 1976; Reyment & Jöreskog 1993; Thurstone, 1947), the elements of e in (3) are particularly referred to as measurement errors. If such errors are considered to be uncorrelated among variables, we can add the assumption that the off-diagonal elements of C[e, e] are zeros, i.e.,to those in Sect. 1.1, with D[e, e] diag(C[e, e]). Here, diag(N) denotes the diagonal matrix whose diagonal elements are those of a square matrix N. In the classic literature, (14) is not explicitly presented, but might be implicitly supposed for the following reasons: In that literature, only latent variable FA (LVFA) is described for estimating parameters, and the constraints considered in LVFA follow from adding (14) to the standard ones in Sect. 1.1, as shown in Sect. 2.2. We can also add the nonrandom matrix version of (14), i.e.,to those in Sect. 1.2, with D diag(C.

We can also consider a version of the CompFA model, in which (14) and (15) are not assumed, that is, the errors are allowed to be correlated among variables. In this correlated error version, C[e, e] in (8) and C in (13) are merely supposed to be unconstrained covariance matrices that are nonnegative-definite.

We should note the difference between uncorrelated error constraints (14) and (15), the latter being stronger, as explained next. The rows of the error matrix E in C E E can be considered the realizations of the transpose of e in (14). This consideration leads to C[e, e] in (14) equaling E[C], as detailed in Appendix 1. Thus, we can rewrite (14) as the expectation of C being diagonal: (14) requires the diagonality of E[C], but not that of C itself. This diagonality is required by (15), in contrast. To emphasize the strength of (15), we call this the strong uncorrelated error condition. We can consider that data meeting (15) is hardly encountered, i.e., is unusual. However, in the later sections, such data would be noted: The FA procedures are shown to perfectly fit the data meeting strong condition (15), which is a motivation to study the related properties of the FA procedures. The next theorem gives the foundation for the perfect fit to be shown later.

Theorem 1

If the error matrix in CompFA model (4) satisfies (15), (4) can be rewritten as the error-free model:Here, D and (S + E) can be regarded as the specific variance and factor score matrices, respectively.

Proof

We can rewrite (4) as (16): X F S E F (S E). The matrix D (p p) is diagonal and nonnegative, thus regarded as the specific variance matrix. We can show why (S E) p) is the specific factor score matrix, as follows: (9)–(12) and (15) lead to 1(S E) 0,and (S E) O. Thus, (S E) can be substituted into S in (9)–(11). Further, (16) can be rewritten as X F, with the error matrix being , which implies that those and can be substituted into E and S in (12), respectively.

Prospects for Relating the CompFA Model to LVFA, MDFA, and CDFA

As the assumptions in the CompFA model have been specified, they can now be used for providing the prospects for the following sections, where the relationships of the CompFA model to LVFA, MDFA, and CDFA will be studied.

Among the relationships, those independent of the uncorrelated and correlated error assumptions (in Sect. 1.3) can be summarized as in Table 1. Its left-hand "Model" and "Note" columns merely present the facts described before Sect. 1: (2) links LVFA model (1) to CompFA model (3), and this matrix version is model (4) underlying MDFA and CDFA.

Table 1

Relationships of the FA procedures to the CompFA model.

Proc.	Model	Note	Specific Factor and Errors	Estimate ≈ True
LVFA	x= Λ f +Ψ u	Ψu=Θs+e	Θ s & e unseparated	Ψ^2≈ Θ_^2+D__EE	Λ ≈ Λ_
MDFA	X = FΛ^′+ SΘ + E	↓	diag(S^′ E) =_p O_p	Θ^2≈ Θ_^2+D__EE	Λ ≈ Λ_
CDFA	X = FΛ^′+ SΘ + E	Nonrandom version	S^′ E =_p O_p	Θ^2≈ Θ_^2	Λ ≈ Λ_

The right-hand columns in Table 1 present the key points in the relationships to be found. The lower cells in the "Specific Factor & Errors" column show the facts to be found in Sects. 3.2 and 4.2: The MDFA solution does not meet (12) in the CompFA assumptions, but only satisfies its diagonal part diag(S E) O, but (12) is satisfied by the CDFA solution. On the other hand, the specific factor and error parts ( s and e) are unseparated in LVFA.

The column furthest right in Table 1 shows how , , and are usually estimated in the FA procedures for the CompFA data matrixi.e., the observations underlaid by (4) with F, , S, , and E set to particular matrices , , , , and , respectively. Here, the latter matrices have been underlined for the sake of indicating that particular values are substituted into the elements of those matrices. The final (third) subsections in Sect. 2–4 are particularly concerned with how the true and in (17) are related to LVFA, MDFA, and CDFA estimates, respectively, as outlined in the following two paragraphs.

In Sects. 2.3 and 3.3, we will discuss that the LVFA estimate of approximates and the MDFA one of approximates , respectively. The latter MDFA property of is restated as being contaminated by . In Sect. 4.3, CDFA will be shown to provide the estimate differently from MDFA, with a discussion of how this difference follows from the CDFA solution satisfying (12), which is not met by the MDFA solution.

In Sects. 2.3, 3.3, and 4.3, we will also discuss that the uncorrelated error assumption (in Sect. 1.3) leads to the facts that are not covered in Table 1. The facts are summarized as follows. Two of the formulas with "" in Table 1 are replaced by equalities only in strong uncorrelated error condition (15): in LVFA and in MDFA and CDFA, with all procedures fitting data perfectly. However, apart from that strong condition, the CDFA estimate of can approximate , as described above.

The good recovery of loadings with in all procedures will be shown numerically in Sect. 5.

Latent Variable Factor Analysis

In Sect. 2.1, we review the formulation of latent variable FA (LVFA) with the assumptions added to (1). Then, in Sect. 2.2, we show how those LVFA assumptions can follow from the CompFA ones. In Sect. 2.3, we discuss how the estimate of approximates , as shown in Table 1.

Formulation

LVFA is modeled as (1), i.e., , where the expectations and covariances for f and are assumed to satisfy(e.g., Bartholomew, Knott, & Moustaki, 2011; Yanai & Ichikawa, 2007). Here, the diagonal elements of are called unique variances, as they stand for the variances of the unique factor part .

LVFA assumptions (18)–(20) imply that inter-variable covariance matrix C[x, x] for (1) is expressed asThus, and can be estimated so that model-based (21) approximates its data-based counterpart C. This estimation can be attained by minimizingover and (Harman & Jones, 1966), with trM M denoting the squared Frobenius norm of a matrix M. Another estimation procedure is to minimize the functionwhich is the negative of the log likelihood following from (21) and the additional normality assumptions for f and u (e.g., Bartholomew, Knott, & Moustaki, 2011; Yanai & Ichikawa, 2007).

Relationship to the CompFA Model

LVFA model (1) and assumptions (18)–(20) do not include the specific factor and error parts, which differs from CompFA model (3) and its assumptions. However, (1) is linked to (3) through (2). Further, it is shown next how LVFA constraints (18)–(20) follow from the CompFA counterparts:

Theorem 2

Under (1)–(3), LVFA assumptions (18)–(20) follow from CompFA ones (5)–(7) and uncorrelated error assumption (14).

Obviously, (18) follows, since its two equations appear in (5) and (6). Next, and C[f, O in (19) follow from (5)–(7) under (2), since (2) and (5) imply , while (2), (6), and (7) lead to O. The remaining task is to show how (20) follows. Its left side is rewritten, using (2), (6), and (7), as . For this to equal the right side of (20), i.e., nonnegative diagonal , C[e, e] must equal – which is also nonnegative diagonal. This holds true for (14), i.e., – .

This theorem shows that the LVFA assumptions follow from adding uncorrelated error assumption (14) to the standard CompFA ones in Sect. 1.1. However, LVFA can be performed for CompFA data (17), independently of whether its errors satisfy (14) or not. In the next subsection, how LVFA behaves for data (17) is discussed.

Behaviors for CompFA Data

CompFA data matrix (17) leads to the covariance matrixi.e., (13) with , , and C set to particular matrices , , and , respectively. Let us consider the LVFA solution for (24).

Loss functions (22) and (23) are known to be minimized forfor a given (e.g., Mulaik, 2010, (8.47), (8.80)). Using (24) in (25), this can be rewritten as diag( diag(. This impliesfor ; if estimated approximates the true counterpart , each of the diagonal elements in , i.e., a unique variance, can be the estimate of the sum of the corresponding true specific and error variances, though these two cannot be estimated separately.

The next corollary, which follows from Theorem 1, shows that formula (26) with "" is replaced by the equality , if (24) satisfies strong uncorrelated error condition (15):

Corollary 1

For (15), (24) is restricted to C. For this matrix, LVFA loss functions (22) and (23) can attain their lower limit zero with the solution of given by and that of satisfying .

The matrix difference C ( in (22) and the product C( in (23) are rewritten, using C, as ( and (( , respectively. The former difference can be O, and the latter product can be I, which allows (22) and (23) to attain zeros, for and .

The result in this corollary is referred to in Sect. 3.3.

Matrix Decomposition Factor Analysis

In Sect. 3.1, we review the original formulation of matrix decomposition FA (MDFA). Then, in Sects. 3.2 and 3.3, we discuss two properties of the MDFA solution shown in Table 1. One property is the solution’s satisfying diag( E) O but not (12), which implies that MDFA can be reformulated with constraints that are less restrictive than the CompFA assumptions, as discussed in Sect. 3.2. The other property is suggested by a theorem to be presented in Sect. 3.3.

Original Formulation

For the data matrix X with 1 X 0, MDFA is formulated as minimizing the least squares function for (4), i.e.,over F, , S, and subjec to constraints (9) and (10). In the MDFA literature (e.g., Adachi & Trendafilov, 2018; Sočan, 2003; Stegeman, 2016), 1 E 0 in (9) has not been described, but this is trivial, since 1 E 0 follows from the other equations in (9), (4), and 1 X 0.

One difference in (27) from the LVFA loss functions, besides the former being based on (4), is that (22) and (23) in LVFA do not include the factor scores, while (27) includes those scores as F and S to be estimated. However, infinite solutions of F and S exist that minimize (27); the optimal F and S are not unique (Adachi & Trendafilov, 2018, Sect. 4). Thus, only the solutions of and are interpreted in MDFA, as are those of and in LVFA.

Though (27) is defined using data matrix X, the MDFA solution of and can be obtained only if the covariance matrix C X is available, i.e., even if the original X is unavailable (Adachi, 2012, 2020). This can be captured in the fact that (27) can be expanded as and simplified using (10) as trC tr 2trC 2trC, in which X does not appear. Here, C F and contain the covariances of variables to factors and are uniquely determined for given and (Adachi & Trendafilov, 2018, p. 411). On the other hand, the optimal and can be obtained for given C and C(Adachi & Trendafilov, 2018, p. 410). Thus, the optimal updates of the block matrices [C,C] and [, ] are iterated alternately to provide the solution of [, ] in Adachi’s (2012, 2020) MDFA algorithm.

Reformulation from Properties of the Solution

Besides (9) and (10) imposed as constraints in MDFA, (11) and (12) are included in the standard CompFA assumptions (Sect. 1.2). Thus, we must note whether (11) and (12) are satisfied by the MDFA solution, i.e., the parameter estimates and E resulting in the minimization of (27) under (9) and (10). Adachi and Trendafilov (2018, Theorem 4.1) show that the solution satisfies (11) and the diagonal part of (12), i.e.,but does not meet the off-diagonal part of (12) with S E– diag(S E) O in general. That is, the MDFA solution satisfies standard CompFA assumptions (9)–(11) but does not meet (12).

The MDFA solution satisfying (11) and (28) implies that these equations can be included in the constraints. That is, MDFA can be reformulated as minimizing (27) over F, , S, and subject to constraints (9)–(11) and (28). Here, (28) being only the diagonal part of (12) implies that the MDFA formulation is less restrictive than the CompFA model, in that (12) is relaxed as (28). In contrast, the theorems presented in Sect. 4.2 show that the CDFA solutions satisfy CompFA constraints (9)–(12) completely.

In this section, we consider how MDFA behaves for CompFA data matrix (17). The next corollary, which follows from Theorem 1, shows the MDFA solution in strong uncorrelated error condition (15).

Corollary 2

For data matrix (17) satisfying , i.e., (15) given by setting E to , MDFA loss function (27) can attain its lower limit zero for the solution satisfying F, , S with and .

Theorem 1 shows that (17) can be rewritten into the error-free form X for . Substituting X in (27), it is rewritten as . This can attain the lower limit zero for F, S , and .

By comparing Corollaries 1 and 2, we can find that both LVFA and MDFA perfectly fit the CompFA data with strong condition (15), then in LVFA, but in MDFA; its estimates of the specific variances are not their true values, and they are contaminated by the error variances in . This shows an undesirable property of MDFA.

Though the data with (15) are unusual, as mentioned in Sect. 1.3, whether the above contamination can occur for usual CompFA data (17) that are not restricted by (15) is to be considered. For this consideration, we reparameterize the error matrix in (17) asand being p p. The next theorem suggests that the MDFA estimate of can be contaminated by , i.e., can be close to :

Theorem 3

Consider the case where MDFA is performed for the data matrix specified as (17) with (29) and p. Let two matrices be defined aswith p diagonal matrix. Then, the matrices and in (30) can be substituted into and S in function (27), respectively, and can be substituted into S in (9) and (10). Moreover, if diag and satisfies , then in (30) can be substituted into S in (28), and we can use , and (17) in (27) to rewrite this function as

See Appendix 2.

Theorem 3 suggests that the MDFA estimate of can be contaminated by withas explained next. The theorem allows us to consider that and in (30) can be the MDFA estimates of and S, respectively. By substituting those and into and S in the loss function (27) to be minimized, it can be rewritten as (31). This value can be small for F and . This use in (30) and lead to (32). It will be confirmed in Sect. 5 that (32) actually arises.

Completely Decomposed Factor Analysis

In Sect. 4.1, we review Stegeman’s (2016) restrictive variant of MDFA that has been rephrased as completely decomposed FA (CDFA) through the theorems present in Sect. 4.2. They also allow CDFA to be reformulated so that it is perfectly matched with the CompFA model. In Sect. 4.3, we argue how the CDFA estimate of approximates as shown in Table 1.

Stegeman’s Factor Analysis Procedure

Stegeman’s (2016) original formulation of CDFA is to add the constraintto MDFA formulated as in Sect. 3.1: In CDFA, (27) is minimized over F, , S, and subject to (9), (10), and (33), for the data matrix X with 1 X 0.

As described in Stegeman (2016, p. 196), the CDFA solution can be obtained through the following three sequential steps: First, the optimal subject to (33) can be obtained by performing ten Berge and Kiers’ (2001) minimum rank factor analysis (MRFA) for C X. Then, the resulting provides the solution of S. Finally, loss function (27), whose and S are fixed to the ones resulting so far, is minimized over F and subject to (9) and (10). This minimization is attained forthrough the singular value decomposition (SVD) of defined asHere, V V W W I, and is the q q diagonal matrix whose diagonal elements are arranged in decreasing order, with rank(X − S and supposed. The matrices V (n m) and W (p m) in (34) contain the first m columns of V and W, respectively, with the upper-left m diagonal block of .

The optimal F and in (34) are found to satisfysince we can use (10), (34), (35), and (from to derive (36) asAs in MDFA, the optimal F and S cannot be uniquely determined, but their estimation can be skipped to obtain the optimal and , only if the covariance matrix C is available. This is shown using the fact that (10) and (33) lead to X S . This, (33), and (35) imply that the SVD of (X − S(X − S X X (C can be defined as n(C W W with given by MRFA for C. This SVD can provide with (34).

Reformulation Matched to the CompFA Model

The CDFA solution, i.e., the parameter estimates resulting in the minimization of (27) under (9), (10), and (33), satisfies (36). This implies that (36) can be included in the constraints: CDFA can also be formulated as minimizing (27) subject to (9), (10), (33), and (36). Here, constraints (33) and (36) are proved to be equivalent to (12) and (11), respectively, under (4) and (10), in the next two theorems.

Theorem 4

Under (4) and (10), (33) is equivalent to (12), i.e., S E O.

First, (33) (12) is proved as follows: Using (4), we can rewrite (33) as S(F E) O, which implies (12), i.e., S E O, from (10). Next, (12) (33) is proved as follows: We can use (4) and (10) to rewrite (12) as S S O, i.e., (33).

Theorem 5

Under (4) and (10), (36) is equivalent to (11), i.e., F E O.

First, let us prove (36)  (11). The former is rewritten as F X. Using this, (4), and (10), we have F E F(X–F S O, i.e., (11). Next, (11) (36) is proved as follows: We can use (4) and (10) to rewrite (11) as F, which leads to (36).

These theorems and (10) show that F S O, F E O, and S E O in (10)–(12) hold true in the CDFA solution. These three equations imply that the common factors, specific factors, and errors are mutually decomposed completely, thus the name CDFA. This name is considered more suitable than another name, constrained uniqueness FA, used in Adachi (2019), as the CompFA model has not been considered in the latter naming. Further, the theorems show that Stegeman’s (2016) procedure in Sect. 4.1 can be reformulated as minimizing least squares function (27) for model (4) subject to its constraints (9)–(12). Thus, CDFA is perfectly matched to the CompFA model and its standard constraints in Sect. 2.2. Here, let us remember that MDFA can be reformulated as minimizing (27) subject to (9)–(11) and (28) (Sect. 3.2); this (28) is strengthened into (12) in CDFA.

In this section, we consider how CDFA behaves for CompFA data (17). At first, the following corollary shows the CDFA solution in strong uncorrelated error condition (15):

Corollary 3

CDFA can be substituted for MDFA in Corollary 2.

This is the same as the proof for Corollary 2, since (27) is also the CDFA loss function.

This corollary shows that the CDFA estimate of (specific variances) is contaminated by (error variances) for the CompFA data with (15), as is the MDFA estimate. However, we can argue that the contamination is less likely to occur in CDFA than in MDFA for the usual CompFA data that are not restricted by (15). This argument follows from the fact that constraint (28), i.e., diag(S E) O, in MDFA is strengthened as (12), i.e., S E O, in CDFA, as explained in the following paragraph.

In Sect. 3.3, we discussed that the MDFA estimate of can be contaminated as (32), which follows from (30) with (. This matrix and E lead to ( as shown by (A6) in Appendix 2. Here, is a p p diagonal matrix and is defined as in (29). In CDFA, the above equation for must be substituted into S E in (12) assince (12) is included in the CDFA constraints as described in Sect. 4.2. The equivalence of (37) to strong uncorrelated error condition (15) is shown next:

Theorem 6

Substituting in (17) into E in (15), it is rewritten as . This is equivalent to (37).

The last identity in (37) holds if and only if is a diagonal matrix: . This equivalence to is shown using (29) rewritten as . That is, implies that is also diagonal: . On the other hand, , i.e., being diagonal, implies that is diagonal: .

This theorem shows that contamination (32) is less likely to occur in CDFA for the data that do not satisfy (15). On the other hand, even for such data, (32) can occur in MDFA, where S E may not be O, but only diag(S E) O is required. These arguments are empirically supported in the next section.

Simulation Study

We assess the performance of the FA procedures for the CompFA data in a simulation study. Its purposes and the data types to be simulated are detailed in Sect. 5.1. Data analysis and assessment procedures are described in Sect. 5.2, and the results are reported in Sects. 5.3–5.5.

Purposes and Data Synthesis Procedures

In this study, the FA procedures are carried out for the CompFA data synthesized with the true loading matrix and specific variance matrix . The major purpose of this study is to numerically assess the following hypotheses:

The CDFA estimate of approximates the true better than the MDFA estimate.

The MDFA estimate of approximates , i.e., is contaminated by , with the diagonal matrix including the true error variances.

The LVFA estimate of unique variance matrix approximates .

The estimates of in all procedures approximate the true .

Here, [H] follows from the discussion in Sect. 4.3, and hypotheses [H] and [H] are equivalent to (32) and (26), respectively. [H] has not been discussed, but rather has been presupposed for the discussions in Sects. 2–4. The estimates in the hypotheses are obtained, given the covariance matrix C XX for the CompFA data. Thus, we describe below how the true and are set and how they lead to C.

Let , denote the uniform distribution for the interval [, ]. Each element of and each diagonal one of are drawn from 1, 1) and U(0.1, 0.8), respectively, subject to rank(. From the resulting and , we generate 12 types of C with rank(C constrained to be p. Here the 12 ( 3 2 2) types are defined by combining the three levels of error correlations, two levels of error magnitudes, and two versions of the CompFA model, which are explained in the following paragraphs.

The two versions of the model correspond to its nonrandom (N) version (4) and random (R) version (3). The covariance matrix for the N version is given by (24) for (17) with :Here, error vectors e ( 1, …, n) are chosen as e, with drawn from (0, , i.e., the p-variate normal distribution whose mean vector is 0 and covariance matrix is p). How and are defined is described later. The matrix for the R version is given byThis follows from (3), whose random vectors are followed by the observation-number subscript i. The factor score vectors in (39) are sampled with [f, s] (0, I.

The three levels of error correlations can be referred to as no, low, and high levels (C, C, and C, while the two levels of error magnitudes can be called low and high levels (E and E, respectively. Here, C, E, N, L, and H in the parentheses are abbreviations of correlation, error, no, low, and high, respectively. At the C level, is set to the diagonal matrix D whose diagonal elements are drawn from U(0.1, 0.8). At the C and C levels, is set to DRD, with R ( the p p symmetric nonnegative-definite matrix whose elements are chosen as 1 and for . Here, is randomly set to 1 or , and is drawn from (0.2, 0.05 subject to , with for C and for C. The value is set so that 0.1 and 0.2 for the E and E levels, respectively. We should notice that is diagonal at the C level, but obtained with is not necessarily diagonal; the C level does not exactly match (15). In Appendix 3, it is reported that a simulation study for C satisfying (15) numerically demonstrated the facts in Corollaries 1–3.

The procedures in the last three paragraphs were replicated 500 times with the settings 200, 12, and 3. Thus, we had 6000 (12 types 500 times) C.

Data Analysis and Assessment Procedures

For each of the 6000 C, we performed MDFA, CDFA, and the two types of LVFA, i.e., the least squares LVFA (LS-LVFA) and maximum likelihood LVFA (ML-LVFA) for minimizing (22) and (23), respectively. Their algorithms are described in Appendix 4. In every procedure, has rotational indeterminacy; thus, it was rotated by the orthogonal Procrustes method (e.g., Adachi, 2020, p. 206), to optimally approximate the true counterpart in a least squares sense. This approximation allows to be comparable to ; thus, the Procrustes method has been typically used in previous simulation studies for FA (e.g., Adachi, 2013, Appendix D; Stegeman, 2016, Sect. 4.2).

The similarities between estimated parameters and their true values can be assessed with the smallness of a mean absolute difference (MAD). The MAD for loadings is defined as MAD(, where denotes the norm of , i.e., the sum of the absolute values of the elements in . The denominator pm in this definition is replaced by p in the following two MAD for p p diagonal matrices:with . Here, MAD in (40) and (41) for LVFA differs from those for MDFA and CDFA, as corresponds to in LVFA. We obtain (41) in addition to (40), as both MAD are necessary for examining [H] and [H] in Sect. 5.1.

Let {, } and {, } denote the LS- and ML-LVFA estimates of {, }, respectively. The broad equivalence between {, } and {, } was found with the averages (standard deviations) of /(pm) and /p over the solutions for 5935 () C being 0.010 (0.011) and 0.022 (0.019), respectively. Here, we have removed the 65 C for which LS-LVFA provided improper solutions with negative unique variances. As described in Appendix 4, such improper solutions show an undesirable property of LS-LVFA. These results allowed us to only consider the ML-LVFA solutions; thus, we refer to only ML-LVFA as LVFA, hereafter.

Now, we have the MAD values for the three matrices (M), , , and , which were obtained from each solution of the three FA procedures (P) performed for the 6000 ( 3 2 2 500) C. Here, these C are classified by combining the three levels of error correlations (C), two levels of error magnitudes (E), two versions (V) of the model, and 500 replications (R). In order to assess the hypotheses in Sect. 5.1, we performed analysis of variance of the randomized block design (ANOVA-RBD) (e.g., Kirk, 2013) for the MAD values. Here, P, M, C, E, and V were treated as treatments and R was treated as a block factor; the factorial design can be expressed as P M C E V with R consisting of 500 blocks, with the sets of the levels in P, M, C, E, and V being {LVFA, MDFA, CDFA}, {, ,}, {E E}, and {N,R}, respectively.

The above ANOVA-RBD provided the F values for main and interaction effects in Table 2. We do not use those values for statistical hypothesis testing, since this is senseless for our simulated data, whose sample size is too large to reject null hypotheses. But rather, we consider the F values as standing for the sizes of the effects, as the number of the levels in each treatment is restricted to two or three; thus, it makes sense to compare the F values. We regard the effects with 24,000 as substantial enough, though the lower limit of 24,000 is a benchmark threshold chosen because 24,000 is far greater than the largest one ( 14,491.0) among the F-values less than 24,000. Thus, the five effects boldfaced in Table 2 are regarded as substantial.

Table 2

F values resulting in ANOVA-RBD with the substantial effects boldfaced whose F values > 24,000.

Effect	Source and F value
Main	P	M		C	E	V		Block
	444.5	195361.0		14491.0	107830.5	26992.8		36.4
Two-way interaction	P×M	P×C		P×E	P×V	M×C		M×E
	42997.4	67.7		213.6	952.1	5845.3		24629.0
	M×V	C×E		C×V	E×V
	8393.9	1938.9		235.3	341.9
Three-way interaction	P×M×C	P×M×E		P× M×V	P×C×E	P×C×V		P×E×V
	2051.2	4591.3		400.7	16.3	27.7		77.7
	M×C×E	M× C×V		M×E×V	C×E× V
	1090.1	908.0		102.3	4.8
Higher-order interactions	P× M×C×E		P× M×C×V		P× M×E×V		P×C×E×V
	291.1		277.6		28.6		1.3
	M×C×E×V		P×M×C× E×V
	32.2		7.8

Averages for Substantial Effects

The averages of MAD for the levels associated with the substantial effects are presented in Table 3. The averages for the main effect of V in Panel (C) show that the MAD for R version (39) is greater than that for N one (38); this can be attributable to the fact that (39) has higher randomness. The averages for the other main effects in the bottom rows of Panels (A) and (B) can be interpreted from the interactions considered next.

Table 3

Averages of MAD for substantial effects with those for main effects shown by boldfaced italic letters.

(A) Procedure × Matrix				(B) Matrix × Error			(C) Version
	Λ_	Θ_^2	Θ_^2+D__EE		E_L	E_H	Nonrandom	0.099
LVFA	0.045	0.228	0.074	Λ_	0.037	0.053	Random	0.131
MDFA	0.044	0.217	0.073	Θ_^2	0.132	0.249
CDFA	0.046	0.126	0.182	Θ_^2+ D__EE	0.082	0.137
Matrix	0.045	0.191	0.110	Error	0.084	0.146

The averages for the P M levels in Panel (A) show that [H]–[H] in Sect. 5.1 were supported: [H] is supported by the CDFA average 0.126 for (40) being less than the MDFA counterpart 0.217, [H] and [H] are supported by the result that the LVFA and MDFA averages for (41) are less than those for (40), and [H] is supported by the averages of MAD( being small and almost equivalent among the procedures.

Panel (B) shows that every MAD increases with the change from E to E, but the increments in (40) and (41) values are much larger than those in MAD(. This difference in the increments can be interpreted by taking into account the related three-way interaction, as explained in the next subsection.

Three-way Interactions Related to the Substantial Two-way Interactions

Figure 1 shows the averages associated with the P M C and P M E interactions, whose F values are the largest among those for the three-way interactions (Table 2). Although those values are not substantial, we note Fig. 1 for exploring the mechanisms that underlie the interactions treated in Sect. 5.3.

Fig. 1

Averages of MAD for three-way interactions (A) and (B), which facilitate the interpretation of the substantial two-way interactions.

Panel (A) in Fig. 1 shows that the changes of C to C and C to C increase MAD for all procedures, but decrease the (40) value for CDFA. This result can be explained by the similarity of the matrices at C to (15) and the deviation of at C from (15). Here, (15) leads to the solution of and , as shown in Corollaries 1–3. However, for deviating from (15), the CDFA estimate of can approximate , as discussed with Theorem 6. Thus, the deviation of from (15) increases MAD(, but decreases (40) only in CDFA. The above explanation can also be used for the increase in the CDFA average of (41) with the deviation from (15). On the other hand, the deviation is not found to decrease the LVFA and MDFA values of (40). This is congruous with (26) and (32). Further, the LVFA and MDFA averages of (40) rather increase with the change from C to C. These increases may be correlated to those in MAD( from C to C, as is jointly estimated with and in (40). The LVFA and MDFA averages of (41) being smaller than those of (40) at all C levels is also congruous with (26) and (32).

Panel (B) shows that the increments in MAD with the change from E to E differ among the procedures and matrices. For explaining the differences, we must consider error magnitudes (EM) and effects, where the EM effect stands for the errors with greater EM disturbing parameter estimation more deeply, and the effect refers to the fact that the diagonal elements of for E have greater values than those for E. We can consider that the increase in MAD( for every procedure follows only from the EM effect, while those in the LVFA and MDFA values of (40) are affected by the EM plus effects, because of (26) and (32). The increments in (40) being greater than their CDFA counterpart can be explained with Theorem 6, which shows that (32) is less likely to occur in CDFA for deviating from (15). On the other hand, (41) values and their increments for MDFA and LVFA are found to be smaller than their (40) counterparts. This result can be attributable to the fact that (41) is affected only by the EM effect in LVFA and MDFA, as the use of their properties (26) and (32) in (41) allows us to find that MAD( can approximate zero, but this approximation can be deteriorated by the EM effect.

Additional Results

The results considered so far show the advantage of CDFA over the two other procedures. Which of those two is better may be answered from the following result: The panels for in Fig. 1 show that the MDFA estimate of is slightly closer to than the LVFA estimate of on average. The generality of this relationship was found: The value, which is defined as the LVFA value of (40) minus its MDFA value, was positive for the 5819 pairs of LVFA–MDFA solutions among the 6000 for all C. Further, we performed ANOVA-RBD for with C, E, and V treated as treatments, and R treated as blocks. Among all resulting F values, the one for the main effect of C (3651.7) and the value for the effect of E (3238.4) were the largest, with the other F values less than 940. However, the two effects for the largest F values are not considered substantial, with the averages at levels C, C, C, E, and E being 0.003, 0.008, 0.019, 0.006, and 0.014, respectively; even the largest of those cannot be regarded as substantial. In conclusion, the MDFA estimate of is significantly closer to than the LVFA estimate of , but not substantially closer.

Real Data Illustration

The results in the previous sections can be summarized as follows: In CDFA whose formulation is exactly matched to the CompFA model, its parameters (factor loadings and specific variances) are well recovered for the CompFA data. In MDFA and LVFA, the loadings can be recovered as well as in CDFA, but the specific variances cannot be recovered, so their MDFA estimates are contaminated by the error variances, and the LVFA estimates of the unique variances approximate the sums of the specific and error variances. These results suggest that CDFA should be used for the CompFA data, particularly for the purpose of estimating the specific variances. Therefore, in this section, we use two real data sets to illustrate how the CDFA estimates should be interpreted, and we show the merits of using CDFA through comparisons among CDFA, MDFA, and LVFA solutions, on the supposition that the data sets are underlaid by the CompFA model. Here, LVFA is restricted to ML-LVFA, as its solutions were almost equivalent to LS-LVFA.

One of the data sets is that from Yanai and Ichikawa (2007) for personality test scores of 200 students for 12 items. The other data set, which we obtained from Izenman’s (2008) website, is known as Holzinger and Swineford’s (1939) 24 psychological tests data and contains intelligence test scores of participants for items. We performed the FA procedures for the correlation matrices for the personality and intelligence test scores, with m set at 3 and 5, respectively. In every procedure, was rotated by the varimax method (Kaiser, 1958) which is typically used in FA for real data. Tables 4 and 5 present the resulting , and ev, with the last three terms being the jth diagonal elements of , , and C, respectively, and ev an abbreviation of error variance. In the tables, the sub-/superscripts of L, M, and C, which stand for LVFA, MDFA, and CDFA, respectively, have been attached to , , , and ev, for distinguishing the solutions for different procedures. We use the above notation with sub-/superscripts in this section.

Table 4

Solutions for personality test data.

Variable	LVFA						MDFA							CDFA
	Λ_L=[λ1L⋯λ12L]^′			λjL^2	ψj2		Λ_M=[λ1M⋯λ12M]^′			λjM^2	θjM2	evjM		Λ_C=[Λ1C⋯λ12C]^′			λjC^2	θjC2	evjC
Extraversion	− 0.32	0.48	0.24	0.39	0.61		− 0.34	0.46	0.25	0.39	0.60	0.011		− 0.35	0.50	0.25	0.44	0.39	0.176
Activity	− 0.25	0.65	0.42	0.66	0.33		− 0.26	0.64	0.43	0.66	0.33	0.007		− 0.25	0.67	0.43	0.70	0.18	0.118
Empathy	− 0.02	0.03	0.58	0.34	0.66		− 0.03	0.01	0.60	0.36	0.62	0.018		− 0.04	0.00	0.63	0.40	0.45	0.151
Novelty	− 0.05	0.62	0.04	0.39	0.62		− 0.06	0.61	0.05	0.38	0.61	0.006		− 0.06	0.65	0.04	0.43	0.44	0.141
Durability	− 0.07	0.04	0.68	0.47	0.53		− 0.07	0.04	0.67	0.46	0.54	0.014		− 0.06	0.05	0.68	0.47	0.38	0.151
Regularity	0.06	0.17	0.71	0.54	0.46		0.07	0.17	0.71	0.54	0.45	0.015		0.09	0.18	0.76	0.62	0.23	0.152
Self-revelation	0.16	0.62	0.03	0.41	0.59		0.14	0.62	0.04	0.41	0.58	0.006		0.16	0.66	0.03	0.46	0.42	0.122
Aggressiveness	0.39	0.54	− 0.14	0.46	0.54		0.39	0.57	− 0.15	0.50	0.49	0.010		0.41	0.56	− 0.17	0.51	0.41	0.084
Uncooperativeness	0.45	0.16	− 0.17	0.26	0.74		0.46	0.18	− 0.19	0.28	0.71	0.020		0.51	0.20	− 0.23	0.35	0.39	0.259
Inferiority feeling	0.62	− 0.30	− 0.17	0.50	0.49		0.63	− 0.29	− 0.17	0.51	0.48	0.007		0.63	− 0.30	− 0.17	0.52	0.37	0.111
Nervousness	0.71	0.00	0.22	0.55	0.45		0.70	0.01	0.23	0.54	0.44	0.011		0.76	− 0.02	0.27	0.65	0.18	0.169
Depression	0.82	0.03	−0.07	0.68	0.32		0.82	0.04	− 0.06	0.68	0.32	0.007		0.82	0.03	− 0.07	0.68	0.22	0.103

Table 5

Solutions for intelligence test data.

Variable	LVFA							MDFA								CDFA
	Λ_L=[λ1L⋯λ24L]^′					λjL^2	ψj2	Λ_M=[λ1M⋯λ24M]^′					λjM^2	θjM2	evjM	Λ_C=[λ1C⋯λ24C]^′					λjC^2	θjC2	evjC
Spatial 1	0.31	0.63	0.10	0.12	0.01	0.52	0.49	0.31	0.63	0.10	0.12	0.03	0.52	0.48	0.006	0.30	0.65	0.10	0.13	0.12	0.55	0.35	0.095
Spatial 2	0.14	0.48	- 0.06	0.01	0.12	0.27	0.73	0.14	0.48	- 0.06	0.00	0.14	0.27	0.73	0.002	0.13	0.46	- 0.07	0.01	0.22	0.28	0.65	0.075
Spatial 3	0.19	0.43	0.06	- 0.03	0.00	0.23	0.77	0.19	0.44	0.07	- 0.03	0.01	0.24	0.76	0.005	0.19	0.49	0.06	- 0.03	0.05	0.28	0.54	0.177
Spatial 4	0.06	0.63	0.09	0.14	0.09	0.44	0.56	0.06	0.64	0.09	0.14	0.10	0.45	0.55	0.002	0.05	0.61	0.08	0.15	0.20	0.44	0.49	0.059
Verbal 1	0.82	0.06	0.12	- 0.02	0.04	0.69	0.30	0.83	0.06	0.12	- 0.02	0.04	0.71	0.29	0.008	0.84	0.07	0.11	- 0.02	0.06	0.73	0.16	0.106
Verbal 2	0.80	0.10	0.09	0.12	0.06	0.68	0.32	0.80	0.10	0.09	0.12	0.07	0.68	0.32	0.007	0.80	0.10	0.08	0.12	0.11	0.68	0.23	0.087
Verbal 3	0.88	0.05	0.06	0.01	- 0.02	0.78	0.22	0.89	0.04	0.06	0.01	0.00	0.80	0.20	0.007	0.91	0.04	0.06	0.00	0.04	0.83	0.07	0.104
Verbal 4	0.71	0.16	0.10	0.10	0.07	0.55	0.44	0.71	0.15	0.10	0.10	0.10	0.56	0.44	0.007	0.72	0.14	0.10	0.10	0.15	0.58	0.29	0.139
Verbal 5	0.82	0.15	0.06	0.08	0.11	0.72	0.28	0.82	0.15	0.05	0.08	0.12	0.72	0.28	0.007	0.83	0.14	0.05	0.08	0.16	0.74	0.15	0.114
Speed 1	0.08	- 0.08	0.79	0.08	0.14	0.66	0.34	0.08	- 0.09	0.78	0.08	0.14	0.65	0.34	0.010	0.07	- 0.12	0.80	0.07	0.14	0.68	0.23	0.098
Speed 2	0.29	0.18	0.57	0.24	- 0.09	0.51	0.50	0.30	0.18	0.57	0.23	- 0.04	0.50	0.49	0.009	0.30	0.17	0.60	0.23	0.02	0.53	0.31	0.155
Speed 3	0.09	0.23	0.62	0.00	0.04	0.45	0.56	0.09	0.22	0.62	0.00	0.06	0.44	0.55	0.005	0.08	0.22	0.65	0.00	0.11	0.49	0.37	0.137
Speed 4	0.17	0.45	0.51	0.06	−0.11	0.51	0.50	0.17	0.44	0.51	0.05	- 0.06	0.49	0.51	0.007	0.16	0.46	0.54	0.04	0.03	0.53	0.32	0.142
Memory 1	0.15	0.05	0.04	0.64	0.08	0.44	0.56	0.15	0.04	0.04	0.63	0.11	0.43	0.56	0.007	0.15	0.00	0.02	0.69	0.14	0.52	0.31	0.179
Memory 2	-0.03	0.16	0.05	0.56	0.08	0.35	0.66	- 0.03	0.17	0.04	0.57	0.08	0.36	0.63	0.010	- 0.05	0.17	0.02	0.63	0.09	0.44	0.39	0.178
Memory 3	0.17	0.37	0.11	0.45	0.18	0.41	0.58	0.17	0.36	0.11	0.44	0.22	0.41	0.58	0.006	0.16	0.32	0.10	0.44	0.30	0.42	0.48	0.105
Memory 4	0.06	0.03	0.34	0.54	- 0.03	0.41	0.60	0.06	0.03	0.34	0.54	- 0.02	0.41	0.58	0.007	0.06	0.02	0.35	0.57	- 0.01	0.45	0.39	0.153
Memory 5	0.12	0.17	0.22	0.42	0.01	0.27	0.73	0.12	0.17	0.23	0.44	0.02	0.29	0.70	0.012	0.12	0.17	0.24	0.49	0.02	0.34	0.43	0.226
Memory 6	0.24	0.16	0.11	0.31	0.22	0.24	0.76	0.24	0.13	0.12	0.30	0.26	0.25	0.74	0.013	0.23	0.06	0.12	0.31	0.35	0.29	0.51	0.200
Math.1	0.34	0.39	- 0.01	0.21	0.31	0.41	0.60	0.33	0.38	- 0.01	0.21	0.32	0.40	0.60	0.004	0.31	0.33	- 0.03	0.21	0.40	0.41	0.49	0.101
Math.2	0.29	0.35	0.35	0.10	0.39	0.49	0.52	0.28	0.34	0.34	0.10	0.42	0.50	0.50	0.007	0.24	0.26	0.33	0.08	0.55	0.54	0.32	0.133
Math.3	0.49	0.36	0.06	0.11	0.24	0.44	0.55	0.49	0.35	0.06	0.11	0.27	0.45	0.55	0.009	0.48	0.29	0.05	0.10	0.38	0.47	0.38	0.155
Math.4	0.42	0.47	0.16	0.13	0.36	0.57	0.43	0.41	0.46	0.15	0.12	0.38	0.56	0.43	0.008	0.39	0.40	0.13	0.11	0.51	0.60	0.23	0.160
Math.5	0.42	0.12	0.36	0.19	0.42	0.53	0.47	0.41	0.09	0.35	0.18	0.47	0.55	0.45	0.005	0.40	0.01	0.34	0.17	0.51	0.56	0.35	0.085

Tables 4 and 5 show that the loadings resulting in all procedures are mutually similar and lead to the identical interpretation of common factors. Although this result does not show the merits of using CDFA, the merits are shown by the other results, as described in the following paragraphs. There, an important role is fulfilled by the fact thatholds for , which denotes the variance for variable j and is also the jth diagonal element of C. Here, the first identity follows from the fact that the CDFA solution meets (9)–(12) and thus (13), the last identity follows from (25), and the second identity is derived as follows: (4), (10), and (11) imply diag(C diag( C S E E S with diag(E S diag( S E) diag(S E) O from (28). Each of the terms (with superscripts) in (42) can be called a common variance (or communality) by abbreviating the variance of the common factor part affecting variable j, as found from equaling and the variance of f under (6) and (10). When C is a correlation matrix as in our case, 1; thus, , , and ev (with superscripts) in (42) stand for the proportions of the common, specific, and error variances in the variance of variable j, respectively, with the proportion of the unique variance.

Let us note the CDFA solution in Table 4 with keeping (42) in mind. For example, the results 0.44, 0.39, and 0.176 for the variable extraversion show that 44 percent of the variations in extraversion are explained by the common factors, 39 percent are accounted for by the factor specific to extraversion, and 17.6 percent of the variations remain unexplained by common and specific factors. The comparison of across j ( 1, …, p) shows that 0.45 for empathy is the largest and empathy is affected most by the corresponding specific factor among all variables. On the other hand, 0.18 for activity is the smallest among all , but 0.70 is the largest among all , implying that activity is affected least by the corresponding specific factor, but is explained best by the common factors, among all variables.

MDFA and LVFA solutions can be interpreted in a parallel manner, except that the term "specific" is replaced by "unique" and an error variance is not obtained in LVFA. However, we can find in Tables 4 and 5 that and are much greater than and rather close to for almost all variables. This finding is congruous with (26) and (32). For example, 0.60 and 0.61 > 0.39 for the variable extraversion in Table 4. We can consider that 0.60 and 0.61 are contaminated by the true error variance for extraversion and greater than its true specific variance.

We can also find that the error variance resulting in MDFA is far smaller than its CDFA counterpart for every variable. This result follows from the fact that MDFA is less restrictive than CDFA; (27) value trC in MDFA cannot be greater than that in CDFA, which suggests that < tends to occur. This property does not imply goodness of the MDFA solution but is rather congruous with its undesirable property shown by (32); comparing this with following from (42) allows us to find that being larger than its true value decreases .

Conclusion

In this paper, latent variable factor analysis (LVFA), matrix decomposition factor analysis (MDFA), and its variant from Stegeman (2016) were revisited from the comprehensive FA (CompFA) model. The variant of MDFA was reformulated to be called CDFA and exactly underlaid by the CompFA model. On the other hand, MDFA was reformulated as the procedure with (12) in the CompFA model assumptions relaxed as (28). We also showed how the model for LVFA can be related to the CompFA model.

A goal of the revisit was to show how LVFA, MDFA, and CDFA behave for the CompFA data based on the CompFA model. Except for the unusual case where the data satisfy strong condition (15), the following results were theoretically and numerically found: The CDFA estimates of the specific variances can approximate their true values, but the MDFA estimates are contaminated by the error variances, and the LVFA estimates of the unique variances approximate the sum of the true specific and error variances. It was also shown numerically that the factor loadings can be recovered well in all three procedures. On the supposition that the data to be analyzed by FA are underlaid by the CompFA model, the above results have the practical implications described in the following paragraph.

When only factor loadings are of interest, LVFA, MDFA, and CDFA are equally useful. However, if the specific variances are also interesting, CDFA is to be used. The LVFA estimate of unique variances and the MDFA estimates of specific variances must be considered as larger than the true specific variances.

However, a problem remains for showing the above implications to FA users, who are interested in only the factor loadings, not the specific variances. This problem would be dealt with, if psychometricians could enlighten the users about the importance of the specific variances. For this enlightenment, CompFA model (4) can be used as follows: By removing the specific factor part by setting O in (4), this model and corresponding least squares function (27) are rewritten as X F E and , respectively. Minimizing this function gives the formulation of principal component analysis (PCA) as approximating X by reduced rank matrix F (Eckart & Young, 1936). This fact demonstrates that FA can be distinguished from PCA simply by the fact that the former has specific factor part S. That is, the significance of using FA (rather than PCA) is in obtaining S, which convinces the users of how it is necessary to interpret the specific variances in together with loading matrix in the FA solution.

Finally, we should remember that MDFA can be regarded as a procedure for a relaxed variant of the CompFA model with (12) replaced by (28). To study such a relaxed CompFA model is beyond the scope of the present study; thus, it remains for future approaches.