Characterizing the Manifest Probability Distributions of Three Latent Trait Models for Accuracy and Response Time.

In this paper we study the statistical relations between three latent trait models for accuracies and response times: the hierarchical model (HM) of van der Linden (Psychometrika 72(3):287-308, 2007), the signed residual time model (SM) proposed by Maris and van der Maas (Psychometrika 77(4):615-633, 2012), and the drift diffusion model (DM) as proposed by Tuerlinckx and De Boeck (Psychometrika 70(4):629-650, 2005). One important distinction between these models is that the HM and the DM either assume or imply that accuracies and response times are independent given the latent trait variables, while the SM does not. In this paper we investigate the impact of this conditional independence property-or a lack thereof-on the manifest probability distribution for accuracies and response times. We will find that the manifest distributions of the latent trait models share several important features, such as the dependency between accuracy and response time, but we also find important differences, such as in what function of response time is being modeled. Our method for characterizing the manifest probability distributions is related to the Dutch identity (Holland in Psychometrika 55(6):5-18, 1990).

Introduction

In this paper we wish to study the statistical relations between three latent trait models for accuracies and response times: the hierarchical model (HM) of van der Linden (2007), the signed residual time model (SM) of Maris and van der Maas (2012), and the drift diffusion model (DM) proposed by Tuerlinckx and De Boeck (2005). These models come from different backgrounds and differ in many respects. A key distinction between the three latent trait models is in the relation they stipulate between accuracy and response time after conditioning on the latent variables. Whereas responses are independent of response times after conditioning on the latent variables in both the DM and the HM, this is not the case for the SM. The conditional independence property has received much attention in the psychometric literature on response time modeling (e.g., Bolsinova, De Boeck, & Tijmstra, 2010; Bolsinova & Maris, 2016; Bolsinova & Tijmstra, 2016; Bolsinova, Tijmstra, & Molenaar, 2017; van der Linden & Glas, 2017).

The way that these and other latent trait models for accuracy and response time are related has been the topic of several publications (e.g., Molenaar, Tuerlinckx, & van der Maas, 2017a; 2015b; van Rijn & Ali, 2015a), but what sets our approach apart from earlier comparison attempts is that we do not work with their latent trait formulations. A serious complication with comparing latent trait models is that they are usually not defined on a common metric or space. As a result, it is unclear what the conditional independence property, for example, says about the distribution of observables, or how one can compare latent variables and their impact on observables across models. The manifest distribution —i.e., the distribution of observables after having integrated out the latent variables—does not suffer from these complications and is easily compared. We therefore work with manifest distributions in this paper.

The comparison of manifest probability distributions crucially depends on having their analytic expressions available to us, but unfortunately, this is not the case for the latent trait models that we study here. To overcome this complication, we reverse-engineer an approach that was originally used by Kac (1968) to find a latent variable expression of a graphical model known now as the Ising model (Ising, 1925). The work of Kac has revealed a broad equivalence between psychometric item response models and network models from statistical physics (Epskamp, Maris, Waldorp, & Borsboom, 2018; Marsman et al., 2018; Marsman, Tanis, Bechger, & Waldorp, 2019). Here we use it to characterize the manifest distribution of latent trait models that are in the exponential family. Another way to express the manifest distributions of latent trait models is the Dutch identity (Holland, 1990). Our approach and the Dutch identity are, of course, very much related, and we will study this relation in detail.

The remainder of this paper is structured as follows: In the next section, we formally introduce the three latent trait models. We will focus on versions of the latent trait models that either use or imply the two-parameter logistic model for the marginal distribution of response accuracies—i.e., the conditional distribution of accuracies given the latent variables after having integrated out the response times. After having introduced the three latent trait models we introduce our approach for characterizing their manifest probability distributions. Here we will also study the relation between our approach and the Dutch identity. We then characterize and analyze the manifest probability distributions that are implied by the three latent trait models. Our paper ends with a discussion of these results.

Models

Before we introduce the three latent trait models we first wish to introduce some notation and clarify our terminology. We will assume that the item parameters of the latent trait models are fixed constants, but that the accuracies, the response times, and the latent variables are random. Since these variables are assumed to be random everywhere in this paper, we do not distinguish between (vectors of) random variables and their realizations. We will use to denote a vector of p response accuracies——and use to denote a vector of p response times— for the HM and DM and for the SM, see below. The two latent variables ability and speed will be denoted with and , respectively. Finally, we will use “marginal distribution” to refer to the conditional distribution of one type of observable, e.g.,where the other observables have been integrated out, and we will use “manifest distribution” to refer to distributions of the formwhere the latent variables have been integrated out.

The Hierarchical Model

The HM, as proposed by van der Linden (2007), is a general statistical framework for modeling accuracies and responses times that is based on the idea that there are two latent traits at work; ability governs the response accuracy distribution and speed the response time distribution. Importantly, the response accuracy distribution is assumed to be independent of speed given ability , the response time distribution is assumed to be independent of ability given speed , and it is also assumed that accuracies and response times are independent given the full set of latent traits, i.e.,This setup provides a plug-and-play framework for modeling accuracy and response times: The measurement model for ability —marginal distribution of accuracies —can be chosen independently of the measurement model for speed —marginal distribution of response times . The HM is concluded with a model for the two latent traits.

Different measurement models for ability have been used in the literature. For example, van der Linden (2007) used normal ogive models, Bolsinova, De Boeck, and Tijmstra (2017) used their logistic counterparts, while Zhan, Jiao, and Liao (2018) used cognitive diagnosis models instead. Here, we use the two-parameter logistic model,where is an item discrimination parameter and is an item easiness parameter. There are two reasons for using the two-parameter logistic model here. Firstly, the marginal distribution of accuracies that is implied by the versions of the SM and DM that are used here is also a two-parameter logistic model. Secondly, the two-parameter logistic model is a member of the exponential family of distributions, which will be convenient for expressing the manifest probability distribution .

Different measurement models for speed have also been used in the literature. For example, van der Linden (2006) used a log-normal distribution, Fox, Klein Entink, and van der Linden (2007) used a linear factor model for log-transformed response times, and Klein Entink, van der Linden, and Fox (2009) used models based on Box–Cox transformations of response times. In this paper we will use the log-normal distribution,where is an item time intensity parameter and an item precision parameter. The item precision is the response time analogue of the item discrimination in the two-parameter logistic model; larger values of imply that speed explains a larger portion of the log-time variance. The log-normal distribution is a common choice for the measurement model for speed in the HM framework and is also a member of the exponential family of distributions.

To conclude the HM we specify a distribution for ability and speed . Typically, a bivariate normal distribution is used in which ability and speed are correlated. To identify the model, however, the means of the bivariate normal need to be constrained to zero and the marginal variance of ability needs to be constrained to one.1

The Signed Residual Time Model

The SM has been proposed by Maris and van der Maas (2012) as a measurement model for ability in the context of tests with item-level time limits. The model was specifically designed for tests that use the following scoring rule,where is the time limit for item i. This scoring rule encourages persons to work fast but punishes guessing: Residual time is gained when the response is correct, but is lost when the response is incorrect. Van Rijn and Ali (2017b) demonstrated that the SM is also appropriate for applications where these time limits are not specified a priori, but “estimated” from the observed response time distributions.

The SM specifies the following distribution for accuracy and response times :where is an item easiness parameter. Observe that the SM is an exponential family model and that the scoring rule is the sufficient statistic for ability . The SM has been generalized by van Rijn and Ali (2017b) allowing the items to differ in their discriminative power even when the time limits are the same across items. In this paper we will use the standard version of the SM.

Whereas the HM characterizes the joint distribution of accuracies and response times by specific choices of the marginals and , the SM directly specifies a joint distribution for accuracies and response times . By integrating out the response times we obtain the marginal distribution for accuracies . Maris and van der Maas (2012) show that this marginal distribution is the two-parameter logistic model in Eq. (1), where the item discrimination is equal to the item time limit . In a similar way, we obtain the marginal distribution of response times by summing out the accuracies,An alternative specification of the SM is in terms of accuracies and what Maris and van der Maas (2012, p. 624) refer to as pseudo-response times . Pseudo-response times are obtained from response times through the transformationThis transformation from response times to pseudo-response times is one-to-one, so that no information is lost. One convenient feature of using pseudo-response times instead of response times is that the pseudo-response times and accuracies are (conditionally) independent in the SM, i.e.,where the marginal distribution is equal to,

The Drift Diffusion Model

The DM was introduced by Ratcliff (1978) as a model for two-choice experiments. In the DM, evidence for either choice accumulates over time until a decision boundary is reached. One way to characterize this evidence accumulation process is in terms of a Wiener process with constant drift and volatility, and absorbing upper and lower boundaries (Cox & Miller, 1970). The drift of the diffusion process determines how fast information is accumulated, the volatility determines how noisy the accumulation process is, and the distance between the two boundaries determines how much evidence needs to be accumulated before a choice is made. The process has two additional parameters: a bias parameter z that indicates the distance from the starting point to the lower boundary, and the non-decision time . A commonly used simplification of the DM assumes that the process is unbiased .

The DM has been extended to model differences between persons and tasks. For example, Tuerlinckx and De Boeck (2005) proposed to decompose the drift of the accumulation process into a person and an item part —i.e., —and to treat the distance between the boundaries as an item characteristic —i.e., . The person component in the drift specification carries the interpretation of an ability in item response theory models, as a higher value of implies an increased probability of choosing the correct alternative. To identify the DM, Tuerlinckx and De Boeck (2005) fixed the volatility to one. The joint distribution of decision times and the chosen alternatives —i.e., response accuracies if the upper and the lower boundaries correspond to the correct and incorrect responses—is then equal to:Both the SM and the DM directly specify a joint distribution of accuracies and decision times (response times) that is based on one latent trait, ability . In contrast to the SM, however, accuracies and response times are independent given ability in the (unbiased) DM. The marginal is the two-parameter logistic model in Eq. (1), where the discrimination parameter equals the distance between the boundaries in the diffusion process, and the easiness parameter is an item effect on drift of the diffusion process. The marginal is equal toEven though the marginal distribution is a member of the exponential family, neither the marginal distribution nor the joint distribution is a member of the exponential family. The primary reason that the latter cannot be written in exponential family form is because it implies a statistic that is sufficient for and another statistic that is sufficient for . If we express the latter as , we end up with an exponential family model subject to constraints on the parameters and : is functionally related to ability . This is known as a curved exponential family model (Efron, 1975, 1978).

Characterizing Manifest Probabilities of Latent Trait Models

We consider the general case of an item response theory (IRT) model for accuracy and response times in an exponential family form:where is a (possibly vector-valued) statistic that is sufficient for the (possibly vector-valued) latent variable , and the function is a base measure that does not depend on the value of this latent variable. The base measure serves as the probability measure when the exponential term in Eq. (7) is given no weight, i.e., when . Finally, the function is a normalizing constant that is defined aswhich, when it exists, ensures that the probabilities add up to one.

We will make use of the fact that the three latent trait models can be written in the form of Eq. (7), and outline an approach for expressing the manifest distribution for latent trait models of this form. But since Eq. (7) ignores any functional relation that may exist between its latent variables, a variant of our approach needs to be used to express the manifest distribution for the DM. We will point out how our approach can be used for models, such as the DM, that can be written in the formFor latent trait models that are of the form of Eq. (7) or Eq. (8) we can make use of the following latent variable distribution to express its manifest distribution.

Definition 1

For models that are of the form of Eq. (7) or Eq. (8) we may define the latent variable distributionwhere is a kernel density and is the normalizing constant of . For every kernel distribution for which the normalizing constant is finite, i.e.,where is the support of , is a valid probability distribution.

The distribution in Definition 1 was inspired by the latent trait distribution that has been introduced with the latent variable expression of a graphical model from physics known as the Ising (1925) model by Kac (1968, see also Marsman et al., 2018; Epskamp, Maris, Waldorp, & Borsboom, 2018), but a similar construction can also be found in, for instance, Cressie and Holland (1983, Eq. A9) and McCullagh (1994). We can now state our first result.

Theorem 1

When is of the form of Eq. (7) and the latent variable distribution in Eq. (9) is a valid probability distribution, then the manifest distribution is given bywhere is a normalizing constant and the expectation is an integral with respect to the kernel density .

We omit the simple proof of Theorem 1, which requires one to fill in the definitions of the models in Eqs. (7) and (9), and then integrate out the latent variable . In a similar way, the manifest distribution for latent trait models that are of the form of Eq. (8) can be expressed asTheorem 1 shows that for any latent trait model of the form of Eq. (7), combined with a latent variable distribution of the form of Eq. (9), the manifest distribution can be characterized in terms of the base measures and the moment generating function of the kernel distribution . This is similar to Holland’s (1990) Dutch identity, which was initially formulated for locally independent binary response models by Holland (1990) and extended to a locally dependent response model by Ip (2002) and a polytomous response model by Hessen (2012). The following theorem gives an extension of the Dutch identity for response models that are of the exponential family form of Eq. (7). Its proof is in the Appendix.

Theorem 2

(The Dutch identity) Suppose that is of the formwhere is the support of , and is of the form of Eq. (7). Then for any vector and , where and denote the support of and , respectively, we havewhere the expectation is an integral with respect to the posterior density .

It is easy to verify that whenever is of the form of in Eq. (9), the identity in Theorem 2 reduces towhich was to be expected from Theorem 1. Observe that in this case the expectations imply integrating with respect to the kernel density that is used to define in Eq. (9).

Both Theorems 1 and 2 characterize the manifest distribution in terms of a moment generating function, and for their practical application it is important to find a convenient form for this moment generating function. The Dutch identity, for example, has provided a general analytic solution for the (extended) Rasch model (Cressie & Holland, 1983; Tjur, 1982), but to come to an analytic expression for other latent trait models an assumption has to be made about the posterior distribution of the latent variable. In a similar way, we have to choose a kernel for the practical application of Theorem 1. The following corollary shows how a multivariate normal kernel distribution can be used to express the manifest distribution in a simple analytic form.

Corollary 1

If is of the form in Eq. (7), and the latent variable distribution is of the form in Eq. (9) with a multivariate normal kernel having a mean vector and covariance matrix , then (i)where is a normalizing constant, and (ii) the posterior distribution is multivariate normal with mean vector and covariance matrix .

We will omit the proof of Corollary 1, which requires one to insert in Theorem 1 the moment generating function of the multivariate normal distribution, i.e.,using as the interpolating parameter vector. Since we cannot make use of this moment generating function for the curved exponential family models in Eq. (8), the normal kernel does not result in the same simple form for these models. We can, however, make use of the following identitywhere the expectation is taken with respect to a normal distribution with mean m and variance . Inserting this identity into the expression of the manifest distribution in Eq. (10) with and , we end up with the following expressionThis is a relatively simple analytic expression when we set m to zero. We will use Corollary 1 and Eq. (12) to characterize the manifest probability distributions of the three latent trait models.

Corollary 1 mirrors the results for assuming posterior normality of the latent trait in combination with the Dutch identity as evidenced in Corollary 1 of Holland (1990), Corollary 1 of Ip (2002), and Theorem 2 of Hessen (2012), see also the log-multiplicative association models of Anderson and Vermunt (2000), and Anderson and Yu (2007), and the fused latent and graphical IRT model of Chen, Li, Liu, and Ying (2018).

There appears to be a deeper connection between the prior assumption that leads to our Corollary 1 and the posterior assumption that leads to Corollary 1 of Holland (1990). What we know is that the latent variable distribution in Eq. (9) with a normal kernel is one way to ensure posterior normality of the latent variables for any latent trait model of the form of Eq. (7). To see why this is the case, consider the posterior with a prior distribution of the form of Eq. (9),which is proportional to a multivariate normal distribution if and only if the kernel is a multivariate normal distribution. The reverse need not be true since there exists at least one counterexample: Suppose that follows a multivariate normal distribution with a mean equal to (or some linear function of ), such that can be written in the form of Eq. (7), then a normal prior distribution for , which is not of the form of Eq. (9), also ensures posterior normality of the latent variables. It may well be the case that this is the only exception to the general correspondence between the prior assumption that leads to our Corollary 1 and the posterior assumption that leads to Corollary 1 of Holland (1990).

The Manifest Probabilities of the Three Latent Trait Models

In this section we use Corollary 1 and Eq. (12) to characterize the manifest distributions for the three latent trait models. For the HM and SM we will use Corollary 1 to generate a manifest distribution over the realizations of some vector random variable that is of the formwhere denotes a vector of intercepts, denotes a symmetric matrix of pairwise interactions, denotes a base measure that serves as the probability measure when the pairwise interactions are equal to zero, and denotes the model’s normalizing constant. For the DM we will use Eq. (12) to generate a manifest distribution over the realizations of a random vector that is of the formwhich resembles the manifest distribution in Eq. (13), except that the pairwise interactions are now “weighted” by h. What we shall see is that the three latent trait models fundamentally differ in what the random variable is, revealing key differences in the function of response time they are modeling, but also that accuracies and responses times are dependent in the manifest distribution of each latent trait model, except for fringe cases that are uncommon in practice. We will now consider each of the models in turn.

The version of the HM that is considered here is of the formwhere the marginal is the two-parameter logistic model introduced in Eq. (1), the marginal is the log-normal distribution introduced in Eq. (2), and is of the form of Eq. (9) using a bivariate normal kernel distribution for ability and speed , using a mean vector and covariance matrixwhere denotes the a priori correlation between ability and speed, and the a priori variance of speed.

To come to an expression for the manifest probability distribution of this version of the HM we first rewrite the conditional distribution of accuracies and response times to fit the form of Eq. (7). To this aim, we introduce the statisticthe base measuresand the normalizing constantsHaving expressed the conditional distribution of accuracies and response times in the form of Eq. (7), we can now apply Corollary 1 to obtain the manifest distribution. It is convenient to characterize this manifest distribution in terms of log-transformed response times instead of response times. The manifest probability distribution of accuracies and log-transformed response times that results is equal towhere refers to Hadamard product. Observe that this manifest distribution is of the form of Eq. (13) for a random variable , with interceptsa rank two matrix of pairwise interactionsand a normal base measurewith precisions . In this model the associations, or pairwise interactions, between accuracies and log-response times, are of the opposite sign of the correlation between the two latent variables of the HM: Faster responses correspond to correct answers when ; slower responses correspond to correct answers when .

There are two important characteristics that can be observed from the manifest probability distribution of our version of the HM. Firstly, the base measure that is used here stipulates an a priori restriction on the variance of the speed parameter and the item-specific precisions . To see this, observe that for this base measure the manifest distribution is a proper probability distribution—i.e., integrates to one—if and only ifSince the variance of the is equal tothis restriction on and implies that the speed variable can account for less than 50% of the total variance of the .

A second important characteristic that can be observed from the manifest probability distribution of our version of the HM concerns the associations between accuracies and log-response times, which are encoded in the matrix of pairwise interactions . First, note that when the interaction between an accuracy and log-transformed response time in the manifest distribution is equal to zero, these variables are independent conditional upon the remaining accuracies and log-transformed response times:Thus, and are conditionally independent whenever one of the following conditions apply: The correlation between speed and ability is zero; the discrimination of item i is zero; the precision of item i is zero, and/or the a priori variance of the speed variable is zero. The only non-trivial condition that leads to conditional independence between accuracy and response time is the a priori independence of ability and speed in the HM. But this entails the extreme case in which all of the accuracies are independent of all of the response times, which is unlikely to occur in psychometric practice.

There are two versions of the SM that are considered here. The first version of the SM stipulates a distribution of accuracies and residual response times, and the second version of the SM stipulates a distribution of accuracies and pseudo-response times. We will first characterize the manifest probability distribution of accuracies and residual times and revert to pseudo-response times after that.

The Manifest Distribution of Accuracy and Residual Response Time

The SM was introduced in Eq. (3) and can be expressed in the exponential family form of Eq. (7) with statisticsbase measuresand normalizing constantsHaving expressed the conditional distribution of accuracies and residual times of the SM in the form of Eq. (7) we can now use Corollary 1 to obtain their manifest distribution. Assuming a normal kernel with mean and variance , Corollary 1 leads us to the following manifest distribution,where is the unit vector of length p. One way to write this distribution more succinctly is to express it in terms of the random variables and residual times , which givesThis is of the form of the manifest distribution in Eq. (13) for the random variable , with intercepts , a rank one matrix of pairwise interactions , and a uniform base measure . In this model, larger residual times (faster responses) are associated with an increased probability that a person responds accurately to easy items () and a decreased probability to respond accurately to difficult items (). The association between the response accuracies of different items is positive and increases with increasing residual response times (faster responses).

There are two important characteristics that can be observed from the manifest probability distribution of the SM. One characteristic appears when we view the manifest distribution as a distribution for the random variables , as this closely resembles a graphical model from physics that is known as the Ising model (Lenz, 1920; Ising, 1925). The Ising model is characterized by the following probability distribution over realizations of where denotes a vector of p intercepts , and is a symmetric matrix of pairwise interactions , similar to our Eq. (13). However, where the intercepts and interactions are fixed effects in the Ising network model, they are random effects here, with and . This view of the SM thus provides a novel way for modeling the intercepts and matrix of associations in the Ising model.

A second important characteristic that can be observed from the manifest probability distribution of the SM concerns the association between accuracies and residual response times. Whereas we have found that for the manifest distribution of the HM that accuracy can be conditionally independent of response time in a non-trivial manner, at least in theory, this is not the case with the SM. This can be observed, for example, from the conditional distribution of accuracy and residual response time for an item i given the accuracies and residual times of the remaining itemsfrom which it is clear that there are no values of (or and ) that render and (conditionally) independent.

The Manifest Distribution of Accuracy and Pseudo-response Time

An alternative formulation of the SM is in terms of accuracies and pseudo-response times, which is of the formwhere the marginal is the two-parameter logistic model in Eq. (1), and the marginal is given in Eq. (5).

To characterize the manifest distribution of accuracies and pseudo-response times for this version of the SM we can take two approaches. Firstly, we may express the conditional distribution of accuracies and pseudo-response times in the exponential family form of Eq. (7) and then apply Corollary 1 using a normal kernel distribution with mean and variance to this conditional distribution. Alternatively, we may rewrite the sufficient statistic for residual response times in the manifest distribution in Eq. (15) through the relationBoth approaches lead to the following manifest distribution of accuracies and pseudo-response timeswhich is of the form of the manifest distribution in Eq. (13) for the random variable , with interceptsa rank one matrix of pairwise interactionsand a uniform base measure . Observe that for this model both the associations between accuracies and the associations between pseudo-response times are positive, yet the associations between accuracies and pseudo-response times are negative: Correct responses are associated with smaller pseudo-time values (faster response times); incorrect responses are associated with larger pseudo-time values (slower response times).

The manifest distribution of accuracies and pseudo-response times of the SM is of the same form as the manifest distribution of accuracies and the log-transformed response times of the HM, and thus they share certain characteristics. For example, both models share the following Markov property: When the association between an accuracy and a response time for an item i is equal to zero, then this implies that these two variables are independent conditional upon the remaining accuracies and response times,However, it is immediately clear that this association is never zero in practice, since would imply a zero second time limit for item i.

The version of the DM that is considered here—which was introduced in Eq. (6)—can be expressed in the form of Eq. (8), with statisticsbase measuresand normalizing constantsHaving expressed the DM in the form of Eq. (8), we may now use Eq. (12) to express its manifest distribution. Assuming a latent trait distribution of the form of Eq. (9) with a normal kernel with a mean and variance , Eq. (12) leads us to the following manifest distributionwhere , and is a base measure,This is of the form of the manifest distribution in Eq. (14) for the random variable , that is characterized by the interceptsa rank one matrix of pairwise interactionsthe weight function , and the aforementioned base measure.

There are two important characteristics that can be observed from the manifest probability distribution of our version of the DM. The first observation is that the association between both accuracies and response times is scaled by the total time that is spent on the test. This implies smaller associations between accuracies and response times for pupils that take longer to complete the test, and larger associations for pupils that take less time to complete the test. In none of the three other manifest probability distributions that were considered here have we seen an influence of the total time that was spent on the test.

A second observation is that since the interaction between accuracy and response time is scaled with the total test time, i.e.,and the same holds for the interactions between accuracies, i.e.,the accuracy of an item is related to all of the response times. There is only one way for the accuracy of an item to be conditionally independent from its response time in the manifest distribution, which is when the associated discrimination is equal to zero. However, the accuracy is then not only independent of all of the response times, but also of all remaining accuracies (and of ability in the latent trait formulation). As a consequence, accuracy and response time are conditionally independent only in a fringe case that we do not expect to see in psychometric practice.

Discussion

The goal of this paper was the statistical comparison of three latent trait models for accuracy and response time: the hierarchical model (HM) of van der Linden (2007), the signed residual time model (SM) of Maris and van der Maas (2012), and the drift diffusion model (DM) as proposed by Tuerlinckx and De Boeck (2005). Our idea was to work with the manifest distributions of observables that were generated by these latent trait models, as they are more easily compared than their original latent trait formulations. To characterize these manifest distributions we have reverse-engineered an approach by Kac (1968), which inspired a new method for expressing manifest distributions. This method is summarized in our Theorem 1 and Corollary 1 and is related to the Dutch identity (Holland, 1990), which is our Theorem 2 for the response models considered in this paper. Our assumption of a normal kernel density for the latent trait parameters appeared to be closely related to the posterior normality assumption that is often used with the Dutch identity, but more importantly, it has allowed us to characterize the manifest distributions of observables analytically. So what did this formal exercise teach us about the three psychometric models?

The observation that accuracies and response times are dependent in the analyzed manifest distributions is a warm reminder of the fact that integrating over a common cause, or set of correlated common causes, will generate a dependency between observables that are conditionally independent. In fact, the statistical modeling of such manifest dependencies is what latent variable models are made for. Viewed in this way, it hardly seems relevant how the three latent trait models treat these dependencies locally, e.g., assuming conditional independence between observables or not, since these local properties have disappeared in the manifest distribution. Given that the manifest probabilities are all that we can ever learn from our observables, the conditional independence property may be a convenient tool to model dependencies at the latent trait level, but for the three response models it is hardly more than that.

A more sensible division of the three latent trait models appears to be the response time function that is being modeled, as it is here that we find major differences between the three response models. For example, the log-transformed response times are modeled in the HM, the residual response times or pseudo-response times are modeled in the SM, and in the DM the response times are modeled directly, although the latter does so in proportion to the total test time. This offers an interesting new view on response models that take response times into account, and one may wonder if there is a way to find out which function tells us the most about the unknown abilities. The manifest distributions in this paper offer one approach to address such questions.

That the conditional independence property at the latent trait level does not resonate in the practical application of the three latent trait models does not imply that this property has no impact on the manifest distribution. To see the impact of the conditional independence property at the manifest level we first note that for exponential family response models our Corollary 1 generates manifest distributions that are of the form of Eq. (13). The distribution in Eq. (13) is a prototypical example of a Markov random field (MRF; Kindermann & Snell, 1980), which is an undirected graphical model with certain conditional independence relations —known as Markov properties (e.g., Lauritzen, 2004)—that are encoded in the matrix of pairwise interactions. When accuracy and response time are independent at the latent trait level we may write this interaction matrix aswhere encodes the interactions between response accuracies, the interactions between (functions of) response times, and the interactions between the two types of observables. The division of these interactions allows us to flesh out dependencies between the two types of observables in the manifest distribution, and to specifically model any patterns of interactions that we might observe. If we wish to model such local properties we could start with models of the form of Eq. (13) or we could use higher-dimensional latent trait models (e.g., Epskamp, Kruis, & Marsman, 2015; Marsman, Maris, Bechger, & Glas, 2017; Marsman, Waldorp, & Maris, 2017).

Even though the manifest probability distributions of the DM and SM for residual times are not MRFs with respect to the two types of observables,2 we observed some interesting properties in their manifest distributions. In the manifest expression for the DM, for example, the associations between accuracies and response times are a function of the total time the pupil has spent on the test, an aspect that is not being modeled in any of the other manifest expressions. This property could be used to inform about the underlying strategies that pupils use, for example. The manifest expression of the SM, on the other hand, provides a new and interesting way to view an old model, the Ising model. The Ising model is an undirected graphical model that is characterized by the following distribution,where is a p-dimensional vector of or variables , a p-dimensional vector of main effects, and a symmetric matrix of pairwise associations between variables. Whereas the pairwise associations are fixed effects in the Ising model, the manifest distribution of the SM indicates one way to model these associations as a random effect.

One famous conjecture from Holland (1990, p. 11) is that if there are large number of items on a test, and a smooth unidimensional IRT model (for accuracies) is used, the posterior distribution of the latent trait will be approximately normal. This conjecture has inspired several publications on the posterior normality of the latent trait in the context of IRT models for response accuracy (e.g., Chang and Stout, 1993; Chang, 1996; Zhang and Stout, 1997). An interesting conclusion that Holland (1990) deduced from this conjecture, in combination with the assumption that the log-likelihoods of the p items can be approximated using a p-variate normal with a rank one covariance matrix, is that the log of the manifest distribution of accuracy is approximately of quadratic form consisting of p main effects and a matrix of associations that was of rank one. This enticed Holland (1990) to add a second conjecture that only two parameters can be consistently estimated per item. This idea points to interesting avenues of future research, such as the asymptotic posterior normality of the latent trait in the context of IRT models for response accuracy and response times. If it is reasonable to approximate the posterior of the latent trait (or the log-likelihood function) with a normal distribution, then we can use this approximation in combination with Corollary 1 or Theorem 2 to investigate the complexity of models for response accuracy and response times, and how model complexity is impacted by the conditional independence property of the underlying latent trait model.

The latent variable distribution has allowed us to express the manifest probability distributions for a large class of latent trait models, but it also generated an unexpected parameter restriction in the manifest distribution of the HM, where we found that the variance of the speed variable needed to be smaller than the smallest log-normal variance . This parameter restriction follows from omitting the normalizing constants of the latent variable model in Eq. (7), which provides prior model structure. When a regular latent variable distribution is used—for example, a normal distribution on —the model structure that is provided by the normalizing constants is integrated instead. Marsman et al. (2018) studied a similar scaling issue of the posterior distribution that results from using the latent variable distribution in the context of multi-dimensional IRT (see also Marsman et al., 2017). The correspondence that we have found between our normal kernel assumption and the posterior normality assumption with the Dutch identity suggests that similar observations can be made for the prior and posterior of the latent variables in Corollary 1 of Holland (1990), Corollary 1 of Hessen (2012), and Theorem 1 in Ip (2002).

The particular restriction that is imposed on the variance of the speed variable in the HM is a rather strong restriction from a substantive point of view. From the perspective of the manifest distribution, however, it might be less of an issue since is simply a scaling factor for the interactions between the log-transformed response times and accuracies . That is, the manifest structure would not change when we absorbed in the precisions and simply use the matrix of associations:Alternatively, we may adopt a different base measure to remove the restriction.