Maximum likelihood estimation of factor and ideal point models for paired comparison data

Many paired comparison data sets reported in the literature are obtained in a multiple judgment setting where each judge compares all possible item pairs one at a time. Despite the repeated measures structure of the data, typically the multiple judgments are analyzed under the strong assumption of independence. This disregard for dependencies in the paired comparison judgments is a serious model misspecification that may lead to incorrect statistical and substantive conclusions. Building on Takane's (1987, Cognition and Communication, 20, 45-62) analysis of covariance structures models, we present a variant of the Monte Carlo expectation maximization (MCEM) algorithm for estimating probabilistic paired comparison models of multiple paired comparison judgments. The MCEM algorithm is straightforward to implement and converges quickly even when the paired comparison data are sparse. A detailed analysis of a paired comparison experiment illustrates the usefulness of this approach for the interpretation of similarity and individual difference effects in preference data.