arXiv research

A new algorithm estimates item parameters in item response theory models.

New algorithm speeds up IRT model fitting for large datasets.

New Bayesian method for sparse multidimensional item response theory.

The paper presents a machine learning approach to multidimensional item response theory.

New estimators improve Rasch model item parameter estimation for sparse data.

FedIRT enables privacy-preserving psychometric estimation without centralizing data.

Item Response Theory (IRT) is a ubiquitous model for understanding humans based on their responses to questions, used in fields as diverse as education, medicine and psychology. Large modern datasets offer opportunities to capture more nuances in human behavior, potentially improving test scoring and better informing p…

Proposes $β^{4}$-IRT, a new IRT model with enhanced discrimination estimation.

Study presents MMC model for better fitting multiple choice data.

New method for scalable learning of IRT models from large datasets.

GPIRT uses Gaussian processes to estimate latent traits and IRFs from binary responses.

Ensemble unsupervised anomaly detection using IRT for hidden ground truth.

LaRT models LLMs' response accuracy and CoT length to evaluate reasoning ability and speed.

Item Response Theory (IRT) aims to assess latent abilities of respondents based on the correctness of their answers in aptitude test items with different difficulty levels. In this paper, we propose the $β^3$-IRT model, which models continuous responses and can generate a much enriched family of Item Characteristic Cur…

Deep learning based knowledge tracing model has been shown to outperform traditional knowledge tracing model without the need for human-engineered features, yet its parameters and representations have long been criticized for not being explainable. In this paper, we propose Deep-IRT which is a synthesis of the item res…

Framework for fast CAT calibration and administration using AutoML and IRT.

Paper uses NLP and IRT to score ESG factors from news articles.

IRT improves algorithm evaluation across datasets.

Item response theory (IRT) models for categorical response data are widely used in the analysis of educational data, computerized adaptive testing, and psychological surveys. However, most IRT models rely on both the assumption that categories are strictly ordered and the assumption that this ordering is known a priori…

VTIRT speeds up IRT inference for dynamic learner proficiency.

A new framework evaluates large language models efficiently and accurately.

Researchers study how teachers' advising relationships influence their perceptions of satisfaction and students, not policy influence.

Item response theory (IRT) models are widely used in psychometrics and educational measurement, being deployed in many high stakes tests such as the GRE aptitude test. IRT has largely focused on estimation of a single latent trait (e.g. ability) that remains static through the collection of item responses. However, in …

In this article, we propose a novel probabilistic framework to improve the accuracy of a weighted majority voting algorithm. In order to assign higher weights to the classifiers which can correctly classify hard-to-classify instances, we introduce the Item Response Theory (IRT) framework to evaluate the samples' diffic…

Item response theory (IRT) is a non-linear generative probabilistic paradigm for using exams to identify, quantify, and compare latent traits of individuals, relative to their peers, within a population of interest. In pre-existing multidimensional IRT methods, one requires a factorization of the test items. For this t…

This paper introduces our solution to the 2018 Duolingo Shared Task on Second Language Acquisition Modeling (SLAM). We used deep factorization machines, a wide and deep learning model of pairwise relationships between users, items, skills, and other entities considered. Our solution (AUC 0.815) hopefully managed to bea…

IRTCI uses IRT for categorical imputation, outperforming other methods in various datasets.

A new dataset tracks user interactions and click responses in online marketplaces.

ESRLCM clusters similar responses, more broadly than traditional models.

Bayesian method improves adaptive testing item selection, ensuring full item exposure.

The paper examines skill estimation and variance under model misspecification in IRT.

Model learns metrics and preferences from user comparisons.

Archetypal analysis helps understand binary data sets.

NASirt automates CNN architecture design for spectral data.

Suppose that we wish to estimate a user's preference vector $w$ from paired comparisons of the form "does user $w$ prefer item $p$ or item $q$?," where both the user and items are embedded in a low-dimensional Euclidean space with distances that reflect user and item similarities. Such observations arise in numerous se…

Computer simulations have become a popular tool of assessing complex skills such as problem-solving skills. Log files of computer-based items record the entire human-computer interactive processes for each respondent. The response processes are very diverse, noisy, and of nonstandard formats. Few generic methods have b…

Adaptive querying learns user psychometrics with AI personas.

Knowledge tracing is a sequence prediction problem where the goal is to predict the outcomes of students over questions as they are interacting with a learning platform. By tracking the evolution of the knowledge of some student, one can optimize instruction. Existing methods are either based on temporal latent variabl…

An efficient algorithm selects the correct number of latent dimensions in multidimensional probit models.

New IRT method identifies useful datasets for ML classifier evaluation.

Previous work on recommender systems mainly focus on fitting the ratings provided by users. However, the response patterns, i.e., some items are rated while others not, are generally ignored. We argue that failing to observe such response patterns can lead to biased parameter estimation and sub-optimal model performanc…

This paper explores the suitability of using automatically discovered topics from MOOC discussion forums for modelling students' academic abilities. The Rasch model from psychometrics is a popular generative probabilistic model that relates latent student skill, latent item difficulty, and observed student-item respons…

The recently proposed SPARse Factor Analysis (SPARFA) framework for personalized learning performs factor analysis on ordinal or binary-valued (e.g., correct/incorrect) graded learner responses to questions. The underlying factors are termed "concepts" (or knowledge components) and are used for learning analytics (LA),…

Marginal maximum likelihood (MML) estimation is the preferred approach to fitting item response theory models in psychometrics due to the MML estimator's consistency, normality, and efficiency as the sample size tends to infinity. However, state-of-the-art MML estimation procedures such as the Metropolis-Hastings Robbi…

The Rasch model is widely used for item response analysis in applications ranging from recommender systems to psychology, education, and finance. While a number of estimators have been proposed for the Rasch model over the last decades, the available analytical performance guarantees are mostly asymptotic. This paper p…

Using the trends of estimated abilities in terms of item response theory for online testing, we can predict the success/failure status for the final examination to each student at early stages in courses. In prediction, we applied the newly developed nearest neighbor method for determining the similarity of learning sk…

Debias recommender systems by accounting for hidden confounders using network information.

Improved IFA with Generative Adversarial Networks for high-dimensional latent variables.