arXiv research

Paper establishes statistical inference for pairwise comparison models.

New proposed models are often compared to state-of-the-art using statistical significance testing. Literature is scarce for classifier comparison using metrics other than accuracy. We present a survey of statistical methods that can be used for classifier comparison using precision, accounting for inter-precision corre…

Paper proposes a sequential statistical test for comparing imitation learning policies with near-optimal stopping.

New method compares classifiers using GSD-front, addressing statistical uncertainty and robustness.

Statistical framework improves LLM chatbot ranking.

Develops a statistical framework to measure uncertainty in model rankings based on human preferences.

SC improves robustness in model comparison for misspecified models.

AdaStop improves statistical testing for Deep RL algorithm comparisons.

New method uses model comparison signals to improve LLM evaluation accuracy.

This paper compares and evaluates methods for evaluating statistical models using benchmarking data and simulations.

This work analyzes Fréchet regression using comparison geometry, providing theoretical and practical insights.

Paper proposes a new method to compare classifiers across multiple datasets.

We develop a new statistical test for comparing variables with varying scales.

New model for pairwise comparisons without stochastic transitivity.

Enhanced 3D shape analysis using information geometry.

The main object of Bayesian statistical inference is the determination of posterior distributions. Sometimes these laws are given for quantities devoid of empirical value. This serious drawback vanishes when one confines oneself to considering a finite horizon framework. However, assuming infinite exchangeability gives…

Active learning aims to obtain a classifier of high accuracy by using fewer label requests in comparison to passive learning by selecting effective queries. Many active learning methods have been developed in the past two decades, which sample queries based on informativeness or representativeness of unlabeled data poi…

Develops a method to infer partial rankings from sparse comparisons.

In this short paper, we overview and extend the results of our papers cond-mat/0001432, cond-mat/0008305, and cond-mat/0103544, where we use an analogy with statistical physics to describe probability distributions of money, income, and wealth in society. By making a detailed quantitative comparison with the available …

The paper ranks items based on top choices in multiway comparisons.

Statistical models outperform mechanistic models in short-term COVID-19 incidence forecasts.

New method improves ABC for Bayesian model comparison.

Study geodesic paths on flat surfaces, comparing length and singularity counts.

We address the classical problem of hierarchical clustering, but in a framework where one does not have access to a representation of the objects or their pairwise similarities. Instead, we assume that only a set of comparisons between objects is available, that is, statements of the form "objects $i$ and $j$ are more …

Measures dependence between two systems using Bayesian model comparison.

The paper improves spectral ranking methods for diverse comparison graphs.

Many complex systems can be represented as networks, and the problem of network comparison is becoming increasingly relevant. There are many techniques for network comparison, from simply comparing network summary statistics to sophisticated but computationally costly alignment-based approaches. Yet it remains challeng…

A statistical toolbox for analyzing model performance in medical imaging.

Unified framework to bridge human and LLM judgments.

MIRA scores assess conditional distribution accuracy using joint samples.

Proposes Population Difference Criterion for visually observed subpopulation differences.

Lossy compression of statistical data using quantum annealing.

A new ranking model with dynamic covariates improves statistical analysis.

New methods resolve conflicting treatment effect estimates in health tech assessments.

A novel method compares 3D point clouds using information geometry.

There are various parametric models for analyzing pairwise comparison data, including the Bradley-Terry-Luce (BTL) and Thurstone models, but their reliance on strong parametric assumptions is limiting. In this work, we study a flexible model for pairwise comparisons, under which the probabilities of outcomes are requir…

Confidence intervals improve evaluation of binary prediction rules in data mining.

Consistently checking the statistical significance of experimental results is one of the mandatory methodological steps to address the so-called "reproducibility crisis" in deep reinforcement learning. In this tutorial paper, we explain how the number of random seeds relates to the probabilities of statistical errors. …

Approximate Bayesian Computation (ABC) are likelihood-free Monte Carlo methods. ABC methods use a comparison between simulated data, using different parameters drew from a prior distribution, and observed data. This comparison process is based on computing a distance between the summary statistics from the simulated da…

The article explains how to estimate confusion matrices for classifiers using unlabeled data.

BSD is a Bayesian framework for analyzing neural spectral data.

Deep learning, computational neuroscience, and cognitive science have overlapping goals related to understanding intelligence such that perception and behaviour can be simulated in computational systems. In neuroimaging, machine learning methods have been used to test computational models of sensory information process…

We investigate the forecasting ability of the most commonly used benchmarks in financial economics. We approach the usual caveats of probabilistic forecasts studies -small samples, limited models and non-holistic validations- by performing a comprehensive comparison of 15 predictive schemes during a time period of over…

Statistical inference using pairwise comparison data is an effective approach to analyzing large-scale sparse networks. In this paper, we propose a general framework to model the mutual interactions in a network, which enjoys ample flexibility in terms of model parametrization. Under this setup, we show that the maximu…

ERM with square loss achieves sublinear error for learnable function classes with smoothed data.

New tests compare regression functions using machine learning, overcoming dimensionality issues.

This paper investigates how to measure common market risk factors using newly proposed Panel Quantile Regression Model for Returns. By exploring the fact that volatility crosses all quantiles of the return distribution and using penalized fixed effects estimator we are able to control for otherwise unobserved heterogen…

Stock markets are complex systems exhibiting collective phenomena and particular features such as synchronization, fluctuations distributed as power-laws, non-random structures and similarity to neural networks. Such specific properties suggest that markets operate at a very special point. Financial markets are believe…