arXiv research

Reply to Tetlock et al. on tail risk and probability gap.

Proposes a flexible tournament design combining knockout and round-robin.

Classifies intrinsically linked tournaments by their score sequences.

We explore a new way to evaluate generative models using insights from evaluation of competitive games between human players. We show experimentally that tournaments between generators and discriminators provide an effective way to evaluate generative models. We introduce two methods for summarizing tournament outcomes…

A directed graph $G$ is $\textit{intrinsically linked}$ if every embedding of that graph contains a non-split link $L$, where each component of $L$ is a consistently oriented cycle in $G$. A $\textit{tournament}$ is a directed graph where each pair of vertices is connected by exactly one directed edge. We consider intr…

A regularized risk minimization procedure for regression function estimation is introduced that achieves near optimal accuracy and confidence under general conditions, including heavy-tailed predictor and response variables. The procedure is based on median-of-means tournaments, introduced by the authors in [8]. It is …

Extracts StarCraft II tournament data for AI and ML studies.

Breaks the hardness conjecture for batch RL with a novel tournament-based approach.

We propose a novel ranking model that combines the Bradley-Terry-Luce probability model with a nonnegative matrix factorization framework to model and uncover the presence of latent variables that influence the performance of top tennis players. We derive an efficient, provably convergent, and numerically stable majori…

The paper tackles optimal level set estimation in crowdsourcing and tournaments.

Elo ratings learn model parameters quickly using Markov chains.

Receiver operating characteristic (ROC) analysis is widely used for evaluating diagnostic systems. Recent studies have shown that estimating an area under ROC curve (AUC) with standard cross-validation methods suffers from a large bias. The leave-pair-out (LPO) cross-validation has been shown to correct this bias. Howe…

Co-designing efficient machine learning based systems across the whole hardware/software stack to trade off speed, accuracy, energy and costs is becoming extremely complex and time consuming. Researchers often struggle to evaluate and compare different published works across rapidly evolving software frameworks, hetero…

We are interested in parallelizing the Least Angle Regression (LARS) algorithm for fitting linear regression models to high-dimensional data. We consider two parallel and communication avoiding versions of the basic LARS algorithm. The two algorithms have different asymptotic costs and practical performance. One offers…

New algorithms for model selection in off-policy evaluation of reinforcement learning.

Directed graphs occur throughout statistical modeling of networks, and exchangeability is a natural assumption when the ordering of vertices does not matter. There is a deep structural theory for exchangeable undirected graphs, which extends to the directed case via measurable objects known as digraphons. Using digraph…

This paper extends Median-of-Means to new learning problems involving pairwise comparisons.

In this paper we demonstrate how genetic algorithms can be used to reverse engineer an evaluation function's parameters for computer chess. Our results show that using an appropriate mentor, we can evolve a program that is on par with top tournament-playing chess programs, outperforming a two-time World Computer Chess …

The paper derives upper bounds on the MLE error for BTL model under general graphs.

The small-ball method was introduced as a way of obtaining a high probability, isomorphic lower bound on the quadratic empirical process, under weak assumptions on the indexing class. The key assumption was that class members satisfy a uniform small-ball estimate: that $Pr(|f| \geq κ\|f\|_{L_2}) \geq δ$ for given const…

The paper studies dynamic ranking and translation synchronization from evolving pairwise comparison graphs.

In this paper we demonstrate how genetic algorithms can be used to reverse engineer an evaluation function's parameters for computer chess. Our results show that using an appropriate expert (or mentor), we can evolve a program that is on par with top tournament-playing chess programs, outperforming a two-time World Com…

A method for dynamic ranking using BTL model and nearest neighbor rank centrality.

Bayesian model infers strengths from noisy tennis match outcomes.

Recently we proposed a general, ensemble-based feature engineering wrapper (FEW) that was paired with a number of machine learning methods to solve regression problems. Here, we adapt FEW for supervised classification and perform a thorough analysis of fitness and survival methods within this framework. Our tests demon…

Fantasy Premier League (FPL) performance predictors tend to base their algorithms purely on historical statistical data. The main problems with this approach is that external factors such as injuries, managerial decisions and other tournament match statistics can never be factored into the final predictions. In this pa…

Optimal reconciliation keeps some forecasts unchanged in hierarchical forecasting.

New method estimates optimizer for convex stochastic problems.

Short-term load forecasting is a critical element of power systems energy management systems. In recent years, probabilistic load forecasting (PLF) has gained increased attention for its ability to provide uncertainty information that helps to improve the reliability and economics of system operation performances. This…

Combining forecasts of 16 ED causes improves accuracy and stability.

Pairwise comparison data arises in many domains, including tournament rankings, web search, and preference elicitation. Given noisy comparisons of a fixed subset of pairs of items, we study the problem of estimating the underlying comparison probabilities under the assumption of strong stochastic transitivity (SST). We…

Conditional forecasts improve performative prediction accuracy.

Study improves seasonal forecasts using deep learning.

A number of applications (e.g., AI bot tournaments, sports, peer grading, crowdsourcing) use pairwise comparison data and the Bradley-Terry-Luce (BTL) model to evaluate a given collection of items (e.g., bots, teams, students, search results). Past work has shown that under the BTL model, the widely-used maximum-likeli…

Two new methods improve forecasting of functional time series data.

Deep learning improves time series forecasting, outperforming other methods.

Nowadays, with the unprecedented penetration of renewable distributed energy resources (DERs), the necessity of an efficient energy forecasting model is more demanding than before. Generally, forecasting models are trained using observed weather data while the trained models are applied for energy forecasting using for…

MPANF improves naive forecast by incorporating directional information.

Simplifies forecast combination by using diversity of out-of-sample forecasts.

Develops forecast hedging for improved calibration of forecasts.

Microdata improves inflation forecasts after major shocks, study finds.

Improved forecast accuracy for Knitwear by 20% using adaptive AI/ML model.

Given a nonlinear model, a probabilistic forecast may be obtained by Monte Carlo simulations. At a given forecast horizon, Monte Carlo simulations yield sets of discrete forecasts, which can be converted to density forecasts. The resulting density forecasts will inevitably be downgraded by model mis-specification. In o…

Paper proposes a new method for selecting the best hierarchical forecasting approach.

Proposes a neural network for accurate and reconciled hierarchical time series forecasting.

The key contribution of this paper is to propose a classification into two dimensions of the load forecasting studies to decide which forecasting tools to use in which case. This classification aims to provide a synthetic view of the relevant forecasting techniques and methodologies by forecasting problem. In addition,…

This paper reviews forecast combinations over 50 years, highlighting their evolution and utility.

This paper improves forecast stability without sacrificing accuracy using dynamic loss weighting.