Proposes a flexible tournament design combining knockout and round-robin.
problem Designing a tournament that eliminates participants linearly.
method Combines knockout and round-robin structures for flexible elimination.
result Flexible tournament design can eliminate participants linearly.
Classifies intrinsically linked tournaments by their score sequences.
problem Classifying intrinsically linked tournaments using their score sequences.
method Examining the score sequences of tournaments and identifying linkless sequences.
result The vast majority of score sequences for 8-vertex tournaments are linkless.
New method uses game theory to rate generative models.
problem Evaluating generative models' performance.
method Tournaments between generators and discriminators to summarize outcomes.
result Tournament win rate and skill rating provide effective model evaluations.
A new ranking model uses nonnegative matrix factorization for tennis players.
problem Modeling latent variables influencing tennis player performance.
method Combines Bradley-Terry-Luce model with nonnegative matrix factorization.
result Model identifies surface type as key determinant of male player performance.
Directed graphs in tournaments can link or knot in various ways.
problem Understanding intrinsic linking and knotting in directed graphs, specifically tournaments.
method Analyzing the minimum number of vertices required for intrinsic linking or knotting properties in tournaments.
result Bounds on the minimum number of vertices required for various intrinsic linking or knotting properties in tournaments.
A new method, TLPO, corrects bias in ROC curve estimation.
problem Bias in estimating AUC with standard cross-validation methods.
method Tournament leave-pair-out (TLPO) cross-validation.
result TLPO provides an unbiased estimate of AUC and allows ROC curve analysis.
Extracts StarCraft II tournament data for AI and ML studies.
problem Lack of accessible esports data for scientific use.
method Gathered and processed StarCraft II tournament replays using an API parser library.
result The largest publicly available StarCraft II esports dataset.
A regularized risk minimization procedure for regression function estimation is introduced that achieves near optimal accuracy and confidence under general conditions, including heavy-tailed predictor and response variables. The procedure is based on median-of-means tournaments, introduced by the authors in [8]. It is …
Expert-driven genetic algorithms improve chess evaluation functions.
problem Creating efficient evaluation functions for complex systems.
method Using genetic algorithms with expert guidance to evolve evaluation functions.
result Evolved programs outperform top chess programs with fewer parameters.
Reply to Tetlock et al. on tail risk and probability gap.
problem Expert judgment fails to account for tail risk.
method Comparison of forecasting tournaments and extreme value theory.
result Greater gap between tail expectation and probability properties.
Breaks the hardness conjecture for batch RL with a novel tournament-based approach.
problem Sample-efficient reinforcement learning from exploratory data.
method BVFT algorithm using pairwise comparison and state-action partition.
result Solves the learning problem in a setting previously thought impossible.
The paper tackles optimal level set estimation in crowdsourcing and tournaments.
problem Deciphering small entries in a partially observed matrix of expert correctness.
method Constructs an efficient polynomial-time algorithm for recovering level sets up to a precision.
result The algorithm is minimax optimal for the classification problem, contrasting with existing literature.
Elo ratings learn model parameters quickly using Markov chains.
problem Ranking players in online settings.
method Bradley--Terry--Luce model and Markov chain theory.
result Elo learns model parameters at a competitive rate.
New algorithms for model selection in off-policy evaluation of reinforcement learning.
problem Hyperparameter tuning for off-policy evaluation methods in reinforcement learning.
method Developed new model-free and model-based selectors with theoretical guarantees and a new experimental protocol.
result New model-free selector, LSTD-Tournament, demonstrates promising empirical performance.
Parallelizes LARS for high-dimensional data with speedups and accuracy trade-offs.
problem Fitting linear regression models to high-dimensional data efficiently.
method Two parallel and communication avoiding versions of LARS: bLARS and Tournament-bLARS.
result Speedups up to 4x compared to LARS, with trade-offs in solution quality.
ReQuEST platform aims to simplify ML system evaluation across diverse hardware and software.
problem Complexity and time-consuming evaluation of machine learning systems across frameworks and hardware.
method Open co-design tournament platform with online public scoreboard, leveraging Collective Knowledge and ACM artifact methodology.
result Validation and sharing of reproducible machine learning system implementations.
Extends small-ball method to broader class without uniform small-ball condition.
problem Obtaining high probability lower bounds on quadratic empirical processes.
method Extends small-ball method to allow broader class without uniform small-ball condition, motivated by tournament learning.
result Obtains high probability, almost-isometric lower bound on quadratic empirical process.
This paper extends Median-of-Means to new learning problems involving pairwise comparisons.
problem Learning from pairwise comparisons in machine learning.
method Segmenting data into blocks, comparing pairs of decision rules, and declaring the winner based on majority performance.
result The Median-of-Means approach maintains robustness and performance under various sampling schemes.
The paper derives upper bounds on the MLE error for BTL model under general graphs.
problem Estimating the MLE of BTL model parameters with ℓ∞-loss under general graphs. method Novel upper bounds on ℓ∞ estimation error dependent on algebraic connectivity and graph topology. result Upper bounds on ℓ∞ error are sharp and match minimax lower bounds under certain graph topologies. Genetic algorithms optimize chess evaluation functions with mentor assistance.
problem Optimizing complex evaluation functions for superior performance.
method Genetic algorithms with mentor-assisted evolution.
result Programs evolved with mentors outperform top chess champions.
Recently we proposed a general, ensemble-based feature engineering wrapper (FEW) that was paired with a number of machine learning methods to solve regression problems. Here, we adapt FEW for supervised classification and perform a thorough analysis of fitness and survival methods within this framework. Our tests demon…
Improved fantasy football performance predictor using human feedback.
problem Lack of external factors in traditional statistical models.
method Combining statistical data with human feedback from various sources.
result Model outperformed regular statistical predictors by over 300 points.
Directed graphs occur throughout statistical modeling of networks, and exchangeability is a natural assumption when the ordering of vertices does not matter. There is a deep structural theory for exchangeable undirected graphs, which extends to the directed case via measurable objects known as digraphons. Using digraph…
New method improves stock return prediction in non-stationary markets.
problem Tackles the challenge of predicting stock returns in non-stationary environments.
method Jointly optimizes model class and training window size using a tournament procedure.
result Consistently outperforms standard benchmarks by 14-23% in out-of-sample R2. New method estimates optimizer for convex stochastic problems.
problem Estimating optimizer for convex stochastic optimization problems.
method Median-of-means tournament procedure for heavy-tailed data.
result Optimal statistical performance in heavy tailed situations.
The paper studies dynamic ranking and translation synchronization from evolving pairwise comparison graphs.
problem Dynamic pairwise comparison graphs in evolving environments.
method Proposes estimators based on smoothness-penalized least squares and projection onto low frequency eigenspace.
result Finite sample bounds for the ℓ2 estimation error, proving consistency of the proposed methods. A method for dynamic ranking using BTL model and nearest neighbor rank centrality.
problem Aggregating evolving pairwise comparisons to recover item strengths over time.
method Adapting Rank Centrality method to handle dynamic pairwise outcomes.
result Consistency of the method in estimating latent strengths over time.
Bayesian model infers strengths from noisy tennis match outcomes.
problem Ranking tennis players from match outcomes.
method Bayesian approach to infer unobserved strengths and mapping function.
result Bayesian approach robust to different model specifications.
DynAEsti models ability as a time-varying curve, improving IRT for dynamic settings.
problem Traditional IRT models assume static ability; new approach models dynamic ability.
method DynAEsti augments traditional IRT Expectation Maximization with CurvFiFE for curve fitting.
result DynAEsti successfully recovers human performance dynamics in golf.
Agents learn to play a first-person multiplayer game at human level performance.
problem Training AI agents for complex, multi-agent, real-time environments.
method Population-based deep reinforcement learning with concurrent training of multiple agents.
result Achieved human-level performance in a first-person multiplayer game.
Optimizes decision-making in dueling bandits with contextual features.
problem Identifying the best arm in dueling bandits with contextual features.
method Develops algorithms for minimizing regret in stochastic contextual dueling bandits.
result Proves optimal regret bounds for contextual dueling bandits.
New estimators show consistent estimation is possible with randomized item assignment.
problem Estimating underlying comparison probabilities from noisy pairwise comparisons.
method Study permutation-based models under strong stochastic transitivity, randomized item assignment.
result Rates of estimators are optimal for a large class of graphs.
Improved fairness in pairwise comparisons using MLE with a simple modification.
problem Fairness in pairwise comparisons using MLE is suboptimal.
method Proposed a simple modification to MLE to improve fairness (bias) without sacrificing accuracy.
result Improved rate in bias while maintaining minimax-optimality in mean squared error.
Random forest model predicts tennis match outcomes with 80% accuracy.
problem Predicting tennis match outcomes before the game starts.
method Used a large database of tennis match information and a random forest model.
result Identified serve strength as a key predictor of match outcome.
This paper tackles bandit optimization with a new pairwise comparison oracle for unknown strongly concave functions.
problem Maximizing an unknown strongly concave function over T periods with a biased pairwise comparison oracle.
method Introduced a discretization technique and local polynomial approximation to relate the problem to linear bandits. Developed a tournament successive elimination technique to localize the discretized cell and run LinUCB algorithm on cells.
result Established optimal regret bounds and improved state-of-the-art results in operations management problems.
Paper learns skill distributions from game outcomes, proving minimax optimality.
problem Learning skill distributions from noisy pairwise game outcomes.
method Proposes a simple algorithm using kernel density estimation and minimax techniques.
result Near-optimal minimax mean squared error scaling for skill density estimation.
New method to decompose portfolio performance ratios.
problem Understanding the drivers of portfolio performance ratios.
method Using Euler's theorem, decomposes performance ratios into modified ratios.
result Derives condition for new asset to improve portfolio performance.
Direct convolution eliminates memory overhead and improves performance.
problem Memory overhead and suboptimal performance in convolution layers.
method Implementing direct convolution without additional memory overhead.
result Performance improvement between 10% to 400% on various architectures.
This paper optimizes performative risk by focusing on convex properties and developing efficient algorithms.
problem Performative risk, the loss experienced by decision makers, is not optimized by stable models.
method Identifying convex properties of loss function and model-induced distribution shift, developing algorithms for optimization.
result Optimization of performative risk with better sample efficiency than generic methods.
Plug-in method improves performative prediction accuracy.
problem Learning under performative feedback with slow convergence rates.
method Plug-in performative optimization using models.
result Plug-in method can be superior to model-agnostic strategies.
New causal models perform poorly when evaluated on biased training sets.
problem Sample selection bias affects the evaluation of causal models' prediction performance.
method Re-evaluated prediction performance of causal models on a genetic perturbation data set, proposing a less-biased evaluation set.
result Causal models have similar or worse performance when evaluated on a less-biased set compared to standard association-based estimators.
A new framework for performative prediction robust to distributional misspecification.
problem Performative prediction models can be influenced by their own predictions, leading to suboptimal outcomes.
method Introduces distributionally robust performative prediction (DRPO) to approximate the true performative optimum (PO) robustly.
result DRPO provides provable guarantees as a robust approximation to the true PO when the nominal distribution map is misspecified.
Study compares Islamic banks' accounting and market performance.
problem Assessing the relationship between Islamic banks' accounting and market performance.
method Selected six Islamic banks, collected data from 2009-2013, used random-effect models.
result Superior accounting performance does not correlate with superior market performance.
The study evaluates AI model performance measures for medical use.
problem Selecting appropriate performance measures for AI models in medical practice.
method Assessed 32 performance measures across five domains for binary outcomes.
result 17 measures are both proper and reflect decision-analytic performance.
SHIFT framework identifies subgroups with large ML model performance decay.
problem Large model performance decay in subgroups when deployed.
method Subgroup-scanning Hierarchical Inference Framework (SHIFT) for performance drift.
result SHIFT identifies interpretable subgroups with large performance decay and suggests targeted actions to mitigate it.
New framework for predicting decisions that influence their own outcomes.
problem Predictions that affect the outcomes they predict, leading to undesirable distribution shift.
method Risk minimization framework combining statistics, game theory, and causality.
result Necessary and sufficient conditions for retraining to converge to a performatively stable point of minimal loss.
The paper explores how machine learning models can be learnable despite label shifts.
problem Learnability of binary classification models in the presence of label shifts.
method Developed a performative empirical risk function that is an unbiased estimate of the true risk on the shifted distribution.
result PAC-learnable hypothesis spaces remain PAC-learnable for performative scenarios.
New approach tackles decision-making under predictions that shape outcomes.
problem Challenges in learning optimal decision rules when predictions influence outcomes.
method Introduces performative omniprediction, a predictor that encodes optimal decision rules for multiple objectives.
result Efficient performative omnipredictors exist under a natural restriction of outcome performativity.