Understanding optimal prompts for binary sequence predictors is challenging.
problem Finding good prompts for binary sequence predictors is difficult.
method Viewing prompting as finding the best conditioning sequence on a near-optimal sequence predictor, using empirical and statistical analysis.
result Optimal prompts can be better understood given the pretraining distribution, which is not usually available.
The paper offers simple, near-optimal algorithms for multi-group learning.
problem Learning predictors within subgroups of a population, addressing fairness and hidden stratification.
method Studies the structure of solutions and provides simple, near-optimal algorithms.
result Simple and near-optimal algorithms for multi-group learning.
New bounds show simple predictors can learn complex concepts online.
problem When can simple predictors learn complex concepts in online learning?
method Characterized optimal mistake bounds for online learning with simple predictors.
result Achieved nearly optimal mistake bounds for online learning using sparse majority-vote of proper predictors.
A regularized risk minimization procedure for regression function estimation is introduced that achieves near optimal accuracy and confidence under general conditions, including heavy-tailed predictor and response variables. The procedure is based on median-of-means tournaments, introduced by the authors in [8]. It is …
In this report we review memory-based meta-learning as a tool for building sample-efficient strategies that learn from past experience to adapt to any task within a target class. Our goal is to equip the reader with the conceptual foundations of this tool for building new, scalable agents that operate on broad domains.…
New insights into when benign overfitting occurs in linear and classification tasks.
problem Understanding when benign overfitting happens in linear and classification models.
method Analysis of a generic data model and comparison of predictors (minimum-norm interpolating and max-margin).
result The minimum-norm interpolating predictor is biased towards an inconsistent solution, preventing benign overfitting in linear regression.
Learning linear predictors with the logistic loss---both in stochastic and online settings---is a fundamental task in machine learning and statistics, with direct connections to classification and boosting. Existing "fast rates" for this setting exhibit exponential dependence on the predictor norm, and Hazan et al. (20…
New method MRI improves machine learning models' ability to generalize to unseen data.
problem Machine learning models often fail to generalize well to out-of-distribution data.
method Introduces a new notion of invariance (MRI) and a practical version (MRI-v1) to improve model generalization.
result MRI-v1 guarantees invariant predictors and outperforms IRM-v1 in various settings.
Neural networks can memorize and learn complex functions with minimal parameters.
problem Learning and memorizing complex functions with neural networks using minimal parameters.
method SGD on depth two neural networks with near optimal network size and parameters.
result SGD on depth two network can memorize and learn polynomials with bounded weights and certain kernel spaces.
Optimizes predictions for specific tasks using parametrized decision analysis.
problem Optimizing predictions for specific decision tasks of interest.
method Designs a class of parametrized actions for Bayesian decision analysis.
result Derives efficient and interpretable solutions for various action parametrizations and loss functions.
Optimizes predictions by recalibrating online forecasts with minimal error.
problem Tackles the challenge of recalibrating online predictions to be more accurate.
method Uses an imbalanced extension of the Blackwell approachability reduction framework to achieve ( ε , ε 2 ) (\varepsilon, \varepsilon^2) ( ε , ε 2 ) -recalibration. result Achieves ( ε , ε 2 ) (\varepsilon, \varepsilon^2) ( ε , ε 2 ) -recalibration for Lipschitz proper losses in T ≈ ε − 3 T \approx \varepsilon^{-3} T ≈ ε − 3 rounds. Bayesian method estimates dynamics from near-optimal trajectories.
problem Estimating dynamics from near-optimal expert trajectories in reinforcement learning.
method Constraint-based Bayesian approach integrating expert near-optimality.
result Significant improvements in decision-making and transfer success.
Paper tackles non-stationary kernelized bandits with near-optimal algorithm.
problem Minimizing regret in a time-varying reward function.
method Near-optimal algorithm with a novel restarting phased elimination with random permutation (R-PERP).
result Regret upper bound matches the lower bound, making the algorithm near-optimal.
System uses conformal prediction to help experts make accurate decisions without understanding when to trust it.
problem Helping experts make accurate decisions in multiclass classification tasks.
method Develops an automated decision support system using conformal prediction to provide precise prediction sets and an efficient search method.
result System improves expert predictions by providing precise prediction sets and forcing experts to predict from these sets.
We study the worst-case adaptive optimization problem with budget constraint that is useful for modeling various practical applications in artificial intelligence and machine learning. We investigate the near-optimality of greedy algorithms for this problem with both modular and non-modular cost functions. In both case…
Efficient streaming algorithms for robust statistics with near-optimal memory.
problem High-dimensional robust statistics tasks in streaming model.
method First efficient streaming algorithms with near-optimal memory requirements.
result Near-optimal error guarantees and space complexity nearly-linear in the dimension for robust mean estimation.
The paper proposes calibration to improve algorithm performance using machine learning predictions.
problem Improving real-world performance of online algorithms with machine learning predictions.
method Calibration as a tool to bridge the gap between prediction uncertainty and algorithm design.
result Calibrated advice leads to more effective guidance in high-variance settings and significant performance improvements in real-world data.
The problem of forecasting conditional probabilities of the next event given the past is considered in a general probabilistic setting. Given an arbitrary (large, uncountable) set C of predictors, we would like to construct a single predictor that performs asymptotically as well as the best predictor in C, on any data.…
New bounds explain deterministic non-smooth deep nets without large Lipschitz constants.
problem Challenges in explaining generalization of deterministic non-smooth deep nets.
method De-randomized PAC-Bayes margin bounds for deterministic non-convex and non-smooth predictors.
result New bounds avoid large Lipschitz constants, providing generalization guarantees.
We study the problem of regret minimization for distributed bandits learning, in which M M M agents work collaboratively to minimize their total regret under the coordination of a central server. Our goal is to design communication protocols with near-optimal regret and little communication cost, which is measured by the…
This paper proposes a method to reduce complexity in GLMs with categorical predictors.
problem Wasteful, hard-to-interpret, and prone to overfitting of traditional one-hot encoding for high-cardinality categorical predictors.
method Clustering categories of categorical predictors through a numerical method that preserves or improves accuracy while reducing the number of coefficients.
result Clustering categories of categorical predictors reduces complexity substantially without harming accuracy.
Paper tackles clustering with ordinal comparisons, achieving near-optimal results.
problem Clustering with ordinal comparisons when similarity measures are not available.
method Two-step procedure: estimate similarity matrix from comparisons, then apply SDP clustering.
result Near-optimal recovery of planted clustering using near-optimal number of comparisons.
The article compares predictor importance in classification problems with categorical outcomes.
problem Comparing predictor importance in classification problems with categorical response variables.
method The approach is based on the categorical Gini correlation (CGC) and tests differences in CGCs across predictor groups.
result The proposed methodology accommodates predictors of arbitrary and unequal dimensions and allows for dependence between predictor groups.
Paper proposes a sparse synthetic control method to select important predictors.
problem Choosing and weighting predictors affects synthetic control estimator performance.
method Sparse synthetic control procedure that penalizes predictors, derived in a linear factor model.
result Sparse synthetic control achieves lower bias and better post-treatment performance.
New algorithm detects changes quickly without knowing parameters, near optimally.
problem Quickest change detection with unknown parameters.
method Leverages theoretical asymptotic properties to derive a scalable approximate algorithm with near optimal performance.
result Detects changes in constant complexity with near optimal performance.
Efficiently find near-optimal medical treatments with less trial and error.
problem Finding effective medical treatments through trial and error.
method Formalizes the problem, uses a causal inference framework, and proposes model-based dynamic programming and greedy algorithms.
result Our methods compare favorably to model-free reinforcement learning, offering a more transparent trade-off between search time and treatment efficacy.
New DP algorithms achieve near-optimal regret bounds for online learning problems.
problem Online learning problems with zero-loss solutions and differential privacy constraints.
method Developed new Differentially Private algorithms with near-optimal regret bounds.
result Achieved near-optimal regret bounds for various online prediction and convex optimization problems.
This paper presents Sparse Partitioning, a Bayesian method for identifying predictors that either individually or in combination with others affect a response variable. The method is designed for regression problems involving binary or tertiary predictors and allows the number of predictors to exceed the size of the sa…
WeakNAS uses a set of weaker predictors to find top architectures with fewer samples.
problem Finding the best neural architecture with heavy computation costs.
method Proposes a paradigm shift from fitting the whole architecture space to progressively fitting a search path through a set of weaker predictors.
result WeakNAS produces coarse-to-fine iteration to gradually refine the ranking of sampling space, requiring fewer samples to find top-performance architectures.
Proposes a method to create fair, robust predictors that remain consistent across different scenarios.
problem Creating fair and robust machine learning models that behave consistently across different scenarios.
method Graphical criteria and a model-agnostic framework called CIP based on HSCIC.
result Demonstrates the effectiveness of CIP in enforcing counterfactual invariance across various datasets.
Algorithm extsc{Pedel} learns near-optimal policies efficiently on specific problems.
problem Learning near-optimal policies in linear MDPs with minimal samples.
method Online experiment design to focus exploration on relevant directions.
result Achieves instance-dependent complexity, outperforming minimax-optimal algorithms.
Improved algorithm for adaptive dueling bandits with near-optimal regret bound.
problem Non-stationary dueling bandits with unknown number of preference changes.
method Elimination-based rescheduling algorithm for adaptive dynamic regret.
result Near-optimal i l d e O ( S e x t t t C W T ) ilde{O}(\sqrt{S^{ exttt{CW}} T}) i l d e O ( S e x ttt C W T ) dynamic regret bound. This paper continues study, both theoretical and empirical, of the method of Venn prediction, concentrating on binary prediction problems. Venn predictors produce probability-type predictions for the labels of test objects which are guaranteed to be well calibrated under the standard assumption that the observations ar…
Derives bounds for deterministic predictors using smooth loss functions.
problem Generalizing probabilistic predictors to deterministic ones.
method Exploits smoothness properties of loss and predictor classes, controlling the Jensen gap class through Rademacher complexity.
result Derives bounds for deterministic predictors involving flatness quantities from Jacobians and Hessians.
Study shows competition feedback can make ML predictors biased towards specific user groups.
problem How competition affects machine learning predictors and user prediction quality.
method Flexible model of competing ML predictors, empirical and mathematical analysis.
result Competition causes predictors to specialize for specific sub-populations at the cost of general performance.
Study on merging predictors in causal and anticausal directions using CMAXENT.
problem Comparing merging predictors in causal and anticausal directions.
method Using CMAXENT as inductive bias, study differences in merging predictors.
result CMAXENT solution reduces to logistic regression in causal direction and LDA in anticausal direction.
A single policy suffices for near-optimal parallel exploration in RL.
problem Quantitative effects of parallel exploration in reward-free RL.
method Using a single policy to guide exploration across all agents.
result Near-linear speedup and near-minimax optimality for linear MDPs.
Short proof for ideal polygons with near optimal orthogeodesic decomposition.
problem Decomposing ideal polygons into orthogeodesics.
method Short proof with orthogeodesic decomposition of length at most 2 log ( n ) 2 \log(n) 2 log ( n ) . result Optimal orthogeodesic decomposition of ideal polygons with length 2 log ( n ) 2 \log(n) 2 log ( n ) . A new UCB algorithm for heavy-tailed bandits with near-optimal regret.
problem Sequential decision making in uncertain environments with heavy-tailed rewards.
method Data-driven, distribution-free UCB algorithm combining resampled median-of-means and UCB.
result Near-optimal regret bound for heavy-tailed distributions.
New algorithm reduces bias in trained models, near-optimal performance proven.
problem Reduction of bias in trained machine learning models.
method Scalable post-processing algorithm for debiasing trained models, including deep neural networks (DNNs).
result Proven to be near-optimal by bounding its excess Bayes risk.
A new method finds diverse near-optimal portfolios using quality-diversity.
problem Optimizing financial portfolios with robustness to input parameter uncertainties.
method Quality-Diversity (QD) optimization using CVT-MAP-Elites algorithm.
result Diverse set of near-optimal portfolios identified.
Paper introduces SUEL model for integrating predictors without labeled data.
problem Combining predictors with unknown accuracy and high correlation.
method Structured unsupervised ensemble learning (SUEL) with correlation-based decomposition algorithms.
result Efficient integration of dependent predictors without labeled data.
Near-optimal algorithms for predicting across multiple loss functions efficiently.
problem Predicting optimally across various loss functions simultaneously.
method Developed near-optimal online and offline learning algorithms for omniprediction.
result Achieved near-optimal complexity for both online and offline settings.
The paper develops predictors for functional data on manifolds.
problem Functional data prediction on time-varying manifolds.
method Least-squares local linear Fréchet curve predictor and weighted Fréchet mean approach.
result Asymptotical optimality of the proposed predictors.
New algorithms minimize risk in MNL bandits, achieving near-optimal performance.
problem Minimizing risk in multi-armed bandit problems.
method Designing algorithms for various risk criteria (e.g., CVaR, Sharpe ratio, entropy risk).
result Near-optimal regret for the designed algorithms.
Adaptive kernels from neural networks improve model performance.
problem Improving neural network performance through adaptive kernels.
method Deriving adaptive kernels from infinite-width neural networks using feature learning and gradient flow training.
result Adaptive kernels achieve lower test loss compared to traditional kernels.
Ensembles of random-feature models can't outperform a single large model.
problem Finding the optimal balance between model size and ensemble size.
method Deterministic equivalent risk estimates and scaling laws analysis.
result Ensembles of random-feature models achieve near-optimal performance only under specific conditions.
Scaffolding sets improve predictor correctness across subsets.
problem Ensuring predictor correctness across multiple subsets.
method Inspired by neural nets, constructing scaffolding sets to ensure correctness.
result Scaffolding sets ensure predictor correctness, not just calibration.