Research
On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,695 papers · 148 categories

Trend · papers per month

21426283 · Jun 202019922001200920172026
48 results for near-optimal predictors

Understanding optimal prompts for binary sequence predictors is challenging.

problem Finding good prompts for binary sequence predictors is difficult.
method Viewing prompting as finding the best conditioning sequence on a near-optimal sequence predictor, using empirical and statistical analysis.
result Optimal prompts can be better understood given the pretraining distribution, which is not usually available.

The paper offers simple, near-optimal algorithms for multi-group learning.

problem Learning predictors within subgroups of a population, addressing fairness and hidden stratification.
method Studies the structure of solutions and provides simple, near-optimal algorithms.
result Simple and near-optimal algorithms for multi-group learning.

New bounds show simple predictors can learn complex concepts online.

problem When can simple predictors learn complex concepts in online learning?
method Characterized optimal mistake bounds for online learning with simple predictors.
result Achieved nearly optimal mistake bounds for online learning using sparse majority-vote of proper predictors.

A regularized risk minimization procedure for regression function estimation is introduced that achieves near optimal accuracy and confidence under general conditions, including heavy-tailed predictor and response variables. The procedure is based on median-of-means tournaments, introduced by the authors in [8]. It is …

2017-01-15abs ↗pdf ↗

In this report we review memory-based meta-learning as a tool for building sample-efficient strategies that learn from past experience to adapt to any task within a target class. Our goal is to equip the reader with the conceptual foundations of this tool for building new, scalable agents that operate on broad domains.…

2019-05-08abs ↗pdf ↗

New insights into when benign overfitting occurs in linear and classification tasks.

problem Understanding when benign overfitting happens in linear and classification models.
method Analysis of a generic data model and comparison of predictors (minimum-norm interpolating and max-margin).
result The minimum-norm interpolating predictor is biased towards an inconsistent solution, preventing benign overfitting in linear regression.

Learning linear predictors with the logistic loss---both in stochastic and online settings---is a fundamental task in machine learning and statistics, with direct connections to classification and boosting. Existing "fast rates" for this setting exhibit exponential dependence on the predictor norm, and Hazan et al. (20…

2018-03-25abs ↗pdf ↗

New method MRI improves machine learning models' ability to generalize to unseen data.

problem Machine learning models often fail to generalize well to out-of-distribution data.
method Introduces a new notion of invariance (MRI) and a practical version (MRI-v1) to improve model generalization.
result MRI-v1 guarantees invariant predictors and outperforms IRM-v1 in various settings.

Optimizes predictions for specific tasks using parametrized decision analysis.

problem Optimizing predictions for specific decision tasks of interest.
method Designs a class of parametrized actions for Bayesian decision analysis.
result Derives efficient and interpretable solutions for various action parametrizations and loss functions.

Optimizes predictions by recalibrating online forecasts with minimal error.

problem Tackles the challenge of recalibrating online predictions to be more accurate.
method Uses an imbalanced extension of the Blackwell approachability reduction framework to achieve (ε,ε2)(\varepsilon, \varepsilon^2)-recalibration.
result Achieves (ε,ε2)(\varepsilon, \varepsilon^2)-recalibration for Lipschitz proper losses in Tε3T \approx \varepsilon^{-3} rounds.

Bayesian method estimates dynamics from near-optimal trajectories.

problem Estimating dynamics from near-optimal expert trajectories in reinforcement learning.
method Constraint-based Bayesian approach integrating expert near-optimality.
result Significant improvements in decision-making and transfer success.

Paper tackles non-stationary kernelized bandits with near-optimal algorithm.

problem Minimizing regret in a time-varying reward function.
method Near-optimal algorithm with a novel restarting phased elimination with random permutation (R-PERP).
result Regret upper bound matches the lower bound, making the algorithm near-optimal.

System uses conformal prediction to help experts make accurate decisions without understanding when to trust it.

problem Helping experts make accurate decisions in multiclass classification tasks.
method Develops an automated decision support system using conformal prediction to provide precise prediction sets and an efficient search method.
result System improves expert predictions by providing precise prediction sets and forcing experts to predict from these sets.

Efficient streaming algorithms for robust statistics with near-optimal memory.

problem High-dimensional robust statistics tasks in streaming model.
method First efficient streaming algorithms with near-optimal memory requirements.
result Near-optimal error guarantees and space complexity nearly-linear in the dimension for robust mean estimation.

The paper proposes calibration to improve algorithm performance using machine learning predictions.

problem Improving real-world performance of online algorithms with machine learning predictions.
method Calibration as a tool to bridge the gap between prediction uncertainty and algorithm design.
result Calibrated advice leads to more effective guidance in high-variance settings and significant performance improvements in real-world data.

The problem of forecasting conditional probabilities of the next event given the past is considered in a general probabilistic setting. Given an arbitrary (large, uncountable) set C of predictors, we would like to construct a single predictor that performs asymptotically as well as the best predictor in C, on any data.…

2016-10-26abs ↗pdf ↗

New bounds explain deterministic non-smooth deep nets without large Lipschitz constants.

problem Challenges in explaining generalization of deterministic non-smooth deep nets.
method De-randomized PAC-Bayes margin bounds for deterministic non-convex and non-smooth predictors.
result New bounds avoid large Lipschitz constants, providing generalization guarantees.

This paper proposes a method to reduce complexity in GLMs with categorical predictors.

problem Wasteful, hard-to-interpret, and prone to overfitting of traditional one-hot encoding for high-cardinality categorical predictors.
method Clustering categories of categorical predictors through a numerical method that preserves or improves accuracy while reducing the number of coefficients.
result Clustering categories of categorical predictors reduces complexity substantially without harming accuracy.

Paper tackles clustering with ordinal comparisons, achieving near-optimal results.

problem Clustering with ordinal comparisons when similarity measures are not available.
method Two-step procedure: estimate similarity matrix from comparisons, then apply SDP clustering.
result Near-optimal recovery of planted clustering using near-optimal number of comparisons.

The article compares predictor importance in classification problems with categorical outcomes.

problem Comparing predictor importance in classification problems with categorical response variables.
method The approach is based on the categorical Gini correlation (CGC) and tests differences in CGCs across predictor groups.
result The proposed methodology accommodates predictors of arbitrary and unequal dimensions and allows for dependence between predictor groups.

Paper proposes a sparse synthetic control method to select important predictors.

problem Choosing and weighting predictors affects synthetic control estimator performance.
method Sparse synthetic control procedure that penalizes predictors, derived in a linear factor model.
result Sparse synthetic control achieves lower bias and better post-treatment performance.

New algorithm detects changes quickly without knowing parameters, near optimally.

problem Quickest change detection with unknown parameters.
method Leverages theoretical asymptotic properties to derive a scalable approximate algorithm with near optimal performance.
result Detects changes in constant complexity with near optimal performance.

Efficiently find near-optimal medical treatments with less trial and error.

problem Finding effective medical treatments through trial and error.
method Formalizes the problem, uses a causal inference framework, and proposes model-based dynamic programming and greedy algorithms.
result Our methods compare favorably to model-free reinforcement learning, offering a more transparent trade-off between search time and treatment efficacy.

New DP algorithms achieve near-optimal regret bounds for online learning problems.

problem Online learning problems with zero-loss solutions and differential privacy constraints.
method Developed new Differentially Private algorithms with near-optimal regret bounds.
result Achieved near-optimal regret bounds for various online prediction and convex optimization problems.

WeakNAS uses a set of weaker predictors to find top architectures with fewer samples.

problem Finding the best neural architecture with heavy computation costs.
method Proposes a paradigm shift from fitting the whole architecture space to progressively fitting a search path through a set of weaker predictors.
result WeakNAS produces coarse-to-fine iteration to gradually refine the ranking of sampling space, requiring fewer samples to find top-performance architectures.

Proposes a method to create fair, robust predictors that remain consistent across different scenarios.

problem Creating fair and robust machine learning models that behave consistently across different scenarios.
method Graphical criteria and a model-agnostic framework called CIP based on HSCIC.
result Demonstrates the effectiveness of CIP in enforcing counterfactual invariance across various datasets.

Algorithm extsc{Pedel} learns near-optimal policies efficiently on specific problems.

problem Learning near-optimal policies in linear MDPs with minimal samples.
method Online experiment design to focus exploration on relevant directions.
result Achieves instance-dependent complexity, outperforming minimax-optimal algorithms.

Improved algorithm for adaptive dueling bandits with near-optimal regret bound.

problem Non-stationary dueling bandits with unknown number of preference changes.
method Elimination-based rescheduling algorithm for adaptive dynamic regret.
result Near-optimal ildeO(SextttCWT) ilde{O}(\sqrt{S^{ exttt{CW}} T}) dynamic regret bound.

This paper continues study, both theoretical and empirical, of the method of Venn prediction, concentrating on binary prediction problems. Venn predictors produce probability-type predictions for the labels of test objects which are guaranteed to be well calibrated under the standard assumption that the observations ar…

2012-10-31abs ↗pdf ↗

Derives bounds for deterministic predictors using smooth loss functions.

problem Generalizing probabilistic predictors to deterministic ones.
method Exploits smoothness properties of loss and predictor classes, controlling the Jensen gap class through Rademacher complexity.
result Derives bounds for deterministic predictors involving flatness quantities from Jacobians and Hessians.

Study shows competition feedback can make ML predictors biased towards specific user groups.

problem How competition affects machine learning predictors and user prediction quality.
method Flexible model of competing ML predictors, empirical and mathematical analysis.
result Competition causes predictors to specialize for specific sub-populations at the cost of general performance.

A new UCB algorithm for heavy-tailed bandits with near-optimal regret.

problem Sequential decision making in uncertain environments with heavy-tailed rewards.
method Data-driven, distribution-free UCB algorithm combining resampled median-of-means and UCB.
result Near-optimal regret bound for heavy-tailed distributions.

Paper introduces SUEL model for integrating predictors without labeled data.

problem Combining predictors with unknown accuracy and high correlation.
method Structured unsupervised ensemble learning (SUEL) with correlation-based decomposition algorithms.
result Efficient integration of dependent predictors without labeled data.

Adaptive kernels from neural networks improve model performance.

problem Improving neural network performance through adaptive kernels.
method Deriving adaptive kernels from infinite-width neural networks using feature learning and gradient flow training.
result Adaptive kernels achieve lower test loss compared to traditional kernels.

Ensembles of random-feature models can't outperform a single large model.

problem Finding the optimal balance between model size and ensemble size.
method Deterministic equivalent risk estimates and scaling laws analysis.
result Ensembles of random-feature models achieve near-optimal performance only under specific conditions.