arXiv research

Optimal algorithm for contextual bandits with unknown context distributions.

Develops a robust learning method for unknown context distributions.

A federated learning algorithm tackles unknown contexts in multi-arm bandits.

Paper solves stochastic contextual linear bandits using linear bandit algorithms.

Optimal pricing strategy for unknown valuation models with noisy feedback.

Enhances activity recognition in wearable computing with context awareness and uncertainty quantification.

New method reduces regret in nonparametric bandits with unknown covariate shifts.

Generalization and adaptation of learned skills to novel situations is a core requirement for intelligent autonomous robots. Although contextual reinforcement learning provides a principled framework for learning and generalization of behaviors across related tasks, it generally relies on uninformed sampling of environ…

New algorithm reduces online learning error for unknown feature distributions.

Algorithm identifies best arm in piecewise stationary linear bandits with minimal samples.

A new algorithm optimizes unknown functions with noisy data and unmatched features.

Paper establishes MLE consistency for market microstructure models.

We study contextual bandits with budget and time constraints, referred to as constrained contextual bandits.The time and budget constraints significantly complicate the exploration and exploitation tradeoff because they introduce complex coupling among contexts over time.Such coupling effects make it difficult to obtai…

Thompson Sampling tackles noisy context in stochastic bandits.

In risk management, tail risks are of crucial importance. The assessment of risks should be carried out in accordance with the regulatory authority's requirement at high quantiles. In general, the underlying distribution function is unknown, the database is sparse, and therefore special tail models are used. Very often…

Minimax PAC bounds for learning in exogenous contextual MDPs

EDRBO optimizes Bayesian optimization with continuous contexts using ensemble models and robust methods.

Unified approach to fair online learning with stochastic contexts.

Bayesian models use hyperparameters to indirectly assign priors, and this work shows how these priors can be derived from maximum entropy principles.

We provide new results for noise-tolerant and sample-efficient learning algorithms under $s$-concave distributions. The new class of $s$-concave distributions is a broad and natural generalization of log-concavity, and includes many important additional distributions, e.g., the Pareto distribution and $t$-distribution.…

Consider a nonparametric contextual multi-arm bandit problem where each arm $a \in [K]$ is associated to a nonparametric reward function $f_a: [0,1] \to \mathbb{R}$ mapping from contexts to the expected reward. Suppose that there is a large set of arms, yet there is a simple but unknown structure amongst the arm reward…

For a dataset of label-count pairs, an anonymized histogram is the multiset of counts. Anonymized histograms appear in various potentially sensitive contexts such as password-frequency lists, degree distribution in social networks, and estimation of symmetric properties of discrete distributions. Motivated by these app…

We explore the sequential decision making problem where the goal is to estimate uniformly well a number of linear models, given a shared budget of random contexts independently sampled from a known distribution. The decision maker must query one of the linear models for each incoming context, and receives an observatio…

Extends JKO scheme for iterative algorithms with unknown parameters.

The paper tackles estimating optimal policy value in linear bandits with general context distributions.

Transformer learns context and regularization for ICL in inverse problems.

A novel bandit problem with context-dependent rewards and blocking.

Statistical test verifies long-term rating system calibration with overlapping time windows.

Generative AI can solve in-context learning problems using a martingale perspective.

We extend Bayesian multi-armed bandit (MAB) algorithms beyond their original setting by making use of sequential Monte Carlo (SMC) methods. A MAB is a sequential decision making problem where the goal is to learn a policy that maximizes long term payoff, where only the reward of the executed action is observed. In the …

Bayesian nonparametric CMS improves frequency estimation for power-law data.

New Thompson Sampling for partially observed context bandits reduces regret logarithmically with time.

The problem of change-point estimation is considered under a general framework where the data are generated by unknown stationary ergodic process distributions. In this context, the consistent estimation of the number of change-points is provably impossible. However, it is shown that a consistent clustering method may …

While normalizing flows have led to significant advances in modeling high-dimensional continuous distributions, their applicability to discrete distributions remains unknown. In this paper, we show that flows can in fact be extended to discrete events---and under a simple change-of-variables formula not requiring log-d…

A new pricing strategy learns customer valuations without noise distribution knowledge.

The paper analyzes meta-learning in a Gaussian setting, providing bounds and matching algorithms.

This paper proposes an online tree-based Bayesian approach for reinforcement learning. For inference, we employ a generalised context tree model. This defines a distribution on multivariate Gaussian piecewise-linear models, which can be updated in closed form. The tree structure itself is constructed using the cover tr…

Faster convergence of kernel mean embeddings using variance information.

We propose and study a multi-scale approach to vector quantization. We develop an algorithm, dubbed reconstruction trees, inspired by decision trees. Here the objective is parsimonious reconstruction of unsupervised data, rather than classification. Contrasted to more standard vector quantization methods, such as K-mea…

In real-world scenarios, different features have different acquisition costs at test-time which necessitates cost-aware methods to optimize the cost and performance trade-off. This paper introduces a novel and scalable approach for cost-aware feature acquisition at test-time. The method incrementally asks for features …

Over the past decades, researchers and ML practitioners have come up with better and better ways to build, understand and improve the quality of ML models, but mostly under the key assumption that the training data is distributed identically to the testing data. In many real-world applications, however, some potential …

Paper proposes using generalized lambda distributions for stochastic simulators.

A new algorithm tackles graph-based contextual bandits with efficient regret bounds.

The study compares Bayesian and frequentist approaches in deep learning.

NP-PROV separates mean and variance spaces to improve function uncertainty.

Paper improves Thompson Sampling for linear contextual bandits.

Nearest neighbor (k-NN) graphs are widely used in machine learning and data mining applications, and our aim is to better understand what they reveal about the cluster structure of the unknown underlying distribution of points. Moreover, is it possible to identify spurious structures that might arise due to sampling va…

ICON-OCnet solves optimal execution problems with neural networks and few examples.