arXiv research

RANDomized-exploration policy Optimization via Multiple Importance Sampling with Truncation (RANDOMIST) for PO with mediator feedback.

Study online learning with set-valued feedback, showing differences between deterministic and randomized approaches.

ESM-CNN uses error feedback to build a random CNN for time series forecasting.

Study the tradeoffs of bandit feedback in multiclass classification.

Motivated by problems in search and detection we present a solution to a Combinatorial Multi-Armed Bandit (CMAB) problem with both heavy-tailed reward distributions and a new class of feedback, filtered semibandit feedback. In a CMAB problem an agent pulls a combination of arms from a set $\{1,...,k\}$ in each round, g…

The paper tackles combinatorial pure exploration with various feedback structures and proposes efficient algorithms.

Random Feedback Alignment helps solve low-rank matrix factorization problems.

We present and study models of adversarial online learning where the feedback observed by the learner is noisy, and the feedback is either full information feedback or bandit feedback. Specifically, we consider binary losses xored with the noise, which is a Bernoulli random variable. We consider both a constant noise r…

Paper studies CLO with partial feedback, improving decision-making in uncertain contexts.

New protocol for online learning with partial feedback, extending classical methods.

New algorithm learns human preferences from few comparisons efficiently.

We consider stochastic multi-armed bandit problems with graph feedback, where the decision maker is allowed to observe the neighboring actions of the chosen action. We allow the graph structure to vary with time and consider both deterministic and Erdős-Rényi random graph models. For such a graph feedback model, we fir…

New algorithm improves distributed SGD with random sparsification for better convergence and generalization.

A predictor that is deployed in a live production system may perturb the features it uses to make predictions. Such a feedback loop can occur, for example, when a model that predicts a certain type of behavior ends up causing the behavior it predicts, thus creating a self-fulfilling prophecy. In this paper we analyze p…

Artificial neural networks are most commonly trained with the back-propagation algorithm, where the gradient for learning is provided by back-propagating the error, layer by layer, from the output layer to the hidden layers. A recently discovered method called feedback-alignment shows that the weights used for propagat…

Meta-algorithm for efficient reinforcement learning from human preferences.

In a wide variety of applications, including personalization, we want to measure the difference in outcome due to an intervention and thus have to deal with counterfactual inference. The feedback from a customer in any of these situations is only 'bandit feedback' - that is, a partial feedback based on whether we chose…

Neural networks are commonly trained to make predictions through learning algorithms. Contrastive Hebbian learning, which is a powerful rule inspired by gradient backpropagation, is based on Hebb's rule and the contrastive divergence algorithm. It operates in two phases, the forward (or free) phase, where the data are …

QS-BO optimizes functions using only rank-based feedback.

Current algorithms for deep learning probably cannot run in the brain because they rely on weight transport, where forward-path neurons transmit their synaptic weights to a feedback path, in a way that is likely impossible biologically. An algorithm called feedback alignment achieves deep learning without weight transp…

We present a simple uniqueness argument for a collection of McKean-Vlasov problems that have seen recent interest. Our first result shows that, in the weak feedback regime, there is global uniqueness for a very general class of random drivers. By weak feedback we mean the case where the contagion parameters are small e…

We present online boosting algorithms for multiclass classification with bandit feedback, where the learner only receives feedback about the correctness of its prediction. We propose an unbiased estimate of the loss using a randomized prediction, allowing the model to update its weak learners with limited information. …

In this paper we explain the wild fluctuations of financial prices from the intrinsic amplifying feedback of speculative supply and demand. Formally, we show that an asset return follows a multiplicative random growth with exogenous input, which is well-known to be a generic power-law generating process, and which coul…

We consider a problem of stochastic online learning with general probabilistic graph feedback, where each directed edge in the feedback graph has probability $p_{ij}$. Two cases are covered. (a) The one-step case, where after playing arm $i$ the learner observes a sample reward feedback of arm $j$ with independent prob…

Capacity-Constrained Online Convex Optimization with Delayed Feedback

Study shows how to control jump-diffusion processes with stable feedback controls in reinforcement learning.

Sayer uses implicit feedback to optimize system policies.

A deterministic apple tasting learner is developed, confirming a conjecture and providing tight bounds for mistake bounds.

New algorithm tackles delayed feedback in Lipschitz bandits with sublinear regret.

New framework tackles submodular welfare with multi-agent combinatorial bandits.

This paper extends combinatorial semi-bandits to graph feedback, improving regret bounds.

New algorithm shows neural networks can learn without full backpropagation.

FP uses random projections to train networks without feedback, achieving comparable performance to backpropagation.

We present two alternative ways to apply PAC-Bayesian analysis to sequences of dependent random variables. The first is based on a new lemma that enables to bound expectations of convex functions of certain dependent random variables by expectations of the same functions of independent Bernoulli random variables. This …

Algorithm allocates budgets to tasks with semi-bandit feedback, achieving near-optimal regret bounds.

Feedback improves ESN performance by 30-60% across various tasks.

Study shows visual feedback and monetary incentives reduce plugload energy consumption in commercial buildings.

Bayesian Neural Networks (BNNs) have been proposed to address the problem of model uncertainty in training and inference. By introducing weights associated with conditioned probability distributions, BNNs are capable of resolving the overfitting issue commonly seen in conventional neural networks and allow for small-da…

New algorithm optimizes noisy function evaluations with delayed feedback.

Online learning with one-sided feedback aims to maximize accuracy while ensuring fairness.

Algorithm identifies best item from subsets with random utility model feedback.

Study cost-effective fairness audits with partial feedback, improving over random exploration.

There were many algorithms to substitute the back-propagation (BP) in the deep neural network (DNN) training. However, they could not become popular because their training accuracy and the computational efficiency were worse than BP. One of them was direct feedback alignment (DFA), but it showed low training performanc…

Recommender systems widely use implicit feedback such as click data because of its general availability. Although the presence of clicks signals the users' preference to some extent, the lack of such clicks does not necessarily indicate a negative response from the users, as it is possible that the users were not expos…

Algometrics analyzes how predictive models affect their own forecasts in algorithmic markets.

Online algorithms stabilize in feedback loops of performative prediction.

A vast majority of computation in the brain is performed by spiking neural networks. Despite the ubiquity of such spiking, we currently lack an understanding of how biological spiking neural circuits learn and compute in-vivo, as well as how we can instantiate such capabilities in artificial spiking circuits in-silico.…

Develops a framework for clustering and distribution matching with bandit feedback.