RANDomized-exploration policy Optimization via Multiple Importance Sampling with Truncation (RANDOMIST) for PO with mediator feedback.
problem Policy Optimization in continuous control tasks.
method RANDomized-exploration policy Optimization via Multiple Importance Sampling with Truncation (RANDOMIST) for regret minimization in PO.
result Achieving constant regret under certain circumstances in PO with mediator feedback.
Study online learning with set-valued feedback, showing differences between deterministic and randomized approaches.
problem Online learning with set-valued feedback, where labels are sets rather than single labels.
method Introduced new combinatorial dimensions (Set Littlestone and Measure Shattering) to characterize learnability.
result Characterized deterministic and randomized online learnability, and established bounds for various learning settings.
ESM-CNN uses error feedback to build a random CNN for time series forecasting.
problem Improving time series forecasting accuracy with CNNs.
method Incrementally adding random filters and neurons to adaptively construct a CNN.
result ESM-CNN outperforms state-of-the-art models in prediction accuracy and efficiency.
Study the tradeoffs of bandit feedback in multiclass classification.
problem The price of using bandit feedback in multiclass classification.
method Mistake bound model, analysis of variants, and comparison of learners and adversaries.
result The optimal mistake bound under bandit feedback is at most O ( k ) O(k) O ( k ) times higher than in full information, with a tight bound of O ( k ) O(k) O ( k ) . Study of online learning with noisy feedback in adversarial settings.
problem Adversarial online learning with noisy feedback.
method Models of adversarial online learning with binary losses xored by noise, considering both constant and variable noise rates.
result Tight regret bounds for learning with noise in adversarial online learning.
Motivated by problems in search and detection we present a solution to a Combinatorial Multi-Armed Bandit (CMAB) problem with both heavy-tailed reward distributions and a new class of feedback, filtered semibandit feedback. In a CMAB problem an agent pulls a combination of arms from a set { 1 , . . . , k } \{1,...,k\} { 1 , ... , k } in each round, g…
The paper tackles combinatorial pure exploration with various feedback structures and proposes efficient algorithms.
problem Identifying the optimal action in a combinatorial space with limited feedback and nonlinear rewards.
method Designs polynomial-time adaptive algorithms for CPE-BL and CPE-PL, providing sample complexity analyses.
result The proposed algorithms achieve sample complexity close to lower bounds and outperform existing methods.
Random Feedback Alignment helps solve low-rank matrix factorization problems.
problem Solving low-rank matrix factorization problems efficiently.
method Random Feedback Alignment (FA) as a bio-plausible alternative to gradient descent.
result FA converges to the optimal solution when the rank of the matrix is at least the rank of the input matrix.
Algorithm learns feature representations from randomized experiments to improve counterfactual inferences.
problem Measuring the impact of interventions with limited feedback.
method Feature learning algorithm from randomized experiments to identify effective and ineffective interventions.
result The algorithm leverages feature representations to derive the value of interventions for each instance, improving decision-making.
Paper studies CLO with partial feedback, improving decision-making in uncertain contexts.
problem Improving decision-making in contexts with uncertain cost coefficients using partial feedback.
method Unified class of offline learning algorithms for CLO with different types of feedback, using IERM framework.
result Fast-rate regret bound for IERM with partial feedback and misspecified model classes.
New protocol for online learning with partial feedback, extending classical methods.
problem Learning with partial feedback where only one acceptable label is observed per round.
method Introducing a collection version space to address the lack of direct extension of classical methods.
result Characterization of learnability in set-realizable regime using Partial-Feedback Littlestone dimension and Partial-Feedback Measure Shattering dimension.
New algorithm learns human preferences from few comparisons efficiently.
problem Learning human preferences from limited comparison feedback.
method Formulated as D-optimal design for Plackett-Luce model, solved using randomized Frank-Wolfe algorithm.
result Proposed algorithm efficiently solves D-optimal design problem for Plackett-Luce objective.
We consider stochastic multi-armed bandit problems with graph feedback, where the decision maker is allowed to observe the neighboring actions of the chosen action. We allow the graph structure to vary with time and consider both deterministic and Erdős-Rényi random graph models. For such a graph feedback model, we fir…
A new method optimizes recommender systems by addressing missing-not-at-random implicit feedback.
problem Missing-not-at-random implicit feedback in recommender systems.
method Defining an ideal loss function and proposing an unbiased estimator to optimize relevance.
result The proposed method improves performance, especially for rare items.
New algorithm improves distributed SGD with random sparsification for better convergence and generalization.
problem Communication bottleneck in distributed deep learning.
method Proposes detached error feedback (DEF) algorithm to improve convergence and generalization of communication-efficient distributed SGD.
result Shows better convergence and generalization bounds than existing methods.
A predictor that is deployed in a live production system may perturb the features it uses to make predictions. Such a feedback loop can occur, for example, when a model that predicts a certain type of behavior ends up causing the behavior it predicts, thus creating a self-fulfilling prophecy. In this paper we analyze p…
Artificial neural networks are most commonly trained with the back-propagation algorithm, where the gradient for learning is provided by back-propagating the error, layer by layer, from the output layer to the hidden layers. A recently discovered method called feedback-alignment shows that the weights used for propagat…
Online boosting for multiclass classification with limited feedback.
problem Online multiclass classification with bandit feedback.
method Proposed unbiased loss estimate and extended full information boosting algorithms to bandit setting.
result Asymptotic error bounds match full information counterparts, with larger sample complexity due to limited feedback.
Simple uniqueness proof for McKean-Vlasov systems with weak feedback.
problem Global and short-time uniqueness for McKean-Vlasov systems with small feedback.
method Simple probabilistic comparison argument robust to solution regularity.
result Global uniqueness for a broad class of McKean-Vlasov problems in the weak feedback regime.
Meta-algorithm for efficient reinforcement learning from human preferences.
problem Learning from human preference comparisons in Markov decision processes.
method Randomized exploration and experimental design for batch comparison queries.
result Meta-algorithm achieves both regret and last-iterate guarantees with minimal preference queries.
Neural networks are commonly trained to make predictions through learning algorithms. Contrastive Hebbian learning, which is a powerful rule inspired by gradient backpropagation, is based on Hebb's rule and the contrastive divergence algorithm. It operates in two phases, the forward (or free) phase, where the data are …
QS-BO optimizes functions using only rank-based feedback.
problem Optimizing expensive functions with unreliable or unavailable metric values.
method Quantile-scaling pipeline to convert ranks into Gaussian targets.
result QS-BO consistently achieves lower objective values and is statistically significant.
This work achieves finite-time stabilization of uncertain LQ systems using random feedbacks.
problem Stabilizing linear systems with unknown dynamics in finite time.
method Random linear feedbacks to achieve finite-time stabilization.
result High probability guarantees for finite time stabilization of LQ systems.
New algorithms avoid weight transport, outperforming current deep learning methods.
problem Current deep learning algorithms rely on weight transport, which is biologically implausible.
method Two mechanisms: weight mirror and modified Kolen-Pollack algorithm, using random feedback weights.
result These mechanisms outperform feedback alignment and other methods on visual recognition tasks.
The paper analyzes online learning with probabilistic graph feedback, matching regret bounds with high probability.
problem Online learning with probabilistic graph feedback, covering both one-step and cascade cases.
method Analyzed asymptotic lower bounds and designed algorithms for both cases, matching regret bounds with high probability.
result Regret upper bounds match asymptotic lower bounds with high probability.
In this paper we explain the wild fluctuations of financial prices from the intrinsic amplifying feedback of speculative supply and demand. Formally, we show that an asset return follows a multiplicative random growth with exogenous input, which is well-known to be a generic power-law generating process, and which coul…
Capacity-Constrained Online Convex Optimization with Delayed Feedback
problem Online learning with delayed feedback under a hard capacity constraint
method Reduction to a delayed and weighted OCO problem using a scheduler
result First regret guarantees for capacity-constrained OCO under convex and strongly convex losses
Random projections of labels enable feedforward training of deep neural networks without feedback.
problem Biological plausibility and low-cost adaptive smart sensors constrained by backpropagation.
method Fixed random projections of targets for feedforward training of hidden layers.
result DRTP algorithm provides a tradeoff between accuracy and computational cost suitable for edge computing.
Study shows how to control jump-diffusion processes with stable feedback controls in reinforcement learning.
problem Control jump-diffusion processes with unknown coefficients in reinforcement learning.
method Lipschitz continuous optimal feedback controls, stability analysis of forward-backward SDEs, least-squares algorithm.
result Achieves O ( N ln N ) O(\sqrt{N\ln N}) O ( N ln N ) regret for linear-convex learning problems with jumps. A deterministic apple tasting learner is developed, confirming a conjecture and providing tight bounds for mistake bounds.
problem Determining the learnability of hypothesis classes in binary online classification with apple tasting feedback.
method Developed a deterministic apple tasting learner and proved tight bounds for mistake bounds.
result Deterministic apple tasting is feasible and provides tight bounds for mistake bounds.
Sayer uses implicit feedback to optimize system policies.
problem Leveraging implicit feedback to improve system policies is difficult due to bias and incompleteness.
method Sayer combines randomized exploration and unbiased counterfactual estimators to evaluate and train new policies using implicit feedback.
result Sayer can accurately evaluate and train new policies that outperform existing ones.
New algorithm tackles delayed feedback in Lipschitz bandits with sublinear regret.
problem Delayed feedback in Lipschitz bandits.
method Design of algorithms for bounded and unbounded stochastic delays.
result Sublinear regret guarantees for both bounded and unbounded delays.
New framework tackles submodular welfare with multi-agent combinatorial bandits.
problem Maximizing total welfare among agents with shared constraints and submodular utilities under bandit feedback.
method Proposes an explore-then-commit strategy with randomized assignments for multi-agent combinatorial bandits.
result Achieves i l d e O ( T 2 / 3 ) ilde{\mathcal{O}}(T^{2/3}) i l d e O ( T 2/3 ) regret, first for partition-based submodular welfare problem under bandit feedback. New algorithm shows neural networks can learn without full backpropagation.
problem Stochastic gradient descent with backpropagation is non-biologically plausible.
method Random and fixed backpropagation weights in a feedback alignment algorithm.
result Error converges to zero exponentially fast in overparameterized networks.
This paper extends combinatorial semi-bandits to graph feedback, improving regret bounds.
problem Adversarial combinatorial semi-bandits with graph feedback.
method Introduced graph feedback in combinatorial semi-bandits, using convexified actions and online stochastic mirror descent.
result Optimal regret scales as S T + α S T S\sqrt{T}+\sqrt{αST} S T + α S T , interpolating between full and semi-bandit feedback. FP uses random projections to train networks without feedback, achieving comparable performance to backpropagation.
problem Training neural networks without feedback from downstream layers.
method Forward Projection (FP) method that uses randomised nonlinear projections and closed-form regression.
result FP achieves comparable generalisation to backpropagation methods with a single forward pass, offering significant speedup.
We present two alternative ways to apply PAC-Bayesian analysis to sequences of dependent random variables. The first is based on a new lemma that enables to bound expectations of convex functions of certain dependent random variables by expectations of the same functions of independent Bernoulli random variables. This …
Algorithm allocates budgets to tasks with semi-bandit feedback, achieving near-optimal regret bounds.
problem Stochastic budget allocation with censored semi-bandit feedback.
method Optimism-based algorithm operating under censored semi-bandit feedback.
result Regret scales polylogarithmically with horizon T in diminishing-returns regimes.
Feedback improves ESN performance by 30-60% across various tasks.
problem Reducing computational complexity in ESNs for complex sequential data processing.
method State feedback to modify the internal reservoir state of ESNs.
result Feedback significantly improves ESN performance, reducing error by 30-60%.
Study shows visual feedback and monetary incentives reduce plugload energy consumption in commercial buildings.
problem Mitigating energy consumption in commercial buildings through occupant plugload control.
method Field experiments with visual feedback and monetary incentives in government and university buildings.
result Mean energy reduction of ~9.52% in office environments and ~21.61% in university environments with visual feedback.
Bayesian Neural Networks (BNNs) have been proposed to address the problem of model uncertainty in training and inference. By introducing weights associated with conditioned probability distributions, BNNs are capable of resolving the overfitting issue commonly seen in conventional neural networks and allow for small-da…
Improved CNN training with BDFA reduces computational cost and improves accuracy.
problem Low training performance of DFA in CNN.
method Combining DFA with BP, introducing feedback weight initialization, and proposing BDFA.
result BDFA shows better performance than conventional BP, especially in small datasets.
New algorithm optimizes noisy function evaluations with delayed feedback.
problem Optimizing unknown functions from noisy, delayed feedback.
method Kernel bandit problem with stochastically delayed feedback.
result Proposes an algorithm with improved regret bound for non-smooth kernels.
Algorithm identifies best item from subsets with random utility model feedback.
problem PAC learning the best item from subsets with random utility model feedback.
method Pairwise relative counts and hierarchical elimination for learning algorithm.
result Near-optimal PAC sample complexity guarantee for identifying ε-optimal item.
Online learning with one-sided feedback aims to maximize accuracy while ensuring fairness.
problem Maximizing accuracy in online learning with limited feedback and ensuring fairness.
method Extending the framework of Bechavod et al. (2020) to incorporate dynamic panels of auditors, reducing the problem to a contextual combinatorial semi-bandit, and leveraging Exp2 and Context-Semi-Bandit-FTPL algorithms.
result Multi-criteria no regret guarantees for accuracy and fairness are provided.
Study cost-effective fairness audits with partial feedback, improving over random exploration.
problem Auditing fairness of classifiers with limited true labels.
method Introduces cost model, proposes near-optimal algorithms for black-box and mixture models.
result Significantly lower audit costs compared to natural baselines.
FCMs model feedback in complex systems, combining expert knowledge and statistical learning.
problem Modeling feedback in complex, interconnected systems.
method Fuzzy cognitive maps (FCMs) as fuzzy signed directed graphs, allowing for feedback loops and nonlinear dynamics.
result FCMs can better represent domain knowledge and simulate policy scenarios compared to DAGs.
Adaptive and robust algorithm for convex optimization under noisy feedback.
problem Minimizing unknown convex functions with noisy gradient feedback.
method Biased random walk on a binary tree with adaptive confidence bounds.
result Achieves optimal regret orders for various function classes.