Paper improves SLCB regret bound for bounded noise.
problem Stochastic linear contextual bandits with bounded noise.
method Set-membership estimation (SME) and optimism in the face of uncertainty (OFU).
result Improved regret bound of .
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,051 papers · 148 categories
Trend · papers per month
48 results for “reward noising”
A new exploration method for RL using parameter space noise.
problem Improving exploration in deep reinforcement learning.
method Switching isotropic and directional exploration in parameter space with parameter space noise.
result The proposed method achieves competitive results and better performance in sparse reward environments.
New method adds noise to rewards to improve reinforcement learning performance.
problem Brittleness in reinforcement learning due to variance differences between states and actions.
method Adaptive Symmetric Reward Noising (ASRN) by adding Gaussian noise to rewards based on their variance.
result ASRN improves reinforcement learning performance in various tasks, including autonomous driving.
Adjoint Matching improves flow and diffusion models with reward fine-tuning.
problem Improving generative models with reward fine-tuning.
method Casting reward fine-tuning as stochastic optimal control (SOC) and enforcing a specific noise schedule.
result Adjoint Matching outperforms existing SOC algorithms.
Asymptotic and Finite Sample Analysis of Nonexpansive Stochastic Approximations with Markovian Noisecs.LG
This work analyzes nonexpansive stochastic approximations with Markovian noise, proving convergence in reinforcement learning.
problem Applying stochastic approximation to reinforcement learning settings with nonexpansive operators.
method Investigates nonexpansive stochastic approximations with Markovian noise, providing asymptotic and finite sample analysis.
result First-time proof of convergence for classical tabular average reward temporal difference learning.
A novel framework refines diffusion models iteratively for better downstream reward optimization.
problem Optimizing reward functions during inference of diffusion models.
method Iterative refinement process with noising and reward-guided denoising steps.
result Superior empirical performance in protein and DNA design.
Paper tackles noisy reinforcement learning with perturbed rewards, improving agent performance.
problem Noisy rewards in reinforcement learning scenarios, leading to unreliable model performance.
method Develops a robust RL framework using a confusion matrix to estimate unbiased surrogate rewards.
result Trained policies using estimated surrogate rewards achieve higher expected rewards and faster convergence.
CausalRM models rewards from user feedback, overcoming noise and bias.
problem Aligning language models with user preferences from noisy, biased feedback.
method Causal-theoretic reward modeling framework addressing noise and bias in observational feedback.
result CausalRM learns accurate reward signals from noisy and biased observational feedback.
This study proposes hidden state curiosity to enhance RL models' resilience against noise.
problem Curiosity traps in RL models distract agents from discovering novel experiences.
method Proposed hidden state curiosity based on the Free Energy Principle to reward agents for KL divergence between predictive priors and posteriors.
result Agents with hidden state curiosity are more resilient against curiosity traps compared to those with prediction error curiosity.
A bandit algorithm reduces regret in noisy, communication-constrained feedback.
problem Distributed stochastic multi-armed bandit with noisy, communication-constrained feedback.
method Proposes a multi-phase bandit algorithm, UE-UCB++, that matches an information-theoretic lower bound.
result Matches an information-theoretic lower bound of Ω(√(KT/σ²)) on the minimax regret.
The paper proposes a model reward scheme for collaborative ML based on Shapley value and information gain.
problem Designing fair incentives for collaborative machine learning.
method The paper proposes a reward scheme based on Shapley value and information gain, with properties like fairness and stability.
result The proposed reward scheme satisfies fairness and trade-offs between desirable properties via an adjustable parameter.
Improved GP bandit algorithms for noiseless, varying noise, and RKHS norms.
problem Minimizing regret in Gaussian process bandits with unknown reward functions.
method New upper bound on maximum posterior variance, refined MVR and PE algorithms.
result Optimal regret bounds for noiseless, varying noise, and RKHS norms.
Reward hacking exploits misspecified rewards, affecting agent capabilities and true performance.
problem Reward hacking in RL models exploiting reward misspecifications.
method Constructed four RL environments with misspecified rewards; analyzed agent capabilities and behavior.
result More capable agents exploit reward misspecifications, achieving higher proxy reward but lower true reward.
Predictive Banditscs.LG
New predictive bandit model with noise for better decision making.
problem Optimizing decisions with limited information and noise.
method Introduced predictive bandits, derived regret bounds, and developed algorithms.
result Developed algorithms matching theoretical regret bounds.
New UCB-type algorithms reduce regret bounds for stochastic bandits with heavy and super heavy noise.
problem Improving regret bounds for stochastic bandits with heavy-tailed noise.
method General convex optimization methods with an inexact oracle, Clipped-SGD-UCB algorithm.
result Achieved an regret bound for symmetric noise, better than general lower bounds.
Proposes a method to boost deep reinforcement learning with sparse rewards.
problem Challenges in learning complex behaviors with long horizons and sparse rewards.
method Predictive coding for reward shaping.
result Achieves better learning by providing reward signals that understand environment dynamics and emphasize useful features.
CORe uses randomization to explore bandit problems without external noise.
problem Exploration in stochastic bandit problems.
method Randomizes past observations to exploit variance in rewards.
result Achieves regret bound in stochastic linear bandits.
We protect value function approximators in continuous state spaces from differential privacy attacks.
problem Protecting reward information from inverse reinforcement learning attacks in continuous state spaces.
method Adding functional noise to the value function iteratively to ensure differential privacy.
result Rigorous privacy guarantees and approximate optimality of the algorithm in discrete state spaces.
Combines human demonstrations and preferences to learn rewards for reinforcement learning.
problem Learning reward functions for complex real-world reinforcement learning problems.
method Trains a deep neural network to model reward functions from human demonstrations and preferences, then uses the model to train a DQN agent.
result Achieves superhuman performance on 2 Atari games without using game rewards.
Reinforcement Learning (RL) agents require the specification of a reward signal for learning behaviours. However, introduction of corrupt or stochastic rewards can yield high variance in learning. Such corruption may be a direct result of goal misspecification, randomness in the reward signal, or correlation of the rew…
A new reward shaping method balances learning efficiency and effectiveness for robot manipulation.
problem Efficient and effective learning in robot manipulations with system uncertainty.
method Dense2Sparse reward shaping method combining dense and sparse rewards.
result Dense2Sparse method achieves higher expected reward and better system uncertainty tolerance.
The paper tackles resource allocation for arms with unknown and random rewards, achieving optimal regret bounds.
problem Allocating resources on arms with unknown and random rewards.
method Developed two algorithms with optimal regret bounds for , demonstrating a phase transition at .
result Achieved optimal gap-dependent and gap-independent regret bounds for .
Efficient algorithm for online learning with Massart noise achieves near-optimal mistake bound.
problem Online learning with adversarial context and Massart noise.
method Developed an efficient algorithm for -margin linear classifiers in the presence of Massart noise.
result Achieved a mistake bound of for the online learning model.
This paper solves inverse reinforcement learning with options framework.
problem Reinforcement learning in complex environments with expert demonstrations.
method Gradient method for parametrized options, deducing Q-feature and reward feature spaces, optimal reward function selection.
result Recovered rewards provide effective solution to IRL problem and accelerate transfer learning.
Paper introduces a novel reward function for noisy financial markets using imitation learning.
problem Noisy reward function in financial markets hinders RL agent performance.
method Integrates imitation learning feedback with reinforcement learning to improve reward function design.
result Improves financial performance metrics compared to traditional benchmarks and RL agents.
EBRM improves robustness and generalization of language model rewards.
problem Challenges in capturing complex human preferences and generalizing to unseen data in reward models.
method Energy-Based Reward Model (EBRM) that models reward distribution explicitly, using conflict-aware data filtering, label-noise-aware contrastive training, and hybrid initialization.
result Significant improvements in robustness and generalization, up to 5.97% improvement in safety-critical alignment tasks.
Improves generative models by optimizing rewards and sample editing.
problem Efficiently generating high-reward samples with structural constraints.
method Introduces MDM-VGB, a discrete diffusion sampler that augments unmasking generation with reward-guided remasking.
result MDM-VGB achieves quadratic complexity and robustness to noise, outperforming heuristics like best-of-.
Thompson Sampling tackles noisy context in stochastic bandits.
problem Designing an action policy for noisy, corrupted contexts in stochastic bandits.
method Introducing a Thompson Sampling algorithm for Gaussian bandits with Gaussian context noise, adopting an information-theoretic analysis.
result Demonstrates the Bayesian regret of the proposed algorithm concerning the oracle's action policy.
This paper tackles open problem of tight bounds for KBs with Bernoulli rewards.
problem Open problem of tight bounds for Kernelized Bandits with Bernoulli rewards.
method Focus on Bernoulli model, not subgaussian noise, and optimize function in RKHS.
result Open problem remains unsolved in this context.
New method tackles confounded bandit problems with dual instrumental variables.
problem Confounded contextual bandit problems where noise affects both contexts and rewards.
method Dual instrumental variable regression applied to reproducing kernel Hilbert spaces.
result Near-optimal convergence rate and computationally efficient algorithms proved.
Bayesian optimization under heavy-tailed noise with improved regret bounds.
problem Optimizing functions with heavy-tailed noise.
method Adaptation of GP-UCB with reward truncation and novel kernel approximation techniques.
result Regret bounds matching the lower bound for SE kernel, improving upon existing methods.
Algorithm for low-rank matrix bandits with heavy-tailed rewards, achieving nearly optimal regret bound.
problem Stochastic low-rank matrix bandit with heavy-tailed rewards.
method LOTUS algorithm using truncation and dynamic exploration.
result Regret bound of order $ ilde O(d^ rac{3}{2}r^ rac{1}{2}T^ rac{1}{1+δ}/ ilde{D}_{rr})$ without knowing .
Extrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning from Observationscs.LG
T-REX improves reinforcement learning from suboptimal demonstrations.
problem Inability of existing IRL methods to significantly outperform the demonstrator.
method Trajectory-ranked Reward EXtrapolation (T-REX) algorithm.
result T-REX outperforms state-of-the-art methods on Atari and MuJoCo tasks.
New method uses hindsight to make exploration robust in stochastic environments.
problem Exploration in sparse-reward or reward-free environments, especially in stochastic settings.
method Learn representations of the future that capture unpredictable aspects, using them to predict and reward only the predictable parts of the world.
result Improves exploration in Atari games and Montezuma's Revenge, robust to stochasticity.
Presents SPEED, an algorithm for optimal policy evaluation in linear bandits with heteroscedastic noise.
problem Optimal data collection for policy evaluation in linear bandits with heteroscedastic reward noise.
method Formulated an optimal design for weighted least squares estimates, derived the optimal sample allocation, introduced SPEED algorithm, and derived regret bounds.
result SPEED leads to policy evaluation with MSE comparable to oracle strategy and significantly lower than random policy execution.
We discuss a natural game of competition and solve the corresponding mean field game with \emph{common noise} when agents' rewards are \emph{rank dependent}. We use this solution to provide an approximate Nash equilibrium for the finite player game and obtain the rate of convergence.
New RL agent learns sparse rewards efficiently.
problem Sparse-reward environments and computational expense.
method Active inference with novel free energy minimization.
result High sample efficiency and online operation.
Demon aligns diffusion models without retraining or backpropagation.
problem Aligning diffusion models with user preferences.
method Stochastic optimization to control noise distribution.
result Significantly improves aesthetics scores for text-to-image generation.
New algorithms reduce contextual bandits' regret without knowing reward noise variances.
problem Reducing regret in contextual bandits with unknown reward noise variances.
method Developed new algorithms based on the optimism principle.
result Regret scales as the square root of the sum of measurement variances, not the time horizon.
NeuralUCB tackles contextual bandits with neural networks and UCB for near-optimal regret.
problem Stochastic contextual bandit problem with unknown bounded reward function.
method NeuralUCB uses neural networks and UCB for efficient exploration.
result Achieves regret under standard assumptions.
Proposes DISCO, the first CVI for density-based clustering with noise.
problem Evaluation of noise assignments in density-based clustering.
method Density-connectivity and Silhouette Coefficient adaptation for noise evaluation.
result DISCO is the first CVI to explicitly assess noise assignments.
Study derivative-free methods for linear policies in linear-quadratic systems.
problem Optimizing policies in linear-quadratic systems with limited derivative information.
method Derivative-free methods applied to linear policies over various noise and reward feedback settings.
result These methods converge to near-optimal policies with a polynomial number of zero-order evaluations.
NADPEx uses dropout to enable temporally consistent exploration in reinforcement learning.
problem Achieving temporally consistent exploration in reinforcement learning agents.
method Integrates dropout into reinforcement learning policies to ensure temporal consistency.
result NADPEx outperforms naive exploration and parameter noise in tasks with sparse rewards.
We introduce NoisyNet, a deep reinforcement learning agent with parametric noise added to its weights, and show that the induced stochasticity of the agent's policy can be used to aid efficient exploration. The parameters of the noise are learned with gradient descent along with the remaining network weights. NoisyNet …
Paper introduces Latent-CLIP for efficient text-image comparison in latent space.
problem Efficiently compare text and images in latent space without costly decoding.
method Trains CLIP model in latent space, uses Latent-CLIP rewards for noise optimization, and guides generation away from harmful content.
result Latent-CLIP matches CLIP performance on text-image classification and harmful content detection.
TT-DAC-PS: Twin-Target Deterministic Actor-Critic with Policy Smoothing for Optimal Trade Executioncs.AI
TT-DAC-PS: A deterministic actor-critic approach for optimal trade execution
problem Optimal execution of large stock sell programs
method Twin-Target Deterministic Actor-Critic with Policy Smoothing
result Reduces mean implementation shortfall percentage
Improved Bayesian regret bound for linear Thompson sampling with general distributions.
problem Proving an improved Bayesian regret bound for linear Thompson sampling with general distributions.
method Generalized elliptical potential lemma for non-Gaussian noise and prior distributions.
result Minimax optimal regret bound for changing action sets with general prior and noise distributions.
Curiosity-Critic improves world model training by focusing on cumulative prediction error.
problem Training world models with intrinsic rewards that consider cumulative prediction error.
method Curiosity-Critic uses a surrogate reward based on the difference between current and asymptotic prediction errors, estimated online by a co-trained critic.
result Curiosity-Critic outperforms other methods in training speed and final world model accuracy.