arXiv research

Local search improves GFlowNets' ability to generate high-reward samples.

In many environments only a tiny subset of all states yield high reward. In these cases, few of the interactions with the environment provide a relevant learning signal. Hence, we may want to preferentially train on those high-reward states and the probable trajectories leading to them. To this end, we advocate for the…

New algorithm finds high-reward combinatorial sets with fewest pulls.

DTS improves inference-time alignment of diffusion models with less compute.

ACE improves GFlowNet exploration efficiency by balancing complementary search strategies.

We present Memory Augmented Policy Optimization (MAPO), a simple and novel way to leverage a memory buffer of promising trajectories to reduce the variance of policy gradient estimate. MAPO is applicable to deterministic environments with discrete actions, such as structured prediction and combinatorial optimization ta…

SA-GFN corrects biases in GFlowNets due to graph symmetries.

We address the problem of regret minimization in logistic contextual bandits, where a learner decides among sequential actions or arms given their respective contexts to maximize binary rewards. Using a fast inference procedure with Polya-Gamma distributed augmentation variables, we propose an improved version of Thomp…

Adaptive teacher improves sample efficiency and mode coverage in sampling tasks.

Improves generative models by optimizing rewards and sample editing.

Boosted GFlowNets improve exploration by sequentially training GFlowNets with residual rewards.

This work improves online fine-tuning of diffusion models for specific properties.

Diffusion LLMs can efficiently generate harmful prompts for adversarial testing.

Improved Thompson Sampling reduces regret in contextual bandits and reinforcement learning.

Demon aligns diffusion models without retraining or backpropagation.

This paper improves sample efficiency for off-policy evaluation with preference data.

Align-RUDDER improves reinforcement learning with few demonstrations by redistributing rewards.

OP-GFNs sample candidates in order-preserving proportion to a learned reward function.

Improves text-to-image diffusion models using GFlowNets.

New method optimizes diffusion models without fine-tuning, integrating soft value functions.

In this paper, we propose an information-theoretic exploration strategy for stochastic, discrete multi-armed bandits that achieves optimal regret. Our strategy is based on the value of information criterion. This criterion measures the trade-off between policy information and obtainable rewards. High amounts of policy …

Study shows challenges in reinforcement learning math problems, proposing enhancements and a hardness measure.

Curriculum learning has been successfully used in reinforcement learning to accelerate the learning process, through knowledge transfer between tasks of increasing complexity. Critical tasks, in which suboptimal exploratory actions must be minimized, can benefit from curriculum learning, and its ability to shape explor…

New meta-learning approach for bandit policies that achieve high average reward.

New method tackles safe reinforcement learning from offline data.

Exploration strategy design is one of the challenging problems in reinforcement learning~(RL), especially when the environment contains a large state space or sparse rewards. During exploration, the agent tries to discover novel areas or high reward~(quality) areas. In most existing methods, the novelty and quality in …

A new algorithm for contextual combinatorial bandits reduces regret.

Proper balance between exploitation and exploration is what makes good decisions, which achieve high rewards like payoff or evolutionary fitness. The Infomax principle postulates that maximization of information directs the function of diverse systems, from living systems to artificial neural networks. While specific a…

Learning reward functions from data is a promising path towards achieving scalable Reinforcement Learning (RL) for robotics. However, a major challenge in training agents from learned reward models is that the agent can learn to exploit errors in the reward model to achieve high reward behaviors that do not correspond …

In reinforcement learning the Q-values summarize the expected future rewards that the agent will attain. However, they cannot capture the epistemic uncertainty about those rewards. In this work we derive a new Bellman operator with associated fixed point we call the `knowledge values'. These K-values compress both the …

Distributed learning adapts to diverse devices, improving performance.

We consider the problem of evaluating the quality of startup companies. This can be quite challenging due to the rarity of successful startup companies and the complexity of factors which impact such success. In this work we collect data on tens of thousands of startup companies, their performance, the backgrounds of t…

In stochastic multi-armed bandits, the reward distribution of each arm is assumed to be stationary. This assumption is often violated in practice (e.g., in recommendation systems), where the reward of an arm may change whenever is selected, i.e., rested bandit setting. In this paper, we consider the non-parametric rott…

A new algorithm for resource-aware multi-armed bandits minimizes regret.

Equivariant neural networks improve performance and generalization in lattice field theory tasks.

Improves inference-time alignment for diffusion models without updating weights.

Meta-learning curiosity algorithms improves exploration across various tasks.

New method prevents RLHF alignment collapse by accounting for policy's influence on reward model updates.

Study designs logging policies to minimize off-policy evaluation error.

QuantaAlpha uses evolutionary algorithms to mine financial alpha robustly across market distributions.

In retail, there are predictable yet dramatic time-dependent patterns in customer behavior, such as periodic changes in the number of visitors, or increases in customers just before major holidays. The current paradigm of multi-armed bandit analysis does not take these known patterns into account. This means that for a…

How effective are the most common trading models? The answer may help investors realize upsides to using each model, act as a segue for investors into more complex financial analysis and machine learning, and to increase financial literacy amongst students. Creating original versions of popular models, like linear regr…

New algorithms optimize decision-making under uncertainty with graph information.

The paper proposes a machine learning framework for detecting DeFi fraud across multiple blockchain chains.

The paper analyzes RLVR's training dynamics, proving convergence depends on aligning update direction with Gradient Gap.

We study a general problem of allocating limited resources to heterogeneous customers over time under model uncertainty. Each type of customer can be serviced using different actions, each of which stochastically consumes some combination of resources, and returns different rewards for the resources consumed. We consid…

ePF improves PF for ITS by balancing exploration and exploitation, outperforming baselines.

Proposes a new RL method to fine-tune flow-based models with arbitrary rewards.