arXiv research

Study tackles infinitely many-armed bandits with rotting rewards, achieving tight regret bounds.

New algorithm reduces regret in infinitely many-armed bandits with decreasing rewards.

New algorithm handles both decaying and non-decaying bandit problems.

In stochastic multi-armed bandits, the reward distribution of each arm is assumed to be stationary. This assumption is often violated in practice (e.g., in recommendation systems), where the reward of an arm may change whenever is selected, i.e., rested bandit setting. In this paper, we consider the non-parametric rott…

The Multi-Armed Bandits (MAB) framework highlights the tension between acquiring new knowledge (Exploration) and leveraging available knowledge (Exploitation). In the classical MAB problem, a decision maker must choose an arm at each time step, upon which she receives a reward. The decision maker's objective is to maxi…

New method tracks shifts in infinite-armed bandits without prior knowledge.

Graph-Triggered Bandits unify rested and restless bandits with graph-defined arm interactions.

TURB-Rot provides a large database of turbulent rotating flow snapshots for research.

ROTS improves sentence similarity by incorporating structural information.

Thomas Rot discusses a topology theorem at a math conference.

We investigate Legendrian graphs in $(\R^3, ξ_{std})$. We extend the classical invariants, Thurston-Bennequin number and rotation number to Legendrian graphs. We prove that a graph can be Legendrian realized with all its cycles Legendrian unknots with $tb=-1$ and $rot=0$ if and only if it does not contain $K_4$ as a mi…

New R-equivalence classes found for torus knot diagrams.

Enhanced rotation prediction improves SSL models by capturing both shape and texture information.

Develops a novel global pooling framework using optimal transport.

Researchers predict butt rot volume using harvester data and remote sensing.

Global existence of Willmore flow with boundary via Li-Yau inequality.

We prove two results on the classification of trivial Legendrian embeddings $g: G \rightarrow (S^3,ξ_{std})$ of planar graphs. First, the oriented Legendrian ribbon $R_g$ and rotation invariant $\text{rot}_g$ are a complete set of invariants. Second, if $G$ is 3-connected or contains $K_4$ as a minor, then the unique t…

In this paper, as the second in our series of papers on differential geometry of microlinear Frolicher spaces, we study differenital forms. The principal result is that the exterior differentiation is uniquely determined geometrically, just as grad (ient), div (ergence) and rot (ation) are uniquely determined geometric…

This paper presents a unified framework for smooth convex regularization of discrete optimal transport problems. In this context, the regularized optimal transport turns out to be equivalent to a matrix nearness problem with respect to Bregman divergences. Our framework thus naturally generalizes a previously proposed …

Let $M_n$ be the topological moduli space of all parallel n-cables of long framed oriented knots in 3-space. We construct in a combinatorial way for each natural number $n>1$ a 1-cocycle $R_n$ which represents a non trivial class in $H^1(M_n; \mathbb{Z} [x_1,x_2,...,x_1^{-1},x_2^{-1},...])$, where the number of variabl…

We advocate an optimization-centric view on and introduce a novel generalization of Bayesian inference. Our inspiration is the representation of Bayes' rule as infinite-dimensional optimization problem (Csiszar, 1975; Donsker and Varadhan; 1975, Zellner; 1988). First, we use it to prove an optimality result of standard…

Let $(M^n,g,\nabla f)$, $n\geq 3$, be an expanding gradient Ricci soliton with nonnegative sectional curvature whose asymptotic cone is isometric to $C(\mathbb{S}^{n-1}(c))$ where $\mathbb{S}^{n-1}(c)$ is the standard $(n-1)$-sphere of curvature $1/c^2$, with $c\in(0,1)$. We prove that if the convergence to the asympto…

New examples of non-simple knots in Lens spaces show rich botany.

New Legendrian bounds for non-fibered knots in 3-manifolds.

The paper proves $n$-transitivity for equivariant diffeomorphisms of manifolds.

The following three geometrical structures on a manifold are studied in detail: (1) Leibnizian: a non-vanishing 1-form $Ω$ plus a Riemannian metric $\h$ on its annhilator vector bundle. In particular, the possible dimensions of the automorphism group of a Leibnizian G-structure are characterized. (2) Galilean: Leibnizi…

Let $Ω$ be a smooth compact oriented 3-dimensional Riemannian manifold with boundary. A quaternion field is a pair $q=\{α,u\}$ of a function $α$ and a vector field $u$ on $Ω$. A field $q$ is {\it harmonic} if $α, u$ are continuous in $Ω$ and $\nablaα={\rm rot\,}u,\,{\rm div\,}u=0$ holds into $Ω$. The space ${\mathscr Q…

We classify Legendrian unknots in overtwisted contact structures on $S^3$. In particular, we show that up to contact isotopy for every pair $(n,\pm(n-1))$ with $n>0$ there are exactly two oriented non-loose Legendrian unknots in $S^3$ with Thurston-Bennequin invariant $n$ and rotation number $\pm(n-1)$. (Only one overt…

We propose a novel visual context-aware filter generation module which incorporates contextual information present in images into Convolutional Neural Networks (CNNs). In contrast to traditional CNNs, we do not employ the same set of learned convolution filters for all input image instances. Our proposed input-conditio…

We present numerical visualizations of Ricci Flow of surfaces and 3-dimensional manifolds of revolution. Ricci_rot is an educational tool which visualizes surfaces of revolution moving under Ricci flow. That these surfaces tend to remain embedded in R3 is what makes direct visualization possible. The numerical lessons …

Reward hacking exploits misspecified rewards, affecting agent capabilities and true performance.

Paper introduces PRMs to learn non-Markovian stochastic rewards for reinforcement learning.

This work analyzes the value of future reward information in RL.

Self-supervised reward prediction improves RL in sparse reward settings.

The study categorizes reward errors in reinforcement learning, finding some can be beneficial.

Reward collapse occurs when ranking-based reward models yield uniform rewards for different prompts.

Reward models need more than just accuracy for effective RLHF.

Proposes a method to boost deep reinforcement learning with sparse rewards.

Action guidance helps agents learn true objectives in games with sparse rewards.

New RL method uses distance between states instead of rewards for sparse reward environments.

Paper proposes RRD to learn proxy rewards for sparse delayed rewards in episodic reinforcement learning.

Learning reward functions from data is a promising path towards achieving scalable Reinforcement Learning (RL) for robotics. However, a major challenge in training agents from learned reward models is that the agent can learn to exploit errors in the reward model to achieve high reward behaviors that do not correspond …

Enhances reward specification in RL with a novel language-based approach.

Reward tweaking optimizes behavior for long-term goals by adjusting the reward function.

We propose a generic, Bayesian, information geometric approach to the exploration--exploitation trade-off in multi-armed bandit problems. Our approach, BelMan, uniformly supports pure exploration, exploration--exploitation, and two-phase bandit problems. The knowledge on bandit arms and their reward distributions is su…

Many reinforcement-learning researchers treat the reward function as a part of the environment, meaning that the agent can only know the reward of a state if it encounters that state in a trial run. However, we argue that this is an unnecessary limitation and instead, the reward function should be provided to the learn…

Extends reinforcement learning alignment to scalar rewards, improving math reasoning.

This work characterizes reward function partial identifiability and its impact on policy optimization.