arXiv research

Paper analyzes Q-learning with neural networks, proving a fast convergence rate.

Paper analyzes finite-time performance of SA in RL with Markovian noise.

The question of the optimality of Thompson Sampling for solving the stochastic multi-armed bandit problem had been open since 1933. In this paper we answer it positively for the case of Bernoulli rewards by providing the first finite-time analysis that matches the asymptotic rate given in the Lai and Robbins lower boun…

Study on SA with heavy-tailed and LRD noise, establishing finite-time bounds.

Paper analyzes NAC with neural networks for efficient policy optimization.

This work analyzes actor-critic methods for faster convergence.

New analysis of stochastic approximation with non-expansive mappings.

Continuous-time analysis shows SGD with noise prefers flat minima.

Paper analyzes finite-time convergence of double Q-learning.

Temporal difference learning (TD) is a simple iterative algorithm used to estimate the value function corresponding to a given policy in a Markov decision process. Although TD is one of the most widely used algorithms in reinforcement learning, its theoretical analysis has proved challenging and few guarantees on its s…

Paper analyzes biased stochastic approximation with a novel multistep Lyapunov function.

Proposes a TS approach for Bayesian optimization with preferential feedback.

Paper analyzes SVGD algorithm for non-asymptotic convergence.

We tackle the problem of online reward maximisation over a large finite set of actions described by their contexts. We focus on the case when the number of actions is too big to sample all of them even once. However we assume that we have access to the similarities between actions' contexts and that the expected reward…

The paper analyzes a neural network two-sample test using kernel analysis.

This paper addresses the problem of distributed detection in multi-agent networks. Agents receive private signals about an unknown state of the world. The underlying state is globally identifiable, yet informative signals may be dispersed throughout the network. Using an optimization-based framework, we develop an iter…

Generative profiling improves real-time task timing for varied resource contexts.

Paper analyzes finite-time guarantees for preference-based RL.

New algorithm for nonstationary multi-armed bandits with optimal performance.

The paper analyzes an actor-critic algorithm with target networks for deep reinforcement learning.

Unified analysis of stochastic iterative algorithms using Lyapunov functions.

New analysis shows actor-critic method converges efficiently in practical settings.

Study on improving the linear two-time-scale stochastic approximation method with a restarting scheme.

In the context of stochastic continuum-armed bandits, we present an algorithm that adapts to the unknown smoothness of the objective function. We exhibit and compute a polynomial cost of adaptation to the H{ö}lder regularity for regret minimization. To do this, we first reconsider the recent lower bound of Locatelli an…

The paper offers precise bounds for averaged LSA iterates in linear systems.

In this paper we analyze the long-time behaviour of 3 dimensional Ricci flow with surgery. We prove that under the topological condition that the initial manifold only has non-aspherical or hyperbolic components in its geometric decomposition, there are only finitely many surgeries and the curvature is bounded by $C t^…

Continuous time analysis of bubble formation in harmonic maps.

We consider the problem of learning a realization for a linear time-invariant (LTI) dynamical system from input/output data. Given a single input/output trajectory, we provide finite time analysis for learning the system's Markov parameters, from which a balanced realization is obtained using the classical Ho-Kalman al…

Improves functional linear regression with shape transfer learning.

New algorithm optimizes unimodal bandits using empirical divergence.

We consider an original problem that arises from the issue of security analysis of a power system and that we name optimal discovery with probabilistic expert advice. We address it with an algorithm based on the optimistic paradigm and on the Good-Turing missing mass estimator. We prove two different regret bounds on t…

This paper investigates stochastic and adversarial combinatorial multi-armed bandit problems. In the stochastic setting under semi-bandit feedback, we derive a problem-specific regret lower bound, and discuss its scaling with the dimension of the decision space. We propose ESCB, an algorithm that efficiently exploits t…

Improved prediction algorithm for 'easy' sequences with reduced regret.

We develop a method that relates the truncated cumulant-function of the fourth order with the Lévian cumulant-function. This gives us explicit formulas for the Lévy-parameters, which allow a real-time analysis of the state of a random-motion. Cumbersome procedures like maximum-likelihood or least-square methods are unn…

Minimal assumptions analysis of Q-learning with time-varying policies.

Policy gradient methods achieve linear convergence in simple MDPs.

Despite remarkable empirical success, the training dynamics of generative adversarial networks (GAN), which involves solving a minimax game using stochastic gradients, is still poorly understood. In this work, we analyze last-iterate convergence of simultaneous gradient descent (simGD) and its variants under the assump…

This paper proposes a novel model of financial prices where: (i) prices are discrete; (ii) prices change in continuous time; (iii) a high proportion of price changes are reversed in a fraction of a second. Our model is analytically tractable and directly formulated in terms of the calendar time and price impact curve. …

This work analyzes $Q$-learning with adaptive stepsizes for finite-time convergence.

Market activity scales near a constant of 0.632 in intrinsic time.

We study the Stochastic Gradient Langevin Dynamics (SGLD) algorithm for non-convex optimization. The algorithm performs stochastic gradient descent, where in each step it injects appropriately scaled Gaussian noise to the update. We analyze the algorithm's hitting time to an arbitrary subset of the parameter space. Two…

We study a multiplayer stochastic multi-armed bandit problem in which players cannot communicate, and if two or more players pull the same arm, a collision occurs and the involved players receive zero reward. We consider the challenging heterogeneous setting, in which different arms may have different means for differe…

We consider sequential decision making problems for binary classification scenario in which the learner takes an active role in repeatedly selecting samples from the action pool and receives the binary label of the selected alternatives. Our problem is motivated by applications where observations are time consuming and…

Coherent Multiplex analyzes real-time wavelet coherence among multiple signals.

This handbook translates lead time analysis into R code.

Mathematical framework for transfer learning feasibility and transfer risk.

In this paper we analyze the long-time behavior of 3 dimensional Ricci flows with surgery. Our main result is that if the surgeries are performed correctly, then only finitely many surgeries occur and after some time the curvature is bounded by $C t^{-1}$. This result confirms a conjecture of Perelman. In the course of…

This paper examines challenges and solutions for solving variational inequalities.