arXiv research

Study uses randomized value functions to enhance exploration in reinforcement learning.

We study the use of randomized value functions to guide deep exploration in reinforcement learning. This offers an elegant means for synthesizing statistically and computationally efficient exploration with common practical approaches to value function learning. We present several reinforcement learning algorithms that…

Randomized value functions offer a promising approach towards the challenge of efficient exploration in complex environments with high dimensional state and action spaces. Unlike traditional point estimate methods, randomized value functions maintain a posterior distribution over action-space values. This prevents the …

We propose randomized least-squares value iteration (RLSVI) -- a new reinforcement learning algorithm designed to explore and generalize efficiently via linearly parameterized value functions. We explain why versions of least-squares value iteration that use Boltzmann or epsilon-greedy exploration can be highly ineffic…

QR-MIX models joint state-action values as a distribution to handle randomness in MARL.

Introduces epistemic deep learning for better uncertainty estimation in neural networks.

We discuss the relative merits of optimistic and randomized approaches to exploration in reinforcement learning. Optimistic approaches presented in the literature apply an optimistic boost to the value estimate at each state-action pair and select actions that are greedy with respect to the resulting optimistic value f…

Paper introduces a method for operator learning using random features.

New method calculates Shapley values for uncertain functions.

Devoted to multi-task learning and structured output learning, operator-valued kernels provide a flexible tool to build vector-valued functions in the context of Reproducing Kernel Hilbert Spaces. To scale up these methods, we extend the celebrated Random Fourier Feature methodology to get an approximation of operator-…

Improved Random Search for hyperparameter optimization.

Sublinear functionals of random variables are known as sublinear expectations; they are convex homogeneous functionals on infinite-dimensional linear spaces. We extend this concept for set-valued functionals defined on measurable set-valued functions (which form a nonlinear space), equivalently, on random closed sets. …

Paper improves worst-case regret bounds for RLSVI in reinforcement learning.

Random walk constructs Morse functions on surfaces.

Improved RL algorithm reduces regret in large state spaces.

Generalizes randomized SVD for better matrix approximations using Gaussian vectors.

We prove a sharp estimate on the expected value of the integral of the index of a simple random walk on the square or triangular lattice. This gives new lower bounds on the averaged Dehn function, which measures the expected area needed to fill a random curve with a disc.

Random feature models approximate functions in Banach spaces efficiently.

Paper analyzes error bounds for learning with vector-valued RF, improving existing analyses.

The paper shows vector-valued risk measures ignore dependence structures.

We study random Morse functions on a Riemann manifold $(M^m,g)$ defined as a random Gaussian weighted superpositions of eigenfunctions of the Laplacian of the metric $g$. The randomness is determined by a fixed Schwartz function $w$ and a small parameter $\varepsilon>0$. We first prove that as $\varepsilon\to 0$ the ex…

Randomized algorithm solves vector-valued regression problems with low-rank operators.

The paper analyzes stability of random matrix products with Markovian noise.

The paper analyzes Random Search and introduces BLiN-MOS for bandit learning.

RANDPOL uses randomized networks for efficient reinforcement learning in continuous state and action MDPs.

We introduce a new functional measure of tail dependence for weakly dependent (asymptotically independent) random vectors, termed weak tail dependence function. The new measure is defined at the level of copulas and we compute it for several copula families such as the Gaussian copula, copulas of a class of Gaussian mi…

UVU simplifies value uncertainty quantification in RL.

We address the problem of automatic generation of features for value function approximation. Bellman Error Basis Functions (BEBFs) have been shown to improve the error of policy evaluation with function approximation, with a convergence rate similar to that of value iteration. We propose a simple, fast and robust algor…

New methods for ordinal classification of interval-valued data and functional data.

The paper computes special values of combinatorial zeta functions to reveal topological properties of manifolds.

New algorithm explores reinforcement learning with noisy data.

We introduce a new method to explain Gaussian processes using Shapley values.

We consider a problem of optimal investment with intermediate consumption and random endowment in an incomplete semimartingale model of a financial market. We establish the key assertions of the utility maximization theory assuming that both primal and dual value functions are finite in the interiors of their domains a…

The paper defines and analyzes set-valued stochastic integrals for Lévy processes.

We study the optimal stopping of an American call option in a random time-horizon under exponential spectrally negative Lévy models. The random time-horizon is modeled as the so-called Omega default clock in insurance, which is the first time when the occupation time of the underlying Lévy process below a level $y$, ex…

We relate the distribution of eigenvalues of a random symmetric matrix in the Gaussian Orthogonal Ensemble to the distribution of critical values of a random linear combination of eigenfunctions of the Laplacian on a compact Riemann manifold. We then prove a central limit theorem describing what happens when the dimens…

Software estimates inequality in random systems with changing communities.

A note on extending Chernoff bound for unit interval random variables.

Online minimization of an unknown convex function over the interval $[0,1]$ is considered under first-order stochastic bandit feedback, which returns a random realization of the gradient of the function at each query point. Without knowing the distribution of the random gradients, a learning algorithm sequentially choo…

The paper constructs random concave functions on the unit simplex.

The paper proposes a Gaussian mixture model for Hilbert-space-valued data.

Develops statistical framework for analyzing functional data extremes.

In distributed function computation, each node has an initial value and the goal is to compute a function of these values in a distributed manner. In this paper, we propose a novel token-based approach to compute a wide class of target functions to which we refer as "Token-based function Computation with Memory" (TCM) …

In this paper we discuss the asymptotic behaviour of random contractions $X=RS$, where $R$, with distribution function $F$, is a positive random variable independent of $S\in (0,1)$. Random contractions appear naturally in insurance and finance. Our principal contribution is the derivation of the tail asymptotics of $X…

Model stock price dynamics using semi-Markov processes.

Paper presents an efficient exploration method for reinforcement learning.

This paper presents a new approach, called perturb-max, for high-dimensional statistical inference that is based on applying random perturbations followed by optimization. This framework injects randomness to maximum a-posteriori (MAP) predictors by randomly perturbing the potential function for the input. A classic re…

Study on risk measures using distorted Choquet integrals with random distortions.