arXiv research

TERA method speeds up derivative Gaussian processes in high dimensions.

Variance reduction (VR) methods boost the performance of stochastic gradient descent (SGD) by enabling the use of larger, constant stepsizes and preserving linear convergence rates. However, current variance reduced SGD methods require either high memory usage or an exact gradient computation (using the entire dataset)…

New method improves convergence and reduces variance in noisy optimization problems.

Two reduction schemes for symplectic manifolds are shown equivalent.

Push-SAGA is a decentralized algorithm for directed graphs that converges linearly.

Some aspects of the multidimensional soliton geometry are considered. It is shown that some simples (2+1)-dimensional equations are exact reductions of the Self-Dual Yang-Mills equation or its higher hierarchy.

Improved EXACT strategy reduces GNN memory consumption and runtime.

This work studies the implicit bias of mini-batch SGD in classification.

Unified convergence analysis of alpha-SVRG under strong convexity.

This paper discusses the exact simulation of the stock price process underlying the 3/2 model. Using a result derived by Craddock and Lennox using Lie Symmetry Analysis, we adapt the Broadie-Kaya algorithm for the simulation of affine processes to the 3/2 model. We also discuss variance reduction techniques and find th…

New method computes affine normal directions efficiently for sparse polynomials.

We study control systems invariant under a Lie group with application to the problem of nonlinear trajectory planning. A theory of symmetry reduction of exterior differential systems is employed to demonstrate how symmetry reduction and reconstruction is effective in the explicit, exact construction of planned system t…

We reduce variance in Bures-Wasserstein variational inference.

Abstract: Generalized reduction methods for symmetries in graded geometry.

Policy gradient methods are very attractive in reinforcement learning due to their model-free nature and convergence guarantees. These methods, however, suffer from high variance in gradient estimation, resulting in poor sample efficiency. To mitigate this issue, a number of variance-reduction approaches have been prop…

Highly expressive directed latent variable models, such as sigmoid belief networks, are difficult to train on large datasets because exact inference in them is intractable and none of the approximate inference methods that have been applied to them scale well. We propose a fast non-iterative approximate inference metho…

Optimization geometrodynamics simplifies adaptive optimizer dynamics.

We obtain the first polynomial-time algorithm for exact tensor completion that improves over the bound implied by reduction to matrix completion. The algorithm recovers an unknown 3-tensor with $r$ incoherent, orthogonal components in $\mathbb R^n$ from $r\cdot \tilde O(n^{1.5})$ randomly observed entries of the tensor…

We study a class of nonlinear pricing models which involves the feedback effect from the dynamic hedging strategies on the price of asset introduced by Sircar and Papanicolaou. We are first to study the case of a nonlinear demand function involved in the model. Using a Lie group analysis we investigate the symmetry pro…

Families of exact solutions are found to a nonlinear modification of the Black-Scholes equation. This risk-adjusted pricing methodology model (RAPM) incorporates both transaction costs and the risk from a volatile portfolio. Using the Lie group analysis we obtain the Lie algebra admitted by the RAPM equation. It gives …

Unified model for reducing dimensions and clustering high-dimensional data.

Gradient descent achieves exact linear convergence rate for symmetric matrix completion.

In this paper, we investigate the non-linear Black--Scholes equation: $u_t+ax^2u_{xx}+bx^3u_{xx}^2+c(xu_x-u)=0,\quad a,b>0,\ c\geq0.$ and show that the one can be reduced to the equation $u_t+(u_{xx}+u_x)^2=0$ by an appropriate point transformation of variables. For the resulting equation, we study the group-theore…

New method extends invariant reduction to rescaled geometric structures.

This work proposes using zero-variance control variates to reduce variance in pathwise gradient estimators for variational inference.

A new method reduces the complexity of decentralized optimization.

We propose Kernel Hamiltonian Monte Carlo (KMC), a gradient-free adaptive MCMC algorithm based on Hamiltonian Monte Carlo (HMC). On target densities where classical HMC is not an option due to intractable gradients, KMC adaptively learns the target's gradient structure by fitting an exponential family model in a Reprod…

Improves gradient estimation for discrete distributions with variance reduction techniques.

Taking advantage of the recent litterature on exact simulation algorithms (Beskos, Papaspiliopoulos and Roberts) and unbiased estimation of the expectation of certain fonctional integrals (Wagner, Beskos et al. and Fearnhead et al.), we apply an exact simulation based technique for pricing continuous arithmetic average…

RevDEQs improve performance on tasks with exact gradients and fewer function evaluations.

Proves new inequality linking spectral numbers of Lagrangians and their reductions.

We construct symplectic and Kähler ray reduced spaces and discuss their relation with the Marsden-Weinstein (point) reduction. This Kähler reduction is well defined even when the momentum value is not totally isotropic. The compatibility of the ray reduction with the cone construction and the Boothby-Wang fibration is …

We discuss Levi-Civita connections on Courant algebroids. We define an appropriate generalization of the curvature tensor and compute the corresponding scalar curvatures in the exact and heterotic case, leading to generalized (bosonic) Einstein-Hilbert type of actions known from supergravity. In particular, we carefull…

Torsion found in knot homology, challenging augmentation theories.

New method improves scalability of SGD for large datasets.

This paper explores the computational hardness of generating latent vectors for generative models.

TrIM improves gradient-based dimension reduction and regression.

Identifies a gradient flow to solve kernel learning problems with noise reduction.

Gradient descent slows significantly in over-parameterized single neuron learning.

Extends dimension reduction to data-driven settings without gradients.

Unified framework for stable RL learning with theoretical guarantees.

Variable selection and dimension reduction are two commonly adopted approaches for high-dimensional data analysis, but have traditionally been treated separately. Here we propose an integrated approach, called sparse gradient learning (SGL), for variable selection and dimension reduction via learning the gradients of t…

We show that certain submanifolds of generalized complex manifolds ("weak branes") admit a natural quotient which inherits a generalized complex structure. This is analog to quotienting coisotropic submanifolds of symplectic manifolds. In particular Gualtieri's generalized complex submanifolds ("branes") quotient to sp…

We generalize stochastic smoothing for gradient estimation of non-differentiable functions.

Variance reduction methods such as SVRG and SpiderBoost use a mixture of large and small batch gradients to reduce the variance of stochastic gradients. Compared to SGD, these methods require at least double the number of operations per update to model parameters. To reduce the computational cost of these methods, we i…

Evolution Strategies (ES) are a powerful class of blackbox optimization techniques that recently became a competitive alternative to state-of-the-art policy gradient (PG) algorithms for reinforcement learning (RL). We propose a new method for improving accuracy of the ES algorithms, that as opposed to recent approaches…

Paper tackles gradient-free minimax optimization with variance reduction for faster convergence.

Paper extends knowledge gradient for preferential BO, overcoming computational challenges.