A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
MTAdam optimizes multiple loss terms in neural models, balancing gradients dynamically.
Paper establishes estimates for nonlinear equations on compact manifolds.
The paper proposes a method to learn differentially private variational autoencoders with term-wise gradient aggregation.
The study analyzes implicit biases in neural networks using backward error analysis.
We consider the complex Monge-Ampère equation with an additional linear gradient term inside the determinant. We prove existence and uniqueness of solutions to this equation on compact Hermitian manifolds.
LEM efficiently models long-term sequences with gradients.
FPG uses fractional calculus for efficient reinforcement learning with long-term memory.
Article provides Bernstein gradient estimates for heat equations with potential terms.
Paper proposes an -policy gradient for online pricing, reducing regret to .
The paper analyzes the intrinsic exploration terms in policy-gradient algorithms.
We apply stochastic average gradient (SAG) algorithms for training conditional random fields (CRFs). We describe a practical implementation that uses structure in the CRF gradient to reduce the memory requirement of this linearly-convergent stochastic gradient method, propose a non-uniform sampling scheme that substant…
New algorithms improve Langevin Monte Carlo efficiency.
New bounds show SGD can match deterministic gradient descent's convergence rate.
Stochastic gradient descent's long-term fluctuations are described by a diffusion limit.
Gradient descent struggles with learning a single neuron with bias.
In this paper, we consider a class of finite-sum convex optimization problems defined over a distributed multiagent network with agents connected to a central server. In particular, the objective function consists of the average of () smooth components associated with each network agent together with a s…
A new correction term improves sample efficiency in deep reinforcement learning.
The paper provides gradient estimates for nonlinear heat-type equations on smooth metric measure spaces.
Adam's hyperparameters implicitly regularize solutions, penalizing or impeding loss gradients' norms.
AdaX improves Adam by exponentially accumulating past gradients, leading to better performance in machine learning tasks.
Develops methods to calculate global index of real polynomials.
We study a hybrid conditional gradient - smoothing algorithm (HCGS) for solving composite convex optimization problems which contain several terms over a bounded set. Examples of these include regularization problems with several norms as penalties and a norm constraint. HCGS extends conditional gradient methods to cas…
New method reduces bias in incomplete data using deliberate missingness.
This paper proposes a novel proximal-gradient algorithm for a decentralized optimization problem with a composite objective containing smooth and non-smooth terms. Specifically, the smooth and nonsmooth terms are dealt with by gradient and proximal updates, respectively. The proposed algorithm is closely related to a p…
New bound relaxes uniform gradient norm assumptions for PAC-Bayesian bounds.
Gradient flows for knot energies ensure long-term existence of knotted loops.
Enhances gradient estimates for Hermitian Monge-Ampère equations.
We propose the stochastic average gradient (SAG) method for optimizing the sum of a finite number of smooth convex functions. Like stochastic gradient (SG) methods, the SAG method's iteration cost is independent of the number of terms in the sum. However, by incorporating a memory of previous gradient values the SAG me…
We study a reinforcement learning setting, where the state transition function is a convex combination of a stochastic continuous function and a deterministic function. Such a setting generalizes the widely-studied stochastic state transition setting, namely the setting of deterministic policy gradient (DPG). We firstl…
Note on gradient estimates for complex Monge-Ampere equation.
Modelling long-term dependencies is a challenge for recurrent neural networks. This is primarily due to the fact that gradients vanish during training, as the sequence length increases. Gradients can be attenuated by transition operators and are attenuated or dropped by activation functions. Canonical architectures lik…
Single ReLU neuron's gradient dynamics reveal support vectors as key to generalization.
A hybrid strategy forecasts short-term loads using Warm-start Gradient Tree Boosting.
Gradient flow on diffeomorphisms for image registration, with well-posedness proven.
In this paper we present CatBoost, a new open-sourced gradient boosting library that successfully handles categorical features and outperforms existing publicly available implementations of gradient boosting in terms of quality on a set of popular publicly available datasets. The library has a GPU implementation of lea…
Proposes log density gradient to improve reinforcement learning sample complexity.
New RNN model handles long-term dependencies in irregularly-sampled time series.
We present a unifying framework for adapting the update direction in gradient-based iterative optimization methods. As natural special cases we re-derive classical momentum and Nesterov's accelerated gradient method, lending a new intuitive interpretation to the latter algorithm. We show that a new algorithm, which we …
The present paper develops a novel aggregated gradient approach for distributed machine learning that adaptively compresses the gradient communication. The key idea is to first quantize the computed gradients, and then skip less informative quantized gradient communications by reusing outdated gradients. Quantizing and…
Recurrent neural networks are known for their notorious exploding and vanishing gradient problem (EVGP). This problem becomes more evident in tasks where the information needed to correctly solve them exist over long time scales, because EVGP prevents important gradient components from being back-propagated adequately …
Asynchronous distributed stochastic gradient descent methods have trouble converging because of stale gradients. A gradient update sent to a parameter server by a client is stale if the parameters used to calculate that gradient have since been updated on the server. Approaches have been proposed to circumvent this pro…
It is a known fact that training recurrent neural networks for tasks that have long term dependencies is challenging. One of the main reasons is the vanishing or exploding gradient problem, which prevents gradient information from propagating to early layers. In this paper we propose a simple recurrent architecture, th…
We analyze (stochastic) gradient descent (SGD) with delayed updates on smooth quasi-convex and non-convex functions and derive concise, non-asymptotic, convergence rates. We show that the rate of convergence in all cases consists of two terms: (i) a stochastic term which is not affected by the delay, and (ii) a higher …
The natural gradient method has been used effectively in conjugate Gaussian process models, but the non-conjugate case has been largely unexplored. We examine how natural gradients can be used in non-conjugate stochastic settings, together with hyperparameter learning. We conclude that the natural gradient can signific…
DIFF2 improves differential privacy in nonconvex optimization with better utility bounds.
Despite recent innovations in network architectures and loss functions, training RNNs to learn long-term dependencies remains difficult due to challenges with gradient-based optimisation methods. Inspired by the success of Deep Neuroevolution in reinforcement learning (Such et al. 2017), we explore the use of gradient-…
We study distributed stochastic gradient (D-SG) method and its accelerated variant (D-ASG) for solving decentralized strongly convex stochastic optimization problems where the objective function is distributed over several computational units, lying on a fixed but arbitrary connected communication graph, subject to loc…
Deep neural networks are vulnerable to adversarial attacks.