A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
AdaX improves Adam by exponentially accumulating past gradients, leading to better performance in machine learning tasks.
AdaGrad fails to adapt to Hölder-smoothness in composite optimization problems.
GradaGrad adapts learning rate non-monotonically, overcoming AdaGrad's step size decrease.
This work addresses the instability in asynchronous data parallel optimization. It does so by introducing a novel distributed optimizer which is able to efficiently optimize a centralized model under communication constraints. The optimizer achieves this by pushing a normalized sequence of first-order gradients to a pa…
A new correction term improves sample efficiency in deep reinforcement learning.
A new method improves training speed and robustness for deep learning models.
Paper explores low-precision SGLD for neural networks, reducing costs without sacrificing performance.
Proposes reducing random error in stochastic optimization by variance regularization.
We consider the problem of minimizing a Lipschitz differentiable function over a class of sparse symmetric sets that has wide applications in engineering and science. For this problem, it is known that any accumulation point of the classical projected gradient (PG) method with a constant stepsize satisfies the $L…
The level curves of an analytic function germ almost always have bumps at unexpected points near the singularity. This profound discovery of N. A'Campo is fully explored in this paper for $f(z,w)\in \C\{z,w\}$, using the Newton-Puiseux infinitesimals and the notion of gradient canyon. Equally unexpected is the Dirac ph…
FedSel uses local differential privacy to protect data privacy in federated SGD.
Reinforcement learning for embodied agents is a challenging problem. The accumulated reward to be optimized is often a very rugged function, and gradient methods are impaired by many local optimizers. We demonstrate, in an experimental setting, that incorporating an intrinsic reward can smoothen the optimization landsc…
A new method for MARL with partial observations reduces communication overhead.
Efforts to reduce the numerical precision of computations in deep learning training have yielded systems that aggressively quantize weights and activations, yet employ wide high-precision accumulators for partial sums in inner-product operations to preserve the quality of convergence. The absence of any framework to an…
New method speeds up lifelong learning of complex tasks.
There is widespread sentiment that it is not possible to effectively utilize fast gradient methods (e.g. Nesterov's acceleration, conjugate gradient, heavy ball) for the purposes of stochastic optimization due to their instability and error accumulation, a notion made precise in d'Aspremont 2008 and Devolder, Glineur, …
Accumulated stock returns exhibit tempered skew t-distribution.
In this paper, we propose a Distributed Accumulated Newton Conjugate gradiEnt (DANCE) method in which sample size is gradually increasing to quickly obtain a solution whose empirical loss is under satisfactory statistical accuracy. Our proposed method is multistage in which the solution of a stage serves as a warm star…
We present a generic framework for trading off fidelity and cost in computing stochastic gradients when the costs of acquiring stochastic gradients of different quality are not known a priori. We consider a mini-batch oracle that distributes a limited query budget over a number of stochastic gradients and aggregates th…
New GFlowNet training framework using policy gradients for combinatorial object generation.
PES method reduces bias in gradient estimation for unrolled graphs.
Two new differentially private optimization algorithms derived from accelerated methods.
Distributed training of massive machine learning models, in particular deep neural networks, via Stochastic Gradient Descent (SGD) is becoming commonplace. Several families of communication-reduction methods, such as quantization, large-batch methods, and gradient sparsification, have been proposed. To date, gradient s…
Unfolding paths in Outer space accumulate on a simplex, not converge.
FedSKETCH and FedSKETCHGATE improve privacy and efficiency in federated learning.
In this paper, we study the problem of distributed multi-agent optimization over a network, where each agent possesses a local cost function that is smooth and strongly convex. The global objective is to find a common solution that minimizes the average of all cost functions. Assuming agents only have access to unbiase…
SLEDGE algorithm reduces gradient computation errors in optimization.
New data accumulation prevents model collapse in generative models.
Wealth inequality is an important matter for economic theory and policy. Ongoing debates have been discussing recent rise in wealth inequality in connection with recent development of active financial markets around the world. Existing literature on wealth distribution connects the origins of wealth inequality with a v…
The paper speeds up hyperparameter optimisation in Gaussian processes.
Low precision operations can provide scalability, memory savings, portability, and energy efficiency. This paper proposes SWALP, an approach to low precision training that averages low-precision SGD iterates with a modified learning rate schedule. SWALP is easy to implement and can match the performance of full-precisi…
Study of historic stock returns distributions, highlighting asymmetry and outliers.
AdEMAMix optimizer improves model performance and convergence speed.
Gradient Boosting Machine (GBM) is an extremely powerful supervised learning algorithm that is widely used in practice. GBM routinely features as a leading algorithm in machine learning competitions such as Kaggle and the KDDCup. In this work, we propose Accelerated Gradient Boosting Machine (AGBM) by incorporating Nes…
Accelerates policy optimization in RL with optimistic and adaptive updates.
This paper presents the Homeo-Heterostatic Value Gradients (HHVG) algorithm as a formal account on the constructive interplay between boredom and curiosity which gives rise to effective exploration and superior forward model learning. We envisaged actions as instrumental in agent's own epistemic disclosure. This motiva…
MQF forecasts multivariate quantiles globally.
The state-of-the-art hardware platforms for training Deep Neural Networks (DNNs) are moving from traditional single precision (32-bit) computations towards 16 bits of precision -- in large part due to the high energy efficiency and smaller bit storage associated with using reduced-precision representations. However, un…
The paper improves QD policy ensembles using distribution ratio estimators.
A new algorithm SRG-DQN reduces variance in deep Q-learning.
In this paper, finite type domains with hyperbolic orbit accumulation points are studied. We prove, in case of , it has to be a (global) pseudoconvex domain, after an assumption of boundary regularity. Moreover, one of the applications will realize the classification of domains within this class, precisel…
We propose Sideways, an approximate backpropagation scheme for training video models. In standard backpropagation, the gradients and activations at every computation step through the model are temporally synchronized. The forward activations need to be stored until the backward pass is executed, preventing inter-layer …
OptEx accelerates first-order optimization with parallelized iterations.
Optimizes neural networks' last layer with closed-form solutions.
This paper presents a numerical model to solve the problem of cash accumulation strategies for products with an unknown future price, like assets. Stock prices are modeled by a discretized Wiener Process, and by the means of ordinary integrals this Wiener Process will be exactly matched at a preset terminal time. Three…
Several statistical and machine learning methods are proposed to estimate the type and intensity of physical load and accumulated fatigue . They are based on the statistical analysis of accumulated and moving window data subsets with construction of a kurtosis-skewness diagram. This approach was applied to the data gat…
Stochastic gradient descent (SGD) with constant momentum and its variants such as Adam are the optimization algorithms of choice for training deep neural networks (DNNs). Since DNN training is incredibly computationally expensive, there is great interest in speeding up the convergence. Nesterov accelerated gradient (NA…
This paper deals with bandit online learning problems involving feedback of unknown delay that can emerge in multi-armed bandit (MAB) and bandit convex optimization (BCO) settings. MAB and BCO require only values of the objective function involved that become available through feedback, and are used to estimate the gra…