A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
New SAGA algorithm with decreasing step for stochastic optimization.
The Expectation-Maximization (EM) algorithm for mixture models often results in slow or invalid convergence. The popular convergence proof affirms that the likelihood increases with Q; Q is increasing in the M -step and non-decreasing in the E-step. The author found that (1) Q may and should decrease in some E-steps; (…
Applying standard Markov chain Monte Carlo (MCMC) algorithms to large data sets is computationally infeasible. The recently proposed stochastic gradient Langevin dynamics (SGLD) method circumvents this problem in three ways: it generates proposed moves using only a subset of the data, it skips the Metropolis-Hastings a…
Paper improves CLT and bootstrap approximations for LSA with decreasing step size.
GradaGrad adapts learning rate non-monotonically, overcoming AdaGrad's step size decrease.
Avare improves optimization and sampling with adaptive importance sampling.
In an adaptive population which models financial markets and distributed control, we consider how the dynamics depends on the diversity of the agents' initial preferences of strategies. When the diversity decreases, more agents tend to adapt their strategies together. This change in the environment results in dynamical…
GD monotonically decreases GFS sharpness in neural networks and scalar models.
The CSA-ES is an Evolution Strategy with Cumulative Step size Adaptation, where the step size is adapted measuring the length of a so-called cumulative path. The cumulative path is a combination of the previous steps realized by the algorithm, where the importance of each step decreases with time. This article studies …
Applying standard Markov chain Monte Carlo (MCMC) algorithms to large data sets is computationally expensive. Both the calculation of the acceptance probability and the creation of informed proposals usually require an iteration through the whole data set. The recently proposed stochastic gradient Langevin dynamics (SG…
TREGO improves EGO for global optimization of high-dimensional problems.
DRAG decreases regularization to accelerate semi-discrete OT convergence.
SALSA automatically adjusts learning rates in stochastic gradient methods.
D2SRM solves complex PDEs using deep learning.
Paper tackles catastrophic overfitting in single-step adversarial training.
Q-learning is a simple and powerful tool in solving dynamic problems where environments are unknown. It uses a balance of exploration and exploitation to find an optimal solution to the problem. In this paper, we propose using four basic emotions: joy, sadness, fear, and anger to influence a Qlearning agent. Simulation…
Gradient descent on neural nets often operates at the Edge of Stability, where loss behavior is complex but loss decreases over time.
The Frank-Wolfe (FW) algorithm has been widely used in solving nuclear norm constrained problems, since it does not require projections. However, FW often yields high rank intermediate iterates, which can be very expensive in time and space costs for large problems. To address this issue, we propose a rank-drop method …
Characterizes corridors in loss surfaces for gradient-based optimization.
Adaptive step-size improves optimization in complex geometries.
Gradually Truncated Log-normal distribution - Size distribution of firms Abstract Many natural and economical phenomena are described through power law or log- normal distributions. In these cases, probability decreases very slowly with step size compared to normal distribution. Thus it is essential to cut-off these di…
The recently announced Energy Union by the European Commission is the most recent step in a series of developments aiming at integrating the EU's gas markets to increase social welfare (SW) and security of gas supply. Based on a spatial partial equilibrium model, we analyze the changes in consumption, prices, and SW up…
ReDi improves few-step generation for discrete data models.
We describe two techniques that significantly improve the running time of several standard machine-learning algorithms when data is sparse. The first technique is an algorithm that effeciently extracts one-way and two-way counts--either real or expected-- from discrete data. Extracting such counts is a fundamental step…
Bayes optimal algorithm under certain conditions doesn't achieve exponential simple regret.
Paper analyzes convergence rates for multi-agent learning in games.
The paper analyzes bank decisions in a three-step model, focusing on equity and debt raising.
Improved averaging method for noisy observations converges strongly.
Recent advances in Bayesian learning with large-scale data have witnessed emergence of stochastic gradient MCMC algorithms (SG-MCMC), such as stochastic gradient Langevin dynamics (SGLD), stochastic gradient Hamiltonian MCMC (SGHMC), and the stochastic gradient thermostat. While finite-time convergence properties of th…
Paper develops Gaussian approximations and bootstrap for federated LSA with trade-off bounds.
ContinuousNet generalizes ResNets to continuous dynamical systems.
LSD distills high-quality samplers for DDMs with fewer steps.
A new simulation method for Volterra processes improves convergence for rough kernels.
Generative adversarial networks (GANs) are a family of generative models that do not minimize a single training criterion. Unlike other generative models, the data distribution is learned via a game between a generator (the generative model) and a discriminator (a teacher providing training signal) that each minimize t…
Paper proposes a new method to detect convergence in SGD.
Large GD stepsizes improve margins and speed up training for non-homogeneous networks.
This paper establishes risk convergence and asymptotic weight matrix alignment --- a form of implicit regularization --- of gradient flow and gradient descent when applied to deep linear networks on linearly separable data. In more detail, for gradient flow applied to strictly decreasing loss functions (with similar re…
Deep learning has achieved impressive prediction performance in the field of sequence learning recently. Dissolved oxygen prediction, as a kind of time-series forecasting, is suitable for this technique. Although many researchers have developed hybrid models or variant models based on deep learning techniques, there is…
Decentralized Bayesian learning reduces KL-divergence exponentially.
In this paper, we propose an implicit gradient descent algorithm for the classic -means problem. The implicit gradient step or backward Euler is solved via stochastic fixed-point iteration, in which we randomly sample a mini-batch gradient in every iteration. It is the average of the fixed-point trajectory that is c…
New convergence results for NGVI with various step sizes and sample sizes.
Deep networks run with low precision operations at inference time offer power and space advantages over high precision alternatives, but need to overcome the challenge of maintaining high accuracy as precision decreases. Here, we present a method for training such networks, Learned Step Size Quantization, that achieves…
The paper decouples shrinkage and selection in Bayesian Quantile Regression.
Model studies money exchange stability in social networks.
A new algorithm solves constrained optimization problems with stochastic gradients.
Recently it has been shown that the step sizes of a family of variance reduced gradient methods called the JacSketch methods depend on the expected smoothness constant. In particular, if this expected smoothness constant could be calculated a priori, then one could safely set much larger step sizes which would result i…
Recent studies have explored the use of deep generative models of speech spectra based of variational autoencoders (VAEs), combined with unsupervised noise models, to perform speech enhancement. These studies developed iterative algorithms involving either Gibbs sampling or gradient descent at each step, making them co…
When learning behavior, training data is often generated by the learner itself; this can result in unstable training dynamics, and this problem has particularly important applications in safety-sensitive real-world control tasks such as robotics. In this work, we propose a principled and model-agnostic approach to miti…