A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
New algorithm improves convergence of gradient boosting trees.
This paper investigates the classical and quantum elementary systems with Newton-Hoooke symmetry. A complete classification is given by explicit computation. In addition, we present an application example of quantization using the Moyal scheme.
A parallel optimization method for convex functions using Hessian sketching and debiasing.
Unified approach to Bayesian inference with guarantees on covariance matrices.
New averaging technique speeds up Newton method convergence.
Proves heat expansion for Laplacian on a singularity.
Paper proposes a new method to efficiently incorporate curvature information in stochastic optimization.
Recurrent Neural Networks (RNNs) are powerful models that achieve exceptional performance on several pattern recognition problems. However, the training of RNNs is a computationally difficult task owing to the well-known "vanishing/exploding" gradient problem. Algorithms proposed for training RNNs either exploit no (or…
In this paper, we develop a variant of the well-known Gauss-Newton (GN) method to solve a class of nonconvex optimization problems involving low-rank matrix variables. As opposed to the standard GN method, our algorithm allows one to handle general smooth convex objective function. We show, under mild conditions, that …
New method solves constrained self-concordant minimization problems efficiently.
Paper proposes a faster federated learning method for logistic regression.
We study the smooth structure of convex functions by generalizing a powerful concept so-called self-concordance introduced by Nesterov and Nemirovskii in the early 1990s to a broader class of convex functions, which we call generalized self-concordant functions. This notion allows us to develop a unified framework for …
In this paper, a novel stochastic extra-step quasi-Newton method is developed to solve a class of nonsmooth nonconvex composite optimization problems. We assume that the gradient of the smooth part of the objective function can only be approximated by stochastic oracles. The proposed method combines general stochastic …
This paper extends Newton's method to distributed learning, avoiding saddle points and handling Byzantine workers.
NS-RGS improves orthogonal group synchronization with faster convergence.
Study uses FEM for HJB in option pricing with borrowing fees, improving accuracy and efficiency.
Regularized nonlinear acceleration (RNA) estimates the minimum of a function by post-processing iterates from an algorithm such as the gradient method. It can be seen as a regularized version of Anderson acceleration, a classical acceleration scheme from numerical analysis. The new scheme provably improves the rate of …
New method achieves superlinear convergence rate with limited memory.
New algorithmic view of ℓ2 regularization using ODEs and path-following methods.
Abstract: Linking field theory to Floer theory via regularization.
We present a novel statistical inference framework for convex empirical risk minimization, using approximate stochastic Newton steps. The proposed algorithm is based on the notion of finite differences and allows the approximation of a Hessian-vector product from first-order information. In theory, our method efficient…
Consider a process, stochastic or deterministic, obtained by using a numerical integration scheme, or from Monte-Carlo methods involving an approximation to an integral, or a Newton-Raphson iteration to approximate the root of an equation. We will assume that we can sample from the distribution of the process from time…
In this paper, we study large-scale convex optimization algorithms based on the Newton method applied to regularized generalized self-concordant losses, which include logistic regression and softmax regression. We first prove that our new simple scheme based on a sequence of problems with decreasing regularization para…
Many machine learning techniques sacrifice convenient computational structures to gain estimation robustness and modeling flexibility. However, by exploring the modeling structures, we find these "sacrifices" do not always require more computational efforts. To shed light on such a "free-lunch" phenomenon, we study the…
We consider the class of convex minimization problems, composed of a self-concordant function, such as the metric, a convex data fidelity term and, a regularizing -- possibly non-smooth -- function . This type of problems have recently attracted a great deal of interest, mainly due to th…
A new method for optimization in probability space using Newton's flows.
We describe stochastic Newton and stochastic quasi-Newton approaches to efficiently solve large linear least-squares problems where the very large data sets present a significant computational burden (e.g., the size may exceed computer memory or data are collected in real-time). In our proposed framework, stochasticity…
Many of the algorithms used to solve minimization problems with sparsity-inducing regularizers are generic in the sense that they do not take into account the sparsity of the solution in any particular way. However, algorithms known as semismooth Newton are able to take advantage of this sparsity to accelerate their co…
Newton's method solves variational problems on manifolds.
Muon with Newton-Schulz converges to the same stationary point as SVD-polar, up to a constant factor.
Momentum is a popular technique to accelerate the convergence in practical training, and its impact on convergence guarantee has been well-studied for first-order algorithms. However, such a successful acceleration technique has not yet been proposed for second-order algorithms in nonconvex optimization.In this paper, …
Simple stochastic Newton and cubic Newton methods with fast convergence.
RNN operators solve Newton's equations with large timesteps for molecular dynamics.
We generalize Newton-type methods for minimizing smooth functions to handle a sum of two convex functions: a smooth function and a nonsmooth function with a simple proximal mapping. We show that the resulting proximal Newton-type methods inherit the desirable convergence behavior of Newton-type methods for minimizing s…
Boosting algorithms are frequently used in applied data science and in research. To date, the distinction between boosting with either gradient descent or second-order Newton updates is often not made in both applied and methodological research, and it is thus implicitly assumed that the difference is irrelevant. The g…
Newton's method tackles nonlinear mappings into vector bundles with connections and retractions.
Paper improves a method for fast global and local convergence in optimization.
Study uses Newton polytopes to distinguish Lagrangian fillings of Legendrian submanifolds.
In this paper, we discuss the problem of minimizing the sum of two convex functions: a smooth function plus a non-smooth function. Further, the smooth part can be expressed by the average of a large number of smooth component functions, and the non-smooth part is equipped with a simple proximal mapping. We propose a pr…
Paper proposes an online covariance estimator for sketched Newton methods.
New Q-Newton's method avoids saddle points and converges quadratically.
A new optimization method improves deep learning accuracy without hyper-parameter tuning.
Proposes a Quasi-Newton trust region method for policy optimization in reinforcement learning.
ADCMs adaptively discretize CMs for efficient training.
This thesis disentangles Gauss-Newton and variational approximations in Bayesian deep learning.
We present a numerical approach for solving the free boundary problem for the Black-Scholes equation for pricing American style of floating strike Asian options. A fixed domain transformation of the free boundary problem into a parabolic equation defined on a fixed spatial domain is performed. As a result a nonlinear t…
SVRN accelerates Newton methods by reducing variance and improving performance.
In this article we present a natural generalization of Newton's Second Law valid in field theory, i.e., when the parameterized curves are replaced by parameterized submanifolds of higher dimension. For it we introduce what we have called the geodesic -vector field, analogous to the ordinary geodesic field and which …