A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Transformers can approximate Newton's method for logistic regression.
In this article we introduce a generalization of the Newton transformation to the case of a system of endomorphisms. We show that it can be used in the context of extrinsic geometry of foliations and distributions yielding new integral formulas containing generalized extrinsic curvatures.
Derives integral formulae on weighted manifolds.
A new EM gradient algorithm for mixture models with skewed components.
We introduce a geometric framework to study Newton's equations on infinite-dimensional configuration spaces of diffeomorphisms and smooth probability densities. It turns out that several important PDEs of hydrodynamical origin can be described in this framework in a natural way. In particular, the Madelung transform be…
The paper generalizes optimization algorithms using category theory.
Nonnegative matrix factorization (NMF) is a popular method for audio spectral unmixing. While NMF is traditionally applied to off-the-shelf time-frequency representations based on the short-time Fourier or Cosine transforms, the ability to learn transforms from raw data attracts increasing attention. However, this adds…
Approximate Newton methods are a standard optimization tool which aim to maintain the benefits of Newton's method, such as a fast rate of convergence, whilst alleviating its drawbacks, such as computationally expensive calculation or estimation of the inverse Hessian. In this work we investigate approximate Newton meth…
Classification of cubics (that is, third order planar curves in the up to certain transformations is interested since Newton, and treated by several authors. We classify cubics up to affine transformations, in seven class, and give a complete set of representatives of the these classes. This result is complete an…
We revisit the geodesic approach to ideal hydrodynamics and present a related geometric framework for Newton's equations on groups of diffeomorphisms and spaces of probability densities. The latter setting is sufficiently general to include equations of compressible and incompressible fluid dynamics, magnetohydrodynami…
Sketching, a dimensionality reduction technique, has received much attention in the statistics community. In this paper, we study sketching in the context of Newton's method for solving finite-sum optimization problems in which the number of variables and data points are both large. We study two forms of sketching that…
The article proves integral formulas for foliated sub-Riemannian manifolds.
Novel approach simplifies VI problems with faster performance.
Bayesian deep learning avoids underfitting by projecting onto null space of generalized Gauss-Newton matrix.
The article derives integral formulas for foliated sub-Riemannian manifolds.
This work improves interpretability and calibration of complex-valued neural networks using Newton-Puiseux analysis.
This is a paper based on a talk given at the conference on Conformal Geometry which held at Roscoff in France in the 2008 summer. We study some aspects of the equation arising from the problem of the existence on a given closed Riemannian manifold of dimension at leat 4, of a conformal metric with constant curvat…
In this article, using the generalized Newton transformation, we define higher order mean curvatures of distributions of arbitrary codimension and we show that they agree with the ones from Brito and Naveira (Ann. Global Anal. Geom. 18, 371-383 (2000)). We also introduce higher order mean curvature vector fields and we…
We describe stochastic Newton and stochastic quasi-Newton approaches to efficiently solve large linear least-squares problems where the very large data sets present a significant computational burden (e.g., the size may exceed computer memory or data are collected in real-time). In our proposed framework, stochasticity…
We introduce a framework for Newton's flows in probability space with information metrics, named information Newton's flows. Here two information metrics are considered, including both the Fisher-Rao metric and the Wasserstein-2 metric. A known fact is that overdamped Langevin dynamics correspond to Wasserstein gradien…
Newton's method solves variational problems on manifolds.
Corrects bias in random sampling matrices for improved ML methods.
Muon with Newton-Schulz converges to the same stationary point as SVD-polar, up to a constant factor.
RNN operators solve Newton's equations with large timesteps for molecular dynamics.
We generalize Newton-type methods for minimizing smooth functions to handle a sum of two convex functions: a smooth function and a nonsmooth function with a simple proximal mapping. We show that the resulting proximal Newton-type methods inherit the desirable convergence behavior of Newton-type methods for minimizing s…
New algorithm improves convergence of gradient boosting trees.
Boosting algorithms are frequently used in applied data science and in research. To date, the distinction between boosting with either gradient descent or second-order Newton updates is often not made in both applied and methodological research, and it is thus implicitly assumed that the difference is irrelevant. The g…
Newton's method tackles nonlinear mappings into vector bundles with connections and retractions.
Study uses Newton polytopes to distinguish Lagrangian fillings of Legendrian submanifolds.
Let be a smooth closed hypersurface with non-negative Ricci curvature, isometrically immersed in a space form. It has been proved in \cite{P}, \cite{CZ}, and \cite{C2} that there are some inequalities on which measure the stability of closed umbilical hypersurfaces or more generally, closed hypersurfaces …
Unified approach to Bayesian inference with guarantees on covariance matrices.
Paper proposes an online covariance estimator for sketched Newton methods.
New Q-Newton's method avoids saddle points and converges quadratically.
AuON is a linear-time optimizer that improves upon Muon's performance without approximate orthogonal matrices.
We study deformations of Riemannian metrics on a given manifold equipped with a codimension-one foliation subject to quantities expressed in terms of its second fundamental form. We prove the local existence and uniqueness theorem and estimate the existence time of solutions for some particular cases. The key step of t…
A new optimization method improves deep learning accuracy without hyper-parameter tuning.
This thesis disentangles Gauss-Newton and variational approximations in Bayesian deep learning.
SVRN accelerates Newton methods by reducing variance and improving performance.
In this article we present a natural generalization of Newton's Second Law valid in field theory, i.e., when the parameterized curves are replaced by parameterized submanifolds of higher dimension. For it we introduce what we have called the geodesic -vector field, analogous to the ordinary geodesic field and which …
The second order method as Newton Step is a suitable technique in Online Learning to guarantee regret bound. The large data is a challenge in Newton method to store second order matrices as hessian. In this paper, we have proposed an modified online Newton step that store first and second order matrices of dimension m …
In this paper we study a geometric configuration of submanifolds of arbitrary codimension in an ambient Riemannian space. We obtain relations between the geometry of a q-codimension submanifold Mn along its boundary and the geometry of the boundary of Mn as an hypersuface of a q-codimensional submanifold Pn in an ambie…
Deep learning involves a difficult non-convex optimization problem, which is often solved by stochastic gradient (SG) methods. While SG is usually effective, it may not be robust in some situations. Recently, Newton methods have been investigated as an alternative optimization technique, but nearly all existing studies…
Study of measured laminations on surfaces using Newton polytopes and Poisson brackets.
We present two new remarkably simple stochastic second-order methods for minimizing the average of a very large number of sufficiently smooth and strongly convex functions. The first is a stochastic variant of Newton's method (SN), and the second is a stochastic variant of cubically regularized Newton's method (SCN). W…
Newton-LESS sparsifies Gaussian sketching for faster optimization.
New quasi-Newton method guarantees global superlinear convergence.
Paper develops a robust PP distributed quasi-Newton estimation for Byzantine machines.
Four decades after their invention, quasi-Newton methods are still state of the art in unconstrained numerical optimization. Although not usually interpreted thus, these are learning algorithms that fit a local quadratic approximation to the objective function. We show that many, including the most popular, quasi-Newto…