Paper develops RGN method for estimating low-rank tensors from noisy measurements.
problem Estimating low-rank tensors from noisy linear measurements.
method Riemannian Gauss-Newton (RGN) method for efficient low-rank tensor estimation.
result First local quadratic convergence guarantee of RGN for low-rank tensor estimation in noisy settings.
Mass calculation extended to manifolds with noncompact boundaries.
problem Computing mass in asymptotically flat and hyperbolic manifolds with boundaries.
method Coordinate-free approach using Einstein and Newton tensors.
result Mass can be computed as surface integrals involving these tensors.
Efficiently balances tensors to sum to one, significantly faster than previous methods.
problem Balancing tensors to sum to one for comparison in various applications.
method Newton's method applied to tensors modeled as probability distributions on statistical manifolds.
result The proposed algorithm is several orders of magnitude faster than existing methods.
Classifies connections on Galilei manifolds, generalizing known results.
problem Classifying general affine connections on Galilei manifolds.
method Classification through tensor fields, extending known Galilei connections.
result Additional freedom in connections not metric-compatible, linked to clock form and space metric.
Hidden symmetries of the Goryachev-Chaplygin and Kovalevskaya gyrostats spacetimes, as well as the Brdička-Eardley-Nappi-Witten pp-waves are studied. We find out that these spacetimes possess higher rank Stäckel-Killing tensors and that in the case of the pp-wave spacetimes the symmetry group of the Stäckel-Killing ten…
The article proves integral formulas for foliated sub-Riemannian manifolds.
problem Integral formulas for foliated sub-Riemannian manifolds.
method Proved a series of integral formulae involving mean curvatures, Newton transformations, and curvature tensor.
result Generalized known integral formulas for codimension-one foliations.
New tensor recovery method uses Riemannian optimization on Segre manifold.
problem Recovering low-rank tensors from noisy measurements.
method Riemannian Gradient Descent (RGD) and Riemannian Gauss-Newton (RGN) algorithms over the Segre manifold.
result Proven convergence rates for RGD and RGN under mild noise assumptions.
The article derives integral formulas for foliated sub-Riemannian manifolds.
problem Integrating geometric concepts in Riemannian manifolds with foliations.
method Deriving integral formulas involving shape operators and curvature tensor.
result Generalizes results for foliated Riemannian manifolds and includes arbitrary functions.
A new method for online tensor dictionary learning reduces complexity and improves robustness.
problem Efficiently learning tensor dictionaries online with reduced complexity and improved stability.
method Online tensor dictionary learning with separable dictionaries, tensor contraction, and stochastic gradient descent with memory.
result The method reduces computational complexity and improves robustness compared to existing methods.
New methods solve tensor-on-tensor regression with unknown rank, revealing benefits of over-parameterization.
problem Connecting tensor responses to tensor covariates with unknown intrinsic rank.
method Riemannian gradient descent and Riemannian Gauss-Newton methods for tensor-on-tensor regression.
result Riemannian optimization methods converge linearly and quadratically to a statistically optimal estimate in rank over-parameterized settings.
New algorithm APHEN improves tensor decomposition for mobile banking user-device authentication.
problem Enhancing user-device authentication in mobile banking for financial services.
method Tensor decomposition using Paratuck2 and APHEN algorithm for faster and more accurate computation.
result Improved user-device authentication for financial services through faster and more accurate tensor decomposition.
Improved tensor GLM estimation for complex data.
problem Complex tensor data in GLMs leads to high-dimensional, ill-posed estimation.
method Proposed LSRTR-M algorithm using Muon updates for faster convergence and lower errors.
result LSRTR-M converges faster and achieves lower errors than LSRTR.
Stochastic Newton and quasi-Newton methods solve large linear least-squares problems efficiently.
problem Efficiently solve large linear least-squares problems with limited computational resources.
method Introduce stochasticity in Newton and quasi-Newton approaches to handle large datasets.
result Stochastic Newton iterates may not converge to the least-squares solution.
A new method for optimization in probability space using Newton's flows.
problem Optimization in probability space with information metrics.
method Information Newton's flows, including Fisher-Rao and Wasserstein-2 metrics, with Newton's Langevin dynamics and variational methods.
result Effective numerical implementation and convergence results for the proposed method.
Newton's method solves variational problems on manifolds.
problem Solving variational equations on manifolds.
method Newton's method with affine covariant damping strategy.
result Numerical results for variational problems demonstrated.
This research compares gradient and Newton boosting methods in classification and regression.
problem The distinction between gradient descent and Newton updates in boosting algorithms is not well understood.
method Presented a unified framework for gradient and Newton boosting, and compared them with tree base learners.
result Newton boosting outperforms gradient and hybrid boosting in predictive accuracy on most datasets.
Muon with Newton-Schulz converges to the same stationary point as SVD-polar, up to a constant factor.
problem Improving the convergence rate of Muon optimizer.
method Using Newton-Schulz steps for momentum orthogonalization, proving convergence rate and constant factor.
result Muon with Newton-Schulz converges to the same stationary point as SVD-polar, up to a constant factor.
Newton methods improve CNN optimization, showing competitive accuracy.
problem Optimizing CNNs for deep learning applications.
method Detailed investigation and implementation of Newton methods for CNNs.
result Newton methods for CNNs achieve competitive test accuracy.
Simple stochastic Newton and cubic Newton methods with fast convergence.
problem Minimizing large numbers of smooth and strongly convex functions.
method Stochastic Newton and cubic Newton methods with simple local linear-quadratic rates.
result Local linear-quadratic convergence results with fast adaptation to problem's curvature.
Generalizes Newton's Second Law for field theory.
problem Applying Newton's Second Law to higher-dimensional parameterized submanifolds.
method Introducing geodesic k-vector field and deriving Hamilton's equations.
result Different forces can lead to the same Hamilton's equations.
Modified Newton step for online learning reduces matrix size for large datasets.
problem Handling large multi-class datasets efficiently in online learning.
method Element-wise multiplication to reduce matrix size of second order matrices.
result Proposed method achieves similar mistake rates to popular methods but with faster computations.
RNN operators solve Newton's equations with large timesteps for molecular dynamics.
problem Solving Newton's equations of motion with large timesteps for molecular dynamics simulations.
method Recurrent Neural Networks (RNN) operators to solve Newton's equations using past trajectory data.
result Significant speedup in molecular dynamics simulations with timesteps up to 4000 times larger.
New algorithm improves convergence of gradient boosting trees.
problem Global convergence of Newton boosting in tabular machine learning.
method Introduces Gradient Regularized Newton Descent for GBDTs, proving linear convergence for smooth, strongly convex losses and O(k21) rate for general convex losses. result Achieves globally convergent second-order GBDT algorithm with rate matching first-order boosting.
We generalize Newton-type methods for minimizing smooth functions to handle a sum of two convex functions: a smooth function and a nonsmooth function with a simple proximal mapping. We show that the resulting proximal Newton-type methods inherit the desirable convergence behavior of Newton-type methods for minimizing s…
Newton's method tackles nonlinear mappings into vector bundles with connections and retractions.
problem Finding zeros of mappings from a manifold into a vector bundle.
method Local convergence using differentiability concepts, Banach space Riemannian distance, and affine covariant damping strategy.
result Illustrated application to generalized non-symmetric eigenvalue problems.
Study uses Newton polytopes to distinguish Lagrangian fillings of Legendrian submanifolds.
problem Distinguishing Lagrangian fillings of Legendrian submanifolds.
method Utilizes Newton polytopes associated with augmented values of Reeb chords.
result Newton polytopes can distinguish infinitely many distinct Lagrangian fillings.
Unified approach to Bayesian inference with guarantees on covariance matrices.
problem Approximate Bayesian inference with PSD guarantees.
method Bayes-Newton methods extending Newton's method for optimisation.
result Novel algorithms with PSD covariance matrices.
Study on Newton-Sketch and Subsampled Newton methods for large-scale optimization.
problem Optimization of large-scale finite-sum problems with high-dimensional data.
method Hessian subsampling and randomized Hadamard transformations for dimensionality reduction in Newton's method.
result Advantages of conjugate gradient vs. stochastic gradient iterations revealed through experiments.
Paper proposes an online covariance estimator for sketched Newton methods.
problem Estimating the limiting covariance matrix of sketched Newton methods.
method Proposes a fully online covariance matrix estimator from Newton iterates.
result Establishes the consistency and convergence rate of the proposed estimator.
New Q-Newton's method avoids saddle points and converges quadratically.
problem Optimizing functions with saddle points and ensuring convergence guarantees.
method Modified New Q-Newton's method with Backtracking line search.
result Theorem for Morse functions: quadratic convergence to local minima.
The paper analyzes Newton-like and inexact Newton methods for stochastic optimization problems.
problem Optimization of stochastic problems with approximated gradients and Hessians.
method Newton-like methods using subsampled gradients and Hessians, and inexact Newton methods using conjugate gradient for Hessian approximation.
result Inexact Newton methods can achieve similar convergence rates to exact methods, with lower computational cost.
We provide integral formulae for the ADM mass of asymptotically flat hypersurfaces in Riemannian manifolds with a certain warped product structure in a neighborhood of infinity, thus extending Lam's recent results on Euclidean graphs to this broader context. As applications we exhibit, in any dimension, new classes of …
Improved quasi-Newton method for convex optimization with linear and accelerated convergence.
problem Efficiently solving composite optimization problems with strong convexity.
method Proximal quasi-Newton algorithm with accelerated variant.
result Acceleration may not improve convergence in quasi-Newton setting.
GIANT optimizes distributed computing by improving Newton method efficiency.
problem Efficiently solving empirical risk minimization problems in distributed environments.
method GIANT combines local ANT directions to form a GIANT direction, averaging communications and computations.
result GIANT achieves faster convergence compared to first-order and existing Newton-type methods.
A new optimization method improves deep learning accuracy without hyper-parameter tuning.
problem Computational demands and convergence behavior in deep learning training.
method Stochastic quasi-Gauss-Newton (SQGN) optimization method combining stochastic quasi-Newton, Gauss-Newton, and variance reduction.
result SQGN provides excellent accuracy without hyper-parameter experimentation, improving convergence and computational performance.
Proposes a Quasi-Newton trust region method for policy optimization in reinforcement learning.
problem Lack of stepsize selection criterion and slow convergence in gradient descent for policy optimization.
method Uses a trust region method with Quasi-Newton approximation for the Hessian.
result Demonstrates improved performance and efficiency in continuous control tasks.
This thesis disentangles Gauss-Newton and variational approximations in Bayesian deep learning.
problem Understanding the interplay between the Gauss-Newton method and variational approximations in Bayesian deep learning.
method Analysis of the Gauss-Newton method and Laplace/Gaussian variational approximations for neural networks.
result The combination of the Gauss-Newton method with approximate inference can be cast as inference in a linear or Gaussian process model.
Newton's method converges linearly for stable Hessians, even with approximations.
problem Finding global linear convergence for functions without strong convexity or Lipschitz gradients.
method Global linear convergence of Newton's method for stable Hessians, using approximate Hessians and subproblems.
result Global linear convergence rate for a broad class of functions, superior to first-order methods.
SVRN accelerates Newton methods by reducing variance and improving performance.
problem Improving the efficiency of Newton methods for large-scale optimization problems.
method Stochastic Variance-Reduced Newton (SVRN) algorithm that accelerates Subsampled Newton and Iterative Hessian Sketch algorithms.
result SVRN accelerates Newton methods by reducing the number of passes over the data, achieving a significant improvement in performance.
Study of measured laminations on surfaces using Newton polytopes and Poisson brackets.
problem Understanding the space of measured laminations on surfaces from a valuative perspective.
method Introducing Newton polytopes for character variety functions, defining tangent spaces, and identifying symplectic structures.
result Trace functions have unit coefficients at the extremal points of their Newton polytopes.
The paper determines the bifurcation set of a real polynomial function of two variables using Newton polygons.
problem Determining the bifurcation set of a real polynomial function of two variables.
method Using toric compactification and toric modifications to count singular phenomena at infinity.
result An upper bound of the number of elements in the bifurcation set is given in terms of its Newton polygon.
Approximate Newton methods are a standard optimization tool which aim to maintain the benefits of Newton's method, such as a fast rate of convergence, whilst alleviating its drawbacks, such as computationally expensive calculation or estimation of the inverse Hessian. In this work we investigate approximate Newton meth…
Newton-LESS sparsifies Gaussian sketching for faster optimization.
problem Computing the Hessian matrix in optimization is computationally expensive.
method Uses a sparsified version of a dense Gaussian sketching matrix.
result Achieves nearly the same convergence rate as dense Gaussian embeddings without the computational cost.
New quasi-Newton method guarantees global superlinear convergence.
problem Global convergence and superlinear convergence of quasi-Newton methods.
method Hybrid proximal extragradient method with online learning for Hessian approximation.
result First globally convergent quasi-Newton method with explicit superlinear convergence rate.
Stochastic quasi-Newton tackles noisy gradients in optimization.
problem Optimizing with noisy data in stochastic settings.
method Extends quasi-Newton methods to handle stochastic gradients through flexible Hessian modeling and line-search regularization.
result Demonstrates superior performance in maximum likelihood estimation for complex models.
Paper develops a robust PP distributed quasi-Newton estimation for Byzantine machines.
problem Byzantine machines in distributed computing under Privacy Protection constraints.
method Robust PP distributed quasi-Newton estimation method that transmits only five vectors.
result Reduces privacy budgeting and transmission cost compared to gradient descent and Newton iteration.
Efficient methods for training deep neural networks using subsampled Gauss-Newton and natural gradient.
problem Training deep neural networks with large datasets and variables.
method Subsampled Gauss-Newton and natural gradient methods with subsampled gradient estimates.
result Methods converge to a stationary point and are efficient to implement.
A new quasi-Newton method uses cubic regularization to avoid saddle points in deep learning.
problem Avoiding saddle points and poor local minima in deep learning models.
method Limited-memory symmetric rank-one quasi-Newton approach with adaptive regularized cubics.
result The method effectively avoids saddle points and converges to better local minima.