Geometrically reformulates wave equation solving method.
problem Solving tensorial wave equations in spacetimes.
method Geometric formulation of the method of descent.
result Representation formula for tensorial wave equation.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,341 papers · 148 categories
Trend · papers per month
48 results for “geometric descent”
Extends geometric descent method for convex composite problems.
problem Nonsmooth and strongly convex composite problems.
method Geometric Proximal Gradient Method (GeoPG)
result Achieves optimal linear convergence rate of (1-1/\sqrtκ).
SGD and stochastic gradient descent converge at optimal rates for certain non-convex functions.
problem Optimal convergence rates for non-convex functions under gradient noise.
method Geometric interpretation of the PL-condition to analyze convergence rates.
result Convergence rates of SGD and stochastic gradient descent match those of strongly convex quadratics.
GeoAdaLer enhances geometric understanding of Adam for stochastic optimization.
problem Understanding geometric principles behind Adam's success in stochastic optimization.
method Introduces GeoAdaLer, an adaptive learning method based on geometric properties.
result Extends interpretability and effectiveness in complex optimization scenarios.
Researchers interpret SGD using diffusion metrics for clearer geometric understanding.
problem Elusiveness of geometrical significance in stochastic gradient descent.
method Study a deterministic model with geodesics of diffusion metrics.
result Establishes parallel with General Relativity models.
Study shows how steepest descent algorithms' geometric margin increases during training.
problem Understanding implicit bias in steepest descent algorithms for neural networks.
method Analysis of steepest descent algorithms with infinitesimal learning rates in homogeneous neural networks.
result Limit points of training trajectories correspond to KKT points of margin-maximization problems.
Paper shows how to use geometric median for robust SGD in high dimensions.
problem Robustifying SGD for high-dimensional optimization problems with gross corruption.
method Applying geometric median to only chosen blocks of coordinates at a time.
result Retains optimal breakdown point of 0.5 for smooth non-convex problems.
Yau's Affine Normal Descent optimizes smooth unconstrained problems with geometrically adapted directions.
problem Optimizing smooth unconstrained problems with geometrically adapted directions.
method Yau's Affine Normal Descent (YAND) uses the equi-affine normal of level-set hypersurfaces as search directions.
result YAND converges globally under standard smoothness assumptions and locally quadratically near nondegenerate minimizers.
Geometric analysis shows gradient descent in linear neural nets converges to global minima.
problem Analyzing convergence of gradient descent in linear neural networks.
method Geometric framework and invariance property of network structure.
result Gradient descent trajectories converge to global minima for linear neural nets.
Geometric framework simplifies CNNs using inner product spaces.
problem Complexity and inefficiency in training CNNs.
method Introduces a geometric approach to CNNs using inner product spaces.
result Gradient calculations simplified for CNN layers, higher-order losses.
Gradient descent converges to perfect classification in neural nets for non-separable data.
problem Classifying linearly non-separable data using neural networks.
method Analysis of gradient descent dynamics in neural networks with sufficient but not large number of neurons.
result Gradient descent converges to global minima with perfect classification in the landscape of minimization problems.
Geometric Occam's Razor shapes deep learning solutions.
problem Understanding the regularization in over-parameterized neural networks.
method Analyzing the geometric model complexity and Dirichlet energy in neural networks.
result Over-parameterized neural networks are implicitly regularized by geometric model complexity.
Gradient descent converges geometrically to optimal self-attention parameters.
problem Training softmax self-attention layers for linear regression.
method Structure-aware gradient descent with preconditioner and regularizer.
result Gradient descent converges geometrically to global minima.
Gradient descent can solve shallow neural network training problems globally.
problem Training shallow linear neural networks.
method Analyzed the geometric properties of the loss landscape.
result Gradient descent can converge globally due to the landscape's properties.
Gradient descent finds better constellations for GMI-based learning.
problem Nonconvexity in end-to-end GMI learning.
method Gradient descent initialized with Gray-labeled APSK constellations.
result State-of-the-art constellations in 2D and 4D provide up to 26% reach increase.
Establishes geometric convergence of iterative optimization algorithms.
problem Analyzes convergence of iterative optimization algorithms under general assumptions.
method General framework for iterative optimization algorithms, proving asymptotic geometric convergence and providing convergence rates.
result Asymptotic geometric convergence of iterative optimization algorithms with exact rate.
The paper improves Kaczmarz algorithm with momentum for linear least squares.
problem Improving convergence of the Kaczmarz algorithm for linear least squares.
method Integrates geometrically smoothed momentum into the randomized Kaczmarz algorithm.
result Proves expected error reduction in singular vector directions.
A new method for manifold learning uses gradient descent in embedding space with geometric constraints.
problem Learning manifolds in high-dimensional spaces with geometric constraints.
method Discretized gradient flow in the space of embeddings with geometric step length bounds.
result Explicit lower bound for step length in embedding space, applicable to manifold learning.
Gradient descent struggles to achieve zero loss in deep learning models due to non-generic data distributions.
problem Achieving zero loss minimizers in deep learning networks.
method Analysis of gradient descent algorithm in deep learning, focusing on underparametrized networks.
result Zero loss minimization cannot be achieved generically in deep learning networks.
We improve SGD convergence on manifolds using averaging.
problem Minimizing functions on Riemannian manifolds with noisy gradients.
method Developed a geometric framework to transform SGD iterates into an averaged sequence with robust and fast convergence.
result Averaged SGD iterates converge at rate, improving on slow convergence of SGD.
Stochastic approximation algorithms show exponential progress bounds.
problem Analyzing the convergence of stochastic approximation algorithms.
method Developed geometric ergodicity proofs to establish exponential concentration bounds.
result Proved faster convergence rates for specific algorithms.
On the geometric and Riemannian structure of the spaces of group equivariant non-expansive operatorsmath.DG
The paper studies geometric properties of group equivariant operators and their Riemannian structure.
problem Understanding the geometric structure of group equivariant operators.
method Endowing the space of group equivariant non-expansive operators with a Riemannian manifold structure and using gradient descent methods.
result Gradient descent methods can be applied to minimize cost functions on the space of group equivariant non-expansive operators.
Randomized algorithms that base iteration-level decisions on samples from some pool are ubiquitous in machine learning and optimization. Examples include stochastic gradient descent and randomized coordinate descent. This paper makes progress at theoretically evaluating the difference in performance between sampling wi…
Two new algorithms solve high-dimensional optimization problems without gradients.
problem Optimizing complex, high-dimensional functions without gradient information.
method GradientLess Descent (GLD) algorithms that use evaluations at adaptively chosen inputs.
result Converges within an ε-ball of the optimum with a number of evaluations that is poly-logarithmic in dimensionality.
This paper uses a geometric approach to understand how normalization layers affect neural network optimization.
problem Understanding the effect of normalization layers on optimization in neural networks.
method Introduces a spherical framework to study optimization dynamics of neural networks with normalization layers from a geometric perspective.
result Derives the first effective learning rate expression of Adam and shows that SGD with NLs is equivalent to a constrained variant of Adam.
The paper studies matrix normalization and graph balancing using a new functional and gradient descent.
problem Matrix normalization and graph balancing.
method A new functional called the non-normal energy, and gradient descent.
result Gradient descent of the non-normal energy converges to balanced graphs and preserves spectra and realness of weights.
Gradient descent with geometrically adapted metrics drives cost to global minimum at uniform rate.
problem Minimizing cost in deep learning networks.
method Adapting gradient descent to output layer metric in deep learning.
result Uniform exponential convergence to global minimum in cost.
Stochastic gradient descent's long-term fluctuations are described by a diffusion limit.
problem Long-term behavior of stochastic gradient descent in non-smooth settings.
method Functional central limit theorem applied to rescaled trajectory of SGD.
result Characterization of long-term fluctuations around the minimizer.
This paper analyzes Stein variational gradient descent for Bayesian inference.
problem Sampling or approximating high-dimensional probability distributions.
method Iterated steepest descent steps with a reproducing kernel Hilbert space norm.
result Performance gains of certain nondifferentiable kernels with adjusted tails.
Unified signSGD and gradient descent analysis for neural networks.
problem Performance of sign-based optimization methods in neural networks.
method Unified analysis of separable smoothness and -smoothness, isolating geometric properties affecting performance.
result Sign-based methods are preferable over gradient descent under specific Hessian properties in deep networks.
Information geometry applies concepts in differential geometry to probability and statistics and is especially useful for parameter estimation in exponential families where parameters are known to lie on a Riemannian manifold. Connections between the geometric properties of the induced manifold and statistical properti…
Enhances SVGD with matrix-valued kernels for faster inference.
problem Efficient approximate inference in complex probability landscapes.
method Integrates geometric information through matrix-valued kernels in SVGD.
result Significant improvement in real-world Bayesian inference tasks.
Byrd-SAGA reduces variance to robustify SGD against Byzantine attacks.
problem Learning over networks with malicious Byzantine attacks.
method Byrd-SAGA uses geometric median for robust aggregation of corrected stochastic gradients.
result Byrd-SAGA achieves provably linear convergence to optimal solution in the presence of Byzantine workers.
New geometric interpretation explains over-parameterized models and adversarial perturbations.
problem Geometric understanding of over-parameterized regression and adversarial perturbations.
method Alternative geometric interpretation of regression in feature space.
result Adversarial perturbations are a natural feature of biased models due to underlying geometry.
Study optimization landscapes for overcomplete representations, showing benign geometric structures.
problem Optimizing overcomplete representations in high-dimensional data analysis.
method Formulate as -norm optimization problems with spherical constraint, analyze geometric properties.
result Nonconvex objectives have benign geometric structures, ensuring local search algorithms find target solutions.
Researchers analyze SGD dynamics using von Mises-Fisher distributions.
problem Understanding the dynamics of stochastic gradient descent in high-dimensional spaces.
method Geometric analysis of minibatch gradient norms and directions through von Mises-Fisher distribution.
result Directional uniformity of minibatch gradients increases over SGD iterations.
Paper analyzes convergence of OMD algorithms with geometric conditions.
problem Analyzing convergence of online mirror descent algorithms.
method Presented necessary and sufficient conditions for convergence of OMD with step size sequences.
result Established conditions for convergence and linear convergence under specific variances.
Review of non-abelian gerbes and their applications in string theory.
problem Anomaly cancellation and geometric description of T-duals in string theory.
method Systematic construction of non-abelian gerbes via descent.
result Extension of non-abelian gerbes to orientifold sigma models and T-duals.
AGNES accelerates gradient descent with noisy gradients.
problem Minimizing smooth convex and strongly convex functions with noisy gradients.
method Generalization of Nesterov's accelerated gradient descent algorithm for noisy conditions.
result AGNES achieves acceleration for noisy gradients with a constant of proportionality up to 1.
New algorithm converges geometrically fast with different step-sizes.
problem Distributed optimization with varying step-sizes.
method Adapt-Then-Combine (ATC) variation of DIGing algorithm.
result Geometric convergence with uncoordinated step-sizes.
SGD converges with perturbed forward-backward passes, explained by geometric amplification.
problem Analyzing convergence of SGD with perturbed forward-backward passes in composite optimization.
method Characterized propagation and amplification of perturbations, derived convergence guarantees for non-convex and PL objectives.
result Perturbations cascade through the computational graph, affecting convergence order under specific conditions.
This paper details a series of experiments in searching for minimal energy configurations for knots and links using the computer program KnotPlot. The most interesting phenomena found in these experiments is the dependence of the trajectories of energy descent upon the initial geometric conditions of the knotted embedd…
Mirror Descent Strikes Again: Optimal Stochastic Convex Optimization under Infinite Noise Variancestat.ML
Study shows Stochastic Mirror Descent optimizes convex problems with infinite noise variance.
problem Optimizing convex problems with infinite noise variance.
method Stochastic Mirror Descent algorithm with uniformly convex mirror maps.
result Demonstrates convergence rate quantified in terms of iterations, dimensionality, and geometric parameters.
Minimization of Functions on Dually Flat Spaces Using Geodesic Descent Based on Dual Connectionsstat.CO
Geodesic descent optimizes likelihood in dually flat spaces.
problem Maximum likelihood estimation in exponential families.
method m-geodesic and e-geodesic updates on dually flat spaces.
result Geodesic updates can reach maximum likelihood estimator in one step.
Hermitian bundle gerbes with connection are geometric objects for which a notion of surface holonomy can be defined for closed oriented surfaces. We systematically introduce bundle gerbes by closing the pre-stack of trivial bundle gerbes under descent. Inspired by structures arising in a representation theoretic approa…
Fast algorithm for online optimization on transport polytopes.
problem Optimizing convex objectives on transport polytopes.
method Mirror Sinkhorn algorithm combining Sinkhorn scaling and mirror descent.
result Robust and efficient online optimization for convex objectives.
Quantum Natural Gradientquant-ph
Quantum Natural Gradient uses quantum geometry for optimization.
problem Optimizing variational quantum circuits efficiently.
method Quantum generalization of Natural Gradient Descent using Quantum Information Geometry.
result Efficient algorithm for computing metric tensor approximations.
Gradient descent with random weights in linear regression analyzed for various noise types.
problem Analyzing the impact of random noise on gradient descent in linear regression.
method Gradient descent with randomly weighted data points, various weighting distributions, geometric moment contraction.
result Characterization of implicit regularization and non-asymptotic convergence bounds.