The study examines polynomial growth functions on gradient shrinking Ricci solitons.
problem Characterizing harmonic and caloric functions with polynomial growth on gradient shrinking Ricci solitons.
method Analysis of polynomial growth functions under different curvature conditions.
result Finite dimensional estimates for harmonic and caloric functions with polynomial growth.
Study on polynomial growth functions and forms on gradient Ricci solitons.
problem Estimating dimensions of polynomial growth holomorphic functions and forms.
method Relating to spectral data of the f-Laplacian, proving estimates under curvature assumptions. result Sharp dimension estimates and almost sharp frequency estimates for polynomial growth holomorphic functions.
Piecewise polynomial interpolation-based gradient descent reduces oracle complexity for smooth loss functions.
problem Optimizing empirical risk minimization loss functions
method Piecewise polynomial interpolation-based gradient descent
result Oracle complexity is reduced for smooth loss functions
Characterizes values at infinity for real polynomial maps with 2D fibers.
problem Understanding atypical values at infinity for real polynomial maps.
method Characterization using indices of gradient vector fields on spheres.
result Analogous to two-variable case, but for maps with 2D fibers.
Develops methods to calculate global index of real polynomials.
problem Calculating the global index of real polynomials.
method Two methods: via atypical fibres and Milnor arcs clusters.
result Derives upper bounds for the global index, refining Durfee's degree-based bound.
Classifies polynomial growth solutions to drift-harmonic equations on asymptotically paraboloidal manifolds.
problem Classifying polynomial growth solutions to drift-harmonic equations on specific types of manifolds.
method Inductive argument that alternates between constructing and asymptotically controlling drift-harmonic functions.
result All drift-harmonic functions with polynomial growth asymptotically separate variables and dimensions of spaces are computed.
In this note, we study properties of the gradient map of the isoparametric polynomial. For a given isoparametric hypersurface in sphere, we calculate explicitly the gradient map of its isoparametric polynomial which turns out many interesting phenomenons and applications. We find that it should map not only the focal s…
New method shows AdaGrad converges globally to neural network minima.
problem Convergence of adaptive gradient methods for neural networks.
method Proposed adaptive gradient method for over-parameterized neural networks.
result Converges to global minimum in polynomial time for two-layer networks.
Abstract reviews algorithms for multi-index models, focusing on polynomial-time methods and their limitations.
problem Estimating the index space in multi-index models efficiently and accurately.
method Polynomial-time algorithms in Gaussian space, nonparametric gradient estimation, and neural network fitting.
result A gap exists between computationally efficient methods and information-theoretical minimum.
Wide networks with polynomial activations have proven asymptotic behavior.
problem Understanding the behavior of neural networks in the large width limit.
method Proving a conjecture for deep networks with polynomial activation functions.
result Tight bounds on the behavior of wide networks during stochastic gradient descent and derivation of their finite-width dynamics.
Accelerates ERM problems with LPI-GD and improved oracle complexity.
problem Empirical Risk Minimization (ERM) problems with strong convexity and smoothness.
method Local Polynomial Interpolation-based Gradient Descent (LPI-GD) and accelerated methods.
result Oracle complexity improved to $ ilde{O}\left(\sqrtσ m^d \log(1/\varepsilon)
ight)$.
Gradient descent efficiently finds global minima in deep neural networks.
problem Training deep neural networks efficiently and reliably.
method Gradient descent, leveraging the stability of the Gram matrix induced by the network architecture.
result Gradient descent achieves zero training loss in polynomial time for deep over-parameterized neural networks with residual connections.
GD outperforms ridge regression and SGD in linear regression problems.
problem Comparing the risks of GD, ridge regression, and SGD in linear regression problems.
method Instance-wise finite-sample risk analysis of GD, ridge regression, and SGD.
result GD outperforms ridge regression and is incomparable with SGD in some cases.
This study analyzes adversarial training on linearly separable data and finds that gradient updates can achieve large margins in polynomial iterations.
problem Ensuring robustness in machine learning models trained on linearly separable data.
method Analysis of adversarial training with gradient updates on linearly separable data.
result Gradient updates in adversarial training can achieve large margins in polynomial iterations, whereas non-smooth methods require exponentially many iterations.
Simple gradient descent algorithm escapes saddle points efficiently.
problem Escaping saddle points in nonconvex optimization.
method Gradient-based algorithm with polynomial iterations.
result Outputs ε-approximate second-order stationary points efficiently.
Gradient descent learns over-param neural nets better than NTK.
problem Learning over-parametrized neural networks with ReLU activations.
method Gradient descent from random initialization on a Gaussian input distribution.
result Gradient descent achieves population loss o(1/d), while NTK achieves Ω(1/d). Fine-grained analysis of gradient descent with momentum provides modified loss equations.
problem Understanding the dynamics of gradient descent with momentum.
method Fine-grained analysis and derivation of modified loss equations.
result Global approximation bounds and continuous modified equations for HB.
New algorithm improves gradient-based ERM for smooth convex losses.
problem Empirical risk minimization of smooth, strongly convex loss functions.
method Iterative gradient-based method with local polynomial regression.
result Oracle complexity of O((pε−1)d/(2η)) for our algorithm. The study bounds dimensions and proves existence of holomorphic sections on Kähler Ricci shrinkers.
problem Estimating dimensions and existence of holomorphic sections with polynomial growth on Kähler Ricci shrinkers.
method Proved upper bounds for dimensions and existence of sections using polynomial growth.
result Upper bounds for dimensions and existence of holomorphic sections with polynomial growth on Kähler Ricci shrinkers.
Houdini finds high-dimensional saddle points under few constraints.
problem Escaping from saddle points in high-dimensional spaces with constraints.
method Gradient descent methods under logarithmic inequality constraints.
result Polynomial time algorithms for escaping saddle points under constraints.
We prove that stochastic gradient descent efficiently converges to the global optimizer of the maximum likelihood objective of an unknown linear time-invariant dynamical system from a sequence of noisy observations generated by the system. Even though the objective function is non-convex, we provide polynomial running …
We prove that the evolution of weight vectors in online gradient descent can encode arbitrary polynomial-space computations, even in very simple learning settings. Our results imply that, under weak complexity-theoretic assumptions, it is impossible to reason efficiently about the fine-grained behavior of online gradie…
Study of two-layer NNs under Gaussian mixtures data, proving polynomial models equivalent to neural networks.
problem Training and generalization performance of two-layer NNs under structured Gaussian mixture data.
method Asymptotic analysis of two-layer NNs after one gradient descent step under Gaussian mixture data assumption.
result High-order polynomial models equivalent to nonlinear neural networks under certain conditions.
Study submanifolds in gradient Ricci solitons with bounded curvature, proving volume growth properties.
problem Volume growth of submanifolds in gradient Ricci solitons with bounded weighted mean curvature.
method Analyzing submanifolds in shrinking gradient Ricci solitons with bounded weighted mean curvature vector.
result Proves polynomial and at least linear volume growth for submanifolds under certain conditions.
Gradient descent fails to learn simple neural networks efficiently.
problem Learning one-layer neural networks efficiently using gradient descent.
method Gradient descent and statistical query algorithms.
result Superpolynomial lower bounds for learning one-layer neural networks.
Efficient method for high-dimensional American option pricing and hedging.
problem High-dimensional American option pricing and hedging.
method Gradient-enhanced sparse Hermite polynomial expansions combined with least squares Monte Carlo.
result Outperforms state-of-the-art methods in high dimensions with comparable computational cost.
This paper improves neural network learning by escaping the NTK regime and efficiently learning sparse polynomials.
problem Learning sparse polynomials efficiently using neural networks.
method Spectral analysis of NTK, identifying 'good' directions, and constructing a regularizer.
result Gradient descent on a two-layer neural network can learn sparse polynomials efficiently, improving over the NTK and QuadNTK.
Gradient EM converges globally for over-parameterized Gaussian mixtures.
problem Recovering ground truth Gaussian mixtures with over-parameterized models.
method Gradient EM with over-parameterization, using Hermite polynomials and tensor decomposition.
result Gradient EM globally converges to ground truth with n=Ω(mlogm) over-parameterization. Gradient Descent with Projection learns low-degree polynomials efficiently.
problem Learning low-degree spherical polynomials with neural networks.
method Over-parameterized two-layer neural network with Gradient Descent with Projection.
result Achieves nearly minimax optimal sample complexity and risk bound.
Holomorphic functions grow polynomially on Kähler-Ricci shrinkers, proving ring finitely generated.
problem Understanding polynomial growth of holomorphic functions on Kähler-Ricci shrinkers.
method Analyzing scalar curvature conditions to prove finite generation of the ring of holomorphic functions.
result The ring of holomorphic functions with polynomial growth on Kähler-Ricci shrinkers is finitely generated.
New algorithm reduces dynamic regret for noisy gradient feedback with piecewise polynomial comparators.
problem Online estimation of piecewise polynomial trends with noisy feedback.
method Introduces variational constraint for piecewise polynomial comparators, designs adaptive algorithm.
result Achieves nearly optimal dynamic regret of $ ilde{O}(n^{ rac{1}{2k+3}}C_n^{ rac{2}{2k+3}})$.
Describes curvature lines on a double torus in 4D space.
problem Analyzing curvature lines on a complex geometric shape.
method Using polynomial gradient and Milnor fibration to define curvature lines, then projecting them into 3D space.
result Complete description of curvature lines on the double torus.
SGD handles label noise with bounds improving over SGLD.
problem Label noise in non-convex optimization.
method Stochastic gradient descent with uniform dissipativity and smoothness conditions, using Wasserstein distance and algorithmic stability.
result Generalization error bounds with a rate of n−2/3, better than SGLD's n−1/2. Reduces learning periodic neural networks to lattice problems, proving hardness under cryptographic assumptions.
problem Learning single periodic neurons in noisy environments.
method Reduction to worst-case lattice problems, using LLL algorithm.
result Polynomial-time algorithms for learning these functions are hard under cryptographic assumptions.
AdaLoss optimizes adaptive learning rates for efficient convergence in various models.
problem Efficiently optimizing adaptive learning rates for gradient descent methods.
method AdaLoss uses loss function information to dynamically adjust step sizes.
result AdaLoss achieves linear convergence in linear regression and robust global convergence in neural networks.
Study growth rates of harmonic functions on curved surfaces.
problem Understanding the growth rates of harmonic functions on curved surfaces.
method Gradient estimate and frequency analysis on complete surfaces and manifolds with non-negative curvature.
result Existence and properties of nonconstant polynomial growth harmonic functions on manifolds with maximal volume growth.
ParamBoost uses gradient boosting to create interpretable non-linear models with constraints.
problem Creating interpretable non-linear models with expert knowledge constraints.
method Gradient Boosting of cubic polynomials with specified constraints.
result ParamBoost outperforms state-of-the-art GAMs in real-world datasets.
Polyak step size GD reaches final radius of convergence after log iterations.
problem Statistical and computational complexities of Polyak step size GD.
method Generalized smoothness and Lojasiewicz conditions, stability of gradients.
result Polyak step size GD reaches final statistical radius of convergence after logarithmic number of iterations.
Neural network learns low-dimensional polynomials with SGD near information-theoretic limit.
problem Learning a single-index target function with gradient descent.
method Two-layer neural network optimized by SGD on squared loss.
result Sample and runtime complexity of n≃T=Θ(d⋅polylogd) for polynomial single-index models, matching information theoretic limit up to polylogarithmic factors. Unique soliton found on resolved cones.
problem Existence of Kähler-Ricci solitons on Calabi-Yau cones.
method Equivariant crepant resolutions and steady gradient Kähler-Ricci solitons.
result Unique complete steady gradient Kähler-Ricci soliton found.
Let f:M→R be a Morse-Bott function on a compact smooth finite dimensional manifold M. The polynomial Morse inequalities and an explicit perturbation of f defined using Morse functions fj on the critical submanifolds Cj of f show immediately that MBt(f)=Pt(M)+(1+t)R(t), where MBt(f)…
Acceleration in Hilbert spaces reduces computations but not accuracy.
problem Improving learning accuracy with fewer computations.
method Analysis of Nesterov acceleration and heavy-ball methods in Hilbert spaces.
result Acceleration can reduce computations but not improve accuracy with respect to gradient descent.
We show that the standard stochastic gradient decent (SGD) algorithm is guaranteed to learn, in polynomial time, a function that is competitive with the best function in the conjugate kernel space of the network, as defined in Daniely, Frostig and Singer. The result holds for log-depth networks from a rich family of ar…
Stochastic gradient descent achieves polynomial convergence rates for noiseless linear models.
problem Convergence analysis of stochastic gradient descent in noiseless linear models.
method Fixed step-size stochastic gradient descent on least-square risk.
result Polynomial convergence rates depend on the regularities of the optimum and feature vectors.
Gradient-trained shallow networks can generalize well but are vulnerable to small-radius adversarial attacks.
problem Adversarial robustness of gradient-trained shallow networks.
method Analysis of neuron alignment and polynomial ReLU activation.
result Gradient-trained shallow networks with polynomial ReLU activation are robust to small-radius adversarial attacks.
The paper shows how neural networks can approximate PDEs with polynomial scaling in dimension.
problem Understanding the complexity of approximating PDE solutions with neural networks.
method Developed a proof technique to simulate gradient descent using neural networks.
result Neural network parameters scale polynomially with input dimension for approximating PDE solutions.
New method computes affine normal directions efficiently for sparse polynomials.
problem Computing affine normal directions is computationally expensive in high dimensions.
method Reduces third-order tensor contraction to matrix-free formulation using log-determinant gradient.
result Scalable implementations with near-linear scaling in dimension and sparsity.
Paper proves a Liouville theorem for solitons with constant curvature.
problem Understanding harmonic functions on specific geometric structures.
method Proved a Liouville theorem without gradient estimates.
result Finite dimensionality of harmonic functions with polynomial growth.