PILAE learns DNNs without gradient descent, achieving better performance.
problem Training deep feedforward neural networks efficiently and accurately.
method PILAE uses a pseudoinverse learning algorithm for autoencoder building blocks of MLP DNNs.
result PILAE achieves better performance on tradeoff between training efficiency and accuracy.
Paper discusses the pseudoinverse learning algorithm and its variants.
problem Improving learning algorithms for neural networks.
method Review and discussion of the pseudoinverse learning algorithm and its variants.
result Extreme Learning Machine (ELM) is a variant of the pseudoinverse learning algorithm.
PairNets optimize AI models for fast IoT applications.
problem Slow training and high memory usage of deep neural networks.
method Developed Pairwise Neural Networks (PairNets) with low memory and fast training.
result PairNets achieve faster training (one epoch) and lower prediction errors.
Study of special Lorentzian Lie groups with 4D isometry group, finding all are non-gradient expanding Ricci solitons.
problem Characterizing homogeneous Lorentzian three-manifolds with a 4D isometry group.
method Explicit global coordinate description and proof of Ricci soliton properties.
result All special examples are non-gradient expanding Ricci solitons.
Study on non-gradient Ricci almost solitons in warped products.
problem Understanding non-gradient Ricci almost solitons.
method Construction method and explicit example in warped products.
result Rigidity result for Gaussian soliton.
Study defends shallow neural networks from data-poisoning attacks.
problem Protecting shallow neural networks from adversarial attacks during training.
method Developed a non-gradient stochastic algorithm for depth-2 neural networks, proving near-optimal trade-offs.
result Demonstrated improved performance over stochastic gradient descent under various data distributions.
Gradient Ricci solitons can be extended to non-gradient Ricci solitons using energy function.
problem Extending the geometry of gradient Ricci solitons to non-gradient Ricci solitons.
method Using energy function E E E to study the geometry. result A non-steady Ricci soliton with symmetric covariant derivative is gradient.
The paper classifies quasi-Einstein 3-manifolds and their properties.
problem Classifying compact locally homogeneous non-gradient quasi-Einstein 3-manifolds.
method Analyzing quotient spaces of Lie groups and using properties of quasi-Einstein metrics.
result Identifies conditions for the existence of nontrivial quasi-Einstein metrics.
A1GM method improves efficiency in reconstructing missing data using KL divergence.
problem Efficiently reconstructing missing data in matrices.
method Fast non-gradient-based rank-1 NMF using KL divergence.
result A1GM outperforms gradient methods in efficiency with competitive reconstruction errors.
This research analyzes how input and output layers affect deep neural networks' resistance to adversarial attacks.
problem The vulnerability of deep neural networks to adversarial inputs, especially non-gradient based attacks.
method Analysis of three different fully connected dense network classes with manipulated input and output layers.
result Manipulating input and output layers can significantly enhance a deep neural network's robustness against adversarial attacks.
We study 3 3 3 -dimensional Ricci solitons which project via a semi-conformal mapping to a surface. We reformulate the equations in terms of parameters of the map; this enables us to give an ansatz for constructing solitons in terms of data on the surface. A complete description of the soliton structures on all the 3 3 3 -di…
The three-dimensional Heisenberg group H 3 H_3 H 3 has three left-invariant Lorentz metrics g 1 g_1 g 1 , g 2 g_2 g 2 and g 3 g_3 g 3 . They are not isometric each other. In this paper, we characterize the left-invariant Lorentzian metric g 1 g_1 g 1 as a Lorentz Ricci soliton. This Ricci soliton g 1 g_1 g 1 is a shrinking non-gradient Ricci soliton. Likew…
Introduces new info-geometric structure for dynamics on graphs and hypergraphs.
problem Modeling dynamics on discrete structures like graphs and hypergraphs.
method Introduces two dually flat structures: one on vertex space and another on edge space.
result Extends gradient flows to include nonequilibrium dynamics.
The paper classifies Ricci solitons and studies harmonic vector fields on a specific Thurston geometry.
problem Classifying Ricci solitons and studying harmonic vector fields in a specific Thurston geometry.
method Left-invariant Riemannian metric classification and analysis of harmonic maps and vector fields.
result All Ricci solitons on ( F 4 , g ) (F^4,g) ( F 4 , g ) are expanding and non-gradient. Paper studies non-gradient almost Yamabe solitons and their properties.
problem Characterizing structures of non-gradient almost Yamabe solitons.
method Investigates conditions for trivial solitons and local warped product structures.
result Almost Yamabe solitons with closed vector fields admit local warped product structures.
A general Boltzmann machine with continuous visible and discrete integer valued hidden states is introduced. Under mild assumptions about the connection matrices, the probability density function of the visible units can be solved for analytically, yielding a novel parametric density function involving a ratio of Riema…
New quasi-Einstein metrics found on a sphere.
problem Finding quasi-Einstein metrics on a sphere.
method Constructing axi-symmetric non-gradient m m m -quasi-Einstein structures using hypergeometric functions. result Found new regular metrics on a two-sphere, including the extreme Kerr black hole horizon.
Study of Bach flow on specific nilmanifolds, converging to a soliton.
problem Analyzing the Bach flow on specific nilmanifolds.
method Fourth order geometric flow on four-dimensional simply connected nilmanifolds.
result The Bach flow converges to an expanding Bach soliton on these manifolds.
SGLD proves geometric ergodicity via reflection coupling for nonconvex log-concave distributions.
problem Proving geometric ergodicity of SGLD in nonconvex, log-concave settings.
method Reflection coupling technique to handle SGLD's time discretization and minibatch issues.
result SGLD has an invariant distribution and geometric ergodicity in W 1 W_1 W 1 distance. With a f-left-invariant Riemannian metric on a Lie group G G G , we mean a Riemannian metric which is conformally equivalent to a left-invariant Riemannian metric, with the conformal factor f f f . In this article, we study the geometry of such metrics and give a necessary and sufficient condition for an f-left-invariant Rie…
The purpose of this article is to study the existence and uniqueness of quasi-Einstein structures on 3 3 3 -dimensional homogeneous Riemannian manifolds. To this end, we use the eight model geometries for 3-dimensional manifolds identified by Thurston. First, we present here a complete description of quasi-Einstein metric…
Killing fields on compact m-quasi-Einstein manifolds are shown under specific curvature conditions.
problem Characterizing Killing fields on compact m-quasi-Einstein manifolds.
method Extending a result by Bahuaud-Gunasekaran-Kunduri-Woolgar, the approach involves proving the existence of Killing fields under certain curvature conditions.
result A sufficient condition for a compact, non-gradient m-quasi-Einstein metric to admit a Killing field is provided, extending the original result to the m = -2 case.
Foundation for robust finance using rough path theory.
problem Mathematical models of financial markets under Knightian uncertainty.
method Introducing Property (RIE) for càdlàg paths, proving existence of rough integrals, verifying admissibility of trading strategies.
result Existence and stability of rough path integrals for non-gradient integrands.
The study explores ( m , ρ ) (m,ρ) ( m , ρ ) -quasi-Einstein structures on contact metric manifolds.
problem Exploring ( m , ρ ) (m,ρ) ( m , ρ ) -quasi-Einstein structures in contact geometry. method Proving properties of ( m , ρ ) (m,ρ) ( m , ρ ) -quasi-Einstein structures on contact metric manifolds. result Compact contact or H H H -contact metric manifolds with ( m , ρ ) (m,ρ) ( m , ρ ) -quasi-Einstein structures have specific properties. Gradient-enhanced deep GPs improve multifidelity model accuracy.
problem Improving accuracy in multifidelity models using gradient data.
method Extending deep Gaussian processes to incorporate gradient data.
result Gradient-enhanced deep GP outperforms other models in predicting aerodynamic coefficients.
Proposes KDA to protect deep nets from adversarial attacks.
problem Machine learning system vulnerability to adversarial attacks.
method Key based diversified aggregation with pre-filtering.
result Demonstrates high robustness and universality against various attacks.
New algorithm improves online learning with reduced discretization.
problem Improving adaptive online learning with refined discretization.
method Continuous time approach to online learning, followed by a new discretization argument.
result Optimal regret bound with O ( V T ) O(\sqrt{V_T}) O ( V T ) dependence on gradient variance. E-LDA offers faster, interpretable LDA topic models.
problem Inferring topics in LDA topic models with strong guarantees.
method Non-gradient combinatorial approach for faster convergence.
result Logarithmic parallel computation time and interpretability.
New algorithm trains ReLU gates provably in linear time.
problem Training ReLU gates in realizable settings with mild conditions.
method Iterative stochastic algorithm with moment assumptions.
result First recovery of true labels under data-poisoning attacks.
SOLO uses DNN to optimize complex topology problems with reduced FEM calculations.
problem Optimizing materials distribution in complex domains with high computational cost.
method Integrates DNN with FEM calculations to learn and substitute objective functions dynamically.
result Optimum predicted by DNN converges to true global optimum through iterations.
The paper analyzes convergence of Langevin dynamics with time-dependent metrics.
problem Analyzing convergence of Langevin dynamics with time-dependent metrics.
method Formulated a modified gradient flow of the Kullback-Leibler divergence, selected a time-dependent relative Fisher information functional, and developed a time-dependent Hessian matrix condition.
result Proved convergence conditions for various Langevin dynamics.
New methods solve inverse structural modification problems using random projections.
problem Quantifying changes in modal properties of structures given limited data.
method First-order gradient-based methods are inefficient. Particle swarm optimization is used instead. Random projections reduce dimensionality.
result Random projections can reduce dimensionality by 80-99%, making optimization problems more tractable.
Blind Descent avoids gradient issues, using a different learning approach.
problem Gradient issues like exploding and vanishing gradients.
method Does not use gradients to guide learning; instead, it is a more fundamental learning process.
result Gradient descent is a specific case of Blind Descent.
Gradient descent and noisy gradient descent explored on simple functions.
problem Behavior of gradient descent and noisy gradient descent on simple functions.
method Computer experiments with gradient descent and noisy gradient descent on simple functions.
result Noise affects the trajectory of gradient descent on simple functions.
New method reconstructs non-equilibrium stochastic systems from data.
problem Reconstructing non-equilibrium stochastic systems from ensemble measurements.
method Schrödinger bridge problem with multivariate Ornstein-Uhlenbeck process.
result Simulation-free algorithm achieves higher accuracy than competing methods.
Reparameterizes mirror descent as gradient descent for efficient sparse learning.
problem Efficiently training small sparse networks with mirror descent.
method Develops a framework to convert mirror descent updates into gradient descent updates on different parameters.
result Mirror descent can be reparameterized as gradient descent on modified parameters, facilitating standard backpropagation.
Study on noisy gradient descent in higher-dimensional minima.
problem Behavior of gradient descent in higher codimension.
method Computer experiments with noisy gradient descent.
result Effects of noise on gradient descent trajectories in higher codimension.
Derives Mirror Descent from gradient flow on a Riemannian manifold.
problem No specific problem stated; focuses on derivation.
method Derives Mirror Descent from gradient flow on a Riemannian manifold with a natural discretization.
result Generalizes Mirror Descent to non-Hessian metrics.
Accelerates coordinate descent methods for machine learning problems.
problem Slowness of coordinate descent methods in machine learning.
method Extrapolation-based accelerated coordinate descent.
result Significant speed-up in practice compared to existing methods.
Discovering quasipotential equations from data using machine learning.
problem Understanding escape mechanisms from metastable states in nonlinear systems.
method Combining neural networks and sparse regression to symbolically reconstruct quasipotential equations.
result Model-unbiased analytical forms of quasipotential discovered directly from data.
Stochastic gradient descent on manifolds improves low-rank approximation.
problem Efficiently approximate large matrices with lower rank.
method Stochastic gradient descent on a manifold.
result Algorithm outperforms Euclidean space methods on Netflix Prize data.
New analysis shows GMD can converge linearly under PL-like conditions.
problem Establishing linear convergence for generalized mirror descent.
method PL-based analysis for time-dependent mirrors, Taylor-series approach for stochastic GMD.
result Linear convergence of stochastic GMD under PL-like conditions.
Gradient descent optimizes deep ReLU networks with proper initialization.
problem Training deep neural networks with ReLU activation.
method Gradient descent and stochastic gradient descent with proper random weight initialization.
result Gradient descent finds global minima for over-parameterized deep ReLU networks.
A new method improves stochastic gradient descent for faster and more efficient estimation.
problem Efficient and fast parametric estimation methods.
method Projected stochastic gradient descent corrected by Fisher scoring.
result The method is faster and more efficient than traditional methods.
Double descent phenomenon explained in simple terms.
problem Understanding the surprising drop in test error in overparameterized models.
method Informal explanation using linear algebra and probability, visual intuition with polynomial regression, mathematical analysis with ordinary linear regression.
result Three factors create double descent: data undersampling, model size, and parameter count. Ablating any one of these factors prevents double descent.
Online gradient descent can simulate complex computations.
problem Understanding the fine-grained behavior of online gradient descent is hard.
method Proving online gradient descent can encode arbitrary polynomial-space computations.
result It is impossible to reason efficiently about the fine-grained behavior of online gradient descent under weak complexity-theoretic assumptions.
New insights into double descent phenomenon in neural networks.
problem Understanding the double descent behavior in deep learning models.
method Linear teacher-student setup and tools from statistical physics.
result Distinct features are learned at different scales, leading to epoch-wise double descent.
New adaptive step-size method for convex optimization without tuning.
problem Optimizing convex functions efficiently with stochastic gradients.
method Adapted Adaptive Gradient Descent Without Descent to stochastic setting.
result Stochastic gradient descent converges under various assumptions.