The study analyzes sharpness dynamics in neural networks, revealing mechanisms and conditions.
problem Understanding sharpness in neural network training.
method Fixed point analysis and edge of stability analysis in a simplified 2-layer linear network.
result Reveals mechanisms behind sharpness trends, conditions for edge of stability, and a period-doubling route to chaos.
SAM improves generalization by operating near the edge of stability.
problem Improving generalization in neural networks.
method Sharpness-Aware Minimization (SAM) approach to training neural networks.
result SAM operates near the 'edge of stability' identified by the analysis.
A simple function shows how neural nets can converge despite high sharpness.
problem Understanding why neural nets converge with high sharpness.
method Constructed a minimal example function and analyzed its training dynamics rigorously.
result Final converging point has sharpness close to 2/η. Sharp boundaries for detecting dense subhypergraphs established.
problem Detecting dense subhypergraphs in random hypergraphs.
method Established sharp detection boundaries for known and unknown edge probabilities.
result Sharp detectable regions differ significantly from graph counterparts.
Gradient descent at edge of stability stabilizes implicitly, following projected gradient descent.
problem Gradient descent's stability and sharpness behavior at the edge of instability.
method Cubic Taylor expansion analysis of gradient descent dynamics.
result Gradient descent at edge of stability implicitly follows projected gradient descent.
New findings show mini-batch SGD operates in a 'Edge of Stochastic Stability' regime.
problem Understanding the stability and convergence of mini-batch SGD.
method Analyzing the mini-batch Hessian and its directional curvature.
result Mini-batch SGD operates in a different stability regime (Edge of Stochastic Stability) compared to full-batch GD.
Gradient descent near stability threshold exhibits sharpness oscillations.
problem Understanding sharpness behavior near stability threshold in non-Euclidean norms.
method Interpreted EoS through Directional Smoothness and generalized sharpness under arbitrary norms.
result Non-Euclidean GD with generalized sharpness shows sharpness oscillations near 2/η. New insights into network generalization show learning rate affects both norm and sharpness.
problem Understanding the generalization of overparameterized networks.
method Empirical analysis and theoretical proof of the trade-off between norm and sharpness.
result Learning rate influences both norm and sharpness, neither alone minimizes generalization error.
Suppose C is a compact, n-edged two-cell of the centered dual decomposition of a locally finite set in the hyperbolic plane, a coarsening of the Delaunay tessellation which was introduced in the author's prior work. We describe an effectively computable lower bound on the area of C, given an n-tuple of positive…
GD monotonically decreases GFS sharpness in neural networks and scalar models.
problem Oscillatory behavior of loss in GD training.
method Analysis of GFS sharpness and empirical validation.
result GFS sharpness decreases monotonically during GD training.
Cohen et al. (2021) show GD trajectories align on a bifurcation diagram.
problem Understanding the Edge of Stability (EoS) phenomenon in gradient descent.
method Empirical studies and rigorous mathematical proofs for two-layer networks and single-neuron networks.
result GD trajectories align on a specific bifurcation diagram independent of initialization.
Training avoids edge of stability by aligning Jacobian matrices.
problem Training neural networks on the edge of stability causes inaccuracies.
method Used an exponential Euler solver to prevent entering the edge of stability.
result Alignment of Jacobian matrices causes sharpness increase in Hessian.
Gradient descent near stability threshold shows sharpness oscillations.
problem Understanding sharpness and stability in non-Euclidean norms during gradient descent.
method Interpreted EoS through Directional Smoothness, defined generalized sharpness for arbitrary norms.
result Non-Euclidean GD exhibits sharpness oscillations around the stability threshold.
Momentum affects optimization differently at small vs large batch sizes near instability.
problem Understanding how momentum impacts optimization near the edge of stability.
method Demonstrated through batch-size dependent behavior of SGD with momentum.
result Momentum operates in two distinct regimes: amplifying stochastic fluctuations at small batch sizes and stabilizing at large batch sizes.
Flow on weighted graphs sharpens Bakry-Émery curvature.
problem Sharp curvature in weighted graphs.
method Bakry-Émery curvature flow on mixed weighted graphs.
result Limits of curvature flow are curvature sharp.
The paper calculates graph Ricci curvature and finds properties of specific graph types.
problem Understanding Ricci curvature on irregular graphs.
method Developed a formula for graph Ricci curvature based on optimal bijections.
result Derived structural and theorem results for specific graph types.
Weight decay stabilizes training dynamics by slowing progressive sharpening.
problem Understanding how weight decay affects training stability in deep learning models.
method Analyzing weight decay effects at the Edge of Stability, developing a mathematical framework.
result Weight decay dampens oscillations and stabilizes sharpness in CNNs, causing a phase transition in MLPs.
The study evaluates forecast risk-adjusted performance using various metrics.
problem Evaluating forecast reliability beyond accuracy.
method Risk-adjusted performance measures (Sharpe, Sortino, Omega ratios) and Edge Ratio.
result Machine learning models often offer attractive risk profiles but not necessarily higher reliability.
Study financial contagion and risk in sparse networks with directed edges.
problem Analyzing systemic risk in sparse financial networks with balance-sheet interactions.
method Linear fraction of institutions with zero out-degree, sender-truncated subgraph G_sh, adversarial and random systemic events, explicit fan-in accumulation bound.
result Maximal forward reachability in G_sh is O(log n) with high probability in the subcritical regime, and multi-hit defaults are negligible in the supercritical regime.
Polygonal meshes provide an efficient representation for 3D shapes. They explicitly capture both shape surface and topology, and leverage non-uniformity to represent large flat regions as well as sharp, intricate features. This non-uniformity and irregularity, however, inhibits mesh analysis efforts using neural networ…
GD converges faster to flatter minima than gradient flow in shallow networks.
problem Understanding the dynamics of gradient descent in shallow linear networks.
method Analyzing the convergence rate and solution of gradient descent in depth-2 linear neural networks.
result GD converges linearly to flatter minima than gradient flow, even with large step sizes.
A labeled oriented graph (LOG) is an oriented graph with a labeling function from the edge set into the vertex set. The complexity of a LOG is the minimal cardinality of an initial set S of vertices such that every vertex can be reached successively from S only using edges with labels in S or already visited vert…
Existence and uniqueness of discrete Einstein metrics on trees proven.
problem Existence and uniqueness of discrete Einstein metrics on trees.
method Using Perron-Frobenius theory and Lin-Lu-Yau Ricci curvature.
result Existence and uniqueness of discrete Einstein metrics on trees established.
Study detects edge correlation between unlabeled random graphs.
problem Detect edge correlation between unlabeled random graphs.
method Hypothesis testing, conditional second-moment method, pseudoforest structure, enumeration of subpseudoforests.
result Sharp threshold for phase transition in testing error probability.
SGD works well with large learning rates at the edge of stability.
problem Stochasticity at the edge of stability in deep learning.
method Sharp convergence guarantees for SGD with multiclass cross-entropy loss.
result SGD self-stabilizes, ensuring convergence with large learning rates.
Study reconstructs hidden perfect matchings in random graphs with specific edge weights.
problem Reconstructing hidden perfect matchings in random weighted bipartite graphs.
method Analyzes the maximum likelihood estimator for matching reconstruction under different probability distributions of edge weights.
result Sharp threshold and infinite-order phase transition in reconstruction error for different probability distributions.
We prove that the asymptotic completion of a developable Möbius strip in Euclidean three-space must have at least one singular point other than cuspidal edge singularities. Moreover, if the strip contains a closed geodesic, then the number of such singular points is at least three. These lower bounds are both sharp.
The study finds the bounds of vertex orbits in maps derived from specific lattices.
problem Determining the bounds of vertex orbits in maps derived from k-vertex-homogeneous lattices. method Analyzing maps as quotients of k-vertex-homogeneous lattices. result Sharp bounds of the number of vertex orbits are identified.
We introduce a scalable measure of curvature for analyzing training dynamics of large language models.
problem Analyzing the training dynamics of large language models due to high computational cost of measuring Hessian sharpness.
method We introduce critical sharpness and relative critical sharpness as computationally efficient measures capturing Hessian sharpness phenomena.
result We provide the first demonstration of sharpness phenomena at scale up to 7B parameters.
New theory explains how chaotic training improves neural network generalization.
problem Understanding how chaotic training improves neural network generalization.
method Representing stochastic optimizers as random dynamical systems and introducing a new dimension concept.
result Generalization in chaotic training depends on the complete Hessian spectrum and partial determinants.
Improved financial network predictability using LLM for edge filtering.
problem Spurious edges in financial networks from textual similarity.
method Two-stage framework: sparse candidate graph + LLM edge classification.
result LLM-based edge filtering improves Sharpe ratio and reduces drawdown.
DIF extends NF with stochastic discrete latent variables for better density estimation.
problem Improving density estimation with discontinuities and fine details.
method Discretely indexed flows as an extension of Normalizing Flows with stochastic latent variables.
result DIF inherit good computational behavior of NF and can capture distributions with discontinuities.
Study sets limits for detecting a subhypergraph in uniform hypergraphs.
problem Recovering a subhypergraph from a uniform hypergraph with different edge probabilities.
method Information-theoretic analysis for weak and exact recovery.
result Sharp conditions for weak or exact recovery of the subhypergraph.
The study characterizes 3-pseudomanifolds with up to two singularities.
problem Characterizing face-number-related invariants of normal 3-pseudomanifolds with up to two singularities.
method Proves properties of normal 3-pseudomanifolds using specific operations and upper bounds.
result Proves that normal 3-pseudomanifolds with up to two singularities are constructed from boundary complexes of 4-simplices.
A new ODE model explains gradient descent dynamics near edge of stability.
problem Understanding gradient-based training over non-convex landscapes.
method Rod Flow, a new ODE approximation of GD dynamics.
result Rod Flow accurately predicts critical sharpness threshold and self-stabilization in quartic potentials.
This paper resolves the all-or-nothing phase transition in graph matching.
problem Recovering vertex correspondence between edge-correlated random graphs.
method Analysis of mutual information, truncated second-moment computation, and maximum likelihood estimator.
result Sharp thresholds for correct matching in both dense and sparse graphs.
New superbridge index calculations for knots with odd edges.
problem Computing superbridge index of knots.
method Polygonal realizations with odd edges and linear programming.
result Exact superbridge index of many new knots, including 9- and 12-crossing knots.
Stable commutator length scl_G(g) of an element g in a group G is an invariant for group elements sensitive to the geometry and dynamics of G. For any group G acting on a tree, we prove a sharp bound scl_G(g)>=1/2 for any g acting without fixed points, provided that the stabilizer of each edge is relatively torsion-fre…
We investigate slicings of combinatorial manifolds as properly embedded co-dimension 1 submanifolds. A focus is given to dimension 3 where slicings are normal surfaces. In the case of 2-neighborly 3-manifolds and quadrangulated slicings, a lower bound on the number of quadrilaterals of normal surfaces depending on the …
Detecting edge correlation between two graphs sharpens a threshold based on densest subgraph.
problem Detecting edge correlation between two Erdős-Rényi graphs.
method Formulated as a hypothesis testing problem, connecting to densest subgraph detection.
result Sharp information-theoretic threshold established for edge correlation detection.
Paper solves curvature assignment on surfaces with sharp points and edges.
problem Prescribing curvatures on surfaces with conical singularities and corners.
method New variational formulation for surfaces with singularities.
result First results for prescribed curvatures on surfaces with singularities.
Expressivity is one of the most significant issues in assessing neural networks. In this paper, we provide a quantitative analysis of the expressivity for the deep neural network (DNN) from its dynamic model, where the Hilbert space is employed to analyze the convergence and criticality. We study the feature mapping of…
Quantum stochastic walks optimize portfolios by leveraging financial networks, improving Sharpe ratios and reducing turnover.
problem Optimizing portfolios in noisy financial markets with superior risk-adjusted returns.
method Embed assets in a weighted graph, using quantum stochastic walks to derive optimal portfolio weights from the stationary distribution.
result Quantum stochastic walks can lift Sharpe ratios by up to 27% and reduce turnover from 480% to 2-90%.
Soft cells fill space without gaps, derived from minimal surfaces and deformed using edge bending.
problem Creating space-filling shapes without sharp corners.
method Edge bending algorithm to deform polyhedral tilings into soft tilings.
result Soft tilings derived from minimal surfaces can be continuously transformed into one another.
Graphs on surfaces have limits for complete walks, impacting ergodicity.
problem Graphs embedded in surfaces have limits for complete leftward walks.
method Analyzes graphs embedded in surfaces, proving limits on valence for complete walks.
result The valence of graphs embedded in surfaces is bounded for complete walks.
Method solves optimisation problems on non-Riemannian surfaces with bilateral curvature bounds.
problem Optimisation problems on non-Riemannian surfaces with sharp edges.
method Forward-backward splitting in Alexandrov spaces with bilateral curvature bounds.
result Convergence of the forward-backward method in Alexandrov spaces with bilateral curvature bounds.
This paper tackles exact recovery of clusters in a stochastic Ising model on a SBM graph.
problem Recovering clusters in a stochastic Ising model on a SBM graph.
method Proposes a Stochastic Ising Block Model (SIBM) and establishes a sharp threshold for exact recovery.
result Sharp threshold m∗ for exact recovery of clusters in SIBM, with O(n) time complexity for m≥m∗. Background: Three-dimensional, whole heart, balanced steady state free precession (WH-bSSFP) sequences provide delineation of intra-cardiac and vascular anatomy. However, they have long acquisition times. Here, we propose significant speed ups using a deep learning single volume super resolution reconstruction, to reco…