SR-GANs combat mode collapse in GANs by monitoring and compensating spectral distributions.
problem Mode collapse in GANs.
method Spectral regularization (SR-GANs) to combat spectral collapse.
result SR-GANs prevent mode collapse and outperform SN-GANs in experiments.
We show that the page at which the Lee spectral sequence collapses gives a bound on the unknotting number, u(K). In particular, for knots with u(K)<3, we show that the Lee spectral sequence must collapse at the E_2 page. An immediate corollary is that the Knight Move Conjecture is true when u(K)<3.
Paper shows stability of metric reconstruction for orbifolds from spectral data.
problem Determining the metric structure of collapsing orbifolds from spectral data.
method Improved quantitative unique continuation for wave operator on Riemannian manifolds.
result Quantitative stability of inverse problem for Riemannian orbifolds.
The paper analyzes graph Laplacians on manifolds with curvature bounds and applies to non-collapsed spaces.
problem Analyzing spectral properties of graph Laplacians on manifolds with curvature constraints.
method Quantitative bounds on eigenvalues and eigenfunctions of graph Laplacians constructed from random variables on manifolds with uniform lower Ricci curvature bounds.
result Spectral convergence of graph Laplacians on manifolds with curvature bounds and in non-collapsed spaces.
Researchers describe a spectral sequence for knots in 3D space.
problem Understanding the Sinha spectral sequence for knots in R^3.
method Explicit description using Fox Neuwirth chain complexes and multicomplex structure.
result A non-trivial third page differential found, contradicting the rational case.
The paper improves collapsing Alexandrov spaces results using good coverings.
problem Collapsing Alexandrov spaces with no proper extremal subsets.
method Application of good coverings of Alexandrov spaces.
result Construction of an infinitely long exact sequence of homotopy groups and a spectral sequence of cohomology groups.
SPECTRE uses spectral conditioning to generate larger graphs without mode collapse.
problem Overcoming expressivity and mode collapse in one-shot graph generators.
method SPECTRE generates graph Laplacian spectrum and eigenvectors to model graph structure.
result SPECTRE outperforms state-of-the-art deep autoregressive generators in fidelity and speed.
Introduce Collapsed Effective Operators for higher-order structures.
problem Existing spectral operators decompose topology into separate ranks, leaving practitioners to fuse information back to vertices.
method Introduce Collapsed Effective Operators via Schur complementation of a graded Laplacian.
result Preserves positive semi-definiteness, lowers system energy under higher-order connectivity.
The purpose of this paper is to compare two spectral sequences converging to the cohomology of a configuration space. The collapsing of these spectral sequences is established, in some cases, using Massey products.
This study analyzes why attention layers in neural networks can cause signal loss and proposes a solution.
problem Pathological behavior of attention layers in neural networks, leading to signal loss.
method Spectral analysis using Random Matrix Theory to identify and mitigate rank collapse in width.
result A novel solution to mitigate rank collapse in width by removing outlier eigenvalues.
The paper explains how continuous language models can produce discrete, interpretable meanings.
problem Semantic collapse in continuous systems of large language models.
method Formalizing large language models as Continuous State Machines (CSMs) and analyzing the associated transfer operator.
result The leading eigenfunctions of the transfer operator induce a finite number of invariant meaning basins, explaining how continuous computation can produce discrete, interpretable semantics.
Batch normalization prevents rank collapse in deep networks, improving training stability.
problem Rank collapse in randomly initialized deep networks with increasing depth.
method Investigates spectral instabilities in random matrices and uses batch normalization to avoid rank collapse.
result Batch normalization prevents rank collapse in both linear and ReLU networks, improving training stability.
The paper tackles model collapse in GPLVMs by improving kernel flexibility and projection variance.
problem Model collapse in GPLVMs leading to vague latent representations.
method Theoretical analysis of projection variance, integration of SM and RFF kernels, and variational inference.
result The advisedRFLVM outperforms competing models in informative latent representations and missing data imputation.
Proposes PSCs for UQ in deep nets without retraining.
problem Estimating uncertainty in deep nets with a single pass.
method Identifies sensitive, smooth intermediate layer, fits probabilistic model.
result PSCs achieve UQ and OOD detection performance matching existing methods.
Study uses satellite data to predict tailings dam collapse risk.
problem Detecting early signs of tailings dam instability.
method Spectral analysis of satellite InSAR displacement time series data.
result Algorithm detects risk milestones up to 5 months before dam collapse.
Recursive training of generative models can lead to model collapse, and the recursion converges to a unique limiting distribution.
problem Model collapse in recursive training of generative models
method Recursive training on their own outputs
result Recursive training converges to a unique limiting distribution
Lobb observed in [arXiv:1103.1412] that each equivariant sl(N) Khovanov-Rozansky homology over C[a] admits a standard decomposition of a simple form. In the present paper, we derive a formula for the corresponding Lee-Gornik spectral sequence in terms of this decomposition. Based on this formula, we give a simple alter…
Study on model collapse in regression models, proposing a mitigation strategy.
problem Model collapse in recursive training of regression models.
method Analytic formulae and adaptive regularization strategy.
result Modified scaling laws and mitigation strategy for model collapse.
We associate a Taylor tower supplied by calculus of the embedding functor to the space of long knots and study its cohomology spectral sequence. The combinatorics of the spectral sequence along the line of total degree zero leads to chord diagrams with relations as in finite type knot theory. We show that the spectral …
New insights explain why β-VAEs fail at disentanglement.
problem Disentanglement performance of β-VAEs peaks at intermediate β and collapses as regularization increases. method Formalized information-theoretic mechanism, introduced λβ-VAE to stabilize disentanglement. result Strong regularization pressure leads to mutual information collapse in β-VAEs. Khovanov homology for pro-tangles and spectral sequences
problem Developing a framework for Khovanov homology for pro-tangles and spectral sequences
method Using pro-tangles, simplicial presheaves, and spectral sequences
result Establishing a fully faithful embedding and an algebraic spectral sequence for pro-tangles
New method prevents entropy collapse in Transformer training, leading to more stable and robust models.
problem Training instability in Transformers, especially in attention layers.
method Spectral normalization with a learned scalar to prevent entropy collapse.
result Prevents entropy collapse, leading to more stable training.
We consider the geometric inverse problem of determining a closed Riemannian manifold from measurements of the heat kernel in an open subset of the manifold. In this paper we analyze the stability of this problem in the class of n-dimensional Riemannian manifolds with bounded diameter and sectional curvature. It is w…
We prove a convergence result for a family of Yang-Mills connections over an elliptic K3 surface M as the fibers collapse. In particular, assume M is projective, admits a section, and has singular fibers of Kodaira type I1 and type II. Let Ξtk be a sequence of SU(n) connections on a principal SU(n)…
We construct a new spectral sequence beginning at the Khovanov homology of a link and converging to the Khovanov homology of the disjoint union of its components. The page at which the sequence collapses gives a lower bound on the splitting number of the link, the minimum number of times its components must be passed t…
We identify spectral conditions for reliable neural probe interpretation.
problem Unreliable performance of linear probes in interpreting neural representations.
method Formalized Spectral Identifiability Principle (SIP) based on eigengap and Fisher error.
result Reliability of neural probes depends on the eigengap relative to Fisher estimation error.
Authors compute stable homology of torus knots using a new deformation technique.
problem Computing stable homology of torus knots.
method Link-splitting deformation (y-ification) of link homology.
result Explicit computation of y-ified glN stable Khovanov--Rozansky homology of torus knots. A theory of feature geometry using spectral analysis of weight matrices.
problem Current methods decompose neural network activations into sparse linear features, losing geometric structure.
method Develops a theory by analyzing the spectra of weight-derived matrices, introducing the frame operator.
result Features collapse onto single eigenspaces, organizing into tight frames, and admit discrete classification.
The study reveals the spectral structure of attention layers and its implications for generalization.
problem Understanding the spectral structure and generalization of trained attention layers.
method Empirical risk minimization in a single-head tied-attention layer, using random matrix theory, spin-glass theory, and approximate message passing.
result Exact high-dimensional characterization of training and test error, interpolation and recovery thresholds, and spectrum of the key and query matrices.
We give an overview of how calculus of the embedding functor can be used for the study of long knots and summarize various results connecting the calculus approach to the rational homotopy type of spaces of long knots, collapse of the Vassiliev spectral sequence, Hochschild homology of the Poisson operad, finite type k…
MLP residual networks implement a selective coarse-graining procedure governed by the spectral structure of the input distribution.
problem Understanding the coarse-graining procedure in MLP residual networks
method Analyzing a pure MLP residual stack on synthetic Markov chain sequences
result MLP residual networks implement a selective coarse-graining procedure governed by the spectral structure of the input distribution
The study prevents model collapse in overparameterized linear regression by mixing real and synthetic labels.
problem Preventing model collapse in overparameterized linear regression.
method Iterative mixing of real and synthetic labels, deriving generalization error formulae.
result Optimal mixing ratio converges to the reciprocal of the golden ratio for isotropic features.
Survey of spectral, probabilistic, and deep metric learning methods.
problem Developing effective distance metrics for various machine learning tasks.
method Divided into spectral, probabilistic, and deep approaches, covering various techniques and their applications.
result Comprehensive overview of metric learning methods, including new developments and applications.
Graph neural networks (GNNs) have become increasingly popular for classification tasks on graph-structured data. Yet, the interplay between graph topology and feature evolution in GNNs is not well understood. In this paper, we focus on node-wise classification, illustrated with community detection on stochastic block m…
Study heat flow on collapsing K3 surfaces, handling conic singularities.
problem Analyzing heat flow on K3 surfaces as they collapse.
method Using semi-flat product approximations and conic-renormalized bilinear functionals.
result Heat operators converge to base Laplacian on regular locus.
We introduce the notion of a Khovanov-Floer theory. Roughly, such a theory assigns a filtered chain complex over Z/2 to a link diagram such that (1) the E_2 page of the resulting spectral sequence is naturally isomorphic to the Khovanov homology of the link; (2) this filtered complex behaves nicely under planar isotopy…
We construct a braid conjugacy class invariant κ by refining Plamenevskaya's transverse element ψ in Khovanov homology via the annular grading. While κ is not an invariant of transverse links, it distinguishes some braids whose closures share the same classical invariants but are not transversely isotopic. Using …
Unified framework detects overfitting in crash classification models.
problem Evaluation metrics fail to detect overfitting in crash classification models.
method Random Matrix Theory and Heavy-Tailed Self-Regularization framework applied to various model types.
result Power-law exponent α reliably distinguishes well-regularized from overfit models.
Let F be a riemannian flow on a closed manifold M. We study the behavior of the first eigenvalues of the Hodge Laplacian acting on differential forms under adiabatic collapsing of the flow. We show that the number of small eigenvalues is related to the basic cohomology of F, and give spectral criteria for the vanishing…
High-dimensional models become unstable when sample size falls below a critical level, leading to a phase transition.
problem Instability in high-dimensional learning models when sample size is insufficient.
method Proved the necessity of a Fisher eigenvalue threshold for stability, introduced Fisher floor for verification.
result A sharp phase transition between reliable concentration and inevitable failure in high-dimensional learning.
New gauge preserves Einstein metrics' interactions, proving rigidity on negatively curved manifolds.
problem Stability and deformation theory of Einstein metrics.
method Introduces Chen-Nagano gauge condition, linking Lichnerowicz Laplacian to shifted scalar operator.
result Chen-Nagano gauge collapses to classical transverse-traceless gauge under spectral pinching assumptions.
The paper classifies links with low rank knot Floer and Khovanov homologies.
problem Detecting and classifying links with low rank knot Floer and Khovanov homologies.
method Generalized link Floer homology, used to obtain rank bounds and classify links.
result Knot Floer homology detects T(2,8) and T(2,10). Deep networks learn clean structure before memorizing corrupted labels, leaving a spectral signature in gradient centered scatter.
problem Deep networks' transition from learning clean structure to memorizing corrupted labels under label noise.
method Analysis of the centered scatter of per-example last-layer gradients to identify Fisher Rank Inflation.
result Fisher Rank Inflation is a spectral signature of memorization under label noise, with effective rank expanding during memorization.
Novel anti-grokking phase discovered in neural networks, revealed by HTSR layer quality metric.
problem Understanding and detecting overfitting in neural networks.
method 3-layer MLP, weight decay, HTSR layer quality metric α, correlation traps, Kolmogorov–Smirnov tests. result Anti-grokking phase detected late in training, revealed by α<2 and correlation traps. Bayesian deep learning faces posterior collapse due to likelihood vs. prior competition.
problem Posterior collapse in Bayesian deep learning models.
method Identified competition between likelihood and prior regularization in a linear latent variable model.
result Posterior collapse is related to neural and dimensional collapse, suggesting a broader learning issue.
The paper presents a new method to create exotic 4-manifolds and surfaces that remain exotic after stabilization.
problem Stabilization of exotic 4-dimensional phenomena and knotted surfaces.
method Elementary approach to constructing exotic 4-manifolds and surfaces, including examples in closed, simply connected 4-manifolds.
result The construction yields exotic surfaces in the 4-ball that remain exotic after stabilization, detected by Khovanov homology.
Proves weakly non-collapsed RCD spaces are strongly non-collapsed.
problem Proving the equivalence of weakly non-collapsed and strongly non-collapsed RCD spaces.
method Analyzes properties of RCD spaces and uses auxiliary results.
result Confirms conjecture about RCD spaces being strongly non-collapsed.
Study on Neural Collapse limits in deep learning.
problem Understanding the limits of Neural Collapse in deep learning.
method Investigated Neural Collapse in the context of generalization and feature learning, refining conjectures and conducting experiments.
result Neural Collapse primarily occurs on the train set and not on the test set, suggesting it is an optimization phenomenon with unclear connections to generalization.