Bayesian deep learning faces posterior collapse due to likelihood vs. prior competition.
problem Posterior collapse in Bayesian deep learning models.
method Identified competition between likelihood and prior regularization in a linear latent variable model.
result Posterior collapse is related to neural and dimensional collapse, suggesting a broader learning issue.
Proves weakly non-collapsed RCD spaces are strongly non-collapsed.
problem Proving the equivalence of weakly non-collapsed and strongly non-collapsed RCD spaces.
method Analyzes properties of RCD spaces and uses auxiliary results.
result Confirms conjecture about RCD spaces being strongly non-collapsed.
Study on Neural Collapse limits in deep learning.
problem Understanding the limits of Neural Collapse in deep learning.
method Investigated Neural Collapse in the context of generalization and feature learning, refining conjectures and conducting experiments.
result Neural Collapse primarily occurs on the train set and not on the test set, suggesting it is an optimization phenomenon with unclear connections to generalization.
Ricci flow smooths locally collapsing manifolds with controlled curvature.
problem Locally collapsing manifolds with controlled Ricci curvature.
method Ricci flow for a definite period of time, detecting collapsing infranil fiber bundles.
result Topological conditions detect collapsing infranil fiber bundles.
Mathematical analysis shows annealing prevents mode collapse in Gaussian mixtures.
problem Mode collapse in variational inference for multimodal distributions.
method Analyzed annealing strategies for Gaussian mixtures, derived formulas, and tested on neural networks.
result Appropriately chosen annealing schemes can robustly prevent mode collapse.
The study characterizes and rules out collapsing in convex ancient mean curvature flow.
problem Characterizing and ruling out collapsing in convex ancient mean curvature flow.
method Characterization and counterexamples.
result Collapsing occurs if and only if the flow is asymptotic to at least one Grim hyperplane.
Despite excellent progress in recent years, mode collapse remains a major unsolved problem in generative adversarial networks (GANs).In this paper, we present spectral regularization for GANs (SR-GANs), a new and robust method for combating the mode collapse problem in GANs. Theoretical analysis shows that the optimal …
New method controls posterior collapse in VAEs without network architecture constraints.
problem Posterior collapse in VAEs reduces diversity of generated samples.
method Introduces Latent Reconstruction (LR) loss to control posterior collapse.
result Controls posterior collapse on various datasets without architectural constraints.
Special Lagrangian submanifolds emerge from K3 surface collapse.
problem Understanding special Lagrangian submanifolds in K3 surface collapse.
method Lifting affine lines to degenerating sequences of special Lagrangian submanifolds.
result Constructing special Lagrangian two-spheres connecting Taub-NUT bubbles.
Collapsibility is a combinatorial strengthening of contractibility. We relate this property to metric geometry by proving the collapsibility of any complex that is CAT(0) with a metric for which all vertex stars are convex. This strengthens and generalizes a result by Crowley. Further consequences of our work are: (1) …
Two-dimensional collapsed spaces with lower Ricci bounds are topological surfaces.
problem Topology of collapsed spaces with lower Ricci bounds
method Prove that collapsed spaces are topological surfaces
result Collapsed spaces are topological surfaces
Prove that collapsing CSC metrics can be perturbed to invariant collapsing CSC metrics.
problem Prove that collapsing constant scalar curvature metrics can be perturbed to invariant collapsing constant scalar curvature metrics.
method Prove that a sequence of constant scalar curvature metrics which is collapsing with bounded curvature to a manifold can be perturbed to a sequence of invariant collapsing constant scalar curvature metrics.
result Prove that a sequence of constant scalar curvature metrics which is collapsing with bounded curvature to a manifold can be perturbed to a sequence of invariant collapsing constant scalar curvature metrics.
We will simplify the earlier proofs of Perelman's collapsing theorem of 3-manifolds given by Shioya-Yamaguchi and Morgan-Tian. Among other things, we use Perelman's semi-convex analysis of distance functions to construct the desired local Seifert fibration structure on collapsed 3-manifolds. The verification of Perelma…
Estimate collapsibility of causal effects in CPDAGs via strong d-convex hulls.
problem Estimate causal effects in CPDAGs.
method Use strong d-convex hulls to characterize minimal collapsible sets.
result Efficient algorithm for obtaining collapsible sets in DAGs and CPDAGs.
We introduce the theory of strong homotopy types of simplicial complexes. Similarly to classical simple homotopy theory, the strong homotopy types can be described by elementary moves. An elementary move in this setting is called a strong collapse and it is a particular kind of simplicial collapse. The advantage of usi…
Lower Ricci curvature bound prevents first Betti number from dropping more than dimension in collapsing manifolds.
problem Understanding how the first Betti number behaves under manifold collapse with Ricci curvature bounds.
method Analyzing sequences of Riemannian manifolds with lower Ricci curvature bounds.
result The first Betti number cannot drop more than the dimension in collapsing manifolds.
Deep nets exhibit 'Neural Collapse' during training's final phase, simplifying decision-making.
problem Understanding and optimizing deep learning training phases.
method Direct measurements on three deepnet architectures across seven datasets.
result Deep nets exhibit 'Neural Collapse' during training's final phase, simplifying decision-making.
Study tackles criterion collapse in learning criteria, showing conditions for loss minimization.
problem Criterion collapse in optimization, focusing on error probability minimizers.
method Analyzes various learning criteria, including DRO, OCE risks, and non-monotonic criteria.
result Non-monotonic criteria can avoid collapse, while monotonic ones cannot.
This is an expositiry article on collapsing theory in Riemannian geometry written for the Modern Encyclopedia of Mathematical Physics (MEMPhys). We focus on describing the geometric and topological structure of collapsed/non-collapsed regions in Riemannian manifold under various curvature assumptions. Numerous applicat…
In this paper we extend the works of Tancer and of Malgouyres and Francés, showing that (d,k)-collapsibility is NP-complete for d≥k+2 except (2,0). By (d,k)-collapsibility we mean the following problem: determine whether a given d-dimensional simplicial complex can be collapsed to some k-dimensional sub…
In this paper, we study collapsed manifolds with boundary, where we assume a lower sectional curvature bound, two sides bounds on the second fundamental forms of boundaries and upper diameter bound. Our main concern is the case when inradii of manifolds converge to zero. This is a typical case of collapsing manifolds w…
This paper examines how skip connections prevent rank collapse in sequence models.
problem Rank collapse in sequence models, leading to reduced expressivity and training instabilities.
method Analytical and ablation studies of lambda-skip connections in SSMs.
result A sufficient condition to prevent rank collapse across various architectures.
Survey on collapsing manifolds using group actions and foliations.
problem Collapsing manifolds with controlled curvature.
method Using group actions and singular Riemannian foliations.
result Recent extensions to singular Riemannian foliations.
Enhances Ricci flow theorem with scalar curvature bound.
problem Improving no-local-collapsing theorem of Ricci flow.
method Derives improved theorem under scalar curvature bound condition.
result Refines Perelman's no-local-collapsing theorem.
The paper studies a relative version of non-positive immersion for 2-complex pairs and shows conditions under which a transitivity law holds.
problem The study of collapsing non-positive immersion for 2-complex pairs and its implications.
method Introduced a relative version of collapsing non-positive immersion for 2-complex pairs (L,K) and proved a transitivity law under certain conditions. result Under certain conditions, a transitivity law holds: If (L,K) has relative collapsing non-positive immersion and K has collapsing non-positive immersion, then L has collapsing non-positive immersion. ContraNorm prevents dimensional collapse in GNNs and Transformers.
problem Dimensional collapse in Graph Neural Networks and Transformers.
method Proposes ContraNorm, a novel normalization layer inspired by contrastive learning.
result Proves ContraNorm alleviates both complete and dimensional collapse under certain conditions.
Topology of non-orientable spaces without boundary is studied.
problem Topology of non-collapsed RCD spaces without boundary.
method Studied the stability of non-orientability and topology under Gromov-Hausdorff convergence.
result Non-orientable spaces without boundary have a stable ramified double cover.
We will simplify earlier proofs of Perelman's collapsing theorem for 3-manifolds given by Shioya-Yamaguchi and Morgan-Tian. Among other things, we use Perelman's critical point theory (e.g., multiple conic singularity theory and his fibration theory) for Alexandrov spaces to construct the desired local Seifert fibratio…
Cryptos remained resilient after SVB's collapse, contrary to expectations.
problem Impact of SVB collapse on crypto markets.
method Factual summary, sentiment analysis, and market performance examination.
result Cryptocurrencies showed resilience after SVB's collapse.
Polyhedra collapse to subpolyhedra if they can be continuously shrunk onto them.
problem Characterizing when a polyhedron can be continuously shrunk onto a subpolyhedron.
method Piecewise-linear free deformation retraction and metric considerations.
result A polyhedron collapses to a subpolyhedron if and only if it admits a free deformation retraction onto that subpolyhedron.
We study collapsed manifolds with Ricci bounded covering geometry i.e., Ricci curvature is bounded below and the Riemannian universal cover is non-collapsed or consists of uniform Reifenberg points. Via Ricci flows' techniques, we partially extend the nilpotent structural results of Cheeger-Fukaya-Gromov, on collapsed …
New metrics prevent event collapse in contrast maximization frameworks.
problem Event collapse in contrast maximization frameworks.
method First principles of space-time deformation based on differential geometry and physics.
result Proposed metrics mitigate event collapse and do not harm well-posed warps.
Finite simply connected 2-complexes with nonpositive planar curvature are collapsible.
problem Understanding the collapsibility of 2-complexes with specific curvature properties.
method Analyzing the fundamental groups and sectional curvatures of 2-complexes.
result Finite simply connected 2-complexes with nonpositive planar curvature are collapsible.
Study identifies topologies of 3D spaces with boundary.
problem Understanding the topologies of compact Alexandrov spaces with boundary.
method Continuation of previous work, determining topologies through analysis.
result Identified topologies of collapsing 3D Alexandrov spaces with boundary.
Gluing theorem for collapsing warped-QAC Calabi-Yau manifolds verified.
problem Behavior of warped-QAC Calabi-Yau metrics on affine quadrics.
method Gluing construction for collapsing warped-QAC Calabi-Yau manifolds.
result Verification of Yang Li's conjecture on warped QAC Calabi-Yau metrics.
New metric space concept and quasi-isometry properties explored.
problem Exploring new metric spaces and quasi-isometry properties.
method Introducing (b,c)-metric and defining collapsing maps.
result Collapsing maps preserve quasi-isometry properties.
Proves Euler characteristic of collapsing Alexandrov spaces.
problem Euler characteristic of collapsing Alexandrov spaces.
method Analyzes strata and fibers of the limit space.
result Euler characteristic equals sum of products of strata and fiber Euler characteristics.
We provide an algebraic description of the Teichmüller space and moduli space of flat metrics on a closed manifold or orbifold and study its boundary, which consists of (isometry classes of) flat orbifolds to which the original object may collapse. It is also shown that every closed flat orbifold can be obtained by col…
In the last two decades, one of the most important developments in Riemannian geometry is the collapsing theory of Cheeger-Fukaya-Gromov. A Riemannian manifold is called (sufficiently) collapsed if its dimension looks smaller than its actual dimension while its sectional curvature remains bounded (say a very thin flat …
A triangulation of a 3-manifold can be shown to be homeomorphic to the 3-sphere by describing a discrete Morse function on it with only two critical faces, that is, a sequence of elementary collapses from the triangulation with one tetrahedron removed down to a single vertex. Unfortunately, deciding whether such a …
New study shows MLE can avoid model collapse with gradual synthetic data addition.
problem Model collapse in generative models trained on synthetic data.
method Theoretical study of maximum likelihood estimation (MLE) under iterative training with accumulating synthetic data.
result Non-asymptotic bounds show MLE can avoid model collapse even as real data fraction vanishes.
Study collapsing geometry of hyperkähler 4-manifolds and prove conjectures.
problem Understanding collapsing geometry of hyperkähler 4-manifolds.
method Investigation of collapsing geometry and proving conjectures.
result Proved two conjectures about collapsed limits and asymptotic behavior of hyperkähler 4-manifolds.
New analysis shows how attention masks and LayerNorm prevent rank collapse in transformers.
problem Rank collapse in transformer models with increasing depth.
method General analysis of rank collapse under self-attention, considering attention masks and LayerNorm.
result Self-attention with LayerNorm can prevent rank collapse and maintain a rich set of equilibria.
This paper explains a mechanism called phase collapse that improves image classification accuracy.
problem Understanding the role of non-linearities and convolutional filters in image classification.
method Demonstrates phase collapse as a mechanism that eliminates spatial variability and linearly separates classes.
result Phase collapse improves classification accuracy, while thresholding operators degrade performance.
Study collapsing geometry with Ricci curvature, proving Kähler metrics and Killing structures.
problem Collapsing geometry of Riemannian manifolds with Ricci curvature constraints.
method Locally bounded Ricci covering geometry and Ricci flow smoothing techniques.
result Volume collapsed Calabi-Yau manifolds admit Ricci-flat Kähler metrics and compatible Killing structures.
Study inradius collapsed manifolds with lower Ricci curvature bounds, proving properties of their limits.
problem Characterizing limits of inradius collapsed manifolds with lower Ricci curvature bounds.
method Analyzing families of manifolds with specific curvature and boundary conditions, proving properties of the limits.
result Limits of inradius collapsed manifolds have at most two boundary components and a lower Ricci curvature bound.
New findings show DNC is not optimal for deep models, revealing a low-rank bias.
problem Theoretical limitations of DNC in non-linear models and multi-class classification.
method Analysis of non-linear models of arbitrary depth in multi-class classification.
result DNC stops being optimal for DUFM when going beyond two layers or two classes, due to a low-rank bias.
New method controls posterior collapse in VAEs with theoretical guarantees.
problem Posterior collapse in VAEs where encoder ignores latent structure.
method Inverse Lipschitz constraint on decoder network.
result Controls degree of posterior collapse for various VAE models.