Post-shifted BN prevents filter collapse in BN networks, improving model performance.
problem Filter collapse in BN networks reduces network capacity and harms model performance.
method Post-shifted BN (psBN) to prevent filter collapse by making BN parameters trainable again.
result psBN prevents filter collapse and increases model performance in various tasks.
This paper explains a mechanism called phase collapse that improves image classification accuracy.
problem Understanding the role of non-linearities and convolutional filters in image classification.
method Demonstrates phase collapse as a mechanism that eliminates spatial variability and linearly separates classes.
result Phase collapse improves classification accuracy, while thresholding operators degrade performance.
A new method improves Bayesian filtering in nonlinear systems.
problem Bayesian filtering in nonlinear dynamical systems with non-Gaussian posteriors.
method Transport maps with block-triangular structure and gradient flows for MMD minimization.
result Accurate approximation of non-Gaussian posteriors without particle collapse.
Framework expands particle filtering to estimate states beyond prior boundaries.
problem Limitations of traditional particle filtering in estimating states outside prior support.
method Diffusion-Enhanced Particle Filtering Framework with adaptive diffusion, entropy-driven regularisation, and kernel-based perturbations.
result Framework significantly improves state estimation accuracy and success rates for out-of-boundary targets.
The redundancy is widely recognized in Convolutional Neural Networks (CNNs), which enables to remove unimportant filters from convolutional layers so as to slim the network with acceptable performance drop. Inspired by the linear and combinational properties of convolution, we seek to make some filters increasingly clo…
New approach prevents model collapse in language generation.
problem Model collapse risk in large language models.
method Introduces a replay adversary to study language generation.
result Replay limits generation in certain ways but not all.
New model improves recommendation systems by analyzing user-item interactions.
problem Improving recommendation systems for better user-item interactions.
method Sliced Anti-symmetric Decomposition (SAD) model using tensor decomposition.
result SAD produces the most consistent personalized preferences compared to SOTA models.
For regular particle filter algorithm or Sequential Monte Carlo (SMC) methods, the initial weights are traditionally dependent on the proposed distribution, the posterior distribution at the current timestamp in the sampled sequence, and the target is the posterior distribution of the previous timestamp. This is techni…
A new framework reduces inconsistencies in chaotic surrogate modeling.
problem Consistency issues between probabilistic objectives and dynamical system dynamics.
method KAFFEE (Kalman-Aware Framework For Ergodic Emulation), a differentiable extended Kalman filter.
result KAFFEE mitigates the dynamic-probabilistic consistency gap, improving reconstruction and predictive scores.
Unified framework for self-supervised learning via latent distribution matching.
problem Lack of a unifying theoretical framework for diverse SSL methods.
method Casting SSL as latent distribution matching (LDM): maximizing alignment and uniformity.
result Derives a Bayesian filtering model and proves identifiable latent representations.
Recursive training of generative models can lead to model collapse, and the recursion converges to a unique limiting distribution.
problem Model collapse in recursive training of generative models
method Recursive training on their own outputs
result Recursive training converges to a unique limiting distribution
A geometric characterization of the Arf invariant of a knot in the 3-sphere is given in terms of two kinds of 4-dimensional bordisms, half-gropes and Whitney towers. These types of bordisms have associated complexities class and order which filter the condition of bordism by an embedded annulus, i.e. knot concordance, …
A new method routes EEG covariance matrices across domains using adaptive subspace selection.
problem Challenges in cross-domain EEG decoding due to distinct SPD manifold regions.
method Dynamic Stiefel routing with expert filters and cross-attention for adaptive subspace projection.
result Consistent gains across three datasets: balanced accuracy improves from 0.773 to 0.823, 0.757 to 0.809, and 0.801 to 0.839.
The paper introduces a new method to detect rough volatility and market states using fractional derivatives.
problem Testing self-similarity in fractional processes from a single observed trajectory is difficult under long-range dependence.
method The paper introduces a regime-adaptive KS/GL--KS framework based on the discrete Grünwald--Letnikov (GL) fractional derivative.
result The method detects rough volatility and persistent, anti-persistent, or efficient market states in financial applications.
Graph Convolutional Networks (GCNs) and their variants have experienced significant attention and have become the de facto methods for learning graph representations. GCNs derive inspiration primarily from recent deep learning approaches, and as a result, may inherit unnecessary complexity and redundant computation. In…
Bayesian deep learning faces posterior collapse due to likelihood vs. prior competition.
problem Posterior collapse in Bayesian deep learning models.
method Identified competition between likelihood and prior regularization in a linear latent variable model.
result Posterior collapse is related to neural and dimensional collapse, suggesting a broader learning issue.
Proves weakly non-collapsed RCD spaces are strongly non-collapsed.
problem Proving the equivalence of weakly non-collapsed and strongly non-collapsed RCD spaces.
method Analyzes properties of RCD spaces and uses auxiliary results.
result Confirms conjecture about RCD spaces being strongly non-collapsed.
Study on Neural Collapse limits in deep learning.
problem Understanding the limits of Neural Collapse in deep learning.
method Investigated Neural Collapse in the context of generalization and feature learning, refining conjectures and conducting experiments.
result Neural Collapse primarily occurs on the train set and not on the test set, suggesting it is an optimization phenomenon with unclear connections to generalization.
Motivated by recently published methods using frequency decompositions of convolutions (e.g. Octave Convolutions), we propose a novel convolution scheme to stabilize the training and reduce the likelihood of a mode collapse. The basic idea of our approach is to split convolutional filters into additive high and low fre…
Ricci flow smooths locally collapsing manifolds with controlled curvature.
problem Locally collapsing manifolds with controlled Ricci curvature.
method Ricci flow for a definite period of time, detecting collapsing infranil fiber bundles.
result Topological conditions detect collapsing infranil fiber bundles.
Mathematical analysis shows annealing prevents mode collapse in Gaussian mixtures.
problem Mode collapse in variational inference for multimodal distributions.
method Analyzed annealing strategies for Gaussian mixtures, derived formulas, and tested on neural networks.
result Appropriately chosen annealing schemes can robustly prevent mode collapse.
The study characterizes and rules out collapsing in convex ancient mean curvature flow.
problem Characterizing and ruling out collapsing in convex ancient mean curvature flow.
method Characterization and counterexamples.
result Collapsing occurs if and only if the flow is asymptotic to at least one Grim hyperplane.
Despite excellent progress in recent years, mode collapse remains a major unsolved problem in generative adversarial networks (GANs).In this paper, we present spectral regularization for GANs (SR-GANs), a new and robust method for combating the mode collapse problem in GANs. Theoretical analysis shows that the optimal …
New method controls posterior collapse in VAEs without network architecture constraints.
problem Posterior collapse in VAEs reduces diversity of generated samples.
method Introduces Latent Reconstruction (LR) loss to control posterior collapse.
result Controls posterior collapse on various datasets without architectural constraints.
Study flat manifolds' collapsed limits as flat orbifolds.
problem Understanding collapsed limits of flat manifolds.
method Analyzing totally geodesic foliations and Gromov-Hausdorff limits.
result Identify collapsed limits as flat orbifolds and provide criteria for singularity.
Special Lagrangian submanifolds emerge from K3 surface collapse.
problem Understanding special Lagrangian submanifolds in K3 surface collapse.
method Lifting affine lines to degenerating sequences of special Lagrangian submanifolds.
result Constructing special Lagrangian two-spheres connecting Taub-NUT bubbles.
The torus cannot collapse to a segment under certain curvature conditions.
problem Impossibility of codimension-one collapse for surfaces of negative Euler characteristic.
method Analysis of Gaussian curvature and homology loops.
result The torus cannot collapse to a segment under similar conditions to surfaces of negative Euler characteristic.
Collapsibility is a combinatorial strengthening of contractibility. We relate this property to metric geometry by proving the collapsibility of any complex that is CAT(0) with a metric for which all vertex stars are convex. This strengthens and generalizes a result by Crowley. Further consequences of our work are: (1) …
Two-dimensional collapsed spaces with lower Ricci bounds are topological surfaces.
problem Topology of collapsed spaces with lower Ricci bounds
method Prove that collapsed spaces are topological surfaces
result Collapsed spaces are topological surfaces
Prove that collapsing CSC metrics can be perturbed to invariant collapsing CSC metrics.
problem Prove that collapsing constant scalar curvature metrics can be perturbed to invariant collapsing constant scalar curvature metrics.
method Prove that a sequence of constant scalar curvature metrics which is collapsing with bounded curvature to a manifold can be perturbed to a sequence of invariant collapsing constant scalar curvature metrics.
result Prove that a sequence of constant scalar curvature metrics which is collapsing with bounded curvature to a manifold can be perturbed to a sequence of invariant collapsing constant scalar curvature metrics.
We will simplify the earlier proofs of Perelman's collapsing theorem of 3-manifolds given by Shioya-Yamaguchi and Morgan-Tian. Among other things, we use Perelman's semi-convex analysis of distance functions to construct the desired local Seifert fibration structure on collapsed 3-manifolds. The verification of Perelma…
Estimate collapsibility of causal effects in CPDAGs via strong d-convex hulls.
problem Estimate causal effects in CPDAGs.
method Use strong d-convex hulls to characterize minimal collapsible sets.
result Efficient algorithm for obtaining collapsible sets in DAGs and CPDAGs.
We introduce the theory of strong homotopy types of simplicial complexes. Similarly to classical simple homotopy theory, the strong homotopy types can be described by elementary moves. An elementary move in this setting is called a strong collapse and it is a particular kind of simplicial collapse. The advantage of usi…
Lower Ricci curvature bound prevents first Betti number from dropping more than dimension in collapsing manifolds.
problem Understanding how the first Betti number behaves under manifold collapse with Ricci curvature bounds.
method Analyzing sequences of Riemannian manifolds with lower Ricci curvature bounds.
result The first Betti number cannot drop more than the dimension in collapsing manifolds.
Deep nets exhibit 'Neural Collapse' during training's final phase, simplifying decision-making.
problem Understanding and optimizing deep learning training phases.
method Direct measurements on three deepnet architectures across seven datasets.
result Deep nets exhibit 'Neural Collapse' during training's final phase, simplifying decision-making.
Study tackles criterion collapse in learning criteria, showing conditions for loss minimization.
problem Criterion collapse in optimization, focusing on error probability minimizers.
method Analyzes various learning criteria, including DRO, OCE risks, and non-monotonic criteria.
result Non-monotonic criteria can avoid collapse, while monotonic ones cannot.
Study on collapsing Calabi-Yau manifolds and their metrics.
problem Understanding degenerations of Calabi-Yau manifolds with Ricci-flat Kahler metrics.
method Survey of recent developments, focusing on volume collapsing metrics.
result New insights into the behavior of Calabi-Yau manifolds under volume collapse.
This is an expositiry article on collapsing theory in Riemannian geometry written for the Modern Encyclopedia of Mathematical Physics (MEMPhys). We focus on describing the geometric and topological structure of collapsed/non-collapsed regions in Riemannian manifold under various curvature assumptions. Numerous applicat…
In this paper we extend the works of Tancer and of Malgouyres and Francés, showing that (d,k)-collapsibility is NP-complete for d≥k+2 except (2,0). By (d,k)-collapsibility we mean the following problem: determine whether a given d-dimensional simplicial complex can be collapsed to some k-dimensional sub…
In this paper, we study collapsed manifolds with boundary, where we assume a lower sectional curvature bound, two sides bounds on the second fundamental forms of boundaries and upper diameter bound. Our main concern is the case when inradii of manifolds converge to zero. This is a typical case of collapsing manifolds w…
This paper examines how skip connections prevent rank collapse in sequence models.
problem Rank collapse in sequence models, leading to reduced expressivity and training instabilities.
method Analytical and ablation studies of lambda-skip connections in SSMs.
result A sufficient condition to prevent rank collapse across various architectures.
Survey on collapsing manifolds using group actions and foliations.
problem Collapsing manifolds with controlled curvature.
method Using group actions and singular Riemannian foliations.
result Recent extensions to singular Riemannian foliations.
Enhances Ricci flow theorem with scalar curvature bound.
problem Improving no-local-collapsing theorem of Ricci flow.
method Derives improved theorem under scalar curvature bound condition.
result Refines Perelman's no-local-collapsing theorem.
The paper studies a relative version of non-positive immersion for 2-complex pairs and shows conditions under which a transitivity law holds.
problem The study of collapsing non-positive immersion for 2-complex pairs and its implications.
method Introduced a relative version of collapsing non-positive immersion for 2-complex pairs (L,K) and proved a transitivity law under certain conditions. result Under certain conditions, a transitivity law holds: If (L,K) has relative collapsing non-positive immersion and K has collapsing non-positive immersion, then L has collapsing non-positive immersion. ContraNorm prevents dimensional collapse in GNNs and Transformers.
problem Dimensional collapse in Graph Neural Networks and Transformers.
method Proposes ContraNorm, a novel normalization layer inspired by contrastive learning.
result Proves ContraNorm alleviates both complete and dimensional collapse under certain conditions.
Topology of non-orientable spaces without boundary is studied.
problem Topology of non-collapsed RCD spaces without boundary.
method Studied the stability of non-orientability and topology under Gromov-Hausdorff convergence.
result Non-orientable spaces without boundary have a stable ramified double cover.
We will simplify earlier proofs of Perelman's collapsing theorem for 3-manifolds given by Shioya-Yamaguchi and Morgan-Tian. Among other things, we use Perelman's critical point theory (e.g., multiple conic singularity theory and his fibration theory) for Alexandrov spaces to construct the desired local Seifert fibratio…
Cryptos remained resilient after SVB's collapse, contrary to expectations.
problem Impact of SVB collapse on crypto markets.
method Factual summary, sentiment analysis, and market performance examination.
result Cryptocurrencies showed resilience after SVB's collapse.