Introduce Collapsed Effective Operators for higher-order structures.
problem Existing spectral operators decompose topology into separate ranks, leaving practitioners to fuse information back to vertices.
method Introduce Collapsed Effective Operators via Schur complementation of a graded Laplacian.
result Preserves positive semi-definiteness, lowers system energy under higher-order connectivity.
Cheeger and Gromov showed that F-structures are related to collapse with a double-sided curvature bound. We define fibered F-structures and extend some of the Cheeger-Gromov results to the setting of collapse with a lower bound on the curvature operator.
We study the behavior of the spectrum of the Dirac operator on collapsing S^1-bundles. Convergent eigenvalues will exist if and only if the spin structure is projectable.
Enhances neural architecture search efficiency and prevents performance collapse.
problem Improving memory efficiency and preventing performance collapse in neural architecture search.
method Employing continuous relaxation strategy and gradient-based optimization for over-parameterized BCNN construction, introducing Confident Learning Rate and partial channel connections.
result NAS-v2 delivers state-of-the-art search efficiency on CIFAR-10 and ImageNet.
Estimate collapsibility of causal effects in CPDAGs via strong d-convex hulls.
problem Estimate causal effects in CPDAGs.
method Use strong d-convex hulls to characterize minimal collapsible sets.
result Efficient algorithm for obtaining collapsible sets in DAGs and CPDAGs.
This paper explains a mechanism called phase collapse that improves image classification accuracy.
problem Understanding the role of non-linearities and convolutional filters in image classification.
method Demonstrates phase collapse as a mechanism that eliminates spatial variability and linearly separates classes.
result Phase collapse improves classification accuracy, while thresholding operators degrade performance.
Formula proves Euler characteristic of singularized surfaces.
problem Calculating the Euler characteristic of singularized surfaces.
method Three operations: collapsing, zipping, and double loop identification.
result Formula for Euler characteristic of singularized surfaces.
We compute the spectrum of the Dirac operator on 3-dimensional Heisenberg manifolds. The behavior under collapse to the 2-torus is studied. Depending on the spin structure either all eigenvalues tend to ±∞ or there are eigenvalues converging to those of the torus. This is shown to be true in general for collap…
Polygonal meshes provide an efficient representation for 3D shapes. They explicitly capture both shape surface and topology, and leverage non-uniformity to represent large flat regions as well as sharp, intricate features. This non-uniformity and irregularity, however, inhibits mesh analysis efforts using neural networ…
We show characterizations of non-collapsed compact RCD(K,N) spaces, which in particular confirm a conjecture of De Philippis-Gigli on the implication from the weakly non-collapsed condition to the non-collapsed one in the compact case. The key idea is to give the explicit formula of the Laplacian associated to the p…
The goal of the paper is to calculate the limit sectrum of the Hodge-Laplace operator under the perturbation of collapse of one part of a connected sum. This gives some new results concerning the 'conformal spectrum' on differential forms.
Paper shows stability of metric reconstruction for orbifolds from spectral data.
problem Determining the metric structure of collapsing orbifolds from spectral data.
method Improved quantitative unique continuation for wave operator on Riemannian manifolds.
result Quantitative stability of inverse problem for Riemannian orbifolds.
Similarity algebra extends algebraic structures with quantitative bounds.
problem Exact algebraic structures with strict axioms.
method Framework for approximate algebraic and Lie structures with ε-estimates. result Similarity structures converge to classical algebraic objects as εightarrow0. We study the behavior of the spectrum of the Dirac operator together with a symmetric W1,∞-potential on spin manifolds under a collapse of codimension one with bounded sectional curvature and diameter. If there is an induced spin structure on the limit space N then there are convergent eigenvalues which co…
Random weights in GNNs match learned weights in performance.
problem Feature rank collapse in GNNs.
method Replacing learned weights with random weights.
result Random weights achieve comparable performance to learned weights, reducing training time and memory usage.
ContraNorm prevents dimensional collapse in GNNs and Transformers.
problem Dimensional collapse in Graph Neural Networks and Transformers.
method Proposes ContraNorm, a novel normalization layer inspired by contrastive learning.
result Proves ContraNorm alleviates both complete and dimensional collapse under certain conditions.
We analyze the limit of the spectrum of a geometric Dirac-type operator under a collapse with bounded diameter and bounded sectional curvature. In the case of a smooth limit space B, we show that the limit of the spectrum is given by the spectrum of a certain first-order differential operator on B, which can be constru…
New model explains GAN training dynamics and mode collapse.
problem Mode collapse in GANs, where generators fail to reproduce diversity.
method Simplified model using particles in output space, coupled by universal kernel.
result Gradient regularizers can optimally yield convergence through critical damping.
Formula for scalar curvature under metric collapse.
problem Finding positive scalar curvature metrics.
method Formula involving scalar curvature and adapted orthonormal frame.
result Effect of metric collapse on scalar curvature.
The paper establishes principles for initializing and designing GNNs with ReLU activations to avoid oversmoothing and correlation collapse.
problem Oversmoothing and correlation collapse in deep ReLU GNNs.
method The paper derives and validates three principles for initialization and architecture selection in finite width graph neural networks with ReLU activations.
result Correct initialization, residual aggregation operators, and residual connections significantly improve early training dynamics in deep ReLU GNNs.
The goal of the present paper is to calculate the limit spectrum of the Hodge-de Rham operator under the perturbation of collapsing one part of a manifold obtained by gluing together two manifolds with the same boundary. It appears to take place in the general problem of blowing up conical singularities as introduced i…
The paper explains how continuous language models can produce discrete, interpretable meanings.
problem Semantic collapse in continuous systems of large language models.
method Formalizing large language models as Continuous State Machines (CSMs) and analyzing the associated transfer operator.
result The leading eigenfunctions of the transfer operator induce a finite number of invariant meaning basins, explaining how continuous computation can produce discrete, interpretable semantics.
VINNAS uses variational inference to avoid mode collapse in neural architecture search.
problem Mode collapse in gradient-based NAS methods, leading to suboptimal architectures.
method Differentiable variational inference with variational dropout and automatic relevance determination.
result State-of-the-art accuracy with up to twice fewer non-zero parameters.
Study on removing sets and uniqueness of diffusion operators on various spaces.
problem Determining the effect of removing small sets on the self-adjointness and uniqueness of diffusion operators.
method Analyzes symmetric diffusion operators on metric measure spaces, proving a truncation result for potentials.
result Characterizes the critical size of removed sets and their effect on operator properties.
DARTS- improves robustness by factoring out skip connections' advantage.
problem Performance collapse in DARTS architecture search.
method Factor out skip connections' advantage with an auxiliary skip connection.
result Significantly improved robustness across various datasets.
Study shows annealing with adaptive schedule reduces mode collapse in NFs for parameter estimation.
problem Mode collapse in normalizing flows for multimodal distributions.
method Annealing with an adaptive schedule based on effective sample size (ESS).
result Our approach reduces mode collapse and converges marginal likelihood faster than MCMC methods.
Deep Gaussian Processes with polynomial kernels can collapse rapidly without proper hyperparameter tuning.
problem The collapse of Deep Gaussian Processes with polynomial kernels without careful hyperparameter tuning.
method Analysis using the Berry-Esseen Theorem and observation of prior behavior.
result The prior of a Deep Gaussian Process collapses rapidly towards zero or places negligible mass on low norm functions without proper hyperparameter tuning.
We show the convergence properties of the eigenvalues of the Dirac operator on a spin manifold with a Riemannian flow when the metric is collapsed along the flow.
Researchers discover a new family of 3D solitons that are flying wings.
problem Verifying a conjecture about 3D steady gradient Ricci solitons.
method Analyzing a family of 3D flying wing solitons and proving properties of these solitons.
result 3D flying wing solitons are non-collapsed and have non-zero scalar curvature at infinity.
New method controls posterior collapse in VAEs with theoretical guarantees.
problem Posterior collapse in VAEs where encoder ignores latent structure.
method Inverse Lipschitz constraint on decoder network.
result Controls degree of posterior collapse for various VAE models.
Deep nets exhibit 'Neural Collapse' during training's final phase, simplifying decision-making.
problem Understanding and optimizing deep learning training phases.
method Direct measurements on three deepnet architectures across seven datasets.
result Deep nets exhibit 'Neural Collapse' during training's final phase, simplifying decision-making.
New metrics prevent event collapse in contrast maximization frameworks.
problem Event collapse in contrast maximization frameworks.
method First principles of space-time deformation based on differential geometry and physics.
result Proposed metrics mitigate event collapse and do not harm well-posed warps.
By Cheeger-Colding's almost splitting theorem, if a domain in a Ricci flat manifold is pointed-Gromov-Hausdorff close to a lower dimensional Euclidean domain, then there is a harmonic almost splitting map. We show that any eigenfunction of the Laplace operator is almost constant along the fibers of the almost splitting…
The paper proves extremal black holes form at a critical point of gravitational collapse.
problem Formation of extremal black holes in gravitational collapse.
method Constructing smooth families of spherically symmetric solutions to the Einstein-Maxwell-Vlasov system.
result Extremal Reissner-Nordström black holes form at the critical collapse threshold.
NPO method improves LLM unlearning without catastrophic collapse.
problem Efficiently unlearning undesirable data from LLMs without losing model utility.
method Negative Preference Optimization (NPO) method based on alignment.
result NPO-based methods achieve better unlearning results and maintain model utility.
Derives operational-time variance kernel for reaction boundaries in financial markets.
problem Separating components in volatility models to better understand market dynamics.
method Derives a variance kernel for a latent-order-book reaction boundary, separating structural boundary cumulant, clock projection, and pricing-measure choice.
result Operational variance has a closed asymptotic form for long-memory forcing, with effective signed-forcing intensity and resilience.
Derives variance kernel for reaction boundary in financial models.
problem Separating components in financial volatility models.
method Operational-time variance kernel, damped Abel response kernel, closed asymptotic form.
result Operational variance has a closed asymptotic form involving various parameters.
The paper studies gravitational instantons with flat limits and finds elliptic regularity estimates.
problem Analyzing gravitational instantons with flat limits using elliptic analysis.
method Establishing elliptic regularity estimates and showing uniform constants for a family of metrics.
result The Laplacian is Fredholm and an isomorphism between specific weighted spaces.
Verification helps prevent model collapse when training on synthesized data.
problem Model collapse when training on generated data.
method Theoretical characterization using Gaussian mixtures, linear classifiers, and linear verifiers to assess synthesized data quality.
result Verifiers can prevent model collapse and correlate with performance.
In narrow asymptotic settings Gaussian VAE models of continuous data have been shown to possess global optima aligned with ground-truth distributions. Even so, it is well known that poor solutions whereby the latent posterior collapses to an uninformative prior are sometimes obtained in practice. However, contrary to c…
NECO detects out-of-distribution data using neural collapse properties.
problem Detecting out-of-distribution data in machine learning models.
method NECO leverages neural collapse geometric properties to identify OOD data.
result NECO achieves state-of-the-art results on OOD detection tasks.
This work tackles posterior collapse in conditional and hierarchical VAEs.
problem Posterior collapse in VAEs leads to poor latent variable representations.
method Theoretical analysis of linear conditional and hierarchical VAEs, empirical validation.
result Theoretical and empirical evidence of posterior collapse causes in conditional and hierarchical VAEs.
New analysis shows how attention masks and LayerNorm prevent rank collapse in transformers.
problem Rank collapse in transformer models with increasing depth.
method General analysis of rank collapse under self-attention, considering attention masks and LayerNorm.
result Self-attention with LayerNorm can prevent rank collapse and maintain a rich set of equilibria.
This work shows dimension regularization can replace skip-gram negative sampling for graph embeddings, improving efficiency and performance.
problem Efficiently enforcing dissimilarity among node embeddings in graph learning.
method Dimension regularization as an alternative to skip-gram negative sampling.
result Dimension regularization is a more efficient approach to enforcing dissimilarity in graph embeddings.
Simplified non-contrastive learning avoids representation collapse.
problem Training failure modes in self-supervised learning.
method Hyperdimensional computing and inductive bias.
result The approach avoids representation collapses.
Generative adversarial networks (GANs) often suffer from unpredictable mode-collapsing during training. We study the issue of mode collapse of Boundary Equilibrium Generative Adversarial Network (BEGAN), which is one of the state-of-the-art generative models. Despite its potential of generating high-quality images, we …
A fast geometric regularizer improves event camera performance.
problem Event collapse in contrast maximization framework.
method Geometric regularizer to mitigate overfitting.
result State-of-the-art accuracy with reduced computational complexity.
Variational autoencoders (VAEs) hold great potential for modelling text, as they could in theory separate high-level semantic and syntactic properties from local regularities of natural language. Practically, however, VAEs with autoregressive decoders often suffer from posterior collapse, a phenomenon where the model l…