The paper examines how the angle between inputs in ReLU networks decreases with depth, impacting training.
problem Depth degeneracy in neural networks, leading to constant function behavior on initialization.
method Combinatorial expansions and Monte Carlo experiments to analyze the angle between inputs in ReLU networks of increasing depth.
result The angle between inputs in ReLU networks decreases exponentially with depth, leading to constant function behavior on initialization.
New Transformer architecture prevents rank degeneracy in deep attention models.
problem Rank degeneracy in deep attention models.
method Modified Softmax-based attention model with skip connections, centered at identity, and scaled logits.
result Existence of a stable SDE implies well-behaved covariance structure, preventing rank degeneracy.
Network degeneracy affects training performance, especially in deep networks.
problem Degeneracy in deep neural networks leads to poor training performance.
method Predicted degeneracy level correlates with training dynamics using finite and infinite width networks.
result Degeneracy in neural networks correlates with training performance and can be predicted.
Polynomial time algorithm learns depth-2 neural networks with ReLU activations.
problem Learning depth-2 neural networks with non-zero bias terms and general ReLU activations.
method Robust tensor decomposition of Hermite expansions.
result Polynomial time and sample efficient learning of depth-2 networks with ReLU activations.
New findings show learning deeper neural networks is hard even with Gaussian inputs and non-degenerate weights.
problem The computational complexity of learning neural networks, especially deeper ones.
method Smoothed analysis framework and local pseudorandom generators.
result Learning depth-3 ReLU networks under Gaussian input distribution is hard even if weight matrices are non-degenerate.
Research on mixed polynomials, extending non-degeneracy concepts to complex variables.
problem Extending non-degeneracy concepts to mixed polynomials in complex variables.
method Generalization of Mondal's partial non-degeneracy to mixed polynomials, introducing new concepts and proving properties.
result Strong partial non-degeneracy implies isolated singularities, and mixed polynomials that are strongly inner non-degenerate satisfy the strong Milnor condition.
Classifies patterns of symmetry breaking and vacuum degeneracy in scalar and gauge fields.
problem Understanding patterns of symmetry breaking and vacuum degeneracy in complex field systems.
method Uses mathematical classification of singular foliations to encode and classify patterns of spontaneous symmetry breaking and vacuum degeneracy.
result Mathematical classification provides a qualitative understanding of possible patterns of vacuum degeneracy.
CP degeneracy affects tensor regression solutions, especially in high dimensions.
problem CP degeneracy in tensor regression.
method Analysis of CP degeneracy and development of a penalized strategy.
result A general penalized strategy to overcome CP degeneracy in tensor regression.
Note removes degeneracy in Kähler geometry estimates.
problem Estimating diameter and inequalities in Kähler geometry with degeneracy.
method Technical improvement of earlier results.
result Established diameter, Green's functions, and Sobolev inequalities without small degeneracy assumption.
Proves SQ lower bounds for learning two-hidden-layer neural networks.
problem Learning two-hidden-layer ReLU networks with Gaussian inputs.
method Refined lifting procedure to reduce Boolean PAC learning to Gaussian.
result Superpolynomial SQ lower bounds for Gaussian inputs.
We construct a complete convergent normal form for a real hypersurface in $\CC{N},\,N\geq 2$ at generic Levi degeneracy. This seems to be the first convergent normal form for a Levi-degenerate hypersurface. In particular, we obtain, in the spirit of the work of Chern and Moser \cite{chern}, distinguished curves in the …
Feedback alignment methods need to be evaluated for accuracy and gradient cosine similarity.
problem Evaluating feedback alignment methods
method Proposed diagnostic evaluation protocol
result Identified silent failures in standard reporting pair
TAMD prevents degeneracy in finite mixtures, offering strong guarantees but modest practical improvements.
problem Degeneracy in maximum likelihood estimation of finite mixtures.
method Transcendental regularization with analytic barrier functions.
result Strong theoretical guarantees (identifiability, consistency, robustness) but modest practical improvements.
We analyze relations between BPS degeneracies related to Labastida-Marino-Ooguri-Vafa (LMOV) invariants, and algebraic curves associated to knots. We introduce a new class of such curves that we call extremal A-polynomials, discuss their special properties, and determine exact and asymptotic formulas for the correspond…
New cylindrical solutions found for Grushin-type problem.
problem Critical Grushin-type problem on CR sphere.
method Local Pohozaev identities for non-degeneracy, Lyapunov-Schmidt reduction for solutions.
result New type of multi-bubbling cylindrical solutions constructed.
Geometric regularisation improves statistical models by avoiding degeneracy loci.
problem Non-identifiability, singular information, and moment indeterminacy in statistical models.
method Develops the geometric regularisation of distribution-kernel pairs (T,φ) using Whitney, Thom, and Mather theorems. result Finite-dimensional weak transversality theorem for generic kernels, avoiding degeneracy strata of high codimension.
Study non-degeneracy of minimal hypersurfaces asymptotic to cones, proving Jacobi equation solvability.
problem Non-degeneracy properties of minimal hypersurfaces asymptotic to cones.
method Analysis of the Jacobi operator and construction of its right inverse.
result Proved solvability of the Jacobi equation under non-degeneracy assumptions.
Study families of Morse functions for manifolds with boundary.
problem Characterize degeneracies in 1-parameter families of Morse functions.
method List all possible degeneracies in generic 1-parameter families.
result Identified all degeneracies in generic 1-parameter families.
Investigates metric degeneracies on symplectic leaves using a generalized gradient flow.
problem Degeneracies in metrics on symplectic leaves of Poisson manifolds.
method Introduces the generalized double bracket (GDB) vector field to generalize gradient dynamics.
result Identifies admissible regions where the double bracket metric remains non-degenerate on symplectic leaves, enabling GDB as a gradient flow.
The spectral properties of p-forms on the fundamental domains of regular tesselations of the d-dimensional sphere are discussed. The degeneracies for all ranks, p, are organised into a double Poincare series which is explicitly determined. In the particular case of coexact forms of rank (d-1)/2, for odd d, it is shown …
A method to automatically and symbolically detect and resolve degenerate parameter combinations from parameter-data pairs.
problem Identifying degenerate parameter combinations in physical models or real-world datasets.
method The degeneracy distillery method detects and resolves degenerate parameter combinations from parameter-data pairs.
result The method reduces the simulation budget required for downstream neural posterior estimation.
For regular particle filter algorithm or Sequential Monte Carlo (SMC) methods, the initial weights are traditionally dependent on the proposed distribution, the posterior distribution at the current timestamp in the sampled sequence, and the target is the posterior distribution of the previous timestamp. This is techni…
Study on automorphisms of complex bk-manifolds, extending previous work.
problem Investigate automorphisms of complex bk-manifolds with higher-order degeneracies. method Extend Mendoza's definition of complex b-manifolds to complex bk-manifolds and study their local and global automorphisms. result Propose bk-analogues for classical spaces of holomorphic functions. Generalizing some results from R. Leung's thesis, we compute, in rational cohomology, the Poincare dual of the degeneracy locus of the family of Dirac operators parameterized by the moduli space of projectively anti-self-dual $\SO(3)$ connections. This is the first step in a program to derive a relation between the Don…
Study on existence of p-Kähler structures on nilmanifolds with nilpotent complex structures.
problem Existence of p-Kähler structures on nilmanifolds with nilpotent complex structures. method Determine optimal p for existence of p-Kähler structures and analyze the relationship between balanced metrics and degeneracy steps of the Frölicher spectral sequence. result No p-Kähler structures exist for an optimal p on nilmanifolds with nilpotent complex structures. Proves existence of proper solutions for inverse mean curvature flow.
problem Existence of proper solutions for inverse mean curvature flow.
method Proves existence theorem assuming non-degeneracy conditions on isoperimetric profile.
result No curvature assumption in existence theorem.
This paper explains why ResNets generalize better than FFNets using neural tangent kernels.
problem Understanding why deep ResNets generalize better than deep FFNets.
method Using neural tangent kernels to compare the learnability of functions induced by the kernels of ResNets and FFNets.
result The kernel of ResNets does not exhibit degeneracy as depth increases, unlike FFNets.
Tian's theorem connects Chern classes of bundles to random section zeros and degeneracy sets.
problem Understanding the distribution of zeros and degeneracy sets of random holomorphic sections.
method Analyzing the pullback of Chern classes and computing currents of integration.
result The limit distribution of zeros of random sections is determined by the Chern form.
Unified Morse-Bott-Smale chain complex, resolves well-definedness issue.
problem Well-definedness of Morse-Bott-Smale chain complex.
method Unified five degeneracy relations into a single condition.
result Quasi-isomorphic to Morse-Smale-Witten chain complex, alternative proof of Morse Homology Theorem.
In this paper, we study the interplay between modules and sub-objects in holomorphic Poisson geometry. In particular, we define a new notion of "residue" for a Poisson module, analogous to the Poincaré residue of a meromorphic volume form. Of particular interest is the interaction between the residues of the canonical …
We develop a mathematically rigorous framework for multilayer neural networks in the mean field regime. As the network's widths increase, the network's learning trajectory is shown to be well captured by a meaningful and dynamically nonlinear limit (the \textit{mean field} limit), which is characterized by a system of …
Counterfactual learning is a natural scenario to improve web-based machine translation services by offline learning from feedback logged during user interactions. In order to avoid the risk of showing inferior translations to users, in such scenarios mostly exploration-free deterministic logging policies are in place. …
Study slopes in 3-manifolds, proving conjectures about knots.
problem Understanding slopes in 3-manifolds and their implications for knots.
method Upper bounds on distances between slopes, applications to knots and surgeries.
result Bounds on boundary and degeneracy slopes for knots in 3-manifolds.
A new method for neural network initialization using graph degeneracy.
problem Improving neural network performance through better initialization.
method Adapted k-hypercore decomposition for neural network initialization.
result k-hypercore outperforms state-of-the-art initialization methods.
The paper studies parallel surfaces of cuspidal cross caps and their degeneracy.
problem Investigating the geometry and singularities of parallel surfaces of cuspidal cross caps.
method Established a criterion for the degeneracy of the distance squared function using geometric invariants.
result Parallel surfaces degenerate into a degenerated cuspidal S1 singularity at specific distances.
Local bifurcation theory typically deals with the response of a degenerate but isolated equilibrium state or periodic orbit of a dynamical system to perturbations controlled by one or more independent parameters, and characteristically uses tools from singularity theory. There are many situations, however, in which the…
Unified interpretation of sub-Riemannian Gauss-Bonnet theorem for surfaces in 3D contact manifolds.
problem Proving a sub-Riemannian Gauss-Bonnet theorem for surfaces in 3D contact manifolds.
method Measure-theoretic perspective, focusing on singular measures and characteristic points.
result Unified interpretation of previous results and natural geometric conditions for the theorem.
The geography problem is usually stated for simply connected symplectic 4-manifolds. When the first cohomology is nontrivial, however, one can restate the problem taking into account how close the symplectic manifold is to satisfying the conclusion of the Hard Lefschetz Theorem, which is measured by a nonnegative integ…
The paper studies the distribution of random degeneracy sets on complex manifolds.
problem Distribution of random degeneracy sets on compact Kähler manifolds.
method Asymptotic expansion of induced Grassmannian Chern forms, meromorphic transforms, and Wishart distribution.
result Normalized currents converge to curvature forms with quantitative estimates.
Geometric theory of projection heads in self-supervised learning.
problem Dimensional collapse and information invariance trade-off in projection heads.
method Geometric modeling of projection heads as Riemannian metrics, analyzing Hessian eigenvalues, and tracking optimization geometry.
result Smooth nonlinear heads induce negative curvature, preventing collapse; linear and ReLU heads cannot.
This paper introduces a scalable benchmark for evaluating local posterior sampling in neural networks.
problem Degeneracy in neural network loss landscapes and its impact on SGMCMC algorithms.
method Development of a scalable benchmark for local posterior sampling.
result RMSProp-preconditioned SGLD is most effective at representing the local geometry of the posterior distribution.
New lower bounds for linear classification problems in high dimensions.
problem Linear classification problems in high-dimensional spaces.
method Reduction from hardness conjectures for Affine Degeneracy testing and k-Sum problems.
result Matching lower bounds of Ω(n^d) and respectively Ω(1/ε^d) for Maximum Halfspace Discrepancy problem.
New pseudometrics defined on knot spaces based on curve thickness and length.
problem Rigidity and non-degeneracy of knot spaces under isotopies.
method Swept-area pseudometrics on ropelength-filtered knot spaces.
result Proved non-degeneracy on polygonal strata and exact distance formulas.
Study circle patterns on tori, linking symplectic forms and homeomorphisms.
problem Understanding circle patterns on tori and their symplectic properties.
method Investigates the space of circle patterns on closed tori with complex projective structures, embedding it into Teichmüller spaces and analyzing symplectic forms.
result Non-degeneracy of the pulled-back Weil-Petersson symplectic form and homeomorphism between circle patterns and Teichmüller spaces.
The paper addresses rigid alignment of noisy patches, providing a polynomial time algorithm and convergence conditions.
problem Finding a rigid alignment of overlapping local views (patches) that minimizes alignment error in a noisy setting.
method Characterizes non-degeneracy based on kernel and positivity of a matrix, provides polynomial time algorithm for testing non-degeneracy, and uses Riemannian gradient descent for alignment.
result The algorithm converges locally linearly to a non-degenerate perfect alignment under certain conditions.
The paper develops Morse homology for a class of elliptic partial differential equations.
problem Developing Morse homology for elliptic partial differential equations.
method Introducing a new notion of non-degeneracy and proving it generically satisfied for a class of functionals defined on Banach spaces.
result The paper enlarges the class of elliptic pde's for which non-degeneracy holds and Morse homology can be defined.
Improved estimation of higher order integrals using shrinkage techniques.
problem Estimating higher order Bochner integrals in non-parametric settings.
method Shrinkage of U-statistic towards a target element, considering kernel degeneracy.
result Consistent shrinkage estimators with fast rates of convergence, even for non-degenerate kernels.
We study and completely describe pairs of compatible Poisson structures near singular points of the recursion operator satisfying natural non-degeneracy condition.