A new method for spectral positional encodings in directed graphs using Hermitian block Krylov subspaces.
problem Challenges in spectral positional encodings for directed graphs, including computational complexity and gauge invariance issues.
method Learnable spectral positional encodings of the form hθ(Aq)R, computed in a Hermitian block Krylov subspace from sparse matrix-vector products. result The method is gauge-invariant and converges to the exact eigendecomposition oracle as the depth grows.
New method for directed graphs using learnable spectral positional encodings.
problem Challenges in magnetic Laplacians and unitary gauge invariance for directed graphs.
method Learnable spectral PEs of the form hθ(Aq)R, computed in Hermitian block Krylov subspace.
result Gauge-invariant and computationally efficient solution for directed graphs.
Spectral flow connects manifold geometry to rigidity criteria.
problem Tackling rigidity of simply-connected closed manifolds.
method Spectral deformation flow and invariant-based approach.
result Spherical profile is the unique manifold-compatible asymptotic realization.
Introduce Collapsed Effective Operators for higher-order structures.
problem Existing spectral operators decompose topology into separate ranks, leaving practitioners to fuse information back to vertices.
method Introduce Collapsed Effective Operators via Schur complementation of a graded Laplacian.
result Preserves positive semi-definiteness, lowers system energy under higher-order connectivity.
Spectral sparsification improves Gaussian graphical models under MTP2 constraints.
problem Learning accurate, sparse graphs from data under MTP2 constraints.
method Spectral graph sparsification applied to Gaussian graphical models.
result Spectral-MTP2 preserves MTP2 and approximates the original model well.
The study shows how geometric Weyl bulk-density exponent rigidifies spectral encodings in O-regularly varying classes.
problem Understanding spectral encodings under Weyl growth conditions.
method Analyzing geometric Weyl bulk-density exponent and proving spectral rigidity.
result The geometric Weyl bulk-density exponent (d−2)/2 rigidifies spectral encodings in the O-regularly varying class, leading to unique admissible exponents and scaling laws. The paper shows how to recover true node positions from a graph or similarity matrix.
problem Recovering true distances and positions from a graph or similarity matrix.
method Two steps: matrix factorisation followed by nonlinear dimension reduction.
result Nonlinear dimension reduction can recover latent positions close to a manifold where geodesic distance is encoded.
A framework uses free probability to analyze Transformer models.
problem Understanding the dynamics and complexity of Transformer-based language models.
method Formal operator-theoretic analysis using free probability theory.
result Entropy-based generalization bounds derived under freeness assumptions.
Constrained clustering has been well-studied for algorithms such as K-means and hierarchical clustering. However, how to satisfy many constraints in these algorithmic settings has been shown to be intractable. One alternative to encode many constraints is to use spectral clustering, which remains a developing area. I…
BiPE blends intra-segment and inter-segment encodings for better length extrapolation.
problem Improving length extrapolation in language models.
method Bilevel Positional Encoding (BiPE) that separates intra-segment and inter-segment encodings.
result BiPE enhances length extrapolation across various text modalities.
New neural architectures invariant to sign flips and basis symmetries for graph representation learning.
problem Learning invariant graph representations from eigenvectors.
method SignNet and BasisNet neural architectures that are invariant to sign flips and basis symmetries.
result Proven to be universal, approximating any continuous function of eigenvectors with desired invariances.
Spectro-Riemannian Graph Neural Networks integrate spectral and curvature signals for better graph representation learning.
problem Enhance graph representation learning by leveraging spectral and curvature signals.
method Proposes Spectro-Riemannian Graph Neural Networks (CUSP) that combines spectral and curvature insights.
result Empirical evaluation shows CUSP outperforms state-of-the-art models by up to 5.3%.
Gerbes encode spectral gaps in topological insulators.
problem Capturing spectral gaps in topological insulators.
method Geometric encoding through 'Real' gerbes.
result Gerbes precisely capture spectral gaps.
Signed graphs encode positive (attractive) and negative (repulsive) relations between nodes. We extend spectral clustering to signed graphs via the one-parameter family of Signed Power Mean Laplacians, defined as the matrix power mean of normalized standard and signless Laplacians of positive and negative edges. We pro…
Sign equivariant networks improve model expressiveness for spectral geometric learning.
problem Limited expressiveness of sign invariant models for tasks like graph link prediction.
method Developed sign equivariant neural network architectures based on new analytic sign equivariant polynomials.
result Sign equivariant models achieve theoretical benefits in spectral geometric learning tasks.
Paper explores embedding methods for detecting pseudo-cliques in random graphs, showing limitations and potential.
problem Detecting planted pseudo-cliques in random dot product graphs.
method Adjacency Spectral Embedding (ASE) and Graph Encoder Embedding (GEE).
result These methods can localize pseudo-cliques with additional clean network data, but not without it.
DOODL learns shared spectral dynamics across related dynamical systems.
problem Learning independent dynamical operators for each system limits discovery of shared structure.
method DOODL learns a dictionary of characteristic spectral dynamics on a manifold of related systems.
result DOODL achieves errors one to two orders of magnitude lower than independent operator estimation methods.
Study reveals class disparities in balanced datasets through spectral imbalance.
problem Class disparities in balanced datasets are overlooked despite model performance gaps.
method Developed a theoretical framework and studied 11 encoders to diagnose spectral imbalance.
result Identified spectral imbalance as a source of class disparities in balanced datasets.
STRING improves 2D and 3D position encodings for better performance.
problem Efficient and accurate position encoding for 2D and 3D applications.
method STRING extends Rotary Position Encodings with a unifying theoretical framework, maintaining translation invariance and low computational cost.
result STRING shows substantial gains in open-vocabulary object detection and robotics.
A fast graph embedding method for large graphs.
problem Efficiently embedding large graphs for various applications.
method One-hot graph encoder embedding with linear complexity.
result Graph encoder embedding is approximately normally distributed and converges to its mean.
Randomized positional encodings boost transformer performance on longer sequences.
problem Transformers struggle with generalizing to sequences of arbitrary length.
method Introduced randomized positional encodings that simulate longer sequences and randomly select positions.
result Randomized positional encodings increase test accuracy by 12.0% on average for sequences of unseen length.
We consider an elliptic self-adjoint first order differential operator acting on pairs (2-columns) of complex-valued half-densities over a connected compact 3-dimensional manifold without boundary. The principal symbol of our operator is assumed to be trace-free. We study the spectral function which is the sum of squar…
New method for Transformer models to encode position information without sequential bias.
problem Lack of flexible and learnable position encoding for Transformer models.
method Continuous dynamical model to learn position encoding.
result Consistent improvements over baselines in various NLP tasks.
Symplectic homology matches dual capacities for convex domains.
problem Understanding symplectic capacities and Reeb flows on convex domains.
method Isomorphic filtered symplectic homology to dual singular homology.
result Gutt-Hutchings capacities match spectral invariants for convex domains.
Paper introduces a new method for Transformers with linear complexity.
problem No efficient relative positional encoding for linear Transformer models.
method Stochastic Positional Encoding (SPE) that replaces classical RPE.
result SPE behaves like RPE and performs well on benchmarks.
Language models fail to process hallucinated responses, and this study diagnoses the failure.
problem Language models fail to process hallucinated responses, leading to over-concentration or diffuse attention.
method The study uses forced scoring of benchmark-labeled responses to compute attention shapes and analyze the symmetric component of the degree-normalized attention operator.
result The study proves that every transpose-invariant spectral diagnostic of the attention operator is orientation-blind and bounds the sensitivity of any Lipschitz diagnostic by the asymmetry coefficient \(G\).
New method clusters directed and undirected graphs without losing directional information.
problem Clustering directed graphs due to asymmetry in edge connectivity.
method Generalized Dirichlet Energy (GDE) and generalized spectral clustering (GSC).
result GSC outperforms existing methods in clustering accuracy and robustness.
To a finite, connected, unoriented graph of Betti-number g>=2 and valencies >=3 we associate a finitely summable, commutative spectral triple (in the sense of Connes), whose induced zeta functions encode the graph. This gives another example where non-commutative geometry provides a rigid framework for classification.
Efficiently accelerates attention calculation for Transformers with relative positional encoding.
problem Quadratic complexity of attention in long sequences.
method Kernelized attention with Fast Fourier Transform (FFT) for RPE.
result Achieves O(n log n) time complexity, mitigates training instability, and outperforms other models.
Positive scalar curvature metrics on cobordisms yield only reducible Seiberg-Witten solutions.
problem Counting solutions to Seiberg-Witten equations on cobordisms.
method Constructing families of metrics with positive scalar curvature.
result Irreducible solutions are absent when positive scalar curvature metrics are used.
Transformer struggles with arithmetic length but improves with explicit structure encoding.
problem Transformers fail to generalize length in arithmetic tasks.
method Explicitly encoding structural symmetries via modified number formatting and custom positional encodings.
result Transformer can generalize up to 50-digit numbers without additional data.
Researchers study spectral asymmetry using pseudodifferential projections on the massless Dirac operator.
problem Understanding spectral asymmetry for the massless Dirac operator.
method Constructing a negative order pseudodifferential asymmetry operator from spectral projections.
result Computed the principal symbol of the asymmetry operator, accounting for gauge invariance.
Spectral sparsification improves Laplacian-constrained graph learning.
problem Improving accuracy of Laplacian-constrained graph learning.
method Spectral graph sparsification as a post-estimation operation.
result Improved accuracy of Laplacian-constrained graph learning.
We propose a flexible framework for spectral conversion (SC) that facilitates training with unaligned corpora. Many SC frameworks require parallel corpora, phonetic alignments, or explicit frame-wise correspondence for learning conversion functions or for synthesizing a target spectrum with the aid of alignments. Howev…
Study surfaces with nonnegative curvature in spectral sense, proving inequalities and bounds.
problem Closed orientable surfaces with nonnegative curvature in spectral sense.
method Spectral condition and associated conformal metrics to prove inequalities and bounds.
result Isoperimetric inequalities, area growth theorems, and diameter bounds for surfaces.
The paper proves conditions for Kähler and Riemannian manifolds to be simply connected.
problem Conditions for Kähler and Riemannian manifolds to be simply connected.
method Spectral positivity assumptions for Kähler manifolds and a specific spectral positivity assumption for Riemannian manifolds.
result Compact Kähler manifolds and Riemannian manifolds under the specified spectral positivity assumptions are simply connected.
For graphs generated from stochastic blockmodels, adjacency spectral embedding is asymptotically consistent. Further, adjacency spectral embedding composed with universally consistent classifiers is universally consistent to achieve the Bayes error. However when the graph contains private or sensitive information, trea…
Improved spectral projection estimates on manifolds of non-positive curvature.
problem Estimating spectral projections on manifolds with non-positive curvature.
method New spectral projection estimates, including sharp ones for tori, using pointwise estimates and microlocal L2oLqc Kakeya-Nikodym estimates. result Stronger and more precise spectral projection estimates, including new sharp estimates for tori.
Define mass invariant for asymptotically hyperbolic 3-manifolds with toroidal infinity.
problem Define mass invariant for asymptotically hyperbolic 3-manifolds with toroidal infinity.
method Define mass invariant for asymptotically hyperbolic 3-manifolds with toroidal infinity.
result Define mass invariant for asymptotically hyperbolic 3-manifolds with toroidal infinity.
New method efficiently learns positive-definite curvature for neural nets.
problem Efficiently learn positive-definite curvature for neural net training.
method Spectral-factorized positive-definite curvature learning approach.
result Efficiently applies arbitrary matrix roots and generic curvature learning.
Positive weights improve kernel quadrature's accuracy.
problem Improving kernel quadrature weights to be positive and stable.
method Using convex geometry to approximate the kernel mean embedding with positive weights.
result Positive weights lead to improved kernel quadrature bounds with Monte-Carlo-beating rates.
This work proposes a novel autoencoder for fusing visible and infrared images.
problem Challenging task to combine spatial and spectral information from visible and infrared images.
method Spatially constrained adversarial autoencoder with residual architecture and adversarial regularizer.
result Generates a more realistic fused image with enhanced spatial and spectral information.
The paper bounds bandwidth and focal radius for manifolds with positive isotropic curvature.
problem Bounding bandwidth and focal radius for manifolds with positive isotropic curvature.
method Using spectral properties of a twisted de Rham-Hodge operator.
result Upper bounds on bandwidth and focal radius are derived for hypersurfaces in PIC manifolds.
In this paper we consider a modified version of the classical optimal dividends problem of de Finetti in which the dividend payments subject to a penalty at ruin. We assume that the risk process is modeled by a general spectrally positive Levy process before dividends are deducted. Using the fluctuation theory of spect…
Paper proposes LCP for structural encodings, outperforming existing methods.
problem Improving Graph Neural Networks performance through effective structural encodings.
method Geometric perspective, Local Curvature Profiles (LCP) for structural encodings, combining with global positional encodings, comparing with rewiring techniques.
result LCP significantly outperforms existing structural encodings and combining LCP with global positional encodings improves performance.
The paper validates Stokes' theorem for differential subcomplexes in positively graded Lie groups.
problem Validating Stokes' theorem for differential subcomplexes in positively graded Lie groups.
method Introducing geometric conditions and spectral complexes to recover Stokes' theorem on locally smooth intrinsic graphs.
result Stokes' theorem holds for Rumin complex and new spectral complexes on Carnot groups.
Study spectral flow for Callias operators to prove obstructions to positive scalar curvature.
problem Prove obstructions to positive scalar curvature on manifolds.
method Use spectral flow and odd K-cowaist to derive obstructions.
result Infinite odd K-cowaist is an obstruction to the existence of PSC metrics.
The paper finds optimal threshold strategies for insurance companies with a positive terminal value at creeping ruin.
problem Optimizing dividend payments in an insurance company's surplus process with a positive terminal value at creeping ruin.
method Using fluctuation theory, the paper derives explicit formulas for the objective function and shows the optimality of threshold strategies.
result Threshold strategies are optimal for the dividend optimization problem under certain conditions.