Constructs coordinate systems from spectral curve sheaves.
problem Creating coordinate systems from spectral curve sheaves.
method Finite-gap integration methods for orthogonal curvilinear coordinates.
result Constructs coordinate systems over reducible spectral curves.
Muon optimizer simplifies matrix optimization with spectral orthogonalization.
problem Matrix optimization challenges, especially with large condition numbers.
method Simplified Muon optimizer using spectral orthogonalization of gradients.
result Simplified Muon converges linearly with independent scalar sequences, outperforming gradient descent and Adam.
New method combines spectral and sparse methods for Gaussian processes.
problem Efficiently fitting Gaussian processes to large datasets.
method Orthogonally decoupled variational Fourier features.
result Competitive performance on synthetic and real-world data.
SpecNet2 improves spectral embedding without orthogonalization, achieving better performance and efficiency.
problem Improving spectral embedding methods for better performance and efficiency.
method Optimizes an equivalent objective of the eigen-problem without orthogonalization, allowing separate row and column sampling.
result Local and global convergence of the new objective using batch-based gradient descent is proven, and improved performance and efficiency are demonstrated on simulated and image datasets.
Method constructs orthogonal curvilinear coordinates in constant curvature spaces.
problem Creating orthogonal coordinates in spaces of constant curvature.
method Modification of Krichever's method for Euclidean space, applied to constant curvature spaces.
result Examples of orthogonal coordinate systems on the sphere and hyperbolic plane constructed.
Optimal spectral estimators and AMP combine for efficient weak recovery in orthogonally invariant GLMs.
problem Parameter estimation from generalized linear models with complex correlation structures.
method Spectral initialization and approximate message passing (AMP) algorithm.
result Established rigorous performance guarantees for spectral initialization and AMP.
Spectral methods achieve near-optimal performance in orthogonal and permutation group synchronization.
problem Recovering group elements from pairwise measurements in computer vision.
method Spectral methods applied with the leave-one-out technique.
result Near-optimal performance bounds for orthogonal and permutation group synchronization established.
Spectral method for joint community detection and group synchronization.
problem Jointly detecting communities and synchronizing orthogonal groups in graphs.
method Spectral decomposition followed by CPQR factorization.
result Near-optimal guarantees for exact and stable recovery of cluster memberships and orthogonal transforms.
Study spectral flow on a warped cylinder with special boundary conditions.
problem Analyzing spectral flow on a warped cylinder with specific boundary conditions.
method Complexifying the twisting bundle, diagonalizing the orthogonal twist, and regrouping conjugate and reflection-paired blocks.
result Explicit formula for RO(O(2))-valued spectral flow, refining ordinary spectral flow. In this paper, we propose a scalable algorithm for spectral embedding. The latter is a standard tool for graph clustering. However, its computational bottleneck is the eigendecomposition of the graph Laplacian matrix, which prevents its application to large-scale graphs. Our contribution consists of reformulating spect…
We give a geometric characterisation of the topological invariants associated to SO(m,m+1)-Higgs bundles through KO-theory and the Langlands correspondence between orthogonal and symplectic Hitchin systems. By defining the split orthogonal spectral data, we obtain a natural grading of the moduli space of SO(m,m+1)-Higg…
Deterministic bounds for tensor singular values and vectors, differing from matrix cases.
problem Spectral learning of higher-order orthogonally decomposable tensors.
method Deterministic perturbation bounds for singular values and vectors of orthogonally decomposable tensors.
result Perturbation affects each essential singular value/vector in isolation, independent of multiplicity and distance from other singular values.
A recent theoretical analysis shows the equivalence between non-negative matrix factorization (NMF) and spectral clustering based approach to subspace clustering. As NMF and many of its variants are essentially linear, we introduce a nonlinear NMF with explicit orthogonality and derive general kernel-based orthogonal m…
We study the limiting case of the Krichever construction of orthogonal curvilinear coordinate systems when the spectral curve becomes singular. We show that the case when the curve is reducible and all its irreducible components are rational curves the construction procedure reduces to solving systems of linear equatio…
We examine Higgs bundles for non-compact real forms of SO(4,C) and the isogenous complex group SL(2,C)XSL(2,C). This involves a study of non-regular fibers in the corresponding Hitchin fibrations and provides interesting examples of non-abelian spectral data.
New framework detects directional influence in multivariate time series.
problem Detecting directional influence in multivariate time series.
method Order-constrained spectral non-invariance.
result Unique diagnostic functional for directional influence.
We study the problem of approximating orthogonal matrices so that their application is numerically fast and yet accurate. We find an approximation by solving an optimization problem over a set of structured matrices, that we call extended orthogonal Givens transformations, including Givens rotations as a special case. …
Through Cayley and Langlands type correspondences, we give a geometric description of the moduli spaces of real orthogonal and symplectic Higgs bundles of any signature in the regular fibres of the Hitchin fibration. As applications of our methods, we complete the concrete abelianization of real slices corresponding to…
Pion optimizes LLMs by preserving weight matrix singular values.
problem Training large language models (LLMs) with standard optimizers leads to unstable weight matrices.
method Pion uses orthogonal transformations to update weight matrices, preserving their singular values.
result Pion offers a stable alternative to standard optimizers for LLM pretraining and finetuning.
New spectral functionals related to noncommutative residue and torsion Dirac operators.
problem Spectral functionals and Dirac operators with torsion.
method Noncommutative residue and Dirac operators with torsion.
result Extension of spectral functionals to noncommutative realm with torsion.
Study optimizes estimation of orthogonal and rotation matrices from noisy data.
problem Estimating orthogonal and rotation matrices from noisy data.
method Iterative polar decomposition algorithm initialized by spectral methods.
result Algorithm achieves optimal error rate of $(1+o(1)) rac{σ^2 d(d-1)}{2np}$.
Study spectral learning for odeco tensors, addressing initialization bottlenecks.
problem Recovering orthogonally decomposable tensors under noise.
method Investigates perturbation bounds, non-convex optimization, and initialization strategies.
result Initialization is the main bottleneck for efficient algorithms.
Corrected whitening restores orthogonality in high-dimensional spherical Gaussian mixtures.
problem In high-dimensional data, standard whitening fails to preserve orthogonality of mixture means.
method Derived exact limits for whitened means dot products using random matrix theory, constructed a corrected whitening matrix.
result Corrected whitening allows for improved estimation of spherical Gaussian mixtures in the large-dimensional regime.
A recent strategy to circumvent the exploding and vanishing gradient problem in RNNs, and to allow the stable propagation of signals over long time scales, is to constrain recurrent connectivity matrices to be orthogonal or unitary. This ensures eigenvalues with unit norm and thus stable dynamics and training. However …
Defines and parametrizes sl(2)-type singular fibres in symplectic and odd orthogonal Hitchin systems.
problem Characterizing and understanding singular fibres in Hitchin systems.
method Stratification by semi-abelian spectral data, study of irreducible components, global description of degenerations.
result Extension of Langlands duality to sl(2)-type Hitchin fibres. Shampoo achieves higher token efficiency than Muon in language models.
problem Understanding the relationship and relative data efficiency of Shampoo and Muon compared to Adam and Signum.
method Extensive experiments on language models, demonstrating Shampoo's higher efficiency and decomposing its updates.
result Shampoo's benefits are attributed to its application to weight matrices, challenging interpretations based on variance adaptation and whitening.
This work proves the asymptotic freeness of layerwise Jacobians in MLPs with Haar orthogonal matrices.
problem Proving the asymptotic freeness of layerwise Jacobians in multilayer perceptrons (MLPs).
method Replacing each layer's parameter matrix with itself multiplied by a Haar orthogonal matrix, and using the invariance of the MLP.
result Proves the asymptotic freeness of layerwise Jacobians in MLPs with Haar orthogonal matrices.
A robust method for decomposing spectral peaks robust to distortion and interference.
problem Decomposing spectral peaks in the presence of distortion and interference.
method Optimizing a nonparametric approach using pseudo-symmetric functions with nonincreasing behavior.
result Decomposed spectral peaks show pseudo-orthogonal behavior and power preserving equality.
Proves nearby Lagrangian cocores are homotopically rigid in certain dimensions.
problem Homotopy rigidity of nearby Lagrangian cocores in Weinstein sectors.
method Spectral wrapped Donaldson-Fukaya category with orthogonal group coefficients.
result Inclusion followed by retract and quotient is null-homotopic.
Nonnegative matrix factorization (NMF) is a popular method for audio spectral unmixing. While NMF is traditionally applied to off-the-shelf time-frequency representations based on the short-time Fourier or Cosine transforms, the ability to learn transforms from raw data attracts increasing attention. However, this adds…
Wasserstein-GANs have been introduced to address the deficiencies of generative adversarial networks (GANs) regarding the problems of vanishing gradients and mode collapse during the training, leading to improved convergence behaviour and improved image quality. However, Wasserstein-GANs require the discriminator to be…
New algorithm finds sparse matrices on Stiefel manifold for optimisation.
problem Finding sparse matrices on Stiefel manifold for optimisation.
method Modified Orthogonal Iteration algorithm for sparse global optimality.
result Proposed method finds globally optimal sparse Stiefel matrices.
The paper analyzes and improves a deep learning optimization technique using matrix gradient orthogonality.
problem Improving deep learning training through more effective optimization methods.
method Develops a stochastic non-Euclidean trust-region gradient method for deep learning optimization.
result Proves state-of-the-art convergence results for the proposed algorithm in various scenarios.
Study uses random matrix theory to improve tensor approximation accuracy.
problem Improving tensor approximation accuracy in the presence of noise.
method Random matrix theory applied to tensor unfoldings.
result Characterizes spectral behavior of tensor unfoldings and predicts reconstruction performance.
Adaptive orthogonalization of data for clustering and visualization.
problem Clustering and visualization of data with high specificity.
method Adaptive orthogonalization process using Gromov-Wasserstein feedback.
result Method refines orthogonality of data to achieve high specificity clustering.
In the past decades, spectral clustering (SC) has become one of the most effective clustering algorithms. However, most previous studies focus on spectral clustering tasks with a fixed task set, which cannot incorporate with a new spectral clustering task without accessing to previously learned tasks. In this paper, we…
Unified analysis of multilabel Fisher discriminants with improved dimensionality and robustness.
problem Improving discriminant analysis for multilabel classification with enhanced dimensionality and robustness.
method Unified theoretical analysis of multilabel Fisher discriminants with algebraic and statistical guarantees.
result Unified characterization of multilabel Fisher objectives and their equivalence under orthogonality constraints.
Random Matrix Theory explains loss surface Hessians in neural networks.
problem Understanding the loss surfaces of neural networks.
method Investigation of local spectral statistics of neural network Hessians.
result Excellent agreement with Gaussian Orthogonal Ensemble statistics.
Unified analysis of multilabel Fisher discriminants with improved dimensionality and robustness.
problem Improving discriminant analysis for multilabel classification with enhanced dimensionality and robustness.
method Unified algebraic and statistical analysis of multilabel Fisher discriminants with Stiefel orthogonality constraints.
result Equivalence of four Fisher objectives under the Stiefel constraint and improved discriminant dimensionality.
NS-RGS improves orthogonal group synchronization with faster convergence.
problem Orthogonal group synchronization from pairwise measurements.
method Newton-Schulz iteration for Riemannian gradient optimization.
result NS-RGS achieves linear convergence and near-optimal accuracy.
Extends spectral number variance convergence to random matrix ensembles for twisted Laplacians.
problem Spectral number variance convergence for twisted Laplacians and Dirac operators.
method Extends Rudnick's approach to Gaussian ensembles for twisted Laplacians and Dirac operators.
result Convergence to Gaussian ensembles for twisted Laplacians and Dirac operators.
Recently mean field theory has been successfully used to analyze properties of wide, random neural networks. It gave rise to a prescriptive theory for initializing feed-forward neural networks with orthogonal weights, which ensures that both the forward propagated activations and the backpropagated gradients are near $…
In this work a novel method to quantify spectral ergodicity for random matrices is presented. The new methodology combines approaches rooted in the metrics of Thirumalai-Mountain (TM) and Kullbach-Leibler (KL) divergence. The method is applied to a general study of deep and recurrent neural networks via the analysis of…
Constructing eigenfunctions for finite-time singularities in Lagrangian mean curvature flow
problem Constructing eigenfunctions for finite-time singularities in Lagrangian mean curvature flow
method Constructing eigenfunctions for finite-time singularities in Lagrangian mean curvature flow
result Identifying the lowest eigenfunction with the scaling mode of the special Lagrangian desingularization
We present a method based on the orthogonal symmetric non-negative matrix tri-factorization of the normalized Laplacian matrix for community detection in complex networks. While the exact factorization of a given order may not exist and is NP hard to compute, we obtain an approximate factorization by solving an optimiz…
State-of-the-art algorithms for sparse subspace clustering perform spectral clustering on a similarity matrix typically obtained by representing each data point as a sparse combination of other points using either basis pursuit (BP) or orthogonal matching pursuit (OMP). BP-based methods are often prohibitive in practic…
Clustering of data sets is a standard problem in many areas of science and engineering. The method of spectral clustering is based on embedding the data set using a kernel function, and using the top eigenvectors of the normalized Laplacian to recover the connected components. We study the performance of spectral clust…
Study smooth linear statistics on random covers of hyperbolic surfaces, showing central limit and variance results.
problem Analyzing fluctuations and energy variance of random covers of compact hyperbolic surfaces.
method Examining fluctuations in a small energy window around a fixed energy level, considering the variance of a typical surface, using a double limit where n and L go to infinity. result Distribution of fluctuations tends to a Gaussian with variance of GOE/GUE, and energy variance of a typical random n-cover is that of GOE/GUE.