Study spectral learning for odeco tensors, addressing initialization bottlenecks.
problem Recovering orthogonally decomposable tensors under noise.
method Investigates perturbation bounds, non-convex optimization, and initialization strategies.
result Initialization is the main bottleneck for efficient algorithms.
S-GAI initializes MLPs using spectral geometry from data, improving performance.
problem Lack of guidance on initial weights encoding data geometry.
method S-GAI uses SVD to estimate spectral class geometry, initializing MLPs from training data.
result S-GAI-initialized MLPs start from a more informative hidden state and achieve comparable accuracy.
Paper proposes AMP with spectral initialization for robust signal estimation.
problem Signal estimation from generalized linear model measurements with correlated initialization.
method Approximate message passing (AMP) with spectral initialization.
result Characterization of AMP with spectral initialization in high-dimensional limit.
Spectral normalization stabilizes GANs by controlling gradient explosion and vanishing.
problem Stability and sample quality issues in GAN training.
method Spectral normalization controls gradient explosion and vanishing, improving GAN training stability and sample quality.
result Bidirectional Scaled Spectral Normalization (BSSN) outperforms standard spectral normalization in sample quality and training stability.
Gradient descent with small random init mimics spectral methods for low-rank matrix recovery.
problem Reconstructing a low-rank matrix from few measurements.
method Gradient descent with small random initialization followed by a few iterations.
result Gradient descent from small random init converges to a well-generalizing solution.
Optimal spectral estimators and AMP combine for efficient weak recovery in orthogonally invariant GLMs.
problem Parameter estimation from generalized linear models with complex correlation structures.
method Spectral initialization and approximate message passing (AMP) algorithm.
result Established rigorous performance guarantees for spectral initialization and AMP.
Phase retrieval refers to the problem of recovering real- or complex-valued vectors from magnitude measurements. The best-known algorithms for this problem are iterative in nature and rely on so-called spectral initializers that provide accurate initialization vectors. We propose a novel class of estimators suitable fo…
Improved performance of factorized neural layers through spectral initialization and Frobenius decay.
problem Improving the performance of factorized neural layers in various deep learning contexts.
method Spectral initialization and Frobenius decay for initialization and regularization.
result Spectral initialization and Frobenius decay lead to improved performance across multiple deep learning settings.
This work analyzes a two-stage algorithm for single index models, showing precise asymptotics of gradient descent.
problem Learning single index models with non-convex optimization.
method Spectral initialization followed by gradient descent, with detailed analysis of dynamics and asymptotics.
result Gradient descent converges to long-time fixed points in the large system limit, representing mean field behavior.
New method for initializing low-rank neural networks improves performance.
problem Training low-rank neural networks efficiently and accurately.
method Inspired by function approximation, proposes a novel low-rank initialization framework.
result Demonstrates significant gap between spectral and low-rank initialization approaches.
Optimal spectral initializers impact phase retrieval phase transitions.
problem Understanding the limits of phase retrieval algorithms.
method Developed Random duality theory (RDT) to characterize optimal spectral initializers.
result Optimal spectral initializers can fall into flat regions of the phase retrieval manifold, making phase retrieval difficult.
Paper revisits graph-CNNs using Laplace-Beltrami spectral filters and polynomials.
problem Improving spectral graph convolutional neural networks (graph-CNNs).
method Developed Laplace-Beltrami CNN (LB-CNN) by replacing graph Laplacian with LB operator and approximating spectral filters using Chebyshev, Laguerre, and Hermite polynomials.
result Classification accuracy of LB-CNN is not dependent on the type of polynomials or operators.
Near-optimal sample complexity for phase retrieval with generative priors.
problem Phase retrieval with magnitude-only measurements and sparse signals.
method Near-optimal sample complexity with i.i.d. Gaussian measurements and generative models.
result O(k log L) samples suffice for phase retrieval with generative priors.
We summarize recent results initiating spectral analysis on pseudo-Riemannian locally symmetric spaces Γ\G/H, beyond the classical setting where H is compact (e.g. theory of automorphic forms for arithmetic Γ) or Γ is trivial (e.g. Plancherel-type formula for semisimple symmetric spaces).
Wedge Sampling improves tensor completion with nearly-linear sample complexity.
problem Efficiently completing low-rank tensors from a subset of entries.
method Non-adaptive wedge sampling to promote structured connections in tensor completion.
result Polynomial-time algorithms achieve weak and exact recovery with nearly linear sample complexity.
We study a spectral initialization method that serves a key role in recent work on estimating signals in nonconvex settings. Previous analysis of this method focuses on the phase retrieval problem and provides only performance bounds. In this paper, we consider arbitrary generalized linear sensing models and present a …
Describes the relationship between two spectral sequences and their joint refinement.
problem Computing the cohomology of a group or space using spectral sequences.
method Joint tri-graded refinement of the Leray--Serre and Eilenberg--Moore spectral sequences.
result One of the spectral sequences always degenerates from its second page, and the other satisfies a local-to-global property.
Proves positive mass theorem for hyperbolic manifolds with ends.
problem Establishing positive mass theorem for complex initial data sets.
method Used spectral PSC, Jang equation, and quantitative shielding theorem.
result Proved positive mass theorem for asymptotically hyperbolic manifolds.
A new convolutional spectral kernel network learns hierarchical and local features.
problem Lack of deep learning in non-stationary spectral kernels.
method Introduces convolutional filters and deep architectures into non-stationary spectral kernels, derives generalization error bounds, and introduces regularizers.
result Validated the effectiveness of the convolutional spectral kernel network on real-world datasets.
Recent work has shown that tight concentration of the entire spectrum of singular values of a deep network's input-output Jacobian around one at initialization can speed up learning by orders of magnitude. Therefore, to guide important design choices, it is important to build a full theoretical understanding of the spe…
Financial frequency combs emerge from macroeconomic long-range memory.
problem Financial economy's long-run cyclic structure
method Incommensurate fractional-order financial model
result Frequency comb structure in steady-state spectrum
New cutoff phenomenon found for geodesic paths on hyperbolic manifolds.
problem Understanding the cutoff phenomenon for geodesic paths on hyperbolic manifolds.
method Spectral strategy and detailed spectral analysis of the spherical mean operator.
result Geodesic paths on compact hyperbolic manifolds exhibit cutoff for spatially localized initial conditions.
Consider the problem of estimating a low-rank matrix when its entries are perturbed by Gaussian noise. If the empirical distribution of the entries of the spikes is known, optimal estimators that exploit this knowledge can substantially outperform simple spectral approaches. Recent work characterizes the asymptotic acc…
Formula for spectral flow connects manifold properties to index theorem.
problem Establishing a formula for spectral flow on manifolds.
method Reduction to Atiyah-Patodi-Singer index theorem for manifolds with boundary.
result Formula for spectral flow expressed in manifold and connection properties.
Domains in infinite jets present the simplest class of diffieties with boundary. In this note some basic elements of geometry of these domains are introduced and an analogue of the C-spectral sequence in this context is studied. This, in particular, allows cohomological interpretation and analysis of initial data, boun…
Paper explores SNN for learning spectral geometric info from data.
problem Challenges in applying traditional eigensolvers to big data.
method Introduces Spectral Neural Networks (SNN) as an alternative.
result Investigates tradeoffs and optimization landscape of SNN.
New method predicts neural network performance using free probability theory.
problem Stability and performance prediction of feed-forward neural networks.
method Free Probability Theory and homotopy method for Jacobian spectral density computation.
result FPT metrics correlate highly with final test accuracies of neural networks.
Efficiently samples multimodal distributions using data-based initialization.
problem Sampling multimodal distributions with limited samples.
method Data-based initialization for Markov chains with spectral gap.
result Efficiently generates samples close to stationary distribution.
Over the past decade there has been considerable interest in spectral algorithms for learning Predictive State Representations (PSRs). Spectral algorithms have appealing theoretical guarantees; however, the resulting models do not always perform well on inference tasks in practice. One reason for this behavior is the m…
The paper studies non-integer curvature flows and proves convergence to spheres under specific conditions.
problem Analyzing the convergence of non-integer curvature flows on rotationally symmetric surfaces.
method Spectral theory of singular Sturm-Liouville operators to construct an eigenbasis and prove convergence.
result The flow converges to a round sphere if the focal points coincide at the poles, otherwise to a non-round Hopf sphere.
Global stability proved for Navier-Stokes equations on hyperbolic space.
problem Stability of the Navier-Stokes equations on hyperbolic space.
method Proved global stability with exponential decay rate for small initial data.
result Exponential decay rate of $μλ_\Def^{(3)}$ for Navier-Stokes equations on hyperbolic space.
Ultra-wideband (UWB) radar systems nowadays typical operate in the low frequency spectrum to achieve penetration capability. However, this spectrum is also shared by many others communication systems, which causes missing information in the frequency bands. To recover this missing spectral information, we propose a gen…
Simple AMP algorithm robust to adversarial corruption.
problem Robust approximate message passing in spiked matrix models.
method Spectral pre-processing combined with robust spectral initialization.
result AMP output is close to correct for corrupted data.
This paper applies the Extreme-Value (EV) Generalised Pareto distribution to the extreme tails of the return distributions for the S&P500, FT100, DAX, Hang Seng, and Nikkei225 futures contracts. It then uses tail estimators from these contracts to estimate spectral risk measures, which are coherent risk measures that r…
The decorated hypercube found in the construction of Khovanov homology for links is an example of a Boolean lattice equipped with a presheaf of modules. One can place this in a wider setting as an example of a coloured poset, that is to say a poset with a unique maximal element equipped with a presheaf of modules. In t…
Study heat kernel on manifolds with fibred boundary metrics.
problem Analyzing spectral problems in manifolds with fibred boundary metrics.
method Construct heat kernel as polyhomogeneous conormal distribution.
result Fundamental step towards analysis of Ray-Singer torsion, eta-invariants and index theorems.
We analyze DMs using spectral methods to design effective noise schedules.
problem Lack of theoretical foundation for synthesis process decisions in DMs.
method Introduced a frequency response perspective based on Gaussianity assumption.
result Proposed a spectral transfer function to understand DM inference process.
Dimensionality reduction (DR) methods have attracted extensive attention to provide discriminative information and reduce the computational burden of the hyperspectral image (HSI) classification. However, the DR methods face many challenges due to limited training samples with high dimensional spectra. To address this …
Neural networks learn spectral representations for group composition.
problem Understanding structured emergence in neural network training.
method Lifting gradient flow to Fourier domain, proving convergence to irreducible representations.
result Neurons converge to single irreducible representations, cross-layer coefficients align.
A new model for dynamic covariance recovery in neuroimaging data.
problem Estimating time-varying covariances in high-dimensional neuroimaging data.
method Nonconvex factorization into sparse spatial and smooth temporal components, combined with spectral initialization and gradient descent.
result The proposed method achieves linear convergence and superior performance compared to existing approaches.
Magnitude of geometric shapes studied for smooth manifolds, revealing spectral geometry insights.
problem Understanding the geometric significance of Leinster's magnitude for smooth manifolds.
method Investigation of magnitude function for various distance functions, including submanifolds and Riemannian manifolds, with asymptotic analysis in the limit.
result Magnitude function is well-defined and meromorphically continued for large distances, revealing volume, surface area, and curvature integrals.
AMP method reconstructs rank-one matrices from noisy data efficiently.
problem Reconstructing rank-one matrices with prior structural information from noisy observations.
method Approximate Message Passing (AMP) with random initialization.
result AMP from random initialization converges rapidly and globally.
In this work, the possibility of clustering correlated random variables was examined, both because of their mutual similarity and because of their similarity to the principal components. The k-means algorithm and spectral algorithms were used for clustering. For spectral methods, the similarity matrix was both the matr…
Gradient descent converges geometrically to optimal self-attention parameters.
problem Training softmax self-attention layers for linear regression.
method Structure-aware gradient descent with preconditioner and regularizer.
result Gradient descent converges geometrically to global minima.
This work analyzes how different layers in deep neural networks contribute to generalization error.
problem Understanding the role of each layer in deep neural networks for generalization.
method Spectral analysis, Neural Tangent Kernel, Hermite polynomials, Spherical Harmonics.
result Initial layers in deep neural networks have a larger bias towards high-frequency functions.
Hyperspectral images (HSI) contain a wealth of information over hundreds of contiguous spectral bands, making it possible to classify materials through subtle spectral discrepancies. However, the classification of this rich spectral information is accompanied by the challenges like high dimensionality, singularity, lim…
We consider the problem of recovering low-rank matrices from random rank-one measurements, which spans numerous applications including covariance sketching, phase retrieval, quantum state tomography, and learning shallow polynomial neural networks, among others. Our approach is to directly estimate the low-rank factor …
The paper examines how gradient descent stabilizes low-rank matrix factorization in noisy conditions.
problem Stability of low-rank implicit regularization in perturbed deep matrix factorization.
method Derives spectral conditions for gradient descent to exhibit a low-rank phase in noiseless settings and analyzes perturbed dynamics.
result Gradient descent converges to a low-rank solution under perturbation, with explicit dependence on perturbation size.