Exact spectral norm regularization improves neural network generalization.
problem Improving neural network generalization while protecting against noise.
method Exact spectral norm regularization of the Jacobian.
result Improved generalization performance compared to previous methods.
Spectral regularization simplifies sequence models by focusing on grammatical simplicity.
problem Sequence modeling challenges in learning tasks.
method Introduces spectral regularization based on Hankel matrices and trace norm, addressing bi-infinite matrices with an unbiased estimator.
result Demonstrates spectral regularization's potential benefits on Tomita grammars.
This paper uses the relationship between graph conductance and spectral clustering to study (i) the failures of spectral clustering and (ii) the benefits of regularization. The explanation is simple. Sparse and stochastic graphs create a lot of small trees that are connected to the core of the graph by only one edge. G…
Dual regularized graph Laplacian improves spectral clustering for community detection.
problem Detecting clusters in networks with improved spectral clustering methods.
method Proposes dual regularized graph Laplacian for three spectral clustering approaches.
result Theoretical analysis shows DRSC and DRSLIM yield stable consistent community detection.
Regularization of the classical Laplacian matrices was empirically shown to improve spectral clustering in sparse networks. It was observed that small regularizations are preferable, but this point was left as a heuristic argument. In this paper we formally determine a proper regularization which is intimately related …
Fiedler regularization uses spectral graph theory to improve neural network performance.
problem Improving neural network performance by penalizing weights based on connectivity.
method Uses the Fiedler value of the neural network's graph as a regularization tool, providing theoretical and computational methods.
result Demonstrates Fiedler regularization's effectiveness in improving neural network performance.
We investigate the generalizability of deep learning based on the sensitivity to input perturbation. We hypothesize that the high sensitivity to the perturbation of data degrades the performance on it. To reduce the sensitivity to perturbation, we propose a simple and effective regularization method, referred to as spe…
Spectral regularization improves learning over combinatorial spaces with limited data.
problem Learning pseudo-Boolean functions with scarce labeled data.
method Regularizing the spectral representation of learned functions using the L_1 norm.
result Regularization allows for data-frugal learning and achieves statistically optimal generalization performance.
Improved MMD test for non-Euclidean data with spectral regularization.
problem Inefficient and impractical MMD goodness-of-fit tests for non-Euclidean data.
method Spectral regularization of MMD test, extending results to general cases.
result Minimax optimal test for non-Euclidean data with appropriate regularization.
Developed a framework for designing filters in spectral GCNNs with improved performance.
problem Designing effective filters for spectral GCNNs with regularization properties.
method Exploring regularization properties of graph Laplacian and proposing a generalized framework for filter design.
result New filters derived from the framework outperform state-of-the-art techniques in semi-supervised node classification.
Despite excellent progress in recent years, mode collapse remains a major unsolved problem in generative adversarial networks (GANs).In this paper, we present spectral regularization for GANs (SR-GANs), a new and robust method for combating the mode collapse problem in GANs. Theoretical analysis shows that the optimal …
Spectral embedding is a popular technique for the representation of graph data. Several regularization techniques have been proposed to improve the quality of the embedding with respect to downstream tasks like clustering. In this paper, we explain on a simple block model the impact of the complete graph regularization…
Novel model captures high-dimensional copulas with spectral dynamics and regularization.
problem Modeling time-varying, asymmetric, tail-dependent copulas in high dimensions.
method Score-driven dynamics for eigenvalues, non-linear shrinkage for biases, parsimonious and scalable.
result Model outperforms recent alternatives in capturing co-movements and diversification potential.
Random feature approximation speeds up spectral methods and improves learning rates.
problem Improving the efficiency and generalization of spectral methods in large-scale algorithms.
method Combining random feature approximation with spectral regularization methods.
result Optimal learning rates for estimators over various regularity classes, including those not in the RKHS.
Estimates Gaussian location model with ridge regularization, comparing variational and spectral methods.
problem Estimating parameters in Gaussian location model with regularization.
method Ridge-regularized log-density-ratio estimation, variational and spectral approaches.
result Regularized variational estimator has lower risk with many observations, spectral estimator with fewer observations.
Improved LDA method for better classification and dimensionality reduction.
problem Improving linear discriminant analysis for better classification performance.
method Integrates spectrally-corrected covariance matrix and regularized discriminant analysis.
result SRLDA has a linear classification global optimal solution under spiked model assumption.
A new method for community detection in networks is presented.
problem Community detection in network analysis.
method Mixed regularized spectral clustering (Mixed-RSC) based on the regularized Laplacian matrix.
result The method is asymptotically consistent under mild conditions.
New spectral clustering method using LASSO regularization for robust graph partitioning.
problem Lack of theoretical guarantees for spectral clustering on general graph models.
method 1-spectral clustering on a new random model with LASSO regularization.
result Effective and robust to small noise perturbations, validated by simulations and real data.
Study reveals learning curves and benign overfitting in spectral algorithms for large dimensions.
problem Understanding learning curves and benign overfitting in spectral algorithms for large-dimensional data.
method Analysis of learning curves and benign overfitting in spectral algorithms for inner-product kernels on the sphere and general domains.
result Characterization of three distinct regimes: over-regularized, under-regularized, and interpolation regimes, revealing benign overfitting across both under-regularized and interpolation regimes.
Paper tackles functional linear regression using spectral algorithms with discrete observations.
problem Functional linear regression problem with discretely observed data.
method Combines distributed spectral algorithms with Sobolev kernels for regularization.
result Derives matching upper and lower bounds for convergence in Sobolev norm.
Estimates linear model from noisy covariates and instruments using spectral regularization.
problem Estimating a linear model from many noisy covariates and instruments.
method Two-stage least squares with spectral regularization of canonical correlations.
result Upper and lower bounds on estimation error, proving optimality of the method with noisy data.
We study a regularizer which is defined as a parameterized infimum of quadratics, and which we call the box-norm. We show that the k-support norm, a regularizer proposed by [Argyriou et al, 2012] for sparse vector prediction problems, belongs to this family, and the box-norm can be generated as a perturbation of the fo…
SC-Net learns interpretable filters for inverse problems, achieving optimal convergence and super-resolution.
problem Solving ill-posed inverse problems with effective regularization and interpretability.
method SC-Net operates in the spectral domain, learning a pointwise adaptive filter function based on signal-to-noise ratio.
result SC-Net achieves optimal convergence rate and zero-shot super-resolution, matching theoretical bounds.
Dimensionality reduction (DR) methods have attracted extensive attention to provide discriminative information and reduce the computational burden of the hyperspectral image (HSI) classification. However, the DR methods face many challenges due to limited training samples with high dimensional spectra. To address this …
Accelerates optimal transport computation by 10x with spectral insights.
problem Exponential slow-down of convergence in Entropic Optimal Transport as regularization weakens.
method Spectral insights and spectral warm-start strategy to mitigate convergence issues.
result Faster convergence compared to the reference method Sinkhorn algorithm.
We consider spectral clustering algorithms for community detection under a general bipartite stochastic block model (SBM). A modern spectral clustering algorithm consists of three steps: (1) regularization of an appropriate adjacency or Laplacian matrix (2) a form of spectral truncation and (3) a k-means type algorithm…
Graph convolutional networks fail to use eigenvectors beyond the first, unlike spectral embedding.
problem Understanding when graph convolutional networks fail compared to spectral embedding.
method Presented a simple generative model to illustrate failure.
result Graph convolutional networks fail to use eigenvectors beyond the first in certain graphs.
A new method for nonstationary Gaussian processes using Fourier features.
problem Efficient simulation of nonstationary Gaussian processes with high-dimensional distributions.
method Discretizes the spectral representation of nonstationary processes, avoiding probability measure assumptions.
result An efficient low-rank approximation of nonstationary spectral densities, consistent and positive semi-definite.
Improved spectral clustering for community detection in networks.
problem Community detection in networks.
method Improved spectral clustering (ISC) based on k-means clustering on weighted eigenvectors of a regularized Laplacian matrix.
result ISC yields stable consistent community detection under mild conditions and outperforms classical methods.
We study the hyperkaehler geometry of a regular semisimple adjoint orbit of SL(k,C) via the algebraic geometry of the corresponding reducible spectral curve.
Hyperspectral images (HSI) contain a wealth of information over hundreds of contiguous spectral bands, making it possible to classify materials through subtle spectral discrepancies. However, the classification of this rich spectral information is accompanied by the challenges like high dimensionality, singularity, lim…
Recently, non-stationary spectral kernels have drawn much attention, owing to its powerful feature representation ability in revealing long-range correlations and input-dependent characteristics. However, non-stationary spectral kernels are still shallow models, thus they are deficient to learn both hierarchical featur…
We study the spectrum of the Finsler--Laplace operator for regular Hilbert geometries, defined by convex sets with C2 boundaries. We show that for an n-dimensional geometry, the spectral gap is bounded above by (n−1)2/4, which we prove to be the infimum of the essential spectrum. We also construct examples of c…
In this paper, we develop an approach to recursively estimate the quadratic risk for matrix recovery problems regularized with spectral functions. Toward this end, in the spirit of the SURE theory, a key step is to compute the (weak) derivative and divergence of a solution with respect to the observations. As such a so…
Spectral methods are popular in detecting global structures in the given data that can be represented as a matrix. However when the data matrix is sparse or noisy, classic spectral methods usually fail to work, due to localization of eigenvectors (or singular vectors) induced by the sparsity or noise. In this work, we …
We consider a distributed learning approach in supervised learning for a large class of spectral regularization methods in an RKHS framework. The data set of size n is partitioned into m=O(nα) disjoint subsets. On each subset, some spectral regularization method (belonging to a large class, including in particular K…
Unified spectral clustering for sparse networks with heterogeneous degrees.
problem Efficiently detecting communities in sparse networks with varying degrees.
method Developed a parametrized regularized Laplacian matrix for spectral clustering.
result Improved parametrization accounts for network heterogeneity and community hardness.
Extends random feature analysis to spectral methods and improves learning rates.
problem Improving generalization properties of spectral methods in large-scale learning.
method Extends random feature analysis to a broad class of spectral regularization techniques, including gradient descent and Nesterov method.
result Obtains optimal learning rates for regularity classes, including those not in the RKHS.
A new method improves graph-based learning for high-dimensional data.
problem Inconsistent high-dimensional learning efficiency of semi-supervised graph regularization.
method Introducing a novel regularization approach involving centering operation.
result Empirical results show improved performance over spectral clustering.
The article deals with intrinsic metrics, Dirac operators and spectral triples induced by regular Dirichlet and resistance forms. We show, in particular, that if a local resistance form is given and the space is compact in resistance metric, then the intrinsic metric yields a geodesic space. Given a regular Dirichlet f…
The paper introduces a trilinear functional to recover torsion in spectral triples.
problem Recovering torsion in noncommutative spectral triples.
method Introduces a trilinear functional for spectral triples and demonstrates its application to recover torsion.
result The trilinear functional recovers the torsion of the linear connection in canonical spectral triples.
Study spectral flow on a warped cylinder with special boundary conditions.
problem Analyzing spectral flow on a warped cylinder with specific boundary conditions.
method Complexifying the twisting bundle, diagonalizing the orthogonal twist, and regrouping conjugate and reflection-paired blocks.
result Explicit formula for RO(O(2))-valued spectral flow, refining ordinary spectral flow. Study Dirac operators on finite warped cylinders with gauge fields.
problem Characterize spectral flow on finite warped cylinders with gauge fields.
method Identify endpoint operators, derive determinant characterization, introduce regularized APS conditions.
result Regularized APS conditions admit a spectral-flow framework, matching zero-mode sets.
Muon optimizer improves deep learning with spectral norm constraints.
problem Improving optimization algorithms in deep learning.
method Theoretical analysis of Muon optimizer within the Lion-K family. result Muon implicitly solves an optimization problem enforcing spectral norm constraints.
Spectral Clustering is a popular technique to split data points into groups, especially for complex datasets. The algorithms in the Spectral Clustering family typically consist of multiple separate stages (such as similarity matrix construction, low-dimensional embedding, and K-Means clustering as post processing), whi…
The paper examines how gradient descent stabilizes low-rank matrix factorization in noisy conditions.
problem Stability of low-rank implicit regularization in perturbed deep matrix factorization.
method Derives spectral conditions for gradient descent to exhibit a low-rank phase in noiseless settings and analyzes perturbed dynamics.
result Gradient descent converges to a low-rank solution under perturbation, with explicit dependence on perturbation size.
Paper introduces a new multilinear functional for spectral triples and computes its properties.
problem Computing properties of spectral triples and their associated Hodge operators.
method Introduces a new multilinear functional for spectral triples and computes its properties using noncommutative residue and perturbed de-Rham Hodge operators.
result Recover two forms, torsion of the linear connection, and four forms by the noncommutative residue and perturbed de-Rham Hodge Dirac triple.
The performance of spectral clustering can be considerably improved via regularization, as demonstrated empirically in Amini et. al (2012). Here, we provide an attempt at quantifying this improvement through theoretical analysis. Under the stochastic block model (SBM), and its extensions, previous results on spectral c…