Spectral ranking methods are improved against semi-random graph sampling.
problem Improving spectral ranking methods in semi-random graph sampling.
method Investigating entry-wise error of spectral algorithms against a semi-random adversary.
result Asymptotic performance can be recovered by reweighting observed edges.
The paper improves tensor completion bounds using spectral gap.
problem Theoretical limitations in tensor completion, especially for deterministic sampling.
method Bounding the generalization error of tensor completion methods using spectral gap.
result Improved bounds on tensor completion error, reducing rank dependence.
The study analyzes spectral algorithms for kernel methods and derives generalization error.
problem Estimating generalization error of spectral algorithms for kernel methods.
method Considered spectral algorithms including KRR and GD, derived generalization error as a functional of learning profile.
result Showed the loss localizes on certain spectral scales and conjectured universality of the loss for noisy observations.
Spectral clustering has been one of the widely used methods for community detection in networks. However, large-scale networks bring computational challenges to the eigenvalue decomposition therein. In this paper, we study the spectral clustering using randomized sketching algorithms from a statistical perspective, whe…
Improved spectral convergence bounds for diffusion maps on tori.
problem Weak theoretical error bounds for diffusion maps.
method Spatial Hardy space estimates, PDE spectral stability, Sinkhorn weights.
result Matched pointwise error bounds for spectral data and operator convergence.
The paper analyzes neural networks for solving high-dimensional Schrödinger eigenvalue problems.
problem Analyzing generalization error of neural networks for high-dimensional Schrödinger eigenvalue problems.
method Proves convergence rate of generalization error independent of dimension d under spectral Barron space assumption. Verifies assumption by proving regularity estimate. result Generalization error rate is independent of dimension d under spectral Barron space assumption. Proposes neural dynamic mode decomposition for end-to-end modeling of nonlinear dynamics.
problem Understanding and modeling nonlinear dynamical systems.
method Trains neural networks to minimize forecast error based on spectral decomposition in the lifted space.
result Demonstrates effectiveness in eigenvalue estimation and forecast performance.
Paper improves spectral learning of HMMs to avoid local optima and improve robustness.
problem Spectral learning of HMMs can get stuck in local optima and degrade due to unchecked error propagation.
method Developed a novel algorithm (PSHMM) and online learning variants to mitigate error propagation and nonstationarity.
result PSHMM provides more robust estimation and forecasting compared to SHMM and B-W algorithm.
Proves polynomial error rate for equidistribution of unipotent flows.
problem Equidistribution of orbits of unipotent subgroups in arithmetic quotients.
method Uses Margulis function, incidence geometry tools, and spectral gap.
result Polynomial error rate for equidistribution theorems.
A large number of algorithms in machine learning, from principal component analysis (PCA), and its non-linear (kernel) extensions, to more recent spectral embedding and support estimation methods, rely on estimating a linear subspace from samples. In this paper we introduce a general formulation of this problem and der…
We identify spectral conditions for reliable neural probe interpretation.
problem Unreliable performance of linear probes in interpreting neural representations.
method Formalized Spectral Identifiability Principle (SIP) based on eigengap and Fisher error.
result Reliability of neural probes depends on the eigengap relative to Fisher estimation error.
Proves error bounds for state representation in RL using graph spectral features.
problem Addressing the curse of dimensionality in RL with unknown transition graphs.
method Proves upper bounds on approximation error of linear value function approximation using learned spectral features of the state-graph.
result Error bounds scale with algebraic connectivity and eigenvector estimation error.
Estimates manifold distances using graph Laplacian, proving consistency.
problem Estimating distances in compact Riemannian manifolds.
method Graph Laplacian estimates of the Laplace-Beltrami operator, bounding errors.
result Proof of consistency for manifold distances.
Study precise sample covariance error for Gaussian centered data.
problem Precise characterization of sample covariance error for Gaussian data.
method Developed a Random Duality Theory (RDT) framework to determine upper and lower bounds.
result Upper and lower bounds match in large-dimensional contexts, matching the spectral norm's limiting value.
Nonlinear kernels can be approximated using finite-dimensional feature maps for efficient risk minimization. Due to the inherent trade-off between the dimension of the (mapped) feature space and the approximation accuracy, the key problem is to identify promising (explicit) features leading to a satisfactory out-of-sam…
This work analyzes how different layers in deep neural networks contribute to generalization error.
problem Understanding the role of each layer in deep neural networks for generalization.
method Spectral analysis, Neural Tangent Kernel, Hermite polynomials, Spherical Harmonics.
result Initial layers in deep neural networks have a larger bias towards high-frequency functions.
Study uses spectral risk for learning with heavy-tailed data.
problem Learning with heavy-tailed loss distributions.
method Spectral risk with Lipschitz-continuous density, derivative-free learning.
result Excess risk guarantees and improved performance over traditional methods.
This paper provides a general framework to study the effect of sampling properties of training data on the generalization error of the learned machine learning (ML) models. Specifically, we propose a new spectral analysis of the generalization error, expressed in terms of the power spectra of the sampling pattern and t…
Graph signal processing detects hallucinations in large language models.
problem Detecting factual reasoning from hallucinations in large language models.
method Modeling transformer layers as dynamic graphs, using spectral analysis to define diagnostics.
result Spectral signatures can distinguish different types of hallucinations and achieve high accuracy.
Spectral feature learning improves IV regression for causal effect estimation.
problem Estimating causal effects in the presence of hidden confounders.
method Two-stage least squares estimator based on spectral features.
result Performance of the method depends on strong spectral alignment and slow eigenvalue decay.
Paper analyzes DRM for solving high-dimensional elliptic PDEs with generalization bounds.
problem Analyzing generalization error of neural network methods for high-dimensional PDEs.
method Developed a new solution theory for spectral Barron space and derived generalization error bounds.
result Generalization error bounds are independent of dimension and solutions lie in spectral Barron space.
Compression techniques for deep neural network models are becoming very important for the efficient execution of high-performance deep learning systems on edge-computing devices. The concept of model compression is also important for analyzing the generalization error of deep learning, known as the compression-based er…
For graphs generated from stochastic blockmodels, adjacency spectral embedding is asymptotically consistent. Further, adjacency spectral embedding composed with universally consistent classifiers is universally consistent to achieve the Bayes error. However when the graph contains private or sensitive information, trea…
Positive weights improve kernel quadrature's accuracy.
problem Improving kernel quadrature weights to be positive and stable.
method Using convex geometry to approximate the kernel mean embedding with positive weights.
result Positive weights lead to improved kernel quadrature bounds with Monte-Carlo-beating rates.
We present a novel spectral learning algorithm for simultaneous localization and mapping (SLAM) from range data with known correspondences. This algorithm is an instance of a general spectral system identification framework, from which it inherits several desirable properties, including statistical consistency and no l…
Spectral regularization improves learning over combinatorial spaces with limited data.
problem Learning pseudo-Boolean functions with scarce labeled data.
method Regularizing the spectral representation of learned functions using the L_1 norm.
result Regularization allows for data-frugal learning and achieves statistically optimal generalization performance.
Extended Gauss-Markov theorem for linear estimation with bounded bias.
problem Linear estimation with bounded bias operator.
method Derive optimal estimator formulas for Nuclear and Spectral norms, analyze generalization error.
result Cross-validated Nuclear and Spectral regressors outperform Ridge regression in simulations.
Randomized spectral co-clustering speeds up large-scale directed networks.
problem Co-clustering directed networks efficiently for large-scale data.
method Randomized spectral co-clustering algorithms using random-projection and random-sampling techniques.
result Theoretical and numerical validation of approximation and misclustering error rates.
New spectral methods improve matrix estimation in RL with low-rank structure.
problem Estimating matrices with low-rank structure in reinforcement learning.
method Spectral-based matrix estimation approaches.
result Spectral methods efficiently recover singular subspaces and minimize entry-wise error.
The paper bounds generalization errors for deep neural networks with Markov datasets.
problem Bounding generalization errors for deep learning with Markov datasets.
method Developed new symmetrization inequalities for Markov chains, using spectral gap of the infinitesimal generator.
result Derived upper bounds on generalization errors for deep neural networks with Markov datasets.
Detects corruption in agentic models during execution.
problem Inconsistent context, retrieval errors, or adversarial inputs corrupt intermediate steps of reasoning chains.
method Analyzes token graphs induced by attention and computes spectral statistics to emit accept/reject signals.
result A single threshold on the high frequency energy ratio optimally detects context inconsistency in agentic models.
Random covers of hyperbolic surfaces have a spectral gap with polynomial rate.
problem Finding spectral gaps in random covers of hyperbolic surfaces.
method Applying recent work on spectral gaps to uniformly random covers of closed hyperbolic surfaces.
result Uniformly random degree-n covers of a closed hyperbolic surface have no new Laplacian eigenvalues below a specific threshold with high probability.
RKD improves clustering in semi-supervised learning with limited labels.
problem Improving clustering accuracy in semi-supervised learning with few labeled examples.
method RKD as spectral clustering on a teacher model's graph, with clustering error quantification.
result RKD provably leads to low clustering error in semi-supervised classification problems.
Spectral clustering is one of the most popular methods for community detection in graphs. A key step in spectral clustering algorithms is the eigen decomposition of the n×n graph Laplacian matrix to extract its k leading eigenvectors, where k is the desired number of clusters among n objects. This is pro…
The generalization performance of kernel methods is largely determined by the kernel, but common kernels are stationary thus input-independent and output-independent, that limits their applications on complicated tasks. In this paper, we propose a powerful and efficient spectral kernel learning framework and learned ke…
Adaptive spectral RL method enhances RL performance and interpretability.
problem Balancing interpretability and performance in reinforcement learning.
method Spectral based linear RL approach with adaptive regularization.
result Near-optimal bounds for parameter estimation and generalization error.
We approximate the spectral data (eigenvalues and eigenfunctions) of compact Riemannian manifold by the spectral data of a sequence of (computable) discrete Laplace operators associated to some graphs immersed in the manifold. We give an upper bound on the error that depends on upper bounds on the diameter and the sect…
Neural network model improves leaf spectral reflectance prediction for grapevines.
problem Inaccurate modeling of grapevine leaf spectral reflectance from traits.
method Multi-head attention neural network trained on grapevine-specific data.
result Model achieved high accuracy (R^2=0.84, NRMSE=1.52%) and outperformed PROSPECT-PRO.
Spectral deconfounding improves machine learning models by reducing hidden confounding effects.
problem Machine learning models can be misled by hidden confounders, leading to unreliable predictions.
method Develops a nonlinear spectral deconfounding framework for gradient boosting that modifies boosting dynamics to slow down in confounding-aligned directions.
result Spectrally deconfounded boosting improves estimation of the target function under hidden confounding and is more scalable.
Improved spectral clustering for community detection in networks.
problem Community detection in networks.
method Improved spectral clustering (ISC) based on k-means clustering on weighted eigenvectors of a regularized Laplacian matrix.
result ISC yields stable consistent community detection under mild conditions and outperforms classical methods.
This paper is concerned with the problem of top-K ranking from pairwise comparisons. Given a collection of n items and a few pairwise comparisons across them, one wishes to identify the set of K items that receive the highest ranks. To tackle this problem, we adopt the logistic parametric model --- the Bradley-Te…
SIC detects elbows in error curves automatically.
problem Automatic elbow detection in error curves.
method Spectral information criterion (SIC) extracts geometric features of error curves.
result SIC provides a subset of models with smaller cardinality than total possible models.
New algorithm for robust regression with subgaussian error bound.
problem Linear regression in the presence of outliers and finite moments.
method Adaptation of spectral method to linear regression problem.
result Optimal sub-gaussian error bound for robust regression.
Spectral clustering for directed graphs using likelihood estimation.
problem Clustering directed graphs with edge directions.
method Maximum likelihood estimation on stochastic block models.
result Significant performance gains over existing methods.
Recently, non-stationary spectral kernels have drawn much attention, owing to its powerful feature representation ability in revealing long-range correlations and input-dependent characteristics. However, non-stationary spectral kernels are still shallow models, thus they are deficient to learn both hierarchical featur…
Fast simulates Volterra processes using RFF, focusing on S-fBM.
problem Efficiently simulate Volterra processes for fractional Brownian motion.
method Random Fourier Features (RFF) approximation of kernel, spectral representation, Hamiltonian Monte Carlo sampling.
result Quantitative guarantees for RFF approximation, competitive in terms of efficiency and error.
Bayesian SAE model with spectral clustering and uncertainty quantification.
problem Small Area Estimation (SAE) with uncertainty quantification.
method Spectral clustering with external covariates, posterior projections, and CPMSE.
result Closed form expressions for posterior mean estimators and CPMSE.
New spectral clustering method for multi-layer networks improves accuracy.
problem Detecting community structure in multi-layer networks.
method Integrative spectral clustering based on adaptive layer aggregation.
result Our methods minimize mis-clustering error and outperform existing methods.