Galerkin method outperforms graph-based methods in spectral decompositions.
problem Improving spectral decomposition methods in machine learning.
method Restricting study to a small set of test functions using the Galerkin method.
result Statistical and computational superiority of Galerkin method over graph-based approaches.
Spectral methods improve signal recovery in mixed GLMs with precise asymptotics.
problem Estimating multiple signals from unlabeled observations in mixed GLMs.
method Developed exact asymptotics for spectral methods in a proportional regime.
result Optimized spectral method combined with a linear estimator minimizes estimation error.
Paper improves robust spectral clustering for noisy data.
problem Noisy data and heavy-tailed entries hinder traditional clustering methods.
method Robust spectral clustering with rank statistics for latent structure recovery.
result Provable recovery of latent block structure in large data matrices.
Study smooth linear statistics on random covers of hyperbolic surfaces, showing central limit and variance results.
problem Analyzing fluctuations and energy variance of random covers of compact hyperbolic surfaces.
method Examining fluctuations in a small energy window around a fixed energy level, considering the variance of a typical surface, using a double limit where n and L go to infinity. result Distribution of fluctuations tends to a Gaussian with variance of GOE/GUE, and energy variance of a typical random n-cover is that of GOE/GUE. Spectral method speeds fitting of binary time series models.
problem Modeling binary time series data with latent linear dynamical systems.
method Spectral learning of probit-Bernoulli latent linear dynamical systems.
result Spectral method provides robust, fixed-cost estimator.
Paper tackles functional linear regression using spectral algorithms with discrete observations.
problem Functional linear regression problem with discretely observed data.
method Combines distributed spectral algorithms with Sobolev kernels for regularization.
result Derives matching upper and lower bounds for convergence in Sobolev norm.
Improves signal detection in non-Gaussian noise using transformed data.
problem Signal detection in rank-one signal-plus-noise data matrices.
method Pre-transforming matrix entries and using linear spectral statistics for hypothesis testing.
result Sharp phase transition of largest eigenvalues in spiked rectangular matrices.
OLS is a special case of Transformer, revealing its linear nature.
problem Understanding the statistical essence of Transformer architecture.
method Algebraic proof and spectral decomposition of covariance matrix.
result Attention mechanism in Transformers is mathematically equivalent to OLS.
Develops methods for spectral estimation and rare-event prediction in complex systems.
problem Challenges in understanding dynamics in complex systems with many degrees of freedom.
method Inexact iterative numerical linear algebra methods for spectral estimation and rare-event prediction.
result Demonstrates methods on low-dimensional and high-dimensional models, showing their effectiveness.
We analyse the learning performance of Distributed Gradient Descent in the context of multi-agent decentralised non-parametric regression with the square loss function when i.i.d. samples are assigned to agents. We show that if agents hold sufficiently many samples with respect to the network size, then Distributed Gra…
Improved algorithm for conditional linear regression with heterogeneous covariances.
problem Identifying a linear predictor for a fraction of data with varying covariances.
method Polynomial time algorithm using Disjunctive Normal Form (DNF) to identify a condition and linear predictor.
result Removed requirement for similar covariances in each condition term, improving algorithm applicability.
Promotes spectral functionals to noncommutative fields and proves a theorem.
problem Spectral functionals on noncommutative fields and manifolds with boundary.
method Carries out promotion to spectral functionals, associates with noncommutative residue, and proves theorem.
result Proves Dabrowski-Sitarz-Zalecki type theorem for statistical de Rham Hodge operators on manifolds with boundary.
New insights into spectral statistics of sample covariance matrix for stable linear systems.
problem Estimating high-dimensional stable state transition matrices from noisy data.
method Combining spectral theorem for non-Hermitian operators, concentration of measure, and perturbation theory.
result The spectral radius of the sample covariance matrix exhibits phase transitions in high dimensions.
This work considers the problem of learning the structure of multivariate linear tree models, which include a variety of directed tree graphical models with continuous, discrete, and mixed latent variables such as linear-Gaussian models, hidden Markov models, Gaussian mixture models, and Markov evolutionary trees. The …
We consider the weak detection problem in a rank-one spiked Wigner data matrix where the signal-to-noise ratio is small so that reliable detection is impossible. We propose a hypothesis test on the presence of the signal by utilizing the linear spectral statistics of the data matrix. The test is data-driven and does no…
GOE statistics emerge from surface moduli space averages.
problem Understanding spectral statistics on hyperbolic surfaces.
method Defined a smooth linear statistic, averaged over moduli space, and analyzed variance.
result GOE statistics are recovered in the large genus and high energy limits.
We introduce RSE to measure robustness in estimation problems.
problem Estimating statistical models from observed data.
method Developed theory for spectral functions of measures to compute RSE.
result RSE reveals a reciprocal relationship with problem complexity.
New algorithm for robust regression with subgaussian error bound.
problem Linear regression in the presence of outliers and finite moments.
method Adaptation of spectral method to linear regression problem.
result Optimal sub-gaussian error bound for robust regression.
In this paper, we study the confounder detection problem in the linear model, where the target variable Y is predicted using its n potential causes Xn=(x1,...,xn)T. Based on an assumption of rotation invariant generating process of the model, recent study shows that the spectral measure induced by the regress…
Auto-regressive conditionally heteroskedastic (ARCH) family models are still used, by practitioners in business and economic policy making, as a conditional volatility forecasting models. Furthermore ARCH models still are attracting an interest of the researchers. In this contribution we consider the well known GARCH(1…
Ridge regression linked to Poisson resetting in statistical physics.
problem Understanding and extending ridge regularization in machine learning.
method Connecting stochastic resetting from statistical physics with ridge regularization in machine learning, using renewal processes.
result Exact filter identities for ridge regularization in various reset laws, including exponential and non-exponential.
Efficiently approximates statistical leverage scores for faster KRR.
problem Accurately estimating statistical leverage scores for fast KRR.
method Analytic formula for statistical leverage scores, leveraging kernel spectral density.
result Linear time approximation with theoretical guarantees, significantly faster than existing methods.
Efficient tests achieve best error rates in high-dimensional hypothesis testing.
problem Achieving optimal error rates in computationally efficient hypothesis testing.
method Linear spectral statistics and low-degree likelihood ratio analysis.
result An efficient test achieves the best possible error rates among all computationally efficient tests.
Deep neural networks decompose SDF into linear and nonlinear components.
problem Constructing accurate stochastic discount factors (SDFs) for pricing.
method Additive decomposition of a deep neural network trained to construct SDFs.
result The PTK representation delivers significant performance gains in equity data.
We consider a discriminative learning (regression) problem, whereby the regression function is a convex combination of k linear classifiers. Existing approaches are based on the EM algorithm, or similar techniques, without provable guarantees. We develop a simple method based on spectral techniques and a `mirroring' tr…
A new model for dynamic covariance recovery in neuroimaging data.
problem Estimating time-varying covariances in high-dimensional neuroimaging data.
method Nonconvex factorization into sparse spatial and smooth temporal components, combined with spectral initialization and gradient descent.
result The proposed method achieves linear convergence and superior performance compared to existing approaches.
Study spectral learning for odeco tensors, addressing initialization bottlenecks.
problem Recovering orthogonally decomposable tensors under noise.
method Investigates perturbation bounds, non-convex optimization, and initialization strategies.
result Initialization is the main bottleneck for efficient algorithms.
New methods tackle statistical inverse problems with random data.
problem Statistical inverse problems with random experimental design.
method Spectral regularization, regularization by projection, convex penalties.
result Minimax rates in expectation and probability for convergence.
New method controls linear systems with partial info and disturbances.
problem Controlling linear dynamical systems under partial observation and adversarial disturbances.
method Double Spectral Control (DSC) using two-level spectral approximation strategy.
result Matches best known regret guarantees with exponential runtime improvement.
We present a novel spectral learning algorithm for simultaneous localization and mapping (SLAM) from range data with known correspondences. This algorithm is an instance of a general spectral system identification framework, from which it inherits several desirable properties, including statistical consistency and no l…
The goal of this paper is to show that there exists a simple, yet universal statistical logic of spectral graph analysis by recasting it into a nonparametric function estimation problem. The prescribed viewpoint appears to be good enough to accommodate most of the existing spectral graph techniques as a consequence of …
Bayesian method uses data spectra to estimate non-sparse high-dimensional models.
problem Handling many parameters in high-dimensional Bayesian statistics.
method Data-adaptive Gaussian prior aligned with leading eigenvectors of sample covariance.
result Posterior contraction rates reveal the effect of spectral mass on prediction error.
Linear cost method approximates Gaussian Matérn processes with exponentially convergent accuracy.
problem High computational cost for Gaussian process inference and prediction.
method Optimal rational approximation of spectral density for Gaussian processes on bounded intervals.
result Exponential decrease in covariance error with increasing order of approximation.
Improved singular value approximation for convolutional layers.
problem Improving accuracy of singular value approximation for linear convolutional layers.
method Developed a new spectral density matrix method for singular value approximation with improved accuracy and reduced computational complexity.
result Obtained moderate improvement in singular value distribution compared to circular approximation.
Study on spectral asymptotics in elasticity on smooth manifolds.
problem Analyzing spectral asymptotics in linear elasticity on smooth manifolds.
method Established two-term spectral asymptotics for boundary value problems in linear elasticity.
result Corrected erroneous results in previous studies.
The study examines robustness auditing for linear regression, improving existing methods and identifying computational challenges.
problem Detecting small subsets of data that can reverse regression coefficients.
method Empirical study of mixed integer quadratically constrained optimization and exact greedy methods, combined with a spectral algorithm.
result Existing methods largely outperform state of the art, but computational bottlenecks remain, especially for higher dimensions.
Spectral methods simplify data analysis, improving accuracy and stability.
problem Extracting meaningful information from noisy, incomplete data.
method Eigenvalues and eigenvectors of matrices constructed from data.
result Spectral methods are effective and can be analyzed using modern statistical theory.
Study on elasticity with mixed boundary conditions, proving spectral asymptotics.
problem Analyzing spectral asymptotics for linear elasticity with mixed boundary conditions.
method Established two-term spectral asymptotics for linear elasticity on smooth compact manifolds.
result Verification of general formulae through explicit examples in 2D and 3D.
Wedge Sampling improves tensor completion with nearly-linear sample complexity.
problem Efficiently completing low-rank tensors from a subset of entries.
method Non-adaptive wedge sampling to promote structured connections in tensor completion.
result Polynomial-time algorithms achieve weak and exact recovery with nearly linear sample complexity.
Recently there has been substantial interest in spectral methods for learning dynamical systems. These methods are popular since they often offer a good tradeoff between computational and statistical efficiency. Unfortunately, they can be difficult to use and extend in practice: e.g., they can make it difficult to inco…
The study examines spectral dynamics in deep neural networks, predicting how outliers evolve during training.
problem Understanding spectral evolution in deep neural networks during training.
method Developed a two-level dynamical mean-field theory (DMFT) to track spectral dynamics.
result The theory predicts how outliers evolve with training time, width, output scale, and initialization variance.
Unified analysis of multilabel Fisher discriminants with improved dimensionality and robustness.
problem Improving discriminant analysis for multilabel classification with enhanced dimensionality and robustness.
method Unified theoretical analysis of multilabel Fisher discriminants with algebraic and statistical guarantees.
result Unified characterization of multilabel Fisher objectives and their equivalence under orthogonality constraints.
Unified analysis of multilabel Fisher discriminants with improved dimensionality and robustness.
problem Improving discriminant analysis for multilabel classification with enhanced dimensionality and robustness.
method Unified algebraic and statistical analysis of multilabel Fisher discriminants with Stiefel orthogonality constraints.
result Equivalence of four Fisher objectives under the Stiefel constraint and improved discriminant dimensionality.
Unified framework explains why overfitting is benign in interpolating learning.
problem Understanding why overfitting is benign in highly overparameterized models.
method Spectral-transport stability framework.
result Sharp benign-overfitting criterion and explicit phase-transition rates.
Stochastic gradient descent achieves polynomial convergence rates for noiseless linear models.
problem Convergence analysis of stochastic gradient descent in noiseless linear models.
method Fixed step-size stochastic gradient descent on least-square risk.
result Polynomial convergence rates depend on the regularities of the optimum and feature vectors.
Develops an ℓ_p theory for PCA and spectral clustering.
problem Lack of precise characterizations of PCA scores for low-dimensional embedding.
method An ℓ_p perturbation theory for PCA in Hilbert spaces, analyzing eigenvectors and Gram matrix.
result Optimal recovery results for Gaussian mixture and stochastic block models.
New method samples DPPs efficiently without downsampling or low-rank approximations.
problem Efficient sampling from DPPs for diverse selections.
method Directly approximates distribution function of linear statistics using Laplace inversion.
result Scalable sampling for general DPPs, beyond symmetric kernels.
The spectral asymptotics for linear elasticity with mixed boundary conditions are shown to be old results.
problem Analyzing the spectral asymptotics for linear elasticity with mixed boundary conditions.
method Demonstrating that the results are essentially old well-known results by other authors.
result The spectral asymptotics results for linear elasticity with mixed boundary conditions are shown to be old results by other authors.