This paper speeds up K-FAC for deep learning by focusing on only a few eigen-modes.
problem Time-consuming computation of Kronecker factors in K-FAC for large layers.
method Theoretical analysis and randomized numerical linear algebra to approximate eigen-spectrum decay.
result Reduces time complexity from cubic to quadratic in layer width, improving efficiency.
Study on KRR with power-law data, showing better sample complexity.
problem High-dimensional kernel ridge regression with anisotropic power-law covariance.
method Explicit characterization of kernel spectrum and asymptotic analysis of excess risk.
result Sample complexity is governed by effective dimension, not ambient dimension.
We investigate various structures associated with the hyperbolic Markov and homological spectra of a pseudoAnosov map φ on a surface. Each unstable eigenvalue of the action of φ on first cohomolgy yields an eigen-cocycle that is transverse and holonomy invariant to the stable foliation Fs of φ. Each …
This paper proposes a new Nystrom-based clustering algorithm for large-scale data.
problem Spectral clustering's high computational complexity for large-scale data.
method Centroid Minimum Sum of Squared Similarities (CMS3) sampling procedure with eigen spectrum shape heuristic.
result Competitive low-rank approximations in test datasets compared to state-of-the-art methods.
This paper aims to address two fundamental challenges arising in eigenvector estimation and inference for a low-rank matrix from noisy observations: (1) how to estimate an unknown eigenvector when the eigen-gap (i.e. the spacing between the associated eigenvalue and the rest of the spectrum) is particularly small; (2) …
Proposes a Gaussian process for graph signals using adaptive spectral kernels.
problem Predicting signals on graph nodes with various structures.
method Spectral kernel learning approach that incorporates a polynomial function in the graph spectral domain.
result The model accurately recovers ground truth spectral filters and outperforms in real-world graph data.
The paper introduces eigen-portfolios using PCA to improve portfolio construction in finance.
problem Overfitting and poor generalization in selecting a single eigen-portfolio.
method Principal Component Analysis (PCA) to derive eigen-portfolios from asset return correlation matrices.
result An ensemble strategy combining multiple top-performing eigen-portfolios significantly improves out-of-sample performance.
Eigen-GNN enhances GNNs by preserving graph structures.
problem Existing shallow GNNs fail to effectively preserve graph structures.
method Integrates eigenspace of graph structures into GNNs as a dimensionality reduction module.
result Eigen-GNN boosts GNNs' ability to preserve graph structures without increasing depth.
The study characterizes harmonic spaces and their radial eigen-functions and vector fields.
problem Characterizing harmonic spaces and their geometric properties.
method Examining radial eigen-spaces of Laplacians and using duality.
result Results extend to spaces harmonic with respect to a single point.
Eigen-stratified models reduce model size and improve performance.
problem Large model size in Laplacian-regularized stratified models.
method Formulate eigen-stratified models with linear combinations of bottom eigenvectors of the graph Laplacian.
result Significant reduction in model size with eigen-stratified models.
Neural networks solve eigen-problems in differential equations.
problem Finding eigenpairs of self-adjoint operators.
method Using neural networks to approximate eigenfunctions and eigenvalues.
result Demonstrates potential of neural networks in solving complex eigen-problems.
EFiGP uses Fourier and eigen-decomposition for efficient ODE parameter estimation.
problem Parameter estimation and trajectory reconstruction for noisy, sparse, nonlinear ODE systems.
method EFiGP integrates Fourier transformation and eigen-decomposition into a physics-informed Gaussian Process framework.
result EFiGP efficiently estimates ODE parameters and recovers trajectories from noisy data.
IKD uses eigen-decomposition for nonlinear dimensionality reduction.
problem Lack of sophisticated and nonlinear dimensionality reduction methods.
method Inverse Kernel Decomposition (IKD) based on eigen-decomposition of sample covariance matrix.
result IKD achieves comparable performance to optimization-based methods with faster running speeds.
We prove polynomial and exponential decay at infinity of eigen-vectors of partial differential operators related to radiation problems for time-harmonic generalized Maxwell systems in an exterior domain with non-smooth inhomogeneous, anisotropic coefficients converging near infinity with a certain rate towards the iden…
Algorithm reduces online regret by leveraging offline data in linear bandits.
problem Online regret minimization in linear bandits with offline data.
method OOPE algorithm using extended D-optimal design.
result Substantial reduction in online regret compared to prior work.
Eigen component analysis combines quantum mechanics with machine learning for efficient data analysis.
problem Efficiently extracting linearly separable components from complex data.
method Eigen component analysis (ECA) incorporates quantum mechanics principles into linear learning models.
result ECA outperforms classical linear models and can be integrated with deep neural networks.
In this paper, energy function is used to investigate the eigen-solutions of −Δu+Vu=λu on the Riemannian manifolds. We give a new way to prove the positivity of the initial energy of energy function, which leads to a simple way to obtain the growth of eigen-solutions.
This study compares two portfolio optimization methods on Indian stocks.
problem Designing an optimal portfolio considering stock returns and risks.
method Hierarchical Risk Parity and Eigen Portfolio approaches on NIFTY 50 sectors.
result Hierarchical Risk Parity portfolio outperforms Eigen portfolio in most sectors tested.
This paper compares three portfolio designs for Indian stocks.
problem Designing an optimum portfolio that balances return and risk.
method Three approaches: minimum risk, optimum risk, and Eigen portfolios.
result Optimum risk portfolios and Eigen portfolios identified for each sector.
In this paper, we introduce an algorithm for performing spectral clustering efficiently. Spectral clustering is a powerful clustering algorithm that suffers from high computational complexity, due to eigen decomposition. In this work, we first build the adjacency matrix of the corresponding graph of the dataset. To bui…
Paper addresses eigenvector perturbation in small eigen-gap scenarios.
problem Fine-grained behavior of eigenvectors in the presence of small eigen-gaps.
method Develops de-biased estimators for linear functions of an unknown eigenvector.
result Achieves minimax lower bounds for a family of scenarios, even with small eigen-gaps.
In this paper, we obtain some properties of biconservative Lorentz hypersurface M1n in E1n+1 having shape operator with complex eigen values. We prove that every biconservative Lorentz hypersurface M1n in E1n+1 whose shape operator has complex eigen values with at most five distinct prin…
A new kernel test reduces noise in MMD by focusing on leading eigen-directions.
problem Noise in trailing directional components degrades power of standard kernel two-sample tests.
method Truncate MMD spectral decomposition, retaining only leading eigen-directions.
result Our method achieves superior power and robustness, especially in high-dimensional and unbalanced settings.
This paper has been withdrawn by the author and it is published in AGAG
Repeated application of machine-learning, eigen-centric methods to an evolving dataset reveals that eigenvectors calculated by well-established computer implementations are not stable along an evolving sequence. This is because the sign of any one eigenvector may point along either the positive or negative direction of…
New spinorial functional connects Perelman's W- and F-functionals.
problem Unifying Perelman's functionals for spin manifolds.
method Introduced a new energy functional on spin manifolds, computed its first variation, and established a gradient flow.
result Critical points of the functional are twisted Ricci solitons and eigen-spinsors.
In this article we give a classification of three dimensional m-quasi Einstein manifolds with two distinct Ricci-eigen values. Our study provides explicit description of local and complete metrics and potential functions. We also describe the associated warped product Einstein manifolds in detail. For the proof we pres…
A novel graph spectral method for mixed categorical and numerical data.
problem Feature learning for mixed data types (numerical and categorical).
method Graph spectral decomposition of the graph Laplacian to model probabilistic dependence structure.
result Increased separability and clusterability of observations in the transformed feature space.
Study on gradient pseudo-Ricci solitons on real hypersurfaces.
problem Characterize gradient pseudo-Ricci solitons on real hypersurfaces.
method Analyze real hypersurfaces in complex space forms with specific eigen properties of the Ricci tensor.
result Show existence of non-trivial gradient pseudo-Ricci solitons on 3D ruled real hypersurfaces.
In this paper, we consider the eigen-solutions of −Δu+Vu=λu, where Δ is the Laplacian on a non-compact complete Riemannian manifold. We develop Kato's methods on manifold and establish the growth of the eigen-solutions as r goes to infinity based on the asymptotical behaviors of Δr and V(x), where r=r(x) i…
2L-FUSE enhances feature sparsity through kernel learning.
problem Sparsity and feature selection in regression tasks.
method 2-Layered kernel machines for learning a shape matrix and feature direction identification.
result Minimal yet informative feature sets are identified without losing predictive performance.
EigenGAN discovers interpretable dimensions in GAN layers for semantic control.
problem Lack of explicit dimensions to control semantic attributes in GAN layers.
method EigenGAN embeds linear subspaces with orthogonal bases into each generator layer, learning eigen-dimensions corresponding to semantic attributes via adversarial training.
result EigenGAN can produce samples with continuous changes corresponding to specific semantic attributes.
Spectral clustering is one of the most popular methods for community detection in graphs. A key step in spectral clustering algorithms is the eigen decomposition of the n×n graph Laplacian matrix to extract its k leading eigenvectors, where k is the desired number of clusters among n objects. This is pro…
Study reveals how neural network smoothness affects their vulnerability to adversarial attacks.
problem Understanding adversarial vulnerability in deep learning networks.
method Analysis of manifold smoothness and generalization capability of deep neural networks trained with local errors.
result High generalization accuracy requires a fast power-law decay of eigen-spectrum of hidden representations.
TOLD++ improves convergence of diffusion models by critically damping the forward transition matrix.
problem Improving the convergence of Denoising Diffusion Probabilistic Models.
method Critically damping the Third-Order Langevin Dynamics (TOLD) forward transition matrix using eigen-analysis.
result TOLD++ converges faster than TOLD, verified on toy and real datasets.
In much of the literature on function approximation by deep networks, the function is assumed to be defined on some known domain, such as a cube or a sphere. In practice, the data might not be dense on these domains, and therefore, the approximation theory results are observed to be too conservative. In manifold learni…
In this paper we give a proof of Lichnerowicz Conjecture for compact simply connected manifolds which is intrinsic in the sense that it avoids the {\it Nice Embeddings} into eigen spaces of the Laplacian. Even if one wants to use these embeddings this paper gives a more streamlined proof.
Financial markets, being spectacular examples of complex systems, display rich correlation structures among price returns of different assets. The correlation structures change drastically, akin to phase transitions in physical phenomena, as do the influential stocks (leaders) and sectors (communities), during market e…
The paper analyzes high-dimensional kernel regression, showing different risk curves based on data and regularization.
problem Characterizing generalization properties of high-dimensional kernel ridge regression.
method Bias-variance decomposition of the expected excess risk, considering different regularization schemes and data eigen-profiles.
result The risk curve of kernel regression can be double-descent-like, bell-shaped, or monotonic, depending on n, d, and regularization level.
Improves MARS for nonparametric multivariate regression with dimension reduction.
problem High number of basis functions in MARS for high-order interactions.
method Linear combinations of covariates for dimension reduction, facilitating gradient calculation and eigen-analysis for estimation.
result Asymptotic theory and numerical studies show improved performance over MARS.
This paper optimizes portfolios of thematic sector stocks using LSTM models.
problem Designing an optimized portfolio of stocks to maximize return and minimize risk.
method Extracted stock prices from Jan 2016 to Dec 2020, used LSTM model for prediction, designed portfolios based on critical stocks.
result LSTM model accurately predicted future stock returns, indicating high accuracy.
A new mathematical approach detects frequency-based alterations in brain networks.
problem Understanding disease-relevant brain alterations through network analysis.
method Proposes a novel connectome harmonic analysis framework using common harmonic waves learned from Stiefel manifolds.
result Identifies more significant and reproducible network dysfunction patterns in Alzheimer's disease.
We study the stochastic Riemannian gradient algorithm for matrix eigen-decomposition. The state-of-the-art stochastic Riemannian algorithm requires the learning rate to decay to zero and thus suffers from slow convergence and sub-optimal solutions. In this paper, we address this issue by deploying the variance reductio…
The Hessian-vector product has been utilized to find a second-order stationary solution with strong complexity guarantee (e.g., almost linear time complexity in the problem's dimensionality). In this paper, we propose to further reduce the number of Hessian-vector products for faster non-convex optimization. Previous a…
MOSAIC detects change points in dynamic networks with low-rank and sparse changes.
problem Detecting change points in dynamic networks with specific structural properties.
method Eigen-decomposition-based test with screened signals and residual-based adjustment.
result MOSAIC achieves minimax-optimal detection and testing rates.
A new classifier uses weighted orthogonal regression for robust classification with limited data.
problem Challenges in classification with insufficient training data.
method Exploits intrinsic structure of data through Eigen components with specific weights determined by eigenvalues.
result Robust learning in classification problems with limited data.
Extensions (entropies) play a central role in the theory of hyperbolic conservation laws by providing intrinsic selection criteria for weak solutions. For a given hyperbolic system u_t+f(u)_x=0, a standard approach is to analyze directly the second order PDE system for the extensions. Instead we find it advantageous to…
SpGAT learns graph representations using spectral attention for efficiency.
problem Efficiently capturing global graph patterns with minimal parameters.
method Introduces Spectral Graph Attention Network (SpGAT) using spectral domain attention mechanisms and a fast Chebychev approximation.
result SpGAT achieves better global pattern recognition with fewer parameters compared to GAT.