Spectral decoupling improves neural network generalization in medical imaging.
problem Poor generalization of neural networks trained on medical imaging data.
method Spectral decoupling, a regularization technique that encourages learning more features.
result Spectral decoupling increases network robustness and performance on external datasets.
ARFF reduces spectral bias in SGD-trained neural networks.
problem Spectral bias in two-layer neural networks.
method Comparison of SGD and ARFF on spectral bias and robustness.
result ARFF yields a closer to zero spectral bias compared to SGD.
Exact spectral norm regularization improves neural network generalization.
problem Improving neural network generalization while protecting against noise.
method Exact spectral norm regularization of the Jacobian.
result Improved generalization performance compared to previous methods.
Paper explores SNN for learning spectral geometric info from data.
problem Challenges in applying traditional eigensolvers to big data.
method Introduces Spectral Neural Networks (SNN) as an alternative.
result Investigates tradeoffs and optimization landscape of SNN.
Paper explains spectral bias in neural networks and its relation to neural tangent kernel.
problem Understanding the spectral bias of deep learning models.
method Decomposes neural network training into eigenfunctions of the neural tangent kernel, proving convergence rates.
result Neural networks learn functions with lower complexity faster, explained by eigenvalues of the neural tangent kernel.
This paper proves a generalization bound for complex-valued neural networks scaling with spectral complexity.
problem Ensuring the performance of complex-valued neural networks on unseen data.
method Theoretical derivation using Maurey Sparsification Lemma and Dudley Entropy Integral, empirical validation on various datasets.
result The spectral complexity of weight matrices is a significant factor in the generalization ability of complex-valued neural networks.
Proposes neural dynamic mode decomposition for end-to-end modeling of nonlinear dynamics.
problem Understanding and modeling nonlinear dynamical systems.
method Trains neural networks to minimize forecast error based on spectral decomposition in the lifted space.
result Demonstrates effectiveness in eigenvalue estimation and forecast performance.
This work analyzes how different layers in deep neural networks contribute to generalization error.
problem Understanding the role of each layer in deep neural networks for generalization.
method Spectral analysis, Neural Tangent Kernel, Hermite polynomials, Spherical Harmonics.
result Initial layers in deep neural networks have a larger bias towards high-frequency functions.
Method detects neural network equivalence via matrix ensembles and spectral analysis.
problem Detecting equivalence among different deep learning architectures.
method Generating Mixed Matrix Ensembles (MMEs) and matching to conjugate circular ensembles.
result Empirical evidence shows vanishing differences in spectral densities with long tail decay rates.
Interpretable neural network for plant traits and species identification.
problem Plant phenotyping and identification.
method Neural network trained on UPWINS spectral library, with visualization of weights for trait-based spectral features.
result 90% accuracy in species identification with interpretable neural network.
Improved neural network predicts spectral functions more accurately than traditional methods.
problem Reconstructing real-time spectral functions from imaginary-time Green's functions is ill-posed and challenging.
method Feature Learning Network (FL-net) for enhanced prediction accuracy.
result FL-net achieves at least 20% improvement over traditional methods like MEM.
Fiedler regularization uses spectral graph theory to improve neural network performance.
problem Improving neural network performance by penalizing weights based on connectivity.
method Uses the Fiedler value of the neural network's graph as a regularization tool, providing theoretical and computational methods.
result Demonstrates Fiedler regularization's effectiveness in improving neural network performance.
Graph convolutional networks fail to use eigenvectors beyond the first, unlike spectral embedding.
problem Understanding when graph convolutional networks fail compared to spectral embedding.
method Presented a simple generative model to illustrate failure.
result Graph convolutional networks fail to use eigenvectors beyond the first in certain graphs.
New framework explains neural network bias in solving differential equations.
problem Understanding and controlling the bias in PINNs for differential equations.
method Deriving an integro-differential equation from PINNs and GPR equivalence.
result PINN predictions are influenced by a kernel term reflecting architecture choices.
Spectral portfolio theory links neural networks to wealth dynamics via SGD weight matrices.
problem Understanding wealth dynamics from neural network training.
method Direct identification of weight matrices as portfolio allocation matrices, linking SGD forces to portfolio dynamics.
result Spectral properties of SGD weight matrices transition between additive and multiplicative regimes, influencing wealth dynamics.
We analyze how kernel machines and neural networks learn different frequency modes of the target function as data size increases.
problem Understanding how kernel machines and neural networks learn different frequency modes of the target function as data size increases.
method Theoretical methods from Gaussian processes and statistical physics, combined with simulations on synthetic data and MNIST dataset.
result Kernel machines and neural networks fit successively higher spectral modes of the target function as the size of the training set grows.
A new method speeds up spectral normalization for neural nets.
problem Efficiently controlling the spectral norm of convolutional layers.
method Depthwise separable convolutions with spectral normalization.
result Significant reduction in computational and memory costs.
Investigates spectral properties of neural networks, showing invariance under certain conditions.
problem Understanding the spectral evolution and invariance in linear-width neural networks.
method Empirical and theoretical analysis of spectra of weight matrices in high-dimensional settings.
result Spectra of weight matrices are invariant under certain training conditions, with implications for feature learning.
New method trains neural networks in spectral domain for improved performance.
problem Training deep neural networks in the space of nodes.
method Trains neural networks in the spectral domain, modifying eigenvalues and eigenvectors of transfer operators.
result Superior performance compared to standard methods, especially when adjusting eigenvalues.
FFN addresses spectral bias in neural value approximation, improving reinforcement learning performance.
problem Spectral bias in neural value approximation, leading to slow convergence and poor performance.
method Proposes Fourier feature networks (FFN) to overcome spectral bias by using a composite neural tangent kernel.
result FFN achieves state-of-the-art performance on challenging continuous control domains with faster convergence and better stability.
The study examines Fisher information matrices and neural tangent kernels for simple ReLU networks with random weights.
problem Understanding the relationship between Fisher information matrices and neural tangent kernels for 2-layer ReLU networks.
method Analyzes Fisher information matrices and neural tangent kernels for 2-layer ReLU networks with random hidden weights, focusing on spectral decomposition and eigenfunctions.
result Obtained an approximation formula for functions represented by 2-layer neural networks.
New neural architectures invariant to sign flips and basis symmetries for graph representation learning.
problem Learning invariant graph representations from eigenvectors.
method SignNet and BasisNet neural architectures that are invariant to sign flips and basis symmetries.
result Proven to be universal, approximating any continuous function of eigenvectors with desired invariances.
New matrix ensembles better match deep neural network spectral densities.
problem Theoretical spectral density models for deep networks do not match empirical observations.
method Introduced new matrix ensemble classes to better fit observed spectral densities.
result Theoretical models for deep networks are significantly flawed.
The paper bridges spectral and spatial graph convolutions, improving model capacity and transferability.
problem Improving graph neural networks by bridging spectral and spatial design.
method Theoretical demonstration and general framework for spectral analysis, new spectral convolutions, and depthwise separable convolutions.
result General framework allows spectral analysis of ConvGNNs, showing their performance and limits, and proposing new spectral convolutions.
This study bridges the gap between spatial and spectral GNNs.
problem Lack of direct comparison and cross-reference of existing GNNs.
method Systematically categorizes and examines GNNs into spatial and spectral domains.
result Establishes a strong relationship between spatial and spectral GNNs.
The paper analyzes neural networks for solving high-dimensional Schrödinger eigenvalue problems.
problem Analyzing generalization error of neural networks for high-dimensional Schrödinger eigenvalue problems.
method Proves convergence rate of generalization error independent of dimension d under spectral Barron space assumption. Verifies assumption by proving regularity estimate. result Generalization error rate is independent of dimension d under spectral Barron space assumption. Neural network model improves leaf spectral reflectance prediction for grapevines.
problem Inaccurate modeling of grapevine leaf spectral reflectance from traits.
method Multi-head attention neural network trained on grapevine-specific data.
result Model achieved high accuracy (R^2=0.84, NRMSE=1.52%) and outperformed PROSPECT-PRO.
SpecNet2 improves spectral embedding without orthogonalization, achieving better performance and efficiency.
problem Improving spectral embedding methods for better performance and efficiency.
method Optimizes an equivalent objective of the eigen-problem without orthogonalization, allowing separate row and column sampling.
result Local and global convergence of the new objective using batch-based gradient descent is proven, and improved performance and efficiency are demonstrated on simulated and image datasets.
Spectral analysis shows neural networks separate from linear methods in approximating functions.
problem Separating two-layer neural networks from linear methods in function approximation.
method Spectral-based approach using Kolmogorov width and kernel spectrum.
result Upper and lower bounds on separation, explicit hard functions identified.
Paper revisits graph-CNNs using Laplace-Beltrami spectral filters and polynomials.
problem Improving spectral graph convolutional neural networks (graph-CNNs).
method Developed Laplace-Beltrami CNN (LB-CNN) by replacing graph Laplacian with LB operator and approximating spectral filters using Chebyshev, Laguerre, and Hermite polynomials.
result Classification accuracy of LB-CNN is not dependent on the type of polynomials or operators.
Study uncovers scaling laws and spectral properties of shallow neural networks.
problem Understanding scaling laws and spectral properties of shallow neural networks.
method Leveraging connections with matrix compressed sensing and LASSO, derived a phase diagram for excess risk.
result Uncovered crossovers between scaling regimes and plateau behaviors, validated empirical observations.
Introduces Spectral Graph Network combining spatial and spectral message passing.
problem Relational reasoning in graph structured data.
method Applies message passing to both spatial and spectral domains of a graph.
result Promotes efficient training with fewer iterations and robustness to edge dropout.
Graph pruning improves neural network performance by addressing squashing and smoothing issues.
problem Over-squashing and over-smoothing in Graph Neural Networks.
method Proposes edge deletions to simultaneously address over-squashing and over-smoothing, optimizing spectral gap.
result Edge deletions improve generalization and distinguishability of nodes of different classes.
Enhances graph neural networks with spectral and topological information.
problem Improving graph neural networks' expressivity beyond Weisfeiler-Leman hierarchy.
method Integrates spectral information into Persistent Homology diagrams.
result SpectRe is strictly more expressive than PH and spectral information alone.
Study identifies cancer genes through graph anomaly analysis of protein interactions.
problem Insufficient modeling of biological information in protein interaction networks for cancer gene identification.
method Proposes HIerarchical-Perspective Graph Neural Network (HIPGNN) to detect weight heterogeneity and spectral flattening in cancer gene nodes.
result HIPGNN detects weight heterogeneity and spectral flattening, leading to improved cancer gene identification.
Study shows deterministic equivalent for neural network kernel convergence.
problem Understanding convergence of neural network kernels.
method Analyzes empirical spectral distribution of Conjugate Kernel, proving convergence to a deterministic limit.
result Obtains a deterministic equivalent for the Stieltjes transform and resolvent of the Conjugate Kernel.
This paper presents a margin-based multiclass generalization bound for neural networks that scales with their margin-normalized "spectral complexity": their Lipschitz constant, meaning the product of the spectral norms of the weight matrices, times a certain correction factor. This bound is empirically investigated for…
Meta-learning for Koopman spectral analysis with short time-series data.
problem Lack of long time-series for training embedding functions in Koopman spectral analysis.
method Meta-learning approach using bidirectional LSTM and neural network to estimate embedding functions from short time-series.
result The proposed method achieves better performance in eigenvalue estimation and future prediction compared to existing methods.
Developed a framework for designing filters in spectral GCNNs with improved performance.
problem Designing effective filters for spectral GCNNs with regularization properties.
method Exploring regularization properties of graph Laplacian and proposing a generalized framework for filter design.
result New filters derived from the framework outperform state-of-the-art techniques in semi-supervised node classification.
Spectro-Riemannian Graph Neural Networks integrate spectral and curvature signals for better graph representation learning.
problem Enhance graph representation learning by leveraging spectral and curvature signals.
method Proposes Spectro-Riemannian Graph Neural Networks (CUSP) that combines spectral and curvature insights.
result Empirical evaluation shows CUSP outperforms state-of-the-art models by up to 5.3%.
SpGAT learns graph representations using spectral attention for efficiency.
problem Efficiently capturing global graph patterns with minimal parameters.
method Introduces Spectral Graph Attention Network (SpGAT) using spectral domain attention mechanisms and a fast Chebychev approximation.
result SpGAT achieves better global pattern recognition with fewer parameters compared to GAT.
New centrality-based graph shift operators improve graph neural networks.
problem Improving graph neural networks by enhancing graph shift operators.
method Proposed Centrality Graph Shift Operators (CGSOs) using global centrality metrics.
result CGSOs lead to improved performance in graph neural networks on real-world datasets.
Unified framework for analyzing graph neural operators converging to graph limits.
problem Analyzing convergence of graph neural operators to graph limits.
method Develops a unified spectral framework for graph neural operators under various graphon assumptions.
result Unified framework enables direct comparison of convergence rates and tradeoffs.
Study on neural networks' sample complexity with one hidden layer.
problem Understanding how sample complexity is affected by network architecture and norm constraints.
method Norm-based uniform convergence bounds for scalar-valued one-hidden-layer networks, focusing on spectral and Frobenius norms.
result Spectral norm control is insufficient for uniform convergence guarantees, but Frobenius norm control is sufficient, with conditions.
Novel GNN for signed and directed networks using magnetic signed Laplacian.
problem Efficiently modeling signed and directed networks for tasks like clustering and link prediction.
method Introduced a magnetic signed Laplacian for directed signed graphs, used it to construct a spectral GNN.
result Demonstrated effective performance on tasks involving signed and directional information.
New method predicts neural network performance using free probability theory.
problem Stability and performance prediction of feed-forward neural networks.
method Free Probability Theory and homotopy method for Jacobian spectral density computation.
result FPT metrics correlate highly with final test accuracies of neural networks.
Gaussian process models simplify neural network behavior for easier understanding.
problem Understanding and predicting the behavior of deep learning systems.
method Constructing surrogate models using Gaussian processes from finite neural networks.
result Surrogate models capture phenomena like spectral bias and predict generalization well.
The study analyzes spectral algorithms for kernel methods and derives generalization error.
problem Estimating generalization error of spectral algorithms for kernel methods.
method Considered spectral algorithms including KRR and GD, derived generalization error as a functional of learning profile.
result Showed the loss localizes on certain spectral scales and conjectured universality of the loss for noisy observations.