Study characterizes cryospheric spectral feature space using joint PC+t-SNE approach.
problem Characterize cryospheric spectral feature space for remote sensing applications.
method Compare and contrast two approaches for identifying feature space basis vectors via dimensionality reduction (PCA and t-SNE).
result Joint characterization reveals distinct continua and clusters of ice reflectance properties.
A theory of feature geometry using spectral analysis of weight matrices.
problem Current methods decompose neural network activations into sparse linear features, losing geometric structure.
method Develops a theory by analyzing the spectra of weight-derived matrices, introducing the frame operator.
result Features collapse onto single eigenspaces, organizing into tight frames, and admit discrete classification.
Irregular features disrupt the desired classification. In this paper, we consider aggressively modifying scales of features in the original space according to the label information to form well-separated clusters in low-dimensional space. The proposed method exploits spectral clustering to derive scaling factors that a…
Random feature approximation speeds up spectral methods and improves learning rates.
problem Improving the efficiency and generalization of spectral methods in large-scale algorithms.
method Combining random feature approximation with spectral regularization methods.
result Optimal learning rates for estimators over various regularity classes, including those not in the RKHS.
A novel graph spectral method for mixed categorical and numerical data.
problem Feature learning for mixed data types (numerical and categorical).
method Graph spectral decomposition of the graph Laplacian to model probabilistic dependence structure.
result Increased separability and clusterability of observations in the transformed feature space.
Nonlinear kernels can be approximated using finite-dimensional feature maps for efficient risk minimization. Due to the inherent trade-off between the dimension of the (mapped) feature space and the approximation accuracy, the key problem is to identify promising (explicit) features leading to a satisfactory out-of-sam…
Spectral dimensionality reduction methods enable linear separations of complex data with high-dimensional features in a reduced space. However, these methods do not always give the desired results due to irregularities or uncertainties of the data. Thus, we consider aggressively modifying the scales of the features to …
Extends random feature analysis to spectral methods and improves learning rates.
problem Improving generalization properties of spectral methods in large-scale learning.
method Extends random feature analysis to a broad class of spectral regularization techniques, including gradient descent and Nesterov method.
result Obtains optimal learning rates for regularity classes, including those not in the RKHS.
Paper proposes JDR to denoise graph features and rewire graphs for better node classification.
problem Jointly denoise noisy graph features and rewire graphs for improved node classification.
method Align leading spectral spaces of graph and feature matrices to solve non-convex optimization problem.
result JDR consistently outperforms existing methods on various node classification tasks.
LASE improves local network structure visualization by targeting locally low-dimensional regions.
problem Global spectral embedding fails to capture local geometric features in sparse, transitive networks.
method Local Adjacency Spectral Embedding (LASE) using weighted spectral decomposition.
result LASE reveals locally low-dimensional structure, improving local reconstruction and visualization.
Deep learning explained through spectral filtering of hierarchical features.
problem Understanding how deep neural networks learn useful representations from data.
method Neural Low-Degree Filtering (Neural LoFi) as a stylized limit of gradient-based training.
result Predicts how representations are selected layer by layer and explains emergence of concepts.
New methods avoid spectral pollution in transfer operators for accurate analysis.
problem Spectral pollution in finite-dimensional approximations of transfer operators.
method Algorithms for computing spectral properties of transfer operators without spectral pollution.
result Accurate spectral estimation across various applications, including protein folding models.
We introduce single-set spectral sparsification as a deterministic sampling based feature selection technique for regularized least squares classification, which is the classification analogue to ridge regression. The method is unsupervised and gives worst-case guarantees of the generalization power of the classificati…
A new method learns outcome-aware spectral features for causal effect estimation.
problem Estimation of causal effects in the presence of hidden confounders.
method Augmented Spectral Feature Learning framework that minimizes a contrastive loss derived from an augmented operator incorporating outcome information.
result Our method remains effective even under spectral misalignment.
Gradient descent reshapes the function space of neural networks.
problem Understanding how feature learning affects the function space of neural networks.
method Characterized the evolution of the feature space during training using a two-layer neural network.
result Gradient descent induces a data-adaptive deformation that selectively enhances signal-aligned directions.
SASE improves attributed graph clustering for large graphs with linear time and space complexity.
problem Challenges in clustering large attributed graphs due to high computational and memory costs.
method SASE combines node features smoothing, scalable spectral clustering, and adaptive order selection.
result SASE achieves a 6.9% improvement in ACC and a 5.87x speedup on the ArXiv dataset.
Spectral feature learning improves IV regression for causal effect estimation.
problem Estimating causal effects in the presence of hidden confounders.
method Two-stage least squares estimator based on spectral features.
result Performance of the method depends on strong spectral alignment and slow eigenvalue decay.
Random Fourier features is one of the most popular techniques for scaling up kernel methods, such as kernel ridge regression. However, despite impressive empirical results, the statistical properties of random Fourier features are still not well understood. In this paper we take steps toward filling this gap. Specifica…
Graph classification improved using spectral features and wavelet filters.
problem Categorizing graphs based on their structure and node attributes.
method Derived spectral features from graph signal processing, designed two Gaussian process models: one simple and one sophisticated.
result Simple and sophisticated Gaussian process models yield competitive performance, including well-calibrated uncertainty estimates.
Survey of spectral, probabilistic, and deep metric learning methods.
problem Developing effective distance metrics for various machine learning tasks.
method Divided into spectral, probabilistic, and deep approaches, covering various techniques and their applications.
result Comprehensive overview of metric learning methods, including new developments and applications.
LLT transforms time series features based on linear laws.
problem Classifying univariate and multivariate time series.
method Time-delay embedding, spectral decomposition, and feature transformation.
result Transformed features improve classification accuracy.
Neural networks are capable of learning rich, nonlinear feature representations shown to be beneficial in many predictive tasks. In this work, we use such models to explore different geographical feature representations in the context of predicting colorectal cancer survival curves for patients in the state of Iowa, sp…
A new method for nonstationary Gaussian processes using Fourier features.
problem Efficient simulation of nonstationary Gaussian processes with high-dimensional distributions.
method Discretizes the spectral representation of nonstationary processes, avoiding probability measure assumptions.
result An efficient low-rank approximation of nonstationary spectral densities, consistent and positive semi-definite.
A new convolutional spectral kernel network learns hierarchical and local features.
problem Lack of deep learning in non-stationary spectral kernels.
method Introduces convolutional filters and deep architectures into non-stationary spectral kernels, derives generalization error bounds, and introduces regularizers.
result Validated the effectiveness of the convolutional spectral kernel network on real-world datasets.
A new method for few-sample FS using manifold learning.
problem Few-sample supervised feature selection in high-dimensional spaces.
method Learn feature associations on manifolds, compute composite kernel, and use spectral analysis for FS score.
result Our method outperforms competitors in feature selection and classification accuracy.
The scale and complexity of modern data sets and the limitations associated with testing large numbers of hypotheses underline the need for feature selection methods. Spectral techniques rank features according to their degree of consistency with an underlying metric structure, but their current graph-based formulation…
Model reduction of Markov processes is a basic problem in modeling state-transition systems. Motivated by the state aggregation approach rooted in control theory, we study the statistical state compression of a discrete-state Markov chain from empirical trajectories. Through the lens of spectral decomposition, we study…
In the past decades, spectral clustering (SC) has become one of the most effective clustering algorithms. However, most previous studies focus on spectral clustering tasks with a fixed task set, which cannot incorporate with a new spectral clustering task without accessing to previously learned tasks. In this paper, we…
New spectral mixture representation for isotropic kernels simplifies random Fourier features.
problem Applying Random Fourier Features to complex kernels.
method Decompose isotropic kernels into scale mixtures of α-stable random vectors.
result Constructive spectral sampling formula for various kernels.
A new framework learns system design using neural features in function space.
problem Learning system design with neural feature extractors.
method Introduces feature geometry in function space, nesting technique for optimal feature approximation.
result Optimal features found from data samples using off-the-shelf architectures and optimizers.
While neural networks are powerful approximators used to classify or embed data into lower dimensional spaces, they are often regarded as black boxes with uninterpretable features. Here we propose Graph Spectral Regularization for making hidden layers more interpretable without significantly impacting performance on th…
Gaussian processes classify graphs using vertex and edge features.
problem Graph classification in machine learning.
method Transform graph features into spectral Euclidean features, apply Hodge decomposition.
result Gaussian processes can classify graphs using vertex and edge features.
Simple neural net outperforms complex uncertainty methods.
problem Reliable uncertainty estimation from deterministic models.
method A simple baseline using a single softmax neural net with residual connections and spectral normalization.
result Simple neural net outperforms DUQ and SNGP on uncertainty prediction.
Proves error bounds for state representation in RL using graph spectral features.
problem Addressing the curse of dimensionality in RL with unknown transition graphs.
method Proves upper bounds on approximation error of linear value function approximation using learned spectral features of the state-graph.
result Error bounds scale with algebraic connectivity and eigenvector estimation error.
Spectral algorithms on manifolds using diffusion kernels improve convergence rates.
problem The limitations of existing spectral algorithms in RKHSs for data on manifolds.
method Integrating manifold structure into spectral algorithms using heat kernel diffusion spaces.
result Spectral algorithms converge to the target function and its derivatives in a strong sense, with rates dependent on manifold intrinsic dimension.
Spectral clustering is one of the most effective clustering approaches that capture hidden cluster structures in the data. However, it does not scale well to large-scale problems due to its quadratic complexity in constructing similarity graphs and computing subsequent eigendecomposition. Although a number of methods h…
We present Spectral Inference Networks, a framework for learning eigenfunctions of linear operators by stochastic optimization. Spectral Inference Networks generalize Slow Feature Analysis to generic symmetric operators, and are closely related to Variational Monte Carlo methods from computational physics. As such, the…
The increasing access to brain signal data using electroencephalography creates new opportunities to study electrophysiological brain activity and perform ambulatory diagnoses of neuronal diseases. This work proposes a pairwise distance learning approach for Schizophrenia classification relying on the spectral properti…
ARFF reduces spectral bias in SGD-trained neural networks.
problem Spectral bias in two-layer neural networks.
method Comparison of SGD and ARFF on spectral bias and robustness.
result ARFF yields a closer to zero spectral bias compared to SGD.
New method combines spectral and sparse methods for Gaussian processes.
problem Efficiently fitting Gaussian processes to large datasets.
method Orthogonally decoupled variational Fourier features.
result Competitive performance on synthetic and real-world data.
Rewiring GNNs to optimize community and feature alignment improves their performance.
problem Improving GNNs' performance by addressing over-squashing and generalization issues.
method Three rewiring strategies: ComMa, FeaSt, and ComFy, targeting community structure, node labels, and their alignment.
result Rewiring strategies enhance GNNs' performance by optimizing label-community alignment.
We present a method for finding high density, low-dimensional structures in noisy point clouds. These structures are sets with zero Lebesgue measure with respect to the D-dimensional ambient space and belong to a d<D dimensional space. We call them "singular features." Hunting for singular features corresponds to f…
New Hilbert bundles with ends defined from indexed bases.
problem Defining new structures in Hilbert bundles.
method Indexed bases and unitary operators of finite propagation.
result Characteristic classes of Hilbert bundles with ends.
Given the apparent difficulty of learning models that are robust to adversarial perturbations, we propose tackling the simpler problem of developing adversarially robust features. Specifically, given a dataset and metric of interest, the goal is to return a function (or multiple functions) that 1) is robust to adversar…
DiSC detects feature clusters that differentiate between conditions.
problem Identifying subsets of features that differentiate between two conditions.
method Construct feature graphs, compute connectivity differences using spectral clustering.
result DiSC uncovers features that better differentiate between conditions.
Improved neural network predicts spectral functions more accurately than traditional methods.
problem Reconstructing real-time spectral functions from imaginary-time Green's functions is ill-posed and challenging.
method Feature Learning Network (FL-net) for enhanced prediction accuracy.
result FL-net achieves at least 20% improvement over traditional methods like MEM.
Generative model controls heterophily in graph signals.
problem Controlling heterophily in graph signals for better model effectiveness.
method Combines graphon-based generator with spectral filtering of Gaussian node features.
result Establishes theoretical guarantees for heterophily control and convergence.
Recently, there is increasing interest and research on the interpretability of machine learning models, for example how they transform and internally represent EEG signals in Brain-Computer Interface (BCI) applications. This can help to understand the limits of the model and how it may be improved, in addition to possi…