NTK-SAP improves neural network pruning by aligning training dynamics.
problem Improving neural network pruning to reduce training time and memory.
method Prune connections based on the spectrum of the Neural Tangent Kernel (NTK), using multiple random weight realizations and random inputs.
result Empirically, NTK-SAP achieves better performance than all baselines on multiple datasets.
Multi-parameter cognition in a cognitive radio network (CRN) provides a more thorough understanding of the radio environments, and could potentially lead to far more intelligent and efficient spectrum usage for a secondary user. In this paper, we investigate the multi-parameter cognition problem for a CRN where the pri…
FreSh shifts model's initial frequency spectrum to match target signal, improving neural representation performance.
problem MLPs' low-frequency bias limits capturing high-frequency details accurately.
method FreSh selects embedding hyperparameters to align model's initial output spectrum with target signal's spectrum.
result FreSh improves performance across various neural representation methods and tasks with minimal computational overhead.
Expressiveness and generalization of deep models was recently addressed via the connection between neural networks (NNs) and kernel learning, where first-order dynamics of NN during a gradient-descent (GD) optimization were related to gradient similarity kernel, also known as Neural Tangent Kernel (NTK). In the majorit…
We propose a flexible framework for spectral conversion (SC) that facilitates training with unaligned corpora. Many SC frameworks require parallel corpora, phonetic alignments, or explicit frame-wise correspondence for learning conversion functions or for synthesizing a target spectrum with the aid of alignments. Howev…
Study detects signal in financial stock correlations using phase-ordering kinetics.
problem Detecting meaningful signals in financial stock return correlations.
method Stochastic field theory model to establish a detection threshold.
result Detection of a signal in the largest eigenvalues of the stock return correlation matrix.
TKRR improves KRR performance by aligning target functions with kernels.
problem Improving kernel ridge regression performance through target alignment.
method Focuses on truncated kernel ridge regression (TKRR) with an additional spectral truncation parameter.
result TKRR can achieve faster rates than full KRR, reaching parametric rates.
Muon replaces matrix gradient with polar factor, optimizing flat spectrum updates
problem Optimization bias in matrix updates
method Using polar factor of gradient
result Muon update maximizes entropy among bounded updates
A common assumption in semi-supervised learning with graph models is that the class label function varies smoothly on the data graph, resulting in the rather strict prior that the label function has low-frequency content. Meanwhile, in many classification problems, the label function may vary abruptly in certain graph …
Proposes a method to compare noisy high-dimensional datasets with low-dimensional manifolds.
problem Comparing distributions on manifolds in noisy high-dimensional datasets.
method Linking low-rank structure to manifold geometry, developing a scale-invariant distance measure.
result Superior robustness and statistical power compared to existing methods.
Muon outperforms GD in associative memory learning by balancing frequency components.
problem Training dynamics and scaling behavior of Muon in associative memory learning.
method Study of Muon in a linear associative memory model with softmax retrieval and hierarchical frequency spectrum over query-answer pairs.
result Muon achieves exponential speedup over GD in noiseless case and superior scaling efficiency in noisy case.
A new debiasing method for high-dimensional regression with applications to PCR.
problem Debiasing in high-dimensional statistics with i.i.d. samples and sub-Gaussian covariates.
method Spectrum-Aware Debiasing using rescaled gradient descent with spectral information.
result Achieves debiasing in broader contexts with structured dependencies, heavy tails, and low-rank structures.
Improved scaling laws in linear regression using data reuse.
problem Sustainability of neural scaling laws when running out of new data.
method Data reuse in multi-pass stochastic gradient descent (multi-pass SGD) for M-dimensional linear models trained on N data with sketched features. result Multi-pass SGD achieves a test error of Θ(M1−b+L(1−b)/a) with L>N, improving scaling laws in data-constrained regimes. A new method matches similar regions in non-rigid shapes using spectra of differential operators.
problem Evaluating similarity of non-rigid shapes with partiality.
method Alignment of spectra of differential operators (SI-LBO and regular LBO) on a manifold with multiple metrics.
result Matching spectra outperforms competing methods on standard benchmarks.
This paper introduces DCE for better counterfactual explanations using optimal transport.
problem Lack of nuanced distributional characteristics in existing counterfactual explanations.
method Formulates a chance-constrained optimization problem using optimal transport to derive counterfactual distributions.
result DCE provides deeper insights into decision-making models by aligning counterfactual distributions with factual ones.
New method uses Riemannian geometry to describe molecular shapes.
problem Predicting drug-like molecules using shape similarity.
method Riemannian geometry applied to molecular surfaces.
result RGMolSA method captures molecular shape effectively.
This paper addresses the problem of localizing audio sources using binaural measurements. We propose a supervised formulation that simultaneously localizes multiple sources at different locations. The approach is intrinsically efficient because, contrary to prior work, it relies neither on source separation, nor on mon…
We define a new spectrum for compact length spaces and Riemannian manifolds called the "covering spectrum" which roughly measures the size of the one dimensional holes in the space. More specifically, the covering spectrum is a set of real numbers δ>0 which identify the distinct δ covers of the space. We investigat…
The subject of this paper is the relationship among the marked length spectrum, the length spectrum, the Laplace spectrum on functions, and the Laplace spectrum on forms on Riemannian nilmanifolds. In particular, we show that for a large class of three-step nilmanifolds, if a pair of nilmanifolds in this class has the …
The subject of this paper is the relationship among the marked length spectrum, the length spectrum, the Laplace spectrum on functions, and the Laplace spectrum on forms on Riemannian nilmanifolds. In particular, we show that for a large class of three-step nilmanifolds, if a pair of nilmanifolds in this class has the …
Study the energy spectrum of metrics on surfaces and its relation to simple length spectrum.
problem Relate the energy spectrum to the simple length spectrum of metrics on surfaces.
method Analyze the energy spectrum of metrics on surfaces and their Teichmüller spaces, considering homotopy conditions.
result The energy spectrum determines the simple length spectrum under certain conditions.
Unified framework explains why overfitting is benign in interpolating learning.
problem Understanding why overfitting is benign in highly overparameterized models.
method Spectral-transport stability framework.
result Sharp benign-overfitting criterion and explicit phase-transition rates.
New approach categorizes objective functions for embodied agents.
problem Understanding how objectives relate to each other and discovering new objectives.
method Introducing Action Perception Divergence (APD) to categorize objective functions.
result Introduces a spectrum of objectives from narrow to general, explaining various unsupervised objectives.
Study shows spectrum properties for specific Hadamard manifolds.
problem Spectrum properties of Hadamard manifolds.
method Absolute continuity and spectrum determination for two classes of Hadamard manifolds.
result Spectrum properties determined for specific Hadamard manifolds.
Proofs high-dimensional spectrum convergence of weighted sample covariance.
problem High-dimensional spectrum convergence of weighted sample covariance.
method Proposes a new, concise proof with stronger assumptions.
result Spectrum convergence proven for different weight distributions.
We study the volatility functional inference by Fourier transforms. This spectral framework is advantageous in that it harnesses the power of harmonic analysis to handle missing data and asynchronous observations without any artificial time alignment nor data imputation. Under conditions, this spectral approach is cons…
Iterative method 'Concent' corrects spectrum bias in covariance matrices.
problem Consistent bias in the spectrum of covariance matrices.
method 'Concent' iterative algorithm.
result Corrects spectrum bias for small and moderate dimensions.
Develops a new spectrum for annular links, recovering a transverse invariant at extreme gradings.
problem Understanding transverse link invariants in the annular setting.
method Constructs a stable homotopy type for annular links and defines a map to the Khovanov skein spectrum.
result At extreme gradings, the map from the Khovanov spectrum to the Khovanov skein spectrum recovers the cohomotopy transverse invariant.
The spectrum of certain manifolds matches that of hyperbolic space if the bottom spectrum is maximal.
problem Investigating spectral rigidity of manifolds with Ricci bounded below and maximal bottom spectrum.
method Analyzing the spectrum of the Laplacian on manifolds with specific Ricci curvature bounds.
result The spectrum of the manifold coincides with that of hyperbolic space if the bottom spectrum is maximal.
TopoNTK kernel captures higher-order interactions in simplicial complexes.
problem Graph neural networks miss higher-order interactions in relational systems.
method Introduces TopoNTK, an infinite-width kernel for simplicial message passing.
result TopoNTK captures topology invisible to graph kernels, improving expressivity and interpretability.
Lower bounds for Hodge-Laplacian spectrum on orbifolds.
problem Finding bounds for the essential spectrum of Hodge-Laplacian.
method Deriving lower bounds for the essential spectrum of the Hodge-Laplacian on geometrically finite orbifolds and their suborbifolds.
result Lower bounds for the essential spectrum of the Hodge-Laplacian.
How to generalize the concept of eigenvalues of quadratic forms to eigenvalues of arbitrary, even, homogeneous continuous functionals, if stability of the set of eigenvalues under small perturbations is required? We compare two possible generalizations, Gromov's homotopy significant spectrum and the Krasnoselskii spect…
We construct Riemannian manifolds with singular continuous spectrum embedded in the absolutely continuous spectrum of the Laplacian. Our manifolds are asymptotically hyperbolic with sharp curvature bounds.
Word embeddings learnt from large corpora have been adopted in various applications in natural language processing and served as the general input representations to learning systems. Recently, a series of post-processing methods have been proposed to boost the performance of word embeddings on similarity comparison an…
Trapezoids uniquely identified by their Dirichlet Laplace spectrum.
problem Identifying trapezoids based on their spectral properties.
method Analyzing the Dirichlet Laplace spectrum of non-obtuse trapezoids.
result Non-obtuse trapezoids are uniquely determined by their Dirichlet Laplace spectrum.
Dynamic Time Warping improves regression accuracy on spectroscopy data.
problem Improving regression accuracy on spectroscopy data with DTW when data is across multiple wavelengths.
method Illustrated DTW's effectiveness on spectroscopy time-series data, showing its benefits in improving regression accuracy when only a single wavelength is considered. DTW combined with k-Nearest Neighbour reveals similarities and differences at the time-series level.
result DTW improves regression accuracy on spectroscopy data, especially when considering a single wavelength.
In 2004, Sormani and Wei introduced the covering spectrum: a geometric invariant that isolates part of the length spectrum of a Riemannian manifold. In their paper they observed that certain Sunada isospectral manifolds share the same covering spectrum, thus raising the question of whether the covering spectrum is a sp…
Survey on bottom of spectrum of Hodge Laplacian on complete noncompact Kähler manifolds
problem Bottom of the spectrum of Hodge Laplacian on complete noncompact Kähler manifolds
method Survey on Kähler hyperbolic manifolds and bounded symmetric domains
result Proposed several open problems
The paper extends decay estimates to graphs with positive spectrum.
problem Proving decay estimates for nonnegative functions on graphs.
method Sharp ℓ2 decay estimates for nonnegative generalized subharmonic functions. result Extends Li and Wang's result to graphs with positive Laplacian spectrum.
Upper bounds for volume spectrum depend on volume, dimension, and a conformal invariant.
problem Bounding the volume spectrum of Riemannian manifolds.
method Proves upper bounds that depend on volume, dimension, and a conformal invariant.
result Upper bounds for the volume spectrum are established.
Upper bounds for essential spectrum of minimal submanifolds linked to volume growth.
problem Estimating the essential spectrum of minimal submanifolds.
method Using volume growth to bound the bottom of the essential spectrum.
result Improved essential spectrum estimate for minimal submanifolds.
Study on magnetic Dirac operators and their spectrum.
problem Understanding the spectrum of magnetic Dirac operators.
method Analysis of magnetic Dirac operators over complete Riemannian manifolds.
result Find sufficient conditions for maximal or discrete spectrum.
This paper surveys scalable automated alignment methods for LLMs.
problem Scalability issues in traditional human-annotated alignment methods for LLMs.
method Categorizes and discusses various automated alignment methods.
result Emerging automated alignment methods are effective and scalable.
Notes on continuity of discrete-spectrum Fredholm operators.
problem Continuity properties of discrete-spectrum families of Fredholm operators.
method Relates recent work on discrete-spectrum families to classical continuity properties.
result Establishes connections between new and classical concepts.
Graph machine learning lacks a balanced theory, focusing on expressive power and optimization.
problem Insufficient theoretical understanding of GNNs' generalization behavior.
method Develop a balanced theory focusing on expressive power, generalization, and optimization.
result Theoretical advancements need to align with practical success in graph machine learning.
Inference-aware meta-alignment of LLMs reduces computational cost.
problem Aligning LLMs to diverse human preferences is challenging due to conflicting criteria.
method IAMA trains a base model to be aligned to multiple tasks via different inference-time alignment algorithms, using non-linear GRPO for optimization.
result IAMA enables effective alignment of LLMs to multiple criteria with limited computational budget.
Conformal Alignment ensures trustworthy outputs from foundation models.
problem Ensuring outputs from foundation models align with human values in high-stakes tasks.
method A framework that trains an alignment predictor using reference data to select trustworthy outputs.
result Conformal Alignment accurately identifies trustworthy outputs via lightweight training over moderate reference data.
The rigidity of marked length spectrum for closed hyperbolic surfaces due to Fricke-Klein [7] has been the motivation of many different rigidity results, specially for manifolds of negative curvature. From the works of Vigneras [18], Sunada [17] and many other authors this result is far from being true for the unmarked…