Spectral regularization simplifies sequence models by focusing on grammatical simplicity.
problem Sequence modeling challenges in learning tasks.
method Introduces spectral regularization based on Hankel matrices and trace norm, addressing bi-infinite matrices with an unbiased estimator.
result Demonstrates spectral regularization's potential benefits on Tomita grammars.
Proves spectral simplicity of Hodge Laplacian and curl operator along metric families.
problem Simplicity of Hodge Laplacian and curl operator eigenvalues along metric families.
method Generalized Teytel's method to compute meagre codimension of metrics with specific eigenvalue multiplicities.
result Simplicity of Hodge Laplacian and curl operator is not a meagre codimension 2 property.
Random SNNs are stable and simple, with low-frequency Fourier spectra.
problem Stability and robustness of spiking neural networks.
method Boolean function analysis and Fourier spectrum concentration.
result Random LIF-SNNs are stable and biased towards simple functions.
Early neural networks can be simplified to linear models, revealing surprising simplicity.
problem Complexity of neural network learning dynamics.
method Formal proof and empirical verification of early-time learning dynamics of neural networks.
result Early learning dynamics of neural networks can be approximated by simple linear models.
We study the sample complexity of learning a high-dimensional simplex from a set of points uniformly sampled from its interior. Learning of simplices is a long studied problem in computer science and has applications in computational biology and remote sensing, mostly under the name of `spectral unmixing'. We theoretic…
New spectral clustering method using LASSO regularization for robust graph partitioning.
problem Lack of theoretical guarantees for spectral clustering on general graph models.
method 1-spectral clustering on a new random model with LASSO regularization.
result Effective and robust to small noise perturbations, validated by simulations and real data.
A theory of feature geometry using spectral analysis of weight matrices.
problem Current methods decompose neural network activations into sparse linear features, losing geometric structure.
method Develops a theory by analyzing the spectra of weight-derived matrices, introducing the frame operator.
result Features collapse onto single eigenspaces, organizing into tight frames, and admit discrete classification.
TopoNTK kernel captures higher-order interactions in simplicial complexes.
problem Graph neural networks miss higher-order interactions in relational systems.
method Introduces TopoNTK, an infinite-width kernel for simplicial message passing.
result TopoNTK captures topology invisible to graph kernels, improving expressivity and interpretability.
We outline a novel clustering scheme for simplicial complexes that produces clusters of simplices in a way that is sensitive to the homology of the complex. The method is inspired by, and can be seen as a higher-dimensional version of, graph spectral clustering. The algorithm involves only sparse eigenproblems, and is …
New spectral mixture representation for isotropic kernels simplifies random Fourier features.
problem Applying Random Fourier Features to complex kernels.
method Decompose isotropic kernels into scale mixtures of α-stable random vectors.
result Constructive spectral sampling formula for various kernels.
The optimization problem behind neural networks is highly non-convex. Training with stochastic gradient descent and variants requires careful parameter tuning and provides no guarantee to achieve the global optimum. In contrast we show under quite weak assumptions on the data that a particular class of feedforward neur…
Proposes a probabilistic framework for stationary topological signals on simplicial complexes.
problem Complex data structures require new models and tools.
method Generalizes stationarity to topological signals on simplicial complexes.
result Defines topological power spectral density (PSD) for stationary signals.
Spectral methods simplify data analysis, improving accuracy and stability.
problem Extracting meaningful information from noisy, incomplete data.
method Eigenvalues and eigenvectors of matrices constructed from data.
result Spectral methods are effective and can be analyzed using modern statistical theory.
New algorithm recovers communities in broader network models.
problem Finding communities in complex networks is challenging.
method Spectral clustering on Preference Frame Models with Normalized Laplacian.
result Spectral clustering works on broader network models with similar guarantees.
A new method for risk-sensitive reinforcement learning using Spectral Risk Measures.
problem Incorporating risk sensitivity into reinforcement learning algorithms.
method Proposes a novel framework for optimizing Spectral Risk Measures in both online and offline RL algorithms.
result Demonstrates consistent outperformance over existing risk-sensitive methods in various domains.
The paper develops formulas for hyperbolic simplices based on edge lengths.
problem Understanding the geometry of hyperbolic simplices using only edge lengths.
method Develops geometric formulas for hyperbolic simplices based on edge lengths.
result Distance and projection formulas in hyperbolic simplices.
Estimates dimensions of maximal simplices for rational and irrational trees in Outer space.
problem Understanding the structure of trees in Outer space.
method Associate simplices to R-trees and estimate their dimensions. result Estimates the dimensions of maximal simplices for both rational and irrational trees.
The paper establishes conditions for Riemannian connections and semi-simplicity of Lie algebras using spray structures.
problem Conditions for Riemannian connections and semi-simplicity of Lie algebras.
method Using almost product structures and spray, the paper provides necessary and sufficient conditions for these properties.
result Equivalence of semi-simplicity of Lie algebras to derived ideal coincidence, interiority of derivations, and adjoint representation semi-simplicity.
It is proved that the volume of spherical or hyperbolic simplices, when considered as a function of the dihedral angles, can be extended continuously to degenerated simplices.
In this article Ehrhart quasi-polynomials of simplices are employed to determine isospectral lens spaces in terms of a finite set of numbers. Using the natural lattice associated with a lens space the associated toric variety of a lens space is introduced. It is proved that if two lens spaces are isospectral then the d…
Geodesic simplices in pseudo-hyperbolic space get a cohomological treatment.
problem Understanding geodesic simplices in pseudo-hyperbolic space.
method Cohomological interpretation and necessary/sufficient condition formulation.
result Every ideal geodesic polytope in (2,2) pseudo-hyperbolic space has finite volume. New algorithms detect and estimate rank-one signals with prior directional information.
problem Detecting and estimating rank-one signals with directional prior information.
method Construct nonlinear Laplacians and examine top eigenvalues and eigenvectors.
result Nonlinear Laplacian algorithms outperform direct spectral methods for biased signals.
New graph embedding method improves link prediction and node classification.
problem Improving graph embedding methods for better node representation.
method Spectral-biased random walks with neighborhood similarity bias.
result Significantly improves link prediction and node classification.
Study PL bordism theories with quantitative bounds on filling simplices.
problem Understanding PL bordism theories with geometric constraints.
method Quantitative analysis of PL manifolds and exotic theories.
result Bounding the number of simplices in fillings of cycles.
Framework reduces simplicity bias in NNs, improving OOD generalization and robustness.
problem Simplicity bias in deep learning models leads to biased predictions and poor OOD generalization.
method Proposes a framework that regularizes conditional mutual information to encourage use of diverse features.
result Demonstrates effectiveness in various settings, enhancing OOD generalization and robustness.
This paper enhances privacy-preserving randomized power method for large datasets.
problem Privacy issues in applying randomized power method to large datasets containing personal information.
method Proposes enhanced privacy-preserving variants of the randomized power method, including a variant with reduced noise and a decentralized framework.
result Tighter convergence bounds and empirical comparisons with previous work in real recommendation datasets.
New framework shows C∗-simplicity for groups without certain subalgebras.
problem Characterizing C∗-simplicity of groups. method Introducing confined subalgebras and Uniformly Recurrent States.
result A countable discrete group is C∗-simple if it has no non-trivial amenable confined subalgebras. In this article, we prove a theorem comparing the dihedral angles of simplices in the hyperbolic, spherical and Euclidean geometries.
We study a natural intrinsic definition of geometric simplices in Riemannian manifolds of arbitrary dimension n, and exploit these simplices to obtain criteria for triangulating compact Riemannian manifolds. These geometric simplices are defined using Karcher means. Given a finite set of vertices in a convex set on t…
New non-separable covariance kernels for spatiotemporal data derived from harmonic oscillator physics.
problem Capturing complex spatiotemporal dependencies in Gaussian processes.
method Hybrid spectral method based on the harmonic oscillator, deriving explicit covariance kernels.
result Explicit non-separable covariance kernels with space-time interactions.
Research reveals simplicity bias in random logistic map, impacting data analysis and forecasting.
problem Simplicity bias in dynamical systems and its impact on data analysis and prediction.
method Examined the logistic map and random logistic map, focusing on simplicity bias and noise effects.
result Simplicity bias is observable in the random logistic map, persisting even with small noise levels.
The Apollonius theorem is generalized for m-simplices, with applications in geometry and optimization.
problem Generalizing the Apollonius theorem for m-simplices.
method Direct generalization of the theorem to m-simplices in n-dimensional space.
result Applications in geometry and optimization, including minimal surface enclosures, simplex thickness, and root-finding methods.
Similar simplices can be inscribed in most smoothly embedded spheres.
problem Inscribing families of similar simplices in spheres.
method Diffeomorphic mapping and techniques from previous work on inscribing triangles.
result A dense family of spheres allows inscribing similar simplices of every pose.
We study prismatics sets analogously to simplical sets except that realization involves prisms, i.e., products of simplices rather than just simplices. Particular examples are the prismatic subdivision of a simplicial set S and the prismatic star of S. Both have the same homotopy type as S and in particular the latter …
Simplicial sets deformation retract onto transverse simplices.
problem Deformation retraction of simplicial sets.
method Showed deformation retraction of singular simplicial set onto transverse simplices.
result Singular simplicial set deformation retracts onto transverse simplices.
We generalize the very well known boundary operator of the ordinary singular homology theory, defined in many books about algebraic topology. We describe a variant of this ordinary simplicial boundary operator where the usual boundary (n-1)-simplices of each n-simplex are replaced by combinations of internal (n-1)- sim…
The paper explores how simplicity leads to better out-of-distribution generalization in models.
problem Understanding the theoretical principles behind out-of-distribution (OOD) generalization in modern models.
method Examining diffusion models in image generation to analyze compositional generalization abilities and develop a theoretical framework for simplicity-based OOD generalization.
result The true, generalizable model corresponds to the simplest among consistent models, and this simplicity can be quantified and used to establish sample complexity guarantees.
The study reveals simplicity bias in neural networks leading to better compositional mappings.
problem Understanding when and how to encourage neural networks to learn compositional mappings.
method Examined compositional mappings through coding length and gradient descent dynamics.
result Neural networks tend to learn the simplest bijections, explaining their good generalization.
Study on simplicity of Lie skew braces, proving new results for compact cases.
problem Simplicity of Lie skew braces, focusing on compact connected cases.
method Reviewing correspondence, investigating ideals and rigidity, proving main result for compact Lie skew braces.
result Compact connected simple Lie skew braces are either trivial or have simple underlying Lie groups.
Triangulations of R^n have at least tensor rank of determinant simplices.
problem Understanding the minimum number of simplices in periodic triangulations of R^n.
method Proving lower bounds on the number of simplices in periodic triangulations of R^n.
result Lower bounds on the number of simplices in periodic triangulations of R^n.
Two-layer networks favor simple features, especially in complex datasets.
problem Simplicity bias in neural networks over-reliing on simple features.
method Characterization of two-layer neural networks with small weights and gradient flow.
result Features learned in middle training stages are more useful for out-of-distribution transfer.
Adam avoids simplicity bias in neural networks, leading to better generalization.
problem Simplicity bias in neural networks trained with SGD.
method Comparison of Adam and GD on binary classification tasks with Gaussian data.
result Adam leads to richer and more diverse features, improving generalization.
We give several new criteria to judge whether a simple convex polytope in a Euclidean space is combinatorially equivalent to a product of simplices. These criteria are mixtures of combinatorial, geometrical and topological conditions that are inspired by the ideas from toric topology.
Neural nets learn simple distributions first, then more complex ones.
problem Understanding how neural networks generalize from simple to complex functions.
method Stochastic gradient descent training, synthetic data, CIFAR10, ImageNet pre-training.
result Neural networks initially use lower-order statistics, then higher-order ones.
SNN architecture shows gradient descent converges to regularized solution in matrix sensing problems.
problem Understanding implicit regularization in neural networks for matrix sensing.
method Developed Spectral Neural Networks (SNN) for matrix learning problems, rigorously demonstrating implicit regularization.
result Gradient descent converges to the solution of a regularized learning problem in matrix sensing problems.
The paper proves eigenvalues are simple for specific operators on bundles.
problem Eigenvalue simplicity for connection Laplacian and G-simplicity on bundles. method Analyzes connections on vector bundles and principal bundles, proving eigenvalue simplicity for a residual set of connections.
result Eigenvalues of the connection Laplacian and Laplace-Beltrami operator are simple for specified conditions.
Proposes a new model for mixed membership in Gaussian mixture.
problem Limited to single component membership in Gaussian mixture models.
method Mixed membership sub-Gaussian model, spectral algorithm.
result Estimation error can be made arbitrarily small with high probability.
Ancient formula connects volume forms and infinitesimal square volumes in manifolds.
problem Relating volume forms and infinitesimal square volumes in Riemannian manifolds.
method Uses Heron's formula to link these concepts.
result Established a connection between volume forms and infinitesimal square volumes.