New framework explains normalizing flows' power and limitations.
problem Understanding the expressive power and limitations of normalizing flows.
method Theoretical framework for well-conditioned coupling-based normalizing flows and volume-preserving flows.
result RealNVP is distributionally universal, but volume-preserving flows are not.
GNNs with random node initialization are shown to be universally expressive.
problem Limitations of standard GNNs in distinguishing graphs.
method Random node initialization (RNI) to enhance GNNs' expressive power.
result GNNs with RNI are proven to be universally expressive.
HDNNs can approximate any continuous function, proving their expressivity.
problem Lack of a comprehensive study on the expressivity of HDNNs.
method Discretization of Hamiltonian Neural Ordinary Differential Equations (HNN-ODEs).
result HDNNs can approximate any continuous function over a compact domain.
The paper extends algebraic expression for subjective spatial patterns to include temporal patterns.
problem Studying subjective spatial and temporal patterns in machine learning.
method Develops X-form for algebraic expression of subjective spatial patterns and extends it to temporal patterns.
result Established algebraic expressions for both spatial and temporal patterns.
Improved bounds on neural network expressivity.
problem Understanding neural network expressivity and approximation capabilities.
method Improved bounds on the maximal number of linear regions of ReLU-networks.
result New insights into the expressivity of neural networks.
PLN-Nets with two linear layers and parallel LN achieve universal approximation.
problem Limitations of standard neural network architectures in universal approximation.
method Introduced PLN-Nets combining two linear layers with parallel LN.
result PLN-Nets achieve universal approximation, while standard LN has limited power.
We study the natural Kähler metrics on moduli spaces of stable oriented pairs in a very general framework, and we prove a universal formula expressing the Kähler class of such a moduli space in terms of characteristic classes of the universal bundle. We use these results to compute explicitly the volumina of certain Qu…
Study of Gödel Universe as Lie group with specific metric.
problem Characterize geodesics in the Gödel Universe.
method Geometric theory of optimal control applied to Lie groups with left-invariant Lorentz metrics.
result No closed timelike or isotropic geodesics in the Gödel Universe.
Residual flows are shown to approximate MMD well.
problem Lack of theoretical understanding of normalizing flows' expressiveness.
method Proved residual flows are universal approximators in MMD.
result Residual flows can approximate MMD with a bounded number of blocks.
Transformers struggle to approximate smooth functions, relying on piecewise constant approximations.
problem Understanding the expressivity of Transformers for function approximation.
method Theoretical analysis and experimental validation of Transformer's ability to approximate smooth functions.
result Transformers cannot reliably approximate smooth functions, relying on piecewise constant approximations.
Quadratic neurons enhance deep networks' approximation capabilities.
problem Improving deep networks' expressive power and efficiency.
method Introduced quadratic neurons and analyzed their performance in deep quadratic networks.
result Quadratic networks can approximate functions more efficiently and uniquely than conventional networks.
A `total Chern class' invariant of knots is defined. This is a universal Vassiliev invariant which is integral `on the level of Lie algebras' but it is not expressible as an integer sum of diagrams. The construction is motivated by similarities between the Kontsevich integral and the topological Chern character.
Universal functions derived for topological correlators in Yang-Mills theory.
problem Expressing correlation functions of topologically twisted Yang-Mills theory.
method Combining Seiberg-Witten geometry, u-plane integral, and blowup formula.
result Closed expressions for universal functions determined.
The study examines the universality of Gaussian data in high-dimensional generalized linear estimation.
problem Understanding when Gaussian data suffices for high-dimensional generalized linear estimation.
method Sharp asymptotic expressions for test and training errors in high-dimensional Gaussian mixture data with labels from a single-index model.
result The universality of Gaussian data in error estimation depends on the alignment between target weights and mixture cluster means and covariances.
Enhances generative models by improving expressivity without high computational cost.
problem Improving expressivity in generative models without increasing computational complexity.
method Proposes a new family of generative flows on an augmented data space, proving they can approximate a Hamiltonian ODE as a universal transport map.
result Demonstrates state-of-the-art performance on flow-based generative modeling benchmarks.
Geometric GNNs improve graph discrimination through GWL.
problem Discriminating geometric graphs embedded in Euclidean space.
method Proposed a geometric version of the Weisfeiler-Leman test (GWL) for geometric graphs.
result Characterized the expressive power of geometric GNNs based on physical symmetries.
We clarify the relation between the Dixmier-Douady class on the space of self adjoint Fredholm operators (`universal B-field') and the curvature of determinant bundles over infinite-dimensional Grassmannians. In particular, in the case of Dirac type operators on a three dimensional compact manifold we obtain a simple a…
Study shows limitations and universality of equivariant QNNs with Sn-equivariant gates.
problem Understanding the expressiveness of Sn-equivariant QNNs with k-body gates. method Investigated the interplay between symmetry and k-bodyness in Sn-equivariant QNN generators. result QNNs are semi-universal but not universal with one- and two-body Sn-equivariant gates. Quantum kernels can be efficiently embedded into classical feature spaces.
problem Can all quantum kernels be efficiently embedded into classical feature spaces?
method Invoking computational universality and using techniques like random Fourier features, the authors show that certain classes of quantum kernels can be efficiently embedded.
result For shift-invariant and composition kernels, embedding quantum kernels are universal and efficient.
Derives adjoint polynomials of torus knots in explicit form.
problem Understanding adjoint invariants of torus knots.
method Closed-form double sum expression derivation.
result Explicit double sum form of adjoint polynomials.
Graphs can't learn certain tasks due to depth vs width limitations.
problem Understanding limitations of graph neural networks in learning specific tasks.
method Analyzing expressive power of GNNmp under depth, width, and node attributes.
result GNNmp can lose significant power when depth and width are restricted.
Polynomial neural networks explore thresholds for maximum expressiveness.
problem Understanding the limits of polynomial neural networks' expressiveness.
method Introducing activation degree threshold to measure network expressiveness and proving its existence and upper bounds.
result Polynomial neural networks with equi-width architectures achieve the maximum expressiveness.
NAF combines neural networks with autoregressive models for better density estimation.
problem Improving density estimation and speech synthesis speed.
method Generalizes autoregressive models using neural networks for invertible transformations.
result NAF is a universal approximator for continuous probability distributions and outperforms IAF.
Unified theorem for deep and shallow joint-equivariant machines.
problem Universal approximation of joint-equivariant machines.
method Constructive universal approximation theorem based on ridgelet transform.
result Unified approximation of deep and shallow networks.
DFMs generalize neural networks for topological layer design.
problem Designing neural networks that can handle high-resolution data without parameter dependence.
method Introducing deep function machines (DFMs) that are invariant to input dimensionality.
result DFMs can approximate bounded non-linear operators between function spaces.
Constructs a universal average for Lie algebra elements.
problem Finding a geometric mean of Lie group elements.
method Develops a totally symmetric geometric mean in a free Lie algebra.
result The universal average preserves group actions on a space.
Quantum models can approximate any function if data encoding allows for a rich enough frequency spectrum.
problem Theoretical properties of quantum machine learning models, particularly their expressive power.
method Investigated how data encoding affects the expressive power of parametrized quantum circuits.
result Quantum models can access increasingly rich frequency spectra by repeating data encoding gates, potentially making them universal function approximators.
MPE framework proves universal approximation for quantum data distribution.
problem Challenges in generating quantum data from underlying distributions.
method Many-body Projected Ensemble (MPE) framework for quantum state design.
result MPE can approximate any quantum distribution within 1-Wasserstein distance error.
By now it is well established that the quantum dimensions of descendants of the adjoint representation can be described in a universal form, independent of a particular family of simple Lie algebras. The Rosso-Jones formula then implies a universal description of the adjoint knot polynomials for torus knots, which in p…
The abstract conjectures a formula for virtual elliptic genera of sheaves on surfaces.
problem Computing virtual elliptic genera of sheaves on surfaces.
method Expressed conjectures in terms of Igusa cusp form, quasi-Jacobi forms, and universal functions.
result Conjectures a formula involving virtual cobordism classes and universal functions.
Quantum machine learning models can approximate any continuous function.
problem Theoretical understanding of quantum feature maps in machine learning.
method Proving universal approximation property of quantum machine learning models in quantum-enhanced feature spaces.
result Quantum machine learning models are universal approximators of continuous functions.
The paper identifies universal features for high-dimensional data inference.
problem Identifying universal low-dimensional features from high-dimensional data for inference tasks.
method Introduces natural notions of universality and shows a local equivalence among them, using information geometry.
result Reveals the complementary roles of various data analysis techniques.
Transformers can approximate any sequence-to-sequence function, surprising given their complexity.
problem Understanding the expressive power of Transformer models for sequence-to-sequence functions.
method Established that Transformers are universal approximators of continuous permutation equivariant sequence-to-sequence functions with compact support, and extended this to arbitrary functions using positional encodings.
result Transformers are universal approximators of arbitrary continuous sequence-to-sequence functions on a compact domain.
PIKS combines physics principles with kernel methods for universal consistency.
problem Lack of learning theory for physics-informed machine learning.
method Physics-Informed Kernel methodS (PIKS) for linear differential constraints.
result PIKS achieves universal consistency for linear differential constraints with Gaussian or Matérn kernels.
Identity parameterization simplifies deep learning models and improves performance.
problem Designing deep neural networks with stable and expressive architectures.
method Theoretical analysis and empirical validation of residual networks with identity parameterization.
result Residual networks with identity parameterization have no spurious local optima and universal expressivity.
New approach to linear regression using universal learning.
problem Applying universal learning to linear regression.
method Using a Gaussian error hypothesis class and Predictive Normalized Maximum Likelihood (pNML) solution.
result Linear regression can generalize even with over-parametrized models under certain conditions.
A coloring scheme improves graph neural networks for node disambiguation.
problem Improving graph neural networks' ability to distinguish identical node attributes.
method Introducing a graph neural network called Colored Local Iterative Procedure (CLIP) that uses colors to disambiguate node attributes.
result CLIP is a universal approximator of continuous functions on graphs with node attributes.
NP-iMCMC algorithm for nonparametric models in universal PPLs.
problem Developing inference algorithms for arbitrary nonparametric models in universal PPLs.
method Unifying involutive MCMC framework with a general procedure for state movement.
result Proves the correctness of the NP-iMCMC sampler and shows significant performance improvements.
Field theory explains optimal scaling in ResNets for signal propagation.
problem Understanding optimal scaling parameter for ResNet performance.
method Finite-size field theory for ResNets to study signal propagation and scaling.
result Analytical expressions for optimal scaling parameter, independent of other hyperparameters.
The study proves Gaussian universality of deep random features learning.
problem Understanding the test error in deep random features learning.
method Proving Gaussian universality of test error in ridge regression and arbitrary convex losses.
result Sharp asymptotic formula for test error in ridge regression setting.
Paper proposes neural networks for learning functions from sets to graphs.
problem Challenges in learning Set2Graph functions, including computational and memory complexity.
method Develops a family of neural network models that are practical and of maximal expressive power, approximating arbitrary continuous Set2Graph functions.
result Models can approximate arbitrary continuous Set2Graph functions over compact sets.
Deep random feature models are analyzed for their performance with exact asymptotic expressions.
problem Understanding the performance of deep random feature models.
method Established a novel universality result and used the convex Gaussian Min-Max theorem.
result Exact asymptotic expressions for the performance of deep random feature models are derived.
Study shows k-NN classifier is not universally consistent on (0,1) but consistent on discrete and specific measure spaces.
problem Consistency of k-NN classifier under Wasserstein distance on measure spaces. method Analysis of k-NN classifier properties under Wasserstein distance, use of σ-finite metric dimension, geodesic structures of Wasserstein spaces. result Consistency of k-NN classifier on specific measure spaces (discrete, Gaussian, wavelet series) but not on (0,1). New neural architectures invariant to sign flips and basis symmetries for graph representation learning.
problem Learning invariant graph representations from eigenvectors.
method SignNet and BasisNet neural architectures that are invariant to sign flips and basis symmetries.
result Proven to be universal, approximating any continuous function of eigenvectors with desired invariances.
This note aims to demonstrate that every parabolic geometry has a naturally defined per-Courant algebroïd structure. This structure is a Courant algebroïd if and only if the the curvature κ of the Cartan connection vanishes. In all other cases, if the parabolic geometry is regular, there does not exist a natural univ…
Paper proves CFlows can approximate any diffeomorphism and applies it in Bayesian optimization.
problem Proving the universality of CFlows in approximating diffeomorphisms.
method Deriving the universality of Para-CFlows through affine coupling layers and invertible linear transforms.
result Para-CFlows can approximate any diffeomorphism in C^k-norm.
Deep neural networks can approximate natural functions with fewer neurons than shallower networks.
problem Understanding the expressibility of neural networks in approximating natural functions.
method Analyzing the neuron requirements for approximating natural multivariate polynomials with deep and shallow networks.
result The neuron requirement for deep networks grows linearly with the number of variables, while shallow networks require exponentially more neurons.
The universal perturbative invariants of rational homology spheres can be extracted from the Chern-Simons partition function by combining perturbative and nonperturbative results. We spell out the general procedure to compute these invariants, and we work out in detail the case of Seifert spaces. By extending some prev…