Develops hierarchical reinforcement learning value function approximators.
problem Estimating long-term returns in reinforcement learning with multiple goals.
method Introduces hierarchical universal value function approximators (H-UVFAs) using the options framework.
result Demonstrates generalization and improved performance of H-UVFAs over UVFAs.
Complex-valued neural networks can approximate any continuous function.
problem Generalizing the universal approximation theorem to complex-valued networks.
method Characterizing activation functions for complex networks to approximate any continuous function.
result Different activation functions are required for deep vs shallow complex networks to achieve universal approximation.
Complex-valued neural networks can approximate any continuous function with bounded widths and depths.
problem Approximating continuous functions with complex-valued neural networks of bounded widths and depths.
method Analyzing activation functions and proving universality for complex-valued networks.
result Deep narrow complex-valued networks are universal if and only if their activation function is neither holomorphic, nor antiholomorphic, nor R-affine. UVU simplifies value uncertainty quantification in RL.
problem Estimating epistemic uncertainty in value functions for reinforcement learning.
method UVU uses squared prediction errors between an online learner and a fixed, randomly initialized target network, incorporating policy-conditional value uncertainty.
result UVU achieves equal performance to large ensembles on challenging offline RL settings, with computational savings.
Deep RBVFs improve continuous control in RL.
problem Challenges in finding optimal actions for continuous actions in RL.
method Introduced deep radial-basis value functions (RBVFs) for continuous control.
result RBF-DQN significantly outperforms value-function-only baselines and is competitive with actor-critic algorithms.
New model improves option pricing with faster convergence and better generalization.
problem Improving classical option pricing models.
method Introducing a time value related decision function and proving a universal approximation theorem.
result The new decision function approximates on the entire domain of definition by neural networks.
Universal approximation for ODENet and ResNet with a single activation function.
problem Approximating complex dynamical systems with limited vector fields.
method Examined ODENet and ResNet with vector fields composed of a single activation function and affine mapping.
result ODENet and ResNet with restricted vector fields can uniformly approximate those with general vector fields.
MLPs can approximate any function in context, challenging the importance of in-context universality.
problem Understanding why transformers are more effective than classical models.
method Proved MLPs with trainable activation functions are universal in context.
result Transformer success is likely due to factors other than in-context universality.
Method maps state space using landmarks for universal goal reaching.
problem Learning the Universal Value Function Approximator (UVFA) for long-range goals is challenging.
method Hierarchical modeling with a dynamic landmark-based map and a value network.
result The method enables agents to reach long-range goals at the early training stage.
USFs capture dynamics for faster RL task transfer.
problem Applying knowledge from one task to another.
method Proposed Universal Successor Features (USFs) for RL.
result USFs accelerate training and transfer knowledge.
The objective of transfer reinforcement learning is to generalize from a set of previous tasks to unseen new tasks. In this work, we focus on the transfer scenario where the dynamics among tasks are the same, but their goals differ. Although general value function (Sutton et al., 2011) has been shown to be useful for k…
Paper introduces a new kernel model for PSD-valued functions with theoretical guarantees and applications.
problem Enforcing positive semi-definiteness (PSD) in function models with good performance and theoretical guarantees.
method Kernel sum-of-squares model for PSD-valued functions, extending previous models for non-negative scalar functions.
result The model constitutes a universal approximator of PSD functions and can represent any smooth and strongly convex function.
Universal approximation theorem for differentiable maps on infinite-dimensional manifolds
problem Approximation of differentiable maps on infinite-dimensional manifolds
method Weighted universal approximation theorem
result Universal approximation theorem for differentiable maps
Unified framework proves neural networks' ability to mimic complex tasks.
problem Lack of a single constructive framework for neural network universality.
method Introduces neural network approximate identity (nAI) and proves it leads to universality.
result Any nAI activation function is universal.
Generalizes neural network approximation to infinite-dimensional manifolds and derivatives.
problem Approximating differentiable maps on infinite-dimensional manifolds.
method Proves a weighted Nachbin theorem to establish universal approximation for differentiable maps, including derivatives.
result Linear functions of the signature can approximate path space functionals including their derivatives.
Single-head attention approximates any function under various norms.
problem Universal approximation of functions using attention mechanisms.
method Interpreting attention as partitioning and summing linear transformations.
result Single-head attention can approximate any continuous function under L∞-norm and Lebesgue integrable functions under Lp-norm. The study extends kernel universality to Riemannian symmetric spaces.
problem Understanding kernel universality in non-Euclidean domains.
method Harmonic analysis on Riemannian symmetric spaces.
result Proves universality of recent kernels on Riemannian symmetric spaces.
Sharp lower bound on GHHs' representation power of CPWL functions.
problem Proving the minimum number of nestings for GHHs to represent arbitrary CPWL functions.
method Using a key lemma about finite sums of periodic functions, proving necessity of n nestings.
result Proving necessity of n nestings for GHHs to achieve universal representation power.
Random feature models approximate functions in Banach spaces efficiently.
problem Approximating functions in Banach spaces efficiently.
method Randomly initialized feature maps and linear readout training.
result Universal approximation in Bochner spaces for Banach space-valued models.
Invariants for singular knots and links using non-commutative cocycles.
problem Creating invariants for singular knots and links.
method Defining universal group and functions for non-commutative cocycles, and computing examples.
result Invariants for singular knots and links defined using non-commutative cocycles.
Dense neural networks can't approximate all functions.
problem Approximation capabilities of dense neural networks.
method Model compression approach combining weak regularity lemma and graph neural networks.
result Existence of Lipschitz continuous functions not approximable by dense neural networks.
Reduces bounded loss learning to binary classification.
problem Universal consistency of non-i.i.d. processes with bounded loss.
method Constructive reduction to binary classification.
result Any bounded loss output setting can be reduced to binary classification.
Functional input neural networks approximate continuous functions on weighted spaces.
problem Approximating continuous functions on infinite-dimensional weighted spaces.
method Additive family mapping, non-linear activation, linear readouts, Stone-Weierstrass theorem.
result Global universal approximation of continuous functions on weighted spaces.
Unified framework for comparing classification metrics across different imbalance rates.
problem Differences in scale and sensitivity to class imbalance rates in classification metrics.
method Introduces outperformance standardization (OPS) function to map metrics to a common scale.
result Unified o-value metric provides clear comparison across different imbalance rates.
Paper develops a privacy-preserving nonparametric regression method.
problem Nonparametric regression with local differential privacy constraints.
method Privatised discretisation and Laplace noise applied to feature vectors and responses.
result Strongly universally consistent estimator for regression and classification.
Novel approach to universal online learning for bounded losses, closing open problems.
problem Characterizing processes for universal online learning under non-i.i.d. conditions.
method Characterization of processes admitting strong and weak universal learning, introduction of optimistically universal learning rule.
result Introduction of a novel 1NN algorithm that is optimistically universal for bounded losses.
Given an L2-acyclic connected finite CW-complex, we define its universal L2-torsion in terms of the chain complex of its universal covering. It takes values in the weak Whitehead group Whw(G). We study its main properties such as homotopy invariance, sum formula, product formula and Poincaré d…
Deep residual networks can approximate any continuous function using control theory.
problem Universal approximation capabilities of deep residual neural networks.
method Relating residual networks to control systems and using Lie algebraic techniques.
result Deep residual networks with adequately deep layers can approximate any continuous function on a compact set.
Minimum width for ReLU networks on compact domain is exactly max{d_x, d_y, 2}
problem Characterizing the minimum width for ReLU networks to approximate functions on compact domains
method Analyzing the minimum width for Lp approximation of Lp functions from [0,1]d to Rdy using ReLU-like activation functions result The minimum width for Lp approximation on a compact domain is exactly max{d_x, d_y, 2} for ReLU-like activation functions Paper establishes rates of universal approximation for neural tangent kernels using transport mappings.
problem Universal approximation for neural tangent kernels with microscopic weight changes.
method Generic scheme to approximate functions with NTK using transport mappings, constructed via Fourier transforms.
result Approximation of continuous functions with roughly 1 / δ^(10d) nodes, where δ depends on function continuity.
We demonstrate that in residual neural networks (ResNets) dynamical isometry is achievable irrespectively of the activation function used. We do that by deriving, with the help of Free Probability and Random Matrix Theories, a universal formula for the spectral density of the input-output Jacobian at initialization, in…
A new approach to continuous-time universal portfolios using pathwise Itô calculus.
problem Continuous-time version of Cover's universal portfolio strategies.
method Pathwise Itô calculus approach to establish existence and properties of universal portfolio strategies.
result The universal portfolio strategy's portfolio value process is the average of all values of constant rebalanced strategies.
New method evaluates LLMs fairness in universal prediction.
problem Evaluating fairness of large language models in universal prediction.
method Introducing batch regret as a modification of average regret for LLMs.
result Asymptotical value of batch regret for add-constant predictors on memoryless and first-order Markov sources.
New conditions ensure deep neural networks can approximate any function on non-Euclidean spaces.
problem Understanding how to modify neural network architectures to approximate functions on non-Euclidean spaces.
method Developed conditions for feature and readout maps that preserve universal approximation capabilities.
result Modified architectures can deterministically approximate any classifier on non-Euclidean spaces.
Researchers identify valid auxiliary functions for extreme value distributions and their max-domains of attraction.
problem Characterize valid auxiliary functions for extreme value distributions and their max-domains of attraction.
method Introduced 'universal' auxiliary functions valid for both VR and vMR representations, identified sets of valid auxiliary functions, and proposed a method for finding appropriate auxiliary functions.
result Characterized valid auxiliary functions for both VR and vMR representations for the entire MDA distribution families.
nUDEs use neural networks to model biology without negative values.
problem Unrealistic negative values in hybrid models of biology.
method Developed non-negative UDEs (nUDEs) with regularization techniques.
result nUDEs provide realistic solutions for biological models.
Path signatures adapted for Lie groups improve action recognition in computer vision.
problem Improving action recognition in computer vision with geometric constraints.
method Lifting path signatures to Lie groups and proving universality and characteristic property.
result Path signatures on Lie groups provide comparable performance to shallow learning approaches in action recognition.
We compute the analytic expression of the probability distributions F{AEX,+} and F{AEX,-} of the normalized positive and negative AEX (Netherlands) index daily returns r(t). Furthermore, we define the αre-scaled AEX daily index positive returns r(t)^αand negative returns (-r(t))^αthat we call, after normalization, the …
Stochastic neural networks can approximate any function, even with correlated outputs.
problem Approximating functions with stochastic outputs and correlations.
method Investigating deep sigmoid belief networks to approximate any stochastic mapping.
result Minimal number of layers and units needed for approximation.
Constraining linear layers in neural networks to respect symmetry transformations from a group G is a common design principle for invariant networks that has found many applications in machine learning. In this paper, we consider a fundamental question that has received little attention to date: Can these networks ap…
The common assumption of universal behavior in stock market data can sometimes lead to false conclusions. In statistical physics, the Hurst exponents characterizing long-range correlations are often closely related to universal exponents. We show, that in the case of time series of the traded value, these Hurst exponen…
We compute the analytic expression of the probability distributions F{FTSE100,+} and F{FTSE100,-} of the normalized positive and negative FTSE100 (UK) index daily returns r(t). Furthermore, we define the alpha re-scaled FTSE100 daily index positive returns r(t)^alpha and negative returns (-r(t))^alpha that we call, aft…
The paper proves universality in optimization problems with i.i.d. random vectors.
problem Optimization problems with i.i.d. random vectors and their projections.
method Proves universality of empirical risk minimization under specific conditions.
result The minimum value of the optimization problem is universal and depends only on the mean and covariance of the random vectors.
New findings show neural network training loss follows a power law over time.
problem Understanding the optimization process of neural networks during training.
method Spectral analysis of the integral operator representing the linearized evolution of a large network.
result The loss function in neural network training follows a power law behavior, L(t)∼t−ξ, with exponent ξ determined by network parameters and data characteristics. We present evidence, that if a large enough set of high resolution stock market data is analyzed, certain analogies with physics -- such as scaling and universality -- fail to capture the full complexity of such data. Despite earlier expectations, the mean value per trade, the mean number of trades per minute and the m…
A new KAN variant uses sinusoidal activations to approximate functions.
problem Approximating multivariable functions using neural networks.
method Replacing inner and outer functions in Kolmogorov-Arnold representation with weighted sinusoidal functions.
result The new KAN variant outperforms fixed-frequency Fourier transform and achieves comparable performance to MLPs.
We use a continuous-time random walk (CTRW) to model market fluctuation data from times when traders experience excessive losses or excessive profits. We analytically derive "superstatistics" that accurately model empirical market activity data (supplied by Bogachev, Ludescher, Tsallis, and Bunde)that exhibit transitio…
We prove Runge-type theorems and universality results for locally univalent holomorphic and meromorphic functions. Refining a result of M. Heins, we also show that there is a universal bounded locally univalent function on the unit disk. These results are used to prove that on any hyperbolic simply connected plane doma…