Single-head attention approximates any function under various norms.
problem Universal approximation of functions using attention mechanisms.
method Interpreting attention as partitioning and summing linear transformations.
result Single-head attention can approximate any continuous function under L∞-norm and Lebesgue integrable functions under Lp-norm. A universal LSTM model outperforms asset-specific models in forecasting stock volatilities.
problem Forecasting stock volatilities across different assets.
method Trained an LSTM network on a pooled dataset of liquid stocks to forecast daily realized volatilities.
result The LSTM model consistently outperforms other asset-specific parametric models in volatility forecasting.
New model suggests universe emerges from single particle quantum mechanics.
problem Exploring how the universe might arise from a single particle in quantum mechanics.
method Novel spontaneous symmetry breaking acting on probability distributions of Hamiltonians.
result Evidence supports the hypothesis that nature seeks tensor decompositions.
Universal model for soft tissue mechanics under shock waves.
problem Modeling shock wave mechanics in soft biological tissues.
method Continuum mixture theory with phase-field mechanics.
result Universal thermodynamically consistent formulation for soft porous tissues.
This text explains how fiber bundle structure is fundamental for classical physics.
problem None explicitly stated, but implied as understanding fiber bundles is crucial for physics.
method Explains the fiber bundle structure and its universality for physics laws.
result Fiber bundle structure is fundamental for classical physics laws.
We prove realizability theorems for vector-valued polynomial mappings, real-algebraic sets and compact smooth manifolds by moduli spaces of planar linkages. We also establish a relation between universality theorems for moduli spaces of mechanical linkages and projective arrangements.
This study approximates neural network features for modeling relations and attention mechanisms.
problem Approximating neural network features for modeling relations and attention mechanisms.
method Analyzes inner products of multi-layer perceptrons for universal approximation of symmetric and asymmetric relation functions.
result Universal approximation of relation functions and attention mechanisms using inner products of neural networks.
Deep Learning reveals universal price formation mechanism in financial markets.
problem Understanding price dynamics in financial markets.
method High-frequency electronic market quotes and transactions for US equities, using Deep Learning.
result The model exhibits stable out-of-sample prediction accuracy across different stocks and time periods.
New findings on universal learning in contextual bandits with adversarial rewards.
problem Learning in contextual bandits with time-varying, adversarial rewards.
method Characterization of learnable processes and necessary/sufficient conditions for universal learning.
result Optimistic universal learning for contextual bandits with adversarial rewards is impossible in general.
Quantum systems learn like machine learning models, influenced by dissipation.
problem Understanding how quantum systems learn and evolve.
method Hydrodynamical formulation of quantum mechanics, gradient descent model, empirical demonstration.
result Quantum systems follow a disrupted gradient descent model, influenced by dissipation.
A new Dirac algebroid approach for nonholonomic systems.
problem Nonholonomic constraints in mechanical systems.
method Developed a Dirac algebroid to generate phase equations for systems with linear nonholonomic constraints.
result Unified approach to describe systems with different potentials.
Study symmetry breaking in quantum mechanics to understand many-body physics.
problem Understanding many-body physics from quantum mechanics.
method Analyzing potentials with unstable critical points and local minima.
result Emergence of many-body physics from spontaneous symmetry breaking.
A mechanical linkage is a mechanism made of rigid rods linked together by flexible joints, in which some vertices are fixed and others may move. The partial configuration space of a linkage is the set of all the possible positions of a subset of the vertices. We characterize the possible partial configuration spaces of…
We present an axiomatic modification of quaternionic quantum mechanics with a possible-worlds semantics capable of predicting essential "nonquantum" features of an observable universe model - the dimensionality and topology of spacetime, the existence, the signature and a specific form of a metric on it, and certain na…
Softmax attention approximates complex functions and subsumes many known universal approximators.
problem Universal approximation of continuous sequence-to-sequence functions.
method Interpolation-based analysis of attention's internal mechanism, showing its ability to approximate ReLU functions.
result Softmax attention is a universal approximator for continuous sequence-to-sequence functions.
Deep neural networks near edge of chaos show universal scaling laws.
problem Understanding the behavior of deep neural networks near critical points.
method Analogy to absorbing phase transitions in statistical mechanics, deterministic propagation dynamics, mean-field and directed percolation universality classes.
result Deep neural networks exhibit universal scaling laws near the edge of chaos.
Unified framework connects physical laws and machine learning.
problem Combining physical laws and machine learning for scientific applications.
method Universal Differential Equations (UDEs) as a unifying framework.
result Wide variety of applications can be efficiently handled through UDE formalism.
Simple construction for universal quantum gates.
problem Designing efficient quantum gates for topological computers.
method Demonstrated a simple construction for unitary solutions of the braided Yang-Baxter equation in any dimension.
result Proved the existence of universal quantum gates in any dimension.
Paper develops a privacy-preserving nonparametric regression method.
problem Nonparametric regression with local differential privacy constraints.
method Privatised discretisation and Laplace noise applied to feature vectors and responses.
result Strongly universally consistent estimator for regression and classification.
This work establishes universality for deep equivariant networks, overcoming limitations of previous approaches.
problem Rarity of universality results for equivariant neural networks, especially in high-dimensional settings.
method Develops a more general account of universality for equivariant networks, introducing entry-wise separability and readout layers.
result Deep equivariant networks achieve universality under entry-wise separability, with or without readout layers.
A mechanical linkage is a mechanism made of rigid rods linked together by flexible joints, in which some vertices are fixed and others may move. The partial configuration space of a linkage is the set of all the possible positions of a subset of the vertices. We characterize the possible partial configuration spaces of…
The Galilean group is the group of symmetries of Newtonian mechanics, with Lie a lgebra $\gal(n)$. We find algebraically independent generators for the center of the universal enveloping algebra of $\gal(n)$ using coadjoint orbits.
Functional integrals explain quantum mechanics and field theory.
problem Explaining quantum mechanics and field theory using functional integrals.
method Describes Feynman's path integral approach to quantum mechanics and field theory.
result Equivalence of path integral formalism to classical mechanics and quantum mechanics.
Proposes RN for unsupervised attention in neural networks.
problem Limited, imbalanced, and non-stationary input distributions in various tasks.
method Inspired by neuronal adaptation, RN uses MDL principle and universal code length for incremental layer-wise computation.
result Outperforms existing normalization methods across diverse tasks.
Paper interprets ResNets via gate-network controls and deep-layer classifications.
problem Understanding the performance mechanism of ResNets.
method Constructs typical solutions using gate-network controls and deep-layer classifications.
result Proves the universal-approximation capability of ResNets.
In this paper we develop a Hamilton-Jacobi theory in the setting of almost Poisson manifolds. The theory extends the classical Hamilton-Jacobi theory and can be also applied to very general situations including nonholonomic mechanical systems and time dependent systems with external forces.
Geometric mechanics approach to constrained and floating multibody systems using Hamel's equations.
problem Analytical mechanics of constrained and floating multibody systems.
method Geometric approach using bundle structures and connections, with Hamel's equations as a universal non-holonomic formulation.
result Achieved intrinsic splitting and inertial decoupling of reduced Euler-Lagrange equations.
This article is written for the online newspaper "The Photon" published by the Department of Physics, University of Maryland. The article describes econophysics research done in the group of Victor Yakovenko. It briefly surveys the subjects "Statistical Mechanics of Money, Income, and Wealth" and "Probability Distribut…
The abstract explores a new wave equation linking quantum mechanics and complex adaptive systems.
problem Understanding the underlying mechanism of distribution formation in complex quantum entanglement.
method Exploring the logical relationship between Schrödinger's wave equation and Shi's trading volume-price wave equation in finance.
result A non-localized wave equation in quantum mechanics reveals the invariance of interaction as a universal law.
Improved BERT model with latent persona and topic variables.
problem Improving BERT's domain-specific utility while maintaining generalization.
method Combining BERT with Universal Transformer, adding latent persona and topic variables.
result Pre-trained model for social texts outperforms baseline.
Novel compression method preserves privacy while reducing communication costs.
problem Reducing communication costs in differential privacy mechanisms.
method Poisson private representation (PPR) for compressing and simulating local randomizers.
result Achieves compression within a logarithmic gap from theoretical lower bound.
New algorithm achieves consistent learning from context in bandit problems.
problem Learning from context in bandit problems with non-i.i.d. contexts.
method Optimistically universal learning rule balancing generalization and personalization.
result Achieves universal consistency for large classes of non-i.i.d. contexts.
Gated Linear Networks bypass feature learning for fast online learning.
problem Fast online learning and feature learning trade-offs in neural networks.
method Distributed and local credit assignment mechanism, data-dependent gating, online convex optimization.
result GLNs achieve universal learning capabilities and resilience to catastrophic forgetting.
Model approximates continuous functions in 1-Wasserstein space.
problem Approximating continuous functions in 1-Wasserstein space.
method Probabilistic Transformer (PT) model with three phases: feature map, deep neural network, and probabilistic extension of attention mechanism.
result Can approximate any continuous function from R^d to P1(R^D) uniformly on compact sets.
Single-layer Transformer can approximate any sequence mapping.
problem Lack of theoretical understanding of Transformers.
method Review of linear algebra, probability, and optimization concepts; detailed analysis of Transformer architecture.
result A single-layer Transformer can approximate any continuous sequence-to-sequence mapping to arbitrary precision.
A plethora of natural, artificial and social systems exist which do not belong to the Boltzmann-Gibbs (BG) statistical-mechanical world, based on the standard additive entropy SBG and its associated exponential BG factor. Frequent behaviors in such complex systems have been shown to be closely related to q-stati…
It is now well established empirically that financial price changes are distributed according to a power law, with cubic exponent. This is a fascinating regularity, as it holds for various classes of securities, on various markets, and on various time scales. The universality of this law suggests that there must be som…
Transformers use ReLUs to approximate softmax efficiently.
problem Analyzing resource usage in softmax transformer models.
method Translating ReLU approximation results to softmax attention mechanisms.
result Economic resource bounds for softmax attention mechanisms.
Generative diffusion models exhibit phase transitions in statistical mechanics, impacting their performance.
problem Understanding the performance and capabilities of generative diffusion models.
method Reformulating generative diffusion models using statistical mechanics, focusing on phase transitions and symmetry breaking.
result Generative diffusion models undergo second-order phase transitions with mean-field universality, critical instability, and mean-field critical exponents.
New framework predicts crypto volatility, outperforming traditional models.
problem Forecasting volatility in cryptocurrencies during the crypto-winter.
method Combines LSTM and rough volatility models, using a parsimonious parametric model.
result Similar prediction performances with fewer parameters, suggesting universality of volatility mechanisms.
Study reveals inefficiencies in EU carbon trading market.
problem Inefficiencies in carbon trading market undermine emission reduction goals.
method Analysis of granular transaction data from 2005-2020.
result 40% of firms never trade in a given year, and many trade only during high-price months.
A new method for private query release using Johnson-Lindenstrauss projection.
problem Private release of query answers with minimal privacy loss.
method Random projection of query answers to a lower dimension, followed by noise addition.
result Optimal worst-case sample complexity for answering a workload of k queries.
Paper models Bitcoin market dynamics using 1+1D field theory.
problem Understanding stylized facts in Bitcoin markets.
method Collects order-book datasets, applies KPZ-like stochastic equations.
result Predicts order book dynamics with high precision.
We argue that an important contributing factor into market inefficiency is the lack of a robust mechanism for the stock price to rise if a company has good earnings, e.g., via buybacks/dividends. Instead, the stock price is prone to volatility due to rather random perception/interpretation of earnings announcements (am…
Many independent studies on stocks and futures contracts have established that market impact is proportional to the square-root of the executed volume. Is market impact quantitatively similar for option markets as well? In order to answer this question, we have analyzed the impact of a large proprietary data set of opt…
Improved inference in probabilistic programs using attention mechanisms.
problem Inference failure in existing IC network architectures due to long-range dependency issues.
method Inference compilation with attention mechanism to model latent variables.
result Attention mechanism enhances proposal distributions to better match true posterior.
Deep User Perception Network learns universal user representations from multiple e-commerce tasks.
problem Lack of shared user information across diverse e-commerce tasks.
method Model user behavior sequences using LSTM and attention mechanism, sharing user representations across multiple tasks.
result Our approach consistently achieves better results in personalization across multiple e-commerce tasks.
Dropout schedules can be optimized to significantly reduce model test loss.
problem Improving model performance in neural networks.
method Developed a mean-field theory of dropout at the edge of chaos, proposing front-loaded dropout schedules.
result Front-loaded dropout schedules reduce test loss by 18-35% over constant dropout.