Single-head attention approximates any function under various norms.
problem Universal approximation of functions using attention mechanisms.
method Interpreting attention as partitioning and summing linear transformations.
result Single-head attention can approximate any continuous function under L∞-norm and Lebesgue integrable functions under Lp-norm. A universal LSTM model outperforms asset-specific models in forecasting stock volatilities.
problem Forecasting stock volatilities across different assets.
method Trained an LSTM network on a pooled dataset of liquid stocks to forecast daily realized volatilities.
result The LSTM model consistently outperforms other asset-specific parametric models in volatility forecasting.
New model suggests universe emerges from single particle quantum mechanics.
problem Exploring how the universe might arise from a single particle in quantum mechanics.
method Novel spontaneous symmetry breaking acting on probability distributions of Hamiltonians.
result Evidence supports the hypothesis that nature seeks tensor decompositions.
Universal model for soft tissue mechanics under shock waves.
problem Modeling shock wave mechanics in soft biological tissues.
method Continuum mixture theory with phase-field mechanics.
result Universal thermodynamically consistent formulation for soft porous tissues.
This text explains how fiber bundle structure is fundamental for classical physics.
problem None explicitly stated, but implied as understanding fiber bundles is crucial for physics.
method Explains the fiber bundle structure and its universality for physics laws.
result Fiber bundle structure is fundamental for classical physics laws.
We prove realizability theorems for vector-valued polynomial mappings, real-algebraic sets and compact smooth manifolds by moduli spaces of planar linkages. We also establish a relation between universality theorems for moduli spaces of mechanical linkages and projective arrangements.
This study approximates neural network features for modeling relations and attention mechanisms.
problem Approximating neural network features for modeling relations and attention mechanisms.
method Analyzes inner products of multi-layer perceptrons for universal approximation of symmetric and asymmetric relation functions.
result Universal approximation of relation functions and attention mechanisms using inner products of neural networks.
Quantum systems learn like machine learning models, influenced by dissipation.
problem Understanding how quantum systems learn and evolve.
method Hydrodynamical formulation of quantum mechanics, gradient descent model, empirical demonstration.
result Quantum systems follow a disrupted gradient descent model, influenced by dissipation.
New findings on universal learning in contextual bandits with adversarial rewards.
problem Learning in contextual bandits with time-varying, adversarial rewards.
method Characterization of learnable processes and necessary/sufficient conditions for universal learning.
result Optimistic universal learning for contextual bandits with adversarial rewards is impossible in general.
A new Dirac algebroid approach for nonholonomic systems.
problem Nonholonomic constraints in mechanical systems.
method Developed a Dirac algebroid to generate phase equations for systems with linear nonholonomic constraints.
result Unified approach to describe systems with different potentials.
Study symmetry breaking in quantum mechanics to understand many-body physics.
problem Understanding many-body physics from quantum mechanics.
method Analyzing potentials with unstable critical points and local minima.
result Emergence of many-body physics from spontaneous symmetry breaking.
A mechanical linkage is a mechanism made of rigid rods linked together by flexible joints, in which some vertices are fixed and others may move. The partial configuration space of a linkage is the set of all the possible positions of a subset of the vertices. We characterize the possible partial configuration spaces of…
We present an axiomatic modification of quaternionic quantum mechanics with a possible-worlds semantics capable of predicting essential "nonquantum" features of an observable universe model - the dimensionality and topology of spacetime, the existence, the signature and a specific form of a metric on it, and certain na…
Softmax attention approximates complex functions and subsumes many known universal approximators.
problem Universal approximation of continuous sequence-to-sequence functions.
method Interpolation-based analysis of attention's internal mechanism, showing its ability to approximate ReLU functions.
result Softmax attention is a universal approximator for continuous sequence-to-sequence functions.
Deep neural networks near edge of chaos show universal scaling laws.
problem Understanding the behavior of deep neural networks near critical points.
method Analogy to absorbing phase transitions in statistical mechanics, deterministic propagation dynamics, mean-field and directed percolation universality classes.
result Deep neural networks exhibit universal scaling laws near the edge of chaos.
Unified framework connects physical laws and machine learning.
problem Combining physical laws and machine learning for scientific applications.
method Universal Differential Equations (UDEs) as a unifying framework.
result Wide variety of applications can be efficiently handled through UDE formalism.
Simple construction for universal quantum gates.
problem Designing efficient quantum gates for topological computers.
method Demonstrated a simple construction for unitary solutions of the braided Yang-Baxter equation in any dimension.
result Proved the existence of universal quantum gates in any dimension.
Paper develops a privacy-preserving nonparametric regression method.
problem Nonparametric regression with local differential privacy constraints.
method Privatised discretisation and Laplace noise applied to feature vectors and responses.
result Strongly universally consistent estimator for regression and classification.
The Galilean group is the group of symmetries of Newtonian mechanics, with Lie a lgebra $\gal(n)$. We find algebraically independent generators for the center of the universal enveloping algebra of $\gal(n)$ using coadjoint orbits.
This work establishes universality for deep equivariant networks, overcoming limitations of previous approaches.
problem Rarity of universality results for equivariant neural networks, especially in high-dimensional settings.
method Develops a more general account of universality for equivariant networks, introducing entry-wise separability and readout layers.
result Deep equivariant networks achieve universality under entry-wise separability, with or without readout layers.
A mechanical linkage is a mechanism made of rigid rods linked together by flexible joints, in which some vertices are fixed and others may move. The partial configuration space of a linkage is the set of all the possible positions of a subset of the vertices. We characterize the possible partial configuration spaces of…
These notes were inspired by the course ''Quantum Field Theory from a Functional Integral Point of View'' given at the University of Zurich in Spring 2017 by Santosh Kandel. We describe Feynman's path integral approach to quantum mechanics and quantum field theory from a functional integral point of view, where the mai…
Paper interprets ResNets via gate-network controls and deep-layer classifications.
problem Understanding the performance mechanism of ResNets.
method Constructs typical solutions using gate-network controls and deep-layer classifications.
result Proves the universal-approximation capability of ResNets.
In this paper we develop a Hamilton-Jacobi theory in the setting of almost Poisson manifolds. The theory extends the classical Hamilton-Jacobi theory and can be also applied to very general situations including nonholonomic mechanical systems and time dependent systems with external forces.
Geometric mechanics approach to constrained and floating multibody systems using Hamel's equations.
problem Analytical mechanics of constrained and floating multibody systems.
method Geometric approach using bundle structures and connections, with Hamel's equations as a universal non-holonomic formulation.
result Achieved intrinsic splitting and inertial decoupling of reduced Euler-Lagrange equations.
This article is written for the online newspaper "The Photon" published by the Department of Physics, University of Maryland. The article describes econophysics research done in the group of Victor Yakovenko. It briefly surveys the subjects "Statistical Mechanics of Money, Income, and Wealth" and "Probability Distribut…
The abstract explores a new wave equation linking quantum mechanics and complex adaptive systems.
problem Understanding the underlying mechanism of distribution formation in complex quantum entanglement.
method Exploring the logical relationship between Schrödinger's wave equation and Shi's trading volume-price wave equation in finance.
result A non-localized wave equation in quantum mechanics reveals the invariance of interaction as a universal law.
Novel compression method preserves privacy while reducing communication costs.
problem Reducing communication costs in differential privacy mechanisms.
method Poisson private representation (PPR) for compressing and simulating local randomizers.
result Achieves compression within a logarithmic gap from theoretical lower bound.
Inspired by the adaptation phenomenon of neuronal firing, we propose the regularity normalization (RN) as an unsupervised attention mechanism (UAM) which computes the statistical regularity in the implicit space of neural networks under the Minimum Description Length (MDL) principle. Treating the neural network optimiz…
New algorithm achieves consistent learning from context in bandit problems.
problem Learning from context in bandit problems with non-i.i.d. contexts.
method Optimistically universal learning rule balancing generalization and personalization.
result Achieves universal consistency for large classes of non-i.i.d. contexts.
Model approximates continuous functions in 1-Wasserstein space.
problem Approximating continuous functions in 1-Wasserstein space.
method Probabilistic Transformer (PT) model with three phases: feature map, deep neural network, and probabilistic extension of attention mechanism.
result Can approximate any continuous function from R^d to P1(R^D) uniformly on compact sets.
This paper describes a language representation model which combines the Bidirectional Encoder Representations from Transformers (BERT) learning mechanism described in Devlin et al. (2018) with a generalization of the Universal Transformer model described in Dehghani et al. (2018). We further improve this model by addin…
Single-layer Transformer can approximate any sequence mapping.
problem Lack of theoretical understanding of Transformers.
method Review of linear algebra, probability, and optimization concepts; detailed analysis of Transformer architecture.
result A single-layer Transformer can approximate any continuous sequence-to-sequence mapping to arbitrary precision.
A plethora of natural, artificial and social systems exist which do not belong to the Boltzmann-Gibbs (BG) statistical-mechanical world, based on the standard additive entropy SBG and its associated exponential BG factor. Frequent behaviors in such complex systems have been shown to be closely related to q-stati…
It is now well established empirically that financial price changes are distributed according to a power law, with cubic exponent. This is a fascinating regularity, as it holds for various classes of securities, on various markets, and on various time scales. The universality of this law suggests that there must be som…
Transformers use ReLUs to approximate softmax efficiently.
problem Analyzing resource usage in softmax transformer models.
method Translating ReLU approximation results to softmax attention mechanisms.
result Economic resource bounds for softmax attention mechanisms.
New framework predicts crypto volatility, outperforming traditional models.
problem Forecasting volatility in cryptocurrencies during the crypto-winter.
method Combines LSTM and rough volatility models, using a parsimonious parametric model.
result Similar prediction performances with fewer parameters, suggesting universality of volatility mechanisms.
Generative diffusion models exhibit phase transitions in statistical mechanics, impacting their performance.
problem Understanding the performance and capabilities of generative diffusion models.
method Reformulating generative diffusion models using statistical mechanics, focusing on phase transitions and symmetry breaking.
result Generative diffusion models undergo second-order phase transitions with mean-field universality, critical instability, and mean-field critical exponents.
Study reveals inefficiencies in EU carbon trading market.
problem Inefficiencies in carbon trading market undermine emission reduction goals.
method Analysis of granular transaction data from 2005-2020.
result 40% of firms never trade in a given year, and many trade only during high-price months.
A new method for private query release using Johnson-Lindenstrauss projection.
problem Private release of query answers with minimal privacy loss.
method Random projection of query answers to a lower dimension, followed by noise addition.
result Optimal worst-case sample complexity for answering a workload of k queries.
Paper models Bitcoin market dynamics using 1+1D field theory.
problem Understanding stylized facts in Bitcoin markets.
method Collects order-book datasets, applies KPZ-like stochastic equations.
result Predicts order book dynamics with high precision.
We argue that an important contributing factor into market inefficiency is the lack of a robust mechanism for the stock price to rise if a company has good earnings, e.g., via buybacks/dividends. Instead, the stock price is prone to volatility due to rather random perception/interpretation of earnings announcements (am…
Using a large-scale Deep Learning approach applied to a high-frequency database containing billions of electronic market quotes and transactions for US equities, we uncover nonparametric evidence for the existence of a universal and stationary price formation mechanism relating the dynamics of supply and demand for a s…
Many independent studies on stocks and futures contracts have established that market impact is proportional to the square-root of the executed volume. Is market impact quantitatively similar for option markets as well? In order to answer this question, we have analyzed the impact of a large proprietary data set of opt…
Dropout schedules can be optimized to significantly reduce model test loss.
problem Improving model performance in neural networks.
method Developed a mean-field theory of dropout at the edge of chaos, proposing front-loaded dropout schedules.
result Front-loaded dropout schedules reduce test loss by 18-35% over constant dropout.
The paper presents instructive interdisciplinary applications of constrained mechanics calculus in economics on a level appropriate for the undergraduate physics education. The aim of the paper is: 1. to meet the demand for illustrative examples suitable for presenting the background of the highly expanding research fi…
Quickly adapts fault diagnosis models for industrial machines.
problem Fault diagnostic models trained for lab machines fail on industrial ones.
method Net2Net transformation followed by fine-tuning.
result Models can be quickly adapted for new operating conditions.
Study shows a universal local obstruction to the Samuelson condition for tangent Lagrangian 2-webs.
problem Obstruction to the Samuelson condition for tangent Lagrangian 2-webs.
method Local analysis of tangent lines and their intersection maps.
result A universal local phenomenon produces a nonzero mixed derivative, obstructing the Samuelson condition.