DDEQs extend DEQs to discrete measure inputs using Wasserstein gradient flows.
problem Applying DEQs to discrete measure inputs like sets or point clouds.
method Wasserstein gradient flows for finding fixed points of discrete measures under permutation-invariance.
result DDEQs can compete with state-of-the-art models in tasks like point cloud classification and completion.
Computing equilibrium states in condensed-matter many-body systems, such as solvated proteins, is a long-standing challenge. Lacking methods for generating statistically independent equilibrium samples in "one shot", vast computational effort is invested for simulating these system in small steps, e.g., using Molecular…
RevDEQs improve performance on tasks with exact gradients and fewer function evaluations.
problem Inexact gradient calculation in DEQs leads to unstable training and requires regularisation or many function evaluations.
method Introduce Reversible Deep Equilibrium Models (RevDEQs) that allow for exact gradient calculation, no regularisation, and far fewer function evaluations.
result RevDEQs significantly improve performance on language modelling and image classification tasks.
DEQs converge to optimal solutions with mild over-parameterization.
problem Training over-parameterized deep equilibrium models.
method Solves equilibrium point directly, uses gradient descent, and analyzes convergence via linear rate.
result Gradient descent converges to a globally optimal solution at a linear rate for quadratic loss.
DREAM learns optimal strategies in imperfect games without needing a simulator.
problem Learning optimal strategies in imperfect-information games with multiple agents.
method DREAM is a deep reinforcement learning algorithm that converges to Nash Equilibria and coarse correlated equilibria.
result DREAM achieves state-of-the-art performance in benchmark games and is competitive with simulator-based algorithms.
Deep learning models replicate Kyle's model's market equilibrium.
problem Understanding market equilibria in asymmetric information settings.
method Using deep neural networks to model agents in Kyle's single period model.
result Trained networks' behavior converges to Kyle's predicted equilibrium.
Deep equilibrium models estimate latent variables from data.
problem Estimating latent variables from data.
method Generalized exponential family models, deep equilibrium networks.
result Deep equilibrium models solve MAP estimates for latent and transformation parameters.
Deep equilibrium models converge globally without explicit computation.
problem Global convergence of deep learning models with implicit layers.
method Analysis of gradient dynamics and proof of convergence rate.
result Deep equilibrium models converge to global optimum at a linear rate.
Gradient descent dynamics studied for DEQs in linear and single-index models.
problem Understanding gradient descent dynamics for DEQs.
method Rigorously studied gradient descent dynamics for DEQs in linear and single-index models.
result Gradient descent converges to a global minimizer for linear DEQs and single-index models.
A new method uses deep learning to predict rare events in complex systems.
problem Predicting rare and extreme events in non-equilibrium systems.
method A deep learning approach that minimizes the geometrical action.
result The method accurately predicts rare events in various complex systems.
This paper extends the convergence rate of DEQs with ReLU to any general activation.
problem Proving global convergence rate for DEQs with general activations.
method Developed a novel population Gram matrix and new form of dual activation with Hermite polynomial expansion.
result Gradient descent converges to a globally optimal solution at a linear rate for DEQs with general activations.
This paper introduces Schur-constant equilibrium distribution models of dimension n for arithmetic non-negative random variables. Such a model is defined through the (several orders) equilibrium distributions of a univariate survival function. First, the bivariate case is considered and analyzed in depth, stressing the…
We consider a financial market model which consists of a financial asset and a large number of interacting agents classified into many types. Different types of agents are heterogeneous in their price expectations. Each agent can change its type based on the current empirical distribution of the types and the equilibri…
New model outperforms Neural ODEs while being more efficient.
problem Stable convergence and existence guarantees for implicit-depth models.
method Developed Monotone Operator Equilibrium Network (monDEQ) based on monotone operator theory.
result MonDEQ models outperform Neural ODEs and are more computationally efficient.
New CGMD model predicts non-equilibrium processes better than existing methods.
problem Inconsistency in conditional distribution of unresolved variables.
method Time-lagged independent component analysis to minimize entropy contribution of unresolved variables.
result The model's generalization ability for non-equilibrium processes is significantly improved.
Deep fictitious play converges to Nash equilibrium in stochastic differential games.
problem Finding Nash equilibrium in large stochastic differential games.
method Decouples the game into sub-optimization problems and solves each player's optimal strategy with deep BSDE method.
result Deep fictitious play converges to the true Nash equilibrium.
We present a new approach to modeling sequential data: the deep equilibrium model (DEQ). Motivated by an observation that the hidden layers of many existing deep sequence models converge towards some fixed point, we propose the DEQ approach that directly finds these equilibrium points via root-finding. Such a method is…
New method stabilizes DEQ models by regularizing Jacobian of fixed-point equations.
problem Stability and performance of DEQ models.
method Jacobian regularization to stabilize DEQ models.
result Significant stabilization of fixed-point convergence in DEQ models.
HomoODE connects DEQs and Neural ODEs via homotopy continuation, improving accuracy and memory efficiency.
problem Connecting DEQs and Neural ODEs for better model performance and efficiency.
method Established a connection between DEQs and Neural ODEs using homotopy continuation, proposing HomoODE.
result HomoODE outperforms existing implicit models in accuracy and memory consumption.
Deep learning solves and estimates complex financial models.
problem Estimating and solving continuous-time financial models.
method Uses deep learning to solve and estimate models simultaneously.
result Demonstrates advantages like generality and large state space handling.
Discovery of atomistic systems with desirable properties is a major challenge in chemistry and material science. Here we introduce a novel, autoregressive, convolutional deep neural network architecture that generates molecular equilibrium structures by sequentially placing atoms in three-dimensional space. The model e…
New approaches improve adversarial robustness of DEQs.
problem Adversarial vulnerability of DEQs.
method Developed approaches to estimate intermediate gradients and integrate them into attacking pipelines.
result Demonstrated adversarial robustness of DEQs competitive with deep networks.
Modeling trading behavior with information signals and limit order books, showing market impact and equilibrium properties.
problem Analyzing the impact of information signals on trading behavior and market equilibrium in limit order books.
method Static equilibrium model with profit-maximizing investors and competitive dealers, using iterative algorithms and asymptotic analysis.
result The market impact of large trades follows a power law with fat tails and a logarithmic law with lighter tails, and the order book flattens as noise trading increases.
We present a simple dynamic equilibrium model for an online exchange where both buyers and sellers arrive according to a exogenously defined stochastic process. The structure of this exchange is motivated by the limit order book mechanism used in stock markets. Both buyers and sellers are elastic in the price-quantity …
We find the explicit expression for the equilibrium wealth distribution of the Directed Random Market process, recently introduced by Martínez-Martínez and López-Ruiz, which turns out to be a Gamma distribution with shape parameter 21. We also prove the convergence of the discrete-time process describing the…
Phase segregation, the process by which the components of a binary mixture spontaneously separate, is a key process in the evolution and design of many chemical, mechanical, and biological systems. In this work, we present a data-driven approach for the learning, modeling, and prediction of phase segregation. A direct …
MDEQ models learn multi-resolution features efficiently.
problem Large-scale, hierarchical pattern recognition.
method Implicit differentiation, multiscale deep equilibrium model.
result MDEQs achieve performance on par with recent models.
Various multi-agent models of wealth distributions defined by microscopic laws regulating the trades, with or without a saving criterion, are reviewed. We discuss and clarify the equilibrium properties of the model with constant global saving propensity, resulting in Gamma distributions, and their equivalence to the Ma…
We consider a simple stochastic model of a urban rental housing market, in which the interaction of tenants and landlords induces rent fluctuations. We simulate the model numerically and measure the equilibrium rent distribution, which is found to be close to a lognormal law. We also study the influence of the density …
Wide neural networks converge to Gaussian processes, improving generalization.
problem Understanding the generalization of wide neural networks, especially deep equilibrium models.
method Investigation of deep equilibrium models (DEQs) with infinite-depth layers, focusing on their convergence to Gaussian processes as width and depth approach infinity.
result Wide DEQs converge to Gaussian processes, maintaining generalization performance.
Develops a method for non-equilibrium importance sampling to estimate expectations and constants.
problem Estimating expectations and normalization constants for complex high-dimensional distributions.
method Generates samples from a base distribution, transports them using a velocity field, and averages along flowlines.
result The method can achieve zero-variance estimation and significantly reduces variance compared to vanilla estimators.
Deep RL solves complex economic models with heterogeneous agents.
problem Solving models with heterogeneous economic actors is difficult.
method Reinforcement Learning techniques for solving general equilibrium models.
result Successfully captures economic behaviors induced by age-based health risks.
We construct a theoretical model for equilibrium distribution of workers across sectors with different labor productivity, assuming that a sector can accommodate a limited number of workers which depends only on its productivity. A general formula for such distribution of productivity is obtained, using the detail-bala…
Proposes a deep learning method for solving complex financial games with delays.
problem Financial modeling with multi-agent interactions and delayed effects.
method Parameterizes controls using recurrent neural networks and trains them with modified fictitious play.
result Demonstrates effectiveness on finance problems with known solutions and new problems with derived Nash equilibria.
Economic systems are similar with physic systems for their large number of individuals and the exist of equilibrium. In this paper, we present a model applying the equilibrium statistical model in economic systems. Consistent with statistical physics, we define a series of concepts, such as economic temperature, econom…
In this communication, complex systems with a near trivial dynamics are addressed. First, under the hypothesis of equiprobability in the asymptotic equilibrium, it is shown that the (hyper) planar geometry of an N-dimensional multi-agent economic system implies the exponential (Boltzmann-Gibss) wealth distribution an…
Path-independent equilibrium models improve network performance on harder problems.
problem Improving network performance on harder problem instances.
method Investigated path-independent equilibrium models and their impact on network performance.
result Path independence correlates with better performance on harder problem instances.
A central problem in machine learning involves modeling complex data-sets using highly flexible families of probability distributions in which learning, sampling, inference, and evaluation are still analytically or computationally tractable. Here, we develop an approach that simultaneously achieves both flexibility and…
Study on Kyle's model with stochastic liquidity impacts asset volatility.
problem Impact of stochastic volatility of noise trading on asset volatility.
method Construct equilibrium for continuous-time Kyle's model with stochastic liquidity.
result In equilibrium, Kyle's Lambda and its inverse are submartingales.
We consider a simple model of a closed economic system where the total money is conserved and the number of economic agents is fixed. In analogy to statistical systems in equilibrium, money and the average money per economic agent are equivalent to energy and temperature, respectively. We investigate the effect of the …
DEQs and explicit networks are nearly equivalent for Gaussian mixtures.
problem Understanding the equivalence between DEQs and explicit neural networks.
method Random matrix theory and analysis of kernel matrices.
result A shallow explicit network can mimic the kernel of a DEQ.
The recent book by T. Piketty (Capital in the Twenty-First Century) promoted the important issue of wealth inequality. In the last twenty years, physicists and mathematicians developed models to derive the wealth distribution using discrete and continuous stochastic processes (random exchange models) as well as related…
We derive asset pricing formula for markets with incomplete information and subjective views.
problem Asset pricing in markets with informational imperfections and subjective investor beliefs.
method Closed-form market equilibrium formula based on Merton's model, non-linear system of equations, conditional posterior distribution.
result Derivation of market reference model for excess returns under random shadow-costs.
The paper introduces Robust Correlated Equilibrium for games with time-varying costs and proposes an algorithm to achieve it.
problem Games with time-varying costs and disturbances.
method Proposes Robust Correlated Equilibrium and a decentralized algorithm to learn optimal strategies.
result The algorithm converges to the Robust Correlated Equilibrium, showing no regret for each controller.
We seek to infer the parameters of an ergodic Markov process from samples taken independently from the steady state. Our focus is on non-equilibrium processes, where the steady state is not described by the Boltzmann measure, but is generally unknown and hard to compute, which prevents the application of established eq…
Study compares cryptocurrency and stock markets using statistical equilibrium models.
problem Comparing the stochastic structure of cryptocurrency and stock markets.
method Applied QRSE model to analyze daily returns of cryptocurrencies and S&P 500 companies.
result Revealed differences in informational efficiency between cryptocurrency and stock markets.
Gradient equilibrium improves online learning performance without requiring sublinear regret.
problem Achieving sublinear regret in online learning.
method Gradient equilibrium: average of gradients converges to zero.
result Gradient equilibrium can be achieved by standard online learning methods.
The paper characterizes equilibrium strategies under random risk aversion, showing unique solutions based on risk aversion distribution.
problem Characterizing equilibrium strategies in a continuous-time portfolio selection problem under random risk aversion.
method Provided a complete characterization of all deterministic equilibrium strategies in closed form, analyzing the structure of the solution based on the distribution of random risk aversion.
result The equilibrium is unique (if exists) when the expectation of random risk aversion is finite, but infinite expectation leads to either infinitely many equilibria or a unique trivial one.