RevDEQs improve performance on tasks with exact gradients and fewer function evaluations.
problem Inexact gradient calculation in DEQs leads to unstable training and requires regularisation or many function evaluations.
method Introduce Reversible Deep Equilibrium Models (RevDEQs) that allow for exact gradient calculation, no regularisation, and far fewer function evaluations.
result RevDEQs significantly improve performance on language modelling and image classification tasks.
DEQs converge to optimal solutions with mild over-parameterization.
problem Training over-parameterized deep equilibrium models.
method Solves equilibrium point directly, uses gradient descent, and analyzes convergence via linear rate.
result Gradient descent converges to a globally optimal solution at a linear rate for quadratic loss.
DDEQs extend DEQs to discrete measure inputs using Wasserstein gradient flows.
problem Applying DEQs to discrete measure inputs like sets or point clouds.
method Wasserstein gradient flows for finding fixed points of discrete measures under permutation-invariance.
result DDEQs can compete with state-of-the-art models in tasks like point cloud classification and completion.
DREAM learns optimal strategies in imperfect games without needing a simulator.
problem Learning optimal strategies in imperfect-information games with multiple agents.
method DREAM is a deep reinforcement learning algorithm that converges to Nash Equilibria and coarse correlated equilibria.
result DREAM achieves state-of-the-art performance in benchmark games and is competitive with simulator-based algorithms.
Deep learning models replicate Kyle's model's market equilibrium.
problem Understanding market equilibria in asymmetric information settings.
method Using deep neural networks to model agents in Kyle's single period model.
result Trained networks' behavior converges to Kyle's predicted equilibrium.
Deep equilibrium models estimate latent variables from data.
problem Estimating latent variables from data.
method Generalized exponential family models, deep equilibrium networks.
result Deep equilibrium models solve MAP estimates for latent and transformation parameters.
Deep equilibrium models converge globally without explicit computation.
problem Global convergence of deep learning models with implicit layers.
method Analysis of gradient dynamics and proof of convergence rate.
result Deep equilibrium models converge to global optimum at a linear rate.
Gradient descent dynamics studied for DEQs in linear and single-index models.
problem Understanding gradient descent dynamics for DEQs.
method Rigorously studied gradient descent dynamics for DEQs in linear and single-index models.
result Gradient descent converges to a global minimizer for linear DEQs and single-index models.
This paper extends the convergence rate of DEQs with ReLU to any general activation.
problem Proving global convergence rate for DEQs with general activations.
method Developed a novel population Gram matrix and new form of dual activation with Hermite polynomial expansion.
result Gradient descent converges to a globally optimal solution at a linear rate for DEQs with general activations.
New model outperforms Neural ODEs while being more efficient.
problem Stable convergence and existence guarantees for implicit-depth models.
method Developed Monotone Operator Equilibrium Network (monDEQ) based on monotone operator theory.
result MonDEQ models outperform Neural ODEs and are more computationally efficient.
Deep fictitious play converges to Nash equilibrium in stochastic differential games.
problem Finding Nash equilibrium in large stochastic differential games.
method Decouples the game into sub-optimization problems and solves each player's optimal strategy with deep BSDE method.
result Deep fictitious play converges to the true Nash equilibrium.
We present a new approach to modeling sequential data: the deep equilibrium model (DEQ). Motivated by an observation that the hidden layers of many existing deep sequence models converge towards some fixed point, we propose the DEQ approach that directly finds these equilibrium points via root-finding. Such a method is…
New method stabilizes DEQ models by regularizing Jacobian of fixed-point equations.
problem Stability and performance of DEQ models.
method Jacobian regularization to stabilize DEQ models.
result Significant stabilization of fixed-point convergence in DEQ models.
HomoODE connects DEQs and Neural ODEs via homotopy continuation, improving accuracy and memory efficiency.
problem Connecting DEQs and Neural ODEs for better model performance and efficiency.
method Established a connection between DEQs and Neural ODEs using homotopy continuation, proposing HomoODE.
result HomoODE outperforms existing implicit models in accuracy and memory consumption.
Deep learning solves and estimates complex financial models.
problem Estimating and solving continuous-time financial models.
method Uses deep learning to solve and estimate models simultaneously.
result Demonstrates advantages like generality and large state space handling.
Discovery of atomistic systems with desirable properties is a major challenge in chemistry and material science. Here we introduce a novel, autoregressive, convolutional deep neural network architecture that generates molecular equilibrium structures by sequentially placing atoms in three-dimensional space. The model e…
New approaches improve adversarial robustness of DEQs.
problem Adversarial vulnerability of DEQs.
method Developed approaches to estimate intermediate gradients and integrate them into attacking pipelines.
result Demonstrated adversarial robustness of DEQs competitive with deep networks.
MDEQ models learn multi-resolution features efficiently.
problem Large-scale, hierarchical pattern recognition.
method Implicit differentiation, multiscale deep equilibrium model.
result MDEQs achieve performance on par with recent models.
Wide neural networks converge to Gaussian processes, improving generalization.
problem Understanding the generalization of wide neural networks, especially deep equilibrium models.
method Investigation of deep equilibrium models (DEQs) with infinite-depth layers, focusing on their convergence to Gaussian processes as width and depth approach infinity.
result Wide DEQs converge to Gaussian processes, maintaining generalization performance.
Deep RL solves complex economic models with heterogeneous agents.
problem Solving models with heterogeneous economic actors is difficult.
method Reinforcement Learning techniques for solving general equilibrium models.
result Successfully captures economic behaviors induced by age-based health risks.
Proposes a deep learning method for solving complex financial games with delays.
problem Financial modeling with multi-agent interactions and delayed effects.
method Parameterizes controls using recurrent neural networks and trains them with modified fictitious play.
result Demonstrates effectiveness on finance problems with known solutions and new problems with derived Nash equilibria.
Computing equilibrium states in condensed-matter many-body systems, such as solvated proteins, is a long-standing challenge. Lacking methods for generating statistically independent equilibrium samples in "one shot", vast computational effort is invested for simulating these system in small steps, e.g., using Molecular…
DEQs and explicit networks are nearly equivalent for Gaussian mixtures.
problem Understanding the equivalence between DEQs and explicit neural networks.
method Random matrix theory and analysis of kernel matrices.
result A shallow explicit network can mimic the kernel of a DEQ.
In this paper, we apply the idea of fictitious play to design deep neural networks (DNNs), and develop deep learning theory and algorithms for computing the Nash equilibrium of asymmetric N-player non-zero-sum stochastic differential games, for which we refer as \emph{deep fictitious play}, a multi-stage learning pro…
A new method uses deep learning to predict rare events in complex systems.
problem Predicting rare and extreme events in non-equilibrium systems.
method A deep learning approach that minimizes the geometrical action.
result The method accurately predicts rare events in various complex systems.
The paper solves a portfolio selection problem in incomplete markets by balancing utility and risk.
problem Time-inconsistent portfolio selection in incomplete markets.
method Characterizes equilibrium via a coupled quadratic BSDE system, introduces approximate equilibrium for general cases.
result Established existence theory for equilibrium strategies in special and general cases.
We propose a deep neural network-based algorithm to identify the Markovian Nash equilibrium of general large N-player stochastic differential games. Following the idea of fictitious play, we recast the N-player game into N decoupled decision problems (one for each player) and solve them iteratively. The individua…
This paper tackles traffic volume estimation challenges with a deep learning method.
problem Underdetermined and non-equilibrium traffic flows.
method Graph-based deep learning method with adaptive attention mechanisms.
result The proposed model achieves high accuracy even with low sensor coverage.
Machine learning detects tipping points in complex systems.
problem Detecting abrupt shifts in complex dynamical systems.
method Equilibrium-informed neural networks (EINNs) trained on candidate equilibrium states.
result EINNs can identify critical thresholds in nonlinear systems.
Improves adversarial robustness of DEQ models by regulating neural dynamics.
problem Limited adversarial robustness of DEQ models.
method Interprets DEQs as neural dynamics, uses entropy reduction and random intermediate states.
result Significantly increases adversarial robustness of DEQ models.
Two-cycle GEILA equilibria are OLG equilibria and vice versa, with applications to indeterminacy and bubbles.
problem Relationship between GEILA and OLG models.
method Proof of equilibrium equivalence and application to indeterminacy and bubbles.
result GEILA and OLG models are equivalent under certain conditions.
Deep RL solves complex macroeconomic models.
problem Solving dynamic stochastic general equilibrium models with bounded rationality.
method Using deep reinforcement learning to model agents as neural networks.
result Artificially intelligent agents can solve models in all policy regimes.
A multi-layer deep Gaussian process (DGP) model is a hierarchical composition of GP models with a greater expressive power. Exact DGP inference is intractable, which has motivated the recent development of deterministic and stochastic approximation methods. Unfortunately, the deterministic approximation methods yield a…
Agents trained with reinforcement learning deviate from Nash equilibrium in optimal execution game.
problem Deviation of reinforcement learning strategies from Nash equilibrium in optimal execution game.
method Two-player optimal execution game with reinforcement learning algorithms (Double Deep Q-Learning).
result Strategies learned by agents deviate significantly from Nash equilibrium, exhibiting supra-competitive solutions.
Generative model solves financial market equilibria with stable reinforcement learning.
problem Financial market equilibria under realistic frictions and multiple agents.
method Generative adversarial reinforcement learning with decoupling feedback.
result Algorithm learns and predicts asset returns and volatilities.
Generative operators solve many convex problems with minimal parameters.
problem Worst-case parameter bounds limit the practical use of neural operators.
method Developed generative equilibrium operators (GEOs) using realizable finite-dimensional layers.
result GEOs can uniformly approximate solutions to convex optimization problems with logarithmic growth in parameters.
We prove the existence of a Radner equilibrium in a model with proportional transaction costs on an infinite time horizon and analyze the effect of transaction costs on the endogenously determined interest rate. Two agents receive exogenous, unspanned income and choose between consumption and investing into an annuity.…
Stochastic normalizing flows improve lattice field theory simulations.
problem Efficiently sample lattice field theories.
method Combining neural-network layers with Monte Carlo updates.
result Stochastic normalizing flows are equivalent to out-of-equilibrium simulations.
Algorithm finds ε-equilibrium policies for multi-agent Markov games with hidden low-rank structure.
problem Designing efficient algorithms for multi-agent Markov games with unknown representation and hidden low-rank structure.
method Model-based and model-free approaches using representation learning to construct an effective representation from data.
result Achieves poly(H,d,A,1/ε) sample complexity for both model-based and model-free approaches. This paper considers the problem of inverse reinforcement learning in zero-sum stochastic games when expert demonstrations are known to be not optimal. Compared to previous works that decouple agents in the game by assuming optimality in expert strategies, we introduce a new objective function that directly pits expert…
A new method relaxes molecules without needing non-equilibrium data.
problem Molecular relaxation requires understanding non-equilibrium structures.
method MoreRed: molecular relaxation by reverse diffusion with time step prediction.
result MoreRed learns a simpler pseudo potential energy surface.
The paper challenges the belief that more inner iterations at test time improve performance in implicit deep learning.
problem The performance improvement of implicit deep learning models with increased inner iterations at test time.
method Theoretical analysis of a simple setting, validation on implicit deep learning problems.
result Overparametrization plays a key role; increasing the number of iterations at test time does not improve performance for overparametrized networks.
In recent years, deep learning researchers have focused on how to find the interpretability behind deep learning models. However, today cognitive competence of human has not completely covered the deep learning model. In other words, there is a gap between the deep learning model and the cognitive mode. How to evaluate…
Existence of Radner equilibrium proven with growing population.
problem Analyzing Radner equilibrium in a model with population growth.
method Proved existence of equilibrium for growing population using mathematical analysis.
result Equilibrium exists for a growing population, with effects on annuity prices.
Deep CapsNet improves sign language recognition from wearable IMUs.
problem Continuous recognition of sign language from wearable devices.
method Custom CapsNet architecture using deep capsule networks and game theory.
result Improved accuracy of 94% and 92.50% for 3 and 5 routings respectively, compared to 87.99% for CNN.
We address the challenge of designing optimal adversarial noise algorithms for settings where a learner has access to multiple classifiers. We demonstrate how this problem can be framed as finding strategies at equilibrium in a two-player, zero-sum game between a learner and an adversary. In doing so, we illustrate the…
Kyle's equilibrium model stability proven for 1-2 trading times, but not for 3 or more.
problem Stability of Kyle's equilibrium model in a dynamic trading setting.
method Proof of stability for 1-2 trading times, non-stability for 3 or more trading times, independent of input parameters.
result Kyle's equilibrium model is stable for 1-2 trading times but not for 3 or more trading times.
The paper explores how symmetries and noise in SGD influence parameter dynamics.
problem Understanding the dynamics of parameter updates in SGD with symmetries.
method Proved the existence of noise equilibria and showed their role in balancing gradient noise.
result Gradient noise creates a systematic motion of parameters to a unique fixed point, called noise equilibria.