RevDEQs improve performance on tasks with exact gradients and fewer function evaluations.
problem Inexact gradient calculation in DEQs leads to unstable training and requires regularisation or many function evaluations.
method Introduce Reversible Deep Equilibrium Models (RevDEQs) that allow for exact gradient calculation, no regularisation, and far fewer function evaluations.
result RevDEQs significantly improve performance on language modelling and image classification tasks.
A new method relaxes molecules without needing non-equilibrium data.
problem Molecular relaxation requires understanding non-equilibrium structures.
method MoreRed: molecular relaxation by reverse diffusion with time step prediction.
result MoreRed learns a simpler pseudo potential energy surface.
This paper analyzes a class of infinite-time-horizon stochastic games with singular controls motivated from the partially reversible problem. It provides an explicit solution for the mean-field game (MFG) and presents sensitivity analysis to compare the solution for the MFG with that for the single-agent control proble…
Machine learning detects tipping points in complex systems.
problem Detecting abrupt shifts in complex dynamical systems.
method Equilibrium-informed neural networks (EINNs) trained on candidate equilibrium states.
result EINNs can identify critical thresholds in nonlinear systems.
A central problem in machine learning involves modeling complex data-sets using highly flexible families of probability distributions in which learning, sampling, inference, and evaluation are still analytically or computationally tractable. Here, we develop an approach that simultaneously achieves both flexibility and…
Unified framework for sampling from complex distributions, including discrete and mixed-variable systems.
problem Sampling from complex unnormalized distributions, especially in discrete or mixed-variable systems.
method Enforces time-reversibility using a prescribed physical transition kernel to minimize Maximum Mean Discrepancy (MMD).
result Demonstrates accurate reproduction of thermodynamic observables and mode-switching behavior across diverse systems.
Study shows price bubbles can exist even with heterogeneous beliefs.
problem Equilibrium price formation in markets with different belief groups.
method Analyzes continuous time asset trading with heterogeneous investors and mean reverting asset.
result Price bubbles may not form even with heterogeneous beliefs, contrary to initial expectations.
Within the description of stochastic differential equations it is argued that the existence of Boltzmann-Gibbs type distribution in economy is independent of the time reversal symmetry in econodynamics. Both power law and exponential distributions can be accommodated by it. The demonstration is based on a mathematical …
DEQs converge to optimal solutions with mild over-parameterization.
problem Training over-parameterized deep equilibrium models.
method Solves equilibrium point directly, uses gradient descent, and analyzes convergence via linear rate.
result Gradient descent converges to a globally optimal solution at a linear rate for quadratic loss.
SARD improves deep learning clinical prediction performance.
problem Deep learning models struggle to match linear models in healthcare predictions.
method Reverse Distillation to initialize deep models, combined with contextual and temporal embeddings.
result SARD outperforms state-of-the-art methods on clinical prediction outcomes.
PETRA enables parallel training of deep models with reversible architectures.
problem Challenges in parallelizing deep model training.
method Introduces PETRA, a novel approach for parallelizing gradient computations in reversible architectures.
result Achieves competitive accuracies on CIFAR-10, ImageNet32, and ImageNet using ResNet models.
DDEQs extend DEQs to discrete measure inputs using Wasserstein gradient flows.
problem Applying DEQs to discrete measure inputs like sets or point clouds.
method Wasserstein gradient flows for finding fixed points of discrete measures under permutation-invariance.
result DDEQs can compete with state-of-the-art models in tasks like point cloud classification and completion.
We found that factors decay over time, with momentum fitting best.
problem Understanding how factors decay over time and their impact on performance.
method Derived a hyperbolic decay model for factors, tested against linear and exponential alternatives.
result Momentum exhibits hyperbolic decay, outperforming linear and exponential models.
Reverse Experience Replay improves Deep Q-learning for sparse rewards.
problem Sparse rewards and reward-maximizing tasks in Deep Q-learning.
method Sampling transitions in reverse order for training.
result Significantly increased performance in tasks with limited experience and memory capacity.
Study on how non-reversible diffusion processes affect homology on manifolds.
problem Understanding the asymptotic behavior of random homology in diffusion processes.
method Investigation of asymptotic properties of random homology associated with stochastic diffusion processes on compact Riemannian manifolds.
result For quadratic rate, manifold is a locally trivial fiber bundle over a flat torus with minimal fibers.
The paper provides a geometric framework for understanding non-equilibrium thermodynamics.
problem Unclear geometric structure of GENERIC in non-equilibrium thermodynamics.
method Cotangent lifts of dynamics, splitting into holonomic and vertical representatives, and formulation within contact geometry.
result Physical meaning and explicit formulation of the second law of thermodynamics within evolution equations.
DREAM learns optimal strategies in imperfect games without needing a simulator.
problem Learning optimal strategies in imperfect-information games with multiple agents.
method DREAM is a deep reinforcement learning algorithm that converges to Nash Equilibria and coarse correlated equilibria.
result DREAM achieves state-of-the-art performance in benchmark games and is competitive with simulator-based algorithms.
Deep learning models replicate Kyle's model's market equilibrium.
problem Understanding market equilibria in asymmetric information settings.
method Using deep neural networks to model agents in Kyle's single period model.
result Trained networks' behavior converges to Kyle's predicted equilibrium.
Study compares employers with and without anticipating strategic labor force responses.
problem Understanding and optimizing strategic interactions in labor markets.
method Formulation of causal strategic classification, theory, and experiments.
result Performatively optimal hiring policies improve employer and labor outcomes, but can also harm labor force utility.
Deep equilibrium models estimate latent variables from data.
problem Estimating latent variables from data.
method Generalized exponential family models, deep equilibrium networks.
result Deep equilibrium models solve MAP estimates for latent and transformation parameters.
Deep equilibrium models converge globally without explicit computation.
problem Global convergence of deep learning models with implicit layers.
method Analysis of gradient dynamics and proof of convergence rate.
result Deep equilibrium models converge to global optimum at a linear rate.
Gradient descent dynamics studied for DEQs in linear and single-index models.
problem Understanding gradient descent dynamics for DEQs.
method Rigorously studied gradient descent dynamics for DEQs in linear and single-index models.
result Gradient descent converges to a global minimizer for linear DEQs and single-index models.
The paper explains how to predict returns based on firm characteristics.
problem Predicting returns based on firm characteristics in equilibrium models.
method Reverse-engineering equilibrium construction process with linear demands in characteristics.
result Linear expressions for returns are derived from scaled net aggregate demands and their variations.
This paper extends the convergence rate of DEQs with ReLU to any general activation.
problem Proving global convergence rate for DEQs with general activations.
method Developed a novel population Gram matrix and new form of dual activation with Hermite polynomial expansion.
result Gradient descent converges to a globally optimal solution at a linear rate for DEQs with general activations.
New model outperforms Neural ODEs while being more efficient.
problem Stable convergence and existence guarantees for implicit-depth models.
method Developed Monotone Operator Equilibrium Network (monDEQ) based on monotone operator theory.
result MonDEQ models outperform Neural ODEs and are more computationally efficient.
Deep fictitious play converges to Nash equilibrium in stochastic differential games.
problem Finding Nash equilibrium in large stochastic differential games.
method Decouples the game into sub-optimization problems and solves each player's optimal strategy with deep BSDE method.
result Deep fictitious play converges to the true Nash equilibrium.
We present a new approach to modeling sequential data: the deep equilibrium model (DEQ). Motivated by an observation that the hidden layers of many existing deep sequence models converge towards some fixed point, we propose the DEQ approach that directly finds these equilibrium points via root-finding. Such a method is…
New method stabilizes DEQ models by regularizing Jacobian of fixed-point equations.
problem Stability and performance of DEQ models.
method Jacobian regularization to stabilize DEQ models.
result Significant stabilization of fixed-point convergence in DEQ models.
We provide a microfoundation for linear price impact models in a stationary market.
problem Deriving linear price impact models in a stationary market with asymmetric information.
method Deriving linear price impact models as the equilibrium of an agent-based system.
result The model shows compatibility with universal price diffusion at small times and non-universal mean-reversion at larger times.
HomoODE connects DEQs and Neural ODEs via homotopy continuation, improving accuracy and memory efficiency.
problem Connecting DEQs and Neural ODEs for better model performance and efficiency.
method Established a connection between DEQs and Neural ODEs using homotopy continuation, proposing HomoODE.
result HomoODE outperforms existing implicit models in accuracy and memory consumption.
Deep learning solves and estimates complex financial models.
problem Estimating and solving continuous-time financial models.
method Uses deep learning to solve and estimate models simultaneously.
result Demonstrates advantages like generality and large state space handling.
Markov state models (MSMs) and Master equation models are popular approaches to approximate molecular kinetics, equilibria, metastable states, and reaction coordinates in terms of a state space discretization usually obtained by clustering. Recently, a powerful generalization of MSMs has been introduced, the variationa…
In the theory of riskfree hedges in continuous time finance, one can start with the delta-hedge and derive the option pricing equation, or one can start with the replicating, self-financing hedging strategy and derive both the delta-hedge and the option pricing partial differential equation. Approximately reversible tr…
Discovery of atomistic systems with desirable properties is a major challenge in chemistry and material science. Here we introduce a novel, autoregressive, convolutional deep neural network architecture that generates molecular equilibrium structures by sequentially placing atoms in three-dimensional space. The model e…
In a closed economic system, money is conserved. Thus, by analogy with energy, the equilibrium probability distribution of money must follow the exponential Gibbs law characterized by an effective temperature equal to the average amount of money per economic agent. We demonstrate how the Gibbs distribution emerges in c…
Deep neural network solves large multi-agent games for Markovian Nash equilibrium.
problem Finding Markovian Nash equilibrium in large multi-agent stochastic differential games.
method Reformulate as decoupled decision problems, solve iteratively using deep BSDE method.
result Proposed algorithm accurately finds Nash equilibrium in large games.
In this paper mechanisms of reversion - momentum transition are considered. Two basic nonlinear mechanisms are highlighted: a slow and fast bifurcation. A slow bifurcation leads to the equilibrium evolution, preceded by stability loss delay of a control parameter. A single order parameter is introduced by Markovian cha…
New approaches improve adversarial robustness of DEQs.
problem Adversarial vulnerability of DEQs.
method Developed approaches to estimate intermediate gradients and integrate them into attacking pipelines.
result Demonstrated adversarial robustness of DEQs competitive with deep networks.
The success of enhanced sampling molecular simulations that accelerate along collective variables (CVs) is predicated on the availability of variables coincident with the slow collective motions governing the long-time conformational dynamics of a system. It is challenging to intuit these slow CVs for all but the simpl…
Rate GENERIC extends thermodynamics principles to non-equilibrium systems.
problem Understanding non-equilibrium thermodynamics and its relation to equilibrium thermodynamics.
method Developed a geometrical framework for rate GENERIC, extending Onsager's variational principle.
result Rate GENERIC structure provides a new perspective on thermodynamics in non-equilibrium systems.
Neural differential equations combine deep learning and differential equations for modeling complex systems.
problem Modeling complex systems with high capacity and efficiency.
method Combining neural networks and differential equations, focusing on neural ordinary, controlled, and stochastic differential equations.
result NDEs offer high-capacity function approximation, strong priors, and handle irregular data efficiently.
The paper develops a new probabilistic framework for denoising diffusion models using free entropy and stochastic analysis.
problem Developing a mathematical framework for denoising diffusion models in noncommutative settings.
method Formulating diffusion and reverse processes governed by operator-valued stochastic dynamics, using tools from free stochastic analysis.
result Establishing an information-geometric link between entropy production, transport, and deconvolution.
The paper characterizes equilibrium strategies under random risk aversion, showing unique solutions based on risk aversion distribution.
problem Characterizing equilibrium strategies in a continuous-time portfolio selection problem under random risk aversion.
method Provided a complete characterization of all deterministic equilibrium strategies in closed form, analyzing the structure of the solution based on the distribution of random risk aversion.
result The equilibrium is unique (if exists) when the expectation of random risk aversion is finite, but infinite expectation leads to either infinitely many equilibria or a unique trivial one.
MDEQ models learn multi-resolution features efficiently.
problem Large-scale, hierarchical pattern recognition.
method Implicit differentiation, multiscale deep equilibrium model.
result MDEQs achieve performance on par with recent models.
Wide neural networks converge to Gaussian processes, improving generalization.
problem Understanding the generalization of wide neural networks, especially deep equilibrium models.
method Investigation of deep equilibrium models (DEQs) with infinite-depth layers, focusing on their convergence to Gaussian processes as width and depth approach infinity.
result Wide DEQs converge to Gaussian processes, maintaining generalization performance.
Deep RL solves complex economic models with heterogeneous agents.
problem Solving models with heterogeneous economic actors is difficult.
method Reinforcement Learning techniques for solving general equilibrium models.
result Successfully captures economic behaviors induced by age-based health risks.
Proposes a deep learning method for solving complex financial games with delays.
problem Financial modeling with multi-agent interactions and delayed effects.
method Parameterizes controls using recurrent neural networks and trains them with modified fictitious play.
result Demonstrates effectiveness on finance problems with known solutions and new problems with derived Nash equilibria.
Recently, deep residual networks have been successfully applied in many computer vision and natural language processing tasks, pushing the state-of-the-art performance with deeper and wider architectures. In this work, we interpret deep residual networks as ordinary differential equations (ODEs), which have long been s…