PANIS learns PDE surrogates for heterogeneous materials without solving the PDE.
problem Learning surrogates for parametrized PDEs in heterogeneous media.
method Physics-aware neural implicit solvers combining probabilistic learning and physics-informed discretization.
result Learned surrogates for effective solutions in heterogeneous materials without solving the reference problem.
Reduced order modeling of energetic materials using physics-aware neural networks.
problem Simulating complex spatiotemporal dynamics in energetic materials.
method Physics-aware recurrent convolutions (PARC) combined with latent space projection to accelerate model training and inference.
result Significant decrease in training and inference time with comparable accuracy.
New model outperforms Neural ODEs while being more efficient.
problem Stable convergence and existence guarantees for implicit-depth models.
method Developed Monotone Operator Equilibrium Network (monDEQ) based on monotone operator theory.
result MonDEQ models outperform Neural ODEs and are more computationally efficient.
Higher-order ODE solvers improve deep learning performance.
problem Improving deep learning performance using higher-order ODE solvers.
method Evaluation and improvement of Runge-Kutta (RK) methods for deep learning.
result Higher-order RK solvers can improve deep learning performance by incorporating key ingredients of optimizers.
RINS-T solves time series inverse problems robustly without pretraining.
problem Recovering original signals from corrupted time series data.
method Implicit neural solvers with robust optimization techniques.
result RINS-T achieves high recovery performance without pretraining.
Bayesian Entropy Neural Networks enforce constraints on deep learning predictions.
problem Deep learning models lack well-defined constraints in their outputs.
method Bayesian Entropy Neural Networks (BENN) using Maximum Entropy principles and the method of multipliers.
result BENN improves model robustness and reliability across various applications.
Unified framework for forward and inverse PDE problems in multiphase media.
problem Non-differentiable inverse problems in discrete-valued material fields.
method GenPANIS: Latent-variable generative framework preserving discrete microstructures.
result Unified bidirectional inference with minimal labeled pairs and physics-aware decoder.
Hessian-free training has become a popular parallel second or- der optimization technique for Deep Neural Network training. This study aims at speeding up Hessian-free training, both by means of decreasing the amount of data used for training, as well as through reduction of the number of Krylov subspace solver iterati…
This work presents an explicit-implicit procedure to compute a model predictive control (MPC) law with guarantees on recursive feasibility and asymptotic stability. The approach combines an offline-trained fully-connected neural network with an online primal active set solver. The neural network provides a control inpu…
Stable neural flows ensure robustness and efficiency in deep learning.
problem Ensuring robustness and stability in deep learning models.
method Introducing a stable variant of neural ODEs with a neural network parametrizing an energy functional, solving as an optimal control problem with adjoint sensitivity analysis.
result The proposed model provides robustness against input perturbations and low computational burden.
Neural Networks improve incompressible flow simulations without complex kernels.
problem Simulating incompressible flows accurately and efficiently.
method Integrates Neural Networks with Random Vortex Dynamics for incompressible Navier-Stokes equations.
result Strictly enforces physical properties like incompressibility and boundary conditions.
A GPU-based workflow for building physics emulators of hypersonic flows
problem Resolving complex physical phenomena in hypersonic flows
method Fully GPU-based workflow integrating accelerated data generation and neural emulators
result Physics emulators remain reliable beyond their training distribution
We describe a set of Gaussian Process based approaches that can be used to solve non-linear Ordinary Differential Equations. We suggest an explicit probabilistic solver and two implicit methods, one analogous to Picard iteration and the other to gradient matching. All methods have greater accuracy than previously sugge…
Gradients of neural networks can be computed efficiently for any architecture, but some applications require differential operators with higher time complexity. We describe a family of restricted neural network architectures that allow efficient computation of a family of differential operators involving dimension-wise…
Developing stable and scalable probabilistic ODE solvers for stiff and high-dimensional problems.
problem Stiff and high-dimensional ODEs
method Matrix-free update step and iterative re-linearization
result Improved stability and scalability
Optimizes neural networks with blackbox solvers using Time-cost Regularization.
problem Improving neural network performance by integrating efficient solvers for complex problems.
method Optimizes both the primary loss function and the performance of the blackbox solver using Time-cost Regularization. Introduces a hyper-blackbox concept to learn blackbox parameters.
result Significant improvement in neural network performance through optimization of blackbox solvers.
EnCF improves data assimilation for implicit, non-smooth observations.
problem Data assimilation challenges with implicit, many-to-one observations.
method EnCF uses a stochastic controlled flow to update forecast distributions.
result EnCF outperforms Kalman filters for non-Gaussian, implicit observations.
A key problem in computational material science deals with understanding the effect of material distribution (i.e., microstructure) on material performance. The challenge is to synthesize microstructures, given a finite number of microstructure images, and/or some physical invariances that the microstructure exhibits. …
New ODE solvers improve training efficiency and accuracy.
problem Training Neural ODEs requires efficient and accurate gradient calculation.
method Presented algebraically reversible ODE solvers that are time and memory efficient, calculate exact gradients, and are numerically stable.
result Reversible solvers strictly improve upon previous architectures in efficiency and accuracy.
Paper studies the theoretical equivalence between implicit and explicit neural networks in high dimensions.
problem Lack of theoretical analysis of implicit and explicit neural networks.
method Examined high-dimensional implicit neural networks and established their equivalence to explicit networks.
result Equivalence between implicit and explicit neural networks in high dimensions.
FNO-DEQ solves steady-state PDEs as fixed points, outperforming traditional FNOs.
problem Lack of understanding in designing neural network architectures for PDEs.
method Proposes FNO-DEQ, a deep equilibrium architecture that solves steady-state PDEs as fixed points.
result FNO-DEQ outperforms FNO-based architectures in predicting solutions to steady-state PDEs.
Proposes efficient, modular method for implicit differentiation.
problem Implicit differentiation of optimization problems.
method Automatic implicit differentiation using autodiff and implicit function theorem.
result Automatic differentiation of optimization problems is made easier and more modular.
New deep learning method for option pricing in jump-diffusion models.
problem Option pricing in jump-diffusion models with high-dimensional assets.
method Implicit-explicit minimizing movement time-stepping approach using deep ANNs.
result Consistent and asymptotically correct solutions for large underlyings.
This paper shows how to train only the implicit layer of overparameterized implicit neural networks.
problem Understanding how the implicit layer contributes to the training of overparameterized implicit neural networks.
method Restricting training to only the implicit layer and analyzing the generalization error for ReLU-activated networks.
result Global convergence is guaranteed even if only the implicit layer is trained, and gradient flow with proper random initialization can achieve small generalization errors.
New neural networks with variable time constants for better time-series prediction.
problem Improving neural network performance in time-series prediction.
method Constructing networks of linear dynamical systems modulated by nonlinear gates, using numerical differential equation solvers.
result Liquid Time-Constant Networks (LTCs) yield superior performance on time-series prediction tasks.
Protocol diagnoses neural HJB-PIDE solvers for Lévy jumps, revealing a missing factor in their importance-proposal density.
problem Neural PDE solvers can match scalar diagnostics but miscompute operators, leading to systematic errors.
method Five-step diagnostic protocol decomposes neural solve into components, compares them with independent reference solutions.
result Corrected a missing 1/2-mixture factor in the neural method's importance-proposal density, improving control accuracy.
MIP-GNN uses graph neural networks to predict variable biases for MIP solvers.
problem Improving combinatorial optimization through data-driven insights.
method Encoding MILP interactions as graphs, training a graph neural network to predict variable biases, and guiding the MIP solver with these predictions.
result Significant improvements in solving binary MILPs compared to default settings of state-of-the-art solvers.
The paper develops a physics-aware method for modeling multiscale dynamics with reduced data.
problem Discovering effective, lower-dimensional models for high-dimensional dynamical systems.
method Probabilistic deep neural networks incorporating physical constraints.
result The method reduces the need for extensive multiscale simulations (Small Data regime).
Moser Flow generates models for complex geometries on manifolds without ODE solvers.
problem Learning generative models for complex geometries like spheres and tori.
method Moser Flow is a new class of continuous normalizing flows that parameterizes the model density as the divergence of a neural network.
result Moser Flow achieves significant improvements in density estimation, sample quality, and training complexity over existing methods.
Partial differential equations (PDEs) are widely used across the physical and computational sciences. Decades of research and engineering went into designing fast iterative solution methods. Existing solvers are general purpose, but may be sub-optimal for specific classes of problems. In contrast to existing hand-craft…
Gaussian processes enhance EO with accurate and uncertain predictions.
problem Challenges in GP models for EO, including data-driven physics and causal inference.
method Data-driven physics-aware models respecting signal characteristics and physical laws.
result GP models need to evolve for better EO applications.
This paper proves SGD converges to global minimum for over-parameterized ReLU networks.
problem Theoretical understanding of implicit neural networks is limited.
method Gradient flow analysis of ReLU activated implicit neural networks.
result Randomly initialized gradient descent converges to global minimum at a linear rate for square loss function in over-parameterized ReLU networks.
The paper explains implicit regularization in hierarchical tensor factorization and deep CNNs.
problem Understanding implicit regularization in complex neural network architectures.
method Theoretical analysis using dynamical systems to overcome challenges in hierarchy.
result Established implicit regularization towards low hierarchical tensor rank, equivalent to locality in CNNs.
This work introduces a new loss function to improve the efficiency of optimization-based PDE solvers.
problem Optimization-based PDE solvers converge slowly and are inefficient compared to classical iterative solvers.
method Proposes a novel Stabilized Gradient Residual (SGR) loss function to modulate the condition number.
result The SGR loss achieves orders-of-magnitude faster convergence than the MSE loss in both ODIL and PINNs frameworks.
Paper optimizes neural network initialization using SMT solvers.
problem Improving neural network performance through better initialization.
method Reduces initialization to SMT problem solving.
result Proposed method achieves better performance than random initialization.
GENOT matches cells across data modalities using neural OT solvers.
problem Scalability, privacy, and out-of-sample estimation issues in traditional OT solvers.
method Learn stochastic maps, parameterize OT maps, relax mass conservation, integrate quadratic solvers.
result Demonstrates significant potential for enhancing therapeutic strategies.
The paper analyzes implicit regularization in tensor factorization using neural networks.
problem Understanding implicit regularization in tensor factorization.
method Dynamical systems perspective and gradient descent analysis.
result Gradient descent induces a form of greedy low tensor rank search.
Single model learns physics from diverse data.
problem Lack of universal physics models for diverse applications.
method General Physics Transformer (GPhyT) trained on diverse physics data.
result Single model achieves superior performance across multiple physics domains.
New method solves PDEs for any initial condition without retraining.
problem Solving PDEs for different initial conditions requires retraining neural solvers.
method Formulate solution as conditional probability distribution.
result Approximates PDE solution for arbitrary initial conditions.
DPM-Solver speeds up DPM sampling to 10-20 function evaluations.
problem Slow sampling from Diffusion Probabilistic Models (DPMs).
method Exact formulation of diffusion ODE solutions, using change-of-variable and exponentially weighted integral.
result Generates high-quality samples in 10-20 function evaluations.
New method for high-fidelity shape representations from raw data.
problem Creating accurate shape representations from raw data.
method A simple loss function encouraging neural network to vanish on input point cloud and have unit norm gradient.
result Our method produces high-fidelity, smooth, and natural zero level set surfaces.
Gradient descent biases towards stable rank networks for nearly-orthogonal data.
problem Understanding implicit bias in non-smooth neural networks trained by gradient descent.
method Analysis of two-layer ReLU and leaky ReLU networks trained by gradient descent on nearly-orthogonal data.
result Gradient descent biases towards networks with stable rank and uniform margin for nearly-orthogonal data.
A scalable deep learning framework accelerates training of large neural networks for solving 3D Poisson equations.
problem Training large-scale neural networks for solving complex PDEs efficiently.
method Combines multigrid techniques with distributed deep learning to accelerate training.
result Solves 3D Poisson equations up to 512x512x512 resolution efficiently.
In his book with Alan Jolis, Vers un monde sans pauvreté (1997) Yunus gives the example of a microcredit loan of 1000BDT reimbursed via 50 weekly settlements of 22BDT and correctly claims that this corresponds to the annual interest rate of 20%. But this is without taking into account that if the borrower has good reas…
Improved neural-ODE for faster convergence and stability.
problem Stability, consistency, and convergence issues in neural-ODE solvers.
method Proposed a first-order Nesterov's accelerated gradient (NAG) based ODE-solver.
result Efficacy demonstrated in three tasks: supervised classification, density estimation, and time-series modelling.
We propose a new forward-backward stochastic differential equation solver for high-dimensional derivatives pricing problems by combining deep learning solver with least square regression technique widely used in the least square Monte Carlo method for the valuation of American options. Our numerical experiments demonst…
This paper presents OptNet, a network architecture that integrates optimization problems (here, specifically in the form of quadratic programs) as individual layers in larger end-to-end trainable deep networks. These layers encode constraints and complex dependencies between the hidden states that traditional convoluti…
Study shows how steepest descent algorithms' geometric margin increases during training.
problem Understanding implicit bias in steepest descent algorithms for neural networks.
method Analysis of steepest descent algorithms with infinitesimal learning rates in homogeneous neural networks.
result Limit points of training trajectories correspond to KKT points of margin-maximization problems.