New model combines physics and machine learning for ocean dynamics.
problem Discovering hidden laws governing ocean dynamics.
method Develops Deep Neural Numerical Models (DNNMs) to learn hidden variables of physical laws.
result Illustrates DNNMs applied to Sea Surface Height dynamics, connecting to QG model.
Deep learning upscales geologic models efficiently.
problem Upscaling large-scale geologic models for efficient simulation.
method Theory-guided convolutional neural network (TgCNN) trained to approximate hydraulic conductivity relationships.
result Deep learning method achieves equivalent upscaling accuracy to numerical methods but with significantly improved efficiency.
Deep neural networks are commonly developed and trained in 32-bit floating point format. Significant gains in performance and energy efficiency could be realized by training and inference in numerical formats optimized for deep learning. Despite advances in limited precision inference in recent years, training of neura…
The development of new classification and regression algorithms based on empirical risk minimization (ERM) over deep neural network hypothesis classes, coined deep learning, revolutionized the area of artificial intelligence, machine learning, and data analysis. In particular, these methods have been applied to the num…
FBSJNN solves PIDEs and FBSDEJs with deep learning, offering theoretical and numerical efficiency.
problem Solving Partial Integro-Differential Equations and Forward-Backward Stochastic Differential Equations with Jumps.
method FBSJNN framework using a single neural network for both solution approximation and non-local integral.
result FBSJNN achieves numerical solutions with a relative error of 10−3, demonstrating efficiency. Paper uses deep learning to solve PDEs without supervision.
problem Solving elliptic PDEs without labeled data.
method Uses deep neural networks and least-squares functionals.
result Demonstrates effectiveness on 1D second-order elliptic PDEs.
Deep quantum neural networks applied to finance for efficient risk management.
problem Efficiently solving numerical problems in finance, especially risk management.
method Application of deep quantum neural networks to finance, focusing on implied volatilities, option prices, and Greeks.
result Deep quantum neural networks can compute Greeks analytically and efficiently solve financial numerical problems.
New ADANNs improve PDE approximations.
problem Approximating operators for parametric PDEs.
method Custom ANN architectures and initialization schemes.
result ADANNs significantly outperform existing methods.
Generalized ResNet learns unknown dynamical systems using neural networks.
problem Learning unknown dynamical systems with deep neural networks.
method A generalized ResNet framework using discrepancy as model correction.
result Generalized ResNet produces more accurate predictions than standard ResNet.
Study deep maxout networks and their equivalence to Gaussian processes.
problem Understanding neural networks with infinite width.
method Derive equivalence between deep maxout networks and Gaussian processes, characterize maxout kernel, and provide efficient numerical implementation.
result Bayesian inference based on deep maxout network kernel leads to competitive results compared to finite-width counterparts and deep neural network kernels.
Deep neural networks struggle with numerical instability during training.
problem Numerical instability in gradient descent training of deep neural networks.
method Analysis of floating-point arithmetic and gradient descent in ReLU neural networks.
result It is highly unlikely for ReLU networks to maintain a superlinear number of affine pieces during training.
Deep-CAPTCHA cracks visual CAPTCHAs using deep learning.
problem Cracking visual CAPTCHAs to assess vulnerabilities.
method Developed a Convolutional Neural Network (Deep-CAPTCHA) to solve numerical and alphanumeric CAPTCHAs.
result Cracking accuracy of 98.94% and 98.31% for numerical and alphanumeric datasets respectively.
Generalization bounds derived for neural ODEs and deep residual networks.
problem Understanding the generalization capability of neural ODEs and deep residual networks.
method Lipschitz-based argument and analogy with deep residual networks.
result A generalization bound involving the magnitude of weight matrix differences.
Langevin algorithms enhance training of deep neural networks for stochastic control problems.
problem Training acceleration for deep neural networks in stochastic control problems.
method Application of Langevin algorithms to minimize the loss of deep neural networks in stochastic control problems.
result Langevin algorithms improve training on various stochastic control problems.
The paper studies how regularization parameters affect sparsity in deep neural networks.
problem Reducing the complexity of deep neural networks by promoting sparsity.
method Derives ℓ1-norm sparsity-promoting models, characterizes sparsity levels, and develops algorithms for selecting optimal regularization parameters. result Developed algorithms to select regularization parameters for desired sparsity levels in neural networks.
Sparse linear models improve neural network debuggability.
problem Improving neural network interpretability and debugging.
method Using sparse linear models over learned deep feature representations.
result The approach leads to more debuggable and accurate neural networks.
Deep neural networks provide meaningful uncertainty estimates for large-scale simulations.
problem Uncertainty estimates for deep neural network predictions from large-scale simulations.
method General variational inference approach to calibrate Bayesian uncertainties.
result Calibrated Bayesian uncertainties preserved physics-correlations in predicted quantities.
Develops deep learning for fast, accurate option pricing models.
problem Computational efficiency and accuracy in option pricing models.
method Neural network generators solving backward Kolmogorov equations for TPDFs.
result Ultra-fast, highly accurate option pricing models for various asset models.
New deep learning method approximates Benes filter model.
problem Approximating high-dimensional SPDEs for filtering.
method Deep learning mesh-free neural network representation.
result First study of neural network method for Benes model.
ST-GAN predicts stock trends using financial news and data.
problem Predicting financial trends in stock markets.
method ST-GAN combines NLP and technical indicators using GAN technology.
result Significant improvement over existing models in stock price forecasting.
In recent years, deep learning methods applying unsupervised learning to train deep layers of neural networks have achieved remarkable results in numerous fields. In the past, many genetic algorithms based methods have been successfully applied to training neural networks. In this paper, we extend previous work and pro…
Theoretical analysis of deep neural networks for time series data.
problem Theoretical development for deep neural networks on temporally dependent observations is lacking.
method Established non-asymptotic bounds for prediction error of deep neural networks under mixing-type assumptions.
result Deep neural networks can model non-linear time series data with additional logarithmic factors due to dependence.
Extends deep learning with interpretable additive models.
problem Identifiability issues between neural networks and additive models.
method Orthogonalization cell to separate deep neural network and structured model parts.
result Stable estimation and interpretability of structured model parts.
Existing deep learning models may encounter great challenges in handling graph structured data. In this paper, we introduce a new deep learning model for graph data specifically, namely the deep loopy neural network. Significantly different from the previous deep models, inside the deep loopy neural network, there exis…
Deep learning solves high-dimensional PDEs efficiently.
problem Solving high-dimensional Kolmogorov PDEs numerically.
method Reformulating as a statistical learning problem using Feynman-Kac formula.
result Single neural network learns entire family of PDEs.
Proposes a method to solve deep neural networks' local minimum problem.
problem Local minimum problem in deep neural networks training.
method Transforms cross-entropy loss into risk-averse error criterion, adjusts RSI, and uses convexity region.
result Trained deep learning machine is expected to be inside a global minimum's attraction basin.
Neural networks improve nonparametric regression with measurement errors.
problem Nonparametric regression with measurement errors.
method Proposes a neural network design using FNN, normalizing flow, and inference network.
result Neural network approach is more flexible and superior or comparable to classical methods.
Deep neural networks can approximate rough functions with high accuracy.
problem Approximating rough functions with neural networks.
method Proved that ENO interpolation can be cast as a deep ReLU neural network, transferring ENO's high-order accuracy.
result Deep neural networks can achieve high-order accuracy in approximating Lipschitz functions.
New framework tackles deep learning issues like local traps and miscalibration.
problem Local traps and miscalibration in deep neural networks.
method Sparse deep learning framework with prior annealing algorithms.
result Proposed method successfully addresses local traps and miscalibration.
The problem of state estimation for unobservable distribution systems is considered. A deep learning approach to Bayesian state estimation is proposed for real-time applications. The proposed technique consists of distribution learning of stochastic power injection, a Monte Carlo technique for the training of a deep ne…
DNA-SE uses deep learning to solve semiparametric problems efficiently.
problem Solving semiparametric integral equations in high dimensions.
method Formulates semiparametric estimation as a bi-level optimization problem and uses DNN to approximate solutions.
result Demonstrates numerical and statistical advantages over traditional methods.
Deep neural networks with memory learn reduced equations from partial data.
problem Constructing governing equations for unknown dynamical systems from limited data.
method Formulate a discrete approximation of memory integrals, use deep neural networks to incorporate history terms.
result Deep neural networks can learn reduced equations with memory from partial data.
Study uses neural networks to solve complex equations efficiently.
problem Solving parametric partial differential equations.
method Machine learning and deep neural networks.
result Performance of the model is independent of parameter space dimension.
Deep neural networks are workhorse models in machine learning with multiple layers of non-linear functions composed in series. Their loss function is highly non-convex, yet empirically even gradient descent minimisation is sufficient to arrive at accurate and predictive models. It is hitherto unknown why are deep neura…
Deep neural networks reduce weather forecast uncertainty estimation costs.
problem Accurate estimation of weather forecast uncertainty using ensemble prediction systems.
method Modified 3D U-Net architecture and models incorporating temporal data.
result Deep neural networks can estimate weather forecast uncertainty with fewer simulations.
Deep neural networks can solve optimal stopping problems without dimensionality issues.
problem Optimal stopping problems in high-dimensional state spaces.
method Established a general framework for deep ReLU neural networks to approximate value functions and continuation values.
result Deep neural networks can approximate value functions and continuation values with error at most ε of size κd^q ε^(-r).
New bounds for neural networks ensure robustness and accuracy.
problem Ensuring robustness of neural networks by computing Lipschitz constants.
method Analyzed and proposed new bounds for l1 and l∞ norms, using explicit and implicit methods for convnets. result One of the new bounds is optimal and more accurate than existing ones.
Deep neural network improves Heston model calibration accuracy and speed.
problem Calibrating the Heston model with numerical stability issues.
method Gradient-based deep learning framework (DDN) to learn Heston model and its derivatives.
result DDN significantly outperforms non-differential neural networks in calibration accuracy and speed.
Deep residual networks implicitly converge to neural ODEs.
problem Link between discrete and continuous deep learning models.
method Establishing implicit regularization for residual networks towards neural ODEs.
result Deep residual networks initialized as discretizations of neural ODEs converge to such ODEs during training.
Neural Networks improve incompressible flow simulations without complex kernels.
problem Simulating incompressible flows accurately and efficiently.
method Integrates Neural Networks with Random Vortex Dynamics for incompressible Navier-Stokes equations.
result Strictly enforces physical properties like incompressibility and boundary conditions.
NMDR estimates complex mixtures of distributions efficiently.
problem Estimating complex finite mixtures of distributions in high-dimensional settings.
method Flexible additive predictors, neural networks, and deep learning optimizers.
result Competitive performance in complex scenarios compared to existing approaches.
Deep learning's success is puzzling from a statistical perspective.
problem Deep learning's success is puzzling from a statistical perspective.
method Physics-informed investigation of deep learning features and surprises.
result Neural scaling laws and their interplay with inductive biases.
Neural Galerkin schemes use active learning to solve high-dimensional equations.
problem Inaccurate function approximations in high dimensions with limited training data.
method Neural Galerkin schemes based on deep learning with active learning for high-dimensional PDEs.
result Active data collection improves the numerical solution of high-dimensional equations.
Unified framework for Riemannian deep learning across manifold-valued representations.
problem Deep learning on manifold-valued representations often relies on Euclidean approximations or costly geometric operations.
method Develops reusable neural modules, manifold-specific network architectures, and geometric designs.
result Generalizes batch normalization and multinomial logistic regression to broader classes of manifolds.
Unified framework for Riemannian deep learning across manifold-valued representations.
problem Deep learning on manifold-valued data lacks reusable modules, specific network architectures, and efficient geometric operations.
method Develops reusable neural modules, manifold-specific network architectures, and geometric designs for broad classes of Lie groups and gyrogroups.
result Generalizes batch normalization and multinomial logistic regression to Riemannian manifolds, including SPD and hyperbolic spaces.
We prove that the binary classifiers of bit strings generated by random wide deep neural networks with ReLU activation function are biased towards simple functions. The simplicity is captured by the following two properties. For any given input bit string, the average Hamming distance of the closest input bit string wi…
Adaptive neural network approximates stochastic system densities.
problem Approximating high-dimensional stochastic dynamical systems.
method Temporal KRnet (tKRnet) trained with adaptive collocation points and temporal decomposition.
result Improves density approximation for stochastic systems without curse of dimensionality.
The paper calibrates the G2++ model using deep learning for interest rates.
problem Calibrating interest rate models with deep learning.
method Calibrated G2++ model using Neural Networks trained on covariances and correlations of Zero-Coupon and Forward rates.
result Deep learning calibration outperforms classic methods.