We propose a deep learning based method, the Deep Ritz Method, for numerically solving variational problems, particularly the ones that arise from partial differential equations. The Deep Ritz method is naturally nonlinear, naturally adaptive and has the potential to work in rather high dimensions. The framework is qui…
Paper uses deep Ritz method for solving stationary Schrödinger equation, proving convergence and feature emergence.
problem Solving stationary Schrödinger equation with high-dimensional features.
method Deep Ritz method, gradient descent, single-index model, two-neuron model.
result Gradient descent converges to near-optimal solution, feature emergence observed in two-neuron model.
Paper studies deep learning for solving elliptic PDEs, proving optimal bounds and neural scaling laws.
problem Solving elliptic PDEs from random samples using machine learning.
method Deep Ritz Method and Physics-Informed Neural Networks (PINNs) for the Schrödinger equation.
result Proves minimax optimal bounds and neural scaling laws for deep PDE solvers.
Deep neural solvers can approximate harmonic functions with low error.
problem Harmonic functions in Barron space are not regular.
method Elliptic regularity theory applied to Barron functions.
result Approximation of harmonic functions by Barron functions with low error.
Paper analyzes DRM for solving high-dimensional elliptic PDEs with generalization bounds.
problem Analyzing generalization error of neural network methods for high-dimensional PDEs.
method Developed a new solution theory for spectral Barron space and derived generalization error bounds.
result Generalization error bounds are independent of dimension and solutions lie in spectral Barron space.
Study analyzes error in neural network solving PDEs, providing convergence and error bounds.
problem Error analysis of neural network solving PDEs.
method Three-layer tanh neural network with projected gradient descent (PGD).
result Comprehensive error analysis including approximation, generalization, and optimization errors.
This paper compares deeper and wider neural networks for optimal generalization error in Sobolev losses.
problem The dilemma of choosing between deeper or wider neural networks for optimal generalization error.
method Analytical investigations into the influence of sample points, parameters, and loss function regularity on neural network architecture.
result A higher number of parameters favors wider neural networks, while more sample points and greater loss function regularity favor deeper neural networks.
Paper develops IFTRR to solve sparse generalized eigenvalue problems efficiently.
problem Finding the leading eigenvector with at most k nonzero entries in sparse generalized eigenvalue problems.
method Inverse-free truncated Rayleigh-Ritz method (IFTRR) with a new truncation strategy.
result IFTRR efficiently finds the support set of the leading eigenvector for large scale problems.
Proposes MscaleDNN for solving high-dimensional PDEs efficiently.
problem Solving high-dimensional PDEs efficiently.
method Radial scaling in frequency domain and compact support activation functions.
result Increased power in multi-scale resolution and high frequency capturing.
NWoS solves high-dimensional Poisson equations using neural networks.
problem Efficiently solving high-dimensional Poisson equations.
method Neural Walk-on-Spheres (NWoS) leveraging stochastic representations and Walk-on-Spheres methods.
result NWoS outperforms competing methods in accuracy, speed, and computational costs.
New neural network approach solves Poisson equations efficiently.
problem Approximating solutions to Poisson equations with Dirichlet boundary conditions.
method Using shallow ReLU α ^α α -networks to solve Laplace operator equations. result Neural networks can approximate solutions to the Laplace operator with Dirichlet boundary conditions efficiently.
For a bounded domain Ω Ω Ω with a piecewise smooth boundary in a complete Riemannian manifold M M M , we study eigenvalues of the Dirichlet eigenvalue problem of the Laplacian. By making use of a fact that eigenfunctions form an orthonormal basis of L 2 ( Ω ) L^2(Ω) L 2 ( Ω ) in place of the Rayleigh-Ritz formula, we obtain inequalities for …
Gradient descent achieves optimal learning for elliptic PDEs via Sobolev norms.
problem Learning elliptic PDEs from noisy data.
method Gradient descent on Sobolev norm objective functions.
result Gradient descent achieves statistical optimality for elliptic PDEs.
We produce a new proof and extend results by Harrell and Stubbe for the discrete spectrum of a self-adjoint operator. An abstract approach--based on commutator algebra, the Rayleigh-Ritz principle, and an ``optimal'' usage of the Cauchy-Schwarz inequality--is used to produce ``parameter-free'', ``projection-free'' vers…
Fast and accurate methods for low-rank learning problems.
problem Partial singular value decomposition and numerical rank estimation of huge matrices.
method Krylov subspaces and Ritz vectors for fast and accurate solutions.
result Advantages over traditional methods in accuracy and speed.
Stochastic volatility (SV) and local stochastic volatility (LSV) processes can be used to model the evolution of various financial variables such as FX rates, stock prices, and so on. Considerable efforts have been devoted to pricing derivatives written on underliers governed by such processes. Many issues remain, thou…
New algorithm updates eigenvectors of evolving graphs efficiently.
problem Updating eigenvectors of dynamic graphs.
method Subspace projection based on Rayleigh-Ritz projections.
result Strong performance in eigenvector approximation and downstream tasks.
Deep learning methods improve overlapping speaker separation across languages and noise.
problem Overlapping speaker separation in realistic scenarios.
method Deep clustering and deep attractor networks.
result Deep learning methods are effective for a broad range of languages and can handle untrained languages with common features.
Bayesian methods enhance deep learning models by improving reliability and uncertainty.
problem Improving reliability and uncertainty awareness in deep learning models.
method Approximate Bayesian inference techniques, including SG-MCMC and VI, applied to deep learning models.
result Enhanced posterior inference for deep learning models, particularly in neural networks and generative models.
Book introduces deep learning methods with math, theory, and applications.
problem Understanding deep learning algorithms and their mathematical foundations.
method Reviews various ANN architectures and optimization methods, covers theoretical aspects.
result Provides a solid mathematical foundation for deep learning.
The study compares shallow and deep learning methods for text classification.
problem High-dimensional, sparse data from customer calls.
method Comparative evaluation of shallow, deep, and ensemble methods.
result Ensemble methods improve classification accuracy.
Deep learning improves asset pricing and risk premium measurement.
problem Improving asset pricing and risk premium measurement using deep learning.
method Investigates various deep learning methods for asset pricing, especially for risk premia measurement.
result RNNs with memory mechanism and attention have the best performance in terms of predictivity.
Proposes a constraint for deep clustering to handle both simple and complex topologies.
problem Limited prior knowledge for deep clustering methods to perform well on complex topologies.
method Introduces a constraint using symmetric InfoNCE to enhance deep clustering performance.
result The constraint improves deep clustering methods' performance on both simple and complex topologies.
Deep learning improves time series forecasting, outperforming other methods.
problem Improving time series forecasting accuracy.
method Deep learning models for time series prediction.
result Deep learning models consistently outperform other methods in forecasting competitions.
New method tackles convergence issues in approximating FBSDEs.
problem Convergence issues in approximating coupled FBSDEs.
method Approximates initial condition for a family of FBSDEs, then uses it to approximate the original FBSDE.
result Method converges even when standard deep BSDE method fails.
This paper provides an overview of deep semi-supervised learning methods.
problem Reducing the need for large annotated datasets in deep learning.
method Summarizes dominant semi-supervised approaches in deep learning.
result Provides a comprehensive overview of deep semi-supervised learning.
Study develops efficient nested deep hedging method for derivatives pricing.
problem Hedging derivatives in market frictions using multiple options.
method Nested deep hedging approach with efficient learning techniques.
result Reduces arbitrage opportunities and improves hedging risks.
Research tackles safety of deep learning in safety-critical tasks.
problem Safety concerns of deep learning in perception tasks for autonomous agents.
method Technical enumeration and discussions on safety concerns and mitigation methods.
result Need for more mitigation methods to ensure safety of deep learning.
In recent years, deep learning methods applying unsupervised learning to train deep layers of neural networks have achieved remarkable results in numerous fields. In the past, many genetic algorithms based methods have been successfully applied to training neural networks. In this paper, we extend previous work and pro…
Hashing has been widely used for large-scale approximate nearest neighbor search because of its storage and search efficiency. Recent work has found that deep supervised hashing can significantly outperform non-deep supervised hashing in many applications. However, most existing deep supervised hashing methods adopt a …
Deep Penalty Method solves high-dimensional optimal stopping problems using deep learning.
problem High-dimensional optimal stopping problems in American option pricing.
method Inspired by penalty method for PDEs, approximates penalized PDE with Deep BSDE framework.
result Error bound of DPM is O ( 1 λ ) + O ( λ h ) + O ( h ) O(\frac{1}{\lambda}) + O(\lambda h) + O(\sqrt{h}) O ( λ 1 ) + O ( λh ) + O ( h ) . Deep learning methods improve subsurface flow modeling efficiency.
problem Efficiently modeling subsurface flow with uncertain parameters.
method Two categories of deep-learning based inverse modeling methods: surrogate-based and direct.
result Deep-learning methods significantly accelerate subsurface flow modeling.
Survey of deep learning methods on graphs.
problem Applying deep learning to graph data is challenging.
method Divided into five categories: graph recurrent neural networks, graph convolutional networks, graph autoencoders, graph reinforcement learning, and graph adversarial methods.
result Comprehensive review of deep learning methods on graphs.
New methods for geometric deep learning on manifolds.
problem Efficiently incorporating rotational effects and sampling on manifolds.
method Horizontal frame bundle flows and non-linear bridge sampling schemes.
result Efficient manifold convolution layers with weighted diffusion mean.
Deep learning improves crime prediction accuracy.
problem Improving crime prediction accuracy using deep learning.
method Comparative study of 10 deep learning methods on crime data.
result Deep learning methods outperform existing methods in crime prediction.
New deep learning method solves stochastic control problems.
problem Solving strongly coupled FBSDEs for stochastic control.
method Modified deep BSDE method with new loss function.
result Empirical convergence of the new method for three problems.
Survey of deep learning methods for forex and stock price prediction.
problem Improving accuracy and return in financial prediction.
method Classification of papers based on different deep learning methods.
result Recent models combining LSTM with other methods yield great returns and performances.
Deep-gKnock uses DNNs to select groups of features with improved interpretability.
problem Feature selection in high-dimensional data with grouping structure.
method Combines deep neural networks with Knockoffs technique for group-feature selection.
result Improves interpretability and accurate gFDR control compared to state-of-the-art methods.
This paper introduces Deep Incremental Boosting, a new technique derived from AdaBoost, specifically adapted to work with Deep Learning methods, that reduces the required training time and improves generalisation. We draw inspiration from Transfer of Learning approaches to reduce the start-up time to training each incr…
Deep Q-learning generates directed acyclic graphs.
problem Generating DAGs with specified structures.
method Deep reinforcement learning, specifically deep Q-learning.
result Demonstrated capability of generating DAGs in sparse reward environments.
Improves deep learning robustness by considering task and model.
problem Adversarial attacks on deep learning systems.
method Binary and interval label encoding strategy to redefine classification tasks and design corresponding loss functions.
result Our method enhances robustness without sacrificing accuracy.
Unified theory linking Bayesian and ensemble methods in deep learning.
problem Uncertainty quantification in deep learning.
method Reformulating optimisation as convex optimisation in probability measures, studying Wasserstein gradient flows.
result Unified theory explaining success of deep ensembles over variational inference.
SDF adapts Deep Forest for evolving data streams with active learning.
problem Adapting Deep Forest for evolving data streams.
method Streaming Deep Forest (SDF) with Augmented Variable Uncertainty (AVU) active learning.
result SDF with AVU outperforms other methods trained with all instances by 70% labeling budget.
New method protects sensitive data in deep learning training.
problem Protecting sensitive data in deep learning training.
method Distributed layer-partitioned training with step-wise activation functions.
result Experimental results show the method is simple and effective.
Bayesian Neural Networks help quantify uncertainty in deep learning predictions.
problem Uncertainty quantification in deep learning predictions.
method Bayesian statistics applied to neural networks.
result Design, implementation, training, and evaluation of Bayesian Neural Networks.
Deep density methods improve filtering in high-dimensional systems.
problem Nonlinear filtering in high-dimensional systems.
method Two deep density methods based on Feynman-Kac formulas and neural networks.
result Logarithmic deep backward stochastic differential equation filter outperforms classical methods in high dimensions.
Deep learning methods are reviewed for preserving structure in neural networks.
problem Challenges in applying deep learning, especially in preserving structure.
method Review of existing deep learning methods and new algorithmic frameworks.
result Mathematical understanding and systematic design of deep learning methods to preserve structure.
This paper highlights the need for explainable AI in video deep learning models.
problem Lack of explainable AI methods for video deep learning models.
method Illustrates the current state of video deep learning and highlights the need for explainability methods.
result Current explainability methods for video deep learning are insufficient and need improvement.