Neural networks improve Bermudan option pricing accuracy.
problem Pricing Bermudan options with conditional expectation challenges.
method Neural network approximations of conditional expectations.
result Longstaff and Schwartz algorithm convergence with neural networks.
Symmetric functions learn better with strategic initial conditions.
problem Understanding how to improve learning efficiency for symmetric functions in neural networks.
method Investigates the impact of initial conditions on SGD training for symmetric functions in neural networks with one hidden layer.
result Strategic initial conditions can lead to efficient generalization guarantees for learning symmetric functions.
Paper develops neural network for Mandarin polyphone disambiguation.
problem Homograph problem in Mandarin Chinese text-to-speech.
method Bidirectional RNN for context, prediction network for mapping embeddings to pronunciations.
result Achieves 94.69% accuracy on polyphonic character dataset.
Study shows depth improves trainability of neural networks by improving kernel conditioning.
problem Improving trainability of neural networks with random initialization and overparameterization.
method Analyzes the role of depth in training neural networks, proving that depth improves conditioning of kernel matrices.
result General result showing depth improves trainability of neural networks by improving the conditioning of kernel matrices.
Method learns conditional distributions using neural entropic optimal transport.
problem Challenges in learning multiple conditional distributions.
method Neural entropic optimal transport method with two networks and regularization.
result Effective learning of conditional distributions with limited samples.
New theory for local parameterization of deep ReLU networks.
problem Determining local parameters of deep ReLU neural networks.
method Introducing local lifting operators and charts of a manifold, deriving necessary and sufficient conditions for local identifiability.
result Sharp and testable conditions for local identifiability of deep ReLU networks.
NCP uses neural networks to efficiently learn conditional distributions.
problem Learning conditional distributions for statistical inference.
method Neural Conditional Probability (NCP) approach.
result NCP efficiently handles complex probability distributions and matches leading methods.
A neural network derived from first principles using MaxEnt.
problem Developing a neural network from first principles.
method Derived a neural network using the principle of Maximum Entropy, with linear dimension-reducing transformations and conditional mean estimators.
result Unified theoretical justification for activation functions like sigmoid, softplus, and relu.
New metric predicts neural network reliability under novel conditions.
problem Verifying neural networks' safety in novel scenarios.
method ML Dependability metric, Task Undependability, Harmful Undependability.
result Accurately predicts reliability under novel conditions.
Neural network based architectures used for sound recognition are usually adapted from other application domains, which may not harness sound related properties. The ConditionaL Neural Network (CLNN) is designed to consider the relational properties across frames in a temporal signal, and its extension the Masked Condi…
Softmax emerges naturally in neural networks as a measure of conditional mutual information.
problem The artificial nature of softmax in neural networks.
method Information-theoretic perspective to derive log-softmax and evaluate conditional mutual information.
result Training deterministic neural networks through log-softmax maximises conditional mutual information.
We study the error landscape of deep linear and nonlinear neural networks with the squared error loss. Minimizing the loss of a deep linear neural network is a nonconvex problem, and despite recent progress, our understanding of this loss surface is still incomplete. For deep linear networks, we present necessary and s…
Estimates conditional distribution function using neural networks for censored and uncensored data.
problem Estimating conditional distribution function for censored and uncensored data.
method Neural network algorithm based on Cox regression with time-dependent covariates, using full likelihood with unconstrained optimization.
result Proposed method yields more accurate estimates than existing methods when model assumptions are violated.
The study characterizes the conditioning of the Gauss-Newton matrix in neural networks.
problem Understanding the conditioning of the Gauss-Newton matrix in neural networks.
method Theoretical analysis of the GN matrix in deep linear and ReLU networks, extending to residual connections and convolutional layers.
result Established tight bounds on the condition number of the GN matrix in neural networks.
The ConditionaL Neural Networks (CLNN) and the Masked ConditionaL Neural Networks (MCLNN) exploit the nature of multi-dimensional temporal signals. The CLNN captures the conditional temporal influence between the frames in a window and the mask in the MCLNN enforces a systematic sparseness that follows a filterbank-lik…
Recently, Neural networks have seen a huge surge in its adoption due to their ability to provide high accuracy on various tasks. On the other hand, the existence of adversarial examples have raised suspicions regarding the generalization capabilities of neural networks. In this work, we focus on the weight matrix learn…
Neural IVP solves IVPs with neural networks, overcoming scaling and conditioning issues.
problem Solving initial value PDEs with neural networks is challenging due to numerical errors and limited scalability.
method Developed an ODE-based approach to solve IVPs with neural networks, preventing ill-conditioning and scaling issues.
result Neural IVP solves challenging PDEs with neural networks efficiently and accurately.
Unified neural network for linear and nonlinear dimension reduction.
problem Efficiently perform linear and nonlinear sufficient dimension reduction.
method Belted and Ensembled Neural Network (BENN) framework.
result Unified framework for both linear and nonlinear dimension reduction.
SGD converges with positive probability for non-convex deep neural networks under specific conditions.
problem Convergence of SGD for non-convex deep neural networks.
method Established local convergence with positive probability under local Łojasiewicz condition and additional structural assumption.
result SGD converges with positive probability for non-convex deep neural networks under specific conditions.
The paper proves neural network identifiability for a broad range of nonlinearities.
problem Can a neural network's architecture, weights, and biases be uniquely determined by its input-output map?
method Derive necessary genericity conditions for identifiability of neural networks of arbitrary depth and connectivity with an arbitrary nonlinearity.
result Construct a family of nonlinearities for which these genericity conditions are minimal, necessary, and sufficient.
Fixed points of nonnegative neural networks are analyzed using fixed point theory.
problem Analyzing fixed points in nonnegative neural networks.
method Fixed point theory, nonlinear Perron-Frobenius theory, monotonic and scalable mappings.
result Conditions for the existence of fixed points in nonnegative neural networks are provided.
StrNN uses neural network structures to learn conditional independencies.
problem Learning conditional independencies in neural networks.
method Designing masks for neural networks based on binary matrix factorization.
result StrNN improves density estimation and causal inference.
Deep neural networks' Jacobian spectrum becomes well-conditioned with orthogonal weights.
problem Understanding and handling the Jacobian spectrum of deep neural networks.
method Applying free probability theory to show almost sure asymptotic freeness of Jacobians in the wide limit.
result Layer-wise Jacobians of deep neural networks with orthogonal weights are almost surely asymptotically free.
Meta-learning neural networks to solve diverse PDEs efficiently.
problem Efficiently solving new PDE problems with minimal training.
method Neural network meta-learning of PDE problem representations.
result Meta-learned neural networks predict PDE solutions with high accuracy.
We compress large neural networks for quick adaptation to specific contexts.
problem How to quickly adapt a pretrained large neural network to specific contexts.
method Propose a Bayesian hypernetwork framework to compress the network and encourage sparsity.
result Generated compressed networks are significantly smaller than baseline methods.
We present the ConditionaL Neural Network (CLNN) and the Masked ConditionaL Neural Network (MCLNN) designed for temporal signal recognition. The CLNN takes into consideration the temporal nature of the sound signal and the MCLNN extends upon the CLNN through a binary mask to preserve the spatial locality of the feature…
New method solves PDEs for any initial condition without retraining.
problem Solving PDEs for different initial conditions requires retraining neural solvers.
method Formulate solution as conditional probability distribution.
result Approximates PDE solution for arbitrary initial conditions.
A neural network approach for efficient conditional SHAP calculations.
problem Efficiently calculating conditional SHAP values for various models.
method Surrogate neural network approach for conditional SHAP.
result Efficiently calculates conditional SHAP values for neural networks and other regression models.
Generative neural network designs novel 3D molecules with specified properties.
problem Designing molecules with desired properties in chemistry.
method Conditional generative neural network for 3D molecular structures.
result Demonstrated utility in generating novel molecules with specified motifs or composition.
A study on preventing catastrophic forgetting in neural networks using conditional computation.
problem Catastrophic forgetting in connectionist neural networks.
method Conditional computation framework where parameters are conditioned on each input example.
result Proposed conditional rehearsal to prevent forgetting of previously learned examples.
Enhances neural network solvers for PDEs with complex boundary conditions.
problem Challenges in solving PDEs with high accuracy and complex boundary conditions.
method Integrates natural gradient optimization with numerical time-stepping schemes to enforce Dirichlet boundary conditions.
result Superior accuracy and computational efficiency of the proposed methods for solving PDEs.
Deep neural network architectures designed for application domains other than sound, especially image recognition, may not optimally harness the time-frequency representation when adapted to the sound recognition problem. In this work, we explore the ConditionaL Neural Network (CLNN) and the Masked ConditionaL Neural N…
Simplified neural network EFTs reveal a single critical condition.
problem Understanding neuron statistics in neural networks at initialization.
method Diagrammatic approach to effective field theories (EFTs).
result A single condition governs criticality of all neuron preactivations.
Neural GARCH models financial time series with time-varying coefficients.
problem Modeling conditional heteroskedasticity in financial time series.
method Neural network adaptation of GARCH and BEKK models with time-varying coefficients parameterized by a recurrent neural network.
result Neural Students t model consistently outperforms other models on financial time series.
Deep learning techniques are increasingly being considered for geological applications where -- much like in computer vision -- the challenges are characterized by high-dimensional spatial data dominated by multipoint statistics. In particular, a novel technique called generative adversarial networks has been recently …
DGNN predicts financial margin calls under stress tests.
problem Forecasting margin calls in dynamic financial networks.
method Dynamic Graph Neural Network (DGNN) architecture.
result DGNN produces accurate forecasts up to 21 days.
The past decade has witnessed a successful application of deep learning to solving many challenging problems in machine learning and artificial intelligence. However, the loss functions of deep neural networks (especially nonlinear networks) are still far from being well understood from a theoretical aspect. In this pa…
Automatically designs neural network structure using matrix conditioning.
problem Choosing an effective size and structure of neural networks for new datasets is time-consuming.
method Adjusts neuron proportions and scales network size using matrix conditioning.
result Small networks achieve high accuracy on various datasets.
HCNAF models complex conditional distributions for probabilistic occupancy forecasting.
problem Modeling complex conditional probability density functions for occupancy forecasting.
method Hyper-Conditioned Neural Autoregressive Flow (HCNAF) combining AF and hyper-network.
result HCNAF achieves state-of-the-art performance in self-driving datasets.
Neural network based architectures used for sound recognition are usually adapted from other application domains such as image recognition, which may not harness the time-frequency representation of a signal. The ConditionaL Neural Networks (CLNN) and its extension the Masked ConditionaL Neural Networks (MCLNN) are des…
Study on deep neural networks for reward modeling with pairwise comparison data.
problem Reward modeling with deep neural networks in non-parametric settings.
method Established a non-asymptotic regret bound for deep reward estimators, introduced a margin-type condition.
result Improved regret bound for deep reward estimators, highlighting the importance of clear human beliefs.
The study characterizes conditions for trainability and generalization in deep neural networks.
problem Understanding the conditions for deep neural networks to be trainable and generalize well.
method Analysis of Neural Tangent Kernel (NTK) for wide and deep networks.
result Large regions of hyperparameter space exist where networks can memorize training data but fail to generalize.
Graphs predict reaction conditions for organic chemistry.
problem Predicting specific reaction conditions in organic chemistry.
method Graph Neural Networks (GNNs) for modeling reaction graphs.
result GNNs can identify specific graph features affecting reaction conditions.
Two-layer CNNs can overfit without harm under certain conditions.
problem Understanding when and how overfitting occurs in neural networks.
method Theoretical analysis of a two-layer CNN trained by gradient descent.
result A sharp phase transition between benign and harmful overfitting based on signal-to-noise ratio.
New neural network approach solves Poisson equations efficiently.
problem Approximating solutions to Poisson equations with Dirichlet boundary conditions.
method Using shallow ReLUα-networks to solve Laplace operator equations. result Neural networks can approximate solutions to the Laplace operator with Dirichlet boundary conditions efficiently.
Improves conditional distribution modeling with simpler training.
problem Analyzing inverse problems with invertible neural networks.
method Uses normalizing flows to maximize posterior likelihood, incorporating conditioning.
result Easier training and natural framework for conditional generation.
Automatic feature extraction using neural networks has accomplished remarkable success for images, but for sound recognition, these models are usually modified to fit the nature of the multi-dimensional temporal representation of the audio signal in spectrograms. This may not efficiently harness the time-frequency repr…
Neural networks can approximate high-dimensional classifiers with ReLU networks under margin conditions.
problem Approximating high-dimensional discontinuous classifiers with neural networks.
method Using ReLU neural networks with three hidden layers, approximating a classifier with a Barron-regular decision boundary.
result High-dimensional discontinuous classifiers can be approximated with a rate of n−1 under strong margin conditions.