Deep neural networks are proven universally powerful using Koopman operator.
problem Proving the universality of deep neural networks.
method Formal deep network as a dual voice transform with Koopman operator, using group actions and Schur's lemma.
result Simple proof of the universality of DNNs.
The paper explores neural scaling laws for deep operator networks, offering a theoretical foundation.
problem Understanding neural scaling laws in deep operator networks.
method Theoretical analysis of approximation and generalization errors.
result Established a theoretical framework to quantify neural scaling laws for deep operator networks.
This paper studies neural network operators and their convergence properties.
problem Understanding the approximation and convergence of neural network operators.
method Proves density results, convergence estimates, and Voronovskaya-type theorems.
result Establishes quantitative convergence estimates and derives Voronovskaya-type theorems.
Develops a new deep learning formulation using Mori-Zwanzig formalism.
problem Improves deep learning by introducing a new concept of memory.
method Uses Mori-Zwanzig formalism to propagate quantities of interest through neural networks.
result Rigorously transforms deep networks into shallow ones using decay property of memory operator.
GIT-Net uses neural networks to approximate PDE operators efficiently.
problem Approximating PDE operators for complex geometries.
method Parametrizes adaptive generalized integral transforms with deep neural networks.
result GIT-Net outperforms existing neural network operators in multiple areas.
Explaining neural network computation in terms of probabilistic/fuzzy logical operations has attracted much attention due to its simplicity and high interpretability. Different choices of logical operators such as AND, OR and XOR give rise to another dimension for network optimization, and in this paper, we study the o…
Deep neural nets estimate operators between infinite-dimensional spaces with fast rates.
problem Estimating operators between infinite-dimensional spaces.
method Deep neural networks for nonparametric estimation of Lipschitz operators.
result Error bounds decay with fast rates depending on intrinsic dimension.
Adaptive weights improve physics-informed neural networks and deep operator networks.
problem Training physics-informed neural networks and deep operator networks can be challenging, leading to unsatisfactory accuracy and efficiency.
method Proposes a pointwise adaptive weighting method that balances the residual decay rate across different training points.
result Our proposed approach of balanced residual decay rates offers advantages including bounded weights, high prediction accuracy, fast convergence rate, low training uncertainty, low computational cost, and ease of hyperparameter tuning.
Paper proposes operator deep Q-learning for quick reward adaptation.
problem Standard RL can only handle one reward function and struggles with unseen rewards.
method Develops operator neural networks to map reward functions to value functions.
result Operator deep Q-learning can quickly adapt to new reward functions.
VIDON learns operators with variable sensors, overcoming sensor limitations.
problem Fixed sensor locations restrict operator learning applicability.
method Variable-Input Deep Operator Network (VIDON) with random, varying sensors.
result VIDON efficiently approximates operators in PDEs and is robust to sensor permutations.
NKN deep neural network learns governing equations and classifies images.
problem Learning governing equations and classifying images with deep neural networks.
method Nonlocal kernel network (NKN) that is resolution independent, deep, and handles various tasks.
result NKN outperforms baseline methods in learning governing equations and image classification tasks.
Deep neural networks solve noisy, complex problems accurately.
problem Reconstructing solutions from noisy, high-dimensional, non-linear inverse problems.
method Restricting infinite-dimensional forward operators to finite-dimensional spaces, training neural networks to approximate these operators robustly to noise.
result Deep neural networks can accurately solve high-dimensional, noisy, non-linear inverse problems.
New model outperforms Neural ODEs while being more efficient.
problem Stable convergence and existence guarantees for implicit-depth models.
method Developed Monotone Operator Equilibrium Network (monDEQ) based on monotone operator theory.
result MonDEQ models outperform Neural ODEs and are more computationally efficient.
RaNNDy uses randomized neural networks to learn transfer operators efficiently.
problem Efficiently learning transfer operators from data.
method Randomized neural network approach with randomly initialized hidden layers and trained output layer.
result Significant reduction in training time and resources with improved stability.
Business analytics refers to methods and practices that create value through data for individuals, firms, and organizations. This field is currently experiencing a radical shift due to the advent of deep learning: deep neural networks promise improvements in prediction performance as compared to models from traditional…
The paper introduces Causal Neural Operators to approximate operators in stochastic analysis.
problem Leveraging temporal structure in non-linear operators for deep learning models.
method Designing a deep learning model framework for infinite-dimensional linear metric spaces.
result Causal Neural Operators can uniformly approximate Hölder or smooth trace class operators.
New ADANNs improve PDE approximations.
problem Approximating operators for parametric PDEs.
method Custom ANN architectures and initialization schemes.
result ADANNs significantly outperform existing methods.
New method uses neural networks to solve high-dimensional eigenvalue problems.
problem Solving eigenvalue problems in high dimensions.
method Reformulates eigenvalue problem as fixed point problem of semigroup flow, approximated by neural networks.
result Accurate eigenvalue and eigenfunction approximations in various high-dimensional operators.
Novel neural operator predicts complex spatiotemporal dynamics from partial observations.
problem Capturing complex operator dynamics in infinite-dimensional function spaces.
method Integrates Koopman operator theory with deep neural networks to approximate nonlinear operators between Banach spaces.
result BNO achieves robust zero-shot super-resolution in unsteady flow prediction and outperforms conventional methods.
NON model improves tabular data classification accuracy.
problem Tabular data classification in real-world applications.
method Field-wise network, across field network, operation fusion network.
result NON significantly outperforms state-of-the-art models.
DeepSVM learns SVMs without PDE solving, achieving high pricing accuracy.
problem Computational bottleneck in real-time calibration of stochastic volatility models.
method Physics-informed Deep Operator Network (PI-DeepONet) that enforces terminal payoffs and no-arbitrage conditions.
result DeepSVM achieves high pricing accuracy across various market dynamics.
SON learns SPDE solutions and uncertainty from noisy data.
problem Uncertainty quantification in SPDEs with unknown model uncertainties.
method Combining DeepONet and SNNs, SON models stochasticity and predicts uncertainty.
result SON accurately captures solution structure and quantifies predictive uncertainty.
Deep learning framework for kernel methods using RKHM and Perron-Frobenius operators.
problem Kernel methods in deep learning with potential overfitting issues.
method Combining RKHM and Perron-Frobenius operator to derive a new Rademacher bound and analyze deep kernel methods.
result Theoretical interpretation of benign overfitting and milder dependency on output dimension.
HOPE uses Hilbert space to deconstruct deep network representations.
problem Deconstructing learned representations in deep networks is challenging.
method Introduces Hilbert Operator for Progressive Encoding (HOPE) to deconstruct network weights.
result HOPE provides an unbiased approach to network compression and fine-tuning.
Polynomial Chaos Expansion improves operator learning for PDEs.
problem Approximating mappings between infinite-dimensional functional spaces.
method Polynomial Chaos Expansion (PCE) for operator learning.
result PCE achieves strong performance in operator learning and uncertainty quantification.
New deep network derived from rate reduction principles, explaining features and efficiency.
problem Understanding and optimizing deep learning architectures.
method Gradient ascent scheme for rate reduction leading to multi-layer deep network.
result Explicitly constructed multi-layer network with precise optimization and interpretation.
New theory for local parameterization of deep ReLU networks.
problem Determining local parameters of deep ReLU neural networks.
method Introducing local lifting operators and charts of a manifold, deriving necessary and sufficient conditions for local identifiability.
result Sharp and testable conditions for local identifiability of deep ReLU networks.
Random feature method approximates operators with theoretical guarantees and reduced computation.
problem Approximating operators between infinite dimensional Banach spaces using machine learning.
method Random feature operator learning method with theoretical guarantees and error bounds.
result The random feature method can achieve similar or better test errors than kernel-based methods and neural networks with significantly reduced training times.
Deep neural networks are powerful learning models that achieve state-of-the-art performance on many computer vision, speech, and language processing tasks. In this paper, we study a fundamental question that arises when designing deep network architectures: Given a target network architecture can we design a smaller ne…
Framework explains how dual deep networks learn features from unlabeled data.
problem Understanding self-supervised learning with dual deep networks.
method Theoretical framework and hierarchical latent tree model.
result Deep ReLU networks learn latent variables through contrastive SSL.
Novel method for nowcasting implied volatility using neural operators.
problem Dynamic and spatially changing option prices in financial markets.
method Operator Deep Smoothing using graph neural operators.
result Highly accurate implied volatility smoothing on ten years of S&P 500 options data.
This paper aims to propose a novel deep learning-integrated framework for deriving reliable simulation input models through incorporating multi-source information. The framework sources and extracts multisource data generated from construction operations, which provides rich information for input modeling. The framewor…
New deep-unfolded network improves video background separation.
problem Video foreground-background separation.
method Deep unfolding of an iterative RPCA algorithm with adaptive learning.
result Proposed network outperforms state-of-the-art in video foreground-background separation.
This article introduces machine learning methods for solving PDEs.
problem Approximating solutions of partial differential equations.
method Machine learning methods, including physics-informed neural networks and deep operator learning.
result Recent advances in machine learning have made PDE solutions more accessible.
Bayesian Neural Networks help quantify uncertainty in deep learning predictions.
problem Uncertainty quantification in deep learning predictions.
method Bayesian statistics applied to neural networks.
result Design, implementation, training, and evaluation of Bayesian Neural Networks.
We consider the use of look-up tables (LUT) to simplify the hardware implementation of a deep learning network for inferencing after weights have been successfully trained. The use of LUT replaces the matrix multiply and add operations with a small number of LUTs and addition operations resulting in a completely multip…
Proposes SDCN to integrate structural information into deep clustering.
problem Lack of attention to structural information in representation learning for clustering.
method Designs a delivery operator to transfer autoencoder representations to GCN layers and uses a dual self-supervised mechanism.
result SDCN consistently outperforms state-of-the-art techniques in clustering tasks.
Random sampling improves DeepONet training efficiency without sacrificing accuracy.
problem Training DeepONet models with high computational and memory costs.
method Random sampling of inputs in the trunk network of DeepONet.
result Significant reduction in training time with comparable accuracy.
Deep neural operators learn complex probabilistic models efficiently.
problem Learning complex probabilistic models with global Lipschitz conditions.
method Deep neural-operator framework under global Lipschitz conditions.
result Explicit network-size bounds for universal approximation of probabilistic models.
Linear algebra approach for parallel deep learning models.
problem Training large DNNs in distributed environments.
method Linear algebraic approach to model parallelism.
result Manual development of backward operators for gradient-based training.
The paper explores stability and generalization of deep GCNs.
problem Understanding the stability and generalization of deep GCNs from a theoretical perspective.
method Theoretical analysis of stability and generalization properties of deep GCNs.
result The stability and generalization of deep GCNs are influenced by the maximum absolute eigenvalue of the graph filter operators and the depth of the network.
GMLP learns feature groups for tabular data without known structure.
problem Deep learning for tabular data with unknown feature interactions.
method Group-wise operations and sparse feature grouping matrix learned through temperature annealing softmax.
result GMLP achieves state-of-the-art classification performance on various datasets.
ReduNet optimizes data compression by maximizing rate reduction in deep networks.
problem Optimizing deep networks for high-dimensional multi-class data.
method Maximizing rate reduction through iterative gradient ascent, leading to a multi-layer deep network.
result ReduNet achieves optimal linear discriminative representation and is more efficient in the spectral domain.
User response prediction makes a crucial contribution to the rapid development of online advertising system and recommendation system. The importance of learning feature interactions has been emphasized by many works. Many deep models are proposed to automatically learn high-order feature interactions. Since most featu…
GNPs learn operators on non-Euclidean geometries using neural networks.
problem Learning operators on complex geometries like manifolds.
method Geometric Neural Operators (GNPs) that incorporate geometric properties.
result GNPs can estimate metrics, solve PDEs, and learn LB operators on manifolds.
dynoNet learns dynamical systems using linear operators.
problem Learning complex dynamical systems.
method dynoNet uses linear dynamical operators for sequence modeling and system identification.
result dynoNet effectively identifies systems on benchmarks.
DeepONet learns operators for PDEs with varying parameters and initial conditions.
problem Learning operators for partial differential equations with different parameters or initial conditions.
method DeepONet uses a Branch net and Trunk net to minimize error between evaluated and expected outputs, incorporating a scalar auxiliary variable approach for energy dissipation.
result DeepONet can accurately approximate operators for PDEs with varying parameters or initial conditions.
Generative networks are analyzed using spline operators to understand their properties and limitations.
problem Understanding and optimizing the properties of deep generative networks.
method Characterizing latent space partition, manifold dimension, and disentanglement using spline operators.
result Characterized the latent space partition, manifold dimension, and disentanglement of GDNs.