SON learns SPDE solutions and uncertainty from noisy data.
problem Uncertainty quantification in SPDEs with unknown model uncertainties.
method Combining DeepONet and SNNs, SON models stochasticity and predicts uncertainty.
result SON accurately captures solution structure and quantifies predictive uncertainty.
The paper introduces Causal Neural Operators to approximate operators in stochastic analysis.
problem Leveraging temporal structure in non-linear operators for deep learning models.
method Designing a deep learning model framework for infinite-dimensional linear metric spaces.
result Causal Neural Operators can uniformly approximate Hölder or smooth trace class operators.
Paper develops SINNOs for approximating stochastic processes.
problem Approximating stochastic processes with neural networks.
method Developed stochastic interpolation neural network operators (SINNOs) with random coefficients.
result Established boundedness, interpolation accuracy, and approximation capabilities of SINNOs.
Proposes a new method to learn operators for stochastic problems using DeepONet with autoencoder.
problem Efficiently solve forward and inverse stochastic problems with limited data.
method MultiAuto-DeepONet, a multi-resolution autoencoder DeepONet model.
result The model effectively handles high-dimensional stochastic inputs and reduces the number of trainable parameters.
Graph convolutional networks adapt the architecture of convolutional neural networks to learn rich representations of data supported on arbitrary graphs by replacing the convolution operations of convolutional neural networks with graph-dependent linear operations. However, these graph-dependent linear operations are d…
We model how Lipschitz continuity changes during neural network training.
problem Understanding how Lipschitz continuity evolves during training.
method We use a system of stochastic differential equations to capture the dynamics of Lipschitz continuity under SGD.
result We identify three factors driving the evolution of Lipschitz continuity: gradient flow projection, gradient noise, and Hessian projection.
NOVAS uses adaptive stochastic search for non-convex optimization in deep networks.
problem Non-convex optimization challenges in deep neural networks.
method Adaptive stochastic search for non-convex optimization.
result NOVAS outperforms existing alternatives in a structured prediction task.
New neural operators model turbulence with memory and randomness.
problem Modeling turbulence in complex fluid dynamics with memory and randomness.
method Symmetrized activation functions, fractional derivatives, and stochastic noise.
result Theoretical guarantees for approximation quality in turbulent phenomena.
GenUQ uses generative models to estimate uncertainty in operator learning.
problem Uncertainty quantification in stochastic operator models.
method Introduces a measure-theoretic approach with a generative hyper-network.
result Outperforms other UQ methods in various example problems.
DeepSVM learns SVMs without PDE solving, achieving high pricing accuracy.
problem Computational bottleneck in real-time calibration of stochastic volatility models.
method Physics-informed Deep Operator Network (PI-DeepONet) that enforces terminal payoffs and no-arbitrage conditions.
result DeepSVM achieves high pricing accuracy across various market dynamics.
New method uses neural networks to solve high-dimensional eigenvalue problems.
problem Solving eigenvalue problems in high dimensions.
method Reformulates eigenvalue problem as fixed point problem of semigroup flow, approximated by neural networks.
result Accurate eigenvalue and eigenfunction approximations in various high-dimensional operators.
NSMs use always-on stochasticity to normalize activations, improving convergence and performance.
problem Improving the robustness and generalizability of deep neural networks.
method Developed Neural Sampling Machines (NSMs) using always-on multiplicative stochasticity and simple threshold neurons.
result NSMs exhibit self-normalizing properties similar to Weight Normalization, speeding up convergence and preventing internal covariate shift.
Neural networks parameterize time-varying Markov dynamics in financial time series.
problem Estimating Markov transition matrices in high-resolution, high-noise financial data.
method Introduces a neural network framework to generate explicit, time-varying Markov transition matrices, constraining neural outputs to formal stochastic operators.
result Learned operators capture regime shifts, with high-volatility regimes homogenizing transition dynamics.
Efficient neural networks compute various differential operators cheaply.
problem Efficient computation of higher time complexity differential operators.
method Restricted neural network architectures with diagonal and hollow Jacobian matrices, allowing efficient extraction of dimension-wise derivatives.
result Demonstrated efficient computation of differential operators for various applications.
Low bit-width weights and activations are an effective way of combating the increasing need for both memory and compute power of Deep Neural Networks. In this work, we present a probabilistic training method for Neural Network with both binary weights and activations, called BLRNet. By embracing stochasticity during tr…
Stochastic approach improves neural network training for kinetic simulations.
problem Training neural networks under physical constraints in kinetic fusion simulations.
method Stochastic augmented Lagrangian approach using pyTorch.
result Higher model prediction accuracy achieved compared to fixed penalty method.
Neural operators solve families of 2BSDEs efficiently.
problem Solving infinite families of 2BSDEs on bounded domains.
method Introduces a mild generative neural operator model to approximate solutions.
result Solution operators can be approximated by neural operators with polynomial parameters.
sFML learns stochastic dynamical systems from data.
problem Learning unknown stochastic dynamical systems from measurement data.
method sFML extends FML for deterministic systems, using a stochastic flow map composed of deterministic and stochastic sub-maps.
result sFML constructs a stochastic evolution model approximating unknown stochastic systems.
A plethora of recent research has focused on improving the memory footprint and inference speed of deep networks by reducing the complexity of (i) numerical representations (for example, by deterministic or stochastic quantization) and (ii) arithmetic operations (for example, by binarization of weights). We propose a s…
Novel parametrized graph shift operators improve graph neural network performance.
problem Improving graph neural network performance on various datasets.
method Proposed a novel parametrized graph shift operator (PGSO) that optimizes parameters during training.
result PGSO improves accuracy in node and graph classification tasks on real-world datasets.
We present Spectral Inference Networks, a framework for learning eigenfunctions of linear operators by stochastic optimization. Spectral Inference Networks generalize Slow Feature Analysis to generic symmetric operators, and are closely related to Variational Monte Carlo methods from computational physics. As such, the…
Enhances RSCNs with hybrid regularization for nonlinear dynamics.
problem Modeling nonlinear dynamic systems with uncertainties.
method Recurrent stochastic configuration networks with hybrid regularization.
result The method outperforms other models in nonlinear system identification and industrial tasks.
Deep neural operators learn complex probabilistic models efficiently.
problem Learning complex probabilistic models with global Lipschitz conditions.
method Deep neural-operator framework under global Lipschitz conditions.
result Explicit network-size bounds for universal approximation of probabilistic models.
We propose a new generic type of stochastic neurons, called q-neurons, that considers activation functions based on Jackson's q-derivatives with stochastic parameters q. Our generalization of neural network architectures with q-neurons is shown to be both scalable and very easy to implement. We demonstrate expe…
RaNNDy uses randomized neural networks to learn transfer operators efficiently.
problem Efficiently learning transfer operators from data.
method Randomized neural network approach with randomly initialized hidden layers and trained output layer.
result Significant reduction in training time and resources with improved stability.
DeepONets enhance spatial-temporal surrogates for structural dynamics.
problem Creating full spatial-temporal surrogates for dynamical systems under uncertainty.
method Proposed Full-Field Extended DeepONet (FExD) to learn full solution operator across multiple degrees of freedom.
result FExD achieves superior accuracy and computational efficiency compared to other models.
Sorting input objects is an important step in many machine learning pipelines. However, the sorting operator is non-differentiable with respect to its inputs, which prohibits end-to-end gradient-based optimization. In this work, we propose NeuralSort, a general-purpose continuous relaxation of the output of the sorting…
New algorithm tackles unknown utility network resource allocation.
problem Maximizing network utility with unknown agent utilities.
method Modeling as a bandit problem, proposing algorithms for resource allocation.
result Proposed algorithms are optimal when all agents have the same utility.
Unified view of federated learning and distributed RL using local stochastic approximation.
problem Finding the root of an operator composed of local operators in a network of agents with dependent data.
method Local stochastic approximation over a network of agents with Markov process-dependent data.
result Convergence rates of local stochastic approximation for both constant and time-varying step sizes, within a logarithmic factor of independent data.
We introduce a simple and effective method for regularizing large convolutional neural networks. We replace the conventional deterministic pooling operations with a stochastic procedure, randomly picking the activation within each pooling region according to a multinomial distribution, given by the activities within th…
Neural SVEs model complex systems with memory, outperforming traditional methods.
problem Modeling systems with memory effects and irregular behavior.
method Introducing neural stochastic Volterra equations as a physics-inspired architecture.
result Neural SVEs outperform neural SDEs and DeepONets in various applications.
CoNBONet improves reliability analysis of complex systems with fast, energy-efficient predictions.
problem Time-dependent reliability analysis of nonlinear systems under stochastic excitations is computationally demanding.
method CoNBONet combines deep operator networks with neuroscience-inspired neuron models for fast, energy-efficient inference.
result CoNBONet provides reliable coverage of failure probabilities with theoretical guarantees.
A new method reduces the complexity of decentralized optimization.
problem Decentralized stochastic non-convex optimization over a network.
method GT-HSGD, a hybrid variance-reduced method.
result Achieves an oracle complexity of O(n^(-1)ε^(-3)) for small ε.
Generative Stochastic Networks (GSNs) have been recently introduced as an alternative to traditional probabilistic modeling: instead of parametrizing the data distribution directly, one parametrizes a transition operator for a Markov chain whose stationary distribution is an estimator of the data generating distributio…
New deep learning model robust to adversarial attacks using stochastic LWTA units.
problem Adversarial robustness in deep learning networks.
method Introduces deep networks with stochastic LWTA activations, combining them with Bayesian non-parametric tools.
result Achieves high robustness to adversarial perturbations, outperforming state-of-the-art methods.
The paper explores how different patterns of heterophily affect Graph Neural Networks.
problem Understanding the impact of heterophily on Graph Neural Networks.
method Theoretical analysis and experiments with Heterophilous Stochastic Block Models (HSBM).
result The impact of heterophily on classification depends on the Euclidean distance of neighborhood distributions and the averaged node degree.
One-pass SGD converges in overparametrized neural networks with random data.
problem Understanding convergence of SGD in neural networks with streaming data.
method Overparameterized two-layer neural networks, one-pass SGD, random initialization, NTK eigen-decomposition, VC dimension, McDiarmid's inequality.
result Prediction error converges in expectation under one-pass SGD in overparametrized neural networks.
Geometry arising from two diffusion operators (smooth semi-elliptic, second order differential operators) on different spaces but intertwined by a smooth map is described. Particular cases arise from Riemannian submersions when the operators are Laplace-Beltrami operators, from equivariant operators on the total space …
In this paper, we consider the stochastic iterative counterpart of the value iteration scheme wherein only noisy and possibly biased approximations of the Bellman operator are available. We call this counterpart as the approximate value iteration (AVI) scheme. Neural networks are often used as function approximators, i…
New neural processes use stacked Markov operators to improve flexibility.
problem Improving flexibility in neural processes.
method Stacking neural parameterized Markov transition operators in function space.
result MNPs outperform baseline models on various tasks.
Existing approaches to resource allocation for nowadays stochastic networks are challenged to meet fast convergence and tolerable delay requirements. The present paper leverages online learning advances to facilitate stochastic resource allocation tasks. By recognizing the central role of Lagrange multipliers, the unde…
A machine learning method for short-maturity options with jumps and stochastic volatility.
problem Short-maturity options with jumps and stochastic volatility.
method Differential machine learning method combining supervision and PIDE-residual penalty.
result Improves jump-term approximation and reduces Greeks errors compared to baselines.
Study variance-reduced method for estimating fixed points in Banach spaces.
problem Estimating fixed points of contractive operators in Banach spaces with noisy evaluations.
method Variance-reduced stochastic approximation scheme in Banach spaces.
result Establish non-asymptotic bounds for operator defect and estimation error.
A new stochastic primal--dual algorithm for solving a composite optimization problem is proposed. It is assumed that all the functions/operators that enter the optimization problem are given as statistical expectations. These expectations are unknown but revealed across time through i.i.d. realizations. The proposed al…
Power of network tests degrades when vertices are misaligned.
problem Power loss in network hypothesis testing due to vertex shuffling.
method Theoretical analysis and simulations of Frobenius norm differences in random dot product and stochastic block models.
result Shuffling vertices can significantly reduce the power of network tests.
Recursive stochastic algorithms have gained significant attention in the recent past due to data driven applications. Examples include stochastic gradient descent for solving large-scale optimization problems and empirical dynamic programming algorithms for solving Markov decision problems. These recursive stochastic a…
We propose a novel method to directly learn a stochastic transition operator whose repeated application provides generated samples. Traditional undirected graphical models approach this problem indirectly by learning a Markov chain model whose stationary distribution obeys detailed balance with respect to a parameteriz…
Stochastic neural ODEs outperform deterministic ones on image classification tasks.
problem Improving generalization in continuous-time models like neural ODEs.
method Empirical study of stochastically regularized neural ODEs using SDEs.
result Data augmentation negates the benefits of stochastic regularization, making neural ODEs and SDEs nearly equivalent.