Stochastic neural networks can approximate any function, even with correlated outputs.
problem Approximating functions with stochastic outputs and correlations.
method Investigating deep sigmoid belief networks to approximate any stochastic mapping.
result Minimal number of layers and units needed for approximation.
A new method quantifies input model uncertainty in streaming data.
problem Quantifying input model uncertainty in streaming data.
method Two-layer importance sampling framework for online uncertainty quantification.
result Consistency and asymptotic convergence rate of the proposed algorithms.
The article develops models for learning and controlling physical systems with unknown inputs.
problem Learning and controlling physical systems with unknown inputs.
method Gaussian process latent force models (GP-LFMs) combining first-principles models and non-parametric GP components.
result Theoretical observability and controllability results for GP-LFMs.
Framework combines generative and predictive models for input design.
problem Maximizing or achieving specified values of properties given stochastic oracles.
method Probabilistic modeling and adaptive sampling algorithm.
result Substantially outperforms other methods in experimental tests.
ISAAC Newton uses input-based curvature for efficient training.
problem Efficient training in small-batch stochastic regimes.
method ISAAC Newton conditions gradients using selected second-order information based on input.
result Effective training even in small-batch stochastic regimes, competitive to first-order and second-order methods.
Paper derives an error bound for stochastic LTI systems.
problem Stochastic LTI systems with inputs in control engineering and econometrics.
method PAC-Bayesian-Like error bound derivation.
result Derived an error bound for stochastic LTI systems.
MF-GLaM models improve stochastic simulator emulation with multifidelity data.
problem Challenging to emulate stochastic simulators' full conditional probability distribution.
method Proposes MF-GLaMs to efficiently emulate HF stochastic simulators using LF data.
result MF-GLaMs achieve improved accuracy or comparable performance at reduced cost.
A method constructs a stochastic surrogate from dimensionality reduction results for high-dimensional uncertainty quantification.
problem High-dimensional uncertainty quantification with physics-based models.
method Constructs a stochastic surrogate model from dimensionality reduction results.
result Preserves convenience of sequential dimensionality reduction and Gaussian process regression while overcoming limitations.
Bayesian approach improves semi-supervised learning with deep generative models.
problem Lack of model uncertainty and flexibility in existing semi-supervised learning methods.
method Proposes a discriminative component with stochastic inputs and extends it to be fully Bayesian.
result Improved handling of model uncertainty and flexibility in capturing complex patterns.
Develops a new approach to optimal control of stochastic systems.
problem Optimal control of stochastic nonlinear dynamical systems is challenging.
method Formulates optimal control as input estimation, using probabilistic inference and Expectation Maximization.
result Extracts time-varying linear Gaussian feedback controllers from the joint state-action distribution.
The paper develops methods to analyze sensitivity in stochastic models using surrogate models.
problem Quantifying the impact of input variability on stochastic simulators with randomness.
method The authors propose using generalized lambda models to emulate response distributions of stochastic simulators and estimate sensitivity indices.
result The proposed method can estimate sensitivity indices even with strong heteroskedasticity and small signal-to-noise ratio.
Generative adversarial networks improve stochastic input parametrization in subsurface flow simulations.
problem Effective parametrization of high-dimensional, correlated stochastic inputs in subsurface flow simulations.
method Training a generative adversarial network to emulate the data generating process of stochastic inputs.
result Generative adversarial networks preserve both visual realism and high-order statistics of flow responses, achieving a significant dimensionality reduction.
When simulating a complex stochastic system, the behavior of output response depends on input parameters estimated from finite real-world data, and the finiteness of data brings input uncertainty into the system. The quantification of the impact of input uncertainty on output response has been extensively studied. Most…
Deep learning for stochastic systems with multi-fidelity data.
problem Predicting stochastic, high-dimensional, and multi-fidelity systems with uncertainty.
method Probabilistic deep learning with variational inference for implicit distributions.
result Effective surrogate models for stochastic systems with quantified uncertainty.
Many random processes can be simulated as the output of a deterministic model accepting random inputs. Such a model usually describes a complex mathematical or physical stochastic system and the randomness is introduced in the input variables of the model. When the statistics of the output event are known, these input …
We introduce and analyze stochastic optimization methods where the input to each gradient update is perturbed by bounded noise. We show that this framework forms the basis of a unified approach to analyze asynchronous implementations of stochastic optimization algorithms.In this framework, asynchronous stochastic optim…
Proposes a new method to selectively access privileged information in reinforcement learning.
problem Selective compression of privileged information in reinforcement learning.
method Formulates a variational bandwidth bottleneck to decide stochastically whether to access privileged information.
result Improves generalization and reduces access to costly information in reinforcement learning experiments.
The paper analyzes how gradient descent learns convolutional filters for non-Gaussian inputs.
problem Learning convolutional filters with ReLU for non-Gaussian input distributions.
method Analysis of gradient descent convergence for ReLU activation with polynomial time complexity.
result Gradient descent can learn convolutional filters in polynomial time, with convergence rate dependent on input distribution smoothness and patch similarity.
Deep learning models converge to Gaussian dynamics with mixed structured inputs.
problem Understanding neural network dynamics with complex input distributions.
method Extended hidden manifold model to Gaussian mixtures, analyzed via SGD.
result Learning dynamics with mixed inputs converge to Gaussian behavior.
Paper introduces a new model to handle multi-task learning across different input domains.
problem Learning correlated tasks across varying input domains.
method Develops a novel heterogeneous stochastic variational linear model of coregionalization (HSVLMC) for multi-task learning.
result The proposed model outperforms existing models in diverse multi-task scenarios.
Paper proposes using generalized lambda distributions for stochastic simulators.
problem Uncertainty quantification with complex stochastic models is computationally challenging.
method Flexible generalized lambda distribution approximates response PDF, parameters are sparse polynomial chaos expansions.
result Local inference of response PDF at each point of experimental design using replicated model evaluations.
Bayesian optimization improves performance with common random numbers.
problem Optimizing expensive stochastic functions with common random numbers.
method Proposes a novel Gaussian process model and Knowledge Gradient for Common Random Numbers.
result Significant performance improvements with moderate computational cost.
Agent learns optimal control inputs for plants with unknown parameters.
problem Optimal control for systems with unknown and changing parameters.
method Personalized control inputs based on stochastic dynamics.
result Demonstrated effectiveness on simulated system.
This paper generalizes SSD for Gaussian RBMs, improving convergence for continuous data.
problem Improving convergence of RBMs with Gaussian inputs using SSD.
method Deriving upper bounds of logarithmic partition function for RBMs via Schatten-infinity norm.
result Empirical improvement of SSD over SGD for Gaussian RBMs.
A new SVRG variant reduces variance with less computation.
problem High variance in stochastic gradient descent hinders convergence.
method CheapSVRG: Uses a surrogate full-gradient on a subset of data.
result Achieves linear convergence rate with complexity trade-off.
A novel method uses GPLFMs for joint input-state estimation in linear structural systems.
problem Combined state and input estimation of linear structural systems.
method Gaussian process latent force models (GPLFMs) combined with Kalman filters.
result GPLFMs outperform conventional Kalman filters in state and input estimation.
A new deep learning method using differential flows and Gaussian processes.
problem Improving deep learning performance through more flexible transformations.
method Differential flows and Gaussian processes to learn stochastic differential equations.
result Exceeds performance of deep Gaussian processes and neural networks.
New method estimates probability of simulator output exceeding threshold.
problem Estimate probability of simulator output exceeding critical threshold.
method Bayesian framework, Gaussian process model, MSUR strategy.
result MSUR strategy selects optimal inputs and fidelity levels.
Nonlinear RNNs' memory capacity varies widely, making it impractical.
problem The usefulness of memory capacity as a metric for linear RNNs is questioned.
method Analysis of random nonlinear RNNs with varying input scales.
result Memory capacity of nonlinear RNNs is arbitrary and impractical.
Study robust control for systems with continuous states using adversarial perturbations.
problem Fragile policies in Markov control models under internal or external perturbations.
method Distributionally robust stochastic control with adaptive adversarial perturbations.
result Optimal robust policies for continuous state systems with uniform learning guarantees.
New framework models neural systems with random architecture on manifolds.
problem Complex, uncertain systems with non-Gaussian outputs.
method Latent random field on compact manifold generates neural architecture and weights.
result Synthetic neural systems can produce stochastic outputs for deterministic inputs.
The paper develops algorithms for computing in stochastic acceptors.
problem Computing input marginals and acceptance probabilities in stochastic acceptors.
method Dynamic programming for input marginals, EM for parameter estimation.
result Efficient algorithms for computing in stochastic acceptors.
Proposes a new method to learn operators for stochastic problems using DeepONet with autoencoder.
problem Efficiently solve forward and inverse stochastic problems with limited data.
method MultiAuto-DeepONet, a multi-resolution autoencoder DeepONet model.
result The model effectively handles high-dimensional stochastic inputs and reduces the number of trainable parameters.
Neural networks learn faster with correlated latent variables.
problem Efficiently learning from higher-order correlations in neural networks.
method Analytical derivation and simulations of two-layer neural networks.
result Correlations between latent variables speed up learning from higher-order correlations.
A new method lifts training of input-convex neural networks to avoid dead weights and plateaued loss.
problem Training input-convex neural networks with non-negative weights.
method Introduces a hypernetwork that emits non-negative weights from a summary of the input batch, adding stochasticity to soften the loss landscape.
result The lift method achieves lower test loss than projected gradient descent and direct softplus reparametrization.
Vanilla SGD learns SIM from anisotropic data without explicit covariance estimation.
problem Learning SIM from anisotropic Gaussian inputs.
method Vanilla Stochastic Gradient Descent (SGD) trained on SIM with anisotropic input.
result Vanilla SGD adapts to anisotropic data's covariance structure.
Expands learning paradigm to stochastic orders using Choquet-Toland distance and Variational Dominance Criterion.
problem Learning high-dimensional distributions with stochastic orders.
method Introduces Choquet-Toland distance and Variational Dominance Criterion, uses input convex maxout networks (ICMNs).
result Proposes surrogates for Choquet-Toland distance and Variational Dominance Criterion with parametric rates.
We present a method to quantify uncertainty in the predictions made by simulations of mathematical models that can be applied to a broad class of stochastic, discrete, and differential equation models. Quantifying uncertainty is crucial for determining how accurate the model predictions are and identifying which input …
Paper studies t-SNE convergence with generalized kernels.
problem Understanding convergence of t-SNE with generalized kernels.
method Concrete formulation of generalized kernels, proving convergence to an equilibrium distribution.
result t-SNE converges to an equilibrium distribution under certain conditions for generalized kernels.
We reduce variance in RL with input-dependent baselines.
problem High variance in RL with standard baselines in input-driven environments.
method Derive and use a bias-free, input-dependent baseline; propose a meta-learning approach.
result Input-dependent baselines improve training stability and policy quality.
Study on estimating unstable open-loop matrices from state trajectories.
problem System identification for stochastic continuous-time dynamics.
method Employing randomized control inputs to estimate unstable open-loop matrix.
result Estimation error decays with trajectory length, signal-to-noise ratio, and excitability.
New method optimizes protein design by sampling from realistic inputs.
problem Optimizing properties of interest in design problems, especially with black box predictive models.
method Conditioning by Adaptive Sampling, using model-based adaptive sampling to estimate conditional input distributions.
result Achieves state-of-the-art results on protein fluorescence problem.
Paper introduces a new method to create adversarial examples against gradient-obfuscating defenses.
problem Crafting adversarial examples to fool gradient-obfuscating defenses.
method Stochastic Substitute Training (SST), a gray-box approach.
result Adversaries can create adversarial examples without knowledge or limited information about the defense.
Paper tackles active labeling for partial supervision.
problem Accessing stochastic gradients with partial supervision.
method Streaming technique to minimize generalization error.
result Proves minimization of generalization error ratio.
Stochastic gradient descent learns weights of state equations with nonlinear activations.
problem Learning weights of state equations with nonlinear activations using SGD.
method Utilizes stochastic gradient descent to learn weight matrices from input/state trajectories.
result SGD converges to ground truth weights with near-optimal sample size and linear convergence.
GANs improve stochastic parameterization of the Lorenz '96 model.
problem Improving stochastic parameterizations for sub-grid processes.
method Developed a GAN-based stochastic parameterization for the Lorenz '96 model.
result GAN configurations outperform a bespoke parameterization in skillful forecasts and climate simulations.
A new method for high-dimensional RBDO using stochastic emulators.
problem Efficient RBDO in high-dimensional settings.
method Unified stochastic representation, stochastic emulators, deterministic mapping.
result Significant computational gains in high-dimensional settings.
The paper quantifies and attributes uncertainty in complex system simulations.
problem Uncertainty in complex system simulations due to unknown or approximated subprocesses.
method Developed a framework for quantifying and attributing submodel uncertainty using bootstrapping, Bayesian model averaging, and tree-based methods.
result Individual submodels contribute to overall uncertainty, and their importance can be quantified.