We improve optimization for data with varying variance.
problem Optimizing data with varying variance.
method Generalized learning and optimization frameworks for data-driven optimization.
result Asymptotic and finite sample guarantees for stochastic programs.
A new deep generative model uses BSDEs for high-dimensional data generation.
problem Generating high-dimensional complex data, especially images.
method Combines BSDEs with deep neural networks for training with MMD loss.
result BSDE-Gen effectively generates high-dimensional data with stochasticity.
sFML learns stochastic dynamical systems from data.
problem Learning unknown stochastic dynamical systems from measurement data.
method sFML extends FML for deterministic systems, using a stochastic flow map composed of deterministic and stochastic sub-maps.
result sFML constructs a stochastic evolution model approximating unknown stochastic systems.
GenFormer uses deep learning to generate complex stochastic data.
problem Creating synthetic stochastic data that matches real-world statistical properties.
method Transformer-based deep learning model that maps Markov state sequences to time series values.
result GenFormer preserves target marginal distributions and other statistical properties in multivariate spatio-temporal data.
New bounds show linear predictors rarely overfit with certain optimization methods.
problem Bounding test error for linear predictors with stochastic optimization methods.
method Coupling argument for fixed point methods like stochastic and batch mirror descent.
result Locally-adapted rates that depend on predictor properties, not global problem structure.
We marry ideas from deep neural networks and approximate Bayesian inference to derive a generalised class of deep, directed generative models, endowed with a new algorithm for scalable inference and learning. Our algorithm introduces a recognition model to represent approximate posterior distributions, and that acts as…
Method learns dynamics of slow variables from stochastic data.
problem Modeling unknown multiscale stochastic systems with limited data.
method Data-driven approach to learn effective dynamics from bursts of observation data.
result Generative model accurately captures effective dynamics of slow variables.
The paper studies a stochastic majority vote approach to improve classifier accuracy.
problem Improving classifier accuracy over ensembles of classifiers.
method Minimizing a PAC-Bayes generalization bound with Dirichlet distributions.
result Achieves state-of-the-art accuracy and tight generalization bounds.
How to model distribution of sequential data, including but not limited to speech and human motions, is an important ongoing research problem. It has been demonstrated that model capacity can be significantly enhanced by introducing stochastic latent variables in the hidden states of recurrent neural networks. Simultan…
New method recovers BSDE from financial data without ergodicity.
problem Discovering probabilistic laws from financial data.
method Stochastic SINDy method under risk-neutral measure.
result Recovery of BSDE from limited financial data.
Improved noise estimation in latent neural SDEs enhances model accuracy.
problem Latent neural SDEs underestimate noise, limiting their stochastic dynamics modeling.
method Explicit additional noise regularization in the loss function.
result Model accurately captures diffusion component of stochastic time series data.
Improved convergence for nonconvex optimization with dependent data.
problem Constrained smooth nonconvex optimization with dependent data.
method Stochastic projected gradient methods under a general dependent data sampling scheme.
result Achieved worst-case rate of convergence ildeO(t−1/4) and complexity ildeO(ε−4). Generative models for complex stochastic dynamics using adversarial learning.
problem Data-driven modeling of multistep stochastic dynamics.
method Adversarial learning with GANs and MMD for stable model classes.
result Stable generative models for long-time prediction and stochastic systems.
Paper tackles robust model training with a new stochastic algorithm.
problem Training robust models against data distribution shift.
method Derives a novel dual formulation and proposes a nested stochastic gradient descent algorithm.
result Establishes polynomial iteration and sample complexities for large-scale DRO problems.
A new framework models uncertainty in structured temporal data using SDEs and neural networks.
problem Uncertainty quantification in machine learning applications involving structured and temporal data.
method Integrates stochastic differential equations (SDEs) with deep generative models in a variational autoencoder framework.
result Improves uncertainty quantification in machine learning applications involving structured and temporal data.
Stochastic neural ODEs outperform deterministic ones on image classification tasks.
problem Improving generalization in continuous-time models like neural ODEs.
method Empirical study of stochastically regularized neural ODEs using SDEs.
result Data augmentation negates the benefits of stochastic regularization, making neural ODEs and SDEs nearly equivalent.
Optimizes pruning masks for neural networks using probabilistic fine-tuning and PAC-Bayes bounds.
problem Improving neural network performance through adaptive pruning of weights.
method Optimizes stochastic pruning masks by minimizing expected loss, considering data-adaptive regularization and feature alignment.
result Probabilistic fine-tuning leads to improved test error over baseline methods in neural networks.
Generative LLE modifies LLE to generate stochastic embeddings.
problem Nonlinear dimensionality reduction and manifold learning.
method Generative LLE modifies LLE by using stochastic linear reconstruction.
result Generative LLE can generate various LLE embeddings stochastically.
New bounds link generalization to stochastic optimizer's lower tail exponents.
problem Understanding the impact of stochastic optimization algorithms on generalization in non-convex settings.
method Proves novel bounds linking generalization to the lower tail exponent of the transition kernel of stochastic optimizers, both discrete- and continuous-time.
result Empirical results show correlations between generalization error and lower tail exponents.
Stochastic variational inference for collapsed models has recently been successfully applied to large scale topic modelling. In this paper, we propose a stochastic collapsed variational inference algorithm in the sequential data setting. Our algorithm is applicable to both finite hidden Markov models and hierarchical D…
Improved algorithm reduces stochastic gradient complexity for large-scale learning problems.
problem High stochastic gradient complexity for large-scale learning problems.
method Hybrid Stochastic-Deterministic Minibatch Proximal Gradient (HSDMPG) algorithm.
result Achieves nearly optimal generalization in less than a single pass over data.
MF-GLaM models improve stochastic simulator emulation with multifidelity data.
problem Challenging to emulate stochastic simulators' full conditional probability distribution.
method Proposes MF-GLaMs to efficiently emulate HF stochastic simulators using LF data.
result MF-GLaMs achieve improved accuracy or comparable performance at reduced cost.
SORSCNs improve nonstationary data modeling by self-organizing and adjusting network parameters.
problem Nonstationary data challenges traditional models in continuous learning.
method SORSCNs autonomously adjust network parameters and structure in real-time using adaptive algorithms.
result SORSCNs outperform other models in generalizing to nonstationary data.
Generative model improves noise estimation in stochastic rotating shallow water models.
problem Improving noise estimation in stochastic partial differential equations for fluid dynamics.
method Replaced PCA with a generative model to avoid constraints on stochastic increments.
result Generative model produces better RMSE, CRPS score, and forecast rank histograms.
New algorithm improves on EM for streaming data, outperforming existing methods.
problem Processing high-volume, streaming data efficiently.
method Incremental stochastic Majorization-Minimization (MM) algorithm.
result The algorithm converges to a stationary point with vanishing gradient.
A machine learning framework predicts self-induced stochastic resonance in neurons.
problem Predicting coherent oscillations in slow-fast excitable systems driven by noise.
method Physics-informed machine learning with a Noise-Augmented State Predictor architecture and Kramers' escape theory constraints.
result Trained PINN accurately predicts spike-train coherence on noise intensity, excitability, and timescale separation.
Paper proposes SRA algorithm for online learning robustness and adaptivity.
problem Quantifying and evaluating tradeoff between robustness and adaptivity in online learning.
method SRA algorithm using biased stochastic approximation scheme with adaptive threshold.
result SRA algorithm provides superior performance in synthetic and real datasets.
This work bridges stochastic interpolants to infinite-dimensional Hilbert spaces.
problem Limited flexibility in generating arbitrary distributions for function-valued data.
method Establishes a rigorous framework for stochastic interpolants in infinite-dimensional Hilbert spaces.
result Achieves state-of-the-art results in conditional generation for complex PDE-based benchmarks.
SGD generalization bounds derived from information theory.
problem Understanding generalization of SGD for non-convex functions.
method Combining information-theoretic bounds with perturbation analysis.
result Upper bounds on SGD's generalization error based on gradient variance and function smoothness.
New method learns policies from past data.
problem Learning decision policies from experience.
method Fully Probabilistic Design (FPD) formalism for stochastic policy finding.
result Proposes a new approach for policy learning.
Developed a diffusion model on spherical data, addressing geometric and stochastic challenges.
problem Diffusion models on spherical data face unique geometric and stochastic issues.
method Extended spectral diffusion to spherical harmonics, introducing modified stochastic differential equations.
result Introduced a geometry-dependent inductive bias in spectral diffusion models.
SymDiff uses stochastic symmetrisation for equivariant diffusion models.
problem Constructing equivariant diffusion models for data augmentation.
method Stochastic symmetrisation for lightweight, efficient, and easy-to-implement equivariance.
result SymDiff achieves significant empirical benefit for E(3)-equivariant molecular generation. SBA improves neural network generalization by dynamically augmenting data.
problem Lack of direct supervision in data augmentation for generalization.
method Stochastic Batch Augmentation with dynamic soft label regularization.
result SBA improves generalization and speeds up training.
Paper presents an efficient algorithm for learning minimax risk classifiers with large-scale data.
problem Efficient learning of minimax risk classifiers for large-scale data with multiple classes.
method Combination of constraint and column generation for efficient learning.
result 10x speedup for general large-scale data and 100x speedup with many classes.
New algorithm guarantees optimal convergence rate for stochastic optimization.
problem Optimal convergence rate for stochastic optimization algorithms.
method Regularized versions of Minimization by Incremental Surrogate Optimization (MISO) with arbitrary recurrent data sampling.
result Expected optimality gap converges at O(n−1/2) under general recurrent sampling schemes. Unified treatment of RC in stochastic and deterministic settings.
problem Understanding and generalizing reservoir computing in both deterministic and stochastic contexts.
method Investigation of state-space systems, analysis of fading memory and solution stability, introduction of stochastic echo states.
result Generality of fading memory and solution stability in state-space systems, even without the echo state property.
We developed a new class of physics-informed generative adversarial networks (PI-GANs) to solve in a unified manner forward, inverse and mixed stochastic problems based on a limited number of scattered measurements. Unlike standard GANs relying only on data for training, here we encoded into the architecture of GANs th…
SA-Solver improves stochastic sampling from DPMs.
problem Efficient sampling from Diffusion Probabilistic Models (DPMs) is time-consuming.
method Proposes SA-Solver, an improved stochastic Adams method for solving diffusion SDE.
result SA-Solver achieves improved or comparable performance compared to SOTA methods for few-step sampling.
Novel framework for learning infinitesimal generator of stochastic processes.
problem Challenges in learning infinitesimal generator due to unbounded nature and state space dimensionality.
method Introduces a novel framework based on energy functional, integrates physical priors, and uses reduced-rank estimator in RKHS.
result Learning bounds independent of state space dimension and non-spurious spectral estimation.
New algorithms for approximating stochastic processes efficiently.
problem Finding accurate finite approximations for stochastic processes.
method Develops new algorithms and fast implementations for approximating stochastic processes.
result Efficient approximations for stochastic processes can be found.
New algorithms improve distributed optimization under mild variance conditions.
problem Improving distributed optimization for large-scale machine learning problems.
method Revisited Federated Averaging and SCAFFOLD algorithms under a general variance condition.
result Established convergence results for smooth nonconvex objective functions under mild variance conditions.
Many recent Markov chain Monte Carlo (MCMC) samplers leverage continuous dynamics to define a transition kernel that efficiently explores a target distribution. In tandem, a focus has been on devising scalable variants that subsample the data and use stochastic gradients in place of full-data gradients in the dynamic s…
The paper tackles robust control with uncertain dependence using data-driven methods.
problem Nonparametric robust control under dependence uncertainty in multi-period stochastic systems.
method Nonparametric adaptive robust control framework using stochastic gradient descent ascent algorithm.
result The controller benefits from knowing more about the uncertain model.
Paper improves neural ODEs for forecasting non-Markovian processes.
problem Forecasting irregularly observed time series with incomplete data.
method Path-dependent Neural Jump ODEs with signature transform.
result Path-dependent NJ-ODE outperforms original framework in non-Markovian data.
We show that stochastic interpolation flow maps are Lipschitz with a sharp constant.
problem High dimensional sampling and transport problems.
method Investigating stochastic interpolation flow for generating data samples.
result Stochastic interpolation flow maps are Lipschitz with a sharp constant matching optimal transport maps.
Estimates neural drift for stochastic equations, improving inference on noisy data.
problem Estimating drift in stochastic differential equations with neural networks.
method Non-parametric estimation using ReLU neural networks, enforcing theoretical bounds.
result Practical method for inference on noisy and rough functional data.
New method generates clean data from corrupted observations.
problem Generating clean data from corrupted observations.
method Iterative update of a transport map using black-box corruption channel access.
result Converges to a self-consistent transport map that effectively inverts the corruption channel.
Paper proposes a new method to compare classifiers across multiple datasets.
problem Comparing classifiers over multiple datasets with multiple criteria.
method Adopting decision theory, the paper introduces generalized stochastic dominance for ranking classifiers.
result Generalized stochastic dominance can be used to rank classifiers and statistically tested.