Unified framework for discrete diffusion modeling with flexible noising processes.
problem Efficient modeling of large discrete state spaces with arbitrary corruption dynamics.
method Generalized Discrete Diffusion from Snapshots (GDDS) framework that supports uniformization for fast noising and snapshot-based ELBO for reverse process.
result GDDS outperforms existing discrete diffusion methods in training efficiency and generation quality.
Discrete noise improves graph generation quality and speed.
problem Generating high-quality discrete graph samples.
method Using discrete noise in diffusion models for graph generation.
result Discrete noise leads to 1.5x better MMDs and 30x faster sampling.
Simplified analysis of diffusion models using discrete random variables.
problem Theoretical analysis of diffusion models is complex and requires rigorous proofs.
method Simplified framework for analyzing Euler--Maruyama discretization of VP-SDEs using Grönwall's inequality.
result Standard Gaussian noise can be replaced by discrete random variables without sacrificing convergence guarantee.
FLDD improves discrete diffusion models by learning a non-Markovian noising process.
problem Efficiency and quality of discrete diffusion models in few-step generation.
method Introduces a learnable non-Markovian forward (noising) process to match the target distribution.
result FLDD produces higher quality samples in fewer steps compared to conventional discrete diffusion models.
Discrete Gaussian noise preserves privacy and accuracy in differential privacy.
problem Finite computers cannot represent continuous Gaussian noise, leading to privacy breaches and loss of interpretability.
method Introduced and analyzed discrete Gaussian noise, providing privacy and accuracy guarantees similar to continuous Gaussian noise.
result Discrete Gaussian noise offers the same privacy and accuracy as continuous Gaussian noise, with efficient sampling algorithms.
A system for federated learning with private data, adding discrete Gaussian noise and secure aggregation.
problem Training models on private data distributed across devices while ensuring privacy.
method Discretizes data, adds discrete Gaussian noise, and uses secure aggregation to protect privacy.
result Matches the accuracy of central differential privacy with less than 16 bits of precision per value.
Unified discrete diffusion for categorical data simplifies training and sampling.
problem Training and sampling in discrete diffusion models for categorical data.
method Mathematical simplifications and elegant unification of discrete-time and continuous-time discrete diffusion.
result Unified Simplified Discrete Denoising Diffusion (USD3) outperforms SOTA baselines.
Safety filter for unknown discrete-time systems with learned models and noise covariance.
problem Ensuring safety for unknown discrete-time linear systems with Gaussian noise.
method Develops a learning-based safety filter using empirical model and noise covariance, optimizing control actions to stay within safety constraints.
result Minimally modifies nominal control actions to ensure safety with high probability, tightening constraints as more data is collected.
Two models incorporate market microstructure noise into asset pricing and option valuation.
problem Effect of market microstructure noise on asset pricing and option valuation.
method Developed two models: a continuous-time Black-Scholes-Merton model and a discrete binomial tree model.
result Extracted coefficients to quantify noise impact on volatility and drift.
Optimizes control of noisy discrete systems without system matrix knowledge.
problem Optimal control of discrete-time systems with additive and multiplicative noises.
method Stochastic Lyapunov and Riccati equations, model-free reinforcement learning.
result Model-free reinforcement learning algorithm converges to optimal control policy.
Masking diffusion outperforms other discrete diffusion models by incorporating jump times into the model.
problem Improving the performance of discrete diffusion models.
method Conditioning on the jump schedule of discrete Markov processes.
result Schedule-conditioned discrete diffusion (SCUD) models outperform classical and masking diffusion models.
Work on SGDm under heavy-tailed noise, revealing its generalization properties.
problem Understanding generalization of SGDm under heavy-tailed noise.
method Analysis of continuous-time limit (SDE) and discrete-time SGDm, establishing generalization bounds.
result SGDm can have worse generalization in the presence of heavy-tailed noise for quadratic loss functions.
CANDI solves the gap between continuous and discrete diffusion models for text generation.
problem Underperformance of continuous diffusion models in discrete data domains.
method Introduces token identifiability and a hybrid framework (CANDI) to decouple discrete and continuous corruption.
result CANDI successfully avoids temporal dissonance, enabling continuous diffusion benefits for discrete spaces.
AlignFlow improves FGMs by optimizing noise and data alignment.
problem Optimal Transport methods for FGMs are limited by scalability issues.
method Introduces Semi-Discrete Optimal Transport (SDOT) to enhance FGM training.
result AlignFlow scales well to large datasets and model architectures.
A new method trains discrete EBMs without sampling.
problem Training EBMs on discrete spaces is hard.
method Energy Discrepancy (ED), a contrastive loss.
result ED offers theoretical guarantees for various perturbation types.
New geometric SDEs and discretizations on Riemannian manifolds with error bounds.
problem Modeling diffusion processes on Riemannian manifolds with geometric SDEs.
method Introduced a new construction of geometric SDEs and provided non-asymptotic error bounds.
result First non-asymptotic error bound for geometric Euler-Murayama discretization.
Paper introduces DMPMs for efficient discrete data generation with sharp convergence bounds.
problem Efficient generation of discrete data with theoretical guarantees.
method Discrete Markov Probabilistic Models (DMPMs) operating in bit space with time-reversal process.
result Sharp convergence bounds established under minimal assumptions, competitive performance in discrete data generation.
Neural Ordinary Differential Equation (Neural ODE) has been proposed as a continuous approximation to the ResNet architecture. Some commonly used regularization mechanisms in discrete neural networks (e.g. dropout, Gaussian noise) are missing in current Neural ODE networks. In this paper, we propose a new continuous ne…
A new model captures irregularly spaced high-frequency prices and their volatility.
problem Modeling high-frequency prices with irregular spacing and market noise.
method Observation-driven model using Skellam distribution with time-varying volatility and smoothing splines.
result The model provides a good fit to IBM stock data and measures daily realized volatility.
Proposes a method to quantify uncertainty in DNN models for discrete inputs.
problem Uncertainty quantification for DNN models with categorical and discrete feature variables.
method Develops a mathematical framework to quantify prediction uncertainty from discrete input noise and model parameters.
result Identifies risk-sensitive cases prone to misclassification due to discrete predictor errors.
Study shows unique linear equilibrium in market with constrained trader.
problem Unique equilibrium in financial market with constrained trader.
method Linear equilibrium model with competitive market makers and noise traders.
result Equilibrium uniquely determined by two state variables.
Inferring the causal structure of a set of random variables from a finite sample of the joint distribution is an important problem in science. Recently, methods using additive noise models have been suggested to approach the case of continuous variables. In many situations, however, the variables of interest are discre…
Replica exchange Langevin diffusion accelerates nonconvex optimization.
problem Nonconvex optimization challenges in machine learning.
method Replica exchange Langevin diffusion, discretization analysis.
result Replica exchange accelerates convergence to global minima.
We introduce two approaches for combining neural evolution strategy (NES) and proximal policy optimization (PPO): parameter transfer and parameter space noise. Parameter transfer is a PPO agent with parameters transferred from a NES agent. Parameter space noise is to directly add noise to the PPO agent`s parameters. We…
The Epps effect helps distinguish between continuous and discrete financial tick data.
problem Determining whether financial tick data represents continuous or discrete events.
method Deriving and correcting the Epps effect, proposing experiments to discriminate between models.
result Tick data is better represented as discrete events rather than continuous Brownian diffusions.
Paper explores neural network approximations on sphere domains.
problem Approximating functionals on sphere domains using neural networks.
method Encoder-decoder framework with spherical harmonics for infinite-dimensional domain.
result Approximation rates of neural networks with different encoder structures.
D2P-Fed improves privacy and communication in federated learning.
problem Achieving both differential privacy and communication efficiency in federated learning.
method Applying discrete Gaussian noise to private data transmission.
result D2P-Fed outperforms state-of-the-art by 4.7% to 13.0% in model accuracy with one-third less communication cost.
Noise-robust Koopman operator framework for control with improved stability and performance.
problem Developing a stable and noise-robust Koopman operator for control tasks.
method Proposes a learning framework using Hankel matrix and neural network approximations for system dynamics, ensuring long-term stability and noise robustness.
result Demonstrates improved model performance and noise robustness in control tasks compared to existing methods.
Paper develops a consistent estimator for discrete mixture models.
problem Estimating mixing probability density in discrete mixture models.
method Develops an L1 consistent estimator under specific conditions. result Consistent estimation of mixing probability density for every density f. New discrete-time model shows insider trading dynamics.
problem Modeling insider trading with discrete time and noise traders.
method Formulated as a game with three types of traders, including an insider, noise traders, and a market maker. Proved existence of sequential Kyle equilibrium for various distributions and information flows.
result Equilibria exist in mixed strategies but not in pure strategies, unlike in Kyle's original model.
This study shows how DDPM can be represented by the OU process.
problem Designing optimal noise schedules for DDPM.
method Formal equivalence between DDPM and OU process, heuristic designs based on Fisher Information.
result Fisher-Information-motivated schedule corresponds to cosine noise schedule.
We investigate the "compass rose" (Crack, T.F. and Ledoit, O. (1996), Journal of Finance, 51(2), pg. 751-762) patterns revealed in phase portraits (delay plots) of stock returns. The structures observed in these diagrams have been attributed mainly to price clustering and discreteness. Using wavelet based denoising, we…
Study on optimal bubble riding with price-dependent entry times in a mean field game model.
problem Optimal bubble riding with price-dependent entry times.
method Mean field game of controls with common noise and random entry time, existence result obtained through discretization and limit analysis.
result Existence of equilibrium in the mean field game model.
Continuous time framework for discrete data denoising models.
problem Efficient training and sampling for discrete data denoising models.
method Formulated as Continuous Time Markov Chains (CTMCs), efficient training using continuous time ELBO, high-dimensional CTMC simulation, novel theoretical error bound.
result Continuous time treatment enables novel theoretical error bound between generated and true data distributions.
Deep reinforcement learning (RL) methods generally engage in exploratory behavior through noise injection in the action space. An alternative is to add noise directly to the agent's parameters, which can lead to more consistent exploration and a richer set of behaviors. Methods such as evolutionary strategies use param…
Stochastic gradient descent (SGD) has been widely used in machine learning due to its computational efficiency and favorable generalization properties. Recently, it has been empirically demonstrated that the gradient noise in several deep learning settings admits a non-Gaussian, heavy-tailed behavior. This suggests tha…
Discrete Flow Maps bypass sequential prediction limits for parallel text generation.
problem Sequential autoregressive prediction limits large language model speed.
method Flow Maps compress generative trajectories into single-step mappings.
result Discrete Flow Maps surpass previous state-of-the-art results in discrete flow modeling.
Robust Q-learning for mean-field control under Wasserstein uncertainty
problem Mean-field control under Wasserstein uncertainty
method Quantization-and-projection scheme with Wasserstein dual reformulation
result Convergence and finite-time iteration bounds
Improved analysis for diffusion models reduces KL divergence error dependence on data dimension and discretization step size.
problem Analyze the convergence of diffusion-based generative models under minimal assumptions.
method Model the generation process as a composition of reverse ODE and noising steps, leveraging Wasserstein-type error control and noise addition.
result Achieved a linear dependence on data dimension and improved dependence on discretization step size for KL divergence error.
Study on the noise in SGD minibatches near local minima.
problem Understanding the noise in SGD minibatches near local minima.
method Detailed analysis of SGD noise in linear regression and derivation of a general formula for different types of minima.
result Provides insight into the stability of training neural networks and suggests large learning rates can help generalization.
Study provides convergence guarantees for discrete diffusion models on finite and infinite state spaces.
problem Challenges in understanding discrete diffusion models on combinatorial state spaces.
method Established convergence bounds for three discrete diffusion models using Euler approximations.
result Optimal non-asymptotic convergence guarantees for discrete diffusion models without boundedness assumptions.
A method for diffusion on probability simplex for generative models.
problem Tension between continuous and discrete data in diffusion models.
method Proposes using softmax function applied to Ornstein-Uhlenbeck Process on probability simplex.
result Method extends to bounded image generation.
SGD handles label noise with bounds improving over SGLD.
problem Label noise in non-convex optimization.
method Stochastic gradient descent with uniform dissipativity and smoothness conditions, using Wasserstein distance and algorithmic stability.
result Generalization error bounds with a rate of n−2/3, better than SGLD's n−1/2. The problem of devising learning strategies for discrete losses (e.g., multilabeling, ranking) is currently addressed with methods and theoretical analyses ad-hoc for each loss. In this paper we study a least-squares framework to systematically design learning algorithms for discrete losses, with quantitative character…
In this paper, we face the problem of simulating discrete random variables with general and varying distributions in a scalable framework, where fully parallelizable operations should be preferred. The new paradigm is inspired by the context of discrete choice models. Compared to classical algorithms, we add paralleliz…
Discrete diffusion models improve data generation for discrete data like language and graphs.
problem Adapting diffusion models to discrete state spaces for better data generation.
method Formulated as CTMCs, used uniformization of continuous Markov chains for sampling.
result Derive guarantees for sampling from any distribution on a hypercube, aligning with state-of-the-art achievements.
PBM mechanism improves privacy and accuracy in federated learning.
problem Secure and private federated learning with limited privacy budget.
method Poisson Binomial mechanism for discrete differential privacy.
result Achieves same privacy-accuracy trade-offs as Gaussian mechanism.
A new RG approach connects discrete and continuous time descriptions of Gaussian processes.
problem Discretization of continuous stochastic processes for accurate simulation or model inference.
method Renormalization Group (RG) approach for Gaussian time series generated by auto-regressive models.
result RG fixed points correspond to discretizations of linear SDEs, providing insights into process accuracy.