Framework improves gradient estimation for faster training convergence.
problem Efficiently estimating noisy gradients in stochastic optimization.
method Dynamic adaptive importance sampling combining multiple distributions.
result Adaptively weighted multiple importance sampling yields superior gradient estimates.
Optimizes Thompson sampling policies using policy gradient methods.
problem Improving Thompson sampling in bandit problems.
method Applies policy gradient algorithms to optimize Thompson sampling policies.
result Direct policy search on Thompson sampling improves performance.
Proposes log density gradient to improve reinforcement learning sample complexity.
problem Residual error in gradient estimation in policy gradient methods.
method Log density gradient method to correct residual error, using state-action discounted distributional formulation.
result Min-max optimization method to approximate log density gradient with on-policy samples, achieving sample complexity of m−1/2. New method for sampling on constrained domains using orthogonal-space gradient flow.
problem Sampling on manifolds defined by constraints is challenging.
method Orthogonal-Space Variational Gradient Descent (O-Gradient)
result O-Gradient converges to the target constrained distribution efficiently.
Stochastic Gradient Descent or SGD is the most popular optimization algorithm for large-scale problems. SGD estimates the gradient by uniform sampling with sample size one. There have been several other works that suggest faster epoch-wise convergence by using weighted non-uniform sampling for better gradient estimates…
New method for constrained sampling using gradient flows.
problem Sampling from constrained domains.
method Introducing a boundary condition for gradient flow to confine particles within the domain.
result Provable continuous-time convergence in total variation for constrained sampling.
DBQPG improves policy gradient estimation with fewer samples.
problem Accurate policy gradient estimation with limited samples.
method Deep Bayesian Quadrature Policy Gradient (DBQPG).
result DBQPG provides more accurate and less variable gradient estimates.
AES improves policy gradient performance by adaptively selecting experience.
problem High variance in gradient estimators from past trajectories.
method AES learns an adaptive sampling distribution to minimise gradient variance.
result AES leads to significantly improved performance in continuous control tasks.
Gradient-guided nested sampling improves posterior inference efficiency.
problem Efficiently sampling from complex posterior distributions.
method Gradient-guided nested sampling combining differentiable programming, Hamiltonian slice sampling, clustering, mode separation, dynamic nested sampling, and parallelization.
result Significantly faster mode discovery and more accurate partition function estimates.
We formulate the problem of sampling and recovering clustered graph signal as a multi-armed bandit (MAB) problem. This formulation lends naturally to learning sampling strategies using the well-known gradient MAB algorithm. In particular, the sampling strategy is represented as a probability distribution over the indiv…
SGBD algorithm improves robustness in Bayesian sampling.
problem Inefficiency of existing MCMC algorithms in large datasets.
method Extends Barker MCMC to stochastic gradient framework, introducing bias-corrected version.
result SGBD is more robust to hyperparameter tuning and gradient noise.
ASVGD accelerates SVGD for efficient sampling.
problem Slow SVGD in high-dimensional sampling.
method Accelerated gradient flow in a metric space of probability densities, using Nesterov's method and momentum-based updates.
result ASVGD outperforms SVGD and other methods in sampling efficiency.
Study shows gradient variance increases during deep learning training, contrary to common belief.
problem Understanding and minimizing gradient variance in deep learning models.
method Gradient Clustering method using stratified sampling to minimize gradient variance.
result Gradient variance increases during training, and smaller learning rates coincide with higher variance.
This work shows synthetic gradients can outperform backpropagation in sample efficiency.
problem The efficiency of backpropagation in training neural networks.
method Unified vectorized feedback framework for loss-based and reward-based learning, introducing synthetic gradients.
result Synthetic gradients can achieve lower gradient-estimation mean squared error than backpropagation under certain conditions.
Improved sampling method using regularized Stein Variational Gradient Flow.
problem Improving the accuracy of sampling methods in machine learning.
method Proposed Regularized Stein Variational Gradient Flow to interpolate between SVGD and Wasserstein Gradient Flow.
result Established theoretical properties and provided preliminary numerical evidence of improved performance.
New analysis improves sample complexity for vanilla policy gradient methods.
problem Improving sample complexity guarantees for vanilla policy gradient methods.
method Adapting tools from SGD analysis to policy gradient methods, with smoothness and gradient approximation assumptions.
result Established improved sample complexity bounds for convergence and global optimum.
In modern data analysis, random sampling is an efficient and widely-used strategy to overcome the computational difficulties brought by large sample size. In previous studies, researchers conducted random sampling which is according to the input data but independent on the response variable, however the response variab…
ARMS improves gradient estimation for binary variables using antithetic samples.
problem Estimating gradients for binary variables in discrete latent variable models.
method ARMS uses antithetic samples generated by a copula to estimate gradients more efficiently and unbiasedly.
result ARMS outperforms competing methods in training generative models and optimizing variational bounds.
Improved sampling accuracy in SG-MCMC methods via non-uniform gradient subsampling.
problem Computational inefficiency and sampling error in stochastic gradient MCMC methods.
method Proposes a non-uniform subsampling scheme to reduce sampling error in EWSG, a variant of SG-MCMC.
result EWSG reduces sampling error compared to uniform subsampling, improving accuracy without sacrificing convergence speed.
New sampling method uses gradient-free IPS with RKHS velocity field.
problem Efficient sampling from unnormalized target densities.
method Gradient-free interacting particle systems (IPS) with RKHS velocity field.
result IPS produce high-quality samples from various target distributions.
This paper explores gradient flows for sampling distributions without normalization constants.
problem Sampling from distributions with unknown normalization constants.
method Gradient flows in the space of probability measures, focusing on Kullback-Leibler divergence, Fisher-Rao metric, and affine invariance.
result Gradient flows derived from Kullback-Leibler divergence do not depend on the normalization constant.
Variational inference approximates the posterior distribution of a probabilistic model with a parameterized density by maximizing a lower bound for the model evidence. Modern solutions fit a flexible approximation with stochastic gradient descent, using Monte Carlo approximation for the gradients. This enables variatio…
Improved convergence for non-log-concave sampling.
problem Sampling from non-log-concave distributions.
method Novel conductance analysis of SGLD with auxiliary Markov Chain.
result SGLD achieves ε-sampling error with fewer evaluations.
Hybrid method improves sampling from multimodal distributions.
problem Sampling from multimodal posterior distributions efficiently.
method Jump-Diffusion Langevin Dynamics hybrid with Metropolis.
result Calibrated hybrid method outperforms pure methods.
REGS samples from unnormalized distributions using gradient flow and neural networks.
problem Sampling from unnormalized distributions with high accuracy and efficiency.
method REGS is a particle method that iteratively transforms samples from a reference distribution to match an unnormalized target distribution using Wasserstein gradient flow and neural networks.
result REGS outperforms state-of-the-art methods in sampling from challenging multimodal distributions and real datasets.
Stochastic gradient descent optimizes Nyström samples for kernel matrix approximation.
problem Optimizing Nyström samples for kernel matrix approximation.
method Stochastic gradient descent applied to multisets of landmark points (Nyström samples) using a surrogate criterion (radial SKD).
result Local minimization of the radial SKD yields improved Nyström approximation accuracy.
Improves deep generative model samples via gradient flow.
problem Inconsistent generation quality across samples.
method Discriminator Gradient Flow (DGflow) using entropy-regularized f-divergences.
result Significant improvement in generated sample quality.
New method uses PDMPs with sub-sampling for efficient sampling from posterior distributions.
problem Efficient sampling from posterior distributions with limited data access.
method Approximate simulation of PDMPs with sub-sampling and stochastic gradient estimation.
result Stochastic-gradient PDMPs are efficient and robust compared to Langevin dynamics.
PS-IG improves feature attribution by reducing noise and variance.
problem Improving feature attribution in machine learning models.
method Path-sampled integrated gradients (PS-IG) computes expected value over sampled baselines.
result PS-IG reduces attribution variance by a factor of 1/3 under uniform sampling.
MT-SGD samples from multiple target distributions using gradient descent.
problem Sampling from multiple unnormalized target distributions.
method Proposes MT-SGD, a flow of intermediate distributions to sample from multiple target distributions.
result Asymptotic analysis shows MT-SGD reduces to multiple-gradient descent for multi-objective optimization.
A new method for sampling high-dimensional distributions overcomes overfitting.
problem Overfitting in energy-based models during gradient descent.
method Mean-field microcanonical gradient descent, which samples multiple data points simultaneously.
result The method reduces entropy loss while maintaining likelihood fit, improving overfitting issues.
Paper proposes an adversarial sampling method for efficient extreme classification.
problem Training classifiers over many classes is computationally expensive.
method Adversarial sampling to draw negative samples from an adversarial model.
result Significantly reduces training time by an order of magnitude.
New algorithms for sampling and optimization without tuning.
problem Efficient sampling and optimization over probability measures.
method Optimization on the space of probability measures, using gradient flows.
result Strong theoretical guarantees and similar performance to optimally tuned algorithms.
Lower bounds show many sampling algorithms need many gradient queries.
problem Sampling from strongly log-concave densities in high dimensions.
method Information theory and stochastic gradient methods.
result Lower bound on number of gradient queries needed.
As an adaptive, interpretable, robust, and accurate meta-algorithm for arbitrary differentiable loss functions, gradient tree boosting is one of the most popular machine learning techniques, though the computational expensiveness severely limits its usage. Stochastic gradient boosting could be adopted to accelerates gr…
A quantum reinforcement learning algorithm reduces sample complexity.
problem Quantum reinforcement learning under model-free settings with quantum oracle access.
method Quantum Natural Policy Gradient (QNPG) algorithm replacing random sampling with deterministic gradient estimation.
result QNPG achieves a sample complexity of ildeO(ε−1.5) for queries to the quantum oracle, significantly improving classical lower bound. Improving sample efficiency has been a longstanding goal in reinforcement learning. This paper proposes VRMPO algorithm: a sample efficient policy gradient method with stochastic mirror descent. In VRMPO, a novel variance-reduced policy gradient estimator is presented to improve sample efficiency.…
SIFG uses noisy particles to efficiently sample from complex distributions.
problem Efficient sampling from complex distributions using particle-based methods.
method SIFG introduces a semi-implicit functional gradient flow with Gaussian noise to improve sampling efficiency and accuracy.
result SIFG achieves strong theoretical convergence guarantees and efficient sampling.
Paper analyzes PSGLD for adaptive IRL with finite-sample bounds.
problem Estimating cost function of a forward learner using noisy gradients.
method Passive stochastic gradient Langevin dynamics (PSGLD) algorithm.
result Explicit bounds on 2-Wasserstein distance between PSGLD sample measure and stationary measure.
Introduces gradient decay in Softmax for better generalization.
problem Improving generalization performance in neural networks.
method Gradient decay hyperparameter in Softmax for varying gradient rates based on probability.
result Gradient decay rate affects generalization performance and can be tuned for better optimization.
New SGD method uses adaptive sampling to converge faster in non-convex problems.
problem Non-convex optimization problems with noisy gradients.
method Adaptive coordinate sampling in stochastic gradient descent (SGD).
result Almost sure convergence and non-asymptotic bounds established.
New algorithm tackles nonconvex machine learning problems with adaptive normalization and independent sampling.
problem Nonconvex machine learning problems with generalized-smoothness.
method Adaptive gradient normalization, independent sampling, and gradient clipping.
result Achieves an O(ε^(-4)) sample complexity for fast convergence.
CARMS improves gradient estimation for categorical variables.
problem Accurately backpropagating gradients through categorical variables.
method CARMS combines REINFORCE with antithetic sampling to create unbiased gradient estimators.
result CARMS outperforms competing methods on various tasks.
New algorithms sample from log concave distributions without gradient Lipschitz continuity.
problem Sampling from log concave distributions without gradient Lipschitz continuity.
method Two algorithms based on monotone polygonal (tamed) Euler schemes.
result Non-asymptotic 2-Wasserstein distance bounds between the process and target measure.
BSVGD improves sampling for multimodal distributions using branching.
problem Sampling from multimodal distributions.
method Random branching in Stein Variational Gradient Descent (SVGD).
result Theoretical convergence guarantee and empirical validation.
GSSBO reduces GP fitting time in Bayesian optimization.
problem High computational cost of fitting Gaussian process surrogate models in Bayesian optimization.
method Gradient-based sample selection to reduce the number of samples used in GP fitting.
result Sublinear regret bounds and significant reduction in computational cost.
Proposes an automatic cyclical scheduling for gradient-based discrete sampling.
problem Gradient-based sampling in high-dimensional models can get stuck in local modes.
method Cyclical step size and balancing schedules with automatic hyperparameter tuning.
result Proves non-asymptotic convergence and inference guarantees for general discrete distributions.
DG improves policy gradient efficiency by selectively backpropagating only valuable samples.
problem Expensive backward passes in policy gradient methods reduce efficiency.
method Introduces 'delight' as a forward-pass signal of learning value and a Kondo gate to selectively backpropagate.
result Selective backpropagation reduces backward pass costs without sacrificing learning quality.