SVRG reduces gradient evaluations for policy evaluation in reinforcement learning.
problem Policy evaluation in reinforcement learning with high computational costs.
method Two variants of SVRG for policy evaluation that reduce gradient calculations.
result Significant reduction in the number of gradient evaluations while preserving linear convergence speed.
In this paper we study the problem of minimizing the average of a large number (n) of smooth convex loss functions. We propose a new method, S2GD (Semi-Stochastic Gradient Descent), which runs for one or several epochs in each of which a single full gradient and a random number of stochastic gradients is computed, fo…
A new method for faster multi-objective optimization by evaluating objectives separately.
problem Finding the Pareto front of trade-offs between multiple objectives efficiently.
method Knowledge Gradient with decoupled evaluations, accounting for different costs.
result The method significantly outperforms existing approaches in terms of evaluation cost.
New method uses noisy function evaluations for sampling Langevin diffusions.
problem Difficulty in obtaining gradient evaluations for Langevin diffusions.
method Stochastic zeroth-order discretizations of Langevin diffusions using Gaussian Stein's identities.
result Comprehensive sample complexity analysis for both overdamped and underdamped Langevin diffusions.
A new hybrid algorithm reduces stochastic gradient evaluations for nonconvex optimization.
problem Solving stochastic composite nonconvex optimization problems efficiently.
method Proposes a new hybrid variance-reduced proximal gradient method with a stochastic gradient estimator.
result Achieves optimal stochastic oracle complexity bound with one less gradient evaluation.
Paper introduces a new multi-kernel algorithm for better gradient approximation.
problem Improving gradient approximation in high-dimensional problems.
method Develops a multi-kernel passive stochastic gradient algorithm with variance reduction.
result The multi-kernel algorithm performs better in high-dimensional problems.
New robustness attacks improve evaluation of neural networks.
problem Difficulty in evaluating robustness of neural networks.
method Gradient-based adversarial attacks that are more reliable and efficient.
result Developed attacks are more reliable and efficient than existing methods.
Improves gradient estimation for discrete distributions with variance reduction techniques.
problem Excessive variance in gradient estimation for discrete distributions.
method Stein operators for discrete distributions and control variates.
result Substantially lower variance in gradient estimation.
This paper evaluates and compares gradient leakage attacks in federated learning.
problem Gradient leakage attacks compromise client privacy in federated learning.
method Formal and experimental analysis of gradient leakage attacks, evaluation of attack effectiveness and cost.
result Gradient leakage attacks can reconstruct private local training data from shared parameter updates.
A new method for optimizing black-box problems with constraints.
problem Optimizing black-box systems with multiple performance criteria and constraints.
method Developed a novel constrained Bayesian optimization approach based on the knowledge gradient method.
result A new acquisition function that balances optimality and feasibility.
Paper extends knowledge gradient for preferential BO, overcoming computational challenges.
problem Extending knowledge gradient to preferential BO with pairwise comparisons.
method Derive exact and analytical knowledge gradient for preferential BO.
result Exact knowledge gradient outperforms existing acquisition functions on benchmark problems.
In many applications of black-box optimization, one can evaluate multiple points simultaneously, e.g. when evaluating the performances of several different neural network architectures in a parallel computing environment. In this paper, we develop a novel batch Bayesian optimization algorithm --- the parallel knowledge…
RevDEQs improve performance on tasks with exact gradients and fewer function evaluations.
problem Inexact gradient calculation in DEQs leads to unstable training and requires regularisation or many function evaluations.
method Introduce Reversible Deep Equilibrium Models (RevDEQs) that allow for exact gradient calculation, no regularisation, and far fewer function evaluations.
result RevDEQs significantly improve performance on language modelling and image classification tasks.
uHMC achieves fast mixing in high dimensions with gradient evaluations.
problem Quantifying mixing time of uHMC in high dimensions.
method Construction of successful couplings for uHMC.
result uHMC mixes in total variation with logarithmic dependence on dimension.
This paper introduces a new method to evaluate multiple policies simultaneously.
problem Estimating the value of many policies for a single set of states.
method Developed a scalable, differentiable fingerprinting mechanism to represent complex policies.
result The method can produce policies that outperform those that generated the training data, in zero-shot manner.
New method combines neural networks with Monte Carlo for complex system reliability.
problem Estimating small failure probabilities in complex systems.
method Subset Simulation with Hamiltonian Neural Networks.
result High acceptance rates and computational efficiency in low-probability regions.
This paper reviews zeroth-order optimization in signal processing and machine learning.
problem Optimization problems without gradient information.
method Iterative steps: gradient estimation, descent direction computation, solution update.
result Demonstrates applications in robustness evaluation and black-box model explanations.
Adaptive correlated MC improves sequence generation stability.
problem High gradient variance in reinforcement learning for sequence generation.
method Adapts policy gradient estimator using correlated Monte Carlo rollouts.
result Reduces gradient variance and improves model performance.
We consider in this paper a class of composite optimization problems whose objective function is given by the summation of a general smooth and nonsmooth component, together with a relatively simple nonsmooth term. We present a new class of first-order methods, namely the gradient sliding algorithms, which can skip the…
New method improves off-policy critic evaluation in reinforcement learning.
problem High variance and instability in off-policy policy evaluation.
method Doubly robust estimators applied to actor-critic algorithms.
result Doubly robust estimation significantly improves performance in continuous control tasks.
PGD-trained models have a preferential direction in their gradients, which improves robustness.
problem Mathematical lack of clarity in the direction of preferential gradient alignment after adversarial training.
method Proposed a novel definition of preferential direction and evaluated it using a metric based on GANs.
result PGD-trained models have higher alignment with the proposed preferential direction than baseline models.
Feedback alignment methods need to be evaluated for accuracy and gradient cosine similarity.
problem Evaluating feedback alignment methods
method Proposed diagnostic evaluation protocol
result Identified silent failures in standard reporting pair
We consider parallel global optimization of derivative-free expensive-to-evaluate functions, and propose an efficient method based on stochastic approximation for implementing a conceptual Bayesian optimization algorithm proposed by Ginsbourger et al. (2007). At the heart of this algorithm is maximizing the information…
New PG method derived from IS, improving variance reduction.
problem Improving policy gradient methods for better performance.
method Starting from the DR estimator, a new PG method is derived.
result Empirically shows the new DR-PG estimator is effective.
TERA method speeds up derivative Gaussian processes in high dimensions.
problem High-dimensional function evaluations and gradient computations are computationally expensive.
method TERA uses exact gradient reduction to decouple n and d from the computational cost. result TERA achieves state-of-the-art predictive accuracy with orders of magnitude faster computation.
We present a unified framework to analyze the global convergence of Langevin dynamics based algorithms for nonconvex finite-sum optimization with n component functions. At the core of our analysis is a direct analysis of the ergodicity of the numerical approximations to Langevin dynamics, which leads to faster conver…
This paper evaluates algorithms for classification and outlier detection accuracies in temporal data. We focus on algorithms that train and classify rapidly and can be used for systems that need to incorporate new data regularly. Hence, we compare the accuracy of six fast algorithms using a range of well-known time-ser…
This paper aims at achieving a "good" estimator for the gradient of a function on a high-dimensional space. Often such functions are not sensitive in all coordinates and the gradient of the function is almost sparse. We propose a method for gradient estimation that combines ideas from Spall's Simultaneous Perturbation …
Gradient filters track moving parameters under noisy data and misspecification.
problem Tracking multidimensional time-varying parameters under noisy observations and model misspecification.
method Gradient-based filters update parameters using the gradient of a postulated objective function, evaluated at either the predicted or updated parameters.
result Novel sufficient conditions for exponential stability of the filtered parameter path, and finite-sample and asymptotic mean squared error bounds.
New method speeds up PGD for CV robustness evaluation.
problem Computational inefficiency of PGD for robustness evaluation.
method Early termination of PGD based on cycle detection.
result Large speedup factors without sacrificing robustness.
Improved convergence for non-log-concave sampling.
problem Sampling from non-log-concave distributions.
method Novel conductance analysis of SGLD with auxiliary Markov Chain.
result SGLD achieves ε-sampling error with fewer evaluations.
New method corrects state distribution mismatch for off-policy policy optimization.
problem Mismatch between behavior and evaluation policy state distributions.
method Off-policy policy gradient with state distribution correction.
result Significantly improved policy quality in simulations.
SREDA optimizes complex machine learning problems with fewer evaluations.
problem Finding an optimal point in nonconvex-strongly-concave minimax problems.
method Stochastic Recursive Gradient Descent Ascent (SREDA) with variance reduction.
result Achieves optimal stochastic gradient complexity of O(κ^3ε^-3).
Enhances optimization and sampling methods using ensemble-based gradient inference.
problem Improving ensemble-based methods for optimization and sampling.
method Ensemble-based gradient inference (EGI) to extract higher-order derivatives from particle ensembles.
result Augmented algorithms outperform gradient-free variants, especially in multimodal and non-Gaussian settings.
Improved text generation using transferable rewards from related tasks.
problem Non-differentiable task-specific scores limit the use of policy gradient methods in text generation.
method Transferable Reward Learner that uses model-based rewards for sentence-level and phrase-level similarity.
result Improved performance on semantic evaluation measures in image captioning tasks.
XRAI improves saliency maps for deep neural networks.
problem Improving understanding of deep neural networks through better saliency methods.
method XRAI is a region-based attribution method built upon integrated gradients, with new evaluation methods and sanity checks.
result XRAI produces better results than other saliency methods for common models and the ImageNet dataset.
Optimizes function networks by selectively evaluating parts of the network, reducing costs.
problem Optimizing expensive function networks where full evaluations are costly.
method Knowledge gradient acquisition function for cost-aware partial evaluations.
result Outperforms existing BOFN methods and benchmarks across various problems.
Enhanced visual feature attribution via adaptive baseline weighting.
problem IG's sensitivity to baseline images leads to noisy or unstable explanations.
method Weighted Integrated Gradients (WG) evaluates and weights baselines for improved reliability.
result WG improves over Expected Gradients (EG) by up to 36% across various models.
ZDPG learns model-free policies without critics, improving on PG.
problem Model-free policy learning in complex dynamic problems.
method Approximates policy-reward gradients via two-point stochastic evaluations of the Q-function.
result Restores true model-free policy learning without critics, with improved stability and efficiency.
Unbiased method for Bayesian posterior means using kinetic Langevin dynamics.
problem Estimating Bayesian posterior means efficiently and accurately.
method Combines advanced splitting methods with enhanced gradient approximations in a multilevel Monte Carlo approach.
result The method achieves unbiased estimates with finite variance and central limit theorem properties.
New gradient estimators simplify policy gradient algorithms in reinforcement learning.
problem Challenges in creating effective gradient estimators for reinforcement learning.
method Total derivative rule and graphical models for gradient estimation.
result New gradient estimators lead to improved performance in reinforcement learning.
AdaSVRG combines adaptive gradient with SVRG for robust optimization.
problem Variance reduction in finite-sum minimization with unknown constants.
method AdaSVRG uses AdaGrad in SVRG's inner loop to make it robust.
result AdaSVRG achieves optimal gradient evaluations with no need for problem-dependent constants.
Develops gradient boosting for multi-label classification.
problem Lack of customizable learning algorithms for multi-label classification.
method Generalizes gradient boosting to multi-output problems and proposes an algorithm for learning multi-label classification rules.
result Ability to minimize both decomposable and non-decomposable loss functions.
Generates multimodal safety-critical scenarios for robustness evaluation of decision-making algorithms.
problem Lack of comprehensive evaluation of neural network robustness under real-world scenarios.
method Proposes a flow-based multimodal scenario generator using weighted likelihood maximization and gradient-based sampling.
result Demonstrates improved testing efficiency and multimodal modeling capability compared to traditional methods.
Paper optimizes summarization of multiple document groups for better distinction.
problem Comparative document summarization to select representative documents from multiple groups.
method Formulated new objective functions based on binary classification and maximum mean discrepancy, using gradient-based optimization.
result Gradient-based optimization outperforms other methods in automatic and crowd-sourced evaluations.
A novel metric and framework for evaluating gradient norm equality in deep neural networks.
problem Evaluation of gradient norm equality in complex DNNs requires strong assumptions or complex analysis.
method Proposes a novel metric called Block Dynamical Isometry and a modularized statistical framework based on free probability.
result Gradient Norm Equality is a universal philosophy behind initialization, normalization, and network structures.
Unified framework for gradient-free MDS improves efficiency and accuracy.
problem Efficiently solving Multidimensional Scaling problems without derivatives.
method Bootstrapped Coordinate Search (BS CSMDS) for MDS, using a probability matrix to guide search.
result BS CSMDS achieves significant speedup and maintains error rate compared to other CSMDS methods.
ICCNLS models complex relationships as convex and concave components.
problem Complex input-output relationships with affine ambiguity.
method Sub-gradient constrained affine functions, global orthogonality constraints, L1, L2, and elastic net regularisation.
result Improved predictive accuracy and model simplicity compared to conventional methods.