Zeroth HOMFLY polynomial coefficients can't tell mutant knots apart.
problem Mutants in knots cannot be distinguished by the zeroth coefficient of colored HOMFLY polynomials.
method Examined the cables of the HOMFLY polynomial to show invariance of the zeroth coefficient.
result The zeroth coefficient of the colored HOMFLY polynomial is invariant to mutation.
We study the finite horizon Merton portfolio optimization problem in a general local-stochastic volatility setting. Using model coefficient expansion techniques, we derive approximations for the both the value function and the optimal investment strategy. We also analyze the `implied Sharpe ratio' and derive a series a…
Method extracts features from signals for classification with explainability.
problem Lack of interpretability in signal classification models.
method Combining scattering transform and multiclass logistic regression with zeroth-order optimization.
result Uncovered the meaning of scattering transform coefficients.
Zeroth-order optimization methods lack inherent privacy guarantees.
problem Ensuring differential privacy in zeroth-order optimization methods.
method Analyzing ZO-GD with and without random initialization for convex and strongly convex objectives.
result ZO-GD is not differentially private for strongly convex objectives and can have superlinear privacy loss.
Paper tackles zeroth-order optimization for nonconvex problems with constraints, high-dimensions, and saddle-points.
problem Optimization of nonconvex functions with constraints and high-dimensionality, avoiding saddle-points.
method Proposes zeroth-order stochastic approximation algorithms, including conditional gradient and truncated gradient methods, and a zeroth-order cubic regularization Newton's method.
result Demonstrates algorithms achieving rates similar to standard stochastic gradient methods, with rates dependent on poly-logarithmic dimensionality.
Paper proposes a new Hessian-aware zeroth-order optimization for improving black-box adversarial attacks.
problem Improving black-box adversarial attacks on neural networks.
method Introduces a Hessian-aware zeroth-order optimization algorithm called ZO-HessAware.
result ZO-HessAware achieves improved success rates with lower query complexity.
We generalize the Omori-Yau almost maximum principle of the Laplace-Beltrami operator on a complete Riemannian manifold M M M to a second-order linear semi-elliptic operator L L L with bounded coefficients and no zeroth order term. Using this result, we prove some Liouville-type theorems for a real-valued C 2 C^{2} C 2 function …
New method for zeroth-order stochastic gradient algorithms provides confidence intervals.
problem Lack of inferential capabilities for zeroth-order stochastic gradient algorithms.
method Established central limit theorem and provided online estimators for asymptotic covariance matrix.
result Asymptotically valid confidence sets for parameter estimation and prediction.
Zeroth-order methods favor flat minima in machine learning.
problem Finding solutions with small Hessian trace in optimization.
method Zeroth-order optimization with two-point estimator.
result Zeroth-order optimization converges to flat minima.
Improves zeroth-order optimization for private machine learning with public data.
problem High computation and memory cost of first-order DP methods.
method PAZO (Public Data Assisted Zeroth-order Optimization) framework.
result Achieves superior privacy/utility tradeoffs across tasks.
New methods improve convergence rate of zeroth-order proximal stochastic algorithms.
problem Nonconvex nonsmooth optimization problems with infeasible gradients.
method ZO-ProxSVRG and ZO-ProxSAGA with variance reduction techniques.
result Convergence rate improved to O ( 1 T ) O(\frac{1}{T}) O ( T 1 ) from O ( 1 T ) O(\frac{1}{\sqrt{T}}) O ( T 1 ) . New optimization method improves generalization across various tasks.
problem Improving zeroth-order optimization for better generalization.
method Exponential tilting objective to connect zeroth-order optimization with sharpness-aware minimization.
result Achieves better generalization compared to vanilla zeroth-order baselines.
Given a Heegaard splitting of a closed 3-manifold, the skein modules of the two handlebodies are modules over the skein algebra of their common boundary surface. The zeroth Hochschild homology of the skein algebra of a surface with coefficients in the tensor product of the skein modules of two handlebodies is interpret…
We generalize A. Borbély's condition for the conclusion of the Omori-Yau maximum principle for the Laplace operator on a complete Riemannian manifold to a second-order linear semi-elliptic operator L L L with bounded coefficients and no zeroth order term. Also, we consider a new sufficient condition for the existence of …
Two algorithms optimize high-dimensional convex functions using sparse gradient or function value queries.
problem Optimizing high-dimensional convex functions with sparse gradient or function value queries.
method Two algorithms: successive component/feature selection and noisy mirror descent using Lasso gradient estimates.
result Both algorithms have logarithmically dependent convergence rates on the problem's dimensionality.
Improved zeroth-order algorithms for nonconvex optimization with reduced complexity and improved performance.
problem Designing efficient zeroth-order algorithms for nonconvex optimization with reduced function query complexities and improved convergence rates.
method Proposed new algorithms ZO-SVRG-Coord-Rand and ZO-SPIDER-Coord, developed new analyses, and addressed issues of function query complexities and stepsize generation.
result New algorithms outperform existing methods in terms of function query complexities and convergence rates.
Two algorithms solve nonconvex minimax problems with linear constraints, achieving complexity guarantees.
problem Nonconvex minimax problems with coupled linear constraints.
method Zeroth-order primal-dual alternating projected gradient (ZO-PDAPG) and zeroth-order regularized momentum primal-dual projected gradient (ZO-RMPDPG) algorithms.
result Iteration complexity guarantees for solving nonconvex-(strongly) concave minimax problems with coupled linear constraints.
Improved zeroth-order algorithms tackle nonconvex minimax problems with reduced complexity.
problem Nonconvex minimax optimization problems in machine learning.
method Design and analysis of Zeroth-Order Gradient Descent Ascent ( exttt{ZO-GDA}) and Zeroth-Order Gradient Descent Multi-Step Ascent ( exttt{ZO-GDMSA}) algorithms.
result Oracle complexity improvements for minimax optimization problems.
ConMeZO speeds up zeroth-order optimization for large language models.
problem Slow convergence in high-dimensional parameter spaces of large language models.
method Adaptive directional sampling in a cone centered around a momentum estimate.
result Achieves the same convergence rate as MeZO but up to 2X faster.
We propose a method for zeroth order stochastic convex optimization that attains the suboptimality rate of O ~ ( n 7 T − 1 / 2 ) \tilde{\mathcal{O}}(n^{7}T^{-1/2}) O ~ ( n 7 T − 1/2 ) after T T T queries for a convex bounded function f : R n → R f:{\mathbb R}^n\to{\mathbb R} f : R n → R . The method is based on a random walk (the \emph{Ball Walk}) on the epigraph of the function. Th…
New method uses noisy function evaluations for sampling Langevin diffusions.
problem Difficulty in obtaining gradient evaluations for Langevin diffusions.
method Stochastic zeroth-order discretizations of Langevin diffusions using Gaussian Stein's identities.
result Comprehensive sample complexity analysis for both overdamped and underdamped Langevin diffusions.
A new method for MARL with partial observations reduces communication overhead.
problem Inefficient MARL algorithms in large-scale problems due to state and action information sharing.
method Distributed zeroth-order policy optimization with local policy gradient estimation using consensus.
result The method converges to a policy that is a stationary point of the global objective function.
New methods solve complex optimization problems with fewer function queries.
problem Solving nonconvex optimization problems with multiple penalties and constraints.
method ZO-SPIDER-ADMM and ZOO-ADMM+ methods for nonconvex finite-sum and online problems.
result Achieved lower function query complexity for finding ε-stationary points.
This paper reviews zeroth-order optimization in signal processing and machine learning.
problem Optimization problems without gradient information.
method Iterative steps: gradient estimation, descent direction computation, solution update.
result Demonstrates applications in robustness evaluation and black-box model explanations.
New method solves nonconvex optimization problems without explicit gradients.
problem Solving nonconvex optimization problems with multiple nonsmooth penalties.
method Zeroth-order stochastic ADMM methods (ZO-SVRG-ADMM and ZO-SAGA-ADMM).
result Convergence rate of O ( 1 / T ) O(1/T) O ( 1/ T ) for nonconvex optimization. Book covers tools for zeroth-order convex optimisation.
problem Zeroth-order convex optimisation.
method Cutting plane methods, interior point methods, continuous exponential weights, gradient descent, online Newton step.
result Improved existing bounds and algorithms.
New algorithm tackles risk-aware learning problems efficiently.
problem Risk-aware learning with mean-semideviation objective.
method Zeroth-order compositional stochastic optimization algorithm.
result Algorithm converges to optimal solutions with explicit rates.
New method improves zeroth-order stochastic optimization with adaptive sampling.
problem Optimization problems without gradient information.
method Adaptive sampling quasi-Newton method using finite differences.
result Significant improvement in performance with adaptive sample sizes.
Optimizes smooth functions with noisy zeroth-order feedback.
problem Global optimization of unknown non-convex smooth functions with noisy evaluations.
method Local minimax framework to study zeroth-order optimization.
result Identifies near global minimizers with fewer queries for functions with fast level set growth.
Paper proposes a new algorithm for solving nonconvex problems with noisy data.
problem Nonconvex nonsmooth optimization problems, especially in signal processing and distributed learning.
method Proximal zeroth-order primal dual algorithm (PZO-PDA) that uses only zeroth-order information.
result The algorithm converges and achieves a rate of convergence for nonconvex nonsmooth problems.
Paper proposes algorithms for solving nonconvex-nonconcave problems with complexity guarantees.
problem Nonconvex-nonconcave minimax problems with PL condition.
method Zeroth-order AGDA and VRAGDA algorithms.
result Iteration complexities for obtaining ε-stationary points.
New algorithms solve complex minimax problems without needing derivatives.
problem Solving nonconvex-concave minimax problems efficiently.
method Zeroth-order alternating and proximal gradient algorithms.
result Iteration complexity and function value estimation bounds established.
In this paper we prove that the zeroth Milnor-Thurston homology group coincides with singular homology for Peano Continua. More- over, we show that the canonical homomorphism between these ho- mology theories may not be injective. However, it is proved that it is injective when a space has Borel path-components.
New method uses zeroth-order queries to approximate proximal sampling efficiently.
problem Approximating proximal sampling with zeroth-order information.
method Direct simulation of heat flow dynamics, treating intermediate distribution as Gaussian mixture.
result Inherits exponential convergence under isoperimetric conditions, avoids rejection sampling.
ZOSPI improves RL policies with global value function exploitation.
problem Limited sample efficiency of PG algorithms in RL.
method Zeroth-order supervised policy improvement, leveraging global value function estimation.
result ZOSPI achieves competitive results with remarkable sample efficiency.
Optimal algorithms for Riemannian optimization with reduced complexity.
problem Stochastic optimization on Riemannian manifolds with limited data.
method Zeroth-order Riemannian Averaging Stochastic Approximation algorithms using Riemannian moving-average estimators and novel geometric conditions.
result Achieves optimal sample complexities for generating approximate first-order stationary solutions.
We provide evidence for the conjecture that the Wodzicki-Chern classes vanish for all bundles with the group Z of invertible zeroth order pseudodifferential operators as structure group. In particular, we prove this vanishing if the structure group reduces to pseudodifferential operators with leading order symbol the i…
Paper proposes estimating gradients for zeroth-order nonconvex optimization.
problem Oracle access of gradients is limited in many applications.
method Develops a gradient descent method using estimated gradients.
result Algorithm finds second-order stationary points efficiently.
New Hessian estimators for Riemannian manifolds with reduced bias.
problem Estimating Hessians on Riemannian manifolds with reduced bias and computational efficiency.
method Introducing new stochastic zeroth-order Hessian estimators using O ( 1 ) O(1) O ( 1 ) function evaluations. result Achieved a bias bound of order O ( γ δ 2 ) O(γδ^2) O ( γ δ 2 ) for analytic real-valued functions. We prove identification of coefficients up to gauge by Cauchy data at the boundary for elliptic systems on oriented compact surfaces with boundary or domains of C \mathbb{C} C . In the geometric setting, we fix a Riemann surface with boundary, and consider both a Dirac-type operator plus potential acting on sections of a …
Simplifies noisy convex optimization with a new algorithm.
problem Finding minimum of noisy, convex functions without gradient information.
method Adapts center of gravity method for noisy, zeroth order optimisation.
result Achieves a faster convergence rate than existing methods.
Paper proposes ZO-NGD for more efficient black-box attacks.
problem Vulnerability of state-of-the-art DNNs to adversarial attacks.
method Zeroth-order natural gradient descent (ZO-NGD) for black-box attacks.
result ZO-NGD achieves significantly lower model query complexities.
New algorithm reduces regret in stochastic bandit convex optimization.
problem Optimizing decisions in uncertain environments with convex losses.
method Introduces a second-order method for zeroth-order stochastic convex bandits.
result Regret bound of ( 1 + r / d ) [ d 1.5 n + d 3 ] p o l y l o g ( n , d , r ) (1 + r/d)[d^{1.5} \sqrt{n} + d^3] polylog(n, d, r) ( 1 + r / d ) [ d 1.5 n + d 3 ] p o l y l o g ( n , d , r ) . Paper tackles non-convex optimization over networks with limited information.
problem Optimizing non-convex functions over a multi-agent network with zeroth-order information.
method Developed efficient distributed algorithms for different network topologies, analyzed convergence and rate of convergence.
result Proved convergence and rate of convergence for the set of stationary solutions.
Paper presents ZO-SVRG for faster nonconvex optimization.
problem Gradient-free optimization challenges in nonconvex settings.
method Comprehensive theoretical analysis, novel ZO-SVRG algorithm, accelerated versions.
result ZO-SVRG achieves best rate for ZO stochastic optimization.
New adaptive methods solve weakly convex stochastic optimization problems.
problem Solving weakly convex stochastic optimization problems.
method Adaptive first and zeroth-order methods using exponential moving averages.
result Established non-asymptotic convergence rates for nonsmooth and nonconvex problems.
This paper analyzes and guarantees convergence of prior-guided ZO algorithms.
problem Understanding convergence properties of prior-guided zeroth-order optimization algorithms.
method Analysis of convergence under a greedy descent framework with various gradient estimators, and development of ARS algorithm.
result Convergence guarantee for prior-guided random gradient-free (PRGF) algorithms and accelerated random search (ARS) algorithm.
ZOO attack bypasses substitute models for black-box DNN attacks.
problem Vulnerability of DNNs to adversarial examples.
method Zeroth order optimization for gradient estimation.
result ZOO attack outperforms existing black-box attacks.