arXiv research

We consider the case of derivative-free algorithms for non-convex optimization, also known as zero order algorithms, that use only function evaluations rather than gradients. For a wide variety of gradient approximators based on finite differences, we establish asymptotic convergence to second order stationary points u…

CyBeR-0 optimizes federated learning with Byzantine resilience and reduced communication costs.

Study optimizes zero-order strongly convex function minimization with higher order smoothness.

A new gradient estimator for online optimization with two function evaluations.

The paper analyzes the efficiency of gradient estimation methods in noisy function evaluations.

New algorithm optimizes convex functions with noisy evaluations in one dimension.

The problem of resource allocation of nonlinear networked control systems is investigated, where, unlike the well discussed case of triggering for stability, the objective is optimal triggering. An approximate dynamic programming approach is developed for solving problems with fixed final times initially and then it is…

We study derivative-free methods for policy optimization over the class of linear policies. We focus on characterizing the convergence rate of these methods when applied to linear-quadratic systems, and study various settings of driving noise and reward feedback. We show that these methods provably converge to within a…

A new method for distributed optimization with noisy function evaluations.

We consider the closely related problems of bandit convex optimization with two-point feedback, and zero-order stochastic convex optimization with two function evaluations per round. We provide a simple algorithm and analysis which is optimal for convex Lipschitz functions. This improves on \cite{dujww13}, which only p…

Derivative-free method solves stochastic optimization problems with noisy objectives and constraints.

The superior interpretability and uncertainty modeling ability of Takagi-Sugeno-Kang fuzzy system (TSK FS) make it possible to describe complex nonlinear systems intuitively and efficiently. However, classical TSK FS usually adopts the whole feature space of the data for model construction, which can result in lengthy …

We consider stochastic zero-order optimization problems, which arise in settings from simulation optimization to reinforcement learning. We propose an adaptive sampling quasi-Newton method where we estimate the gradients of a stochastic function using finite differences within a common random number framework. We emplo…

Paper tackles dynamic pricing in a geometrically decaying environment, achieving better occupancy with lower rates.

Unified algorithm for stochastic optimization with time-varying momentum converges under general conditions.

We describe a conjectural formula via intersection numbers for the Masur-Veech volumes of strata of quadratic differentials with prescribed zero orders, and we prove the formula for the case when the zero orders are odd. For the principal strata of quadratic differentials with simple zeros, the formula reduces to compu…

We show that the eigenvalues of the intrinsic Dirac operator on the boundary of a Euclidean domain can be obtained as the limits of eigenvalues of Euclidean Dirac operators, either in the domain with a MIT-bag type boundary condition or in the whole space, with a suitably chosen zero order mass term.

This paper tackles the computational complexity of finding approximate stationary points in non-convex optimization.

Improved analysis and new algorithm for gradient-free optimization of smooth functions.

A stochastic model for pure-jump diffusion (the compound renewal process) can be used as a zero-order approximation and as a phenomenological description of tick-by-tick price fluctuations. This leads to an exact and explicit general formula for the martingale price of a European call option. A complete derivation of t…

We propose a novel interpretation of the collapsed variational Bayes inference with a zero-order Taylor expansion approximation, called CVB0 inference, for latent Dirichlet allocation (LDA). We clarify the properties of the CVB0 inference by using the alpha-divergence. We show that the CVB0 inference is composed of two…

In this small note we use results derived in Berestycki et al. to correct the celebrated formulae of Hagan et al. We derive explicitly the correct zero order term in the expansion of the implied volatility in time to maturity. The new term is consistent as $β\to 1$. Furthermore, numerical simulations show that it reduc…

Reinforcement learning algorithms, though successful, tend to over-fit to training environments hampering their application to the real-world. This paper proposes $\text{W}\text{R}^{2}\text{L}$ -- a robust reinforcement learning algorithm with significant robust performance on low and high-dimensional control tasks. Ou…

Gradient-free optimization for additive models achieves optimal error.

The purpose of this paper is to prove the a priori estimates for constant scalar curvature Kaehler metrics with conic singularities along normal crossing divisors. The zero order estimates are proved by a reformulated version of Alexandrov's maximum principle. The higher order estimates follow from Chen-Cheng's frame …

HAR regression improves performance on small datasets.

Proposes a framework for generating explainable AI exemplars.

We consider spaces of smooth immersed plane curves (modulo translations and/or rotations), equipped with reparameterization invariant weak Riemannian metrics involving second derivatives. This includes the full $H^2$-metric without zero order terms. We find isometries (called $R$-transforms) from some of these spaces i…

This note explains Ricci flow method for Kähler-Einstein metrics.

Let $M$ be a complete Riemannian manifold and let $Ω^*(M)$ denote the space of differential forms on $M$. Let $d:Ω^*(M) \to Ω^{*+1}(M)$ be the exterior differential operator and let $\Del=dd^*+d^*d$ be the Laplacian. We establish a sufficient condition for the Schroedinger operator $H=\Del+V(x)$ (where the potential $V…

We consider derivative-free algorithms for stochastic and non-stochastic convex optimization problems that use only function values rather than gradients. Focusing on non-asymptotic bounds on convergence rates, we show that if pairs of function values are available, algorithms for $d$-dimensional optimization that use …

This work speeds up hyperparameter selection for non-smooth convex models using implicit differentiation.

The paper calculates option prices using Mellin transform for stochastic volatility models.

John Lott has computed an integer-valued signature for the orbit space of a compact orientable $(4k+1)$ manifold with a semi-free $S^1$-action, which is a homotopy invariant of that space, but he did not construct a Dirac type operator which has this signature as its index. In this Thesis, we construct such operator on…

ZeroS improves Transformers by adding negative weights, matching or beating softmax attention.

A C*algebra A generated by a class of zero-order classical pseudodifferential operator on a cylinder RxB, where B is a compact riemannian manifold, containing operators with periodic symbols, is considered. A description of the K-theory index map associated to the continuous extension to A of the principal-symbol map i…

The digital telecommunications receiver is an important context for inference methodology, the key objective being to minimize the expected loss function in recovering the transmitted information. For that criterion, the optimal decision is the Bayesian minimum-risk estimator. However, the computational load of the Bay…

We introduce a new stochastic smoothing perspective to study adversarial contextual bandit problems. We propose a general algorithm template that represents random perturbation based algorithms and identify several perturbation distributions that lead to strong regret bounds. Using the idea of smoothness, we provide an…

We address some global solvability issues for classes of smooth nonsingular vector fields $L$ in the plane related to cohomological equations $Lu=f$ in geometry and dynamical systems. The first main result is that $L$ is not surjective in $C^\infty(\R^2)$ iff the geometrical condition -- the existence of separatrix str…

The paper proves Gorenstein contractions for multiscale differentials on nodal curves.

We investigate serial correlation, periodic, aperiodic and scaling behaviour of eigenmodes, i.e. daily price fluctuation time-series derived from eigenvectors, of correlation matrices of shares listed on the Johannesburg Stock Exchange (JSE) from January 1993 to December 2002. Periodic, or calendar, components are dete…

GRAC improves reinforcement learning by self-guiding and self-regularizing.

In Maslov (2003), a two level model of the occurrence of financial pyramid (bubbles) has been considered. We also considered the mathematical analogy of this model to Bose condensation. In the present paper, we explain why Ponzi schemes and bubbles result in a crisis in real economics. In Maslov (2005), the law of incr…

Study meromorphic k-differentials with prescribed singularities on Riemann surfaces.

Proposes a method for private aggregation in heterogeneous federated learning.

We apply random matrix theory to compare correlation matrix estimators C obtained from emerging market data. The correlation matrices are constructed from 10 years of daily data for stocks listed on the Johannesburg Stock Exchange (JSE) from January 1993 to December 2002. We test the spectral properties of C against ra…

Unified interpretation of sub-Riemannian Gauss-Bonnet theorem for surfaces in 3D contact manifolds.

Despite the fact that an intraday market price distribution is not normal, the random walk model of price behaviour is as important for the understanding of basic principles of the market as the pendulum model is a starting point of many fundamental theories in physics. This model is a good zero order approximation for…