A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Derives a method to optimize high-dimensional functions on low-dimensional manifolds.
Recent advances in derivative-free optimization allow efficient approximation of the global-optimal solutions of sophisticated functions, such as functions with many local optima, non-differentiable and non-continuous functions. This article describes the ZOOpt (Zeroth Order Optimization) toolbox that provides efficien…
Efficiently tunes hyperparameters with dynamic accuracy method.
We study derivative-free methods for policy optimization over the class of linear policies. We focus on characterizing the convergence rate of these methods when applied to linear-quadratic systems, and study various settings of driving noise and reward feedback. We show that these methods provably converge to within a…
Derivative-free method solves stochastic optimization problems with noisy objectives and constraints.
The problem of stochastic convex optimization with bandit feedback (in the learning community) or without knowledge of gradients (in the optimization community) has received much attention in recent years, in the form of algorithms and performance upper bounds. However, much less is known about the inherent complexity …
Study analyzes derivative-free loss method for solving PDEs and fluid problems.
In this paper, we will provide an introduction to the derivative-free optimization algorithms which can be potentially applied to train deep learning models. Existing deep learning model training is mostly based on the back propagation algorithm, which updates the model variables layers by layers with the gradient desc…
A new imitation learning method uses random search for simple policies, outperforming complex models.
New method tackles inexact bilevel optimization for faster parameter learning.
BOBYQA optimizes deep networks with fewer queries than other methods.
Blade uses diffusion priors to accurately and calibratedly infer complex systems.
We develop an algorithm for minimizing a function using batched function value measurements at each of rounds by using classifiers to identify a function's sublevel set. We show that sufficiently accurate classifiers can achieve linear convergence rates, and show that the convergence rate is tied to the difficu…
In this paper, we consider derivative free optimization problems, where the objective function is smooth but is computed with some amount of noise, the function evaluations are expensive and no derivative information is available. We are motivated by policy optimization problems in reinforcement learning that have rece…
Enhanced DFO using adaptive batch-based FD estimates.
CBO interprets as SGD, leading to global convergence for nonconvex functions.
New sampling strategy improves TR algorithms for stochastic optimization.
This paper provides lower bounds on the convergence rate of Derivative Free Optimization (DFO) with noisy function evaluations, exposing a fundamental and unavoidable gap between the performance of algorithms with access to gradients and those with access to only function evaluations. However, there are situations in w…
Optimizes variational autoencoder for detecting missing data in Mars rover transmissions.
A dissertation on scalable projection-free optimization methods.
This paper discusses online algorithms for inverse dynamics modelling in robotics. Several model classes including rigid body dynamics (RBD) models, data-driven models and semiparametric models (which are a combination of the previous two classes) are placed in a common framework. While model classes used in the litera…
Derivative-free optimization has become an important technique used in machine learning for optimizing black-box models. To conduct updates without explicitly computing gradient, most current approaches iteratively sample a random search direction from Gaussian distribution and compute the estimated gradient along that…
Derives RL framework for systems without velocity or acceleration measurements.
Study optimizes zero-order strongly convex function minimization with higher order smoothness.
In this paper, we study the problem of monotone (weakly) DR-submodular continuous maximization. While previous methods require the gradient information of the objective function, we propose a derivative-free algorithm LDGM for the first time. We define and to characterize how close a function is to continuous D…
In this paper, we will provide an introduction to the derivative-free optimization algorithms which can be potentially applied to train deep learning models. Existing deep learning model training is mostly based on the back propagation algorithm, which updates the model variables layers by layers with the gradient desc…
We learn recurrent neural network optimizers trained on simple synthetic functions by gradient descent. We show that these learned optimizers exhibit a remarkable degree of transfer in that they can be used to efficiently optimize a broad range of derivative-free black-box functions, including Gaussian process bandits,…
Recently, neural networks trained as optimizers under the "learning to learn" or meta-learning framework have been shown to be effective for a broad range of optimization tasks including derivative-free black-box function optimization. Recurrent neural networks (RNNs) trained to optimize a diverse set of synthetic non-…
Machine learning applications often require hyperparameter tuning. The hyperparameters usually drive both the efficiency of the model training process and the resulting model quality. For hyperparameter tuning, machine learning algorithms are complex black-boxes. This creates a class of challenging optimization problem…
This paper presents the application of a newly developed nature-inspired metaheuristic optimization method, namely the Adaptive Wind Driven Optimization (AWDO), to the training of feedforward artificial neural networks (NN) and presents a discussion into the future research of AWDO implementation in Deep Learning (DL).…
We consider stochastic zero-order optimization problems, which arise in settings from simulation optimization to reinforcement learning. We propose an adaptive sampling quasi-Newton method where we estimate the gradients of a stochastic function using finite differences within a common random number framework. We emplo…
This study introduces a framework for the forecasting, reconstruction and feature engineering of multivariate processes along with its renewable energy applications. We integrate derivative-free optimization with an ensemble of sequence-to-sequence networks and design a new resampling technique called additive resampli…
We consider derivative-free black-box global optimization of expensive noisy functions, when most of the randomness in the objective is produced by a few influential scalar random inputs. We present a new Bayesian global optimization algorithm, called Stratified Bayesian Optimization (SBO), which uses this strong depen…
Study uses spectral risk for learning with heavy-tailed data.
BOOOM optimizes orthonormal matrices without needing gradients.
EnKG solves inverse problems without derivatives, using diffusion models.
We apply numerical methods in combination with finite-difference-time-domain (FDTD) simulations to optimize transmission properties of plasmonic mirror color filters using a multi-objective figure of merit over a five-dimensional parameter space by utilizing novel multi-fidelity Gaussian processes approach. We compare …
New algorithm optimizes convex functions with noisy evaluations in one dimension.
Quantum algorithm improves portfolio optimization quality measured by Wasserstein distance.
The paper is devoted to modeling optimal exercise strategies of the behavior of investors and issuers working with convertible bonds. This implies solution of the problems of stock price modeling, payoff computation and min-max optimization. Stock prices (underlying asset) were modeled under the assumption of the geome…
New algorithms solve complex minimax problems without needing derivatives.
New optimization algorithm for mixed-variable problems improves efficiency.
We consider derivative-free algorithms for stochastic and non-stochastic convex optimization problems that use only function values rather than gradients. Focusing on non-asymptotic bounds on convergence rates, we show that if pairs of function values are available, algorithms for -dimensional optimization that use …
New samplers improve MCMC efficiency in high dimensions.
We introduce a deep neural network based method for solving a class of elliptic partial differential equations. We approximate the solution of the PDE with a deep neural network which is trained under the guidance of a probabilistic representation of the PDE in the spirit of the Feynman-Kac formula. The solution is giv…
New method improves zeroth-order stochastic optimization with adaptive sampling.
This paper optimizes performative risk by focusing on convex properties and developing efficient algorithms.
Bayesian optimization outperformed random search in machine learning hyperparameter tuning challenge.