arXiv research

New bounds found for optimizing non-convex functions with noisy data.

Study efficient graph optimization with noisy data.

Simplifies noisy convex optimization with a new algorithm.

Paper presents an efficient algorithm for estimating Lipschitz functions from noisy data.

Study improves estimation of functions from noisy data using convex penalties.

New algorithm optimizes convex functions with noisy evaluations in one dimension.

The study examines how many samples are needed to minimize a noisy convex function with inaccurate gradient estimates.

No accelerated gradient method for hyperbolic convex functions.

New algorithms help machines forget old data efficiently.

Study on privacy leakage in noisy gradient descent algorithms.

We propose a novel Bayesian approach to solve stochastic optimization problems that involve finding extrema of noisy, nonlinear functions. Previous work has focused on representing possible functions explicitly, which leads to a two-step procedure of first, doing inference over the function space and second, finding th…

Convolutional neural network (CNN)-based feature learning has become state of the art, since given sufficient training data, CNN can significantly outperform traditional methods for various classification tasks. However, feature learning becomes more difficult if some training labels are noisy. With traditional regular…

We propose a method for zeroth order stochastic convex optimization that attains the suboptimality rate of $\tilde{\mathcal{O}}(n^{7}T^{-1/2})$ after $T$ queries for a convex bounded function $f:{\mathbb R}^n\to{\mathbb R}$. The method is based on a random walk (the \emph{Ball Walk}) on the epigraph of the function. Th…

Paper analyzes convergence rate of noisy Bayesian Optimization with Expected Improvement.

Paper analyzes convergence of proximal algorithm in metric spaces without geodesic convexity.

Improved private AdaGrad achieves faster convergence rates for convex functions.

Interesting theoretical associations have been established by recent papers between the fields of active learning and stochastic convex optimization due to the common role of feedback in sequential querying mechanisms. In this paper, we continue this thread in two parts by exploiting these relations for the first time …

Study on reproducibility in optimization with bounds on limits.

AGNES accelerates gradient descent with noisy gradients.

We consider variational inequalities coming from monotone operators, a setting that includes convex minimization and convex-concave saddle-point problems. We assume an access to potentially noisy unbiased values of the monotone operators and assess convergence through a compatible gap function which corresponds to the …

We address the problem of estimating a sparse low-rank matrix from its noisy observation. We propose an objective function consisting of a data-fidelity term and two parameterized non-convex penalty functions. Further, we show how to set the parameters of the non-convex penalty functions, in order to ensure that the ob…

Preconditioned non-convex gradient descent improves noisy matrix estimation.

The dueling bandit is a learning framework wherein the feedback information in the learning process is restricted to a noisy comparison between a pair of actions. In this research, we address a dueling bandit problem based on a cost function over a continuous space. We propose a stochastic mirror descent algorithm and …

New method estimates chirp parameters robustly from noisy mixtures.

This paper studies noisy low-rank matrix completion: given partial and noisy entries of a large low-rank matrix, the goal is to estimate the underlying matrix faithfully and efficiently. Arguably one of the most popular paradigms to tackle this problem is convex relaxation, which achieves remarkable efficacy in practic…

A meta-learning method learns adaptive robust loss functions for noisy labels.

A new method for distributed optimization with noisy function evaluations.

Stochastic GD converges linearly for CV@R learning under certain conditions.

New oracles improve stochastic optimization with noisy or biased measurements.

Noisy Feature Mixup improves model robustness with noise-perturbed convex combinations.

Paper tackles private optimization for non-smooth objectives efficiently.

Unified stability bounds for noisy SGD across convex and non-convex losses.

We consider the problem of global optimization of an unknown non-convex smooth function with zeroth-order feedback. In this setup, an algorithm is allowed to adaptively query the underlying function at different locations and receives noisy evaluations of function values at the queried points (i.e. the algorithm has ac…

Deep learning with noisy gradient descent outperforms linear estimators in high dimensions.

Improved function approximation for noisy data.

This paper provides lower bounds on the convergence rate of Derivative Free Optimization (DFO) with noisy function evaluations, exposing a fundamental and unavoidable gap between the performance of algorithms with access to gradients and those with access to only function evaluations. However, there are situations in w…

GONs improve predictions of maximizers from noisy black-box functions.

Symmetrizes loss functions to improve neural network robustness against noisy labels.

Algorithms for bandit convex optimization and online learning often rely on constructing noisy gradient estimates, which are then used in appropriately adjusted first-order algorithms, replacing actual gradients. Depending on the properties of the function to be optimized and the nature of ``noise'' in the bandit feedb…

In this paper, we study stochastic non-convex optimization with non-convex random functions. Recent studies on non-convex optimization revolve around establishing second-order convergence, i.e., converging to a nearly second-order optimal stationary points. However, existing results on stochastic non-convex optimizatio…

Stochastic gradient descent in continuous time (SGDCT) provides a computationally efficient method for the statistical learning of continuous-time models, which are widely used in science, engineering, and finance. The SGDCT algorithm follows a (noisy) descent direction along a continuous stream of data. The parameter …

Reconstructing polytopes with fixed facet directions from support function evaluations.

Algorithm ensures safe optimization under unknown constraints.

Fast algorithm recovers principal eigenvector from noisy matrices.

We consider a non-stationary sequential stochastic optimization problem, in which the underlying cost functions change over time under a variation budget constraint. We propose an $L_{p,q}$-variation functional to quantify the change, which yields less variation for dynamic function sequences whose changes are constrai…

A new method for faster optimization of noisy functions.

Two-stage nonconvex algorithm and convex relaxation both achieve optimal accuracy in noisy blind deconvolution.

New estimator avoids overfitting in convex regression.