arXiv research

Overview of non-stochastic-gradient SA algorithms in signal processing and ML.

This paper studies a non-stochastic version of Fernholz's stochastic portfolio theory for a simple model of stock markets with continuous price paths. It establishes non-stochastic versions of the most basic results of stochastic portfolio theory and discusses connections with Stroock-Varadhan martingales.

Motivated by the task of hyperparameter optimization, we introduce the non-stochastic best-arm identification problem. Within the multi-armed bandit literature, the cumulative regret objective enjoys algorithms and analyses for both the non-stochastic and stochastic settings while to the best of our knowledge, the best…

We consider the problem of controlling a possibly unknown linear dynamical system with adversarial perturbations, adversarially chosen convex loss functions, and partially observed states, known as non-stochastic control. We introduce a controller parametrization based on the denoised observations, and prove that apply…

New algorithm reduces decision switching in dynamic environments.

We propose a framework to perform streaming covariance selection. Our approach employs regularization constraints where a time-varying sparsity parameter is iteratively estimated via stochastic gradient descent. This allows for the regularization parameter to be efficiently learnt in an online manner. The proposed fram…

New algorithm achieves optimal regret in non-stochastic control, showing stochasticity is not beneficial.

Improved privacy analysis for stochastic gradient descent.

We consider derivative-free algorithms for stochastic and non-stochastic convex optimization problems that use only function values rather than gradients. Focusing on non-asymptotic bounds on convergence rates, we show that if pairs of function values are available, algorithms for $d$-dimensional optimization that use …

Study finds optimal regret bound for multi-armed bandit problem with expert advice.

We present an extension of Monte Carlo Tree Search (MCTS) that strongly increases its efficiency for trees with asymmetry and/or loops. Asymmetric termination of search trees introduces a type of uncertainty for which the standard upper confidence bound (UCB) formula does not account. Our first algorithm (MCTS-T), whic…

Stochastic gradient descent is the method of choice for large-scale machine learning problems, by virtue of its light complexity per iteration. However, it lags behind its non-stochastic counterparts with respect to the convergence rate, due to high variance introduced by the stochastic updates. The popular Stochastic …

New algorithm minimizes worst-case regret in uncertain, time-varying dynamics.

We consider the non-stochastic version of the (cooperative) multi-player multi-armed bandit problem. The model assumes no communication at all between the players, and furthermore when two (or more) players select the same action this results in a maximal loss. We prove the first $\sqrt{T}$-type regret guarantee for th…

Adaptive gradient methods such as AdaGrad and its variants update the stepsize in stochastic gradient descent on the fly according to the gradients received along the way; such methods have gained widespread use in large-scale optimization for their ability to converge robustly, without the need to fine-tune the stepsi…

Unified analysis of FL with arbitrary client participation.

This paper makes a small step towards a non-stochastic version of superhedging duality relations in the case of one traded security with a continuous price path. Namely, we prove the coincidence of game-theoretic and measure-theoretic expectation for lower semicontinuous positive functionals. We consider a new broad de…

This work addresses the problem of regret minimization in non-stochastic multi-armed bandit problems, focusing on performance guarantees that hold with high probability. Such results are rather scarce in the literature since proving them requires a large deal of technical effort and significant modifications to the sta…

Develops a new framework for conditional independence.

Study optimal arms in combinatorial bandits with semi-bandit feedback and finite budget.

New method tracks significant arm switches to improve bandit algorithms.

MetaGrad adapts multiple learning rates for faster online optimization.

New algorithm controls systems with unknown, changing losses.

We design differentially private algorithms for the problem of online linear optimization in the full information and bandit settings with optimal $\tilde{O}(\sqrt{T})$ regret bounds. In the full-information setting, our results demonstrate that $ε$-differential privacy may be ensured for free -- in particular, the reg…

Unified framework for analyzing online convex optimization across various settings.

SGD vs quasi-Newton optimization in neural networks: different landscapes, different generalizability.

We consider a non-stochastic online learning approach to price financial options by modeling the market dynamic as a repeated game between the nature (adversary) and the investor. We demonstrate that such framework yields analogous structure as the Black-Scholes model, the widely popular option pricing model in stochas…

New algorithm controls linear systems with bandit feedback, achieving optimal regret.

Performance of machine learning algorithms depends critically on identifying a good set of hyperparameters. While recent approaches use Bayesian optimization to adaptively select configurations, we focus on speeding up random search through adaptive resource allocation and early-stopping. We formulate hyperparameter op…

New model for display advertising with stochastic and adversarial components.

The problem of non-stationarity in financial markets is discussed and related to the dynamic nature of price volatility. A new measure is proposed for estimation of the current asset volatility. A simple and illustrative explanation is suggested of the emergence of significant serial autocorrelations in volatility and …

Forré introduces a new conditional independence notion for mixed variables.

We demonstrate that, in the classical non-stochastic regret minimization problem with $d$ decisions, gains and losses to be respectively maximized or minimized are fundamentally different. Indeed, by considering the additional sparsity assumption (at each stage, at most $s$ decisions incur a nonzero outcome), we derive…

This paper establishes a non-stochastic analogue of the celebrated result by Dubins and Schwarz about reduction of continuous martingales to Brownian motion via time change. We consider an idealized financial security with continuous price path, without making any stochastic assumptions. It is shown that typical price …

Optimal trading patterns adjust based on market efficiency and slippage costs.

Guaranteed bounds for posterior inference in probabilistic programs.

Efficient exploration is a long-standing problem in sensorimotor learning. Major advances have been demonstrated in noise-free, non-stochastic domains such as video games and simulation. However, most of these formulations either get stuck in environments with stochastic dynamics or are too inefficient to be scalable t…

One of the applications of center-based clustering algorithms such as K-Means is partitioning data points into K clusters. In some examples, the feature space relates to the underlying problem we are trying to solve, and sometimes we can obtain a suitable feature space. Nevertheless, while K-Means is one of the most ef…

A new federated bandit problem with multiple adversaries, solved with a near-optimal algorithm.

Bandit algorithms have been predominantly analyzed in the convex setting with function-value based stationary regret as the performance measure. In this paper, motivated by online reinforcement learning problems, we propose and analyze bandit algorithms for both general and structured nonconvex problems with nonstation…

Learning theory has largely focused on two main learning scenarios. The first is the classical statistical setting where instances are drawn i.i.d. from a fixed distribution and the second scenario is the online learning, completely adversarial scenario where adversary at every time step picks the worst instance to pro…

Industrial process control systems try to keep an output variable within a given tolerance around a target value. PID control systems have been widely used in industry to control input variables in order to reach this goal. However, this kind of Transfer Function based approach cannot be extended to complex processes w…

We consider high dimensional sparse regression, and develop strategies able to deal with arbitrary -- possibly, severe or coordinated -- errors in the covariance matrix $X$. These may come from corrupted data, persistent experimental errors, or malicious respondents in surveys/recommender systems, etc. Such non-stochas…

Study shows gradient variance increases during deep learning training, contrary to common belief.

Researchers compare different gradient methods for ridge regression, finding conjugate gradients have similar performance.

Gradient descent implicitly regularizes neural networks by penalizing large loss gradients.

Improved sampling method using regularized Stein Variational Gradient Flow.

In this work we revisit gradient regularization for adversarial robustness with some new ingredients. First, we derive new per-image theoretical robustness bounds based on local gradient information. These bounds strongly motivate input gradient regularization. Second, we implement a scaleable version of input gradient…