arXiv research

Survey of universal portfolio techniques for minimizing investment regret.

A universal learner achieves best rates for all distributions.

Convolutional neural networks (CNN) have become one of the most popular machine learning tools and are being applied in various tasks, however, CNN models are vulnerable to universal perturbations, which are usually human-imperceptible but can cause natural images to be misclassified with high probability. One of the s…

Universal online optimization for dynamic environments using uniclass prediction.

NP-iMCMC algorithm for nonparametric models in universal PPLs.

We extend a recently proposed 1-nearest-neighbor based multiclass learning algorithm and prove that our modification is universally strongly Bayes-consistent in all metric spaces admitting any such learner, making it an "optimistically universal" Bayes-consistent learner. This is the first learning algorithm known to e…

Transformers enable in-context learning with guarantees for a wide range of tasks.

A new risk budgeting scheme derived from universal portfolio theory.

New algorithm achieves consistent learning from context in bandit problems.

Characterizes concept classes for optimistic online learning.

New algorithms for regression with adversarial responses on various metric spaces.

To deal with changing environments, a new performance measure -- adaptive regret, defined as the maximum static regret over any interval, was proposed in online learning. Under the setting of online convex optimization, several algorithms have been successfully developed to minimize the adaptive regret. However, existi…

To a rational homology sphere graph manifold one can associate a weighted tree invariant called splice diagram. It was shown earlier that the splice diagram determines the universal abelian cover of the manifold. We will in this article turn the proof of this in to an algorithm to explicitly construct the universal abe…

New findings on universal learning in contextual bandits with adversarial rewards.

New method makes robust estimators work without knowing corruption levels.

Novel approach to universal online learning for bounded losses, closing open problems.

In this paper, we study adaptive online convex optimization, and aim to design a universal algorithm that achieves optimal regret bounds for multiple common types of loss functions. Existing universal methods are limited in the sense that they are optimal for only a subclass of loss functions. To address this limitatio…

We show that the Subgradient algorithm is universal for online learning on the simplex in the sense that it simultaneously achieves $O(\sqrt N)$ regret for adversarial costs and $O(1)$ pseudo-regret for i.i.d costs. To the best of our knowledge this is the first demonstration of a universal algorithm on the simplex tha…

Recently, deep learning based natural language processing techniques are being extensively used to deal with spam mail, censorship evaluation in social networks, among others. However, there is only a couple of works evaluating the vulnerabilities of such deep neural networks. Here, we go beyond attacks to investigate,…

Paper explores universal rates of ERM in machine learning.

Paper establishes universal lower bounds and optimal rates for clustering sub-exponential mixture models.

The theory of optimal trading under proportional transaction costs has been considered from a variety of perspectives. In this paper, we show that all the results can be interpreted using a universal law, illustrating the results in trading algorithm design.

Sig-Splines model uses signatures and splines for time series data, achieving universality and convexity.

Prototype rules simplify multiclass classification in metric spaces, achieving consistency and reduced complexity.

Algorithm decides if pseudo-Anosov flows have perfect fits.

Lazy Gradient Descent outperforms existing polytope algorithms in pseudo-regret.

Framework for universal graph function approximators outperforms existing methods.

Given a state-of-the-art deep neural network classifier, we show the existence of a universal (image-agnostic) and very small perturbation vector that causes natural images to be misclassified with high probability. We propose a systematic algorithm for computing universal perturbations, and show that state-of-the-art …

OptFormer learns universal HPO from diverse datasets.

A new method for faster optimization of noisy functions.

New method achieves both universality and adaptivity in online convex optimization.

Estimating symmetric properties of a distribution, e.g. support size, coverage, entropy, distance to uniformity, are among the most fundamental problems in algorithmic statistics. While each of these properties have been studied extensively and separate optimal estimators are known for each, in striking recent work, Ac…

A fast method computes class-specific adversarial perturbations for deep networks.

This paper explores the limits of deep learning in poly-time.

Paper shows how online betting algorithms' regret can be used to create tight confidence sequences.

We present a universal algorithm for online trading in Stock Market which performs asymptotically at least as good as any stationary trading strategy that computes the investment at each step using a fixed function of the side information that belongs to a given RKHS (Reproducing Kernel Hilbert Space). Using a universa…

Given a state-of-the-art deep neural network text classifier, we show the existence of a universal and very small perturbation vector (in the embedding space) that causes natural text to be misclassified with high probability. Unlike images on which a single fixed-size adversarial perturbation can be found, text is of …

Market sectors play a key role in the efficient flow of capital through the modern Global economy. We analyze existing sectorization heuristics, and observe that the most popular - the GICS (which informs the S&P 500), and the NAICS (published by the U.S. Government) - are not entirely quantitatively driven, but rather…

We consider the problem of universal joint clustering and registration of images and define algorithms using multivariate information functionals. We first study registering two images using maximum mutual information and prove its asymptotic optimality. We then show the shortcomings of pairwise registration in multi-i…

This paper improves support recovery in universal one-bit compressed sensing.

A Hilbert space embedding for probability measures has recently been proposed, wherein any probability measure is represented as a mean element in a reproducing kernel Hilbert space (RKHS). Such an embedding has found applications in homogeneity testing, independence testing, dimensionality reduction, etc., with the re…

This paper improves support recovery in universal one-bit compressed sensing with fewer measurements.

Reduces bounded loss learning to binary classification.

We study the problem of minimizing a strongly convex, smooth function when we have noisy estimates of its gradient. We propose a novel multistage accelerated algorithm that is universally optimal in the sense that it achieves the optimal rate both in the deterministic and stochastic case and operates without knowledge …

Single-head transformers with a single self-attention layer can approximate any sequence-to-sequence function and are efficient under certain conditions.

Universal preconditioning reduces sequential prediction regret.

Transformers can emulate various algorithms by prompting, proving universality.

Quantum algorithm speeds up MIP solving by a near-quadratic factor.