arXiv research

Develops a parameter-free SGD algorithm with optimal convergence rate.

Develops parameter-free online mirror descent for optimal dynamic regret.

New algorithm closes empirical gap in PFSGD performance.

New algorithm provides robust uncertainty quantification without parameter tuning.

New method achieves optimal performance without needing problem parameters.

Simpler, parameter-free AdaGrad and Adam variants with convergence guarantees.

New algorithms achieve high-probability parameter-free regret in online convex optimization with heavy-tailed data.

A parameter-free PGD algorithm for convex optimization.

We introduce several new black-box reductions that significantly improve the design of adaptive and parameter-free online learning algorithms by simplifying analysis, improving regret guarantees, and sometimes even improving runtime. We reduce parameter-free online learning to online exp-concave optimization, we reduce…

New algorithms solve nonconvex-concave minimax problems without parameter knowledge.

Parameter-free clustering method using cluster catch digraphs (CCDs).

We introduce an efficient algorithmic framework for model selection in online learning, also known as parameter-free online learning. Departing from previous work, which has focused on highly structured function classes such as nested balls in Hilbert space, we propose a generic meta-algorithm framework that achieves o…

We consider the problem of unconstrained online convex optimization (OCO) with sub-exponential noise, a strictly more general problem than the standard OCO. In this setting, the learner receives a subgradient of the loss functions corrupted by sub-exponential noise and strives to achieve optimal regret guarantee, witho…

Subspace clustering, the task of clustering high dimensional data when the data points come from a union of subspaces is one of the fundamental tasks in unsupervised machine learning. Most of the existing algorithms for this task require prior knowledge of the number of clusters along with few additional parameters whi…

A new algorithm solves minimax problems without needing parameters.

Squint bound improved by removing $\ln \ln T$ term.

COPOD detects outliers efficiently and interpretable using copulas.

PF-LaCG removes the need for knowing smoothness and strong convexity parameters for locally accelerated CG.

DoWG optimizer automatically adapts to convex and nonsmooth problems without tuning.

New algorithm clusters GRBs into two groups: short and long duration.

New algorithms reduce online learning regret by tracking gradient variation.

New RL algorithm tackles nonstationary MDPs with linear approximations and varying rewards.

Algorithm optimizes functions without parameters, converging to global minima.

New algorithm for online learning with noisy side observations.

We propose the first contextual bandit algorithm that is parameter-free, efficient, and optimal in terms of dynamic regret. Specifically, our algorithm achieves dynamic regret $\mathcal{O}(\min\{\sqrt{ST}, Δ^{\frac{1}{3}}T^{\frac{2}{3}}\})$ for a contextual bandit problem with $T$ rounds, $S$ switches and $Δ$ total var…

AuToMATo clusters data without tuning parameters, outperforming others.

This paper describes a new parameter-free online learning algorithm for changing environments. In comparing against algorithms with the same time complexity as ours, we obtain a strongly adaptive regret bound that is a factor of at least $\sqrt{\log(T)}$ better, where $T$ is the time horizon. Empirical results show tha…

New method tracks shifts in infinite-armed bandits without prior knowledge.

Robust PCA, the problem of PCA in the presence of outliers has been extensively investigated in the last few years. Here we focus on Robust PCA in the column sparse outlier model. The existing methods for column sparse outlier model assumes either the knowledge of the dimension of the lower dimensional subspace or the …

Due to the growing ubiquity of unlabeled data, learning with unlabeled data is attracting increasing attention in machine learning. In this paper, we propose a novel semi-supervised kernel learning method which can seamlessly combine manifold structure of unlabeled data and Regularized Least-Squares (RLS) to learn a ne…

New algorithm optimally identifies best arm in both stochastic and adversarial settings.

Robust PCA, the problem of PCA in the presence of outliers has been extensively investigated in the last few years. Here we focus on Robust PCA in the outlier model where each column of the data matrix is either an inlier or an outlier. Most of the existing methods for this model assumes either the knowledge of the dim…

Book introduces online learning via convex optimization, focusing on regret minimization.

AdaSDBO solves decentralized bilevel optimization without problem parameters, achieving competitive performance.

The power of sparse signal modeling with learned over-complete dictionaries has been demonstrated in a variety of applications and fields, from signal processing to statistical inference and machine learning. However, the statistical properties of these models, such as under-fitting or over-fitting given sets of data, …

We study a specific \textit{combinatorial pure exploration stochastic bandit problem} where the learner aims at finding the set of arms whose means are above a given threshold, up to a given precision, and \textit{for a fixed time horizon}. We propose a parameter-free algorithm based on an original heuristic, and prove…

We study the problem of optimizing a function under a \emph{budgeted number of evaluations}. We only assume that the function is \emph{locally} smooth around one of its global optima. The difficulty of optimization is measured in terms of 1) the amount of \emph{noise} $b$ of the function evaluation and 2) the local smo…

We show how to take any two parameter-free online learning algorithms with different regret guarantees and obtain a single algorithm whose regret is the minimum of the two base algorithms. Our method is embarrassingly simple: just add the iterates. This trick can generate efficient algorithms that adapt to many norms s…

A new algorithm reduces online exp-concave optimization runtime.

Paper proposes an ensemble of attacks to evaluate adversarial robustness more reliably.

PARMESAN learns from memory without parameters for fast, efficient continual learning.

Rapid overlay of chemical structures (ROCS) is a standard tool for the calculation of 3D shape and chemical ("color") similarity. ROCS uses unweighted sums to combine many aspects of similarity, yielding parameter-free models for virtual screening. In this report, we decompose the ROCS color force field into "color com…

In this paper we propose a new parameter-free method for trajectory classification which finds the best trajectory partition and dimension combination for robust trajectory classification. Preliminary experiments show that our approach is very promising.

We present a novel methodology able to distinguish meaningful level shifts from typical signal fluctuations. A two-stage regularization filtering can accurately identify the location of the significant level-shifts with an efficient parameter-free algorithm. The developed methodology demands low computational effort an…

We consider the problem of minimizing a convex risk with stochastic subgradients guaranteeing $ε$-locally differentially private ($ε$-LDP). While it has been shown that stochastic optimization is possible with $ε$-LDP via the standard SGD (Song et al., 2013), its convergence rate largely depends on the learning rate, w…

This work provides simple algorithms for multi-class (and multi-label) prediction in settings where both the number of examples n and the data dimension d are relatively large. These robust and parameter free algorithms are essentially iterative least-squares updates and very versatile both in theory and in practice. O…

New algorithm broadens BART models applicability.

We introduce GLR-klUCB, a novel algorithm for the piecewise iid non-stationary bandit problem with bounded rewards. This algorithm combines an efficient bandit algorithm, kl-UCB, with an efficient, parameter-free, changepoint detector, the Bernoulli Generalized Likelihood Ratio Test, for which we provide new theoretica…