arXiv research

New algorithms solve nonconvex-concave minimax problems without parameter knowledge.

New algorithm clusters GRBs into two groups: short and long duration.

Develops a parameter-free SGD algorithm with optimal convergence rate.

New method achieves optimal performance without needing problem parameters.

Develops parameter-free online mirror descent for optimal dynamic regret.

The power of sparse signal modeling with learned over-complete dictionaries has been demonstrated in a variety of applications and fields, from signal processing to statistical inference and machine learning. However, the statistical properties of these models, such as under-fitting or over-fitting given sets of data, …

New algorithm closes empirical gap in PFSGD performance.

Simpler, parameter-free AdaGrad and Adam variants with convergence guarantees.

PARMESAN learns from memory without parameters for fast, efficient continual learning.

We introduce several new black-box reductions that significantly improve the design of adaptive and parameter-free online learning algorithms by simplifying analysis, improving regret guarantees, and sometimes even improving runtime. We reduce parameter-free online learning to online exp-concave optimization, we reduce…

A new algorithm solves minimax problems without needing parameters.

New algorithm provides robust uncertainty quantification without parameter tuning.

We propose clustering algorithms based on a recently developed geometric digraph family called cluster catch digraphs (CCDs). These digraphs are used to devise clustering methods that are hybrids of density-based and graph-based clustering methods. CCDs are appealing digraphs for clustering, since they estimate the num…

A parameter-free PGD algorithm for convex optimization.

New algorithm for online learning with noisy side observations.

New algorithms achieve high-probability parameter-free regret in online convex optimization with heavy-tailed data.

We consider the problem of unconstrained online convex optimization (OCO) with sub-exponential noise, a strictly more general problem than the standard OCO. In this setting, the learner receives a subgradient of the loss functions corrupted by sub-exponential noise and strives to achieve optimal regret guarantee, witho…

DoWG optimizer automatically adapts to convex and nonsmooth problems without tuning.

Subspace clustering, the task of clustering high dimensional data when the data points come from a union of subspaces is one of the fundamental tasks in unsupervised machine learning. Most of the existing algorithms for this task require prior knowledge of the number of clusters along with few additional parameters whi…

We introduce an efficient algorithmic framework for model selection in online learning, also known as parameter-free online learning. Departing from previous work, which has focused on highly structured function classes such as nested balls in Hilbert space, we propose a generic meta-algorithm framework that achieves o…

Algorithm refines matrix ratings using hierarchical graph clustering.

COPOD detects outliers efficiently and interpretable using copulas.

PF-LaCG removes the need for knowing smoothness and strong convexity parameters for locally accelerated CG.

Squint bound improved by removing $\ln \ln T$ term.

Paper proposes an ensemble of attacks to evaluate adversarial robustness more reliably.

In this paper we propose a new parameter-free method for trajectory classification which finds the best trajectory partition and dimension combination for robust trajectory classification. Preliminary experiments show that our approach is very promising.

Robust PCA, the problem of PCA in the presence of outliers has been extensively investigated in the last few years. Here we focus on Robust PCA in the column sparse outlier model. The existing methods for column sparse outlier model assumes either the knowledge of the dimension of the lower dimensional subspace or the …

New algorithms reduce online learning regret by tracking gradient variation.

Independent component analysis (ICA) is a powerful method for blind source separation based on the assumption that sources are statistically independent. Though ICA has proven useful and has been employed in many applications, complete statistical independence can be too restrictive an assumption in practice. Additiona…

New method tracks shifts in infinite-armed bandits without prior knowledge.

Traditional recognition methods typically require large, artificially-balanced training classes, while few-shot learning methods are tested on artificially small ones. In contrast to both extremes, real world recognition problems exhibit heavy-tailed class distributions, with cluttered scenes and a mix of coarse and fi…

Adaptive conformal inference without data exchangeability assumptions.

New RL algorithm tackles nonstationary MDPs with linear approximations and varying rewards.

Due to the growing ubiquity of unlabeled data, learning with unlabeled data is attracting increasing attention in machine learning. In this paper, we propose a novel semi-supervised kernel learning method which can seamlessly combine manifold structure of unlabeled data and Regularized Least-Squares (RLS) to learn a ne…

Algorithm optimizes functions without parameters, converging to global minima.

New algorithms minimize regret in SSP with optimal sparse updates.

Robust PCA, the problem of PCA in the presence of outliers has been extensively investigated in the last few years. Here we focus on Robust PCA in the outlier model where each column of the data matrix is either an inlier or an outlier. Most of the existing methods for this model assumes either the knowledge of the dim…

Efficient CF approach using fast adaptive PCA for recommender systems.

CANDECOMP/PARAFAC (CP) tensor factorization of incomplete data is a powerful technique for tensor completion through explicitly capturing the multilinear latent factors. The existing CP algorithms require the tensor rank to be manually specified, however, the determination of tensor rank remains a challenging problem e…

New algorithm optimally identifies best arm in both stochastic and adversarial settings.

AdaSDBO solves decentralized bilevel optimization without problem parameters, achieving competitive performance.

We propose the first contextual bandit algorithm that is parameter-free, efficient, and optimal in terms of dynamic regret. Specifically, our algorithm achieves dynamic regret $\mathcal{O}(\min\{\sqrt{ST}, Δ^{\frac{1}{3}}T^{\frac{2}{3}}\})$ for a contextual bandit problem with $T$ rounds, $S$ switches and $Δ$ total var…

A new Q-learning variant reduces underestimation bias in deep reinforcement learning.

This paper describes a new parameter-free online learning algorithm for changing environments. In comparing against algorithms with the same time complexity as ours, we obtain a strongly adaptive regret bound that is a factor of at least $\sqrt{\log(T)}$ better, where $T$ is the time horizon. Empirical results show tha…

AuToMATo clusters data without tuning parameters, outperforming others.

Automatic machine learning performs predictive modeling with high performing machine learning tools without human interference. This is achieved by making machine learning applications parameter-free, i.e. only a dataset is provided while the complete model selection and model building process is handled internally thr…

We introduce a simple and effective method for regularizing large convolutional neural networks. We replace the conventional deterministic pooling operations with a stochastic procedure, randomly picking the activation within each pooling region according to a multinomial distribution, given by the activities within th…

High throughput biomedical measurements normally capture multiple overlaid biologically relevant signals and often also signals representing different types of technical artefacts like e.g. batch effects. Signal identification and decomposition are accordingly main objectives in statistical biomedical modeling and data…