arXiv research

Paper proves multiplicative weight updates can train neural networks without learning rate tuning.

New algorithm for online optimization over symmetric cones, unifying previous methods.

Paper adapts multiplicative weights method to Gaussian graphical models.

New algorithm provably converges to second-order stationary points in NMF.

Algorithm learns Sherrington-Kirkpatrick model parameters at low temperatures.

A model for deciding which facts to remember in a lifelong learning scenario.

We show that a simple randomized sketch of the matrix multiplicative weight (MMW) update enjoys (in expectation) the same regret bounds as MMW, up to a small constant factor. Unlike MMW, where every step requires full matrix exponentiation, our steps require only a single product of the form $e^A b$, which the Lanczos …

Study shows memory needs grow with task sequence length in continual learning.

Improved MMWU algorithm achieves instance-optimal regret bound for matrix LEA.

Motivated by applications in Game Theory, Optimization, and Generative Adversarial Networks, recent work of Daskalakis et al \cite{DISZ17} and follow-up work of Liang and Stokes \cite{LiangS18} have established that a variant of the widely used Gradient Descent/Ascent procedure, called "Optimistic Gradient Descent/Asce…

Develops theory of weightings for Lie groupoids and algebroids.

This paper analyzes implicit bias in Deep Linear Discriminant Analysis.

In distributed learning, the goal is to perform a learning task over data distributed across multiple nodes with minimal (expensive) communication. Prior work (Daume III et al., 2012) proposes a general model that bounds the communication required for learning classifiers while allowing for $\eps$ training error on lin…

In this paper, we provide a novel construction of the linear-sized spectral sparsifiers of Batson, Spielman and Srivastava [BSS14]. While previous constructions required $Ω(n^4)$ running time [BSS14, Zou12], our sparsification routine can be implemented in almost-quadratic running time $O(n^{2+\varepsilon})$. The funda…

Explains weightings along submanifolds, focusing on Lie groupoids.

We introduce a framework for performing regression between two Hilbert spaces. This is done based on Kirszbraun's extension theorem, to the best of our knowledge, the first application of this technique to supervised learning. We analyze the statistical and computational aspects of this method. We decompose this task i…

We address the challenge of designing optimal adversarial noise algorithms for settings where a learner has access to multiple classifiers. We demonstrate how this problem can be framed as finding strategies at equilibrium in a two-player, zero-sum game between a learner and an adversary. In doing so, we illustrate the…

A new method estimates parameters in heavy-tailed corrupted regression with unknown covariance and heterogeneous noise.

The online problem of computing the top eigenvector is fundamental to machine learning. In both adversarial and stochastic settings, previous results (such as matrix multiplicative weight update, follow the regularized leader, follow the compressed leader, block power method) either achieve optimal regret but run slow,…

Study online learning of quantum processes, showing feasibility for certain types.

This work analyzes how users and services adapt to reduce risk, leading to specialization.

In this paper, we study the problem of robust influence maximization in the independent cascade model under a hyperparametric assumption. In social networks users influence and are influenced by individuals with similar characteristics and as such, they are associated with some features. A recent surging research direc…

Improved algorithm reduces regret in corrupted expert advice setting.

We study the interplay between sequential decision making and avoiding discrimination against protected groups, when examples arrive online and do not follow distributional assumptions. We consider the most basic extension of classical online learning: "Given a class of predictors that are individually non-discriminato…

New algorithm reduces expert prediction regret for two experts.

New algorithm for optimizing statistical utilities in bandits.

Many generative models have to combat $\textit{missing modes}$. The conventional wisdom to this end is by reducing through training a statistical distance (such as $f$-divergence) between the generated distribution and provided data distribution. But this is more of a heuristic than a guarantee. The statistical distanc…

OMWU shows last iterate convergence in convex-concave games.

Private PGB boosts synthetic data quality using GANs and privacy techniques.

New algorithm learns multiclass concepts with finite Littlestone dimension.

Adversarial CBO optimizes under interventions by adversaries and non-stationarities.

The study explores weightings on submanifolds and their geometric properties.

This paper certifies cluster assignments from sum-of-norms clustering algorithms.

Paper tackles ranking items with a semi-random comparison graph and a monotone adversary.

Independent Component Analysis (ICA) - one of the basic tools in data analysis - aims to find a coordinate system in which the components of the data are independent. In this paper we present Multiple-weighted Independent Component Analysis (MWeICA) algorithm, a new ICA method which is based on approximate diagonalizat…

This work improves robustness guarantees for neural networks using low rank representations.

We study the problem of online clustering where a clustering algorithm has to assign a new point that arrives to one of $k$ clusters. The specific formulation we use is the $k$-means objective: At each time step the algorithm has to maintain a set of k candidate centers and the loss incurred is the squared distance bet…

Improved convergence rates for saddle-point optimization algorithms.

Unified approach for robust and heavy-tailed mean estimation in high dimensions.

We consider the problem of multi-objective maximization of monotone submodular functions subject to cardinality constraint, often formulated as $\max_{|A|=k}\min_{i\in\{1,\dots,m\}}f_i(A)$. While it is widely known that greedy methods work well for a single objective, the problem becomes much harder with multiple objec…

New OLO algorithms use Stein's method for better performance tradeoffs.

The article is devoted to investigating the application of hedging strategies to online expert weight allocation under delayed feedback. As the main result, we develop the General Hedging algorithm $\mathcal{G}$ based on the exponential reweighing of experts' losses. We build the artificial probabilistic framework and …

New loss function improves classification for imbalanced and sensitive groups.

New bounds show agnostic multiclass learning depends on two dimensions: Natarajan and Daniely-Shalev-Shwartz.

Fixed-parameter tractability of private synthetic data generation

New algorithm for robust mean estimation in high dimensions with nearly-PCA time complexity.

LGV boosts adversarial attacks by improving surrogate models.

We study the collaborative PAC learning problem recently proposed in Blum et al.~\cite{BHPQ17}, in which we have $k$ players and they want to learn a target function collaboratively, such that the learned function approximates the target function well on all players' distributions simultaneously. The quality of the col…