arXiv research

New bounds improve generalization in learning scenarios.

New information-theoretic bounds improve machine learning generalization.

The information-theoretic analysis by Russo and Van Roy (2014) in combination with minimax duality has proved a powerful tool for the analysis of online learning algorithms in full and partial information settings. In most applications there is a tantalising similarity to the classical analysis based on mirror descent.…

Optimizes SGLD noise structure for better generalization bounds.

Unified framework for removing unwanted information from machine learning models.

Improved robust regression with clean covariates achieves better rates than Huber's model.

New framework for fair ranking with noisy protected attributes.

Information-theoretic quantities, such as conditional entropy and mutual information, are critical data summaries for quantifying uncertainty. Current widely used approaches for computing such quantities rely on nearest neighbor methods and exhibit both strong performance and theoretical guarantees in certain simple sc…

SDP approach recovers communities in multilayer hypergraphs from aggregated similarity matrices.

Algorithm recovers permutations of high-dimensional Gaussian vectors with constant correlation.

Efficiently learns polytrees with known skeleton in polynomial time and sample complexity.

We study the localization of a cluster of activated vertices in a graph, from adaptively designed compressive measurements. We propose a hierarchical partitioning of the graph that groups the activated vertices into few partitions, so that a top-down sensing procedure can identify these partitions, and hence the activa…

We present a data-driven framework called generative adversarial privacy (GAP). Inspired by recent advancements in generative adversarial networks (GANs), GAP allows the data holder to learn the privatization mechanism directly from the data. Under GAP, finding the optimal privacy mechanism is formulated as a constrain…

We develop efficient algorithms for estimating low-degree moments of unknown distributions in the presence of adversarial outliers. The guarantees of our algorithms improve in many cases significantly over the best previous ones, obtained in recent works of Diakonikolas et al, Lai et al, and Charikar et al. We also sho…

Statistical guarantees for hyperparameter selection

We solve matrix denoising with both row and column correlations, setting limits and designing optimal methods.

Optimal best-arm identification with known number of optimal arms.

We study the problem of robust subspace recovery (RSR) in the presence of adversarial outliers. That is, we seek a subspace that contains a large portion of a dataset when some fraction of the data points are arbitrarily corrupted. We first examine a theoretical estimator that is intractable to calculate and use it to …

We consider the mixed regression problem with two components, under adversarial and stochastic noise. We give a convex optimization formulation that provably recovers the true solution, and provide upper bounds on the recovery errors for both arbitrary noise and stochastic noise settings. We also give matching minimax …

New framework connects online learning to statistical learning for better generalization bounds.

Value-function approximation methods that operate in batch mode have foundational importance to reinforcement learning (RL). Finite sample guarantees for these methods often crucially rely on two types of assumptions: (1) mild distribution shift, and (2) representation conditions that are stronger than realizability. H…

Crowdsourcing systems are popular for solving large-scale labelling tasks with low-paid workers. We study the problem of recovering the true labels from the possibly erroneous crowdsourced labels under the popular Dawid-Skene model. To address this inference problem, several algorithms have recently been proposed, but …

In correlation clustering, we are given $n$ objects together with a binary similarity score between each pair of them. The goal is to partition the objects into clusters so to minimise the disagreements with the scores. In this work we investigate correlation clustering as an active learning problem: each similarity sc…

New bounds show limitations of sample-wise information-theoretic generalization.

Through the lens of information-theoretic reductions, we examine a reductions approach to fair optimization and learning where a black-box optimizer is used to learn a fair model for classification or regression. Quantifying the complexity, both statistically and computationally, of making such models satisfy the rigor…

New framework converts offline to online estimation using black-box offline estimators.

Crowdsourced data used in machine learning services might carry sensitive information about attributes that users do not want to share. Various methods have been proposed to minimize the potential information leakage of sensitive attributes while maximizing the task accuracy. However, little is known about the theory b…

We present theoretical guarantees for an alternating minimization algorithm for the dictionary learning/sparse coding problem. The dictionary learning problem is to factorize vector samples $y^{1},y^{2},\ldots, y^{n}$ into an appropriate basis (dictionary) $A^*$ and sparse vectors $x^{1*},\ldots,x^{n*}$. Our algorithm …

Information-theoretic Bayesian optimisation techniques have demonstrated state-of-the-art performance in tackling important global optimisation problems. However, current information-theoretic approaches require many approximations in implementation, introduce often-prohibitive computational overhead and limit the choi…

Develops a method to ensure fairness across multiple sensitive attributes in machine learning.

New research shows existing information-theoretic methods can't establish minimax rates for gradient descent in stochastic convex optimization.

This work connects conformal prediction to information theory for uncertainty estimation.

Information-theoretic Bayesian regret bounds of Russo and Van Roy capture the dependence of regret on prior uncertainty. However, this dependence is through entropy, which can become arbitrarily large as the number of actions increases. We establish new bounds that depend instead on a notion of rate-distortion. Among o…

Study optimizes fairness in predictive models by balancing utility and separation.

We integrate information-theoretic concepts into the design and analysis of optimistic algorithms and Thompson sampling. By making a connection between information-theoretic quantities and confidence bounds, we obtain results that relate the per-period performance of the agent with its information gain about the enviro…

We propose a semidefinite programming (SDP) algorithm for community detection in the stochastic block model, a popular model for networks with latent community structure. We prove that our algorithm achieves exact recovery of the latent communities, up to the information-theoretic limits determined by Abbe and Sandon (…

In this paper we consider an information theoretic approach for the accounting classification process. We propose a matrix formalism and an algorithm for calculations of information theoretic measures associated to accounting classification. The formalism may be useful for further generalizations and computer-based imp…

Efficient algorithm for contextual bandits with first-order guarantees.

Robustly estimates mean in incomplete data with corrupted examples.

Privacy amplification improved through contraction coefficients and $E_γ$-divergence.

Paper analyzes InstaHide's security, recovering all private images with provable guarantee.

An efficient LDP protocol for QMLE with improved practicality and theoretical guarantees.

We study the problem of high-dimensional linear regression in a robust model where an $ε$-fraction of the samples can be adversarially corrupted. We focus on the fundamental setting where the covariates of the uncorrupted samples are drawn from a Gaussian distribution $\mathcal{N}(0, Σ)$ on $\mathbb{R}^d$. We give near…

Paper studies vertex correspondence recovery in correlated graphs with node features.

Proposes a method for private aggregation in heterogeneous federated learning.

Study reveals mutual information is crucial for understanding algorithm performance in stochastic convex optimization.

New tighter bounds for learning algorithms from Steinke & Zakynthinou's supersample setting.

Abstract reviews algorithms for multi-index models, focusing on polynomial-time methods and their limitations.