arXiv research

The paper addresses learner privacy in convex optimization with feedback.

We compare the sample complexity of private learning [Kasiviswanathan et al. 2008] and sanitization~[Blum et al. 2008] under pure $ε$-differential privacy [Dwork et al. TCC 2006] and approximate $(ε,δ)$-differential privacy [Dwork et al. Eurocrypt 2006]. We show that the sample complexity of these tasks under approxima…

The paper introduces a privacy-preserving method for estimating treatment effects that maintains accuracy.

We formulate a private learning model to study an intrinsic tradeoff between privacy and query complexity in sequential learning. Our model involves a learner who aims to determine a scalar value, $v^*$, by sequentially querying an external database and receiving binary responses. In the meantime, an adversary observes…

We study statistical risk minimization problems under a privacy model in which the data is kept confidential even from the learner. In this local privacy framework, we establish sharp upper and lower bounds on the convergence rates of statistical estimation procedures. As a consequence, we exhibit a precise tradeoff be…

We introduce a simple framework for designing private boosting algorithms. We give natural conditions under which these algorithms are differentially private, efficient, and noise-tolerant PAC learners. To demonstrate our framework, we use it to construct noise-tolerant and private PAC learners for large-margin halfspa…

Improved private agnostic learning with near-optimal sample complexity.

Based on differential privacy (DP) framework, we introduce and unify privacy definitions for the multi-armed bandit algorithms. We represent the framework with a unified graphical model and use it to connect privacy definitions. We derive and contrast lower bounds on the regret of bandit algorithms satisfying these def…

Two-stage architecture helps learners collaborate on data with privacy and transmission constraints.

Lower bounds on query complexity for reconstructing private learner's training data.

DMF improves POI recommendation privacy and efficiency.

We study the query complexity of a learner-private sequential learning problem, motivated by the privacy and security concerns due to eavesdropping that arise in practical applications such as pricing and Federated Learning. A learner tries to estimate an unknown scalar value, by sequentially querying an external datab…

Popular approaches to differential privacy, such as the Laplace and exponential mechanisms, calibrate randomised smoothing through global sensitivity of the target non-private function. Bounding such sensitivity is often a prohibitively complex analytic calculation. As an alternative, we propose a straightforward sampl…

In this work, we define a collaborative and privacy-preserving machine teaching paradigm with multiple distributed teachers. We focus on consensus super teaching. It aims at organizing distributed teachers to jointly select a compact while informative training subset from data hosted by the teachers to make a learner l…

The paper examines how to test if two learning algorithms produce similar outcomes.

We present a private learner for halfspaces over an arbitrary finite domain $X\subset \mathbb{R}^d$ with sample complexity $mathrm{poly}(d,2^{\log^*|X|})$. The building block for this learner is a differentially private algorithm for locating an approximate center point of $m>\mathrm{poly}(d,2^{\log^*|X|})$ points -- a…

We study the relationship between the notions of differentially private learning and online learning in games. Several recent works have shown that differentially private learning implies online learning, but an open problem of Neel, Roth, and Wu \cite{NeelAaronRoth2018} asks whether this implication is {\it efficient}…

In machine learning, boosting is one of the most popular methods that designed to combine multiple base learners to a superior one. The well-known Boosted Decision Tree classifier, has been widely adopted in many areas. In the big data era, the data held by individual and entities, like personal images, browsing histor…

Active learning method reduces label queries for positive examples.

Contextual bandits are online learners that, given an input, select an arm and receive a reward for that arm. They use the reward as a learning signal and aim to maximize the total reward over the inputs. Contextual bandits are commonly used to solve recommendation or ranking problems. This paper considers a learning s…

In large-scale statistical learning, data collection and model fitting are moving increasingly toward peripheral devices---phones, watches, fitness trackers---away from centralized data collection. Concomitant with this rise in decentralized data are increasing challenges of maintaining privacy while allowing enough in…

Asynchronous algorithms reduce privacy costs in distributed machine learning.

New method reduces communication in distributed learning, improving privacy and utility.

Private distribution learning with public data, leveraging sample compression schemes.

Improved private learning for Littlestone classes with a doubly-exponential mistake bound.

We study the contextual linear bandit problem, a version of the standard stochastic multi-armed bandit (MAB) problem where a learner sequentially selects actions to maximize a reward which depends also on a user provided per-round context. Though the context is chosen arbitrarily or adversarially, the reward is assumed…

Private classification and online prediction are shown to be equivalent.

Federated learning linked to mean-field games for large-scale learning.

Paper proposes a privacy-preserving method for estimating complex models.

Estimates conversion probabilities from click sequences with privacy constraints.

Study on computational aspects of replicable learning, bridging statistical and algorithmic perspectives.

Improved private learning of halfspaces with reduced sample complexity.

A new validation scheme improves Federated Learning performance.

A new DP approach for Conformal Prediction using quantile search.

We study the query complexity of Bayesian Private Learning: a learner wishes to locate a random target within an interval by submitting queries, in the presence of an adversary who observes all of her queries but not the responses. How many queries are necessary and sufficient in order for the learner to accurately est…

PILLAR improves SP learning with less private data.

Two private algorithms improve domain adaptation with privacy guarantees.

Generative AI tasks analyzed for text, images, audio, video, code, and molecules.

We present novel, computationally efficient, and differentially private algorithms for two fundamental high-dimensional learning problems: learning a multivariate Gaussian and learning a product distribution over the Boolean hypercube in total variation distance. The sample complexity of our algorithms nearly matches t…

Differentially-private Bayes consistency rule for binary classification and density estimation.

DOMKL learns functions from IoT data with minimal regret and consensus constraints.

This paper compares two loss functions for learning from aggregated responses and introduces an interpolating estimator.

Randomness is crucial for stability in learning and statistics, especially for differential privacy.

Optimal algorithm converts weak to strong learner with less data.

Inverse reinforcement learning (IRL) enables an agent to learn complex behavior by observing demonstrations from a (near-)optimal policy. The typical assumption is that the learner's goal is to match the teacher's demonstrated behavior. In this paper, we consider the setting where the learner has its own preferences th…

Meta clustering categorizes learners for collaborative learning.

New meta-learners estimate time-varying treatment effects without model assumptions.

Improves privacy amplification by shuffling for differential privacy.