New private algorithms for online learning improve regret in high privacy regimes.
problem Private online learning from experts and convex optimization.
method Transformed lazy algorithms for differential privacy.
result Improved regret bounds for DP-OPE and DP-OCO.
We study the relationship between the notions of differentially private learning and online learning in games. Several recent works have shown that differentially private learning implies online learning, but an open problem of Neel, Roth, and Wu \cite{NeelAaronRoth2018} asks whether this implication is {\it efficient}…
New findings show limitations in converting private learning to online learning efficiently.
problem Limitations in converting private learning to online learning efficiently.
method Assuming one-way functions, we show an efficient conversion from pure-private learners to online learners is impossible.
result Efficient conversion from pure-private learners to online learners is impossible under certain assumptions.
New algorithms for privately learning decision lists and halfspaces.
problem Private learning of decision lists and halfspaces.
method Differentially private algorithms for PAC and online models.
result Private algorithms match or surpass non-private guarantees.
Bounds on Littlestone dimension for private learning and online prediction.
problem Understanding the Littlestone dimension of composed classes for private learning.
method Deriving bounds on Littlestone dimension and transforming private learners.
result Improved bounds on sample complexity for private learning.
Private classification and online prediction are shown to be equivalent.
problem Learning with differential privacy and online prediction equivalence.
method Introducing global stability and proving equivalence between online learnability and private PAC learnability.
result Every concept class with finite Littlestone dimension can be learned by a differentially-private algorithm.
In this paper, we consider the problem of preserving privacy in the online learning setting. We study the problem in the online convex programming (OCP) framework---a popular online learning setting with several interesting theoretical and practical implications---while using differential privacy as the formal privacy …
Locally private algorithm improves online federated learning with correlated noise.
problem Privacy-preserving online federated learning with non-IID data.
method Locally differentially private algorithm using temporally correlated noise.
result Established dynamic regret bound for nonconvex loss functions.
Decentralized Online Learning (online learning in decentralized networks) attracts more and more attention, since it is believed that Decentralized Online Learning can help the data providers cooperatively better solve their online problems without sharing their private data to a third party or other providers. Typical…
New algorithm learns regression models privately under growth condition.
problem Private learning of nonparametric regression models.
method Novel filtering procedure to output stable hypotheses for nonparametric function classes.
result Established first nonparametric private learnability guarantee for diverging fat shattering dimensions.
Study extends learnability equivalence to multi-class and regression, overcoming binary classification limits.
problem Equivalence of online and private learnability in multi-class and regression settings.
method Introduced a novel Littlestone dimension variant and threshold functions for multi-class classification.
result Online learnability implies private learnability in multi-class classification but not in regression.
Locally private online quantile regression method addresses privacy constraints.
problem Estimating and inferring quantile regression under local differential privacy constraints.
method Developed a finite-alphabet channel where users compute local contributions, apply randomized response, and send reports. A public decoder corrects distortion and reconstructs inputs for averaging.
result Established local privacy, decoder unbiasedness, consistency, asymptotic normality, and inference for scalar contrasts.
Paper addresses private online convex optimization with optimal algorithms in various geometries and high-dimensional bandits.
problem Private online convex optimization with streaming and continual release data.
method Proposes a private variant of online Frank-Wolfe algorithm with recursive gradients for variance reduction.
result Achieves optimal excess risk in linear time for 1 < p ≤ 2 1<p\leq 2 1 < p ≤ 2 and state-of-the-art excess risk for 2 < p ≤ ∞ 2<p\leq\infty 2 < p ≤ ∞ . This work analyzes DP-SGD for online LDP problems with practical convergence rates.
problem Analyzing DP-SGD for online LDP problems with practical convergence rates.
method Developed a general framework for online LDP model in stochastic optimization problems, conducted non-asymptotic convergence analysis.
result Comprehensive non-asymptotic convergence analysis of the proposed estimators in finite-sample situations.
AriaNN enables private deep learning with minimal interaction and reduced key sizes.
problem Private deep learning with minimal interaction and reduced key sizes.
method Semi-honest 2-party computation protocol with function secret sharing, optimized primitives for neural network operations.
result Efficient private comparison for ReLU operations with reduced key size and improved performance.
Algorithm learns optimal coordination for strategic agents in uncertain settings.
problem Optimizing rewards for strategic agents with private types and actions.
method Combines delaying mechanism, reward angle estimation, and LinUCB algorithm.
result Near optimal regret bound of O ~ ( T ) \tilde{O}(\sqrt{T}) O ~ ( T ) for learning optimal policy. New algorithm reduces regret in private online learning with optimal gap-dependent rate.
problem Optimal gap-dependent regret rate for private stochastic decision-theoretic online learning.
method Horizon-free pure-DP algorithm with exponential block partitioning and softmax selection.
result Explicit regret bound of 1000 ⋅ ( log K Δ min + log K ε ) 1000 \cdot (\frac{\log K}{Δ_{\min}}+\frac{\log K}{\varepsilon}) 1000 ⋅ ( Δ m i n l o g K + ε l o g K ) . New DP algorithms achieve near-optimal regret bounds for online learning problems.
problem Online learning problems with zero-loss solutions and differential privacy constraints.
method Developed new Differentially Private algorithms with near-optimal regret bounds.
result Achieved near-optimal regret bounds for various online prediction and convex optimization problems.
In this paper we develop the first algorithms for online submodular minimization that preserve differential privacy under full information feedback and bandit feedback. A sequence of T T T submodular functions over a collection of n n n elements arrive online, and at each timestep the algorithm must choose a subset of $[n]…
New algorithms improve online prediction from experts with privacy constraints.
problem Online prediction from experts under privacy constraints.
method Proposed and analyzed new algorithms for approximate and pure differential privacy.
result Achieved improved regret bounds for various adversaries.
Improved learning algorithms with privacy using smoothed analysis.
problem Designing robust and private learning algorithms.
method Smoothed analysis of adversarial and differentially private learning.
result Stronger regret and privacy error guarantees with smoothed adversaries.
This work proves DP learnability implies online learnability for general classification tasks.
problem Link between differential privacy and online learning for general classification tasks.
method Establishes Ramsey-type theorems for trees to prove DP learnability implies online learnability.
result DP learnability implies online learnability for general classification tasks.
Private online FDR control for adaptive testing under differential privacy.
problem Controlling false discoveries in adaptive multiple hypothesis testing with privacy constraints.
method Private online algorithms based on non-private results, ensuring privacy and statistical performance.
result Strong guarantees for privacy and statistical performance in FDR and power.
Distributed estimation and learning with privacy preserved.
problem Privacy-preserving distributed estimation and learning in a networked environment.
method Linear aggregation schemes with differential privacy constraints.
result Noise minimizes convergence time to best estimates, using Laplace noise.
Improved private learning for Littlestone classes with a doubly-exponential mistake bound.
problem Private learning of Littlestone classes with approximate differential privacy constraints.
method Combines refined interpretation of irreducibility technique, improved sparse selection algorithm, and Exponential Mechanism.
result Achieved a mistake bound of \(\tilde{O}(d^{9.5} \cdot \log(T))\) for online learning of Littlestone classes.
New algorithm achieves small-loss bounds in online learning with improved rates.
problem Achieving strong stability in online learning algorithms.
method Introduces ρ ρ ρ -separation to enforce strong stability, unifying previous approaches. result Oracle-efficient algorithm achieves small-loss bounds with improved rates.
New insights into learning from distributional adversaries and private data.
problem Understanding minimal assumptions for learning and generalization under distributional constraints.
method Generalized smoothness as a characterization of learnability and privacy under distributional adversaries.
result Near complete characterization of families that admit learnability and privacy under distributional adversaries.
Transform non-private e-values into differentially private ones.
problem Leaking sensitive data through non-private e-values.
method Developed a novel biased multiplicative noise mechanism.
result Differentially private e-values maintain strong statistical power and asymptotic equivalence to non-private ones.
In this paper, we study the non-stationary online second price auction problem. We assume that the seller is selling the same type of items in T T T rounds by the second price auction, and she can set the reserve price in each round. In each round, the bidders draw their private values from a joint distribution unknown t…
Differentially Private algorithms often need to select the best amongst many candidate options. Classical works on this selection problem require that the candidates' goodness, measured as a real-valued score function, does not change by much when one person's data changes. In many applications such as hyperparameter o…
Improved privacy in RL with near-optimal regret bounds.
problem Privacy-preserving reinforcement learning in personalized decision-making systems.
method Differentially private algorithm based on LSVI-UCB++ with privacy-preserving techniques.
result Achieved a near-optimal regret bound of O(d * sqrt(H^3 * K) + H^(15/4) * d^(7/6) * K^(1/2) / ε).
AdOBEst-LDP improves privacy-preserving frequency estimation for categorical data.
problem Estimating categorical distributions online while preserving privacy.
method AdOBEst-LDP uses adaptive randomized response mechanism to enhance future data utility.
result AdOBEst-LDP selects optimal subset for LDP mechanism with high probability.
Study improves privacy-preserving online prediction from experts with speed-ups.
problem Privacy-preserving online prediction from experts with speed-ups.
method Differentially private federated online prediction algorithms.
result Achieves m m m -fold regret speed-up with low-loss expert in federated setting. Improved privacy and efficiency in online convex optimization.
problem Differentially private online convex optimization in high dimensions.
method Improves upon Agarwal et al. [2023] by reducing dimension factors and removing smoothness requirement.
result Best known rates for ( ε , δ ) (ε, δ) ( ε , δ ) -differentially private online convex optimization in the regime of ε not being very small. New methods for private statistical inference under local differential privacy.
problem Private statistical inference for population means with bounded observations.
method Nonparametric, nonasymptotic statistical inference using a generalized randomized response mechanism.
result Private confidence intervals and sequences for population means under LDP constraints.
This paper achieves optimal regret bounds for locally private linear contextual bandit.
problem Designing locally private linear contextual bandit algorithms with optimal regret bounds.
method New algorithmic and analytical ideas, including mean absolute deviation analysis and layered principal component regression.
result Achieves an i l d e O ( T ) ilde O(\sqrt{T}) i l d e O ( T ) regret upper bound for locally private linear contextual bandit. New algorithm for personalized healthcare with privacy guarantees.
problem Online exploration in reinforcement learning with differential privacy constraints.
method ε-JDP algorithm with privately released exploration bonuses and visitation statistics.
result Regret bound of O ( S A H 2 T + S 2 A H 3 / ε ) O(\sqrt{SAH^2T}+S^2AH^3/ε) O ( S A H 2 T + S 2 A H 3 / ε ) matching information-theoretic lower bound. Study shows observing order book can significantly improve online market making performance.
problem Online market making with private valuations and limited feedback.
method Introduces action-dependent feedback model and proposes elimination-based and explore-then-perturb algorithms.
result Achieves O ( T ) O(\sqrt{T}) O ( T ) regret bounds with high probability in various settings. This paper tackles online strategic decision making with asymmetry and knowledge transportability.
problem Strategic decision making with information asymmetry and knowledge transportability challenges.
method Developed a sample-efficient algorithm for online learning under these conditions.
result Proved sample complexity of O ( 1 / ε 2 ) O(1/ε^2) O ( 1/ ε 2 ) for learning an ε ε ε -optimal policy. This paper tackles optimal bidding strategies in adversarial first-price auctions.
problem How to bid optimally and efficiently in adversarial first-price auctions.
method Developed a minimax optimal online bidding algorithm leveraging expert-chaining structure and exploiting product structure.
result Achieved an O ~ ( T ) \widetilde{O}(\sqrt{T}) O ( T ) regret, superior to existing algorithms. Privacy-preserving inference for clinical trials using differential privacy.
problem Balancing knowledge sharing and privacy in healthcare data.
method Differential privacy (DP) applied to log-linear belief updates in distributed settings.
result Differentially private, distributed inference methods outperform existing techniques.
Algorithm learns to bid optimally in repeated first-price auctions with censored feedback.
problem Learning to bid optimally in repeated first-price auctions with incomplete feedback.
method Developed an algorithm exploiting the specific feedback structure and payoff function of first-price auctions.
result Achieved a near-optimal O ~ ( T ) \widetilde{O}(\sqrt{T}) O ( T ) regret bound for first-price auctions. Review of privacy-preserving linear models for high-dimensional data.
problem Overfitting and data memorization in high-dimensional linear models.
method Comprehensive comparison of optimization techniques for differentially private high-dimensional linear models.
result Coordinate-optimized algorithms perform best in empirical tests.
We design differentially private algorithms for the problem of online linear optimization in the full information and bandit settings with optimal O ~ ( T ) \tilde{O}(\sqrt{T}) O ~ ( T ) regret bounds. In the full-information setting, our results demonstrate that ε ε ε -differential privacy may be ensured for free -- in particular, the reg…
Adaptive truncation improves privacy in online Bayesian estimation.
problem Ensuring privacy in online Bayesian estimation of a static parameter.
method Sequential Monte Carlo, adaptive truncation, Thompson sampling.
result Adaptive truncation reduces privacy-preserving noise, enabling more accurate estimation.
In this paper, we resolve many of the key algorithmic questions regarding robustness, memory efficiency, and differential privacy of tensor decomposition. We propose simple variants of the tensor power method which enjoy these strong properties. We present the first guarantees for online tensor power method which has a…
Contextual bandits are online learners that, given an input, select an arm and receive a reward for that arm. They use the reward as a learning signal and aim to maximize the total reward over the inputs. Contextual bandits are commonly used to solve recommendation or ranking problems. This paper considers a learning s…
This paper studies trade-offs in private prediction methods.
problem Leakage of training data information in machine learning predictions.
method Private training and private prediction methods with trade-offs.
result Private training methods outperform private prediction methods in various settings.