New private algorithms for online learning improve regret in high privacy regimes.
problem Private online learning from experts and convex optimization.
method Transformed lazy algorithms for differential privacy.
result Improved regret bounds for DP-OPE and DP-OCO.
Paper addresses private online convex optimization with optimal algorithms in various geometries and high-dimensional bandits.
problem Private online convex optimization with streaming and continual release data.
method Proposes a private variant of online Frank-Wolfe algorithm with recursive gradients for variance reduction.
result Achieves optimal excess risk in linear time for 1 < p ≤ 2 1<p\leq 2 1 < p ≤ 2 and state-of-the-art excess risk for 2 < p ≤ ∞ 2<p\leq\infty 2 < p ≤ ∞ . Bounds on Littlestone dimension for private learning and online prediction.
problem Understanding the Littlestone dimension of composed classes for private learning.
method Deriving bounds on Littlestone dimension and transforming private learners.
result Improved bounds on sample complexity for private learning.
This work analyzes DP-SGD for online LDP problems with practical convergence rates.
problem Analyzing DP-SGD for online LDP problems with practical convergence rates.
method Developed a general framework for online LDP model in stochastic optimization problems, conducted non-asymptotic convergence analysis.
result Comprehensive non-asymptotic convergence analysis of the proposed estimators in finite-sample situations.
We study the relationship between the notions of differentially private learning and online learning in games. Several recent works have shown that differentially private learning implies online learning, but an open problem of Neel, Roth, and Wu \cite{NeelAaronRoth2018} asks whether this implication is {\it efficient}…
In this paper we develop the first algorithms for online submodular minimization that preserve differential privacy under full information feedback and bandit feedback. A sequence of T T T submodular functions over a collection of n n n elements arrive online, and at each timestep the algorithm must choose a subset of $[n]…
New findings show limitations in converting private learning to online learning efficiently.
problem Limitations in converting private learning to online learning efficiently.
method Assuming one-way functions, we show an efficient conversion from pure-private learners to online learners is impossible.
result Efficient conversion from pure-private learners to online learners is impossible under certain assumptions.
New algorithms for privately learning decision lists and halfspaces.
problem Private learning of decision lists and halfspaces.
method Differentially private algorithms for PAC and online models.
result Private algorithms match or surpass non-private guarantees.
Study extends learnability equivalence to multi-class and regression, overcoming binary classification limits.
problem Equivalence of online and private learnability in multi-class and regression settings.
method Introduced a novel Littlestone dimension variant and threshold functions for multi-class classification.
result Online learnability implies private learnability in multi-class classification but not in regression.
New algorithm learns regression models privately under growth condition.
problem Private learning of nonparametric regression models.
method Novel filtering procedure to output stable hypotheses for nonparametric function classes.
result Established first nonparametric private learnability guarantee for diverging fat shattering dimensions.
Private classification and online prediction are shown to be equivalent.
problem Learning with differential privacy and online prediction equivalence.
method Introducing global stability and proving equivalence between online learnability and private PAC learnability.
result Every concept class with finite Littlestone dimension can be learned by a differentially-private algorithm.
Algorithm learns optimal coordination for strategic agents in uncertain settings.
problem Optimizing rewards for strategic agents with private types and actions.
method Combines delaying mechanism, reward angle estimation, and LinUCB algorithm.
result Near optimal regret bound of O ~ ( T ) \tilde{O}(\sqrt{T}) O ~ ( T ) for learning optimal policy. New DP algorithms achieve near-optimal regret bounds for online learning problems.
problem Online learning problems with zero-loss solutions and differential privacy constraints.
method Developed new Differentially Private algorithms with near-optimal regret bounds.
result Achieved near-optimal regret bounds for various online prediction and convex optimization problems.
New algorithm reduces regret in private online learning with optimal gap-dependent rate.
problem Optimal gap-dependent regret rate for private stochastic decision-theoretic online learning.
method Horizon-free pure-DP algorithm with exponential block partitioning and softmax selection.
result Explicit regret bound of 1000 ⋅ ( log K Δ min + log K ε ) 1000 \cdot (\frac{\log K}{Δ_{\min}}+\frac{\log K}{\varepsilon}) 1000 ⋅ ( Δ m i n l o g K + ε l o g K ) . Improved privacy and efficiency in online convex optimization.
problem Differentially private online convex optimization in high dimensions.
method Improves upon Agarwal et al. [2023] by reducing dimension factors and removing smoothness requirement.
result Best known rates for ( ε , δ ) (ε, δ) ( ε , δ ) -differentially private online convex optimization in the regime of ε not being very small. Locally private online quantile regression method addresses privacy constraints.
problem Estimating and inferring quantile regression under local differential privacy constraints.
method Developed a finite-alphabet channel where users compute local contributions, apply randomized response, and send reports. A public decoder corrects distortion and reconstructs inputs for averaging.
result Established local privacy, decoder unbiasedness, consistency, asymptotic normality, and inference for scalar contrasts.
AriaNN enables private deep learning with minimal interaction and reduced key sizes.
problem Private deep learning with minimal interaction and reduced key sizes.
method Semi-honest 2-party computation protocol with function secret sharing, optimized primitives for neural network operations.
result Efficient private comparison for ReLU operations with reduced key size and improved performance.
In this paper, we consider the problem of preserving privacy in the online learning setting. We study the problem in the online convex programming (OCP) framework---a popular online learning setting with several interesting theoretical and practical implications---while using differential privacy as the formal privacy …
Review of privacy-preserving linear models for high-dimensional data.
problem Overfitting and data memorization in high-dimensional linear models.
method Comprehensive comparison of optimization techniques for differentially private high-dimensional linear models.
result Coordinate-optimized algorithms perform best in empirical tests.
Locally private algorithm improves online federated learning with correlated noise.
problem Privacy-preserving online federated learning with non-IID data.
method Locally differentially private algorithm using temporally correlated noise.
result Established dynamic regret bound for nonconvex loss functions.
New algorithms improve online prediction from experts with privacy constraints.
problem Online prediction from experts under privacy constraints.
method Proposed and analyzed new algorithms for approximate and pure differential privacy.
result Achieved improved regret bounds for various adversaries.
Decentralized Online Learning (online learning in decentralized networks) attracts more and more attention, since it is believed that Decentralized Online Learning can help the data providers cooperatively better solve their online problems without sharing their private data to a third party or other providers. Typical…
Differentially Private algorithms often need to select the best amongst many candidate options. Classical works on this selection problem require that the candidates' goodness, measured as a real-valued score function, does not change by much when one person's data changes. In many applications such as hyperparameter o…
Improved privacy in RL with near-optimal regret bounds.
problem Privacy-preserving reinforcement learning in personalized decision-making systems.
method Differentially private algorithm based on LSVI-UCB++ with privacy-preserving techniques.
result Achieved a near-optimal regret bound of O(d * sqrt(H^3 * K) + H^(15/4) * d^(7/6) * K^(1/2) / ε).
AdOBEst-LDP improves privacy-preserving frequency estimation for categorical data.
problem Estimating categorical distributions online while preserving privacy.
method AdOBEst-LDP uses adaptive randomized response mechanism to enhance future data utility.
result AdOBEst-LDP selects optimal subset for LDP mechanism with high probability.
This paper achieves optimal regret bounds for locally private linear contextual bandit.
problem Designing locally private linear contextual bandit algorithms with optimal regret bounds.
method New algorithmic and analytical ideas, including mean absolute deviation analysis and layered principal component regression.
result Achieves an i l d e O ( T ) ilde O(\sqrt{T}) i l d e O ( T ) regret upper bound for locally private linear contextual bandit. Private online FDR control for adaptive testing under differential privacy.
problem Controlling false discoveries in adaptive multiple hypothesis testing with privacy constraints.
method Private online algorithms based on non-private results, ensuring privacy and statistical performance.
result Strong guarantees for privacy and statistical performance in FDR and power.
Distributed estimation and learning with privacy preserved.
problem Privacy-preserving distributed estimation and learning in a networked environment.
method Linear aggregation schemes with differential privacy constraints.
result Noise minimizes convergence time to best estimates, using Laplace noise.
We design differentially private algorithms for the problem of online linear optimization in the full information and bandit settings with optimal O ~ ( T ) \tilde{O}(\sqrt{T}) O ~ ( T ) regret bounds. In the full-information setting, our results demonstrate that ε ε ε -differential privacy may be ensured for free -- in particular, the reg…
This paper tackles optimal bidding strategies in adversarial first-price auctions.
problem How to bid optimally and efficiently in adversarial first-price auctions.
method Developed a minimax optimal online bidding algorithm leveraging expert-chaining structure and exploiting product structure.
result Achieved an O ~ ( T ) \widetilde{O}(\sqrt{T}) O ( T ) regret, superior to existing algorithms. Strategic information is valuable either by remaining private (for instance if it is sensitive) or, on the other hand, by being used publicly to increase some utility. These two objectives are antagonistic and leaking this information might be more rewarding than concealing it. Unlike classical solutions that focus on …
Improved private learning for Littlestone classes with a doubly-exponential mistake bound.
problem Private learning of Littlestone classes with approximate differential privacy constraints.
method Combines refined interpretation of irreducibility technique, improved sparse selection algorithm, and Exponential Mechanism.
result Achieved a mistake bound of \(\tilde{O}(d^{9.5} \cdot \log(T))\) for online learning of Littlestone classes.
Study improves privacy-preserving online prediction from experts with speed-ups.
problem Privacy-preserving online prediction from experts with speed-ups.
method Differentially private federated online prediction algorithms.
result Achieves m m m -fold regret speed-up with low-loss expert in federated setting. Transform non-private e-values into differentially private ones.
problem Leaking sensitive data through non-private e-values.
method Developed a novel biased multiplicative noise mechanism.
result Differentially private e-values maintain strong statistical power and asymptotic equivalence to non-private ones.
In this paper, we study the non-stationary online second price auction problem. We assume that the seller is selling the same type of items in T T T rounds by the second price auction, and she can set the reserve price in each round. In each round, the bidders draw their private values from a joint distribution unknown t…
New algorithm achieves small-loss bounds in online learning with improved rates.
problem Achieving strong stability in online learning algorithms.
method Introduces ρ ρ ρ -separation to enforce strong stability, unifying previous approaches. result Oracle-efficient algorithm achieves small-loss bounds with improved rates.
New algorithm for personalized healthcare with privacy guarantees.
problem Online exploration in reinforcement learning with differential privacy constraints.
method ε-JDP algorithm with privately released exploration bonuses and visitation statistics.
result Regret bound of O ( S A H 2 T + S 2 A H 3 / ε ) O(\sqrt{SAH^2T}+S^2AH^3/ε) O ( S A H 2 T + S 2 A H 3 / ε ) matching information-theoretic lower bound. Algorithm learns to bid optimally in repeated first-price auctions with censored feedback.
problem Learning to bid optimally in repeated first-price auctions with incomplete feedback.
method Developed an algorithm exploiting the specific feedback structure and payoff function of first-price auctions.
result Achieved a near-optimal O ~ ( T ) \widetilde{O}(\sqrt{T}) O ( T ) regret bound for first-price auctions. New methods for private statistical inference under local differential privacy.
problem Private statistical inference for population means with bounded observations.
method Nonparametric, nonasymptotic statistical inference using a generalized randomized response mechanism.
result Private confidence intervals and sequences for population means under LDP constraints.
Study online linear regression with paid noise reduction.
problem Online linear regression with noisy features and the ability to pay for reduced noise.
method Analyzes regret against optimal predictor, uses matrix martingale concentration.
result Optimal regret rates for known and unknown noise covariance.
Privacy-preserving inference for clinical trials using differential privacy.
problem Balancing knowledge sharing and privacy in healthcare data.
method Differential privacy (DP) applied to log-linear belief updates in distributed settings.
result Differentially private, distributed inference methods outperform existing techniques.
Differentially private hyperparameter tuning improves privacy in machine learning.
problem Hyperparameter tuning leaks private information through selected configurations.
method Local Bayesian optimization using Gaussian Process surrogate for private gradient approximation.
result DP-GIBO converges to locally optimal hyperparameters with polynomial dimensional dependence.
Improved learning algorithms with privacy using smoothed analysis.
problem Designing robust and private learning algorithms.
method Smoothed analysis of adversarial and differentially private learning.
result Stronger regret and privacy error guarantees with smoothed adversaries.
This paper tackles online strategic decision making with asymmetry and knowledge transportability.
problem Strategic decision making with information asymmetry and knowledge transportability challenges.
method Developed a sample-efficient algorithm for online learning under these conditions.
result Proved sample complexity of O ( 1 / ε 2 ) O(1/ε^2) O ( 1/ ε 2 ) for learning an ε ε ε -optimal policy. This work proves DP learnability implies online learnability for general classification tasks.
problem Link between differential privacy and online learning for general classification tasks.
method Establishes Ramsey-type theorems for trees to prove DP learnability implies online learnability.
result DP learnability implies online learnability for general classification tasks.
Optimizes private statistics with noisy methods.
problem Private inference in statistical models.
method Noisy optimization for M-estimators and confidence regions.
result Private estimators converge to non-private ones with high probability.
We study differentially private (DP) algorithms for stochastic convex optimization (SCO). In this problem the goal is to approximately minimize the population loss given i.i.d. samples from a distribution over convex and Lipschitz loss functions. A long line of existing work on private convex optimization focuses on th…
We present a provably optimal differentially private algorithm for the stochastic multi-arm bandit problem, as opposed to the private analogue of the UCB-algorithm [Mishra and Thakurta, 2015; Tossou and Dimitrakakis, 2016] which doesn't meet the recently discovered lower-bound of Ω ( K log ( T ) ε ) Ω\left(\frac{K\log(T)}ε \right) Ω ( ε K l o g ( T ) ) [Shar…