This paper analyzes user-level local differential privacy in distributed systems.
problem The relationship between user-level and item-level local differential privacy under the local model is complex.
method The paper analyzes the mean estimation problem and applies it to stochastic optimization, classification, and regression. It proposes adaptive strategies to achieve optimal performance at all privacy levels.
result The proposed methods are minimax optimal up to logarithmic factors and show that user-level DP can lead to faster convergence rates than item-level DP.
Paper studies user-level differential privacy in federated linear contextual bandits.
problem Federated learning with user-level differential privacy constraints.
method Unified federated bandits framework, CDP and LDP definitions, ROBIN algorithm.
result Near-optimal learning under user-level CDP with privacy budget and number of clients.
New algorithms protect user-level privacy in learning tasks.
problem Protecting user-level privacy in learning tasks with stringent constraints.
method Proposes algorithms for learning tasks under user-level differential privacy constraints.
result Privacy cost decreases as users provide more samples or as the number of users increases.
Novel mean estimation method under user-level differential privacy reduces noise in continual mean estimates.
problem Maintaining accurate running mean estimates under user-level differential privacy.
method Developed a novel mean estimation specific factorization under approximate differential privacy.
result Achieved asymptotically lower mean-squared error bounds in continual mean estimation.
Estimates population mean from user-level data with privacy, accounting for heterogeneity.
problem Heterogeneous user data with varying numbers of data points and distributions.
method Simple model of heterogeneous user data, differential privacy mechanism for estimation.
result Asymptotic optimality of the proposed estimator and general lower bounds on error.
Develops privacy-preserving methods for longitudinal linear regression.
problem Protecting individual information in longitudinal data with privacy-preserving statistics.
method Proposes a user-level private regression estimator and a privatized covariance estimator for longitudinal linear regression under user-level differential privacy.
result Establishes theoretical guarantees for practical user-level differential privacy estimation and inference in longitudinal linear regression.
New privacy mechanism for user-level discrete distributions with reduced penalty.
problem Achieving privacy for all items of a single user in practical applications.
method Study of learning discrete distributions with user-level differential privacy, proposing a new mechanism with reduced privacy penalty.
result Proposed mechanism reduces privacy penalty to i l d e O ( k / ( m α 2 ) + k / m ε α ) ilde{\mathcal{O}}(k/(mα^2) + k/\sqrt{m}εα) i l d e O ( k / ( m α 2 ) + k / m ε α ) , significantly smaller than standard mechanisms. A new algorithm estimates mean under varying user data sizes with local differential privacy.
problem Mean estimation with user-level local differential privacy under varying data sizes.
method Distribution-aware mean estimation algorithm for users with varying data sizes.
result Upper and lower bounds on the worst-case risk for mean estimation are derived.
New algorithm optimizes privacy and utility in multi-task learning with skewed data.
problem Privacy constraints in multi-task learning with uneven data distribution.
method Adaptive reweighting of privacy budget allocation among tasks.
result Significant improvement in utility with state-of-the-art performance on benchmarks.
Private ALS method improves matrix completion with tighter rates and better privacy.
problem Differential privacy in matrix completion for user-level privacy.
method Joint differentially private ALS method with tighter sample complexity and privacy trade-offs.
result Achieves nearly optimal sample complexity and best privacy/utility trade-off.
Novel algorithm reduces privacy noise in machine learning.
problem High privacy noise in machine learning algorithms.
method Robust statistics, specifically median and trimmed mean, to bound sensitivity of SGD iterates.
result Improved privacy-utility trade-off with reduced noise and computational efficiency.
New method uses Winsorized mean estimators for privacy-preserving statistics on dependent data.
problem Privacy-preserving statistics on dependent data with sensitive information.
method Adapting noisy Winsorized mean estimators to handle dependence via log-Sobolev inequalities.
result Asymptotic and finite sample guarantees for item-level and user-level mean estimation similar to \iid{} settings.
Study shows FL reduces unintended memorization by clustering data and using strong user-level privacy.
problem Unintended memorization in federated learning.
method Examined the effect of clustering data and using strong user-level differential privacy in FL.
result Clustering data and strong user-level differential privacy reduce unintended memorization.
Paper improves privacy-accuracy balance in federated learning.
problem Privacy-accuracy tradeoffs in federated learning.
method Personalized federated learning with joint differential privacy.
result Coordination of local and centralized learning improves accuracy while maintaining privacy.
CDA framework infers channel influence from aggregated data without user identifiers.
problem Lack of user-level path data due to privacy regulations and platform restrictions.
method CDA integrates PCMCI for causal discovery and Structural Causal Model for effect estimation.
result CDA achieves strong accuracy in estimating channel influence, even under structural uncertainty.
Structured subsampling improves privacy in deep time series forecasting.
problem Incompatible privacy guarantees with time series forecasting.
method Structured subsampling of sequential data for privacy amplification.
result Structured subsampling enables training with strong privacy guarantees.
Locally private online quantile regression method addresses privacy constraints.
problem Estimating and inferring quantile regression under local differential privacy constraints.
method Developed a finite-alphabet channel where users compute local contributions, apply randomized response, and send reports. A public decoder corrects distortion and reconstructs inputs for averaging.
result Established local privacy, decoder unbiasedness, consistency, asymptotic normality, and inference for scalar contrasts.
Improved locally private sparse estimation with multiple samples per user.
problem Challenges in high-dimensional locally private sparse estimation.
method Proposes a framework for user-level locally private sparse linear regression with multiple samples per user.
result Eliminates the dependency of dimensionality on error bounds, achieving tighter error bounds.
New DP mechanisms improve ML privacy-utility-computational tradeoffs.
problem Improving privacy in machine learning with multiple passes over data.
method Formalized DP for adaptive streams, extended matrix factorization techniques, Fourier-transform-based mechanism.
result Substantial improvements in privacy-utility-computational tradeoffs over previous methods.
FedIRT enables privacy-preserving psychometric estimation without centralizing data.
problem Privacy and data governance concerns in centralized IRT estimation.
method Federated Item Response Theory (FedIRT) and FedIRT-DP for differentially private estimation.
result FedIRT matches accuracy of centralized estimators while preserving privacy.
Generative framework unifies and improves personalized learning and estimation methods.
problem Statistical heterogeneity in client data motivates personalized learning models.
method Generative framework unifies and suggests new personalized learning and estimation algorithms.
result AdaPeD algorithm numerically outperforms known algorithms.
Existing approaches for training neural networks with user-level differential privacy (e.g., DP Federated Averaging) in federated learning (FL) settings involve bounding the contribution of each user's model update by clipping it to some constant value. However there is no good a priori setting of the clipping norm acr…
New attack recovers user-level information from large batch images.
problem Recovering private information from user-level gradients in distributed learning.
method Proposes a gradient inversion attack using a denoising diffusion model as a prior.
result Demonstrates recovery of realistic facial images and private attributes.
Paper trains language models without memorizing user data.
problem Training production-scale language models with user privacy.
method Federated Learning with Differentially Private Federated Averaging (DP-FedAvg).
result Demonstrated training of a production NWP model with user-level differential privacy.
This paper shows how differential privacy can be achieved naturally in federated learning over fading channels without artificial noise.
problem Achieving differential privacy in federated learning over fading channels without artificial noise.
method Study of AirFL over multiple-access fading channels with a multi-antenna base station, deriving novel bounds on differential privacy.
result DP can be achieved naturally in federated learning over fading channels without artificial noise, revealing convergence-privacy trade-offs.
This paper improves privacy in federated learning without a trusted server.
problem Privacy in federated learning with silos that distrust each other.
method Introduces Inter-Silo Record-Level Differential Privacy (ISRL-DP) and accelerated algorithms for convex and smooth losses.
result Achieves optimal privacy and accuracy tradeoffs in federated learning.
We establish the first benchmark for federated learning with differential privacy in ASR, achieving competitive performance.
problem Challenges in training large transformer models for ASR in federated learning with differential privacy.
method Per-layer clipping and layer-wise gradient normalization to mitigate gradient heterogeneity.
result FL with DP is viable in ASR with strong privacy guarantees, achieving competitive performance.
Recommender systems often face heterogeneous datasets containing highly personalized historical data of users, where no single model could give the best recommendation for every user. We observe this ubiquitous phenomenon on both public and private datasets and address the model selection problem in pursuit of optimizi…
Model clusters authors and topics in short texts like social media posts.
problem Analysis of short texts is difficult due to brevity and lack of context.
method Expands Latent Dirichlet Allocation to model word dependencies and cluster users.
result Improves topic and user clustering, outperforming traditional methods.
Social Media has seen a tremendous growth in the last decade and is continuing to grow at a rapid pace. With such adoption, it is increasingly becoming a rich source of data for opinion mining and sentiment analysis. The detection and analysis of sentiment in social media is thus a valuable topic and attracts a lot of …
We introduce the BriarPatch, a pixel-space intervention that obscures sensitive attributes from representations encoded in pre-trained classifiers. The patches encourage internal model representations not to encode sensitive information, which has the effect of pushing downstream predictors towards exhibiting demograph…
This paper describes a practical system for Multi Touch Attribution (MTA) for use by a publisher of digital ads. We developed this system for JD.com, an eCommerce company, which is also a publisher of digital ads in China. The approach has two steps. The first step ('response modeling') fits a user-level model for purc…
Improves privacy amplification by shuffling for differential privacy.
problem Enhancing privacy guarantees in systems with anonymous data contributions.
method Theoretical and numerical analysis of Rényi differential privacy parameters and privacy amplification by shuffling.
result First asymptotically optimal analysis of Rényi differential privacy parameters for shuffled outputs.
Proposes a privacy-preserving recommendation system using matrix factorization and differential privacy.
problem Privacy leakage in recommendation systems when anonymizing user data is not sufficient.
method Uses matrix factorization and differential privacy via the Gaussian mechanism.
result Demonstrates excellent utility for privacy-preserving recommendation systems.
This work optimizes mean estimation under varying privacy constraints.
problem Mean estimation with heterogeneous privacy constraints.
method Proposes an algorithm for mean estimation under different privacy levels for users.
result Shows a saturation phenomenon in performance as privacy levels are relaxed.
Improves privacy guarantees by analyzing randomness in privacy-preserving mechanisms.
problem Balancing user privacy and business constraints in privacy-preserving mechanisms.
method Analyzes explicit and implicit randomness in privacy mechanisms and proposes a probabilistic calibration method.
result Proposes privacy at risk, providing stronger privacy guarantees with quantifiable risks.
The paper analyzes and proposes methods for privately sharing individual privacy losses using per-instance differential privacy.
problem The standard differential privacy framework provides a worst-case bound that may not accurately reflect individual privacy losses.
method The paper analyzes per-instance differential privacy and proposes methods to privately and accurately publish per-instance privacy losses.
result The methods privately and accurately publish per-instance differential privacy losses with minimal additional privacy cost.
This work optimizes mean estimation under varying user privacy demands.
problem Mean estimation with heterogeneous privacy levels.
method Proposes an algorithm that is minimax optimal and has near-linear run-time.
result Privacy requirements of the most stringent users dictate overall error rates.
A new privacy accountant for Gaussian differential privacy measures individual privacy losses.
problem Bounding differential privacy loss for each participant in data analysis.
method Developed a privacy accountant for adaptive compositions of randomised mechanisms using Gaussian differential privacy.
result Provided optimal bounds for the Gaussian mechanism and constructed an approximative individual privacy accountant.
Paper simplifies DP composition for adaptive privacy budgets, enabling better privacy and accuracy in deep learning.
problem Tension between efficiency and flexibility in DP composition theorems.
method Rényi Differential Privacy (RDP) for adaptive privacy budgets, proving simpler composition theorem with smaller constants.
result Practical DP composition for adaptive privacy budgets, enabling better privacy and accuracy in deep learning.
This paper bounds min-entropy leakage for Blowfish privacy using graph symmetries.
problem Bounding min-entropy leakage for Blowfish privacy mechanisms.
method Organizing analysis over symmetrical partitions corresponding to orbits of graph automorphism groups.
result Demonstrates a construction meeting the bound with asymptotic equality, showing tightness.
New federated f f f -differential privacy for collaborative learning.
problem Privacy in federated learning.
method Introducing federated f f f -differential privacy and proposing a generic private federated learning framework. result Proves federated f f f -differential privacy provides privacy guarantee on each record of one client's data. We summarize the potential impact that the European Union's new General Data Protection Regulation will have on the routine use of machine learning algorithms. Slated to take effect as law across the EU in 2018, it will restrict automated individual decision-making (that is, algorithms that make decisions based on user…
Differential privacy is a statistical concept that can be explained through hypothesis testing.
problem Formalizing differential privacy as a statistical concept.
method Using David Blackwell's informativeness theorem, the paper shows differential privacy can be understood through hypothesis testing.
result The definition of f f f -differential privacy provides a unified framework for analyzing privacy bounds. A new method for tighter privacy loss accounting in adaptive analyses.
problem Ensuring individual privacy in adaptive analyses while staying within a privacy budget.
method A personalized privacy loss estimate and a Rényi differential privacy filter.
result Personalized privacy loss accounting can be practical and tighter than existing methods.
Paper tackles federated learning with privacy, enhancing target data analysis.
problem Heterogeneity and privacy of distributed data in federated learning.
method Formulates federated differential privacy, studies statistical problems under privacy constraints.
result Federated differential privacy offers a balance between privacy and knowledge transfer.
Unified framework for subsampling mechanisms with tighter privacy guarantees.
problem Improving privacy in machine learning models through subsampling.
method Conditional optimal transport for deriving mechanism-specific subsampling guarantees.
result Tighter privacy bounds for subsampled mechanisms compared to traditional methods.
New findings show privacy affects generalization error in a non-monotonic way.
problem Privacy and robustness in distributed learning.
method Theoretical analysis and matching lower/upper bounds on algorithmic stability.
result Generalization error is non-monotonically affected by privacy, depending on noise level.