Improves privacy amplification by shuffling for differential privacy.
problem Enhancing privacy guarantees in systems with anonymous data contributions.
method Theoretical and numerical analysis of Rényi differential privacy parameters and privacy amplification by shuffling.
result First asymptotically optimal analysis of Rényi differential privacy parameters for shuffled outputs.
Framework purifies approximate differential privacy to pure differential privacy.
problem Achieving pure differential privacy from approximate differential privacy.
method Randomized post-processing with calibrated noise to eliminate δ parameter.
result First statistically and computationally efficient reduction from approximate DP to pure DP.
Gaussian processes with differential privacy protect both inputs and outputs.
problem Previous DP methods only protected model outputs, not inputs.
method Sparse GP with private variational approximation, adjusting covariance for DP noise.
result Accurate models can be produced under strong privacy protection with sufficient data.
Differential privacy is a cryptographically-motivated definition of privacy which has gained significant attention over the past few years. Differentially private solutions enforce privacy by adding random noise to a function computed over the data, and the challenge in designing such algorithms is to control the added…
We compare the sample complexity of private learning [Kasiviswanathan et al. 2008] and sanitization~[Blum et al. 2008] under pure ε-differential privacy [Dwork et al. TCC 2006] and approximate (ε,δ)-differential privacy [Dwork et al. Eurocrypt 2006]. We show that the sample complexity of these tasks under approxima…
Privacy constraints affect learning Markov Random Fields differently.
problem Learning Markov Random Fields under differential privacy constraints.
method Algorithms for structure and parameter learning under pure, concentrated, and approximate differential privacy.
result Privacy constraints impose a strong separation between structure and parameter learning in high-dimensional data.
We develop a novel approximate Bayesian computation (ABC) framework, ABCDP, that produces differentially private (DP) and approximate posterior samples. Our framework takes advantage of the Sparse Vector Technique (SVT), widely studied in the differential privacy literature. SVT incurs the privacy cost only when a cond…
Improved shuffling technique amplifies privacy guarantees for anonymous data contributions.
problem Enhancing privacy in systems where data is contributed anonymously.
method Developed a new approach to random shuffling that amplifies differential privacy guarantees.
result Achieved asymptotically optimal privacy amplification with nearly optimal dependence in ε.
Three DP variants linked, improving SGD privacy bounds.
problem Relating different DP variants for tighter privacy bounds.
method Developed machinery to relate approximate DP to RDP and hypothesis test DP.
result Improved privacy guarantees for noisy SGD.
Novel mean estimation method under user-level differential privacy reduces noise in continual mean estimates.
problem Maintaining accurate running mean estimates under user-level differential privacy.
method Developed a novel mean estimation specific factorization under approximate differential privacy.
result Achieved asymptotically lower mean-squared error bounds in continual mean estimation.
We derive the optimal differential privacy (DP) parameters of a mechanism that satisfies a given level of Rényi differential privacy (RDP). Our result is based on the joint range of two f-divergences that underlie the approximate and the Rényi variations of differential privacy. We apply our result to the moments acc…
New algorithm reduces privacy breach in posterior sampling.
problem Combining pure DP with MCMC for efficient sampling.
method ASAP algorithm that perturbs MCMC samples with Wasserstein-infinity noise.
result First nearly linear-time algorithm achieving optimal DP-ERM rates.
Paper proposes no-regret algorithms for private GP bandit optimization.
problem Private Gaussian process bandit optimization.
method Combines uniform kernel approximator with random perturbations for differentially private GP bandit algorithms.
result Provable no-regret algorithms for stationary kernel functions in two DP settings.
Sharp privacy bounds for sequential analysis of sensitive data.
problem Privacy degradation under sequential analysis of sensitive data.
method Edgeworth expansion in f-differential privacy framework.
result Improved privacy bounds under composition with refined approximation accuracy.
A continuing challenge for machine learning is providing methods to perform computation on data while ensuring the data remains private. In this paper we build on the provable privacy guarantees of differential privacy which has been combined with Gaussian processes through the previously published \emph{cloaking metho…
A new method for privacy-preserving Bayesian learning in federated learning.
problem Privacy-preserving learning of models from distributed sensitive data.
method Differentially private partitioned variational inference (DPVI) for federated learning.
result First general framework for federated Bayesian learning with differential privacy.
A new privacy accountant for Gaussian differential privacy measures individual privacy losses.
problem Bounding differential privacy loss for each participant in data analysis.
method Developed a privacy accountant for adaptive compositions of randomised mechanisms using Gaussian differential privacy.
result Provided optimal bounds for the Gaussian mechanism and constructed an approximative individual privacy accountant.
Differential privacy is a leading protection setting, focused by design on individual privacy. Many applications, in medical / pharmaceutical domains or social networks, rather posit privacy at a group level, a setting we call integral privacy. We aim for the strongest form of privacy: the group size is in particular n…
Private statistics estimation faces a bias, accuracy, and privacy trilemma.
problem Balancing privacy, accuracy, and bias in statistical estimation.
method Use differential privacy (DP) for private statistics, but clip samples to control sensitivity and add noise for privacy, introducing bias.
result No algorithm can simultaneously have low bias, low error, and low privacy loss for arbitrary distributions.
Efficiently private clustering algorithms with tight approximation ratios.
problem Differentially private clustering of various types.
method Efficient algorithms achieving tight approximation ratios for clustering problems.
result Achieves approximation ratios similar to non-private algorithms with small additive errors.
Paper optimizes privacy-preserving distribution estimation for sparse data.
problem Sparse distribution estimation under local differential privacy constraints.
method Compressive sensing approaches for privacy-preserving estimation.
result Significant reduction in sample complexity for approximately sparse distributions.
Improved ADMM for convex distributed learning with differential privacy.
problem Privacy concerns in distributed learning with sensitive data.
method Approximate multi-step ADMM with calibrated noise.
result Higher utility and error bounds asymptotic to state-of-the-art.
A new Gaussian mechanism for differential privacy in the shuffle model is introduced.
problem Improving differential privacy in distributed learning environments.
method Characterization and upper-bounding of Rényi differential privacy (RDP) for the shuffle Gaussian mechanism.
result The shuffle Gaussian mechanism provides improved privacy guarantees compared to existing methods.
DP-SEP privatizes EP by refining a single factor per data point.
problem Private inference of complex models with limited memory.
method Stochastic Expectation Propagation with differential privacy.
result DP-SEP provides better posterior estimates with guaranteed privacy.
Paper tightens privacy and generalization bounds for iterative learning.
problem Balancing privacy and generalization in iterative learning algorithms.
method Established alignment between generalization and privacy, derived composition theorems for iterative algorithms.
result Generalization bounds for iterative learning algorithms are strictly tighter than existing works.
Paper proposes a privacy-preserving RL algorithm for linear MDPs with theoretical guarantees.
problem Protecting users' private data in personalized services using RL.
method Local differential privacy (LDP) for RL with linear function approximation.
result Achieves a regret bound of $O(d^{5/4}H^{7/4}T^{3/4}\left(\log(1/δ)
ight)^{1/4}\sqrt{1/\varepsilon})$ for linear mixture MDPs.
The remarkable development of deep learning in medicine and healthcare domain presents obvious privacy issues, when deep neural networks are built on users' personal and highly sensitive data, e.g., clinical records, user profiles, biomedical images, etc. However, only a few scientific studies on preserving privacy in …
The paper proposes methods to infer from privacy-protected data using simulation-based techniques.
problem Valid statistical inference from privacy-protected data is computationally challenging.
method Simulation-based inference methods, including sequential Monte Carlo and neural conditional density estimators.
result Valid statistical inferences can be made from privacy-protected data.
New filters match advanced composition for adaptive privacy, with practical constants.
problem Limitations of existing adaptive composition methods.
method Constructed new filters and odometers that match advanced composition rates, including constants.
result Achieved fully adaptive privacy with practical filters and odometers.
Privacy preserving mechanisms such as differential privacy inject additional randomness in the form of noise in the data, beyond the sampling mechanism. Ignoring this additional noise can lead to inaccurate and invalid inferences. In this paper, we incorporate the privacy mechanism explicitly into the likelihood functi…
Alternating Direction Method of Multipliers (ADMM) is a widely used tool for machine learning in distributed settings, where a machine learning model is trained over distributed data sources through an interactive process of local computation and message passing. Such an iterative process could cause privacy concerns o…
Paper uses statistical depth to create DP estimators for regression.
problem Creating differentially private estimators in high dimensions.
method Uses halfspace and regression depth to analyze maximum influence and construct DP estimators.
result New DP estimators for location and regression show favorable performance.
New bounds for private learning of high-dimensional Gaussian distributions.
problem Learning high-dimensional Gaussian distributions under differential privacy constraints.
method Analytic tools for constructing global covers from local covers, modified hypothesis selection techniques.
result Near-optimal sample complexity bounds for general Gaussians, conjectured to be near-optimal in the general case.
Noisy Max and Sparse Vector are selection algorithms for differential privacy and serve as building blocks for more complex algorithms. In this paper we show that both algorithms can release additional information for free (i.e., at no additional privacy cost). Noisy Max is used to return the approximate maximizer amon…
New algorithms improve online prediction from experts with privacy constraints.
problem Online prediction from experts under privacy constraints.
method Proposed and analyzed new algorithms for approximate and pure differential privacy.
result Achieved improved regret bounds for various adversaries.
New mechanisms improve differential privacy for scalar queries.
problem Improving differential privacy for scalar, real-valued query functions.
method Mixing multiple Gaussian distributions to satisfy differential privacy.
result Mechanisms yield lower noise amplitudes and variances compared to the analytic Gaussian mechanism.
Novel privatization framework for high-dimensional variable selection with differential privacy.
problem High-dimensional controlled variable selection with rigorous FDR control under differential privacy constraints.
method Gaussian Johnson-Lindenstrauss Transformation for privatizing the knockoff matrix.
result The proposed private variable selection procedure maintains statistical power even under strict privacy budgets.
A DP method selects best sparse models in high dimensions efficiently.
problem Model selection in high-dimensional sparse linear regression under privacy constraints.
method Differential privacy (DP) with exponential mechanism and Metropolis-Hastings algorithm.
result The method identifies active features quickly under privacy constraints.
Study improves privacy-preserving online prediction from experts with speed-ups.
problem Privacy-preserving online prediction from experts with speed-ups.
method Differentially private federated online prediction algorithms.
result Achieves m-fold regret speed-up with low-loss expert in federated setting. Optimal Gaussian noise mechanisms achieve nearly optimal error in unbiased mean estimation.
problem Efficiently estimating the mean of high-dimensional data while preserving privacy.
method Differential privacy mechanisms with Gaussian noise, focusing on optimal covariance.
result Gaussian noise mechanisms achieve nearly optimal error among all private unbiased mean estimation mechanisms.
Unified framework for subsampling mechanisms with tighter privacy guarantees.
problem Improving privacy in machine learning models through subsampling.
method Conditional optimal transport for deriving mechanism-specific subsampling guarantees.
result Tighter privacy bounds for subsampled mechanisms compared to traditional methods.
Paper develops a new method for differential privacy sampling using Wasserstein distance.
problem Sampling from distributions under differential privacy constraints with geometric structure consideration.
method Develops a novel framework with Wasserstein Projection Mechanism (WPM) for minimax optimal mechanisms.
result Proposes efficient algorithms for approximate computation of the Wasserstein Projection Mechanism.
This work develops sampling methods for differential privacy using SHK geometry.
problem Approximating sampling for the exponential mechanism in differential privacy.
method Develops perturbation theory for SHK gradient flows and applies to differential privacy.
result Derives time-dependent Pure-DP guarantees and Approximate-DP certificates.
Paper introduces efficient top-k selection with differential privacy.
problem Efficiently selecting top-k elements with differential privacy.
method Oneshot Laplace mechanism, generalizing Report Noisy Max.
result Noise level of O(sqrt(k)/eps) for approximate differential privacy.
Novel PP-ADMM and IPP-ADMM algorithms improve differential privacy in distributed machine learning.
problem Privacy concerns in ADMM-based distributed machine learning.
method Proposes PP-ADMM and IPP-ADMM algorithms to provide differential privacy while improving model accuracy and convergence.
result The proposed algorithms achieve better model accuracy and convergence under the same privacy guarantee.
Private mean estimation with multiple samples requires a certain number of people to maintain privacy.
problem Private mean estimation with person-level differential privacy for multiple samples.
method The approach involves estimating the mean up to a distance α in ℓ_2-norm under ε-differential privacy, using algorithms based on the clip-and-noise framework and new analyses.
result The necessary and sufficient number of people to estimate the mean up to distance α in ℓ_2-norm is given by a specific formula.
Proposes a private empirical bootstrap for Gaussian Differential Privacy.
problem Quantifying uncertainty in massive data under Differential Privacy.
method Gaussian Differential Private Bootstrap by Subsampling.
result Consistent and efficient private inference method.
Privacy affects how much data is needed for CVaR optimization.
problem Privacy constraints impact the effective sample size for CVaR optimization.
method Analyzes the privacy-relevant sample size and decomposes CVaR excess risk.
result The effective private tail sample size is εnτ, affecting CVaR learning rates.