The paper analyzes and proposes methods for privately sharing individual privacy losses using per-instance differential privacy.
problem The standard differential privacy framework provides a worst-case bound that may not accurately reflect individual privacy losses.
method The paper analyzes per-instance differential privacy and proposes methods to privately and accurately publish per-instance privacy losses.
result The methods privately and accurately publish per-instance differential privacy losses with minimal additional privacy cost.
New method for certified unlearning reduces noise injection.
problem Achieving formal unlearning guarantees with adaptive noise calibration.
method Adaptive per-instance noise calibration based on individual data point sensitivities.
result Derivation of high-probability per-instance sensitivity bounds for ridge regression.
We consider a refinement of differential privacy --- per instance differential privacy (pDP), which captures the privacy of a specific individual with respect to a fixed data set. We show that this is a strict generalization of the standard DP and inherits all its desirable properties, e.g., composition, invariance to …
DP-SGD analysis shows many datapoints leak less privacy than previously thought.
problem Empirical evidence suggests DP-SGD leaks less privacy than current analysis predicts.
method Developed a per-instance DP analysis of DP-SGD, introducing dependence on dataset distribution.
result Formally shows DP-SGD leaks significantly less privacy for many datapoints on common benchmarks.
It has long been observed that for practically any computational problem that has been intensely studied, different instances are best solved using different algorithms. This is particularly pronounced for computationally hard problems, where in most cases, no single algorithm defines the state of the art; instead, the…
New method speeds up causal sensitivity analysis.
problem Bounding causal effects in unobserved confounding.
method Amortized approach using prior-data fitted networks.
result Orders of magnitude faster computation.
Crowd-sourcing is a cheap and popular means of creating training and evaluation datasets for machine learning, however it poses the problem of `truth inference', as individual workers cannot be wholly trusted to provide reliable annotations. Research into models of annotation aggregation attempts to infer a latent `tru…
Paper estimates optimal classification error with soft labels and calibration.
problem Estimating the optimal classification error with soft labels and calibration.
method Extends previous work on soft labels to estimate Bayes error, addressing bias and corrupted labels.
result The method provides a statistically consistent estimator of the Bayes error, even with imperfectly calibrated soft labels.
Improves privacy amplification by shuffling for differential privacy.
problem Enhancing privacy guarantees in systems with anonymous data contributions.
method Theoretical and numerical analysis of Rényi differential privacy parameters and privacy amplification by shuffling.
result First asymptotically optimal analysis of Rényi differential privacy parameters for shuffled outputs.
Proposes a privacy-preserving recommendation system using matrix factorization and differential privacy.
problem Privacy leakage in recommendation systems when anonymizing user data is not sufficient.
method Uses matrix factorization and differential privacy via the Gaussian mechanism.
result Demonstrates excellent utility for privacy-preserving recommendation systems.
Federated learning with Bayesian differential privacy offers improved privacy and accuracy.
problem Privacy in federated learning with similar data distributions.
method Bayesian differential privacy for federated learning, with improved privacy budgeting.
result Significant advantage over state-of-the-art privacy bounds, with lower noise and improved accuracy.
This work optimizes mean estimation under varying privacy constraints.
problem Mean estimation with heterogeneous privacy constraints.
method Proposes an algorithm for mean estimation under different privacy levels for users.
result Shows a saturation phenomenon in performance as privacy levels are relaxed.
Improves privacy guarantees by analyzing randomness in privacy-preserving mechanisms.
problem Balancing user privacy and business constraints in privacy-preserving mechanisms.
method Analyzes explicit and implicit randomness in privacy mechanisms and proposes a probabilistic calibration method.
result Proposes privacy at risk, providing stronger privacy guarantees with quantifiable risks.
This work optimizes mean estimation under varying user privacy demands.
problem Mean estimation with heterogeneous privacy levels.
method Proposes an algorithm that is minimax optimal and has near-linear run-time.
result Privacy requirements of the most stringent users dictate overall error rates.
A new privacy accountant for Gaussian differential privacy measures individual privacy losses.
problem Bounding differential privacy loss for each participant in data analysis.
method Developed a privacy accountant for adaptive compositions of randomised mechanisms using Gaussian differential privacy.
result Provided optimal bounds for the Gaussian mechanism and constructed an approximative individual privacy accountant.
Paper improves deep learning privacy with new f-differential privacy framework.
problem Training neural networks on sensitive data while maintaining privacy.
method Introduced and analyzed f-differential privacy for neural networks training. result Improved privacy guarantees for neural networks training without sacrificing accuracy.
Paper simplifies DP composition for adaptive privacy budgets, enabling better privacy and accuracy in deep learning.
problem Tension between efficiency and flexibility in DP composition theorems.
method Rényi Differential Privacy (RDP) for adaptive privacy budgets, proving simpler composition theorem with smaller constants.
result Practical DP composition for adaptive privacy budgets, enabling better privacy and accuracy in deep learning.
This paper bounds min-entropy leakage for Blowfish privacy using graph symmetries.
problem Bounding min-entropy leakage for Blowfish privacy mechanisms.
method Organizing analysis over symmetrical partitions corresponding to orbits of graph automorphism groups.
result Demonstrates a construction meeting the bound with asymptotic equality, showing tightness.
New federated f-differential privacy for collaborative learning.
problem Privacy in federated learning.
method Introducing federated f-differential privacy and proposing a generic private federated learning framework. result Proves federated f-differential privacy provides privacy guarantee on each record of one client's data. Differential privacy is a statistical concept that can be explained through hypothesis testing.
problem Formalizing differential privacy as a statistical concept.
method Using David Blackwell's informativeness theorem, the paper shows differential privacy can be understood through hypothesis testing.
result The definition of f-differential privacy provides a unified framework for analyzing privacy bounds. A new method for tighter privacy loss accounting in adaptive analyses.
problem Ensuring individual privacy in adaptive analyses while staying within a privacy budget.
method A personalized privacy loss estimate and a Rényi differential privacy filter.
result Personalized privacy loss accounting can be practical and tighter than existing methods.
Paper tackles federated learning with privacy, enhancing target data analysis.
problem Heterogeneity and privacy of distributed data in federated learning.
method Formulates federated differential privacy, studies statistical problems under privacy constraints.
result Federated differential privacy offers a balance between privacy and knowledge transfer.
Unified framework for subsampling mechanisms with tighter privacy guarantees.
problem Improving privacy in machine learning models through subsampling.
method Conditional optimal transport for deriving mechanism-specific subsampling guarantees.
result Tighter privacy bounds for subsampled mechanisms compared to traditional methods.
New findings show privacy affects generalization error in a non-monotonic way.
problem Privacy and robustness in distributed learning.
method Theoretical analysis and matching lower/upper bounds on algorithmic stability.
result Generalization error is non-monotonically affected by privacy, depending on noise level.
New privacy framework tailored to specific data distributions.
problem Protecting individual data points in decision-making processes.
method Introducing tangent differential privacy, a new form of differential privacy.
result Entropic regularization guarantees tangent differential privacy under general conditions.
Addresses theoretical and practical aspects of Gaussian differential privacy.
problem Theoretical and practical challenges in privacy-preserving data analysis.
method Discussion of f-differential privacy and Gaussian differential privacy.
result Gaussian differential privacy can enhance privacy in various applications.
New filters match advanced composition for adaptive privacy, with practical constants.
problem Limitations of existing adaptive composition methods.
method Constructed new filters and odometers that match advanced composition rates, including constants.
result Achieved fully adaptive privacy with practical filters and odometers.
DiffSketch combines privacy and communication efficiency in distributed learning.
problem Privacy and communication efficiency in distributed machine learning.
method DiffSketch uses Count Sketch for data stream summarization to achieve both privacy and efficiency.
result DiffSketch provides strong differential privacy guarantees and significant communication compression.
Missing data enhances privacy in differential privacy.
problem Privacy preservation in datasets with missing values.
method Formalized missing data as a privacy amplification mechanism within differential privacy.
result Incomplete data can yield privacy amplification for differentially private algorithms.
This paper proposes a new differential privacy definition using Rao distance.
problem Improving differential privacy definitions for better sequential composition.
method Using Rao distance instead of divergences of densities to define privacy.
result Proposed definition shares interpretation with previous definitions but improves sequential composition.
BCDP enhances privacy by protecting sensitive features more precisely.
problem Uniform privacy protection in LDP degrades performance for sensitive features.
method Bayesian Coordinate Differential Privacy (BCDP) adjusts privacy protection per feature sensitivity.
result BCDP improves accuracy in downstream tasks without sacrificing privacy.
Edgeworth Accountant calculates privacy loss under differential privacy compositions efficiently.
problem Efficiently computing overall privacy loss under composition of private algorithms.
method Analytical approach using f-differential privacy framework and Edgeworth expansion. result Non-asymptotic (ε,δ)-differential privacy bounds with reduced computational cost. Improved privacy bounds enhance deep learning training efficiency.
problem Enhancing privacy guarantees in deep learning models.
method Deriving optimal DP parameters using f-divergences. result Significantly reduces the number of iterations needed for training deep learning models.
Differential privacy is a strong notion for privacy that can be used to prove formal guarantees, in terms of a privacy budget, ε, about how much information is leaked by a mechanism. However, implementations of privacy-preserving machine learning often select large values of ε in order to get acceptable utility of …
BUDS balances privacy and utility by shuffling data, achieving strong privacy with minimal loss.
problem Balancing privacy and utility in crowd-sourced statistical databases.
method One-hot encoding, iterative shuffling, loss estimation, risk minimization.
result Achieves ε=0.02 for privacy, maintaining a privacy bound of ε=ln[t/((n1−1)S)]. Proposes element-level differential privacy for better privacy and utility in statistical learning.
problem Challenges of strong differential privacy in statistical learning applications.
method Introduces element-level differential privacy, extending classical DP to protect specific user elements.
result Provides better utility and more robust results compared to classical DP by allowing finer privacy protections.
DPNR preserves privacy of text representations using differential privacy.
problem Privacy leakage in deep learning text representations.
method DPNR uses Differential Privacy to provide formal privacy guarantees and dropout masking for enhanced privacy.
result DPNR reduces privacy leakage without significantly sacrificing main task performance.
The paper presents a method to preserve privacy in text analysis using calibrated noise.
problem Accurately learning from user data while maintaining privacy.
method Calibrated multivariate perturbations applied to word embeddings to achieve geo-indistinguishability.
result The method provides better privacy guarantees than baseline models with minimal utility loss.
DP-FedTabDiff generates private synthetic tabular data using diffusion models and differential privacy.
problem Privacy-preserving synthetic data generation for tabular data in regulated domains.
method Combines Differential Privacy, Federated Learning, and Denoising Diffusion Probabilistic Models.
result Achieves significant privacy improvements without compromising data quality.
Integrates differential privacy and demographic parity in multi-class classification.
problem Ensuring fairness and privacy in sensitive applications.
method Designs DP2DP algorithm that enforces both demographic parity and differential privacy.
result DP2DP converges towards demographic parity at nearly the same rate as non-private methods, achieving state-of-the-art trade-offs.
Differential privacy improves AI security, fairness, and learning.
problem Privacy violations, security issues, and model fairness in AI.
method Application of differential privacy in various AI areas.
result Differential privacy enhances AI performance in multiple areas.
Enhanced stability improves privacy in machine learning.
problem Improving privacy in machine learning training while maintaining accuracy.
method Study of stability in private empirical risk minimization, focusing on strongly-convex loss functions and uniform stability.
result An algorithm with uniform stability of β implies a bound of O(√β) on the scale of noise required for differential privacy.
This paper quantifies privacy loss in exploratory data analysis.
problem Privacy loss in exploratory data analysis is often overlooked in privacy budgets.
method Quantitative analysis of privacy loss for statistical functions.
result Privacy loss must be considered in calculating machine learning privacy budgets.
This paper benchmarks privacy-preserving machine learning on medical images.
problem Ensuring privacy in medical image analysis while maintaining model accuracy.
method Comparing Local-DP and DP-SGD for differential privacy in medical imagery.
result Theoretical privacy guarantees do not fully align with real-world performance.
The paper proposes a privacy-preserving method for text data using Hyperbolic space.
problem Preserving user privacy in text data while maintaining utility for machine learning.
method Word representations in Hyperbolic space to provide privacy, sampling from a probability distribution.
result Demonstrates significant privacy guarantees (20x greater) compared to Euclidean space.
New algorithm reduces offline RL sample complexity for MDPs.
problem Learning optimal policies from offline data in unknown MDPs.
method Adaptive Pessimistic Value Iteration (APVI) algorithm.
result Suboptimality bound nearly matches theoretical limits.
Paper studies optimal federated learning for nonparametric regression with privacy constraints.
problem Federated learning for nonparametric regression with heterogeneous differential privacy constraints.
method Proposes distributed privacy-preserving estimators and investigates their risk properties.
result Establishes matching minimax lower bounds for global and pointwise estimation.
Differential privacy protects data privacy by adding noise to data.
problem Leakage of sensitive data through common methods like encryption and endpoint protection.
method Randomized response technique to add noise to data collection.
result Differential privacy ensures strong privacy with better utility.