DP-RandP improves privacy-utility tradeoff in DP-SGD by learning priors from random processes.
problem Improving the performance of differentially private stochastic gradient descent (DP-SGD) on private data.
method A three-phase approach that learns priors from images generated by random processes and transfers these priors to private data.
result New state-of-the-art accuracy on CIFAR10, CIFAR100, MedMNIST, and ImageNet for various privacy budgets.
Constructs models to meet any DP requirement without retraining.
problem Evolving privacy requirements at inference time.
method Two post-processing techniques: random selection and linear combination.
result Final private models satisfying any target privacy parameter.
New DP mechanisms improve ML privacy-utility-computational tradeoffs.
problem Improving privacy in machine learning with multiple passes over data.
method Formalized DP for adaptive streams, extended matrix factorization techniques, Fourier-transform-based mechanism.
result Substantial improvements in privacy-utility-computational tradeoffs over previous methods.
One-pass private sketch supports various machine learning tasks.
problem Efficiently supporting multiple machine learning tasks with differential privacy.
method Randomized contingency tables indexed with locality-sensitive hashing, constructed in one pass.
result Competitive error bounds for DP kernel density estimation, faster than existing methods.
Framework purifies approximate differential privacy to pure differential privacy.
problem Achieving pure differential privacy from approximate differential privacy.
method Randomized post-processing with calibrated noise to eliminate δ parameter.
result First statistically and computationally efficient reduction from approximate DP to pure DP.
New approach connects robust learning to privacy issues.
problem Vulnerability of deep neural networks to adversarial examples.
method Privacy-utility tradeoff and rate-distortion theory.
result Found saddle point via maximum conditional entropy problem.
Active learning holds promise of significantly reducing data annotation costs while maintaining reasonable model performance. However, it requires sending data to annotators for labeling. This presents a possible privacy leak when the training set includes sensitive user data. In this paper, we describe an approach for…
Improved DMs with DP-SGD for generating private images.
problem Low privacy of DMs and lack of good privacy-utility tradeoff.
method Adopt LDMs with DP-SGD on attention modules of LDMs.
result Generates high-quality DP images with reduced parameters.
Improves zeroth-order optimization for private machine learning with public data.
problem High computation and memory cost of first-order DP methods.
method PAZO (Public Data Assisted Zeroth-order Optimization) framework.
result Achieves superior privacy/utility tradeoffs across tasks.
New method reduces communication in distributed learning, improving privacy and utility.
problem Distributed learning with minimal communication and privacy protection.
method Non-interactive blind model averaging (BlindAvg) with output perturbation.
result BlindAvg converges to centralized learning with strong L2-regularization and SoftmaxReg for better privacy-utility tradeoff.
Study on privacy-preserving health care models that sacrifice accuracy for data protection.
problem Privacy-preserving models in health care neglect data from the tails, reducing accuracy for small groups.
method Used state-of-the-art differentially private learning methods for clinical prediction tasks.
result Privacy-preserving models in health care exhibit steep tradeoffs between privacy and utility, and disproportionately influence large demographic groups.
Paper improves privacy-utility trade-off in federated learning.
problem Repeated parameter sharing in federated learning leaks private data.
method Proposes a new representation federated learning objective with differential privacy guarantees.
result Algorithm \DPFEDREP\ converges to a global optimal solution with a linear rate and privacy budget-dependent radius.
New framework for private convex optimization in arbitrary norms.
problem Private optimization of convex functions in non-Euclidean settings.
method Regularized exponential mechanism based on localization tools from convex geometry.
result First optimal privacy-utility tradeoffs for ℓp norms and Schatten-p norms. We consider the problem of obfuscating sensitive information while preserving utility, and we propose a machine learning approach inspired by the generative adversarial networks paradigm. The idea is to set up two nets: the generator, that tries to produce an optimal obfuscation mechanism to protect the data, and the c…
The Sampled Gaussian Mechanism's noise level decreases with larger subsampling rates, improving privacy-utility trade-offs.
problem Improving privacy-utility trade-offs in differentially private stochastic optimization.
method Proof of a conjecture about the Sampled Gaussian Mechanism's noise level and subsampling rate relationship.
result A rigorous proof of the conjecture, completing the proof of Theorem 6.2 in the original paper.
Differential privacy has emerged as a gold standard in privacy-preserving data analysis. A popular variant is local differential privacy, where the data holder is the trusted curator. A major barrier, however, towards a wider adoption of this model is that it offers a poor privacy-utility tradeoff. In this work, we add…
SMOTE-DP enhances synthetic data privacy without sacrificing utility.
problem Balancing privacy and utility in synthetic data generation.
method Integrating SMOTE with differential privacy mechanisms.
result SMOTE-DP produces synthetic data that is both private and useful.
Exact optimality achieved in distributed mean estimation with shared randomness.
problem Achieving optimal communication, privacy, and utility tradeoffs in distributed mean estimation.
method Utilization of a rotationally symmetric shared random codebook and a k-closest encoding mechanism. result Proposed mechanism achieves exact optimality for randomly rotated simplex codebook.
Enhanced stability improves privacy in machine learning.
problem Improving privacy in machine learning training while maintaining accuracy.
method Study of stability in private empirical risk minimization, focusing on strongly-convex loss functions and uniform stability.
result An algorithm with uniform stability of β implies a bound of O(√β) on the scale of noise required for differential privacy.
Companies increasingly expose machine learning (ML) models trained over sensitive user data to untrusted domains, such as end-user devices and wide-access model stores. We present Sage, a differentially private (DP) ML platform that bounds the cumulative leakage of training data through models. Sage builds upon the ric…
Paper improves KNN-Shapley for privacy-friendly data valuation.
problem Privacy challenges in data valuation methods.
method Introduces TKNN-Shapley, a privacy-friendly variant of KNN-Shapley.
result TKNN-Shapley offers superior privacy-utility tradeoff compared to naively privatized KNN-Shapley.
Regularization can improve both privacy and performance in machine learning models.
problem Privacy vs. Utility trade-off in machine learning models.
method The study uses logistic regression with ridge regularization and a leave-one-out analysis tool.
result Increasing the number of parameters can improve both privacy and performance when coupled with proper regularization.
Paper relaxes differential privacy for correlated features, improving privacy-utility trade-off.
problem Standard differential privacy ignores feature correlation, leading to suboptimal privacy-utility balance.
method Introduces CorrDP framework that accounts for feature correlation, using total variation distance for quantification.
result CorrDP algorithms outperform standard DP in synthetic and real-world datasets with insensitive features.
Tempered sigmoids improve deep learning privacy.
problem Privacy-preserving deep learning with strict differential privacy guarantees.
method Developed tempered sigmoid activation functions for deep learning models.
result Tempered sigmoids outperform ReLU in achieving state-of-the-art accuracy.
DPLTM uses private winners to improve privacy and utility.
problem Balancing privacy and utility in machine learning models.
method DPLTM applies differential privacy to the lottery ticket method.
result DPLTM converges faster and uses less privacy budget.
Our study analyzes how neural network initialization affects privacy and utility in overparameterized models.
problem Privacy and utility trade-off in overparameterized neural networks.
method Analytical proof of KL divergence privacy bound, focusing on initialization, width, and depth.
result Privacy bound improvement with increasing depth under certain initializations, degradation under others.
Conformal-DP improves differential privacy on manifold data by calibrating perturbations based on local densities.
problem Lack of density-awareness in existing differential privacy mechanisms for manifold data leads to biased and suboptimal privacy-utility trade-offs.
method Proposes Conformal-DP, a density-aware differential privacy mechanism using conformal transformations to calibrate perturbations based on local densities.
result Demonstrates improved privacy-utility trade-off in heterogeneous data distribution settings compared to state-of-the-art mechanisms.
Differential privacy provides strong privacy guarantees simultaneously enabling useful insights from sensitive datasets. However, it provides the same level of protection for all elements (individuals and attributes) in the data. There are practical scenarios where some data attributes need more/less protection than ot…
Differentially private GANs improve image privacy without significant quality loss.
problem Anonymizing image data sets while maintaining image quality.
method Training GANs with differential privacy on MNIST, analyzing privacy-utility trade-offs and explaining optimization methods.
result An increasing privacy budget adds little to generated image quality, revealing a saturated training regime.
A new method for tighter privacy loss accounting in adaptive analyses.
problem Ensuring individual privacy in adaptive analyses while staying within a privacy budget.
method A personalized privacy loss estimate and a Rényi differential privacy filter.
result Personalized privacy loss accounting can be practical and tighter than existing methods.
Paper assesses the market value of sharing privacy-protected smart meter data.
problem Value of sharing privacy-protected smart meter data between consumers and load serving entities.
method Discounted differential privacy model, ANN-based load forecasting, optimal procurement problem.
result Significant value in sharing smart meter data while retaining individual consumer privacy.
Differential privacy is a mathematical framework for privacy-preserving data analysis. Changing the hyperparameters of a differentially private algorithm allows one to trade off privacy and utility in a principled way. Quantifying this trade-off in advance is essential to decision-makers tasked with deciding how much p…
PrAda-GAN improves synthetic data generation under differential privacy.
problem Generating synthetic data under differential privacy with marginal-based methods.
method Sequential generator architecture integrating GAN and marginal-based approaches, with adaptive regularization of Bayes network structure.
result PrAda-GAN outperforms existing methods in privacy-utility trade-off on synthetic and real-world datasets.
Noise-aware Bayesian inference framework for locally private data collection.
problem Privacy-preserving data collection with non-trustworthy aggregators.
method Noise-aware probabilistic modeling framework for Bayesian inference under LDP.
result Demonstrated efficacy in parameter estimation for various distributions and regression models.
New bounds for KANs trained with DP-SGD, addressing correlated noise.
problem Risk bounds for Kolmogorov-Arnold Networks trained by DP-SGD with correlated noise.
method Established new optimization and population risk analysis for KANs trained with DP-SGD, addressing correlated noise.
result First optimization and population risk analysis of correlated-noise mechanisms for DP training in non-convex settings, including neural networks.
Improved privacy-preserving statistical estimates with customizable noise reduction.
problem Balancing privacy and accuracy in statistical estimation.
method Introducing the Brownian mechanism, which adds Gaussian noise to a sequence of estimates, gradually reducing it based on the practitioner's needs.
result The Brownian mechanism produces more accurate estimates while maintaining strong privacy guarantees, outperforming existing methods.
The main goal of this thesis is to point out that the bias-variance tradeoff is not always true (e.g. in neural networks). We advocate for this lack of universality to be acknowledged in textbooks and taught in introductory courses that cover the tradeoff. We first review the history of the bias-variance tradeoff, its …
TVineSynth generates synthetic data to balance privacy and utility.
problem Balancing privacy and utility in synthetic data generation.
method Uses vine copula with truncation to control privacy and utility trade-off.
result Achieves superior privacy-utility balance compared to competitors.
Study privacy vs. utility in estimating network parameters with aggregated data.
problem Privacy-preserving estimation of network parameters from aggregated node degrees.
method β model, local and central differential privacy, minimax lower bounds, simple estimators.
result Achieved minimax-optimal risk bounds for parameter estimation under privacy constraints.
Data privacy is an important concern in machine learning, and is fundamentally at odds with the task of training useful learning models, which typically require the acquisition of large amounts of private user data. One possible way of fulfilling the machine learning task while preserving user privacy is to train the m…
A new tradeoff between regularization and sharpness improves model performance in overparameterized settings.
problem Improving model performance in overparameterized settings with minimum-norm interpolators.
method Proposes a regularization-sharpness tradeoff for overparameterized linear regression with an ℓ^p penalty.
result Empirical validation shows the tradeoff terms can distinguish performant linear interpolators.
Enhanced privacy, utility, and efficiency through MUST subsampling.
problem Balancing privacy, utility, and computational efficiency in data analysis.
method MUltistage Sampling Technique (MUST) for privacy amplification in differential privacy.
result MUST offers stronger privacy guarantees (ϵ) than one-stage subsampling methods while maintaining similar utility and computational efficiency. Study the tradeoff between signal distortion and human perception over finite channels.
problem Characterize the distortion-perception tradeoff for finite channels with arbitrary metrics.
method Solve linear programming problems to compute the distortion-perception function and optimal reconstructions.
result DP function is piecewise linear in the perception index.
We perform the first study of the tradeoff space of access methods and replication to support statistical analytics using first-order methods executed in the main memory of a Non-Uniform Memory Access (NUMA) machine. Statistical analytics systems differ from conventional SQL-analytics in the amount and types of memory …
The paper studies adversarial training for linear regression models.
problem Understanding the tradeoffs between robust and standard accuracy in adversarial training.
method Characterizes the fundamental tradeoff and specific adversarial training approach for linear regression with Gaussian features.
result Precise characterization of the standard and robust accuracy tradeoff in high-dimensional settings.
Novel algorithm reduces privacy noise in machine learning.
problem High privacy noise in machine learning algorithms.
method Robust statistics, specifically median and trimmed mean, to bound sensitivity of SGD iterates.
result Improved privacy-utility trade-off with reduced noise and computational efficiency.
Paper explores tradeoff between standard and robust accuracy for latent models.
problem Tradeoff between standard accuracy and robust accuracy in adversarial training.
method Revisits adversarial training for latent models, considering Gaussian mixture and generalized linear models.
result Low-dimensional manifold structure mitigates the tradeoff between standard and robust accuracy.
Proposes a new adversarial model to avoid accuracy vs. adversarial accuracy tradeoff.
problem Inherent tradeoff between accuracy and adversarial accuracy in existing adversarial robustness definitions.
method Introduces Voronoi-epsilon adversary that balances perturbation constraints.
result Voronoi-epsilon adversary avoids accuracy vs. adversarial accuracy tradeoff even with large ε.