Private PGB boosts synthetic data quality using GANs and privacy techniques.
problem Differentially private GANs struggle with convergence and poor output quality.
method Combines reweighted samples from GAN training using Private Multiplicative Weights method.
result Improves synthetic data quality across various datasets and tasks.
Differentially private algorithms protect model explanations from leaking training data.
problem Model explanations can leak training data, compromising privacy.
method Adaptive differentially private gradient descent algorithm to produce accurate, private explanations.
result Privacy amplification and reduction of overall privacy loss on explanation data.
New methods speed up training of differentially private deep learning models.
problem Training differentially private deep learning models is slower than non-private models.
method Derive and implement new per-example gradient clipping methods compatible with auto-differentiation.
result Significant training speed-ups (54x - 94x) for various models and architectures.
PriMORL trains private RL policies on offline data.
problem Private reinforcement learning on offline data.
method PriMORL learns DP models of the environment and optimizes a policy on the penalized private model.
result PriMORL enables training of private RL agents on complex tasks.
DPlis improves privacy in deep learning models by smoothing loss functions.
problem Privacy leakage in deep learning models trained on private data and low model performance.
method DPlis constructs a smooth loss function to favor noise-resilient models.
result DPlis effectively boosts model quality and training stability under privacy constraints.
DPDMs use DP-SGD to generate private synthetic data.
problem Generating private synthetic data from sensitive datasets.
method Introduced DP-SGD for DMs, investigated DM parameterization and sampling, proposed noise multiplicity.
result Achieved state-of-the-art performance in image generation benchmarks.
Normalization layers improve the accuracy of Differentially Private training of deep neural networks.
problem Reduced accuracy in deep neural networks with Differentially Private training.
method Proposed a novel method for integrating batch normalization with Differentially Private Stochastic Gradient Descent (DPSGD) without additional privacy loss.
result Training deeper networks with better utility-privacy trade-off is possible.
Differentially private GANs improve image privacy without significant quality loss.
problem Anonymizing image data sets while maintaining image quality.
method Training GANs with differential privacy on MNIST, analyzing privacy-utility trade-offs and explaining optimization methods.
result An increasing privacy budget adds little to generated image quality, revealing a saturated training regime.
Public pretraining improves private model training even in extreme distribution shift scenarios.
problem Improving private model training accuracy in settings with large distribution shift.
method Empirical evaluation and theoretical explanation of public representations improving private training accuracy.
result Public representations can improve private training accuracy by up to 67% over private training from scratch in settings with large distribution shift.
Private learning needs more data or better features.
problem Improving differentially private machine learning performance.
method Demonstrates the need for either more private data or better features.
result Private learning requires either more data or better features.
DiPriMe forests use private medians to create balanced tree splits for privacy-protected data.
problem Privacy concerns in training random forests due to multiple data queries.
method Proposes DiPriMe forests, which use a private median to generate balanced splits, ensuring differential privacy.
result DiPriMe forests achieve high utility while maintaining differential privacy, as shown both theoretically and empirically.
Improved differentially private deep learning with group-wise clipping techniques.
problem Efficiency and privacy trade-offs in deep learning models.
method Group-wise clipping techniques (per-layer and per-device) to reduce compute time and memory overhead.
result Private learning with group-wise clipping achieves similar or better performance than non-private learning with less wall time.
Differentially private learning avoids iterative optimization in parameter space.
problem Differentially private training of neural networks
method Hypernetworks trained on public datasets
result Significantly reduces noise in parameter space
Paper introduces a differentially private generative model using gradient flow and sliced Wasserstein distance.
problem Protecting privacy in sensitive training data for generative models.
method Gradient flow in the space of probability measures, Gaussian-smoothed Sliced Wasserstein Distance, and numerical scheme for SDE.
result Demonstrates higher-fidelity data generation at low privacy budget compared to existing methods.
Differentially private learning on real-world data poses challenges for standard machine learning practice: privacy guarantees are difficult to interpret, hyperparameter tuning on private data reduces the privacy budget, and ad-hoc privacy attacks are often required to test model privacy. We introduce three tools to ma…
Differentially private ensemble classifiers adapt to data streams while protecting privacy.
problem Adapting to evolving data characteristics while protecting private information.
method Unbounded ensemble updates, model agnostic approach.
result Outperforms competitors on various privacy, drift, and distribution settings.
Differentially private conformal prediction improves statistical efficiency.
problem Quantifying uncertainty in private data analysis.
method Introducing differential conformal prediction and developing Differentially Private Conformal Prediction (DPCP).
result DPCP produces tighter prediction sets than existing private split conformal approaches.
We study the problem of estimating high dimensional models with underlying sparse structures while preserving the privacy of each training example. We develop a differentially private high-dimensional sparse learning framework using the idea of knowledge transfer. More specifically, we propose to distill the knowledge …
Catastrophic forgetting can be a significant problem for institutions that must delete historic data for privacy reasons. For example, hospitals might not be able to retain patient data permanently. But neural networks trained on recent data alone will tend to forget lessons learned on old data. We present a differenti…
The process of data mining with differential privacy produces results that are affected by two types of noise: sampling noise due to data collection and privacy noise that is designed to prevent the reconstruction of sensitive information. In this paper, we consider the problem of designing confidence intervals for the…
Algorithm selects public datasets for private machine learning.
problem Choosing the most suitable public dataset for private machine learning.
method Measures gradient subspace distance between public and private datasets.
result Excess risk scales with the subspace distance between gradients.
DPLTM uses private winners to improve privacy and utility.
problem Balancing privacy and utility in machine learning models.
method DPLTM applies differential privacy to the lottery ticket method.
result DPLTM converges faster and uses less privacy budget.
New method for differentially private optimization with general Lipschitz conditions.
problem Differentially private optimization under general Lipschitz conditions.
method Generalized Lipschitz condition for per-sample gradients, tuning clip norm based on minimum per-sample Lipschitz constant.
result Efficacy of the recommended clip norm tuning method verified on 8 datasets.
FPFL mitigates unfairness in private federated learning.
problem Differential privacy degrades model performance on under-represented groups.
method Extends modified method of differential multipliers to private federated learning.
result FPFL reduces unfairness in trained models on private federated learning.
We improve private training accuracy with learning rate schedules and matrix factorizations.
problem Private training with learning rate schedules and correlated noise.
method General upper and lower bounds for learning rate schedules, memory-efficient constructions, and schedule-aware factorizations.
result Schedule-aware factorizations improve accuracy in private training.
New algorithms improve privacy and utility of large language models.
problem Privacy-preserving fine-tuning of large language models.
method Meta-framework for differentially private fine-tuning, inspired by recent success in fine-tuning.
result Private fine-tuned models achieve utility close to non-private models, with improved privacy and efficiency.
Improved image generation with private data using perceptual features.
problem Difficulty in training generative models with differential privacy.
method Use pre-trained perceptual features to learn private data distribution.
result Generative models can generate high-quality images with low privacy budget ( ϵ ≈ 2 \epsilon \approx 2 ϵ ≈ 2 ). Optimizes sparse fine-tuning for privacy in neural networks.
problem Performance gap between DP-SGD and non-private fine-tuning.
method Optimization-based approach using private gradient information for selecting trainable weights.
result Our selection method leads to better prediction accuracy compared to existing approaches.
The paper investigates how data imbalance affects fairness and accuracy in differentially private deep learning.
problem Impact of data imbalance on fairness and accuracy in differentially private deep learning.
method Study the effects of different levels of imbalance in the data on the accuracy and fairness of decisions made by a model trained with differential privacy.
result Small imbalances and loose privacy guarantees can cause disparate impacts on model accuracy and fairness.
Gradient clipping helps private SGD converge despite potential bias.
problem Gradient clipping in private SGD can bias convergence.
method Theoretical analysis and empirical evaluation of gradient clipping effects.
result Gradient clipping can prevent convergence to stationary points and introduces bias.
Study shows privacy and utility trade-offs in synthetic data models, impacting fairness and real-world performance.
problem Understanding the impact of differential privacy on fairness and model performance in synthetic data.
method Systematic analysis of differentially private synthetic datasets on classification models, measuring utility and bias using fairness metrics.
result More privacy does not necessarily mean more bias, but it can affect model performance when deployed on real data.
DPZero fine-tunes large models privately without backpropagation.
problem Memory and privacy challenges in fine-tuning large language models.
method DPZero uses zeroth-order methods for private fine-tuning, avoiding backpropagation.
result DPZero achieves private fine-tuning of RoBERTa and OPT on various tasks.
A key factor in developing high performing machine learning models is the availability of sufficiently large datasets. This work is motivated by applications arising in Software as a Service (SaaS) companies where there exist numerous similar yet disjoint datasets from multiple client companies. To overcome the challen…
Transforms robust algorithms into private ones with optimal error rates.
problem Balancing privacy and robustness in machine learning.
method Black-box transformation method to convert robust algorithms to private ones with optimal error rates.
result Optimal private estimators for various tasks, including Gaussian and PCA.
DP-SGD can update fewer coordinates while maintaining privacy.
problem How to update fewer coordinates in DP-SGD without losing optimization signal.
method TP-TopK (Two-Phase TopK DP-SGD), a two-phase method for coordinate-sparse private training.
result Private training can update fewer coordinates without losing optimization signal, scaling noise with active dimension \(k\) instead of full dimension \(d\).
This paper presents a method to train a public model with private data using GANs and differential privacy.
problem Privacy concerns in training deep learning models on sensitive data.
method A three-player learning framework with differential privacy protection.
result The proposed method achieves a balance between privacy and model accuracy.
HDP-VFL hybridizes DP for VFL, reducing privacy costs.
problem Privacy-preserving collaborative learning from vertically partitioned data.
method Hybrid DP framework combining HE and MPC for VFL.
result Achieves DP and JDP with negligible training time and accuracy trade-offs.
A framework for private prediction sets using conformal prediction and differential privacy.
problem Jointly addressing reliability and privacy in machine learning predictions.
method Split conformal prediction with privatized quantile subroutine.
result Private prediction sets can be generated from privately-trained models.
Recent advances in machine learning have largely benefited from the massive accessible training data. However, large-scale data sharing has raised great privacy concerns. In this work, we propose a novel privacy-preserving data Generative model based on the PATE framework (G-PATE), aiming to train a scalable differenti…
DPConvCNP learns to predict private data accurately and privately.
problem Balancing privacy and accuracy in machine learning models.
method Meta-learning combined with improved DP mechanism.
result DPConvCNP outperforms DP GP baseline, especially on non-Gaussian data.
New regularizer for machine learning using private data.
problem Machine learning with private data.
method Distributionally-robust optimization with locally-differentially-private datasets.
result New regularizer for training linear regression models.
DP-RandP improves privacy-utility tradeoff in DP-SGD by learning priors from random processes.
problem Improving the performance of differentially private stochastic gradient descent (DP-SGD) on private data.
method A three-phase approach that learns priors from images generated by random processes and transfers these priors to private data.
result New state-of-the-art accuracy on CIFAR10, CIFAR100, MedMNIST, and ImageNet for various privacy budgets.
Optimal DP model training with public data improves privacy and accuracy.
problem Ensuring privacy while training models with public data.
method Proves optimal error rates for DP model training with public data, develops novel algorithms.
result Optimal error rates can be achieved by using public data or optimal DP algorithms.
Improved Frank-Wolfe algorithm speeds up training of differentially private LASSO models.
problem Training differentially private LASSO models on sparse data.
method Adapted Frank-Wolfe algorithm for sparse inputs, reducing runtime.
result Training time reduced from O ( T D S + T N s ) \mathcal{O}(TDS + TNs) O ( T D S + T N s ) to O ( N S + T D log D + T S 2 ) \mathcal{O}(N S + T \sqrt{D} \log{D} + TS^2) O ( N S + T D log D + T S 2 ) . Private cancer prediction model trained on federated genomic data.
problem Train a private cancer prediction model on federated genomic data.
method Differentially private federated learning (FL) for genomic cancer prediction.
result Ranked 3rd in a competition for private cancer prediction.
Differentially private log-location-scale regression models improve privacy in statistical analysis.
problem Ensuring privacy in statistical regression models while maintaining accuracy.
method Integrates differential privacy into LLS regression using the functional mechanism.
result Proposed DP-LLS models satisfy ε-differential privacy and perform well under various conditions.
Improved DMs with DP-SGD for generating private images.
problem Low privacy of DMs and lack of good privacy-utility tradeoff.
method Adopt LDMs with DP-SGD on attention modules of LDMs.
result Generates high-quality DP images with reduced parameters.
Gradient sparsification enhances privacy-preserving machine learning models.
problem Improving performance of differentially-private machine learning models under privacy constraints.
method Gradient sparsification combined with compressed sensing and additive Laplace noise.
result Gradient sparsification can improve performance of differentially-private machine learning models for small privacy budgets.