Proposes a framework for private data augmentation in federated learning.
problem Privacy and performance issues in non-IID training datasets.
method Multi-hop federated augmentation with sample compression.
result Significantly improves privacy, transmission delay, and local training performance.
FD and FAug reduce communication in on-device ML with non-IID data.
problem Minimize communication overhead in on-device ML with non-IID data.
method Federated distillation (FD) and federated augmentation (FAug).
result FD with FAug reduces communication by 26x while maintaining high accuracy.
New methods reduce bias in synthetic data for machine learning.
problem Statistical bias in synthetic data generated for privacy.
method Re-weighting strategies using privatised likelihood ratios.
result Private importance weighting enhances synthetic data utility.
Proposes private model aggregation methods to enhance machine learning models without sharing client data.
problem Lack of sufficient data for new clients in SaaS companies.
method Two private model aggregation approaches based on differential privacy techniques.
result Private model aggregation enables data utility and privacy guarantees.
A framework for private causal effect estimation without structural assumptions.
problem Estimating causal effects from private observational data.
method Model-agnostic framework that privatizes predictions and aggregation steps.
result Maintains competitive performance under realistic privacy budgets.
New attacks prevent both supervised and contrastive learning from private data.
problem Preventing unauthorized use of private data and commercial datasets.
method Contrastive-like data augmentations in supervised error minimization or maximization frameworks.
result Achieve state-of-the-art worst-case unlearnability across SL and CL algorithms.
Structured subsampling improves privacy in deep time series forecasting.
problem Incompatible privacy guarantees with time series forecasting.
method Structured subsampling of sequential data for privacy amplification.
result Structured subsampling enables training with strong privacy guarantees.
A framework for private prediction sets using conformal prediction and differential privacy.
problem Jointly addressing reliability and privacy in machine learning predictions.
method Split conformal prediction with privatized quantile subroutine.
result Private prediction sets can be generated from privately-trained models.
NAPP-ERM improves ERM with differential privacy guarantees by iteratively achieving target regularization and delivering strong convexity.
problem Over-regularization in privacy-preserving ERM approaches.
method Noise-Augmented Privacy-Preserving Empirical Risk Minimization (NAPP-ERM) with a dual-purpose l2 regularizer and privacy budget retrieval strategy.
result Mitigates over-regularization and achieves strong convexity through a single regularizer.
New algorithms improve privacy-preserving data release using external predictions.
problem Privacy-preserving data release with improved utility using external information.
method Learning-augmented algorithms for multiple quantile release.
result Error guarantees scale with prediction quality, almost recovering state-of-the-art guarantees.
A new decentralized algorithm DESTINY solves optimization over Stiefel manifold with single communication round.
problem Decentralized optimization over the Stiefel manifold with private data.
method Gradient tracking with approximate augmented Lagrangian function.
result DESTINY achieves global convergence with a single communication round.
Enhances FL with domain adaptation for better model accuracy.
problem Improving model accuracy in Federated Learning.
method Domain adaptation for per-user model improvement in Federated Learning.
result Domain adaptation improves model accuracy for all users, especially under differential privacy constraints.
Private method measures nonlinear correlations between data hosted across two entities.
problem Measuring nonlinear correlations between sensitive data hosted across multiple parties while preserving privacy.
method Differentially private estimator of distance correlation.
result First private estimator of nonlinear correlations in a multi-party setup.
Alternating Direction Method of Multipliers (ADMM) is a widely used tool for machine learning in distributed settings, where a machine learning model is trained over distributed data sources through an interactive process of local computation and message passing. Such an iterative process could cause privacy concerns o…
Astraea improves federated learning accuracy on imbalanced data.
problem Accuracy degradation in federated learning due to imbalanced data distribution.
method Self-balancing federated learning framework with data augmentation and client rescheduling.
result Astraea shows +5.59% and +5.89% improvement in top-1 accuracy on imbalanced datasets.
This paper studies trade-offs in private prediction methods.
problem Leakage of training data information in machine learning predictions.
method Private training and private prediction methods with trade-offs.
result Private training methods outperform private prediction methods in various settings.
Private PGB boosts synthetic data quality using GANs and privacy techniques.
problem Differentially private GANs struggle with convergence and poor output quality.
method Combines reweighted samples from GAN training using Private Multiplicative Weights method.
result Improves synthetic data quality across various datasets and tasks.
Private estimation with public data reduces sample complexity.
problem Estimating private distributions with limited public data.
method Differentially private estimation with public data under constraints of pure or concentrated DP.
result Public data can significantly reduce private sample complexity for estimation.
Enhances generative model for clinical data privacy and accuracy.
problem Data privacy in electronic patient records.
method Improves a time-series generative model with privacy safeguards.
result DP-TimeGAN achieves a mean authenticity of 0.778 on the CKD dataset.
New method generates private synthetic data without domain size dependence.
problem Private synthetic data generation for unbounded query classes.
method Constructs a private synthetic data generator for privately PAC learnable query classes.
result Sample complexity independent of domain size for privately PAC learnable query classes.
Public pretraining improves private model training even in extreme distribution shift scenarios.
problem Improving private model training accuracy in settings with large distribution shift.
method Empirical evaluation and theoretical explanation of public representations improving private training accuracy.
result Public representations can improve private training accuracy by up to 67% over private training from scratch in settings with large distribution shift.
Extends private optimization to non-convex problems efficiently.
problem Private optimization of non-convex functions over discrete and continuous domains.
method Two algorithms: one for discrete domains and one for continuous domains, both requiring boundedness and Lipschitz continuity.
result Oracle-efficient optimization algorithms for non-convex problems, outperforming standard approaches in some cases.
Efficiently learns private models using public data.
problem Improving private learning performance with public data.
method Proves computationally efficient algorithms for private learning with public data.
result First computationally efficient algorithms for private learning with public data.
Private learning needs more data or better features.
problem Improving differentially private machine learning performance.
method Demonstrates the need for either more private data or better features.
result Private learning requires either more data or better features.
LeanDojo removes barriers to theorem proving with open-source tools and data.
problem Difficulty in reproducing and building on existing theorem proving methods.
method Introduces LeanDojo, an open-source Lean playground with toolkits, data, models, and benchmarks.
result ReProver, an LLM-based prover augmented with retrieval, outperforms non-retrieval baselines and GPT-4.
The paper explores learning with a mix of private and public data while maintaining privacy.
problem Learning with a mix of private and public data while ensuring differential privacy.
method Designing a learning algorithm that satisfies differential privacy only with respect to private examples.
result A hypothesis class of VC-dimension d can be agnostically learned up to an excess error of α using only (roughly) d/α public examples and d/α^2 private labeled examples.
Private algorithms adapt from public to private domains with minimal labeled data.
problem Adapting from a public source domain to a private target domain with few labeled data.
method Differentially private discrepancy minimization algorithms based on Frank-Wolfe and Mirror-Descent methods.
result Effective adaptation with strong generalization and privacy guarantees.
Paper develops private synthetic data for sensitive data analysis.
problem Analyzing sensitive data while preserving privacy.
method Adapting a private density estimation algorithm for synthetic data generation.
result Better computational guarantees for synthetic data generation.
Improves zeroth-order optimization for private machine learning with public data.
problem High computation and memory cost of first-order DP methods.
method PAZO (Public Data Assisted Zeroth-order Optimization) framework.
result Achieves superior privacy/utility tradeoffs across tasks.
Improved image generation with private data using perceptual features.
problem Difficulty in training generative models with differential privacy.
method Use pre-trained perceptual features to learn private data distribution.
result Generative models can generate high-quality images with low privacy budget (ϵ≈2). Transform non-private e-values into differentially private ones.
problem Leaking sensitive data through non-private e-values.
method Developed a novel biased multiplicative noise mechanism.
result Differentially private e-values maintain strong statistical power and asymptotic equivalence to non-private ones.
DiPriMe forests use private medians to create balanced tree splits for privacy-protected data.
problem Privacy concerns in training random forests due to multiple data queries.
method Proposes DiPriMe forests, which use a private median to generate balanced splits, ensuring differential privacy.
result DiPriMe forests achieve high utility while maintaining differential privacy, as shown both theoretically and empirically.
Private release of sensitive data enables fair learning.
problem Learning fair predictors with restricted sensitive data.
method Private release of sensitive demographic data, adapting non-discriminatory learners.
result The approach provides theoretical guarantees on performance for fair predictors.
PILLAR improves SP learning with less private data.
problem Efficiently learning with semi-private data under privacy constraints.
method Uses pre-trained public data features to reduce private data requirements.
result Significantly lower private labelled sample complexity achieved.
Many applications of Bayesian data analysis involve sensitive information, motivating methods which ensure that privacy is protected. We introduce a general privacy-preserving framework for Variational Bayes (VB), a widely used optimization-based Bayesian inference method. Our framework respects differential privacy, t…
Differentially private ensemble classifiers adapt to data streams while protecting privacy.
problem Adapting to evolving data characteristics while protecting private information.
method Unbounded ensemble updates, model agnostic approach.
result Outperforms competitors on various privacy, drift, and distribution settings.
Decouples data privatization from user preferences for privacy-preserving data.
problem Privacy-preserving data with user-specific private information.
method Decouples data privatization from user preferences using a Variational Autoencoder (VAE) and a generative filter trained by a GAN-type robust optimization.
result Effective privatization of data with minimal disturbance to utility, as shown by experiments on MNIST, UCI-Adult, and CelebA.
Private distribution learning with public data, leveraging sample compression schemes.
problem Private distribution learning with public and private samples under differential privacy constraints.
method Connection to sample compression schemes and list learning.
result At least d public samples are necessary for private learnability of Gaussians in R^d.
Algorithm selects public datasets for private machine learning.
problem Choosing the most suitable public dataset for private machine learning.
method Measures gradient subspace distance between public and private datasets.
result Excess risk scales with the subspace distance between gradients.
New privacy-preserving learning model for mixtures of private and public data.
problem Learning from datasets with both private and public data, where privacy concerns differ.
method Designing a differential privacy-preserving learning algorithm for a mixture of private and public sub-populations.
result Linear classifiers can be learned with sample complexity comparable to non-private PAC-learning, even when privacy status correlates with labels.
Study private query release with public data, reducing sample sizes.
problem Answering a wide range of statistical queries while maintaining privacy.
method Combines public and private samples to answer queries with differential privacy.
result Private and public sample complexities for different query classes.
DP-RandP improves privacy-utility tradeoff in DP-SGD by learning priors from random processes.
problem Improving the performance of differentially private stochastic gradient descent (DP-SGD) on private data.
method A three-phase approach that learns priors from images generated by random processes and transfers these priors to private data.
result New state-of-the-art accuracy on CIFAR10, CIFAR100, MedMNIST, and ImageNet for various privacy budgets.
Improved differentially private drug sensitivity prediction using compact representations.
problem Challenges in differentially private machine learning with genomic data.
method Representation learning using variational autoencoders, PCA, and random projection.
result Variational autoencoders provide the most accurate predictions for differentially private drug sensitivity prediction.
PASS protects private attributes by stochastically substituting data.
problem Protecting private attributes in ML services while maintaining data utility.
method PASS uses stochastic data substitution with a novel loss function derived from information theory.
result PASS effectively protects private attributes across various datasets.
Data is continuously generated by modern data sources, and a recent challenge in machine learning has been to develop techniques that perform well in an incremental (streaming) setting. In this paper, we investigate the problem of private machine learning, where as common in practice, the data is not given at once, but…
New method optimizes privacy and compute trade-offs for deep learning.
problem Privacy and compute trade-offs in deep learning training.
method Decoupling privacy analysis and experimental behavior, using TAN and scaling laws for DP-SGD.
result Stronger privacy guarantees with significant reduction in computational budget.
Private algorithms approximate matrices with private data.
problem Approximate matrices with same spectrum using private data.
method Differential privacy algorithms for unitary orbit optimization.
result Upper and lower bounds on approximation error.
Private statistical inference methods improve confidence interval lengths.
problem Constructing private confidence intervals with differential privacy.
method Proposed two private variants of non-parametric bootstrap.
result Achieve similar coverage accuracy to non-private methods with shorter intervals.