Private estimation with public data reduces sample complexity.
problem Estimating private distributions with limited public data.
method Differentially private estimation with public data under constraints of pure or concentrated DP.
result Public data can significantly reduce private sample complexity for estimation.
Study private query release with public data, reducing sample sizes.
problem Answering a wide range of statistical queries while maintaining privacy.
method Combines public and private samples to answer queries with differential privacy.
result Private and public sample complexities for different query classes.
Improved private sample complexity for answering classification queries.
problem Designing an algorithm to accurately answer classification queries while maintaining differential privacy.
method Formally studied in agnostic PAC model, derived new upper bound on private sample complexity.
result Improved private sample complexity bound for answering classification queries.
Proves exponential sample complexity separations in local differential privacy.
problem Sample complexity in locally private protocols.
method Connection between communication complexity and sample complexity, using specific lower bounds for two problems.
result Exponential separations between differentially private protocols.
Improved private learning of halfspaces with reduced sample complexity.
problem Private learning of halfspaces with reduced sample complexity.
method Iterative algorithm for solving linear feasibility problem, improving state-of-the-art results.
result Sample complexity reduced to d 2.5 ⋅ 2 log ∗ ∣ G ∣ d^{2.5} \cdot 2^{\log^*|G|} d 2.5 ⋅ 2 l o g ∗ ∣ G ∣ , improving d 2 d^2 d 2 factor. Improved private agnostic learning with near-optimal sample complexity.
problem Private agnostic learning with arbitrary privacy parameters.
method Near-optimal sample complexity construction.
result Near-optimal extra sample complexity of \(\widetilde{O}(\mathrm{VC}(\mathcal{C})/α^2)\) for any \(\varepsilon \leq 1\).
We study the sample complexity of private synthetic data generation over an unbounded sized class of statistical queries, and show that any class that is privately proper PAC learnable admits a private synthetic data generator (perhaps non-efficient). Previous work on synthetic data generators focused on the case that …
Private learner for halfspaces with efficient sample complexity.
problem Private learning of halfspaces over arbitrary domains.
method Differential privacy for center point location and its application to halfspace learning.
result Private halfspace learning with sample complexity p o l y ( d , 2 log ∗ ∣ X ∣ ) poly(d, 2^{\log^*|X|}) p o l y ( d , 2 l o g ∗ ∣ X ∣ ) . We compare the sample complexity of private learning [Kasiviswanathan et al. 2008] and sanitization~[Blum et al. 2008] under pure ε ε ε -differential privacy [Dwork et al. TCC 2006] and approximate ( ε , δ ) (ε,δ) ( ε , δ ) -differential privacy [Dwork et al. Eurocrypt 2006]. We show that the sample complexity of these tasks under approxima…
Private optimization faster on interpolation problems with quadratic growth.
problem Private optimization in interpolation problems.
method Adaptive algorithm with improved sample complexity.
result Exponential improvement in private sample complexity for quadratic growth.
New private algorithms learn large-margin halfspaces efficiently.
problem Learning large-margin halfspaces with privacy constraints.
method Differentially private algorithms based on a new approach.
result Sample complexity depends only on the margin, not dimension.
New findings show limitations in converting private learning to online learning efficiently.
problem Limitations in converting private learning to online learning efficiently.
method Assuming one-way functions, we show an efficient conversion from pure-private learners to online learners is impossible.
result Efficient conversion from pure-private learners to online learners is impossible under certain assumptions.
The paper explores learning with a mix of private and public data while maintaining privacy.
problem Learning with a mix of private and public data while ensuring differential privacy.
method Designing a learning algorithm that satisfies differential privacy only with respect to private examples.
result A hypothesis class of VC-dimension d can be agnostically learned up to an excess error of α using only (roughly) d/α public examples and d/α^2 private labeled examples.
Framework for private, noise-tolerant, and efficient learning algorithms.
problem Private and efficient learning of large-margin halfspaces in noisy environments.
method Simple framework using differential privacy and noise tolerance conditions.
result Noise-tolerant and private PAC learners for large-margin halfspaces with sample complexity independent of dimension.
New algorithm learns Gaussian mixtures privately with optimal sample complexity.
problem Learning parameters of Gaussian mixtures under differential privacy constraints.
method Differentially private algorithm based on Achlioptas and McSherry's approach.
result Sample complexity matches non-private algorithm up to lower order terms.
Study differentially private methods for learning Hawkes processes.
problem Lack of thorough analysis on sample complexity for learning Hawkes processes parameters and releasing differentially private versions.
method Developed non-private and differentially private estimators for Hawkes processes parameters.
result Obtained sample complexity results for both private and non-private settings.
New private identity testers for high-dimensional distributions with improved sample complexity.
problem Testing goodness-of-fit for high-dimensional product distributions under differential privacy.
method Developed novel differentially private testers for multivariate product distributions, including Gaussians and binary product distributions.
result Achieved sample complexity matching the minimax sample complexity of O ( d 1 / 2 / α 2 ) O(d^{1/2}/α^2) O ( d 1/2 / α 2 ) in many parameter regimes. New method for private density estimation of high-dimensional Gaussian mixtures.
problem Private density estimation for mixtures of unrestricted high-dimensional Gaussians.
method Exploits list global stability to prove upper bound on sample complexity.
result First upper bound on sample complexity for agnostic private density estimation.
Private learning of Gaussian Mixture Models without boundedness assumptions.
problem Private estimation of parameters of Gaussian Mixture Models with unbounded components.
method Reduction to non-private problem, blackbox privatization, Moitra and Valiant's algorithm.
result First sample complexity upper bound and polynomial time algorithm for privately learning GMMs.
Private distribution learning with public data, leveraging sample compression schemes.
problem Private distribution learning with public and private samples under differential privacy constraints.
method Connection to sample compression schemes and list learning.
result At least d public samples are necessary for private learnability of Gaussians in R^d.
Improved sample efficiency for private learning of Gaussian mixtures.
problem Learning mixtures of Gaussians with differential privacy.
method Inverse sensitivity mechanism, sample compression, sumset volume bounds.
result Proved optimal sample complexity for private learning of mixtures of Gaussians.
Paper analyzes InstaHide's security, recovering all private images with provable guarantee.
problem Protecting privacy of training data in neural networks.
method Unified framework to understand and analyze attacks on InstaHide, presenting a new algorithm to recover all private images with provable guarantee.
result InstaHide is computationally secure but not information-theoretically secure when mixing two private images.
New bounds for private learning of high-dimensional Gaussian distributions.
problem Learning high-dimensional Gaussian distributions under differential privacy constraints.
method Analytic tools for constructing global covers from local covers, modified hypothesis selection techniques.
result Near-optimal sample complexity bounds for general Gaussians, conjectured to be near-optimal in the general case.
New bounds on private mean estimation for heavy-tailed distributions.
problem Estimating the mean of heavy-tailed distributions under differential privacy constraints.
method Upper and lower bounds on sample complexity for differentially private mean estimation.
result Qualitatively different sample complexity compared to non-private estimation, with a factor of O ( d ) O(d) O ( d ) larger for multivariate cases. Algorithm selects private hypothesis from unknown distribution.
problem Private selection of hypothesis from unknown distribution.
method Differentially private algorithm for hypothesis selection.
result Sample complexity of O ( log m α 2 + log m α ε ) O(\frac{\log m}{α^2} + \frac{\log m}{α\varepsilon}) O ( α 2 l o g m + α ε l o g m ) . Study quantile multi-armed bandits for identifying the best arm with a specified quantile level.
problem Identifying the arm with the highest quantile in multi-armed bandits with private rewards.
method Proposed a (non-private) and differentially private successive elimination algorithms for best-arm identification.
result The proposed algorithms are essentially optimal for quantile bandit problems, with finite sample complexity even for distributions with infinite support-size.
Private ALS method improves matrix completion with tighter rates and better privacy.
problem Differential privacy in matrix completion for user-level privacy.
method Joint differentially private ALS method with tighter sample complexity and privacy trade-offs.
result Achieves nearly optimal sample complexity and best privacy/utility trade-off.
Public pretraining improves private model training even in extreme distribution shift scenarios.
problem Improving private model training accuracy in settings with large distribution shift.
method Empirical evaluation and theoretical explanation of public representations improving private training accuracy.
result Public representations can improve private training accuracy by up to 67% over private training from scratch in settings with large distribution shift.
We present novel, computationally efficient, and differentially private algorithms for two fundamental high-dimensional learning problems: learning a multivariate Gaussian and learning a product distribution over the Boolean hypercube in total variation distance. The sample complexity of our algorithms nearly matches t…
PILLAR improves SP learning with less private data.
problem Efficiently learning with semi-private data under privacy constraints.
method Uses pre-trained public data features to reduce private data requirements.
result Significantly lower private labelled sample complexity achieved.
Efficiently estimates Gaussian distributions privately and robustly.
problem Private and robust estimation of Gaussian distributions.
method Efficient algorithms for pure and approximate differential privacy models.
result Optimal sample complexity in both pure and approximate differential privacy settings.
Paper optimizes privacy-preserving distribution estimation for sparse data.
problem Sparse distribution estimation under local differential privacy constraints.
method Compressive sensing approaches for privacy-preserving estimation.
result Significant reduction in sample complexity for approximately sparse distributions.
Hypothesis testing plays a central role in statistical inference, and is used in many settings where privacy concerns are paramount. This work answers a basic question about privately testing simple hypotheses: given two distributions P P P and Q Q Q , and a privacy level ε \varepsilon ε , how many i.i.d. samples are needed to…
New algorithms achieve stable and private predictions with reduced sample complexity.
problem Achieving stable and private predictions in binary classification.
method Developed new algorithms for stable and private prediction, reducing sample complexity.
result Achieved stable and private predictions with reduced sample complexity for VC dimension d d d . New private mean estimation method works well for anisotropic data.
problem Private mean estimation for high-dimensional anisotropic distributions.
method Developed ( ε , δ ) (\varepsilon,δ) ( ε , δ ) -differentially private estimators with dimension-independent sample complexity. result Achieved optimal sample complexity for anisotropic subgaussian distributions.
Polynomial-time DP algorithm for learning Gaussians with matching sample complexity.
problem Learning Gaussian distributions while maintaining privacy.
method General framework for reducing DP estimation to non-private, polynomial-time algorithm for Gaussian learning.
result Matching sample complexity to information-theoretic upper bound for Gaussian learning.
Paper tackles private optimization for non-smooth objectives efficiently.
problem Private stochastic convex optimization for non-smooth objectives.
method Noisy mirror descent algorithm.
result Achieves optimal rates in statistical complexity and number of queries.
New lower bounds for private covariance estimation of Gaussian distributions are proven.
problem Proving tight lower bounds for private estimation tasks under differential privacy.
method Generalized fingerprinting method for exponential families and private Assouad method.
result Tight lower bounds for private covariance estimation in Frobenius and spectral norms.
Efficiently learns private models using public data.
problem Improving private learning performance with public data.
method Proves computationally efficient algorithms for private learning with public data.
result First computationally efficient algorithms for private learning with public data.
Paper introduces DP TDA for near-optimal private persistence diagrams.
problem Challenges in privatizing topological data analysis.
method Sensitivity analysis of persistence diagrams, use of exponential mechanism.
result Proposes near-optimal privacy mechanism for TDA.
Improved privacy-preserving methods for estimating multiple samples from distributions.
problem Estimating multiple samples from distributions while maintaining privacy.
method Developed new multi-sampling techniques for differentially private data estimation.
result Achieved significant reduction in sample complexity for multi-sampling from finite domains and Gaussian distributions.
Lower bounds on queries for Bayesian private learning.
problem Estimating a target within error ε while keeping it hidden from an adversary.
method Lower bound proof using Fano's inequality and proportional-sampling estimators.
result Query complexity is on the order of L log ( 1 / ε ) L\log(1/\varepsilon) L log ( 1/ ε ) for accurate estimation. Differentially private statistical inference using β β β -divergence.
problem Achieving differential privacy without altering data generation.
method Sampling from a generalised posterior minimizing β β β -divergence. result More precise inference with broader applicability.
Differentially-private Bayes consistency rule for binary classification and density estimation.
problem Privacy constraints limit private learning in the distribution-free PAC model.
method Constructs a universally Bayes consistent learning rule that satisfies differential privacy.
result Private learning is possible for arbitrary distributions, even with a single algorithm.
Study privacy and robustness in learning halfspaces, proving hard trade-offs.
problem Balancing privacy and robustness in learning halfspaces.
method Proves nearly tight bounds on sample complexity for robust private learning of halfspaces.
result Robust and private learning is harder than robust or private learning alone.
Efficiently estimates prediction error in regression with Gaussian covariates under privacy constraints.
problem Private regression with Gaussian covariates under differential privacy constraints.
method Sum-of-Squares framework combined with robust estimators.
result Sample-optimal private regression algorithm with optimal error rates.
Differentially private random block coordinate descent improves utility in machine learning.
problem Lack of privacy in classical CD methods when handling sensitive information.
method Proposes a differentially private random block coordinate descent method using sketch matrices and importance sampling.
result Demonstrates improved convergence rates and utility guarantees compared to non-private methods.
Improved DP optimization for nonconvex, nonsmooth objectives with reduced sample complexity.
problem Differentially private optimization of nonconvex, nonsmooth objectives.
method Proposes single-pass and multi-pass DP algorithms with improved sample complexity.
result Sample complexity bounds improved by factors of Ω ( d ) Ω(\sqrt{d}) Ω ( d ) and Ω ( d 3 / 4 ) Ω(d^{3/4}) Ω ( d 3/4 ) .