New algorithms for privately learning decision lists and halfspaces.
problem Private learning of decision lists and halfspaces.
method Differentially private algorithms for PAC and online models.
result Private algorithms match or surpass non-private guarantees.
We present a provably optimal differentially private algorithm for the stochastic multi-arm bandit problem, as opposed to the private analogue of the UCB-algorithm [Mishra and Thakurta, 2015; Tossou and Dimitrakakis, 2016] which doesn't meet the recently discovered lower-bound of Ω(εKlog(T)) [Shar…
New private algorithms for online learning improve regret in high privacy regimes.
problem Private online learning from experts and convex optimization.
method Transformed lazy algorithms for differential privacy.
result Improved regret bounds for DP-OPE and DP-OCO.
Private algorithms adapt from public to private domains with minimal labeled data.
problem Adapting from a public source domain to a private target domain with few labeled data.
method Differentially private discrepancy minimization algorithms based on Frank-Wolfe and Mirror-Descent methods.
result Effective adaptation with strong generalization and privacy guarantees.
Private learning of Gaussian Mixture Models without boundedness assumptions.
problem Private estimation of parameters of Gaussian Mixture Models with unbounded components.
method Reduction to non-private problem, blackbox privatization, Moitra and Valiant's algorithm.
result First sample complexity upper bound and polynomial time algorithm for privately learning GMMs.
Differentially private algorithms for submodular maximization under various constraints.
problem Maximizing decomposable submodular functions under constraints while preserving privacy.
method Designing differentially private algorithms for both monotone and non-monotone decomposable submodular maximization under general matroid constraints.
result Improved utility guarantees and competitive performance compared to non-private algorithms.
Efficiently learns private models using public data.
problem Improving private learning performance with public data.
method Proves computationally efficient algorithms for private learning with public data.
result First computationally efficient algorithms for private learning with public data.
Efficient private algorithms for estimating block models and mixture models.
problem Estimating block models and mixture models in high-dimensional settings.
method General tools for designing efficient private estimation algorithms.
result First efficient private algorithms for weak and exact recovery of stochastic block models.
We study differentially private (DP) algorithms for stochastic convex optimization (SCO). In this problem the goal is to approximately minimize the population loss given i.i.d. samples from a distribution over convex and Lipschitz loss functions. A long line of existing work on private convex optimization focuses on th…
We present new differentially private algorithms for learning a large-margin halfspace. In contrast to previous algorithms, which are based on either differentially private simulations of the statistical query model or on private convex optimization, the sample complexity of our algorithms depends only on the margin of…
Efficiently private clustering algorithms with tight approximation ratios.
problem Differentially private clustering of various types.
method Efficient algorithms achieving tight approximation ratios for clustering problems.
result Achieves approximation ratios similar to non-private algorithms with small additive errors.
Transforms robust algorithms into private ones with optimal error rates.
problem Balancing privacy and robustness in machine learning.
method Black-box transformation method to convert robust algorithms to private ones with optimal error rates.
result Optimal private estimators for various tasks, including Gaussian and PCA.
New algorithms reduce private bandit regret to nearly non-private levels.
problem Differentially private adversarial bandits and expert advice.
method Conversion of non-private algorithms to private, new algorithms for bandits and expert advice.
result Improved regret bounds for private bandits, sublinear for small ε.
Private algorithms approximate matrices with private data.
problem Approximate matrices with same spectrum using private data.
method Differential privacy algorithms for unitary orbit optimization.
result Upper and lower bounds on approximation error.
Two private algorithms improve domain adaptation with privacy guarantees.
problem Improving predictions for a private target domain using public data.
method Two (ε,δ)-differentially private algorithms for supervised domain adaptation. result Private algorithms maintain performance close to non-private versions.
New algorithms for community detection in graphs with privacy constraints.
problem Community recovery in stochastic block models with node-wise privacy.
method Spectral clustering with privacy mechanisms, including privatized PCA, convex optimization, and matrix estimation.
result Developed algorithms that are computable in polynomial-time and achieve consistent community estimation under node differential privacy.
The paper proposes differentially private sliced inverse regression algorithms for high-dimensional data.
problem Privacy concerns in high-dimensional data analysis.
method Differentially private sliced inverse regression algorithms designed for privacy preservation.
result Achieves minimax lower bounds up to logarithmic factors.
Bounds on Littlestone dimension for private learning and online prediction.
problem Understanding the Littlestone dimension of composed classes for private learning.
method Deriving bounds on Littlestone dimension and transforming private learners.
result Improved bounds on sample complexity for private learning.
New private learning algorithms improve utility in tasks with public features.
problem Private learning with public features in recommendation and ad prediction.
method Developed algorithms that protect only certain sufficient statistics, improving utility for linear regression and private recommendation benchmarks.
result Achieved state-of-the-art performance on private recommendation benchmarks.
Paper introduces algorithms for private decision tree learning.
problem Private decision tree learning in distributed settings.
method Proposes DP-TopDown, NoisyCounts, and LocalRNM.
result First utility guarantees for private decision tree learning.
New framework reduces private mean estimation error with optimal efficiency.
problem Locally private mean estimation of high-dimensional vectors.
method ProjUnit framework: random projections, normalization, and optimal algorithm execution in lower dimensions.
result Optimal error up to a 1+o(1)-factor with computational efficiency and low communication complexity.
Differentially private graph learning via bounded sensitivity PPR.
problem Protecting user data in graph learning algorithms.
method Proposes a sensitivity-bounded personalized PageRank (PPR) algorithm.
result Achieves similar accuracy to non-private algorithms with large degrees.
Paper develops a private algorithm for multi-agent learning in bandits.
problem Private cooperative learning in decentralized systems.
method Developed extsc{FedUCB} algorithm for multi-agent learning.
result Improves pseudoregret bounds and empirical performance.
Private adaptive methods improve on traditional SGD for convex optimization.
problem Differential privacy constraints in gradient optimization.
method Differentially private variants of SGD and AdaGrad with adaptive stepsizes and non-isotropic clipping.
result Private AdaGrad outperforms private SGD in high-dimensional problems.
Improved private learning of halfspaces with reduced sample complexity.
problem Private learning of halfspaces with reduced sample complexity.
method Iterative algorithm for solving linear feasibility problem, improving state-of-the-art results.
result Sample complexity reduced to d2.5⋅2log∗∣G∣, improving d2 factor. Paper develops private synthetic data for sensitive data analysis.
problem Analyzing sensitive data while preserving privacy.
method Adapting a private density estimation algorithm for synthetic data generation.
result Better computational guarantees for synthetic data generation.
Algorithm selects public datasets for private machine learning.
problem Choosing the most suitable public dataset for private machine learning.
method Measures gradient subspace distance between public and private datasets.
result Excess risk scales with the subspace distance between gradients.
Efficient private matrix analysis algorithms for recent variants.
problem Private analysis of recent matrix updates.
method Identifying sufficient conditions on positive semidefinite matrices.
result First efficient differentially private algorithms for various matrix analysis tasks.
Paper proposes no-regret algorithms for private GP bandit optimization.
problem Private Gaussian process bandit optimization.
method Combines uniform kernel approximator with random perturbations for differentially private GP bandit algorithms.
result Provable no-regret algorithms for stationary kernel functions in two DP settings.
Optimizes differentially private kernel learning with random projection.
problem Privacy-preserving learning algorithms with optimal performance.
method Differentially private kernel ERM algorithm based on random projection in reproducing kernel Hilbert space.
result Achieves minimax-optimal excess risk rates for various loss functions.
We study the contextual linear bandit problem, a version of the standard stochastic multi-armed bandit (MAB) problem where a learner sequentially selects actions to maximize a reward which depends also on a user provided per-round context. Though the context is chosen arbitrarily or adversarially, the reward is assumed…
Private learning is hard when data is long-tailed.
problem Achieving both privacy and fairness in machine learning with long-tailed data.
method Theoretical analysis and experimental validation on various datasets and algorithms.
result Relaxing overall accuracy can lead to good fairness even with strict privacy requirements.
New algorithms improve privacy and utility of large language models.
problem Privacy-preserving fine-tuning of large language models.
method Meta-framework for differentially private fine-tuning, inspired by recent success in fine-tuning.
result Private fine-tuned models achieve utility close to non-private models, with improved privacy and efficiency.
We show that every approximately differentially private learning algorithm (possibly improper) for a class H with Littlestone dimension~d requires Ω(log∗(d)) examples. As a corollary it follows that the class of thresholds over N can not be learned in a private manner; this resolves open qu…
Efficiently estimates private least squares with linear error growth.
problem Private estimation of ordinary least squares with bounded residuals and leverage.
method Scaled noise added to a stable nonprivate estimator of the regression vector.
result Near-optimal accuracy guarantee with linear error growth in dimension.
We present novel, computationally efficient, and differentially private algorithms for two fundamental high-dimensional learning problems: learning a multivariate Gaussian and learning a product distribution over the Boolean hypercube in total variation distance. The sample complexity of our algorithms nearly matches t…
Many applications of machine learning, such as human health research, involve processing private or sensitive information. Privacy concerns may impose significant hurdles to collaboration in scenarios where there are multiple sites holding data and the goal is to estimate properties jointly across all datasets. Differe…
Differentially private algorithms protect model explanations from leaking training data.
problem Model explanations can leak training data, compromising privacy.
method Adaptive differentially private gradient descent algorithm to produce accurate, private explanations.
result Privacy amplification and reduction of overall privacy loss on explanation data.
In this paper, we consider efficient differentially private empirical risk minimization from the viewpoint of optimization algorithms. For strongly convex and smooth objectives, we prove that gradient descent with output perturbation not only achieves nearly optimal utility, but also significantly improves the running …
New algorithm estimates Gaussian means and covariances efficiently and privately.
problem Estimating Gaussian parameters privately and efficiently.
method Differentially private preconditioner to transform arbitrary Gaussian samples.
result First polynomial-time, sample-efficient estimator for arbitrary Gaussian distributions.
We present differentially private algorithms for the stochastic Multi-Armed Bandit (MAB) problem. This is a problem for applications such as adaptive clinical trials, experiment design, and user-targeted advertising where private information is connected to individual rewards. Our major contribution is to show that the…
Optimal private ERM and SCO with subquadratic gradient complexity.
problem Private optimization of non-smooth convex functions.
method Subquadratic gradient complexity algorithm using subsampling and smoothing.
result Achieved optimal excess empirical risk and population loss.
Polynomial-time algorithm estimates mean with bounded covariance using differential privacy.
problem Estimating mean of a d-variate distribution with differential privacy constraints.
method Sum of Squares (SoS) exponential mechanism for polynomial-time differentially private estimation.
result First polynomial-time algorithm with O(d) samples for mean estimation under pure differential privacy. Develops Merton's model for private companies using DDM.
problem Lack of observable asset values for private companies.
method Uses dividend discount model (DDM) to develop structural model.
result Obtains closed-form formulas for equity and liability values, default probability.
Improved privacy in RL with near-optimal regret bounds.
problem Privacy-preserving reinforcement learning in personalized decision-making systems.
method Differentially private algorithm based on LSVI-UCB++ with privacy-preserving techniques.
result Achieved a near-optimal regret bound of O(d * sqrt(H^3 * K) + H^(15/4) * d^(7/6) * K^(1/2) / ε).
Two new differentially private optimization algorithms derived from accelerated methods.
problem Improving privacy in optimization algorithms while maintaining convergence rates.
method Polyak's heavy ball method and Nesterov's accelerated gradient method with differential privacy.
result The proposed algorithms outperform existing differentially private optimization methods.
FPFL mitigates unfairness in private federated learning.
problem Differential privacy degrades model performance on under-represented groups.
method Extends modified method of differential multipliers to private federated learning.
result FPFL reduces unfairness in trained models on private federated learning.
Three new oracle-efficient algorithms for private synthetic data release.
problem Constructing private synthetic data that preserves statistical query answers.
method Oracle-efficient algorithms using optimization oracles for differential privacy.
result Better accuracy in large workload and high privacy regime compared to state-of-the-art.