Differentially private hyperparameter tuning improves privacy in machine learning.
problem Hyperparameter tuning leaks private information through selected configurations.
method Local Bayesian optimization using Gaussian Process surrogate for private gradient approximation.
result DP-GIBO converges to locally optimal hyperparameters with polynomial dimensional dependence.
Optimizes private statistics with noisy methods.
problem Private inference in statistical models.
method Noisy optimization for M-estimators and confidence regions.
result Private estimators converge to non-private ones with high probability.
Optimizes differentially private kernel learning with random projection.
problem Privacy-preserving learning algorithms with optimal performance.
method Differentially private kernel ERM algorithm based on random projection in reproducing kernel Hilbert space.
result Achieves minimax-optimal excess risk rates for various loss functions.
Transforms robust algorithms into private ones with optimal error rates.
problem Balancing privacy and robustness in machine learning.
method Black-box transformation method to convert robust algorithms to private ones with optimal error rates.
result Optimal private estimators for various tasks, including Gaussian and PCA.
The paper proposes differentially private sliced inverse regression algorithms for high-dimensional data.
problem Privacy concerns in high-dimensional data analysis.
method Differentially private sliced inverse regression algorithms designed for privacy preservation.
result Achieves minimax lower bounds up to logarithmic factors.
Near-optimal private tests for simple and MLR hypotheses developed under Gaussian differential privacy.
problem Developing private tests for simple and MLR hypotheses under Gaussian differential privacy.
method A private mean estimator with data-driven clamping bounds, constructing private test statistics.
result Private tests achieve the same asymptotic relative efficiency as non-private most powerful tests.
Paper proposes no-regret algorithms for private GP bandit optimization.
problem Private Gaussian process bandit optimization.
method Combines uniform kernel approximator with random perturbations for differentially private GP bandit algorithms.
result Provable no-regret algorithms for stationary kernel functions in two DP settings.
The paper introduces a differentially private method for optimization on Riemannian manifolds.
problem Differential privacy in optimization constrained to Riemannian manifolds.
method Adding Gaussian noise to the Riemannian gradient on the tangent space, with privacy and utility guarantees.
result Privacy and utility guarantees for differentially private Riemannian optimization.
Gradient perturbation, widely used for differentially private optimization, injects noise at every iterative update to guarantee differential privacy. Previous work first determines the noise level that can satisfy the privacy requirement and then analyzes the utility of noisy gradient updates as in the non-private cas…
DIFF2 improves differential privacy in nonconvex optimization with better utility bounds.
problem Improving differential privacy in nonconvex optimization with better utility bounds.
method DIFF2 constructs a differential private global gradient estimator using gradient differences.
result DIFF2 achieves a utility of \(\widetilde O(d^{2/3}/(n\varepsilon_{\mathrm{DP}})^{4/3})\), significantly better than \(\widetilde O(\sqrt{d}/(n\varepsilon_{\mathrm{DP}}))\).
Private adaptive methods improve on traditional SGD for convex optimization.
problem Differential privacy constraints in gradient optimization.
method Differentially private variants of SGD and AdaGrad with adaptive stepsizes and non-isotropic clipping.
result Private AdaGrad outperforms private SGD in high-dimensional problems.
Differentially private GANs improve image privacy without significant quality loss.
problem Anonymizing image data sets while maintaining image quality.
method Training GANs with differential privacy on MNIST, analyzing privacy-utility trade-offs and explaining optimization methods.
result An increasing privacy budget adds little to generated image quality, revealing a saturated training regime.
Two new differentially private optimization algorithms derived from accelerated methods.
problem Improving privacy in optimization algorithms while maintaining convergence rates.
method Polyak's heavy ball method and Nesterov's accelerated gradient method with differential privacy.
result The proposed algorithms outperform existing differentially private optimization methods.
We present new differentially private algorithms for learning a large-margin halfspace. In contrast to previous algorithms, which are based on either differentially private simulations of the statistical query model or on private convex optimization, the sample complexity of our algorithms depends only on the margin of…
We study differentially private (DP) algorithms for stochastic convex optimization (SCO). In this problem the goal is to approximately minimize the population loss given i.i.d. samples from a distribution over convex and Lipschitz loss functions. A long line of existing work on private convex optimization focuses on th…
New algorithms for private generalized linear contextual bandits.
problem Private estimation and optimization for generalized linear models under differential privacy.
method Developed algorithms for stochastic and adversarial contexts under shuffle and joint differential privacy.
result Achieved private regret bounds for generalized linear models, differing from non-private rates by factors of d / ε \sqrt{d/\varepsilon} d / ε and d / ε \sqrt{d/\varepsilon} d / ε respectively. Optimal private ERM and SCO with subquadratic gradient complexity.
problem Private optimization of non-smooth convex functions.
method Subquadratic gradient complexity algorithm using subsampling and smoothing.
result Achieved optimal excess empirical risk and population loss.
Improved privacy and utility in machine learning with adaptive differential privacy.
problem Enhancing privacy in machine learning models while maintaining utility.
method Adaptive differentially private (ADP) learning method that optimally adapts noise to stepsize.
result ADP method significantly improves utility compared to standard differentially private methods.
New method uses public data to achieve optimal nonparametric classification with privacy constraints.
problem Optimal nonparametric classification under local differential privacy constraints.
method Locally differentially private classification tree with data-driven pruning.
result Achieves mini-max optimal convergence rate with public data.
Improved privacy in RL with near-optimal regret bounds.
problem Privacy-preserving reinforcement learning in personalized decision-making systems.
method Differentially private algorithm based on LSVI-UCB++ with privacy-preserving techniques.
result Achieved a near-optimal regret bound of O(d * sqrt(H^3 * K) + H^(15/4) * d^(7/6) * K^(1/2) / ε).
Optimizes sparse fine-tuning for privacy in neural networks.
problem Performance gap between DP-SGD and non-private fine-tuning.
method Optimization-based approach using private gradient information for selecting trainable weights.
result Our selection method leads to better prediction accuracy compared to existing approaches.
Differential privacy is concerned about the prediction quality while measuring the privacy impact on individuals whose information is contained in the data. We consider differentially private risk minimization problems with regularizers that induce structured sparsity. These regularizers are known to be convex but they…
New DP algorithm improves privacy and efficiency for convex optimization.
problem Efficient, DP algorithms for convex optimization with strong excess risk bounds.
method Output perturbation for a broad class of tilted loss functions.
result Near optimal DP excess risk and runtime bounds for convex optimization.
Paper improves privacy in SGD with low noise, achieving optimal risk rates.
problem Privacy-preserving machine learning with good performance.
method Differentially private SGD with low-noise analysis.
result Achieves optimal excess risk rates for non-smooth losses.
Algorithm ensures privacy while strictly adhering to constraints.
problem Differential privacy with linear constraints that must be strictly followed.
method Developed an algorithm that releases a nearly-optimal solution satisfying constraints with probability 1.
result Achieved nearly optimal performance while preserving privacy and strictly adhering to constraints.
Differential privacy is a cryptographically-motivated definition of privacy which has gained significant attention over the past few years. Differentially private solutions enforce privacy by adding random noise to a function computed over the data, and the challenge in designing such algorithms is to control the added…
Review of privacy-preserving linear models for high-dimensional data.
problem Overfitting and data memorization in high-dimensional linear models.
method Comprehensive comparison of optimization techniques for differentially private high-dimensional linear models.
result Coordinate-optimized algorithms perform best in empirical tests.
New private algorithms for online learning improve regret in high privacy regimes.
problem Private online learning from experts and convex optimization.
method Transformed lazy algorithms for differential privacy.
result Improved regret bounds for DP-OPE and DP-OCO.
Differentially private random block coordinate descent improves utility in machine learning.
problem Lack of privacy in classical CD methods when handling sensitive information.
method Proposes a differentially private random block coordinate descent method using sketch matrices and importance sampling.
result Demonstrates improved convergence rates and utility guarantees compared to non-private methods.
First DP algorithm for Wasserstein barycenters on private data.
problem Computing Wasserstein barycenters on private datasets.
method Differentially private algorithms for Wasserstein barycenters.
result High-quality private barycenters with strong accuracy-privacy tradeoffs.
Paper develops DP methods for low-rank matrix estimation with near-optimal performance.
problem Estimating a low-rank matrix under differential privacy constraints.
method Introduced computationally efficient DP-initialization and Riemannian optimization-based DP-RGrad algorithm.
result DP-RGrad achieves near-optimal convergence rate under weak differential privacy constraints.
Paper introduces DP TDA for near-optimal private persistence diagrams.
problem Challenges in privatizing topological data analysis.
method Sensitivity analysis of persistence diagrams, use of exponential mechanism.
result Proposes near-optimal privacy mechanism for TDA.
Efficiently learns private models using public data.
problem Improving private learning performance with public data.
method Proves computationally efficient algorithms for private learning with public data.
result First computationally efficient algorithms for private learning with public data.
Optimizes private learning with differential privacy for LASSO problems.
problem Private optimization of convex functions over ℓ 1 \ell_1 ℓ 1 -bounded domains. method Combines iterative localization with private regularized mirror descent and variance-reduced Frank-Wolfe algorithm.
result Achieves optimal excess population loss rates in ℓ 1 \ell_1 ℓ 1 geometry. New framework for private convex optimization in arbitrary norms.
problem Private optimization of convex functions in non-Euclidean settings.
method Regularized exponential mechanism based on localization tools from convex geometry.
result First optimal privacy-utility tradeoffs for ℓ p \ell_p ℓ p norms and Schatten- p p p norms. Develops a high-dimensional differentially-private EM algorithm with near-optimal statistical guarantees.
problem Designing differentially-private EM algorithms for high-dimensional latent variable models.
method Noisy iterative hard-thresholding, statistical guarantees, near-optimal convergence rates.
result Near-optimal statistical guarantees and minimax rate optimality in high-dimensional settings.
Second-order methods improve differential privacy in convex optimization.
problem Improving differential privacy in convex optimization.
method Developed a private variant of the regularized cubic Newton method for strongly convex loss functions.
result Achieves quadratic convergence and optimal excess loss for strongly convex loss functions.
New algorithms for differentially private optimization in convex and non-convex settings with near-optimal rates.
problem Differentially private optimization in convex and non-convex settings.
method Developed algorithms for convex and non-convex settings with near-optimal excess population risk.
result Achieved near-optimal rates in near-linear time for convex settings and nearly dimension independent rates for non-convex settings.
Study on gradient complexity of private optimization with private oracles.
problem Analyzing the efficiency of differentially private optimization algorithms.
method Lower bounds on the number of first-order oracle queries for private optimization.
result Lower bounds on the number of queries for private optimization algorithms, showing a dimension-dependent runtime penalty.
Paper proposes a differentially private test for joint dependence among random vectors.
problem Detecting joint dependence among sensitive data while maintaining privacy.
method Differentially private permutation methodology for dHSIC test.
result Proposed test attains minimax optimal power across privacy regimes.
PriMORL trains private RL policies on offline data.
problem Private reinforcement learning on offline data.
method PriMORL learns DP models of the environment and optimizes a policy on the penalized private model.
result PriMORL enables training of private RL agents on complex tasks.
Study private submodular maximization in streaming data.
problem Private maximization of submodular functions in streaming data.
method Established differentially private baselines and derived better trade-offs for decomposable submodular functions.
result Improved trade-offs between privacy and utility for decomposable submodular functions.
Paper tackles LDP bandits learning with improved results and sub-linear regret.
problem Contextual bandits learning with LDP privacy constraints.
method Simple black-box reduction frameworks for context-free bandits, extended to GLB.
result First result for BCO with multi-point feedback under LDP, sub-linear regret for GLB.
Improves zeroth-order optimization for private machine learning with public data.
problem High computation and memory cost of first-order DP methods.
method PAZO (Public Data Assisted Zeroth-order Optimization) framework.
result Achieves superior privacy/utility tradeoffs across tasks.
Paper optimizes private PCA for covariance estimation in statistics.
problem Private estimation of covariance matrices and principal components.
method Developed differentially private estimators for spiked covariance model.
result Established minimax rates of convergence for principal components and covariance matrix estimation.
Simple DP algorithms find approximate solutions for nonconvex ERM.
problem Finding approximate solutions to nonconvex ERM problems with privacy.
method Differential privacy, descent directions, line search, mini-batching, two-phase strategy.
result Effective algorithms for nonconvex ERM with privacy guarantees.
Private algorithms approximate matrices with private data.
problem Approximate matrices with same spectrum using private data.
method Differential privacy algorithms for unitary orbit optimization.
result Upper and lower bounds on approximation error.
Improved differentially private deep learning with group-wise clipping techniques.
problem Efficiency and privacy trade-offs in deep learning models.
method Group-wise clipping techniques (per-layer and per-device) to reduce compute time and memory overhead.
result Private learning with group-wise clipping achieves similar or better performance than non-private learning with less wall time.