Differentially private hyperparameter tuning improves privacy in machine learning.
problem Hyperparameter tuning leaks private information through selected configurations.
method Local Bayesian optimization using Gaussian Process surrogate for private gradient approximation.
result DP-GIBO converges to locally optimal hyperparameters with polynomial dimensional dependence.
Optimizes private statistics with noisy methods.
problem Private inference in statistical models.
method Noisy optimization for M-estimators and confidence regions.
result Private estimators converge to non-private ones with high probability.
We study differentially private (DP) algorithms for stochastic convex optimization (SCO). In this problem the goal is to approximately minimize the population loss given i.i.d. samples from a distribution over convex and Lipschitz loss functions. A long line of existing work on private convex optimization focuses on th…
We present a provably optimal differentially private algorithm for the stochastic multi-arm bandit problem, as opposed to the private analogue of the UCB-algorithm [Mishra and Thakurta, 2015; Tossou and Dimitrakakis, 2016] which doesn't meet the recently discovered lower-bound of Ω ( K log ( T ) ε ) Ω\left(\frac{K\log(T)}ε \right) Ω ( ε K l o g ( T ) ) [Shar…
Near-optimal private tests for simple and MLR hypotheses developed under Gaussian differential privacy.
problem Developing private tests for simple and MLR hypotheses under Gaussian differential privacy.
method A private mean estimator with data-driven clamping bounds, constructing private test statistics.
result Private tests achieve the same asymptotic relative efficiency as non-private most powerful tests.
Private adaptive methods improve on traditional SGD for convex optimization.
problem Differential privacy constraints in gradient optimization.
method Differentially private variants of SGD and AdaGrad with adaptive stepsizes and non-isotropic clipping.
result Private AdaGrad outperforms private SGD in high-dimensional problems.
Improves zeroth-order optimization for private machine learning with public data.
problem High computation and memory cost of first-order DP methods.
method PAZO (Public Data Assisted Zeroth-order Optimization) framework.
result Achieves superior privacy/utility tradeoffs across tasks.
Study on gradient complexity of private optimization with private oracles.
problem Analyzing the efficiency of differentially private optimization algorithms.
method Lower bounds on the number of first-order oracle queries for private optimization.
result Lower bounds on the number of queries for private optimization algorithms, showing a dimension-dependent runtime penalty.
New framework reduces private mean estimation error with optimal efficiency.
problem Locally private mean estimation of high-dimensional vectors.
method ProjUnit framework: random projections, normalization, and optimal algorithm execution in lower dimensions.
result Optimal error up to a 1+o(1)-factor with computational efficiency and low communication complexity.
Optimizes differentially private kernel learning with random projection.
problem Privacy-preserving learning algorithms with optimal performance.
method Differentially private kernel ERM algorithm based on random projection in reproducing kernel Hilbert space.
result Achieves minimax-optimal excess risk rates for various loss functions.
Optimal private ERM and SCO with subquadratic gradient complexity.
problem Private optimization of non-smooth convex functions.
method Subquadratic gradient complexity algorithm using subsampling and smoothing.
result Achieved optimal excess empirical risk and population loss.
New private algorithms for online learning improve regret in high privacy regimes.
problem Private online learning from experts and convex optimization.
method Transformed lazy algorithms for differential privacy.
result Improved regret bounds for DP-OPE and DP-OCO.
Transforms robust algorithms into private ones with optimal error rates.
problem Balancing privacy and robustness in machine learning.
method Black-box transformation method to convert robust algorithms to private ones with optimal error rates.
result Optimal private estimators for various tasks, including Gaussian and PCA.
Optimizes sparse fine-tuning for privacy in neural networks.
problem Performance gap between DP-SGD and non-private fine-tuning.
method Optimization-based approach using private gradient information for selecting trainable weights.
result Our selection method leads to better prediction accuracy compared to existing approaches.
We present new differentially private algorithms for learning a large-margin halfspace. In contrast to previous algorithms, which are based on either differentially private simulations of the statistical query model or on private convex optimization, the sample complexity of our algorithms depends only on the margin of…
Paper proposes no-regret algorithms for private GP bandit optimization.
problem Private Gaussian process bandit optimization.
method Combines uniform kernel approximator with random perturbations for differentially private GP bandit algorithms.
result Provable no-regret algorithms for stationary kernel functions in two DP settings.
Private algorithms approximate matrices with private data.
problem Approximate matrices with same spectrum using private data.
method Differential privacy algorithms for unitary orbit optimization.
result Upper and lower bounds on approximation error.
Private optimization faster on interpolation problems with quadratic growth.
problem Private optimization in interpolation problems.
method Adaptive algorithm with improved sample complexity.
result Exponential improvement in private sample complexity for quadratic growth.
New framework for private convex optimization in arbitrary norms.
problem Private optimization of convex functions in non-Euclidean settings.
method Regularized exponential mechanism based on localization tools from convex geometry.
result First optimal privacy-utility tradeoffs for ℓ p \ell_p ℓ p norms and Schatten- p p p norms. New method uses public data to achieve optimal nonparametric classification with privacy constraints.
problem Optimal nonparametric classification under local differential privacy constraints.
method Locally differentially private classification tree with data-driven pruning.
result Achieves mini-max optimal convergence rate with public data.
Gradient perturbation, widely used for differentially private optimization, injects noise at every iterative update to guarantee differential privacy. Previous work first determines the noise level that can satisfy the privacy requirement and then analyzes the utility of noisy gradient updates as in the non-private cas…
Efficiently learns private models using public data.
problem Improving private learning performance with public data.
method Proves computationally efficient algorithms for private learning with public data.
result First computationally efficient algorithms for private learning with public data.
Paper introduces DP TDA for near-optimal private persistence diagrams.
problem Challenges in privatizing topological data analysis.
method Sensitivity analysis of persistence diagrams, use of exponential mechanism.
result Proposes near-optimal privacy mechanism for TDA.
Private KL distribution estimation improved with instance-optimality.
problem Minimizing KL divergence between true and estimated distributions.
method Construct minimax optimal private estimators, then focus on instance-optimality.
result Achieved instance-optimality up to constant factors for KL estimation.
The paper proposes differentially private sliced inverse regression algorithms for high-dimensional data.
problem Privacy concerns in high-dimensional data analysis.
method Differentially private sliced inverse regression algorithms designed for privacy preservation.
result Achieves minimax lower bounds up to logarithmic factors.
New algorithms optimize private convex optimization with faster rates for functions with κ-growth.
problem Optimizing private convex functions with varying difficulty and growth conditions.
method Adapts inverse sensitivity mechanism and localization techniques to achieve faster rates without knowing growth constant.
result Achieves faster privacy rates ( d / n ε ) f r a c κ κ − 1 ({\sqrt{d}}/{n\varepsilon})^{ fracκ{κ- 1}} ( d / n ε ) f r a c κ κ − 1 for functions with κ-growth. In this paper, we consider efficient differentially private empirical risk minimization from the viewpoint of optimization algorithms. For strongly convex and smooth objectives, we prove that gradient descent with output perturbation not only achieves nearly optimal utility, but also significantly improves the running …
A fund manager invests both the fund's assets and own private wealth in separate but potentially correlated risky assets, aiming to maximize expected utility from private wealth in the long run. If relative risk aversion and investment opportunities are constant, we find that the fund's portfolio depends only on the fu…
Algorithm ensures privacy while strictly adhering to constraints.
problem Differential privacy with linear constraints that must be strictly followed.
method Developed an algorithm that releases a nearly-optimal solution satisfying constraints with probability 1.
result Achieved nearly optimal performance while preserving privacy and strictly adhering to constraints.
The paper introduces a differentially private method for optimization on Riemannian manifolds.
problem Differential privacy in optimization constrained to Riemannian manifolds.
method Adding Gaussian noise to the Riemannian gradient on the tangent space, with privacy and utility guarantees.
result Privacy and utility guarantees for differentially private Riemannian optimization.
Improved privacy in RL with near-optimal regret bounds.
problem Privacy-preserving reinforcement learning in personalized decision-making systems.
method Differentially private algorithm based on LSVI-UCB++ with privacy-preserving techniques.
result Achieved a near-optimal regret bound of O(d * sqrt(H^3 * K) + H^(15/4) * d^(7/6) * K^(1/2) / ε).
Improved private agnostic learning with near-optimal sample complexity.
problem Private agnostic learning with arbitrary privacy parameters.
method Near-optimal sample complexity construction.
result Near-optimal extra sample complexity of \(\widetilde{O}(\mathrm{VC}(\mathcal{C})/α^2)\) for any \(\varepsilon \leq 1\).
Paper addresses private online convex optimization with optimal algorithms in various geometries and high-dimensional bandits.
problem Private online convex optimization with streaming and continual release data.
method Proposes a private variant of online Frank-Wolfe algorithm with recursive gradients for variance reduction.
result Achieves optimal excess risk in linear time for 1 < p ≤ 2 1<p\leq 2 1 < p ≤ 2 and state-of-the-art excess risk for 2 < p ≤ ∞ 2<p\leq\infty 2 < p ≤ ∞ . Dynamic model considers private asset markets' complexities.
problem Understanding and optimizing private asset allocation.
method State-of-the-art dynamic model with machine learning.
result Optimal investment policies quantified over fund life.
Optimizes survey design for private mean estimation with reduced variance.
problem Minimizing variance in private mean estimation with privacy constraints.
method Formulates optimal survey design as an optimization problem, determining optimal subsampling sizes to minimize variance.
result Identifies the first privacy-aware stratified sampling scheme that minimizes variance under different privacy mechanisms.
Study public-data assisted private stochastic optimization with labeled or unlabeled public data.
problem Limits and capability of public-data assisted differentially private (PA-DP) algorithms in stochastic convex optimization.
method Lower bounds for PA-DP mean estimation and novel methods for leveraging public data in private supervised learning.
result Achieved dimension independent rate for GLM with unlabeled public data, showing optimality.
New method for private linear regression under privacy constraints, achieving optimal rates.
problem Statistical complexity of private linear regression under unknown, ill-conditioned covariates.
method Information-Weighted Regression method
result Optimal convergence rates for both central and local privacy models.
Study private submodular maximization in streaming data.
problem Private maximization of submodular functions in streaming data.
method Established differentially private baselines and derived better trade-offs for decomposable submodular functions.
result Improved trade-offs between privacy and utility for decomposable submodular functions.
Paper uses relaxation techniques to find optimal brokerage fees with private signals.
problem Finding optimal brokerage fees for clients with private trading signals.
method Relaxation techniques to establish contract existence in asymmetric information settings.
result Existence of optimal brokerage fees established in a market model with private signals.
New DP algorithm improves privacy and efficiency for convex optimization.
problem Efficient, DP algorithms for convex optimization with strong excess risk bounds.
method Output perturbation for a broad class of tilted loss functions.
result Near optimal DP excess risk and runtime bounds for convex optimization.
DIFF2 improves differential privacy in nonconvex optimization with better utility bounds.
problem Improving differential privacy in nonconvex optimization with better utility bounds.
method DIFF2 constructs a differential private global gradient estimator using gradient differences.
result DIFF2 achieves a utility of \(\widetilde O(d^{2/3}/(n\varepsilon_{\mathrm{DP}})^{4/3})\), significantly better than \(\widetilde O(\sqrt{d}/(n\varepsilon_{\mathrm{DP}}))\).
Optimizes private learning with differential privacy for LASSO problems.
problem Private optimization of convex functions over ℓ 1 \ell_1 ℓ 1 -bounded domains. method Combines iterative localization with private regularized mirror descent and variance-reduced Frank-Wolfe algorithm.
result Achieves optimal excess population loss rates in ℓ 1 \ell_1 ℓ 1 geometry. Privacy affects how much data is needed for CVaR optimization.
problem Privacy constraints impact the effective sample size for CVaR optimization.
method Analyzes the privacy-relevant sample size and decomposes CVaR excess risk.
result The effective private tail sample size is εnτ, affecting CVaR learning rates.
Bayesian optimization is a powerful tool for fine-tuning the hyper-parameters of a wide variety of machine learning models. The success of machine learning has led practitioners in diverse real-world settings to learn classifiers for practical problems. As machine learning becomes commonplace, Bayesian optimization bec…
New algorithms for private generalized linear contextual bandits.
problem Private estimation and optimization for generalized linear models under differential privacy.
method Developed algorithms for stochastic and adversarial contexts under shuffle and joint differential privacy.
result Achieved private regret bounds for generalized linear models, differing from non-private rates by factors of d / ε \sqrt{d/\varepsilon} d / ε and d / ε \sqrt{d/\varepsilon} d / ε respectively. AdaDPS uses side information to improve private adaptive optimization.
problem Private adaptive optimization methods degrade when training with differential privacy.
method AdaDPS uses non-sensitive side information to precondition gradients.
result AdaDPS reduces the amount of noise needed for similar privacy guarantees, improving optimization performance.
Two private algorithms improve domain adaptation with privacy guarantees.
problem Improving predictions for a private target domain using public data.
method Two ( ε , δ ) (ε, δ) ( ε , δ ) -differentially private algorithms for supervised domain adaptation. result Private algorithms maintain performance close to non-private versions.
Three new oracle-efficient algorithms for private synthetic data release.
problem Constructing private synthetic data that preserves statistical query answers.
method Oracle-efficient algorithms using optimization oracles for differential privacy.
result Better accuracy in large workload and high privacy regime compared to state-of-the-art.