This paper improves privacy bounds for DP algorithms using f-DP.
problem Difficulty in analyzing randomness in DP algorithms due to mixture distributions.
method Derives a closed-form expression for trade-off functions and analyzes f-DP. result Enhances privacy of DP-GD with random initialization and shuffling models.
DPConvCNP learns to predict private data accurately and privately.
problem Balancing privacy and accuracy in machine learning models.
method Meta-learning combined with improved DP mechanism.
result DPConvCNP outperforms DP GP baseline, especially on non-Gaussian data.
New DP framework using data truncation for efficient estimation.
problem Differential privacy in unbounded data support.
method Data truncation, exponential family distributions, maximum likelihood estimation, DP stochastic gradient descent.
result Near-optimal sample complexity for Gaussian mean and covariance estimation.
Lower bounds on query complexity for reconstructing private learner's training data.
problem Query complexity of reconstructing private learner's training data.
method Minimax analysis, Rényi DP, Metric DP framework.
result First known lower bounds on adversary's query complexity for various DP learners.
BNN-DP improves robustness analysis of Bayesian Neural Networks.
problem Ensuring robustness of Bayesian Neural Networks against adversarial attacks.
method Dynamic Programming applied to Bayesian Neural Networks as stochastic dynamical systems.
result BNN-DP provides tighter and more efficient bounds on prediction ranges compared to existing methods.
Many applications of machine learning, for example in health care, would benefit from methods that can guarantee privacy of data subjects. Differential privacy (DP) has become established as a standard for protecting learning results. The standard DP algorithms require a single trusted party to have access to the entir…
New privacy bounds for DP-SGD's last iterate, even with cyclic sampling.
problem Privacy of the last iterate in DP-SGD with cyclic sampling.
method Established new RDP upper bounds for the last iterate under realistic assumptions.
result Privacy bounds for DP-SGD's last iterate with cyclic sampling and clipping, even for nonconvex losses.
Second-order methods improve differential privacy in convex optimization.
problem Improving differential privacy in convex optimization.
method Developed a private variant of the regularized cubic Newton method for strongly convex loss functions.
result Achieves quadratic convergence and optimal excess loss for strongly convex loss functions.
Improved privacy-preserving methods for convex optimization with heavy-tailed data.
problem Privacy-preserving optimization of convex functions with heavy-tailed data.
method Developed algorithms for private mean estimation and convex optimization under concentrated differential privacy constraints.
result Achieved improved upper bounds on excess population risk for convex and strongly convex loss functions.
We propose Dirichlet Process mixtures of Generalized Linear Models (DP-GLM), a new method of nonparametric regression that accommodates continuous and categorical inputs, and responses that can be modeled by a generalized linear model. We prove conditions for the asymptotic unbiasedness of the DP-GLM regression mean fu…
Study the tradeoff between signal distortion and human perception over finite channels.
problem Characterize the distortion-perception tradeoff for finite channels with arbitrary metrics.
method Solve linear programming problems to compute the distortion-perception function and optimal reconstructions.
result DP function is piecewise linear in the perception index.
DP-means clustering was obtained as an extension of K-means clustering. While it is implemented with a simple and efficient algorithm, it can estimate the number of clusters simultaneously. However, DP-means is specifically designed for the average distortion measure. Therefore, it is vulnerable to outliers in data, …
Paper develops DP algorithms for isotonic regression over posets.
problem Differential privacy in isotonic regression over partially ordered sets.
method Developed pure-DP and near-matching lower bound algorithms for isotonic regression.
result Achieved near-matching bounds for isotonic regression with and without poset structure.
Paper addresses DP-SCO on heavy-tailed data, providing methods and results.
problem Designing DP algorithms for SCO on heavy-tailed data.
method Sample-and-aggregate framework, gradient smoothing and trimming.
result Achieved DP guarantees for various loss functions with different excess population risks.
DP models misspecify LF dependencies, leading to significant performance errors.
problem Misspecification of LF dependencies in DP models.
method Theoretical bounds and empirical analysis of modeling errors.
result Modeling errors can be substantial, even with sensible LF structures.
This paper improves privacy in federated learning without a trusted server.
problem Privacy in federated learning with silos that distrust each other.
method Introduces Inter-Silo Record-Level Differential Privacy (ISRL-DP) and accelerated algorithms for convex and smooth losses.
result Achieves optimal privacy and accuracy tradeoffs in federated learning.
New mechanism for pure differential privacy on functional summaries using Laplace-like process.
problem Challenges in achieving differential privacy for complex, structured functional summaries.
method Independent Component Laplace Process (ICLP) mechanism for infinite-dimensional Hilbert space.
result Effective enhancement of utility of private summaries through oversmoothing.
Paper revisits DP-SCO in Euclidean and ℓpd spaces, focusing on constrained and bounded sets.
problem Differentially private stochastic convex optimization in constrained and bounded sets in Euclidean and ℓpd spaces. method Proposes methods achieving excess population risks dependent on Gaussian width of the constraint set, and novel algorithms for unconstrained and heavy-tailed data.
result Theoretical results for DP-SCO in ℓpd spaces, including optimal bounds for strongly convex functions. Machine learning (ML) models trained by differentially private stochastic gradient descent (DP-SGD) have much lower utility than the non-private ones. To mitigate this degradation, we propose a DP Laplacian smoothing SGD (DP-LSSGD) to train ML models with differential privacy (DP) guarantees. At the core of DP-LSSGD is…
New DP-CD method outperforms DP-SGD in solving composite DP-ERM problems.
problem Privacy-preserving machine learning with differential privacy.
method Differentially Private proximal Coordinate Descent (DP-CD) for composite Empirical Risk Minimization (ERM).
result DP-CD outperforms DP-SGD due to larger step sizes and better gradient exploitation.
DOPPLER optimizes DP training with low-pass filtering, improving model accuracy.
problem Privacy concerns in deep learning models and performance degradation of DP optimizers.
method Developed DOPPLER, a low-pass filter for DP optimizers, to reduce privacy noise and enhance model quality.
result DOPPLER optimizers outperform non-DOPPLER counterparts by 3%-10% in test accuracy.
Optimal DP mechanisms for vector queries are found to be staircase distributions.
problem Designing optimal additive mechanisms for vector-valued queries under differential privacy.
method Reduction to radially symmetric distributions and convex rearrangement theory.
result Staircase mechanisms are optimal for any norm and cost function.
DP-BNNs improve accuracy, privacy, and reliability in neural networks.
problem Balancing privacy and accuracy in neural networks.
method Proposed three DP-BNNs: DP-SGLD, DP-BBP, and DP-MC Dropout.
result DP-SGLD achieves high accuracy under strong privacy guarantees.
Proactive DP optimizes privacy and utility in DP-SGD with a fixed privacy budget.
problem Balancing privacy and utility in differential privacy for machine learning.
method Proposes a pro-active DP framework that allows a-priori selection of DP-SGD parameters to maximize test accuracy.
result Proactive DP can optimize utility of DP-SGD with a fixed privacy budget (ε, δ).
Dynamic Programming (DP) provides standard algorithms to solve Markov Decision Processes. However, these algorithms generally do not optimize a scalar objective function. In this paper, we draw connections between DP and (constrained) convex optimization. Specifically, we show clear links in the algorithmic structure b…
Framework purifies approximate differential privacy to pure differential privacy.
problem Achieving pure differential privacy from approximate differential privacy.
method Randomized post-processing with calibrated noise to eliminate δ parameter.
result First statistically and computationally efficient reduction from approximate DP to pure DP.
Paper defines new topological invariants for DP tangles.
problem Classifying and understanding doubly periodic tangles.
method Organized components into interlinked compounds; introduced axis-motif.
result Directional type is an invariant of DP tangles.
PMT uses public data moments to make DP feasible for unbounded data.
problem Applying differential privacy to unbounded data distributions.
method Public-moment-guided Truncation (PMT) using second-moments from public data.
result PMT improves the accuracy and stability of DP models.
Three DP variants linked, improving SGD privacy bounds.
problem Relating different DP variants for tighter privacy bounds.
method Developed machinery to relate approximate DP to RDP and hypothesis test DP.
result Improved privacy guarantees for noisy SGD.
A new DP algorithm for weighted ERM protects sensitive data in predictive models.
problem Protecting sensitive personal information in predictive models trained via ERM.
method Proposes the first differentially private algorithm for weighted ERM with formal privacy guarantees.
result Demonstrates strong DP guarantees while maintaining robust performance in real-world data.
This paper shows how differential privacy can be achieved naturally in federated learning over fading channels without artificial noise.
problem Achieving differential privacy in federated learning over fading channels without artificial noise.
method Study of AirFL over multiple-access fading channels with a multi-antenna base station, deriving novel bounds on differential privacy.
result DP can be achieved naturally in federated learning over fading channels without artificial noise, revealing convergence-privacy trade-offs.
New DP algorithm improves privacy and efficiency for convex optimization.
problem Efficient, DP algorithms for convex optimization with strong excess risk bounds.
method Output perturbation for a broad class of tilted loss functions.
result Near optimal DP excess risk and runtime bounds for convex optimization.
Optimal DP-GD achieves high-dimensional linear regression risk.
problem Differentially private linear regression in high-dimensional settings.
method One-pass DP gradient descent (DP-GD) with gradient clipping and decaying learning rate.
result Achieves optimal non-asymptotic risk of O(γ+γ2/ρ2) for well-conditioned data. This paper improves privacy accounting in decentralized FL using f-Differential Privacy.
problem Challenges in accurately quantifying privacy budget in decentralized FL.
method Develops two new f-DP-based accounting methods for decentralized FL.
result Yields tighter (ε,δ) bounds and improved utility compared to existing methods.
Differentially private log-location-scale regression models improve privacy in statistical analysis.
problem Ensuring privacy in statistical regression models while maintaining accuracy.
method Integrates differential privacy into LLS regression using the functional mechanism.
result Proposed DP-LLS models satisfy ε-differential privacy and perform well under various conditions.
Paper improves privacy-preserving optimization rates for convex functions.
problem Differentially private stochastic convex optimization.
method Algorithmic improvements for convex and strongly convex functions under TNC and non-negative loss.
result Excess population risk bounds for DP-SCO are faster than previous results.
New divergences help audit DP in high dimensions.
problem Challenges in auditing DP in high-dimensional data.
method Propose kernel Rényi divergence and its regularized version for auditing.
result Regularized kernel Rényi divergence can be estimated from samples in high dimensions.
DP-Net uses dynamic programming for efficient deep neural network compression.
problem Efficiently compressing deep neural networks while maintaining accuracy.
method Dynamic Programming for optimal weight quantization and clustering-friendly training.
result Achieves up to 77X compression ratio on Wide ResNet with minimal accuracy loss.
DPPS uses DP priors for Bayesian non-parametric multi-arm bandits.
problem Optimizing multi-arm bandit environments with prior beliefs.
method Bayesian non-parametric algorithm based on Dirichlet Process priors.
result DPPS provides principled incorporation of prior beliefs and is optimal in Bayesian regret setup.
Balls-and-Bins sampling improves DP-SGD privacy and utility.
problem Improving privacy and utility in DP-SGD implementations.
method Introducing Balls-and-Bins sampling as an alternative to shuffling in DP-SGD.
result Balls-and-Bins sampling achieves utility comparable to shuffling while offering better privacy amplification.
New DP bootstrap method for statistical inference with improved privacy and accuracy.
problem Lack of general techniques for conducting statistical inference under differential privacy.
method DP bootstrap procedure to infer sampling distribution and construct confidence intervals.
result DP bootstrap estimates provide consistent point estimates and asymptotically valid standard CIs.
A new differentiable model for sampling DAGs that speeds up optimization.
problem Efficiently sampling and learning DAG structures in continuous optimization.
method DP-DAG model with VI-DP-DAG for DAG learning from data.
result VI-DP-DAG outperforms other methods in DAG structure and causal mechanism learning.
A new DP algorithm improves privacy in hashing and sampling for search and learning.
problem Improving privacy in hashing and sampling for large-scale applications.
method Combines differential privacy with one permutation hashing and bin-wise consistent weighted sampling.
result Proposes DP-OPH and DP-BCWS algorithms that enhance privacy while maintaining utility.
Gaussian DP improves reporting of ML algorithms' differential privacy guarantees.
problem Incomplete and misleading DP guarantees for ML algorithms.
method Using non-asymptotic Gaussian Differential Privacy (GDP) to provide accurate bounds on privacy profiles of ML algorithms.
result GDP captures the entire privacy profile of DP-SGD and related algorithms with virtually no error.
New analysis shows how to balance privacy and accuracy in deep learning.
problem Balancing privacy and accuracy in deep learning models.
method Continuous time analysis through neural tangent kernel (NTK) for arbitrary architectures.
result Large clipping norm improves calibration without sacrificing accuracy.
The paper introduces DP algorithms using random projections and sign random projections for improved privacy in machine learning.
problem Improving differential privacy in machine learning applications.
method Developed algorithms based on random projections and sign random projections, focusing on individual differential privacy (iDP) and standard differential privacy (DP).
result DP-SignOPORP and iDP-SignRP achieve superior performance in differential privacy, especially for small epsilon values.
Recent developments in differentially private (DP) machine learning and DP Bayesian learning have enabled learning under strong privacy guarantees for the training data subjects. In this paper, we further extend the applicability of DP Bayesian learning by presenting the first general DP Markov chain Monte Carlo (MCMC)…
Paper constructs motifs from planar tilings for DP weaves and polycatenanes.
problem Creating complex entangled structures from periodic tilings.
method Combinatorial methodology using polygonal link transformations.
result Predicting the type of motif from a given tiling and polygonal link method.