Machine learning (ML) models trained by differentially private stochastic gradient descent (DP-SGD) have much lower utility than the non-private ones. To mitigate this degradation, we propose a DP Laplacian smoothing SGD (DP-LSSGD) to train ML models with differential privacy (DP) guarantees. At the core of DP-LSSGD is…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “Gaussian DP”
Expands differential privacy mechanisms to include the Generalized Gaussian mechanism for improved private machine learning.
problem Improving privacy in machine learning algorithms while maintaining utility.
method Introduces and analyzes the Generalized Gaussian (GG) mechanism for differential privacy.
result The GG mechanism provides better performance than the Laplace and Gaussian mechanisms across various values of β.
Gaussian DP improves reporting of ML algorithms' differential privacy guarantees.
problem Incomplete and misleading DP guarantees for ML algorithms.
method Using non-asymptotic Gaussian Differential Privacy (GDP) to provide accurate bounds on privacy profiles of ML algorithms.
result GDP captures the entire privacy profile of DP-SGD and related algorithms with virtually no error.
Polynomial-time DP algorithm for learning Gaussians with matching sample complexity.
problem Learning Gaussian distributions while maintaining privacy.
method General framework for reducing DP estimation to non-private, polynomial-time algorithm for Gaussian learning.
result Matching sample complexity to information-theoretic upper bound for Gaussian learning.
Differential privacy has seen remarkable success as a rigorous and practical formalization of data privacy in the past decade. This privacy definition and its divergence based relaxations, however, have several acknowledged weaknesses, either in handling composition of private algorithms or in analyzing important primi…
DPConvCNP learns to predict private data accurately and privately.
problem Balancing privacy and accuracy in machine learning models.
method Meta-learning combined with improved DP mechanism.
result DPConvCNP outperforms DP GP baseline, especially on non-Gaussian data.
Efficiently estimates Gaussian distributions privately and robustly.
problem Private and robust estimation of Gaussian distributions.
method Efficient algorithms for pure and approximate differential privacy models.
result Optimal sample complexity in both pure and approximate differential privacy settings.
Large batch training with DP-SGD reduces model performance due to implicit bias.
problem Large batch training with DP-SGD reduces model performance.
method The study analyzes the phenomenon of implicit bias in Noisy-SGD (DP-SGD without clipping) and its theoretical solutions for linear models.
result The implicit bias in large batch training with DP-SGD is amplified by additional noise, similar to SGD.
Paper presents a reparameterized DP-DLGMM for clustering.
problem Non-parametric DP priors in DLGMM are hard to couple with variational inference.
method Closed-form updates for DP-DLGMM's variational posterior.
result Model generates realistic samples and performs competitively in semi-supervised settings.
Private estimation with public data reduces sample complexity.
problem Estimating private distributions with limited public data.
method Differentially private estimation with public data under constraints of pure or concentrated DP.
result Public data can significantly reduce private sample complexity for estimation.
A major challenge for machine learning is increasing the availability of data while respecting the privacy of individuals. Here we combine the provable privacy guarantees of the differential privacy framework with the flexibility of Gaussian processes (GPs). We propose a method using GPs to provide differentially priva…
New DP framework using data truncation for efficient estimation.
problem Differential privacy in unbounded data support.
method Data truncation, exponential family distributions, maximum likelihood estimation, DP stochastic gradient descent.
result Near-optimal sample complexity for Gaussian mean and covariance estimation.
Many applications of machine learning, for example in health care, would benefit from methods that can guarantee privacy of data subjects. Differential privacy (DP) has become established as a standard for protecting learning results. The standard DP algorithms require a single trusted party to have access to the entir…
Gaussian processes with differential privacy protect both inputs and outputs.
problem Previous DP methods only protected model outputs, not inputs.
method Sparse GP with private variational approximation, adjusting covariance for DP noise.
result Accurate models can be produced under strong privacy protection with sufficient data.
A new Gaussian mechanism for differential privacy in the shuffle model is introduced.
problem Improving differential privacy in distributed learning environments.
method Characterization and upper-bounding of Rényi differential privacy (RDP) for the shuffle Gaussian mechanism.
result The shuffle Gaussian mechanism provides improved privacy guarantees compared to existing methods.
New algorithm reduces privacy breach in posterior sampling.
problem Combining pure DP with MCMC for efficient sampling.
method ASAP algorithm that perturbs MCMC samples with Wasserstein-infinity noise.
result First nearly linear-time algorithm achieving optimal DP-ERM rates.
Differential privacy of Gaussian process posterior sampling
problem Privacy of posterior sample paths from Gaussian process
method Intrinsic randomness yields DP guarantees
result Intrinsic randomness yields DP guarantees
Principled mapping from pure-DP ε to GDP μ for Gaussian differential privacy
problem Choosing the μ parameter in Gaussian differential privacy
method Matching the worst-case success of a membership inference attack
result Recommendation of μ ≈ ε/5 as a conservative general-purpose conversion
New method audits DP guarantees without noise or subsampling info.
problem Auditing DP guarantees of ML models without prior info.
method Histogram-based density estimation for lower bounds.
result Natural generalization of membership inference auditing.
Paper revisits DP-SCO in Euclidean and spaces, focusing on constrained and bounded sets.
problem Differentially private stochastic convex optimization in constrained and bounded sets in Euclidean and spaces.
method Proposes methods achieving excess population risks dependent on Gaussian width of the constraint set, and novel algorithms for unconstrained and heavy-tailed data.
result Theoretical results for DP-SCO in spaces, including optimal bounds for strongly convex functions.
New DP algorithms with margin guarantees for various hypothesis sets.
problem Differential privacy in machine learning with margin guarantees.
method Developed pure and efficient DP learning algorithms for linear, kernel-based, and neural network hypotheses.
result Margin guarantees are independent of input dimension and hypothesis type.
We propose Dirichlet Process mixtures of Generalized Linear Models (DP-GLM), a new method of nonparametric regression that accommodates continuous and categorical inputs, and responses that can be modeled by a generalized linear model. We prove conditions for the asymptotic unbiasedness of the DP-GLM regression mean fu…
Noise-Calibrated Inference from Differentially Private Sufficient Statistics in Exponential Familiescs.LG
This paper provides a method for noise-calibrated inference from DP synthetic data.
problem Inference from DP synthetic data is often miscalibrated and lacks principled uncertainty quantification.
method Release DP sufficient statistics, perform noise-calibrated likelihood-based inference, and optional synthetic data generation.
result Asymptotic normality and valid confidence intervals for the plug-in DP MLE.
Improved privacy-preserving methods for estimating multiple samples from distributions.
problem Estimating multiple samples from distributions while maintaining privacy.
method Developed new multi-sampling techniques for differentially private data estimation.
result Achieved significant reduction in sample complexity for multi-sampling from finite domains and Gaussian distributions.
New DP bootstrap method for statistical inference with improved privacy and accuracy.
problem Lack of general techniques for conducting statistical inference under differential privacy.
method DP bootstrap procedure to infer sampling distribution and construct confidence intervals.
result DP bootstrap estimates provide consistent point estimates and asymptotically valid standard CIs.
Algorithm estimates top k eigenvectors of shared covariance matrices while preserving privacy.
problem Differentially private PCA with adaptive noise for arbitrary k.
method Iterative algorithm with adaptive noise reduction.
result First algorithm for estimating top k eigenvectors with near-optimal statistical error.
Improved privacy-preserving linear regression via iterative Hessian mixing.
problem Differentially private linear regression with improved accuracy and efficiency.
method Iterative Hessian Mixing (IHM) for differentially private ordinary least squares (DP-OLS).
result IHM provides better utility guarantees and outperforms AdaSSP in empirical evaluations.
New method improves privacy in linear regression with optimal error bounds.
problem Differentially private linear regression with suboptimal error bounds.
method One-pass mini-batch stochastic gradient descent (DP-AMBSSGD) with adaptive clipping.
result Nearly optimal error bounds in terms of key parameters like dimensionality, number of points, and noise standard deviation.
New mechanisms improve differential privacy for scalar queries.
problem Improving differential privacy for scalar, real-valued query functions.
method Mixing multiple Gaussian distributions to satisfy differential privacy.
result Mechanisms yield lower noise amplitudes and variances compared to the analytic Gaussian mechanism.
Differentially Private Linear Regression and Synthetic Data Generation with Statistical Guaranteescs.LG
Proposes a method for differentially private linear regression and synthetic data generation.
problem Lack of valid inference and synthetic data generation methods for small-scale datasets in privacy-aware settings.
method Gaussian differentially private linear regression with bias-corrected estimator and SDG procedure.
result Improves accuracy and provides valid confidence intervals for downstream tasks.
DP-SPRT improves privacy in sequential tests with near-optimal error rates.
problem Privacy constraints in sequential probability ratio tests.
method A wrapper for SPRT that uses a private mechanism to determine when to stop based on predefined intervals.
result DP-SPRT achieves near-optimal error rates and privacy guarantees.
Differentially private (DP) machine learning has recently become popular. The privacy loss of DP algorithms is commonly reported using -DP. In this paper, we propose a numerical accountant for evaluating the privacy loss for algorithms with continuous one dimensional output. This accountant can be appl…
A new privacy accountant for Gaussian differential privacy measures individual privacy losses.
problem Bounding differential privacy loss for each participant in data analysis.
method Developed a privacy accountant for adaptive compositions of randomised mechanisms using Gaussian differential privacy.
result Provided optimal bounds for the Gaussian mechanism and constructed an approximative individual privacy accountant.
Improved DP-SGD for variational inference reduces noise and variance.
problem Poor convergence and high variance in variational parameter outputs due to gradient noise in DP-SGD.
method Introduced aligned gradients and iterate averaging to reduce DP-induced noise, and noise-aware posteriors.
result Less noisy gradient estimator and improved parameter estimates for variational inference.
Near-Optimal differentially private low-rank trace regression with guaranteed private initializationstat.ML
Paper develops DP methods for low-rank matrix estimation with near-optimal performance.
problem Estimating a low-rank matrix under differential privacy constraints.
method Introduced computationally efficient DP-initialization and Riemannian optimization-based DP-RGrad algorithm.
result DP-RGrad achieves near-optimal convergence rate under weak differential privacy constraints.
DP-PCA improves privacy in PCA computations with optimal statistical error.
problem Differentially private principal component analysis with sub-linear sample complexity.
method Private minibatch gradient ascent with private mean estimation.
result Achieves optimal statistical error rates for sub-Gaussian data with samples.
Study shows private learning of mixtures of Gaussians is possible with polynomial samples.
problem Estimating mixtures of Gaussians under differential privacy constraints.
method Developed a new framework for privately learning mixtures of Gaussians without structural assumptions.
result Polynomial number of samples (poly(k,d,1/α,1/ε,log(1/δ))) sufficient for estimation up to total variation distance α with (ε, δ)-DP.
Paper proposes no-regret algorithms for private GP bandit optimization.
problem Private Gaussian process bandit optimization.
method Combines uniform kernel approximator with random perturbations for differentially private GP bandit algorithms.
result Provable no-regret algorithms for stationary kernel functions in two DP settings.
Dirichlet processes (DP) are widely applied in Bayesian nonparametric modeling. However, in their basic form they do not directly integrate dependency information among data arising from space and time. In this paper, we propose location dependent Dirichlet processes (LDDP) which incorporate nonparametric Gaussian proc…
Private training and synthetic data generation using DP clustering.
problem Protecting sensitive data in deep neural networks training.
method Approximate input dataset with privately generated synthetic dataset using DP clustering.
result Simple two-layer neural network achieves SOTA classification accuracy on standard benchmark datasets.
Private statistics estimation faces a bias, accuracy, and privacy trilemma.
problem Balancing privacy, accuracy, and bias in statistical estimation.
method Use differential privacy (DP) for private statistics, but clip samples to control sensitivity and add noise for privacy, introducing bias.
result No algorithm can simultaneously have low bias, low error, and low privacy loss for arbitrary distributions.
Privacy-preserving machine learning methods add randomness, leading to varying predictions.
problem Privacy-preserving machine learning methods add randomness, leading to varying predictions.
method The study analyzes three DP-ensuring algorithms: output perturbation, objective perturbation, and DP-SGD.
result The degree of predictive multiplicity rises as the level of privacy increases, and is unevenly distributed across individuals and demographic groups.
Differentially private hyperparameter tuning improves privacy in machine learning.
problem Hyperparameter tuning leaks private information through selected configurations.
method Local Bayesian optimization using Gaussian Process surrogate for private gradient approximation.
result DP-GIBO converges to locally optimal hyperparameters with polynomial dimensional dependence.
Differentially Private (Gradient) Expectation Maximization Algorithm with Statistical Guaranteescs.LG
New DP EM algorithm with statistical guarantees for mixture models.
problem Preserving privacy in EM algorithms for mixture models.
method Proposed a DP EM algorithm with statistical guarantees.
result Near optimal estimation error for GMM in DP model.
Proposes a method to generate private synthetic data in a decentralized setting using correlated noise.
problem Challenges of generating private synthetic data in a decentralized setting with limited client data.
method Integrates CAPE protocol into federated DP-CDA framework to generate anti-correlated noise.
result Improves privacy-utility trade-off in federated setting compared to centralized approach.
Paper optimizes private ReLU regression with relaxed assumptions.
problem Private ReLU regression with relaxed assumptions.
method One-pass mini-batch Generalized Linear Model Perceptron algorithm (DP-MBGLMtron).
result Achieves optimal utility bound up to logarithmic factors.
Addresses theoretical and practical aspects of Gaussian differential privacy.
problem Theoretical and practical challenges in privacy-preserving data analysis.
method Discussion of f-differential privacy and Gaussian differential privacy.
result Gaussian differential privacy can enhance privacy in various applications.
New bounds for KANs trained with DP-SGD, addressing correlated noise.
problem Risk bounds for Kolmogorov-Arnold Networks trained by DP-SGD with correlated noise.
method Established new optimization and population risk analysis for KANs trained with DP-SGD, addressing correlated noise.
result First optimization and population risk analysis of correlated-noise mechanisms for DP training in non-convex settings, including neural networks.