Machine learning (ML) models trained by differentially private stochastic gradient descent (DP-SGD) have much lower utility than the non-private ones. To mitigate this degradation, we propose a DP Laplacian smoothing SGD (DP-LSSGD) to train ML models with differential privacy (DP) guarantees. At the core of DP-LSSGD is…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “Smooth DP”
Framework evaluates privacy cost of non-private pre-processing in DP pipelines.
problem Privacy cost of non-private data-dependent pre-processing in DP machine learning pipelines.
method Establishes upper bounds on overall privacy guarantees using Smooth DP and bounded sensitivity.
result Explicit overall privacy guarantees for various pre-processing algorithms.
Dynamic programming (DP) solves a variety of structured combinatorial problems by iteratively breaking them down into smaller subproblems. In spite of their versatility, DP algorithms are usually non-differentiable, which hampers their use as a layer in neural networks trained by backpropagation. To address this issue,…
New framework for DP-SMO with near-optimal privacy-loss trade-off.
problem Optimal trade-off between privacy and population loss in DP-SMO.
method General framework using Phased-ERM method and black-box optimization.
result Near-linear time algorithms with near-optimal guarantees.
Paper addresses DP-SCO on heavy-tailed data, providing methods and results.
problem Designing DP algorithms for SCO on heavy-tailed data.
method Sample-and-aggregate framework, gradient smoothing and trimming.
result Achieved DP guarantees for various loss functions with different excess population risks.
New method improves optimization and DP in FL.
problem Combining strong DP and optimization in FL.
method Combining clipping, momentum, and error feedback.
result Optimal convergence rate and near optimal DP guarantees.
The paper introduces DP algorithms using random projections and sign random projections for improved privacy in machine learning.
problem Improving differential privacy in machine learning applications.
method Developed algorithms based on random projections and sign random projections, focusing on individual differential privacy (iDP) and standard differential privacy (DP).
result DP-SignOPORP and iDP-SignRP achieve superior performance in differential privacy, especially for small epsilon values.
New privacy bounds for DP-SGD's last iterate, even with cyclic sampling.
problem Privacy of the last iterate in DP-SGD with cyclic sampling.
method Established new RDP upper bounds for the last iterate under realistic assumptions.
result Privacy bounds for DP-SGD's last iterate with cyclic sampling and clipping, even for nonconvex losses.
New DP algorithm improves privacy and efficiency for convex optimization.
problem Efficient, DP algorithms for convex optimization with strong excess risk bounds.
method Output perturbation for a broad class of tilted loss functions.
result Near optimal DP excess risk and runtime bounds for convex optimization.
Enhances privacy in federated learning with Laplacian smoothing.
problem Protecting data privacy in federated learning while maintaining model accuracy.
method Laplacian smoothing for differentially private federated learning (DP-Fed-LS).
result Improves model accuracy with differential privacy guarantee and membership privacy.
In this paper, we consider efficient differentially private empirical risk minimization from the viewpoint of optimization algorithms. For strongly convex and smooth objectives, we prove that gradient descent with output perturbation not only achieves nearly optimal utility, but also significantly improves the running …
This paper improves privacy in federated learning without a trusted server.
problem Privacy in federated learning with silos that distrust each other.
method Introduces Inter-Silo Record-Level Differential Privacy (ISRL-DP) and accelerated algorithms for convex and smooth losses.
result Achieves optimal privacy and accuracy tradeoffs in federated learning.
New algorithm reduces privacy breach in posterior sampling.
problem Combining pure DP with MCMC for efficient sampling.
method ASAP algorithm that perturbs MCMC samples with Wasserstein-infinity noise.
result First nearly linear-time algorithm achieving optimal DP-ERM rates.
Improved DP optimization for nonconvex, nonsmooth objectives with reduced sample complexity.
problem Differentially private optimization of nonconvex, nonsmooth objectives.
method Proposes single-pass and multi-pass DP algorithms with improved sample complexity.
result Sample complexity bounds improved by factors of and .
DP synthetic data may inflate statistical test results, caution advised.
problem Inflated Type I errors in statistical tests on DP-synthetic data.
method Evaluation of Mann-Whitney U test, t-test, chi-squared test, and median test on DP-synthetic data generated from real-world and simulated datasets using various DP-synthetic data generation methods.
result A large portion of evaluation results showed inflated Type I errors, especially at low privacy levels.
New algorithm for differentially private distributed optimization of smooth, non-convex problems.
problem No differentially private distributed method for smooth, non-convex optimization problems.
method Smoothed normalization integrated with an error-feedback mechanism.
result Achieves superior convergence rate and first differentially private distributed optimization algorithm with provable convergence guarantees.
This is the next step of uncovering the relation between string theory and exotic smooth R^4. Exotic smoothness of R^4 is correlated with D6 brane charges in IIA string theory. We construct wild embeddings of spheres and relate them to a class of topological quantum Dp-branes as well to KK theory. These branes emerge w…
Paper tackles DP-SCO with heavy-tailed data in high dimensions, improving error bounds.
problem Differentially private stochastic optimization with heavy-tailed data in high-dimensional spaces.
method Proposes methods for DP-SCO with polytope constraints and LASSO, analyzing sparsity constraints.
result Achieved near optimal error bounds for DP-SCO with heavy-tailed data.
Differentially private random block coordinate descent improves utility in machine learning.
problem Lack of privacy in classical CD methods when handling sensitive information.
method Proposes a differentially private random block coordinate descent method using sketch matrices and importance sampling.
result Demonstrates improved convergence rates and utility guarantees compared to non-private methods.
Paper addresses privacy in combinatorial semi-bandits with improved bounds.
problem Privacy-preserving learning in combinatorial semi-bandits with additional dimension dependence.
method Proposes novel algorithms and proves optimal regret bounds for LDP and DP settings.
result Achieves nearly optimal regret bounds for LDP and DP settings, matching non-private rates.
This paper shows how differential privacy can be achieved naturally in federated learning over fading channels without artificial noise.
problem Achieving differential privacy in federated learning over fading channels without artificial noise.
method Study of AirFL over multiple-access fading channels with a multi-antenna base station, deriving novel bounds on differential privacy.
result DP can be achieved naturally in federated learning over fading channels without artificial noise, revealing convergence-privacy trade-offs.
Large-batch stochastic gradient descent (SGD) is widely used for training in distributed deep learning because of its training-time efficiency, however, extremely large-batch SGD leads to poor generalization and easily converges to sharp minima, which prevents naive large-scale data-parallel SGD (DP-SGD) from convergin…
DIFF2: Differential Private Optimization via Gradient Differences for Nonconvex Distributed Learningcs.LG
DIFF2 improves differential privacy in nonconvex optimization with better utility bounds.
problem Improving differential privacy in nonconvex optimization with better utility bounds.
method DIFF2 constructs a differential private global gradient estimator using gradient differences.
result DIFF2 achieves a utility of \(\widetilde O(d^{2/3}/(n\varepsilon_{\mathrm{DP}})^{4/3})\), significantly better than \(\widetilde O(\sqrt{d}/(n\varepsilon_{\mathrm{DP}}))\).
New DP-CD method outperforms DP-SGD in solving composite DP-ERM problems.
problem Privacy-preserving machine learning with differential privacy.
method Differentially Private proximal Coordinate Descent (DP-CD) for composite Empirical Risk Minimization (ERM).
result DP-CD outperforms DP-SGD due to larger step sizes and better gradient exploitation.
DOPPLER optimizes DP training with low-pass filtering, improving model accuracy.
problem Privacy concerns in deep learning models and performance degradation of DP optimizers.
method Developed DOPPLER, a low-pass filter for DP optimizers, to reduce privacy noise and enhance model quality.
result DOPPLER optimizers outperform non-DOPPLER counterparts by 3%-10% in test accuracy.
DP-BNNs improve accuracy, privacy, and reliability in neural networks.
problem Balancing privacy and accuracy in neural networks.
method Proposed three DP-BNNs: DP-SGLD, DP-BBP, and DP-MC Dropout.
result DP-SGLD achieves high accuracy under strong privacy guarantees.
Proactive DP optimizes privacy and utility in DP-SGD with a fixed privacy budget.
problem Balancing privacy and utility in differential privacy for machine learning.
method Proposes a pro-active DP framework that allows a-priori selection of DP-SGD parameters to maximize test accuracy.
result Proactive DP can optimize utility of DP-SGD with a fixed privacy budget (ε, δ).
Improved DP SO with large Lipschitz parameters, handling outliers and heavy-tailed data.
problem Differential privacy in stochastic optimization with large Lipschitz parameters.
method Assumes bounded k-th order moments, provides linear-time algorithms for smooth convex and non-smooth convex losses.
result Improved risk bounds scaling with k-th moment, not uniform Lipschitz parameter.
We consider a refinement of differential privacy --- per instance differential privacy (pDP), which captures the privacy of a specific individual with respect to a fixed data set. We show that this is a strict generalization of the standard DP and inherits all its desirable properties, e.g., composition, invariance to …
Framework purifies approximate differential privacy to pure differential privacy.
problem Achieving pure differential privacy from approximate differential privacy.
method Randomized post-processing with calibrated noise to eliminate δ parameter.
result First statistically and computationally efficient reduction from approximate DP to pure DP.
Paper defines new topological invariants for DP tangles.
problem Classifying and understanding doubly periodic tangles.
method Organized components into interlinked compounds; introduced axis-motif.
result Directional type is an invariant of DP tangles.
New DP algorithms achieve near-optimal regret bounds for online learning problems.
problem Online learning problems with zero-loss solutions and differential privacy constraints.
method Developed new Differentially Private algorithms with near-optimal regret bounds.
result Achieved near-optimal regret bounds for various online prediction and convex optimization problems.
Three DP variants linked, improving SGD privacy bounds.
problem Relating different DP variants for tighter privacy bounds.
method Developed machinery to relate approximate DP to RDP and hypothesis test DP.
result Improved privacy guarantees for noisy SGD.
FLORAS uses orthogonal sequences for SISO FL, offering both DP and convergence guarantees.
problem Privacy-preserving wireless federated learning in SISO systems.
method Leverages orthogonal sequences to eliminate CSIT requirement and provide DP guarantees.
result FLORAS achieves a smooth tradeoff between convergence rate and DP levels.
Mitigating Privacy-Utility Trade-off in Decentralized Federated Learning via -Differential Privacycs.LG
This paper improves privacy accounting in decentralized FL using f-Differential Privacy.
problem Challenges in accurately quantifying privacy budget in decentralized FL.
method Develops two new f-DP-based accounting methods for decentralized FL.
result Yields tighter (ε,δ) bounds and improved utility compared to existing methods.
New divergences help audit DP in high dimensions.
problem Challenges in auditing DP in high-dimensional data.
method Propose kernel Rényi divergence and its regularized version for auditing.
result Regularized kernel Rényi divergence can be estimated from samples in high dimensions.
DP-Net uses dynamic programming for efficient deep neural network compression.
problem Efficiently compressing deep neural networks while maintaining accuracy.
method Dynamic Programming for optimal weight quantization and clustering-friendly training.
result Achieves up to 77X compression ratio on Wide ResNet with minimal accuracy loss.
Paper relaxes SGD privacy and generalization guarantees for non-smooth convex losses.
problem Privacy and generalization in SGD for non-smooth convex losses.
method Relaxes Lipschitz and strong smoothness assumptions to Hölder smoothness, proving -DP and optimal excess risk.
result Noisy SGD with -Hölder smooth losses achieves optimal excess risk with linear gradient complexity for .
Balls-and-Bins sampling improves DP-SGD privacy and utility.
problem Improving privacy and utility in DP-SGD implementations.
method Introducing Balls-and-Bins sampling as an alternative to shuffling in DP-SGD.
result Balls-and-Bins sampling achieves utility comparable to shuffling while offering better privacy amplification.
DP-GD achieves dimension-independent convergence for unconstrained private GLMs.
problem Differentially private empirical risk minimization for unconstrained GLMs.
method Differentially private gradient descent (DP-GD).
result DP-GD achieves an excess empirical risk of $ ilde O\left(\sqrt{ exttt{rank}}/εn
ight)$ for unconstrained GLMs.
Researchers developed a differentially private method for computing Wasserstein distances.
problem Computing divergences between distributions while preserving privacy.
method They focused on the Sliced Wasserstein Distance and added Gaussian perturbations to make it differentially private.
result They introduced a new differentially private distance, the Smoothed Sliced Wasserstein Distance, which performs well in generative models and domain adaptation.
This paper improves privacy bounds for DP algorithms using -DP.
problem Difficulty in analyzing randomness in DP algorithms due to mixture distributions.
method Derives a closed-form expression for trade-off functions and analyzes -DP.
result Enhances privacy of DP-GD with random initialization and shuffling models.
New DP bootstrap method for statistical inference with improved privacy and accuracy.
problem Lack of general techniques for conducting statistical inference under differential privacy.
method DP bootstrap procedure to infer sampling distribution and construct confidence intervals.
result DP bootstrap estimates provide consistent point estimates and asymptotically valid standard CIs.
A new differentiable model for sampling DAGs that speeds up optimization.
problem Efficiently sampling and learning DAG structures in continuous optimization.
method DP-DAG model with VI-DP-DAG for DAG learning from data.
result VI-DP-DAG outperforms other methods in DAG structure and causal mechanism learning.
A new DP algorithm improves privacy in hashing and sampling for search and learning.
problem Improving privacy in hashing and sampling for large-scale applications.
method Combines differential privacy with one permutation hashing and bin-wise consistent weighted sampling.
result Proposes DP-OPH and DP-BCWS algorithms that enhance privacy while maintaining utility.
Gaussian DP improves reporting of ML algorithms' differential privacy guarantees.
problem Incomplete and misleading DP guarantees for ML algorithms.
method Using non-asymptotic Gaussian Differential Privacy (GDP) to provide accurate bounds on privacy profiles of ML algorithms.
result GDP captures the entire privacy profile of DP-SGD and related algorithms with virtually no error.
Improved privacy analysis for stochastic gradient descent.
problem Analyzing privacy leakage in noisy stochastic gradient descent.
method Modeling Rényi divergence dynamics with Langevin diffusions, proving exponential privacy loss convergence for smooth and strongly convex objectives.
result Privacy loss converges exponentially fast for smooth and strongly convex objectives under constant step size.
Dp-CLIP preserves privacy in multimodal AI training.
problem Privacy concerns in multimodal AI, especially in vision-language tasks.
method Differentially private adaptation of CLIP model.
result Dp-CLIP retains accuracy while ensuring privacy.