New privacy mechanism for user-level discrete distributions with reduced penalty.
problem Achieving privacy for all items of a single user in practical applications.
method Study of learning discrete distributions with user-level differential privacy, proposing a new mechanism with reduced privacy penalty.
result Proposed mechanism reduces privacy penalty to i l d e O ( k / ( m α 2 ) + k / m ε α ) ilde{\mathcal{O}}(k/(mα^2) + k/\sqrt{m}εα) i l d e O ( k / ( m α 2 ) + k / m ε α ) , significantly smaller than standard mechanisms. Alternating direction method of multiplier (ADMM) is a popular method used to design distributed versions of a machine learning algorithm, whereby local computations are performed on local data with the output exchanged among neighbors in an iterative fashion. During this iterative process the leakage of data privacy a…
PASE method protects machine learning models from membership inference attacks without significant accuracy loss.
problem Membership inference attacks on machine learning models.
method Switching ensembles approach to mitigate privacy leakage.
result PASE method provides effective privacy protection with minimal accuracy penalty.
Develops a privacy-preserving algorithm for sparse robust regression.
problem Privacy-preserving machine learning for sparse robust regression.
method Develops FRAPPE algorithm for non-smooth loss under differential privacy.
result Achieves better privacy and statistical accuracy trade-off.
Cluster-DP improves differential privacy in randomized experiments by clustering data.
problem Reducing variance in causal effect estimation from differentially private data.
method Cluster-DP leverages a given cluster structure to improve the privacy-variance trade-off.
result Selecting higher-quality clusters decreases the variance penalty without compromising privacy guarantees.
This paper enhances privacy-preserving randomized power method for large datasets.
problem Privacy issues in applying randomized power method to large datasets containing personal information.
method Proposes enhanced privacy-preserving variants of the randomized power method, including a variant with reduced noise and a decentralized framework.
result Tighter convergence bounds and empirical comparisons with previous work in real recommendation datasets.
Following the publication of an attack on genome-wide association studies (GWAS) data proposed by Homer et al., considerable attention has been given to developing methods for releasing GWAS data in a privacy-preserving way. Here, we develop an end-to-end differentially private method for solving regression problems wi…
Adapts Lifelong Learning for Federated Learning to prevent model drift.
problem Federated Learning on non-i.i.d. data leads to model divergence and forgetting.
method Adds a penalty term to the loss function to encourage convergence of local models to a shared optimum.
result The method improves performance in image recognition tasks compared to existing approaches.
We consider the decentralized exploration problem: a set of players collaborate to identify the best arm by asynchronously interacting with the same stochastic environment. The objective is to insure privacy in the best arm identification problem between asynchronous, collaborative, and thrifty players. In the context …
Privacy is a major issue in learning from distributed data. Recently the cryptographic literature has provided several tools for this task. However, these tools either reduce the quality/accuracy of the learning algorithm---e.g., by adding noise---or they incur a high performance penalty and/or involve trusting externa…
Generative adversarial networks create synthetic insurance datasets from confidential originals.
problem Difficulty in accessing or sharing confidential insurance datasets for research.
method Design and use of three GAN architectures tailored for multi-categorical insurance data.
result MC-WGAN-GP synthesizes the best data, CTGAN is easiest to use, and MNCDP-GAN ensures differential privacy.
Proposes a game-theoretic framework for ML trust regulation.
problem Lack of coordination between ML model builders and regulators.
method Formulates trustworthy ML as a multi-objective multi-agent optimization problem and introduces regulation games and ParetoPlay.
result Enables efficient enforcement of ML model specifications without discouraging participation.
A method for sharing synthetic data without revealing actual data or model parameters.
problem Sharing data privately between enterprises and their clients.
method privGAN architecture with local GANs and a central discriminator.
result Synthetic data leads to better utility than real small datasets.
This work generates synthetic EHRs with privacy guarantees for machine learning tasks.
problem Privacy concerns and heterogeneity in EHR data limit their use in machine learning.
method Generative Adversarial Networks (GANs) with differential privacy (DP) for synthetic data generation.
result Synthetic EHRs maintain performance close to real data, even with DP applied.
Regularization methods are often employed in deep learning neural networks (DNNs) to prevent overfitting. For penalty based DNN regularization methods, convex penalties are typically considered because of their optimization guarantees. Recent theoretical work have shown that nonconvex penalties that satisfy certain reg…
Gradient penalty improves GAN performance by inducing a large-margin classifier.
problem Improving GAN performance and addressing vanishing gradients.
method A unifying framework of expected margin maximization, showing gradient penalties induce large-margin classifiers.
result Gradient penalties reduce vanishing gradients and produce better generated outputs.
One-bit measurements widely exist in the real world, and they can be used to recover sparse signals. This task is known as the problem of learning halfspaces in learning theory and one-bit compressive sensing (1bit-CS) in signal processing. In this paper, we propose novel algorithms based on both convex and nonconvex s…
The paper studies robust risk measures with linear penalties under uncertain distributions.
problem Risk measurement under distributional uncertainty.
method Robust distortion risk measures with linear penalty function under distributional constraints.
result Explicit characterization of optimal quantile distribution and value function.
The use of machine-learning in neuroimaging offers new perspectives in early diagnosis and prognosis of brain diseases. Although such multivariate methods can capture complex relationships in the data, traditional approaches provide irregular (l2 penalty) or scattered (l1 penalty) predictive pattern with a very limited…
Improves privacy amplification by shuffling for differential privacy.
problem Enhancing privacy guarantees in systems with anonymous data contributions.
method Theoretical and numerical analysis of Rényi differential privacy parameters and privacy amplification by shuffling.
result First asymptotically optimal analysis of Rényi differential privacy parameters for shuffled outputs.
Proposes a privacy-preserving recommendation system using matrix factorization and differential privacy.
problem Privacy leakage in recommendation systems when anonymizing user data is not sufficient.
method Uses matrix factorization and differential privacy via the Gaussian mechanism.
result Demonstrates excellent utility for privacy-preserving recommendation systems.
Federated learning with Bayesian differential privacy offers improved privacy and accuracy.
problem Privacy in federated learning with similar data distributions.
method Bayesian differential privacy for federated learning, with improved privacy budgeting.
result Significant advantage over state-of-the-art privacy bounds, with lower noise and improved accuracy.
This work optimizes mean estimation under varying privacy constraints.
problem Mean estimation with heterogeneous privacy constraints.
method Proposes an algorithm for mean estimation under different privacy levels for users.
result Shows a saturation phenomenon in performance as privacy levels are relaxed.
Improves privacy guarantees by analyzing randomness in privacy-preserving mechanisms.
problem Balancing user privacy and business constraints in privacy-preserving mechanisms.
method Analyzes explicit and implicit randomness in privacy mechanisms and proposes a probabilistic calibration method.
result Proposes privacy at risk, providing stronger privacy guarantees with quantifiable risks.
The paper analyzes and proposes methods for privately sharing individual privacy losses using per-instance differential privacy.
problem The standard differential privacy framework provides a worst-case bound that may not accurately reflect individual privacy losses.
method The paper analyzes per-instance differential privacy and proposes methods to privately and accurately publish per-instance privacy losses.
result The methods privately and accurately publish per-instance differential privacy losses with minimal additional privacy cost.
This work optimizes mean estimation under varying user privacy demands.
problem Mean estimation with heterogeneous privacy levels.
method Proposes an algorithm that is minimax optimal and has near-linear run-time.
result Privacy requirements of the most stringent users dictate overall error rates.
A new privacy accountant for Gaussian differential privacy measures individual privacy losses.
problem Bounding differential privacy loss for each participant in data analysis.
method Developed a privacy accountant for adaptive compositions of randomised mechanisms using Gaussian differential privacy.
result Provided optimal bounds for the Gaussian mechanism and constructed an approximative individual privacy accountant.
Paper improves deep learning privacy with new f f f -differential privacy framework.
problem Training neural networks on sensitive data while maintaining privacy.
method Introduced and analyzed f f f -differential privacy for neural networks training. result Improved privacy guarantees for neural networks training without sacrificing accuracy.
Paper simplifies DP composition for adaptive privacy budgets, enabling better privacy and accuracy in deep learning.
problem Tension between efficiency and flexibility in DP composition theorems.
method Rényi Differential Privacy (RDP) for adaptive privacy budgets, proving simpler composition theorem with smaller constants.
result Practical DP composition for adaptive privacy budgets, enabling better privacy and accuracy in deep learning.
New sparse penalty improves biclustering for gene expression data.
problem Identifying significant clusters in gene expression data.
method Prenet penalty applied to SSVD for biclustering.
result Mixed Prenet penalty effectively clusters non-overlapped data.
New approach avoids excess empirical risk in domain generalization.
problem Learning models that generalize to unseen distributions from diverse data sets.
method Minimizes penalty under constraint of optimal empirical risk, leveraging rate-distortion theory.
result Significant improvements in domain generalization performance across multiple methods.
We study the problem of estimating high-dimensional regression models regularized by a structured sparsity-inducing penalty that encodes prior structural information on either the input or output variables. We consider two widely adopted types of penalties of this kind as motivating examples: (1) the general overlappin…
This paper bounds min-entropy leakage for Blowfish privacy using graph symmetries.
problem Bounding min-entropy leakage for Blowfish privacy mechanisms.
method Organizing analysis over symmetrical partitions corresponding to orbits of graph automorphism groups.
result Demonstrates a construction meeting the bound with asymptotic equality, showing tightness.
New federated f f f -differential privacy for collaborative learning.
problem Privacy in federated learning.
method Introducing federated f f f -differential privacy and proposing a generic private federated learning framework. result Proves federated f f f -differential privacy provides privacy guarantee on each record of one client's data. Differential privacy is a statistical concept that can be explained through hypothesis testing.
problem Formalizing differential privacy as a statistical concept.
method Using David Blackwell's informativeness theorem, the paper shows differential privacy can be understood through hypothesis testing.
result The definition of f f f -differential privacy provides a unified framework for analyzing privacy bounds. Curvature penalties improve interpretability of KANs without sacrificing accuracy.
problem Pathologically high-curvature oscillations in KANs activations make them hard to interpret.
method Derived a curvature penalty and proved an upper bound on model curvature.
result KANs with curvature penalties achieve substantially smoother activations while maintaining accuracy.
New method reduces bias in sparse Bayesian learning.
problem High sparsity in statistical models leads to significant bias.
method Variable-coefficient ℓ 1 \ell_1 ℓ 1 penalty with hyperpriors. result Reduces bias in sparse Bayesian learning.
A new method for tighter privacy loss accounting in adaptive analyses.
problem Ensuring individual privacy in adaptive analyses while staying within a privacy budget.
method A personalized privacy loss estimate and a Rényi differential privacy filter.
result Personalized privacy loss accounting can be practical and tighter than existing methods.
Paper tackles federated learning with privacy, enhancing target data analysis.
problem Heterogeneity and privacy of distributed data in federated learning.
method Formulates federated differential privacy, studies statistical problems under privacy constraints.
result Federated differential privacy offers a balance between privacy and knowledge transfer.
Unified framework for subsampling mechanisms with tighter privacy guarantees.
problem Improving privacy in machine learning models through subsampling.
method Conditional optimal transport for deriving mechanism-specific subsampling guarantees.
result Tighter privacy bounds for subsampled mechanisms compared to traditional methods.
We consider a one-period Kyle (1985) framework where the insider can be subject to a penalty if she trades. We establish existence and uniqueness of equilibrium for virtually any penalty function when noise is uniform. In equilibrium, the demand of the insider and the price functions are in general non-linear and remai…
New nonconvex penalty smooths at origin for deep learning.
problem Improving variable selection and bias in high-dimensional statistical learning.
method Developed a new nonconvex penalty function smooth at origin.
result Asymptotic bias of new penalty function vanishes exponentially fast.
New findings show privacy affects generalization error in a non-monotonic way.
problem Privacy and robustness in distributed learning.
method Theoretical analysis and matching lower/upper bounds on algorithmic stability.
result Generalization error is non-monotonically affected by privacy, depending on noise level.
New privacy framework tailored to specific data distributions.
problem Protecting individual data points in decision-making processes.
method Introducing tangent differential privacy, a new form of differential privacy.
result Entropic regularization guarantees tangent differential privacy under general conditions.
We study the problem of learning high dimensional regression models regularized by a structured-sparsity-inducing penalty that encodes prior structural information on either input or output sides. We consider two widely adopted types of such penalties as our motivating examples: 1) overlapping group lasso penalty, base…
Addresses theoretical and practical aspects of Gaussian differential privacy.
problem Theoretical and practical challenges in privacy-preserving data analysis.
method Discussion of f-differential privacy and Gaussian differential privacy.
result Gaussian differential privacy can enhance privacy in various applications.
New filters match advanced composition for adaptive privacy, with practical constants.
problem Limitations of existing adaptive composition methods.
method Constructed new filters and odometers that match advanced composition rates, including constants.
result Achieved fully adaptive privacy with practical filters and odometers.
DiffSketch combines privacy and communication efficiency in distributed learning.
problem Privacy and communication efficiency in distributed machine learning.
method DiffSketch uses Count Sketch for data stream summarization to achieve both privacy and efficiency.
result DiffSketch provides strong differential privacy guarantees and significant communication compression.