Paper evaluates and mitigates privacy risks in deep learning models.
problem Quantifying and defending against privacy attacks in deep learning.
method Quantitative evaluation of trade-offs, reformulating attacks, and proposing a novel SPN.
result Model accuracy improved by 5-20% while maintaining data privacy.
This study examines how model architecture affects deep learning model privacy.
problem Privacy concerns in deep learning models due to potential leakage of sensitive information.
method Investigation of CNNs and Transformers, focusing on activation layers, stem layers, LN layers, and attention modules.
result Transformers generally exhibit higher vulnerability to privacy attacks than CNNs.
New method calibrates noise for attack risk, improving ML model accuracy.
problem Improving accuracy of privacy-preserving ML models while maintaining privacy.
method Directly calibrates noise scale to a desired attack risk level, bypassing the standard ε-calibration. result Significantly decreases noise scale, leading to increased utility at the same risk level.
Paper improves privacy for language models against reconstruction attacks.
problem Reconstruction attacks can regenerate training data from language models.
method Uses Rényi differential privacy with optimized privacy budgets.
result Better privacy guarantees for extraction of rare secrets.
Synth-MIA assesses privacy leakage in synthetic tabular data models.
problem Challenges in evaluating privacy leakage in synthetic tabular data.
method Unified threat framework deploying multiple attacks.
result Higher synthetic data quality correlates with greater privacy leakage.
New method improves privacy risk evaluation of machine learning models.
problem Machine learning models can be vulnerable to membership inference attacks.
method Proposed new inference attack method based on prediction entropy, and introduced privacy risk score metric.
result Existing defense approaches are not as effective as previously reported.
Proposes a new privacy notion for membership inference attacks on machine learning models.
problem Membership inference attacks on machine learning models.
method Introduces f-Membership Inference Privacy (f-MIP) and μ-Gaussian Membership Inference Privacy (μ-GMIP) to quantify and mitigate privacy risks. result Analyzes likelihood ratio-based attacks and derives μ-GMIP guarantees for stochastic gradient descent (SGD) models. To promote secure and private artificial intelligence (SPAI), we review studies on the model security and data privacy of DNNs. Model security allows system to behave as intended without being affected by malicious external influences that can compromise its integrity and efficiency. Security attacks can be divided bas…
This paper evaluates and compares gradient leakage attacks in federated learning.
problem Gradient leakage attacks compromise client privacy in federated learning.
method Formal and experimental analysis of gradient leakage attacks, evaluation of attack effectiveness and cost.
result Gradient leakage attacks can reconstruct private local training data from shared parameter updates.
Machine learning models, especially deep neural networks have been shown to be susceptible to privacy attacks such as membership inference where an adversary can detect whether a data point was used for training a black-box model. Such privacy risks are exacerbated when a model's predictions are used on an unseen data …
Score attack method provides a lower bound on privacy-constrained minimax risk.
problem Characterizing the optimality of privacy-constrained statistical models.
method Score attack based on tracing attack concept.
result Optimally lower bounds the minimax risk of estimating unknown model parameters.
SAPAG attacks distributed learning by reconstructing true training data from gradients.
problem Privacy attacks on distributed learning systems through gradients.
method SAPAG uses a Gaussian kernel-based gradient difference distance measure.
result SAPAG can reconstruct training data on various DNNs and at different training phases.
Efficient method defends privacy in federated learning without accuracy loss.
problem Privacy attacks on federated learning by reconstructing and identifying local data.
method Random noise perturbation method that allows recovery of true gradients.
result Strong privacy protection without sacrificing learning accuracy.
New attacks improve privacy audits by analyzing model updates.
problem Improving privacy audits of AI models through sequence analysis.
method Developed SeMI attacks to identify target insertions in model sequences.
result SeMI attacks achieve higher power and tighter privacy audits.
New defense method protects client data privacy in federated learning.
problem Gradient leakage attacks in federated learning.
method Learning to obscure data to generate synthetic samples.
result Synthetic samples preserve predictive features and protect privacy.
New MCMC method estimates differential privacy from multiple MIAs without worst-case assumptions.
problem Bayesian estimation of differential privacy from membership inference attacks.
method Bayesian estimation via MCMC algorithm (MCMC-DP-Est).
result More cautious privacy analysis with joint estimation of MIA strengths and privacy parameter.
New attacks can infer model training membership using only label predictions, not confidence.
problem Inferring whether a data point was used to train a machine learning model.
method Evaluate model's predicted labels under perturbations to infer membership.
result Label-only attacks perform as well as confidence-based attacks and break defenses that rely on confidence masking.
Paper proposes efficient privacy-preserving matrix encryption for secure collaborative learning against malicious adversaries.
problem Secure collaborative learning of sensitive data across different agencies is challenging with malicious adversaries.
method The paper applies matrix encryption to secure data against chosen plaintext attack, known plaintext attack, and collusion attack, achieving local differential privacy and high computation efficiency.
result The proposed schemes are computationally efficient and secure against malicious adversaries compared to existing techniques.
Unified framework for analyzing model stealing attacks and defenses.
problem Vulnerability of ML applications to model stealing attacks.
method Developed a rigorous threat model and evaluation criteria, proposed methods to quantify attack and defense strategies.
result Demonstrated the importance of attack-specific perturbations for effective defenses.
Efficient framework for training machine learning models at edge without data movement.
problem Lack of privacy-preserving and computationally efficient methods for deep learning model training.
method Privacy preserving FedCollabNN framework for federated learning.
result Framework is computationally efficient and robust against adversarial attacks.
Gen-LRA attacks synthetic data leakage without model knowledge.
problem Auditing synthetic data privacy leakage.
method Generative Likelihood Ratio Attack (Gen-LRA).
result Gen-LRA outperforms other attacks across metrics.
New mechanism protects neural network weights from privacy attacks during self-supervised learning.
problem Privacy risks during fine-tuning stage of self-supervised learning.
method Proposes a novel differential privacy mechanism using additive logistic noise.
result Reduces membership inference attack accuracy to 50% while maintaining below 5% performance loss.
Deep learning model developers often use cloud GPU resources to experiment with large data and models that need expensive setups. However, this practice raises privacy concerns. Adversaries may be interested in: 1) personally identifiable information or objects encoded in the training images, and 2) the models trained …
Designing a data sharing mechanism without sacrificing too much privacy can be considered as a game between data holders and malicious attackers. This paper describes a compressive adversarial privacy framework that captures the trade-off between the data privacy and utility. We characterize the optimal data releasing …
This paper analyzes FL privacy risks and defensive strategies.
problem Privacy leakage in federated learning.
method Literature review of attack methods and defensive strategies.
result No single defensive strategy is sufficient for all attacks.
Log-Loss scores expose membership privacy breaches.
problem Privacy leakage from statistical aggregates like Log-Loss scores.
method Proved that Log-Loss scores enable full accuracy membership inference in a single query.
result Complete membership privacy breach is possible with Log-Loss scores.
New attacks show ML models can be compromised even when targeting one concept.
problem Vulnerability of ML models to multi-concept attacks.
method Developed novel multi-concept attack techniques for deep learning.
result Successfully attacked one set of classifiers without impacting others.
Study compares federated learning and coreset approaches for privacy in distributed machine learning.
problem Measuring privacy in distributed machine learning approaches.
method Comparison of federated learning and coreset approaches using membership inference attack.
result Federated learning offers better privacy than coreset, but with higher communication cost.
Membership inference attacks seek to infer the membership of individual training instances of a privately trained model. This paper presents a membership privacy analysis and evaluation system, called MPLens, with three unique contributions. First, through MPLens, we demonstrate how membership inference attack methods …
Principled mapping from pure-DP ε to GDP μ for Gaussian differential privacy
problem Choosing the μ parameter in Gaussian differential privacy
method Matching the worst-case success of a membership inference attack
result Recommendation of μ ≈ ε/5 as a conservative general-purpose conversion
Survey on threats to federated learning models.
problem Vulnerabilities in federated learning protocols.
method Taxonomy of threat models and attacks.
result Important future research directions.
Generative text classifiers are most vulnerable to membership inference attacks.
problem Privacy threat from Membership Inference Attacks on generative text classifiers.
method Comprehensive empirical evaluation of generative, discriminative, and pseudo-generative classifiers across various datasets.
result Generative classifiers explicitly modeling P(X,Y) are most vulnerable to membership leakage. The paper analyzes privacy leakage in federated learning using linear algebra and optimization theory.
problem Privacy leakage in federated learning despite its promise for data privacy.
method Theoretical analysis from linear algebra and optimization theory perspectives.
result Derives sufficient conditions to prevent data reconstruction attacks and establishes an upper bound on privacy leakage.
We consider membership inference attacks, one of the main privacy issues in machine learning. These recently developed attacks have been proven successful in determining, with confidence better than a random guess, whether a given sample belongs to the dataset on which the attacked machine learning model was trained. S…
Machine learning models leak information about the datasets on which they are trained. An adversary can build an algorithm to trace the individual members of a model's training dataset. As a fundamental inference attack, he aims to distinguish between data points that were part of the model's training set and any other…
APGE protects graph node representations from inference attacks.
problem Privacy leakage in graph embedding methods.
method Adversarial training framework with disentangling and purging mechanisms.
result APGE preserves structural and utility attributes while concealing private information.
This paper analyzes privacy risks in neural network pruning and proposes a defense mechanism.
problem Privacy risks in neural network pruning due to membership inference attacks.
method Investigates the impact of pruning on prediction divergence and proposes a self-attention membership inference attack.
result Proposed defense mechanism mitigates privacy risks while maintaining sparsity and accuracy.
New attacks show data augmentation may not improve privacy.
problem Measuring privacy risk in models trained with data augmentation.
method Formulated membership inference as a set classification problem, designed input permutation invariant features.
result Proposed approach universally outperforms original methods on models trained with data augmentation.
This paper protects rankings from differential privacy breaches.
problem Leakage of personal information in rankings.
method Develops ε-ranking differential privacy and a multistage ranking algorithm.
result Establishes the connection between Mallows model and ε-ranking differential privacy.
Unified framework analyzes privacy risks from gradients in distributed learning.
problem Analyzing inference privacy risks from gradients in machine learning.
method Unified game-based framework for various attacks, including attribute, property, distributional, and user disclosures.
result Demonstrates inefficacy of data aggregation for privacy against inference attacks.
Paper explores how poisoning data can increase privacy risks in machine learning models.
problem Increasing privacy risks of benign training samples through data poisoning attacks.
method Proposes generic and optimization-based attacks to amplify membership exposure.
result Demonstrates substantial increase in membership inference precision with minimal model performance degradation.
Adversarial training was introduced as a way to improve the robustness of deep learning models to adversarial attacks. This training method improves robustness against adversarial attacks, but increases the models vulnerability to privacy attacks. In this work we demonstrate how model inversion attacks, extracting trai…
Machine learning algorithms, when applied to sensitive data, pose a distinct threat to privacy. A growing body of prior work demonstrates that models produced by these algorithms may leak specific private information in the training data to an attacker, either through the models' structure or their observable behavior.…
Efficiently preserves privacy in logistic regression for IoT data.
problem Balancing data privacy and utility in collaborative learning.
method Matrix encryption approach for secure multi-party computation.
result Proposes a privacy-preserving logistic regression model with fast convergence.
Deep neural networks are susceptible to various inference attacks as they remember information about their training data. We design white-box inference attacks to perform a comprehensive privacy analysis of deep learning models. We measure the privacy leakage through parameters of fully trained models as well as the pa…
Federated learning is vulnerable to backdoor attacks; a new defense method is proposed.
problem Backdoor attacks in federated learning that can misclassify models.
method Adjusting the learning rate based on sign information of agents' updates.
result Our defense significantly reduces or eliminates backdoor attacks in federated learning.
Poisoning datasets can reveal private details of other users' training points.
problem Integrity and privacy of machine learning training data.
method Active inference attacks that poison a small fraction of the training dataset.
result Poisoning as little as 0.1% of the training dataset can significantly boost inference attacks.
Paper evaluates membership inference attacks on transfer learning models.
problem Evaluating membership inference attacks on transfer learning models.
method Shadow model training strategy to derive data for membership inference classifier.
result Membership inference attacks can achieve effective performance against transfer learning models.