Paper introduces attacks to infer GAN training dataset properties.
problem Security and privacy risks of generative models like GANs.
method Proposes a general attack pipeline for two attack scenarios.
result Demonstrates strong performance in inferring GAN training dataset properties.
This paper analyzes how data and model properties affect membership inference attacks.
problem Understanding and mitigating the vulnerability of machine learning models to membership inference attacks.
method Empirical analysis of data and model properties on MIA success.
result Data and model properties, not just model overfitting, influence MIA success.
Researchers disrupt Gaussian model inference to test adversarial attacks.
problem Disrupting conditional inference in multivariate Gaussian models under adversarial conditions.
method Considered white- and grey-box settings with complete and incomplete knowledge of the Gaussian distribution, respectively. Reduced to quadratic and stochastic quadratic programs. Derived structural properties for solution methods.
result Demonstrated the impact and efficacy of attacks in various applications, including real estate evaluation, interest rate estimation, and signals processing.
Graph embedding leaks sensitive graph properties and subgraphs.
problem Privacy risks in graph embedding sharing.
method Three inference attacks and a defense mechanism.
result High accuracy in inferring graph properties and subgraphs.
The arms race between attacks and defenses for machine learning models has come to a forefront in recent years, in both the security community and the privacy community. However, one big limitation of previous research is that the security domain and the privacy domain have typically been considered separately. It is t…
Quantile regression attacks outperform shadow models in unseen class membership inference attacks.
problem Failure of shadow model attacks on unseen classes due to limited data access.
method Quantile regression attacks that learn features of member examples.
result Quantile regression attacks achieve up to 11x higher TPR than shadow model-based approaches.
Causal models offer stronger privacy guarantees and better generalization than associational models in machine learning.
problem Privacy attacks on machine learning models, especially membership inference attacks.
method Demonstrated the benefit of causal learning in machine learning models, showing better generalization and stronger privacy guarantees.
result Causal models provide stronger differential privacy guarantees and are more robust to membership inference attacks compared to associational models.
Defends against ML inference attacks using adversarial examples.
problem Automated inference attacks using ML classifiers pose privacy and security threats.
method Turns ML classifier vulnerabilities into defenses by adding adversarial noise to public data.
result Adversarial examples can mislead ML classifiers and protect private data.
New attacks can infer model training membership using only label predictions, not confidence.
problem Inferring whether a data point was used to train a machine learning model.
method Evaluate model's predicted labels under perturbations to infer membership.
result Label-only attacks perform as well as confidence-based attacks and break defenses that rely on confidence masking.
New attacks improve privacy audits by analyzing model updates.
problem Improving privacy audits of AI models through sequence analysis.
method Developed SeMI attacks to identify target insertions in model sequences.
result SeMI attacks achieve higher power and tighter privacy audits.
Bayesian models can be tricked into believing false data.
problem Vulnerability of Bayesian inference to data poisoning attacks.
method Developed attacks to manipulate Bayesian posterior through deletion and replication of data.
result Demonstrated that Bayesian inference can be steered to target distributions.
In machine learning (ML) security, attacks like evasion, model stealing or membership inference are generally studied in individually. Previous work has also shown a relationship between some attacks and decision function curvature of the targeted model. Consequently, we study an ML model allowing direct control over t…
Regularization may not fully protect against membership inference attacks.
problem Mitigating membership inference attacks in deep learning models.
method Evaluation of 8 regularization mechanisms on three image classification tasks.
result Certain regularization mechanisms can inadvertently help membership inference attacks.
New method bounds membership inference attack success using mutual information.
problem Vulnerability of deep neural networks to membership inference attacks.
method Extended Fano's inequality to measure mutual information between inputs and activations.
result Empirical evaluation shows strong correlation between mutual information and model susceptibility.
Paper assesses holistic risks of inference attacks on ML models.
problem Lack of comprehensive risk assessment of inference attacks on ML models.
method Presented a threat model taxonomy for four inference attacks on five model architectures and four image datasets.
result Complexity of training dataset influences attack performance; model stealing and membership inference attacks are negatively correlated.
New attacks reveal membership in label-only ML models.
problem Vulnerability of ML models to membership inference attacks.
method Developed decision-based membership inference attacks.
result Label-only exposures are vulnerable to membership leakage.
Study membership inference under skewed priors and adaptive thresholds, improving attack accuracy.
problem Membership inference in imbalanced settings with selective thresholding.
method Developed PPV metric for skewed priors, threshold selection procedure, and a new inference attack.
result Improved inference attack accuracy in imbalanced settings.
Proposes a new privacy notion for membership inference attacks on machine learning models.
problem Membership inference attacks on machine learning models.
method Introduces f-Membership Inference Privacy (f-MIP) and μ-Gaussian Membership Inference Privacy (μ-GMIP) to quantify and mitigate privacy risks. result Analyzes likelihood ratio-based attacks and derives μ-GMIP guarantees for stochastic gradient descent (SGD) models. Poisoning datasets can reveal private details of other users' training points.
problem Integrity and privacy of machine learning training data.
method Active inference attacks that poison a small fraction of the training dataset.
result Poisoning as little as 0.1% of the training dataset can significantly boost inference attacks.
Attack graphs provide compact representations of the attack paths that an attacker can follow to compromise network resources by analysing network vulnerabilities and topology. These representations are a powerful tool for security risk assessment. Bayesian inference on attack graphs enables the estimation of the risk …
Paper evaluates membership inference attacks on transfer learning models.
problem Evaluating membership inference attacks on transfer learning models.
method Shadow model training strategy to derive data for membership inference classifier.
result Membership inference attacks can achieve effective performance against transfer learning models.
Unified framework analyzes privacy risks from gradients in distributed learning.
problem Analyzing inference privacy risks from gradients in machine learning.
method Unified game-based framework for various attacks, including attribute, property, distributional, and user disclosures.
result Demonstrates inefficacy of data aggregation for privacy against inference attacks.
This paper analyzes how differential privacy and data skewness affect membership inference attacks.
problem Membership inference attacks on privately trained models.
method Developed MPLens system to evaluate membership inference vulnerability.
result Membership inference risk is higher with skewed training data and differential privacy has trade-offs.
Study shows membership inference requires many more samples than training data.
problem Membership inference attacks require more reference samples than training data.
method Investigated the sample complexity for membership inference in Gaussian mean estimation.
result Ω(n + n^2 ρ^2) samples are necessary for successful membership inference attacks.
Deep neural networks are susceptible to various inference attacks as they remember information about their training data. We design white-box inference attacks to perform a comprehensive privacy analysis of deep learning models. We measure the privacy leakage through parameters of fully trained models as well as the pa…
Membership inference (MI) attacks exploit the fact that machine learning algorithms sometimes leak information about their training data through the learned model. In this work, we study membership inference in the white-box setting in order to exploit the internals of a model, which have not been effectively utilized …
Survey on privacy issues in deep learning and proposed solutions.
problem Privacy concerns in deep learning models due to sensitive data.
method Review of existing privacy techniques and gaps in research.
result Identification of test-time inference privacy as a research gap.
Paper defends diffusion models from membership inference attacks using Langevin dynamics.
problem Defending diffusion models against membership inference attacks.
method Uses critically-damped higher-order Langevin dynamics with auxiliary variables.
result Demonstrates improved resistance to membership inference attacks through theoretical investigation and validation.
MI attacks often mislabel non-training samples, making them impractical.
problem MI attacks mislabel non-training samples, leading to high false positive rate.
method Analyzed new features like distance to decision boundary and gradient norms.
result MI attacks cannot achieve high accuracy and low false positive rate simultaneously.
New method uses model's generalization gap to predict membership inference attacks.
problem Predicting membership inference attacks on machine learning models.
method Uses the model's generalization gap as a metric to estimate the vulnerability to membership inference attacks.
result Demonstrates that the model's generalization gap provides an upper bound on MIA security.
Machine learning algorithms, when applied to sensitive data, pose a distinct threat to privacy. A growing body of prior work demonstrates that models produced by these algorithms may leak specific private information in the training data to an attacker, either through the models' structure or their observable behavior.…
Machine learning (ML) has progressed rapidly during the past decade and the major factor that drives such development is the unprecedented large-scale data. As data generation is a continuous process, this leads to ML model owners updating their models frequently with newly-collected data in an online learning scenario…
New attack reveals memorization patterns in pre-trained LLMs.
problem Determining if a data point was part of a pre-trained LLM's training set.
method Adapts MIA statistical tests to LLM's perplexity dynamics of subsequences.
result Significantly outperforms prior approaches in membership inference attacks.
Enhanced attacks quantify machine learning data leakage.
problem Quantifying how much machine learning models reveal about their training data.
method Hypothesis testing framework for membership inference attacks.
result New attacks achieve higher true positive rates with lower false positive rates.
New attacks reveal training label proportions in federated learning.
problem Adversaries can infer training label proportions from limited data.
method Proposed three types of attacks: Class Sniffing, Quantity Inference, Whole Determination.
result Our attacks work well on various tasks and datasets.
Membership inference determines, given a sample and trained parameters of a machine learning model, whether the sample was part of the training set. In this paper, we derive the optimal strategy for membership inference with a few assumptions on the distribution of the parameters. We show that optimal attacks only depe…
TAMIS improves MIA on synthetic data, reducing cost and requiring less knowledge.
problem Empirical assessment of privacy in machine learning algorithms.
method Improves MAMA-MIA by recovering graphical model from synthetic data and introducing a more accurate attack score.
result TAMIS achieves better or similar performance to MAMA-MIA on synthetic data challenges.
PASE method protects machine learning models from membership inference attacks without significant accuracy loss.
problem Membership inference attacks on machine learning models.
method Switching ensembles approach to mitigate privacy leakage.
result PASE method provides effective privacy protection with minimal accuracy penalty.
Attack graphs are a powerful tool for security risk assessment by analysing network vulnerabilities and the paths attackers can use to compromise network resources. The uncertainty about the attacker's behaviour makes Bayesian networks suitable to model attack graphs to perform static and dynamic analysis. Previous app…
AdvMind detects adversary intent in black-box attacks with high accuracy.
problem Detecting adversary intent in black-box adversarial attacks is challenging.
method AdvMind accounts for adversary adaptiveness and synthesizes queries to expose intent.
result AdvMind detects adversary intent with over 75% accuracy after observing less than 3 query batches.
Recently it's been shown that neural networks can use images of human faces to accurately predict Body Mass Index (BMI), a widely used health indicator. In this paper we demonstrate that a neural network performing BMI inference is indeed vulnerable to test-time adversarial attacks. This extends test-time adversarial a…
APGE protects graph node representations from inference attacks.
problem Privacy leakage in graph embedding methods.
method Adversarial training framework with disentangling and purging mechanisms.
result APGE preserves structural and utility attributes while concealing private information.
DeepPeep attacks DNN architectures to reveal design details, posing IP theft risks.
problem Protecting DNN architecture from IP theft in cloud-based services.
method Two-stage attack methodology exploiting design characteristics.
result DeepPeep successfully reverses-engineers compact DNN architectures.
Improved neural network robustness to adversarial attacks through smoothed inference.
problem Vulnerability of deep neural networks to adversarial attacks.
method Randomized smoothing applied to adversarial training, improving both robustness and performance.
result Significant improvement in accuracy on adversarial attacks (e.g., 60.4% on CIFAR-10 with ResNet-20, outperforming previous methods by 11.7%).
New method detects if data points were used in training models with low cost and high power.
problem Detecting if a particular data point was used in training a model.
method Fine-grained modeling of null hypothesis in likelihood ratio tests, leveraging reference models and population data.
result RMIA has superior test power compared to prior methods, even at extremely low false positive rates.
Most of the data manipulation attacks on deep neural networks (DNNs) during the training stage introduce a perceptible noise that can be catered by preprocessing during inference or can be identified during the validation phase. Therefore, data poisoning attacks during inference (e.g., adversarial attacks) are becoming…
As online systems based on machine learning are offered to public or paid subscribers via application programming interfaces (APIs), they become vulnerable to frequent exploits and attacks. This paper studies adversarial machine learning in the practical case when there are rate limitations on API calls. The adversary …
New method improves privacy risk evaluation of machine learning models.
problem Machine learning models can be vulnerable to membership inference attacks.
method Proposed new inference attack method based on prediction entropy, and introduced privacy risk score metric.
result Existing defense approaches are not as effective as previously reported.