Proposes a new privacy notion for membership inference attacks on machine learning models.
problem Membership inference attacks on machine learning models.
method Introduces f-Membership Inference Privacy (f-MIP) and μ-Gaussian Membership Inference Privacy (μ-GMIP) to quantify and mitigate privacy risks. result Analyzes likelihood ratio-based attacks and derives μ-GMIP guarantees for stochastic gradient descent (SGD) models. Principled mapping from pure-DP ε to GDP μ for Gaussian differential privacy
problem Choosing the μ parameter in Gaussian differential privacy
method Matching the worst-case success of a membership inference attack
result Recommendation of μ ≈ ε/5 as a conservative general-purpose conversion
Overparameterized models are more vulnerable to membership inference attacks.
problem Vulnerability of overparameterized models to membership inference attacks.
method Theoretical and empirical analysis of overparameterized linear and ridge-regularized linear regression models in the Gaussian data setting.
result Increased number of parameters and model complexity increase vulnerability to membership inference attacks.
Membership inference attacks seek to infer the membership of individual training instances of a privately trained model. This paper presents a membership privacy analysis and evaluation system, called MPLens, with three unique contributions. First, through MPLens, we demonstrate how membership inference attack methods …
Study shows membership inference requires many more samples than training data.
problem Membership inference attacks require more reference samples than training data.
method Investigated the sample complexity for membership inference in Gaussian mean estimation.
result Ω(n + n^2 ρ^2) samples are necessary for successful membership inference attacks.
Log-Loss scores expose membership privacy breaches.
problem Privacy leakage from statistical aggregates like Log-Loss scores.
method Proved that Log-Loss scores enable full accuracy membership inference in a single query.
result Complete membership privacy breach is possible with Log-Loss scores.
Large capacity machine learning (ML) models are prone to membership inference attacks (MIAs), which aim to infer whether the target sample is a member of the target model's training dataset. The serious privacy concerns due to the membership inference have motivated multiple defenses against MIAs, e.g., differential pr…
Generative text classifiers are most vulnerable to membership inference attacks.
problem Privacy threat from Membership Inference Attacks on generative text classifiers.
method Comprehensive empirical evaluation of generative, discriminative, and pseudo-generative classifiers across various datasets.
result Generative classifiers explicitly modeling P(X,Y) are most vulnerable to membership leakage. Paper evaluates membership inference attacks on transfer learning models.
problem Evaluating membership inference attacks on transfer learning models.
method Shadow model training strategy to derive data for membership inference classifier.
result Membership inference attacks can achieve effective performance against transfer learning models.
New mechanism protects neural network weights from privacy attacks during self-supervised learning.
problem Privacy risks during fine-tuning stage of self-supervised learning.
method Proposes a novel differential privacy mechanism using additive logistic noise.
result Reduces membership inference attack accuracy to 50% while maintaining below 5% performance loss.
New attacks can infer model training membership using only label predictions, not confidence.
problem Inferring whether a data point was used to train a machine learning model.
method Evaluate model's predicted labels under perturbations to infer membership.
result Label-only attacks perform as well as confidence-based attacks and break defenses that rely on confidence masking.
Proposes MIP, a privacy notion that requires less randomness than DP, leading to better utility.
problem Preserving privacy in machine learning models with sensitive data.
method Introduces membership inference privacy (MIP) as a new privacy notion and shows its relationship with differential privacy (DP).
result MIP can be achieved with less randomness than DP, resulting in better utility.
Study quantized models' privacy against membership inference attacks.
problem Privacy risk in quantized machine learning models.
method Proposed a new MIS indicator for post-training quantization procedures, minimizing empirical loss.
result Demonstrated effectiveness of new MIS indicator in assessing and ranking privacy risk.
Regularization can improve both privacy and performance in machine learning models.
problem Privacy vs. Utility trade-off in machine learning models.
method The study uses logistic regression with ridge regularization and a leave-one-out analysis tool.
result Increasing the number of parameters can improve both privacy and performance when coupled with proper regularization.
This paper quantifies how hard it is to identify specific data points in machine learning models.
problem Quantifying the difficulty of identifying specific data points in machine learning models.
method Characterizing optimal attacks and privacy defences, deriving impacts of noise and misspecification, and proposing a new covariance attack.
result The Mahalanobis distance explains the hardness of fixed-target membership inference attacks.
Machine learning models leak information about the datasets on which they are trained. An adversary can build an algorithm to trace the individual members of a model's training dataset. As a fundamental inference attack, he aims to distinguish between data points that were part of the model's training set and any other…
Machine unlearning can compromise privacy, study shows.
problem Machine unlearning may leave data imprints in ML models, risking privacy.
method Proposed a membership inference attack to detect leakage.
result Machine unlearning can lead to unintended privacy risks.
Differential privacy of Gaussian process posterior sampling
problem Privacy of posterior sample paths from Gaussian process
method Intrinsic randomness yields DP guarantees
result Intrinsic randomness yields DP guarantees
New method improves privacy risk evaluation of machine learning models.
problem Machine learning models can be vulnerable to membership inference attacks.
method Proposed new inference attack method based on prediction entropy, and introduced privacy risk score metric.
result Existing defense approaches are not as effective as previously reported.
New MCMC method estimates differential privacy from multiple MIAs without worst-case assumptions.
problem Bayesian estimation of differential privacy from membership inference attacks.
method Bayesian estimation via MCMC algorithm (MCMC-DP-Est).
result More cautious privacy analysis with joint estimation of MIA strengths and privacy parameter.
This paper analyzes privacy risks in neural network pruning and proposes a defense mechanism.
problem Privacy risks in neural network pruning due to membership inference attacks.
method Investigates the impact of pruning on prediction divergence and proposes a self-attention membership inference attack.
result Proposed defense mechanism mitigates privacy risks while maintaining sparsity and accuracy.
Study compares federated learning and coreset approaches for privacy in distributed machine learning.
problem Measuring privacy in distributed machine learning approaches.
method Comparison of federated learning and coreset approaches using membership inference attack.
result Federated learning offers better privacy than coreset, but with higher communication cost.
Regularization may not fully protect against membership inference attacks.
problem Mitigating membership inference attacks in deep learning models.
method Evaluation of 8 regularization mechanisms on three image classification tasks.
result Certain regularization mechanisms can inadvertently help membership inference attacks.
Privacy and transparency are two key foundations of trustworthy machine learning. Model explanations offer insights into a model's decisions on input data, whereas privacy is primarily concerned with protecting information about the training data. We analyze connections between model explanations and the leakage of sen…
New method assesses individual training points' privacy risk without retraining.
problem Privacy vulnerability of individual training points in membership inference attacks.
method Derives a closed-form decomposition of individual black-box MIA vulnerability, extending to deep networks.
result Proposes a surrogate score operating on last-layer representations that requires only a single trained model.
New method bounds membership inference attack success using mutual information.
problem Vulnerability of deep neural networks to membership inference attacks.
method Extended Fano's inequality to measure mutual information between inputs and activations.
result Empirical evaluation shows strong correlation between mutual information and model susceptibility.
Machine learning algorithms, when applied to sensitive data, pose a distinct threat to privacy. A growing body of prior work demonstrates that models produced by these algorithms may leak specific private information in the training data to an attacker, either through the models' structure or their observable behavior.…
A new, low-cost method speeds up membership inference attacks on large language models.
problem Membership inference attacks on large language models.
method An ensemble of small quantile regression models to determine model training set membership.
result Comparable or improved accuracy with significantly reduced computational cost.
Paper improves privacy for language models against reconstruction attacks.
problem Reconstruction attacks can regenerate training data from language models.
method Uses Rényi differential privacy with optimized privacy budgets.
result Better privacy guarantees for extraction of rare secrets.
We consider membership inference attacks, one of the main privacy issues in machine learning. These recently developed attacks have been proven successful in determining, with confidence better than a random guess, whether a given sample belongs to the dataset on which the attacked machine learning model was trained. S…
New method detects if data points were used in training models with low cost and high power.
problem Detecting if a particular data point was used in training a model.
method Fine-grained modeling of null hypothesis in likelihood ratio tests, leveraging reference models and population data.
result RMIA has superior test power compared to prior methods, even at extremely low false positive rates.
Deep neural networks are susceptible to various inference attacks as they remember information about their training data. We design white-box inference attacks to perform a comprehensive privacy analysis of deep learning models. We measure the privacy leakage through parameters of fully trained models as well as the pa…
Paper explores how poisoning data can increase privacy risks in machine learning models.
problem Increasing privacy risks of benign training samples through data poisoning attacks.
method Proposes generic and optimization-based attacks to amplify membership exposure.
result Demonstrates substantial increase in membership inference precision with minimal model performance degradation.
Paper develops a method to detect training data used in machine learning models.
problem Detecting training data used in machine learning models.
method Measures leakage of training data using a proxy of a model's total variation near training samples.
result Empirical evidence supports the effectiveness of the proposed method.
Data privacy is an important issue for "machine learning as a service" providers. We focus on the problem of membership inference attacks: given a data sample and black-box access to a model's API, determine whether the sample existed in the model's training data. Our contribution is an investigation of this problem in…
Pruning improves DNNs against MIA while reducing model size and computation.
problem Vulnerability of DNNs to membership inference attacks (MIA).
method Proposes a pruning algorithm to reduce model size and computational operations.
result Pruned subnetwork prevents privacy leakage from MIA with competitive accuracy.
Fairness in machine learning increases privacy risks, especially for underrepresented groups.
problem Privacy risks in fair machine learning models, particularly for underrepresented groups.
method Membership inference attacks to measure information leakage and analyze fairness vs. privacy trade-offs.
result Achieving fairness in machine learning models increases privacy risks, especially for underrepresented groups.
New attacks show data augmentation may not improve privacy.
problem Measuring privacy risk in models trained with data augmentation.
method Formulated membership inference as a set classification problem, designed input permutation invariant features.
result Proposed approach universally outperforms original methods on models trained with data augmentation.
We quantitatively investigate how machine learning models leak information about the individual data records on which they were trained. We focus on the basic membership inference attack: given a data record and black-box access to a model, determine if the record was in the model's training dataset. To perform members…
Improved canary crafting for one-run privacy auditing reduces leakage estimates.
problem Detecting canaries in one-run privacy auditing to estimate leakage effectively.
method Optimizes canaries for detectability and diversity, using a greedy initialization and bilevel optimization.
result Achieves stronger leakage estimates at lower computational cost.
Machine learning models, especially deep neural networks have been shown to be susceptible to privacy attacks such as membership inference where an adversary can detect whether a data point was used for training a black-box model. Such privacy risks are exacerbated when a model's predictions are used on an unseen data …
The arms race between attacks and defenses for machine learning models has come to a forefront in recent years, in both the security community and the privacy community. However, one big limitation of previous research is that the security domain and the privacy domain have typically been considered separately. It is t…
Challenge hides and seeks privacy in clinical time-series data.
problem De-identifying clinical time-series data while preserving utility and privacy.
method Synthetic data generation to preserve temporal dynamics and limit re-identification risk.
result A novel competition tracks synthetic data generation and patient re-identification.
TAMIS improves MIA on synthetic data, reducing cost and requiring less knowledge.
problem Empirical assessment of privacy in machine learning algorithms.
method Improves MAMA-MIA by recovering graphical model from synthetic data and introducing a more accurate attack score.
result TAMIS achieves better or similar performance to MAMA-MIA on synthetic data challenges.
New framework evaluates MIA without retraining, addressing biases.
problem Evaluate MIA without retraining and under distribution shift.
method Causal inference approach to MIA evaluation.
result Practical estimators for MIA metrics without retraining.
New method audits DP guarantees without noise or subsampling info.
problem Auditing DP guarantees of ML models without prior info.
method Histogram-based density estimation for lower bounds.
result Natural generalization of membership inference auditing.
Poisoning datasets can reveal private details of other users' training points.
problem Integrity and privacy of machine learning training data.
method Active inference attacks that poison a small fraction of the training dataset.
result Poisoning as little as 0.1% of the training dataset can significantly boost inference attacks.
Face recognition models can be inferred from student models, posing privacy risks.
problem Privacy threats in transfer learning models for face recognition.
method Membership inference attacks and attribute inference from aggregate-level information.
result Sensitive attributes can be inferred from student models, even with limited auxiliary information.