This paper bounds min-entropy leakage for Blowfish privacy using graph symmetries.
problem Bounding min-entropy leakage for Blowfish privacy mechanisms.
method Organizing analysis over symmetrical partitions corresponding to orbits of graph automorphism groups.
result Demonstrates a construction meeting the bound with asymptotic equality, showing tightness.
Differential privacy protects data privacy by adding noise to data.
problem Leakage of sensitive data through common methods like encryption and endpoint protection.
method Randomized response technique to add noise to data collection.
result Differential privacy ensures strong privacy with better utility.
Synth-MIA assesses privacy leakage in synthetic tabular data models.
problem Challenges in evaluating privacy leakage in synthetic tabular data.
method Unified threat framework deploying multiple attacks.
result Higher synthetic data quality correlates with greater privacy leakage.
The paper analyzes privacy leakage in federated learning using linear algebra and optimization theory.
problem Privacy leakage in federated learning despite its promise for data privacy.
method Theoretical analysis from linear algebra and optimization theory perspectives.
result Derives sufficient conditions to prevent data reconstruction attacks and establishes an upper bound on privacy leakage.
It has been widely understood that differential privacy (DP) can guarantee rigorous privacy against adversaries with arbitrary prior knowledge. However, recent studies demonstrate that this may not be true for correlated data, and indicate that three factors could influence privacy leakage: the data correlation pattern…
New defense method protects client data privacy in federated learning.
problem Gradient leakage attacks in federated learning.
method Learning to obscure data to generate synthetic samples.
result Synthetic samples preserve predictive features and protect privacy.
This paper evaluates and compares gradient leakage attacks in federated learning.
problem Gradient leakage attacks compromise client privacy in federated learning.
method Formal and experimental analysis of gradient leakage attacks, evaluation of attack effectiveness and cost.
result Gradient leakage attacks can reconstruct private local training data from shared parameter updates.
Improved canary crafting for one-run privacy auditing reduces leakage estimates.
problem Detecting canaries in one-run privacy auditing to estimate leakage effectively.
method Optimizes canaries for detectability and diversity, using a greedy initialization and bilevel optimization.
result Achieves stronger leakage estimates at lower computational cost.
Log-Loss scores expose membership privacy breaches.
problem Privacy leakage from statistical aggregates like Log-Loss scores.
method Proved that Log-Loss scores enable full accuracy membership inference in a single query.
result Complete membership privacy breach is possible with Log-Loss scores.
Differential privacy of Gaussian process posterior sampling
problem Privacy of posterior sample paths from Gaussian process
method Intrinsic randomness yields DP guarantees
result Intrinsic randomness yields DP guarantees
Study on privacy leakage in noisy gradient descent algorithms.
problem Information leakage of iterative randomized learning algorithms about training data.
method Analyzes the dynamics of Rényi differential privacy loss in noisy gradient descent algorithms.
result Privacy loss converges exponentially fast for smooth and strongly convex loss functions.
We consider membership inference attacks, one of the main privacy issues in machine learning. These recently developed attacks have been proven successful in determining, with confidence better than a random guess, whether a given sample belongs to the dataset on which the attacked machine learning model was trained. S…
DPNR preserves privacy of text representations using differential privacy.
problem Privacy leakage in deep learning text representations.
method DPNR uses Differential Privacy to provide formal privacy guarantees and dropout masking for enhanced privacy.
result DPNR reduces privacy leakage without significantly sacrificing main task performance.
Split learning preserves privacy in 1D CNN models for detecting heart abnormalities.
problem Privacy leakage in 1D CNN models under split learning.
method Implemented and validated an 1D CNN model under split learning, applied privacy leakage mitigation techniques.
result Split learning alone is insufficient to maintain raw data privacy in 1D CNN models.
Deep Learning algorithms have recently become the de-facto paradigm for various prediction problems, which include many privacy-preserving applications like online medical image analysis. Presumably, the privacy of data in a deep learning system is a serious concern. There have been several efforts to analyze and explo…
Powered by machine learning services in the cloud, numerous learning-driven mobile applications are gaining popularity in the market. As deep learning tasks are mostly computation-intensive, it has become a trend to process raw data on devices and send the deep neural network (DNN) features to the cloud, where the feat…
This work provides bounds on generalization error and privacy leakage in federated learning.
problem Bounding generalization error and privacy leakage in federated learning.
method Information-theoretic framework for classical, distributed, and federated learning.
result Upper and lower bounds on generalization error and privacy leakage.
Privacy improves robustness in statistical estimation.
problem Sparse mean estimation under privacy constraints.
method Sum-of-Squares method and exponential-time mechanisms.
result Private algorithms matching optimal tradeoffs are not known, but achieved via Sum-of-Squares.
In this paper, we aim to understand the generalization properties of generative adversarial networks (GANs) from a new perspective of privacy protection. Theoretically, we prove that a differentially private learning algorithm used for training the GAN does not overfit to a certain degree, i.e., the generalization gap …
Paper proposes incentive mechanism to encourage participation in federated learning.
problem Users are reluctant to participate in federated learning due to privacy concerns.
method Formulated as a two-stage Stackelberg game, designed an incentive mechanism to select and compensate users.
result Demonstrated effectiveness of the proposed incentive mechanism through simulations.
Foundation models leak sensitive data in synthetic tabular data generation, especially LLaMA 3.3 70B.
problem Privacy leakage in synthetic tabular data generation using foundation models.
method Benchmarked three foundation models (GPT-4o-mini, LLaMA 3.3 70B, TabPFN v2) against four baselines on 35 real-world tables.
result Foundation models, especially LLaMA 3.3 70B, have the highest privacy risk in synthetic tabular data generation.
Fairness in machine learning increases privacy risks, especially for underrepresented groups.
problem Privacy risks in fair machine learning models, particularly for underrepresented groups.
method Membership inference attacks to measure information leakage and analyze fairness vs. privacy trade-offs.
result Achieving fairness in machine learning models increases privacy risks, especially for underrepresented groups.
Researchers found PP-GANs can hide sensitive data in sanitized images, undermining privacy checks.
problem Lack of formal proofs of privacy in PP-GANs for image sanitization.
method Subverted PP-GANs for facial expression recognition to hide sensitive data in sanitized images.
result It is possible to hide sensitive identification data in sanitized PP-GAN output images, even allowing reconstruction of entire input images.
Fine-tuning LLMs on privacy-sensitive data introduces privacy risk, and synthetic data audits can quantify this risk.
problem Fine-tuning LLMs on privacy-sensitive data introduces privacy risk.
method Generate synthetic canaries via high-temperature sampling from LLMs.
result Synthetic canaries are high-influence outliers that ensure strong audits.
A new DP approach for Conformal Prediction using quantile search.
problem Privacy leakage in uncertainty quantification methods like Conformal Prediction.
method Private Conformity via Quantile Search (P-COQS) using randomized binary search.
result The approach targets the desired (1−α)-level of coverage with slight under-covering. There is an increasing concern that most current published research findings are false. The main cause seems to lie in the fundamental disconnection between theory and practice in data analysis. While the former typically relies on statistical independence, the latter is an inherently adaptive process: new hypotheses a…
A new method to protect enterprise data privacy in AI models.
problem Enterprise data leakage risks in AI models.
method ABack, a training-free mechanism using Hidden State Model.
result Improves privacy utility by up to 15% over strong baselines.
Machine learning services pose privacy risks if data or model parameters are compromised.
problem Privacy risks in machine learning services when data or model parameters are exposed.
method Systematic review of privacy challenges, adversarial models, attacks, and defenses.
result Need for better evaluations, targeted defenses, and policy studies.
To continuously improve quality and reflect changes in data, machine learning applications have to regularly retrain and update their core models. We show that a differential analysis of language model snapshots before and after an update can reveal a surprising amount of detailed information about changes in the train…
Paper improves privacy-utility trade-off in federated learning.
problem Repeated parameter sharing in federated learning leaks private data.
method Proposes a new representation federated learning objective with differential privacy guarantees.
result Algorithm \DPFEDREP\ converges to a global optimal solution with a linear rate and privacy budget-dependent radius.
Machine learning has been widely applied to various applications, some of which involve training with privacy-sensitive data. A modest number of data breaches have been studied, including credit card information in natural language data and identities from face dataset. However, most of these studies focus on supervise…
This paper proposes a distributed deep learning framework for privacy-preserving medical data training. In order to avoid patients' data leakage in medical platforms, the hidden layers in the deep learning framework are separated and where the first layer is kept in platform and others layers are kept in a centralized …
Machine unlearning can compromise privacy, study shows.
problem Machine unlearning may leave data imprints in ML models, risking privacy.
method Proposed a membership inference attack to detect leakage.
result Machine unlearning can lead to unintended privacy risks.
Regularization may not fully protect against membership inference attacks.
problem Mitigating membership inference attacks in deep learning models.
method Evaluation of 8 regularization mechanisms on three image classification tasks.
result Certain regularization mechanisms can inadvertently help membership inference attacks.
Improved privacy analysis for stochastic gradient descent.
problem Analyzing privacy leakage in noisy stochastic gradient descent.
method Modeling Rényi divergence dynamics with Langevin diffusions, proving exponential privacy loss convergence for smooth and strongly convex objectives.
result Privacy loss converges exponentially fast for smooth and strongly convex objectives under constant step size.
Latent factor models for recommender systems represent users and items as low dimensional vectors. Privacy risks of such systems have previously been studied mostly in the context of recovery of personal information in the form of usage records from the training data. However, the user representations themselves may be…
Paper explores how poisoning data can increase privacy risks in machine learning models.
problem Increasing privacy risks of benign training samples through data poisoning attacks.
method Proposes generic and optimization-based attacks to amplify membership exposure.
result Demonstrates substantial increase in membership inference precision with minimal model performance degradation.
Bayesian deep learning is recently regarded as an intrinsic way to characterize the weight uncertainty of deep neural networks~(DNNs). Stochastic Gradient Langevin Dynamics~(SGLD) is an effective method to enable Bayesian deep learning on large-scale datasets. Previous theoretical studies have shown various appealing p…
Gen-LRA attacks synthetic data leakage without model knowledge.
problem Auditing synthetic data privacy leakage.
method Generative Likelihood Ratio Attack (Gen-LRA).
result Gen-LRA outperforms other attacks across metrics.
Large-scale collections of electronic records constitute both an opportunity for the development of more accurate prediction models and a threat for privacy. To limit privacy exposure new privacy-enhancing techniques are emerging such as federated learning which enables large-scale data analysis while avoiding the cent…
New analysis shows reconstruction attacks are unreliable without prior data knowledge.
problem Privacy and security risks from neural network memorization of training data.
method Complementary analysis of reconstruction methods, proving their unreliability without prior data knowledge.
result Reconstruction attacks are fundamentally unreliable without prior data knowledge, and networks trained more extensively are less susceptible.
MAPPING debiases GNNs for fair node classification with limited leakage.
problem Graph Neural Networks inherit and exacerbate historical discrimination in high-stake domains.
method MAPPING uses distance covariance-based fairness constraints and adversarial debiasing.
result MAPPING achieves better trade-offs between fairness and utility, mitigating privacy risks.
New algorithm protects privacy in IVaR regression while maintaining accuracy.
problem Privacy leakage in classical IVaR methods.
method Noisy two-stage gradient descent with differential privacy guarantees.
result Achieves statistical efficiency and privacy in IVaR regression.
Privacy-preserving inference for clinical trials using differential privacy.
problem Balancing knowledge sharing and privacy in healthcare data.
method Differential privacy (DP) applied to log-linear belief updates in distributed settings.
result Differentially private, distributed inference methods outperform existing techniques.
This paper improves privacy accounting in decentralized FL using f-Differential Privacy.
problem Challenges in accurately quantifying privacy budget in decentralized FL.
method Develops two new f-DP-based accounting methods for decentralized FL.
result Yields tighter (ε,δ) bounds and improved utility compared to existing methods.
Paper improves differential privacy analysis for machine learning.
problem Quantifying privacy leakage in noisy gradient descent.
method Shifted interpolation in f-differential privacy. result First exact privacy analysis for strongly convex optimization.
Image recognition systems have demonstrated tremendous progress over the past few decades thanks, in part, to our ability of learning compact and robust representations of images. As we witness the wide spread adoption of these systems, it is imperative to consider the problem of unintended leakage of information from …
This study examines how model architecture affects deep learning model privacy.
problem Privacy concerns in deep learning models due to potential leakage of sensitive information.
method Investigation of CNNs and Transformers, focusing on activation layers, stem layers, LN layers, and attention modules.
result Transformers generally exhibit higher vulnerability to privacy attacks than CNNs.