Gradient sparsification enhances privacy-preserving machine learning models.
problem Improving performance of differentially-private machine learning models under privacy constraints.
method Gradient sparsification combined with compressed sensing and additive Laplace noise.
result Gradient sparsification can improve performance of differentially-private machine learning models for small privacy budgets.
The Willmore flow preserves surface volume, leading to convergence to a sphere.
problem Long-term behavior of volume-preserving Willmore flow on surfaces.
method Volume-preserving Willmore flow, blow-up analysis, constrained Lojasiewicz-Simon inequality.
result Smooth solutions exist for spherical surfaces with Willmore energy below 8π and converge to a sphere.
Despite the impressive performance of random forests (RF), its theoretical properties have not been thoroughly understood. In this paper, we propose a novel RF framework, dubbed multinomial random forest (MRF), to analyze the \emph{consistency} and \emph{privacy-preservation}. Instead of deterministic greedy split rule…
Paper proposes G-CRD to improve GNNs by preserving global graph topology.
problem Improving lightweight GNNs for robust performance on large-scale real-world graphs.
method Introduces Graph Contrastive Representation Distillation (G-CRD) using contrastive learning.
result G-CRD consistently boosts GNN performance and robustness, outperforming existing methods.
New graph coarsening method preserves GNN message-passing signals.
problem Preserving GNN message-passing on coarsened graphs.
method Proposed a new oriented message-passing operation for coarsened graphs.
result Improved node classification results compared to naive methods.
Novel strategy for federated learning with privacy-preserving predictors and nonvacuous generalization bounds.
problem Privacy-preserving federated learning with nonvacuous generalization bounds.
method Randomized predictors, PAC-Bayesian generalization bound, synchronous and heterogeneous/homogenous cases.
result Achieves comparable predictive performance to batch approach while preserving privacy.
Preserving the privacy of individuals by protecting their sensitive attributes is an important consideration during microdata release. However, it is equally important to preserve the quality or utility of the data for at least some targeted workloads. We propose a novel framework for privacy preservation based on the …
Enhances machine learning models by preserving data structure, addressing statistical distortions.
problem Statistical distortions in synthetic data generated by Mixup.
method Proposes a generalized mixup method with a flexible weighting scheme to preserve data structure.
result Preserves statistical properties of original data while maintaining model performance.
SASSL improves self-supervised learning by preserving image structure.
problem Distorted augmented samples in self-supervised learning.
method Neural Style Transfer to decouple semantic and stylistic attributes.
result Boosts ImageNet top-1 accuracy by up to 2 percentage points.
A new aggregation strategy improves GNN performance and learning dynamics.
problem Improving expressivity and learning dynamics of GNNs.
method Proposes a variance-preserving aggregation function (VPA) for GNNs.
result VPA leads to increased predictive performance and improved learning dynamics.
Text style transfer aims to modify the style of a sentence while keeping its content unchanged. Recent style transfer systems often fail to faithfully preserve the content after changing the style. This paper proposes a structured content preserving model that leverages linguistic information in the structured fine-gra…
In clinical research, the lack of events of interest often necessitates imbalanced learning. One approach to resolve this obstacle is data integration or sharing, but due to privacy concerns neither is practical. Therefore, there is an increasing demand for a platform on which an analysis can be performed in a federate…
New defense method protects client data privacy in federated learning.
problem Gradient leakage attacks in federated learning.
method Learning to obscure data to generate synthetic samples.
result Synthetic samples preserve predictive features and protect privacy.
Safeguarding privacy in machine learning is highly desirable, especially in collaborative studies across many organizations. Privacy-preserving distributed machine learning (based on cryptography) is popular to solve the problem. However, existing cryptographic protocols still incur excess computational overhead. Here,…
Method preserves correlations in synthetic data.
problem Preserving dependence structure of original data.
method Orthogonal Procrustes problem for restoring Pearson correlation.
result Restores Pearson correlation structure while preserving feature distributions and downstream tasks performance.
IVFS simplifies feature selection for high-dimensional data preservation.
problem Maintaining structure and pairwise distances in high-dimensional data.
method IVFS framework based on persistent diagrams from computational topology.
result IVFS well preserves pairwise distances and topological patterns of full data.
L-CNNs preserve gauge symmetry in lattice simulations.
problem Breaking gauge symmetry in neural network models.
method Lattice gauge equivariant convolutional neural networks (L-CNNs).
result L-CNNs represent gauge invariant functions on the lattice.
Proposes a privacy-preserving recommendation system using matrix factorization and differential privacy.
problem Privacy leakage in recommendation systems when anonymizing user data is not sufficient.
method Uses matrix factorization and differential privacy via the Gaussian mechanism.
result Demonstrates excellent utility for privacy-preserving recommendation systems.
Graph-based clustering has shown promising performance in many tasks. A key step of graph-based approach is the similarity graph construction. In general, learning graph in kernel space can enhance clustering accuracy due to the incorporation of nonlinearity. However, most existing kernel-based graph learning mechanism…
We introduce general scattering transforms as mathematical models of deep neural networks with l2 pooling. Scattering networks iteratively apply complex valued unitary operators, and the pooling is performed by a complex modulus. An expected scattering defines a contractive representation of a high-dimensional probabil…
Spectral graph sparsification preserves geometry of GNN embeddings.
problem Maintaining geometric properties of graph neural network embeddings during sparsification.
method Proving spectral sparsification preserves squared pairwise distances, class means, and covariance structure in embedding space.
result Spectral sparsification preserves the geometry of learned embeddings in GNNs.
New method preserves privacy while detecting communities in distributed networks.
problem Privacy-preserving community detection in locally distributed multi-layer networks.
method Privacy-preserving Distributed Spectral Clustering (ppDSC) using randomized response mechanism.
result Developed a novel algorithm that maintains community structure while protecting privacy.
New DR algorithm preserves both local and global structure.
problem Trade-off between preserving local and global structure in DR methods.
method Analysis of existing DR methods and design principles for loss functions.
result Design of PaCMAP algorithm that preserves both local and global structure.
In this paper we investigate the usage of adversarial perturbations for the purpose of privacy from human perception and model (machine) based detection. We employ adversarial perturbations for obfuscating certain variables in raw data while preserving the rest. Current adversarial perturbation methods are used for dat…
We study the interplay between sequential decision making and avoiding discrimination against protected groups, when examples arrive online and do not follow distributional assumptions. We consider the most basic extension of classical online learning: "Given a class of predictors that are individually non-discriminato…
PCA-Guided Quantile Sampling preserves data structure in large datasets.
problem Preserving data structure in large-scale subsampling.
method PCA QS uses leading principal components to guide stratified sampling.
result PCA QS maintains statistical and geometric structure without distorting semantics.
CycleMorph improves image registration by preserving topology with cycle consistency.
problem Preserving original topology during deformation in image registration.
method Cycle-consistent deformable image registration approach.
result Effective and accurate registration on diverse image pairs within seconds.
We detail a new framework for privacy preserving deep learning and discuss its assets. The framework puts a premium on ownership and secure processing of data and introduces a valuable representation based on chains of commands and tensors. This abstraction allows one to implement complex privacy preserving constructs …
In this work we study the properties of deep neural networks (DNN) with random weights. We formally prove that these networks perform a distance-preserving embedding of the data. Based on this we then draw conclusions on the size of the training data and the networks' structure. A longer version of this paper with more…
Paper improves privacy in SGD with low noise, achieving optimal risk rates.
problem Privacy-preserving machine learning with good performance.
method Differentially private SGD with low-noise analysis.
result Achieves optimal excess risk rates for non-smooth losses.
Recent advances in computing have allowed for the possibility to collect large amounts of data on personal activities and private living spaces. To address the privacy concerns of users in this environment, we propose a novel framework called PR-GAN that offers privacy-preserving mechanism using generative adversarial …
Adaptive optimization algorithms, such as Adam and RMSprop, have shown better optimization performance than stochastic gradient descent (SGD) in some scenarios. However, recent studies show that they often lead to worse generalization performance than SGD, especially for training deep neural networks (DNNs). In this wo…
The Internet of Things (IoT) will be a main data generation infrastructure for achieving better system intelligence. However, the extensive data collection and processing in IoT also engender various privacy concerns. This paper provides a taxonomy of the existing privacy-preserving machine learning approaches develope…
Paper tackles privacy-preserving data density issues using deconvolution.
problem Privacy-preserving noise affects data density, leading to under/over-estimation.
method Develops deconvoluting kernel density estimators and regression models.
result Demonstrates improved accuracy in estimating heavy-hitters with locally differential data.
Paper presents a privacy-preserving method for dynamic assortment selection.
problem Personalized assortment recommendations with data privacy concerns.
method Perturbed upper confidence bound method integrating calibrated noise.
result Policy satisfies Joint Differential Privacy (JDP) with near-optimal regret bound.
Bayesian methods promise to fix many shortcomings of deep learning, but they are impractical and rarely match the performance of standard methods, let alone improve them. In this paper, we demonstrate practical training of deep networks with natural-gradient variational inference. By applying techniques such as batch n…
This study improves graph coarsening methods by preserving graph spectrum and distances.
problem Solving large-scale graph problems by working on a smaller graph.
method Developed a geometric approach using Gromov--Wasserstein distance to minimize the difference between graph distances and their coarsened versions.
result Minimizing the difference between graph distances and their coarsened versions can be achieved using the weighted kernel K-means method. A new method learns Hamiltonian functions from noisy data.
problem Learning Hamiltonian functions from noisy observations.
method Structure-preserving kernel ridge regression method.
result The method yields excellent numerical performances.
Lifelong learning aims to develop machine learning systems that can learn new tasks while preserving the performance on previous learned tasks. In this paper we present a method to overcome catastrophic forgetting on convolutional neural networks, that learns new tasks and preserves the performance on old tasks without…
This review explores federated learning for IoT data privacy.
problem Privacy risks and data transfer costs in IoT data analytics.
method Federated learning approach to protect privacy and reduce data transfer.
result Survey of methods for improving communication efficiency and privacy in IoT federated learning.
ResNets can approximate input distances under certain conditions, but existing theory is flawed.
problem Theoretical justification for regularizing ResNets to preserve input distances is flawed.
method Frequency analysis perspective to explain effectiveness of regularization schemes.
result Regularization schemes enforce a lower Lipschitz bound on low-frequency projections of images.
Network embedding is the process of learning low-dimensional representations for nodes in a network, while preserving node features. Existing studies only leverage network structure information and focus on preserving structural features. However, nodes in real-world networks often have a rich set of attributes providi…
GCML preserves geometric structure in manifold clustering for diverse data types.
problem Loss functions in manifold clustering can corrupt latent space structure.
method GCML framework with isometric and ranking losses for geometric structure preservation.
result GCML outperforms other methods in latent space structure preservation and performance metrics.
P3LS preserves privacy while integrating data across companies.
problem Privacy concerns in cross-organizational data exchange and integration.
method Privacy-preserving federated learning technique using SVD-based PLS and random masks.
result Improves prediction performance on process-related indicators.
The paper introduces a framework to select efficient datasets for preserving model rankings.
problem Efficient evaluation of machine learning models on small, representative datasets.
method Bootstrap aggregation, clustering, design criteria, random baselines, and greedy farthest-first (FAFI).
result Several selection strategies improve rank preservation compared to random subsets, especially in time series classification.
We propose a data-driven framework for optimizing privacy-preserving data release mechanisms to attain the information-theoretically optimal tradeoff between minimizing distortion of useful data and concealing specific sensitive information. Our approach employs adversarially-trained neural networks to implement random…
While many solutions for privacy-preserving convex empirical risk minimization (ERM) have been developed, privacy-preserving nonconvex ERM remains a challenge. We study nonconvex ERM, which takes the form of minimizing a finite-sum of nonconvex loss functions over a training set. We propose a new differentially private…
Active learning holds promise of significantly reducing data annotation costs while maintaining reasonable model performance. However, it requires sending data to annotators for labeling. This presents a possible privacy leak when the training set includes sensitive user data. In this paper, we describe an approach for…