We study the problem of estimating high dimensional models with underlying sparse structures while preserving the privacy of each training example. We develop a differentially private high-dimensional sparse learning framework using the idea of knowledge transfer. More specifically, we propose to distill the knowledge …
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “private knowledge transfer”
This paper presents a method to train a public model with private data using GANs and differential privacy.
problem Privacy concerns in training deep learning models on sensitive data.
method A three-player learning framework with differential privacy protection.
result The proposed method achieves a balance between privacy and model accuracy.
GDA-HIN adapts across heterogeneous networks by aligning shared and private node types.
problem Domain adaptation challenges in heterogeneous networks with shared and private node types.
method Generalized Domain Adaptive model across HINs (GDA-HIN) that aligns identical-type nodes and edges while utilizing different-type nodes and edges.
result GDA-HIN outperforms state-of-the-art methods in various domain adaptation tasks across heterogeneous networks.
This paper tackles online strategic decision making with asymmetry and knowledge transportability.
problem Strategic decision making with information asymmetry and knowledge transportability challenges.
method Developed a sample-efficient algorithm for online learning under these conditions.
result Proved sample complexity of for learning an -optimal policy.
Reinforcement learning algorithms are known to be sample inefficient, and often performance on one task can be substantially improved by leveraging information (e.g., via pre-training) on other related tasks. In this work, we propose a technique to achieve such knowledge transfer in cases where agent trajectories conta…
A new framework for federated continual learning reduces interference and improves performance.
problem Learning from a sequence of tasks with limited data from each client.
method Federated Weighted Inter-client Transfer (FedWeIT) framework.
result FedWeIT significantly outperforms existing methods with reduced communication cost.
Proposes a transfer learning method for PCA studies.
problem Enhancing PCA estimation accuracy across multiple studies.
method Two-step algorithm integrating shared subspace information via Grassmannian barycenter.
result Knowledge transfer improves PCA estimation accuracy through enlarged eigenvalue gap.
A key factor in developing high performing machine learning models is the availability of sufficiently large datasets. This work is motivated by applications arising in Software as a Service (SaaS) companies where there exist numerous similar yet disjoint datasets from multiple client companies. To overcome the challen…
FedZKT enables resource-constrained devices to participate in federated learning with heterogeneous models.
problem Inequality in resource allocation hinders participation from resource-constrained devices in federated learning.
method Zero-shot knowledge transfer through a server-assigned distillation process.
result FedZKT effectively transfers knowledge across heterogeneous on-device models without requiring comparable local training efforts.
Purveyors of malicious network attacks continue to increase the complexity and the sophistication of their techniques, and their ability to evade detection continues to improve as well. Hence, intrusion detection systems must also evolve to meet these increasingly challenging threats. Machine learning is often used to …
This paper explores the connection between adversarial and knowledge transferability.
problem Understanding the factors affecting knowledge transferability.
method Theoretical analysis and practical metrics for adversarial transferability.
result Adversarial transferability and knowledge transferability are closely related.
On the Benefits of Public Representations for Private Transfer Learning under Distribution Shiftcs.LG
Public pretraining improves private model training even in extreme distribution shift scenarios.
problem Improving private model training accuracy in settings with large distribution shift.
method Empirical evaluation and theoretical explanation of public representations improving private training accuracy.
result Public representations can improve private training accuracy by up to 67% over private training from scratch in settings with large distribution shift.
This work transfers causal knowledge between tasks for Individual Treatment Effect estimation.
problem Estimating Individual Treatment Effects (ITE) requires a large amount of data, making it challenging.
method The authors introduce a practical framework for efficient transfer of causal knowledge between tasks, using a Causal Inference Task Affinity (CITA) measure.
result ITE knowledge transfer can significantly reduce the amount of data needed for ITE estimation.
Optimizes sparse fine-tuning for privacy in neural networks.
problem Performance gap between DP-SGD and non-private fine-tuning.
method Optimization-based approach using private gradient information for selecting trainable weights.
result Our selection method leads to better prediction accuracy compared to existing approaches.
This study investigates how much knowledge from natural images can be transferred to pathology images.
problem Quantifying how much knowledge from natural images can be transferred to pathology images.
method Proposes a framework to quantify knowledge gain by a particular layer, conducts empirical investigation in pathology image centered transfer learning.
result Early layers of deep models can transfer knowledge to pathology image classification tasks.
Collaborative (federated) learning enables multiple parties to train a model without sharing their private data, but through repeated sharing of the parameters of their local models. Despite its advantages, this approach has many known privacy and security weaknesses and performance overhead, in addition to being limit…
UDA learns target domain from unlabeled data via source knowledge transfer.
problem Learning unlabeled target domain data.
method Transferring both source knowledge and target-relatedness.
result Improves UDA performance by simultaneous knowledge transfer.
Face recognition models can be inferred from student models, posing privacy risks.
problem Privacy threats in transfer learning models for face recognition.
method Membership inference attacks and attribute inference from aggregate-level information.
result Sensitive attributes can be inferred from student models, even with limited auxiliary information.
Parameter-transfer is a well-known and versatile approach for meta-learning, with applications including few-shot learning, federated learning, and reinforcement learning. However, parameter-transfer algorithms often require sharing models that have been trained on the samples from specific tasks, thus leaving the task…
Position: Considerations for Differentially Private Learning with Large-Scale Public Pretrainingcs.LG
Publicly pretraining models on Web data may undermine differential privacy.
problem The use of large Web-scraped datasets in differential privacy models.
method Critical review of leveraging pretrained models on public datasets for differential privacy.
result Publicizing pretrained models as 'private' could harm trust and generalize poorly.
Federated learning enables the creation of a powerful centralized model without compromising data privacy of multiple participants. While successful, it does not incorporate the case where each participant independently designs its own model. Due to intellectual property concerns and heterogeneous nature of tasks and d…
Enhances generative model accuracy through knowledge transfer.
problem Improving generative model precision across different tasks.
method Introduces a novel framework for transfer learning using shared structures.
result Demonstrates enhanced performance in diffusion and normalizing flows models.
Private method measures nonlinear correlations between data hosted across two entities.
problem Measuring nonlinear correlations between sensitive data hosted across multiple parties while preserving privacy.
method Differentially private estimator of distance correlation.
result First private estimator of nonlinear correlations in a multi-party setup.
We propose the differentially private lottery ticket mechanism (DPLTM). An end-to-end differentially private training paradigm based on the lottery ticket hypothesis. Using "high-quality winners", selected via our custom score function, DPLTM significantly improves the privacy-utility trade-off over the state-of-the-ar…
A new method prevents forgetting during knowledge transfer.
problem Catastrophic forgetting in transfer learning.
method Transfer without Forgetting (TwF) using a fixed pretrained network.
result TwF outperforms other CL methods by 4.81% in Class-Incremental accuracy.
SETrLUSI combines diverse knowledge from multiple domains for faster convergence.
problem Handling diverse knowledge from multiple domains in transfer learning.
method Stochastic Ensemble Multi-Source Transfer Learning Using Statistical Invariant (SETrLUSI).
result SETrLUSI accelerates convergence and outperforms related methods.
Weight Squeezing transfers knowledge from large models to smaller ones, improving performance and speed.
problem Transfer learning and model compression for faster and more efficient training.
method Reparameterization of weights from a large model to a smaller one.
result Weight Squeezing outperforms other methods on GLUE benchmark with faster training.
Often we wish to transfer representational knowledge from one neural network to another. Examples include distilling a large network into a smaller one, transferring knowledge from one sensory modality to a second, or ensembling a collection of models into a single estimator. Knowledge distillation, the standard approa…
DERWENT learns paths for distant transfer learning via deep random walk.
problem Transfer learning between distant domains is challenging.
method DERWENT uses deep random walk to explicitly learn paths between source and target domains.
result DERWENT achieves state-of-the-art performance on benchmark datasets.
Knowledge Transfer (KT) techniques tackle the problem of transferring the knowledge from a large and complex neural network into a smaller and faster one. However, existing KT methods are tailored towards classification tasks and they cannot be used efficiently for other representation learning tasks. In this paper a n…
Information Propagation Across Investor Types: Transfer Entropy Networks in the Korean Equity Marketq-fin.ST
Investor flows in Korean equity market transmit shared information, not private signals.
problem Whether investor flows transmit private information or only public signals.
method Transfer Entropy networks constructed from investor-type flows over
umNDates{} trading days.
result Investor flows transmit shared information, not private signals.
Autonomous Vehicles(AV) are one of the brightest promises of the future which would help cut down fatalities and improve travel time while working in harmony. Autonomous vehicles will face with challenging situations and experiences not seen before. These experiences should be converted to knowledge and help the vehicl…
Paper tackles hypothesis transfer learning for black-box models.
problem Difficult to build universal machine learning models across different institutions.
method Dynamic Knowledge Distillation (dkdHTL) with instance-wise weighting.
result Empirical results show the effectiveness of dkdHTL.
In recent years, the economic policy of privatization, which is defined as the transfer of property or responsibility from public sector to private sector, is one of the global phenomenon that increases use of markets to allocate resources. One important motivation for privatization is to help develop factor and produc…
DP-RandP improves privacy-utility tradeoff in DP-SGD by learning priors from random processes.
problem Improving the performance of differentially private stochastic gradient descent (DP-SGD) on private data.
method A three-phase approach that learns priors from images generated by random processes and transfers these priors to private data.
result New state-of-the-art accuracy on CIFAR10, CIFAR100, MedMNIST, and ImageNet for various privacy budgets.
Paper tackles robust knowledge transfer in parallel RL tasks.
problem Transfer knowledge from low-tier to high-tier tasks in parallel RL without shared dynamics or reward functions.
method Identifies Optimal Value Dominance condition and proposes online learning algorithms for both tasks.
result Achieves constant regret on partial states and near-optimal regret when tasks are dissimilar.
Framework for lifelong learning using eigentasks to avoid forgetting and transfer knowledge.
problem Avoiding forgetting and transferring knowledge in lifelong learning.
method Eigentask framework: skills paired with generative models, wake-sleep cycle for learning and consolidation.
result Improved performance in supervised continual learning, evidence of forward knowledge transfer.
This paper applies secure multi-party computation to K-means clustering to protect private data.
problem Privacy-preserving K-means clustering for distributed private data.
method Secure multi-party computation (MPC) techniques to protect private data during K-means clustering.
result Privacy-preserving K-means clustering is feasible and effective for both horizontal and vertical data distribution.
Bayesian method transfers knowledge between brain tumor datasets.
problem Applying deep learning to small medical datasets.
method Generative Bayesian Prior network for knowledge transfer.
result Best results in Dice Similarity Coefficient for BRATS2018.
Taxonomy of knowledge modalities in RL for better transfer.
problem Tackling the transfer of knowledge in reinforcement learning.
method Systematic analysis of knowledge modalities in RL.
result A taxonomy of knowledge modalities in reinforcement learning.
New algorithm improves knowledge transfer in dynamic decision-making.
problem Utilizing data from existing ventures to improve decision-making in new ventures.
method Proposes Transferred Fitted -Iteration algorithm for estimating optimal action-state function .
result Significantly improved final learning error of function.
Learning to Transfercs.AI
Transfer learning borrows knowledge from a source domain to facilitate learning in a target domain. Two primary issues to be addressed in transfer learning are what and how to transfer. For a pair of domains, adopting different transfer learning algorithms results in different knowledge transferred between them. To dis…
Sharing knowledge between tasks is vital for efficient learning in a multi-task setting. However, most research so far has focused on the easier case where knowledge transfer is not harmful, i.e., where knowledge from one task cannot negatively impact the performance on another task. In contrast, we present an approach…
KG-WDRO optimizes transfer learning with external knowledge.
problem Over-pessimism in WDRO for small target samples.
method KG-WDRO incorporates multiple sources of external knowledge to construct smaller Wasserstein ambiguity sets.
result KG-WDRO improves transfer learning performance and adaptivity.
Mathematical framework for transfer learning feasibility and transfer risk.
problem Theoretical analysis of transfer learning.
method Reformulated transfer learning as an optimization problem, introduced transfer risk concept.
result Demonstrated the potential and benefits of incorporating transfer risk in transfer learning evaluation.
Transfer learning which aims at utilizing knowledge learned from one problem (source domain) to solve another different but related problem (target domain) has attracted wide research attentions. However, the current transfer learning methods are mostly uninterpretable, especially to people without ML expertise. In thi…
Paper explores how knowledge distillation transfers inductive biases between models.
problem Transferring inductive biases between models for tasks with limited data.
method Knowledge distillation applied to models with different inductive biases (LSTMs vs. Transformers, CNNs vs. MLPs).
result Effect of inductive biases is transferred through knowledge distillation, impacting both performance and solution characteristics.
Learning transferable knowledge across similar but different settings is a fundamental component of generalized intelligence. In this paper, we approach the transfer learning challenge from a causal theory perspective. Our agent is endowed with two basic yet general theories for transfer learning: (i) a task shares a c…