AdaTrans adapts to feature and sample transfer in high-dimensional regression.
problem High-dimensional linear regression with more features than samples.
method F-AdaTrans and S-AdaTrans methods using fused-penalties and adaptive weights.
result AdaTrans achieves convergence rates close to oracle estimators and near-minimax optimal rates.
New approach combines multi-view learning for improved convergence in knowledge transfer.
problem Improving convergence in knowledge transfer settings like learning with privileged information and distillation.
method Adopting a multi-view approach under reasonable assumptions about hypothesis spaces, encouraging agreement between teacher and student.
result Improved convergence rate achieved with regularized empirical risk minimization.
The paper improves high-dimensional linear regression prediction and estimation using auxiliary samples.
problem Estimating and predicting high-dimensional linear regression models with auxiliary samples.
method Proposes Trans-Lasso for data-driven transfer learning, establishing optimality for prediction and estimation.
result Knowledge from auxiliary samples can improve learning performance in target problems.
DARec adapts rating patterns across domains without auxillary info.
problem Cross-domain recommendation challenges.
method Deep domain adaptation model (DARec) that transfers rating patterns.
result Best performance on public datasets.
Paper proposes a transfer learning method for improving matrix completion.
problem Improving estimation of a low-rank target matrix using auxiliary data.
method Transfer learning procedure leveraging prior information on favorable source datasets.
result Method outperforms traditional methods when source datasets are close to the target matrix.
The paper uses thermodynamics to improve machine learning representation quality.
problem Improving the quality of learned representations for transfer learning.
method Formal connection with thermodynamics, iso-classification process, traversing the equilibrium surface.
result Demonstrates how to transfer representations while keeping classification loss constant.
Improves black-box adversarial attacks with a transfer-based prior.
problem Low attack success rates and poor query efficiency in black-box adversarial attacks.
method P-RGF method that integrates a transfer-based prior and query information.
result Significantly reduces the number of queries needed for successful attacks.
New study reveals surprising adaptive rates in model selection for transfer learning.
problem Model selection in transfer learning with varying transfer distances.
method Theoretical analysis focusing on classification.
result Adaptive rates can be arbitrarily slower than oracle rates.
Optimizes SSVEP-based BCI performance by automating threshold selection.
problem Improving classification accuracy of SSVEP-based brain-computer interfaces.
method Formalizes ITR maximization, derives a general formula, and automates threshold selection.
result Achieved ITR of 62 bit/min, outperforming previous methods by a factor of 2.
Improves functional linear regression with shape transfer learning.
problem Data scarcity in functional linear models.
method Shape-based transfer learning from auxiliary to target domains.
result Enhances robustness and generalizability of functional linear models.
The paper characterizes the efficiency of transferring knowledge from a teacher to a student classifier over finite domains.
problem Characterizing the statistical efficiency of knowledge transfer over finite domains.
method Three progressive levels of privileged information: hard labels, teacher probabilities, and soft labels. Novel empirical loss functions used to achieve the fundamental limits.
result Achieving the fundamental limits of knowledge transfer through specific levels of privileged information and novel loss functions.
Transfer learning for bandits with latent Lipschitz continuity.
problem Learning to transfer structural information from prior tasks to new tasks.
method Proposes a framework to estimate Lipschitz constant from prior tasks and apply it to new tasks.
result Regret bound close to oracle algorithm with full knowledge of Lipschitz constant under mild assumptions.
A simple strategy prevents negative transfer in transfer learning.
problem Negative transfer in transfer learning where source representations harm target performance.
method Residual feature integration with a trainable target-side encoder.
result The method provably prevents negative transfer with theoretical guarantees.
Adaptive transfer learning model for varying mechanisms across domains.
problem Improving inference in a target domain by leveraging related source domains with varying mechanisms.
method Semi-parametric domain-varying coefficient model (DVCM) for structured transfer learning.
result Minimax rate-optimal adaptive transfer learning estimator with provable negative transfer safeguards.
Paper introduces a novel measure to analyze excess error in classification under covariate shift.
problem Analyzing excess error in classification under covariate shift.
method Utilizes vicinity information to characterize excess error.
result Faster or competitive convergence rates compared to previous techniques.
This paper introduces a new member of the family of Variational Autoencoders (VAE) that constrains the rate of information transferred by the latent layer. The latent layer is interpreted as a communication channel, the information rate of which is bound by imposing a pre-set signal-to-noise ratio. The new constraint s…
We derive expressions for the predicitive information rate (PIR) for the class of autoregressive Gaussian processes AR(N), both in terms of the prediction coefficients and in terms of the power spectral density. The latter result suggests a duality between the PIR and the multi-information rate for processes with mutua…
nGPT learns to transfer learning rates across model dimensions and token horizons.
problem nGPT does not transfer learning rates across model size and token horizon.
method Combining numerical experiments with alignment exponents, a novel nGPT parameterization νGPT is developed.
result νGPT exhibits learning rate transfer across width, depth, and token horizon.
Trans-GLMC tackles source heterogeneity in transfer learning for structured clusters.
problem Source heterogeneity makes it hard to use multiple related auxiliary sources effectively.
method Trans-GLMC constructs clusters of sources, then combines global fusion, within-cluster refinement, and target debiasing.
result Improves facility-specific prediction and identifies interpretable communities of hospitals with mutual transferability.
Proof of learning rate transfer in MLPs with μ μ μ P parameterization.
problem Understanding and optimizing learning rates in neural networks with different parameterizations.
method Theoretical analysis and empirical validation of learning rate transfer in MLPs with μ μ μ P, SP, and NTP parameterizations. result The optimal learning rate converges to a non-zero constant as width goes to infinity under μ μ μ P, explaining learning rate transfer. Localized transfer learning improves nonparametric regression performance.
problem Improving nonparametric regression performance on target tasks.
method Localized transfer learning framework that models heterogeneity and partition covariate space into cells.
result Sharp minimax rates show local transfer mitigates the curse of dimensionality.
Proposes a transfer learning framework to improve U.S. election prediction models.
problem Limited spatial data and spatial dependence challenges in presidential election prediction.
method Proposes a novel transfer learning framework within the SAR model, using a two-stage algorithm with transferring and debiasing stages.
result Substantially improves prediction accuracy and outperforms traditional methods in U.S. presidential swing states.
In this paper, we quantify the statistical coherence between financial time series by means of the Renyi entropy. With the help of Campbell's coding theorem we show that the Renyi entropy selectively emphasizes only certain sectors of the underlying empirical distribution while strongly suppressing others. This accentu…
This paper develops geometric tools for causal inference using information flow concepts.
problem Developing a geometric interpretation of causal inference from probabilistic measures.
method Introducing a new measure, GeoC_{y
ightarrow x}, based on fractal correlation dimension.
result Avoids boundedness issues in transfer entropy, providing a more robust measure of causal inference.
Paper proposes AoA, a new adversarial attack method that significantly increases transferability.
problem Developing universal adversarial attacks on deep neural networks.
method AoA attacks the semantic property of attention in DNNs, enhancing transferability by altering the loss function.
result AoA generates adversarial samples that defeat 13 well-trained DNNs, achieving over 85% error rate.
Optimal transfer learning for missing not-at-random matrix completion using source data.
problem Matrix completion in a Missing Not-at-Random setting with incomplete and noisy source data.
method Active sampling of rows and columns, feature shift in latent space, minimax lower bounds, computationally efficient estimation framework.
result Achieves minimax lower bound for active sampling setting, avoiding incoherence assumptions.
AdaGCN transfers labels across networks via adversarial domain adaptation and graph convolution.
problem Cross-network node classification with limited labeled data.
method Adversarial domain adaptation and graph convolution.
result AdaGCN successfully transfers labels with low labeled data on source networks and significant domain divergence.
New method uses correlation-ratio for transfer learning, improving target model inference.
problem Difficulty in meeting similarity condition for transfer learning.
method Introduces linear correlation-ratio for building accurate relationships between models.
result Improves inference on target model by transferring information from source models.
Paper tackles federated learning with privacy, enhancing target data analysis.
problem Heterogeneity and privacy of distributed data in federated learning.
method Formulates federated differential privacy, studies statistical problems under privacy constraints.
result Federated differential privacy offers a balance between privacy and knowledge transfer.
New methods improve black-box adversarial attacks with fewer queries.
problem Efficiently craft adversarial examples without access to gradients.
method Prior-guided random gradient-free algorithms based on biased sampling and gradient averaging.
result Both methods require fewer queries and achieve higher success rates.
Enhances performance on downstream tasks using multi-domain data.
problem Improving performance on tasks with limited downstream data.
method Deep transfer learning framework leveraging shared and domain-specific features.
result Significantly improves convergence rate for learning Lipschitz functions.
Flexible framework for transfer learning with optimal rates.
problem Inference about a target population using related source data.
method Adaptive transfer learning framework allowing covariate-dependent relationships.
result Achieves minimax optimal rates of convergence by adapting to transfer relationship.
Study improves infant cry-based asphyxia diagnosis using transfer learning.
problem Improving accuracy of diagnosing perinatal asphyxia in newborns.
method Neural transfer learning from adult speech to infant cries.
result Transfer learning models are resilient to noise and signal loss.
CONCERT improves transfer learning by borrowing partial information from auxiliary datasets.
problem Inefficiency of global similarity measures in transfer learning for high-dimensional data.
method Conditional spike-and-slab prior with covariate-specific priors for robust partial information transfer.
result CONCERT achieves variable selection and information transfer simultaneously, improving performance on the target.
Transfer learning improves loan recovery rate forecasting under data scarcity.
problem Data scarcity in loan portfolios limits RR modeling accuracy.
method Introduces FT-MDN-Transformer, a mixture-density tabular Transformer architecture for TL.
result FT-MDN-Transformer outperforms baseline models in RR forecasting, especially under covariate and conditional shifts.
Bayesian method mitigates negative transfer in unknown source data.
problem Negative transfer in transfer learning where target performance worsens after source data consideration.
method Proxy-informed robust method for probabilistic transfer learning (PROMPT).
result Negative transfer can be mitigated without prior knowledge of source data.
New research on limits of transfer learning, proving key selection and dependence requirements.
problem Insufficient theoretical foundation for transfer learning.
method Proved novel results on transfer learning, emphasizing selection of information and dependence between domains.
result Upper bound on improvement possible with transfer learning, highlighting the need for careful selection.
IGMC learns inductive matrix completion without side info.
problem Inductive matrix completion without side information.
method Graph Neural Network (GNN) trained on 1-hop subgraphs of the rating matrix.
result Achieves competitive performance with state-of-the-art transductive baselines.
End-to-end analysis of SGD for STL with adaptive sub-sampling.
problem Designing SGD for STL with statistical guarantees without prior knowledge of source quality.
method Mixed-sample SGD procedure that alternates between source and target data, maintaining transfer guarantees.
result Mixed-sample SGD converges to a target-adaptive solution with 1 / T 1/\sqrt{T} 1/ T rate. DAmageNet generates universal adversarial samples with high transferability.
problem Vulnerability of deep neural networks to adversarial attacks.
method Generated 96,000 transferable adversarial samples from ImageNet.
result Adversarial samples misclassify various models with up to 90% error rate.
This paper quantifies hyperparameter transfer and finds embedding layer learning rate is key.
problem Quantifying optimal hyperparameters for large language models across scales.
method Developed three metrics to quantify hyperparameter transfer and investigated the importance of embedding layer learning rate.
result Maximal Update (μP) parameterization offers high-quality learning rate transfer compared to standard parameterization (SP).
Calculates local Granger causality for Gaussian and nonlinear systems.
problem Understanding causal influence in complex systems.
method Vector autoregression and information-theoretic approach.
result Local Granger causality offers a robust and fast method for time-directed information transfer.
Physics-informed neural networks simulate radiative transfer efficiently.
problem Simulating radiative transfer accurately and efficiently.
method Physics-informed neural networks trained to minimize radiative transfer equations.
result PINNs provide an easy-to-implement, robust, and accurate method for radiative transfer simulation.
Measures neural network information transfer for generalization.
problem Estimating the generalizable information in neural networks.
method Proposes Information Transfer ( L I T L_{IT} L I T ) based on prequential coding. result Consistently correlates with generalizable information in neural networks.
Paper studies transfer learning for nonparametric classification, establishing rates and proposing adaptive classifiers.
problem Transfer learning in nonparametric classification under different distributions.
method Established minimax rates and proposed adaptive classifiers based on weighted K-NN approach.
result Data-driven adaptive classifier achieves near-optimal rates over various parameter spaces.
JSCN improves cross-domain recommendation by learning domain-invariant user representations.
problem Cross-domain recommendation data sparsity and domain-incompatibility issues.
method JSCN uses multi-layer spectral convolutions on different graphs to learn domain-invariant user representations and domain adaptive user mappings.
result Significant improvement in cross-domain recommendation performance (9.2% recall, 36.4% MAP improvements).
Improved TTS style transfer across disjoint datasets with adversarial cycle consistency.
problem Suboptimal TTS style transfer on disjoint datasets with underrepresented styles.
method Adversarial cycle consistency training with paired and unpaired triplets.
result 78% improvement in style transfer with minimal reduction in fidelity and naturalness.
Transfer learning improves chaotic dynamics predictions with less data.
problem Efficiently predicting chaotic dynamics with limited data.
method Transfer learning for nonlinear dynamics, optimizing transfer rate and leveraging small-scale turbulence universality.
result Significantly more accurate inference of chaotic dynamics achieved.