arXiv research

We extend graph neural networks to transfer performance across different input sizes.

ATLAS separates invariant and transferable latent factors across diverse environments.

Framework for transferring discount curve estimates across fixed-income product classes.

New method transfers emotions in facial images.

nGPT learns to transfer learning rates across model dimensions and token horizons.

New method upscales models and transfers hyperparameters efficiently.

Transfer learning has received a lot of attention in the machine learning community over the last years, and several effective algorithms have been developed. However, relatively little is known about their theoretical properties, especially in the setting of lifelong learning, where the goal is to transfer information…

We explain how the Transference Principles from Diophantine approximation can be interpreted in terms of geometry of the locally symmetric spaces $T_n=SO(n) \backslash SL(n,R) /SL(n,Z)$ with $n>1$, and how, via this dictionary, they become transparent geometric remarks and can be easily proved. Indeed, a finite family …

Parameters in deep neural networks which are trained on large-scale databases can generalize across multiple domains, which is referred as "transferability". Unfortunately, the transferability is usually defined as discrete states and it differs with domains and network architectures. Existing works usually heuristical…

Bayesian methods improve learning from related domains.

Transfer learning aims at transferring knowledge from a well-labeled domain to a similar but different domain with limited or no labels. Unfortunately, existing learning-based methods often involve intensive model selection and hyperparameter tuning to obtain good results. Moreover, cross-validation is not possible for…

This paper introduces a new transfer learning method for regression.

Proves Riemannian positive mass theorem with singularities.

In this work, we address the problem of musical timbre transfer, where the goal is to manipulate the timbre of a sound sample from one instrument to match another instrument while preserving other musical content, such as pitch, rhythm, and loudness. In principle, one could apply image-based style transfer techniques t…

Recently, in order to explore the mechanism behind wealth or income distribution, several models have been proposed by applying principles of statistical mechanics. These models share some characteristics, such as consisting of a group of individual agents, a pile of money and a specific trading rule. Whatever the trad…

New framework explains fast transfer of hyperparameters across model scales.

While state-of-the-art NLP explainability (XAI) methods focus on explaining per-sample decisions in supervised end or probing tasks, this is insufficient to explain and quantify model knowledge transfer during (un-)supervised training. Thus, for TX-Ray, we modify the established computer vision explainability principle…

We investigate the use of sparse coding and dictionary learning in the context of multitask and transfer learning. The central assumption of our learning method is that the tasks parameters are well approximated by sparse linear combinations of the atoms of a dictionary on a high or infinite dimensional space. This ass…

A new framework for robust transfer learning that avoids negative transfer in domains with unequal information.

The paper uses thermodynamics to improve machine learning representation quality.

We present a number of related comparison results, which allow to compare moment explosion times, moment generating functions and critical moments between rough and non-rough Heston models of stochastic volatility. All results are based on a comparison principle for certain non-linear Volterra integral equations. Our u…

A simple strategy prevents negative transfer in transfer learning.

Adversarial training enhances model transferability without sacrificing accuracy.

Though the deep learning is pushing the machine learning to a new stage, basic theories of machine learning are still limited. The principle of learning, the role of the a prior knowledge, the role of neuron bias, and the basis for choosing neural transfer function and cost function, etc., are still far from clear. In …

Framework improves target domain prediction using quantile matching.

Deep Neural Networks are vulnerable to adversarial attacks even in settings where the attacker has no direct access to the model being attacked. Such attacks usually rely on the principle of transferability, whereby an attack crafted on a surrogate model tends to transfer to the target model. We show that an ensemble o…

A new autoencoder architecture captures multiscale data.

New framework using Jensen-Shannon divergence improves domain adaptation theory.

Unsupervised domain transfer is the task of transferring or translating samples from a source distribution to a different target distribution. Current solutions unsupervised domain transfer often operate on data on which the modes of the distribution are well-matched, for instance have the same frequencies of classes b…

This paper explores how imperfect reward models can improve online RLHF.

Classifies periodic points on regular and double n-gon surfaces.

Proposes a method to improve treatment policies in data-scarce clinical settings.

New model predicts radiative properties of nanoparticle layers with high accuracy and uncertainty.

A Systemic Optimal Risk Transfer Equilibrium (SORTE) was introduced in: "Systemic optimal risk transfer equilibrium", Mathematics and Financial Economics (2021), for the analysis of the equilibrium among financial institutions or in insurance-reinsurance markets. A SORTE conjugates the classical Bühlmann's notion of a …

Study controlled contagion with state-dependent killing, proving a comparison principle.

Unified approach to domain generalization by aligning gradients and Hessians.

Two Anosov metrics with same boundary distance are isometric.

In multi-task learning, a learner is given a collection of prediction tasks and needs to solve all of them. In contrast to previous work, which required that annotated training data is available for all tasks, we consider a new setting, in which for some tasks, potentially most of them, only unlabeled training data is …

Proposes a Gaussian process model for constrained dynamics learning.

ShotgunCSP predicts crystal structures using machine learning, achieving high accuracy with minimal computation.

Model improves information transfer from visual streams.

UCB-TQL learns from multiple tasks with shared dynamics and adapts to task-specific variations.

This work investigates how neural collapse improves transfer learning for large-scale models.

Smooth structures on diffeological spaces and sheaves, resolving conjectures.

Compared with shallow domain adaptation, recent progress in deep domain adaptation has shown that it can achieve higher predictive performance and stronger capacity to tackle structural data (e.g., image and sequential data). The underlying idea of deep domain adaptation is to bridge the gap between source and target d…

Kichenassamy's work spans theoretical physics, from relativity to applications.

One-step Bellman alignment improves online RL by reducing task mismatch.

Paper tackles efficient IRL in offline settings with polynomial samples and runtime.