arXiv research

Wide neural networks can benefit from multi-task learning in their infinite-width limit.

Infinite-Task Learning uses RKHSs to learn functions over hyperparameter space.

The study analyzes transfer learning in infinite-width neural networks, improving generalization on target tasks.

Graph convolutional deep kernel machine learns representations for graph tasks.

New method transfers emotions in facial images.

We present the Infinite Latent Events Model, a nonparametric hierarchical Bayesian distribution over infinite dimensional Dynamic Bayesian Networks with binary state representations and noisy-OR-like transitions. The distribution can be used to learn structure in discrete timeseries data by simultaneously inferring a s…

Infinitely wide neural nets perform well on small datasets.

Derives PF-ODE for infinite-dimensional functions, improving function generation tasks.

Adaptive kernels from neural networks improve model performance.

Lower bounds set for infinite-precision transformers.

Multi-task learning leverages shared information among data sets to improve the learning performance of individual tasks. The paper applies this framework for data where each task is a phase-shifted periodic time series. In particular, we develop a novel Bayesian nonparametric model capturing a mixture of Gaussian proc…

GP-PCA reduces infinite-dimensional GP posteriors to a finite space for meta-learning.

This paper conditions non-linear infinite-dimensional diffusion processes.

The paper explains generalization in kernel regression and deep neural networks using spectral bias and task-model alignment.

We discuss multi-task online learning when a decision maker has to deal simultaneously with M tasks. The tasks are related, which is modeled by imposing that the M-tuple of actions taken by the decision maker needs to satisfy certain constraints. We give natural examples of such restrictions and then discuss a general …

Transformers capture combinatorial tasks with bounded error and logarithmic sample dependence.

We study the stability properties of nonlinear multi-task regression in reproducing Hilbert spaces with operator-valued kernels. Such kernels, a.k.a. multi-task kernels, are appropriate for learning prob- lems with nonscalar outputs like multi-task learning and structured out- put prediction. We show that multi-task ke…

We investigate the use of sparse coding and dictionary learning in the context of multitask and transfer learning. The central assumption of our learning method is that the tasks parameters are well approximated by sparse linear combinations of the atoms of a dictionary on a high or infinite dimensional space. This ass…

Deep neural nets estimate operators between infinite-dimensional spaces with fast rates.

The paper introduces modular neural networks for faster learning in complex tasks.

Paper tackles deep learning confounding factors, learns unseen factors.

Paper tackles infinite action linear bandits with tight regret bounds.

Proposes a new MIL formulation using infinitely many shapelets.

Develops new reinforcement learning methods for complex constrained decision-making problems.

The paper reduces sample complexity for estimating novel task parameters with few meta-learning tasks.

Cyber-physical systems, such as mobile robots, must respond adaptively to dynamic operating conditions. Effective operation of these systems requires that sensing and actuation tasks are performed in a timely manner. Additionally, execution of mission specific tasks such as imaging a room must be balanced against the n…

This paper tackles online reinforcement learning for unseen tasks with unknown boundaries.

Trace norm regularization is a popular method of multitask learning. We give excess risk bounds with explicit dependence on the number of tasks, the number of examples per task and properties of the data distribution. The bounds are independent of the dimension of the input space, which may be infinite as in the case o…

Paper improves reinforcement learning efficiency with deterministic value gradients.

Framework for multi-task learning with semiparametric models and nuisance parameters.

Paper reduces variance in infinite horizon off-policy evaluation with bias reduction.

New meta-learning method handles nonlinear tasks for faster convergence.

Model shows feature learning can improve neural scaling laws for hard tasks.

Algorithm learns optimal parameters from infinite space for computational resource optimization.

Deep neural networks perform well on local tasks but struggle with global tasks.

Proto-value networks improve deep reinforcement learning representations using auxiliary tasks.

In machine learning ensemble methods have demonstrated high accuracy for the variety of problems in different areas. Two notable ensemble methods widely used in practice are gradient boosting and random forests. In this paper we present InfiniteBoost - a novel algorithm, which combines important properties of these two…

Method distills reward and strategies from diverse demonstrators.

Analysis of deep neural networks under various learning rules reveals dynamics of feature and prediction learning.

NeuTSFlow models continuous functions behind time series forecasting.

First-order ANIL learns shared representations even with overparametrization.

This work characterizes reward function partial identifiability and its impact on policy optimization.

Study identifies specialist representations from generalist models without parametric constraints.

Establishes connection between MTDNN and multitask GP, revealing weight correlation as key to task sharing.

AGMs outperform EGMs in generalizing to unseen inference tasks.

More features and data lead to better model performance in random feature regression.

Existing Bayesian models, especially nonparametric Bayesian methods, rely on specially conceived priors to incorporate domain knowledge for discovering improved latent representations. While priors can affect posterior distributions through Bayes' rule, imposing posterior regularization is arguably more direct and in s…

Kernelized Q-learning achieves good performance with minimal data.