arXiv research

Study the cost of overfitting in noisy KRR models.

More features and data lead to better model performance in random feature regression.

Optimal classifiers derived from GMMs are approximated by deep neural networks.

New principle reveals how neural networks learn complex interactions.

We consider classifiers for high-dimensional data under the strongly spiked eigenvalue (SSE) model. We first show that high-dimensional data often have the SSE model. We consider a distance-based classifier using eigenstructures for the SSE model. We apply the noise reduction methodology to estimation of the eigenvalue…

This paper considers the problem of brain disease classification based on connectome data. A connectome is a network representation of a human brain. The typical connectome classification problem is very challenging because of the small sample size and high dimensionality of the data. We propose to use simultaneous app…

Revisits Gaussian process model with spherical harmonics for scalable deep learning.

Ensembles of random-feature models can't outperform a single large model.

Study shows that ridgeless Gaussian kernel regression overfits even with varying bandwidth or dimensionality.

New method preserves spectral clustering performance under aggressive sparsification and quantization.

The paper analyzes Nordic stock markets' correlation structures and regime shifts.

Study eigenvalues and eigenvectors in neural networks, focusing on signal propagation.

Inference of space-time varying signals on graphs emerges naturally in a plethora of network science related applications. A frequently encountered challenge pertains to reconstructing such dynamic processes, given their values over a subset of vertices and time instants. The present paper develops a graph-aware kernel…

Neural networks can learn kernel machines with a data-dependent kernel.

The smallest eigenvalues and the associated eigenvectors (i.e., eigenpairs) of a graph Laplacian matrix have been widely used for spectral clustering and community detection. However, in real-life applications the number of clusters or communities (say, $K$) is generally unknown a-priori. Consequently, the majority of …

Many important problems are characterized by the eigenvalues of a large matrix. For example, the difficulty of many optimization problems, such as those arising from the fitting of large models in statistics and machine learning, can be investigated via the spectrum of the Hessian of the empirical loss function. Networ…

The smallest eigenvalues and the associated eigenvectors (i.e., eigenpairs) of a graph Laplacian matrix have been widely used in spectral clustering and community detection. However, in real-life applications the number of clusters or communities (say, $K$) is generally unknown a-priori. Consequently, the majority of t…

We consider the problem of approximate joint triangularization of a set of noisy jointly diagonalizable real matrices. Approximate joint triangularizers are commonly used in the estimation of the joint eigenstructure of a set of matrices, with applications in signal processing, linear algebra, and tensor decomposition.…

Paper improves robust spectral clustering for noisy data.

A new model decomposes equity returns and volatilities into memory components.

Neural networks can interpolate random data but still generalize well, studied in the NT regime.

This paper reviews dMTL and methods for selecting auxiliary tasks.

APT-Gen generates tasks to help RL learn in hard problems.

OCEAN infers online task identities from context variables.

Sharing information between multiple tasks enables algorithms to achieve good generalization performance even from small amounts of training data. However, in a realistic scenario of multi-task learning not all tasks are equally related to each other, hence it could be advantageous to transfer information only between …

Proposes a new method for clustering tasks in multi-task learning.

Multi-task learning is a learning paradigm which seeks to improve the generalization performance of a learning task with the help of some other related tasks. In this paper, we propose a regularization formulation for learning the relationships between tasks in multi-task learning. This formulation can be viewed as a n…

HydaLearn dynamically adjusts task weights for better MTL performance.

New distance metric for neural architecture search reduces search space complexity.

We investigate task clustering for deep-learning based multi-task and few-shot learning in a many-task setting. We propose a new method to measure task similarities with cross-task transfer performance matrix for the deep learning scenario. Although this matrix provides us critical information regarding similarity betw…

ST-MAML tackles task ambiguity in meta-learning by encoding tasks with stochastic representations.

New algorithm combines curriculum learning with HER for complex object manipulation tasks.

Transformers infer tasks from context via two modes, geometrically shaped task vectors explain their behavior.

MTL-NAS combines NAS with GP-MTL for task-agnostic multi-task learning.

Meta-learning framework uses task similarity through nonparametric kernel regression.

Deep multi-task learning attracts much attention in recent years as it achieves good performance in many applications. Feature learning is important to deep multi-task learning for sharing common information among tasks. In this paper, we propose a Hierarchical Graph Neural Network (HGNN) to learn augmented features fo…

Novel meta-RL strategy improves efficiency in learning novel tasks.

Study shows perceptual boost of visual attention varies with task difficulty and size.

RTE enables extrapolation to new tasks by learning task transformations.

In this paper we investigate two hypothesis regarding the use of deep reinforcement learning in multiple tasks. The first hypothesis is driven by the question of whether a deep reinforcement learning algorithm, trained on two similar tasks, is able to outperform two single-task, individually trained algorithms, by more…

Multi-task learning (MTL) has recently contributed to learning better representations in service of various NLP tasks. MTL aims at improving the performance of a primary task, by jointly training on a secondary task. This paper introduces automated tasks, which exploit the sequential nature of the input data, as second…

Robot learns multiple tasks hierarchically by transferring knowledge.

Multi-task learning (MTL) has achieved success over a wide range of problems, where the goal is to improve the performance of a primary task using a set of relevant auxiliary tasks. However, when the usefulness of the auxiliary tasks w.r.t. the primary task is not known a priori, the success of MTL models depends on th…

In the paradigm of multi-task learning, mul- tiple related prediction tasks are learned jointly, sharing information across the tasks. We propose a framework for multi-task learn- ing that enables one to selectively share the information across the tasks. We assume that each task parameter vector is a linear combi- nat…

Transformers can generalize to a large task family with only a few demonstrations.

Tool manipulation is vital for facilitating robots to complete challenging task goals. It requires reasoning about the desired effect of the task and thus properly grasping and manipulating the tool to achieve the task. Task-agnostic grasping optimizes for grasp robustness while ignoring crucial task-specific constrain…

Framework adapts to new tasks based on prior knowledge.

Despite significant progress, deep reinforcement learning (RL) suffers from data-inefficiency and limited generalization. Recent efforts apply meta-learning to learn a meta-learner from a set of RL tasks such that a novel but related task could be solved quickly. Though specific in some ways, different tasks in meta-RL…