arXiv research

FAST improves fast and stable task adaptation in DNNs.

Sparse Meta Networks adapt deep neural networks incrementally for fast learning.

Gradient-based meta-learning has proven to be highly effective at learning model initializations, representations, and update rules that allow fast adaptation from a few samples. The core idea behind these approaches is to use fast adaptation and generalization -- two second-order metrics -- as training signals on a me…

FSNet improves online time series forecasting by balancing fast adaptation and old knowledge.

Efficient CF approach using fast adaptive PCA for recommender systems.

FAWMF adapts weights for implicit feedback recommendation efficiently.

A concise review of recent few-shot meta-learning methods.

This paper proposes a new method for estimating sparse precision matrices in the high dimensional setting. It has been popular to study fast computation and adaptive procedures for this problem. We propose a novel approach, called Sparse Column-wise Inverse Operator, to address these two issues. We analyze an adaptive …

Model-Agnostic Meta-Learning (MAML) and its variants have achieved success in meta-learning tasks on many datasets and settings. On the other hand, we have just started to understand and analyze how they are able to adapt fast to new tasks. For example, one popular hypothesis is that the algorithms learn good represent…

Paper presents a fast and adaptive filter for SI suppression in full-duplex transceivers.

M-L2O adapts fast to new tasks by self-adapting during test-time.

A fast method estimates group-adaptive elastic net penalties using co-data.

Fast algorithms developed for adaptive and fully adaptive submodular maximization problems.

Error bound conditions (EBC) are properties that characterize the growth of an objective function when a point is moved away from the optimal set. They have recently received increasing attention in the field of optimization for developing optimization algorithms with fast convergence. However, the studies of EBC in st…

MetaVRF learns adaptive kernels for fast few-shot learning.

Adaptive gradient methods, which adopt historical gradient information to automatically adjust the learning rate, despite the nice property of fast convergence, have been observed to generalize worse than stochastic gradient descent (SGD) with momentum in training deep neural networks. This leaves how to close the gene…

In this work, we are concerned with the spherical quasiconformal parameterization of genus-0 closed surfaces. Given a genus-0 closed triangulated surface and an arbitrary user-defined quasiconformal distortion, we propose a fast algorithm for computing a spherical parameterization of the surface that satisfies the pres…

Novel Bayesian meta-reinforcement learning framework improves traffic signal control robustness.

Deep neural networks have excelled on a wide range of problems, from vision to language and game playing. Neural networks very gradually incorporate information into weights as they process data, requiring very low learning rates. If the training distribution shifts, the network is slow to adapt, and when it does adapt…

New algorithm reduces best-in-class regret in contextual bandits.

In this paper we describe a new algorithm called Fast Adaptive Sequencing Technique (FAST) for maximizing a monotone submodular function under a cardinality constraint $k$ whose approximation ratio is arbitrarily close to $1-1/e$, is $O(\log(n) \log^2(\log k))$ adaptive, and uses a total of $O(n \log\log(k))$ queries. …

AdaX improves Adam by exponentially accumulating past gradients, leading to better performance in machine learning tasks.

A new method improves EEG classification across subjects efficiently.

In this letter, we propose an algorithm for recovery of sparse and low rank components of matrices using an iterative method with adaptive thresholding. In each iteration, the low rank and sparse components are obtained using a thresholding operator. This algorithm is fast and can be implemented easily. We compare it w…

Adaptive workflow combines fast amortized inference with MCMC for many datasets.

The high-order relations between the content in social media sharing platforms are frequently modeled by a hypergraph. Either hypergraph Laplacian matrix or the adjacency matrix is a big matrix. Randomized algorithms are used for low-rank factorizations in order to approximately decompose and eventually invert such big…

New method learns robust meta-representations for fast task adaptation.

New guarantees for ERM with adaptively collected data.

Learning representation from relative similarity comparisons, often called ordinal embedding, gains rising attention in recent years. Most of the existing methods are based on semi-definite programming (\textit{SDP}), which is generally time-consuming and degrades the scalability, especially confronting large-scale dat…

Linearized neural networks provide a fast and interpretable way to adapt models to new settings.

CoDA adapts dynamics models to new physical systems by conditioning on context.

This paper addresses classification tasks on a particular target domain in which labeled training data are only available from source domains different from (but related to) the target. Two closely related frameworks, domain adaptation and domain generalization, are concerned with such tasks, where the only difference …

In this paper, we consider the problem of fast and efficient indexing techniques for sequences evolving in non-Euclidean spaces. This problem has several applications in the areas of human activity analysis, where there is a need to perform fast search, and recognition in very high dimensional spaces. The problem is ma…

Improved RL algorithm stabilizes unknown linear systems with polynomial regret.

Classification algorithms to mine data stream have been extensively studied in recent years. However, a lot of these algorithms are designed for supervised learning which requires labeled instances. Nevertheless, the labeling of the data is costly and time-consuming. Because of this, alternative learning paradigms have…

Meta-learning has emerged as an important framework for learning new tasks from just a few examples. The success of any meta-learning model depends on (i) its fast adaptation to new tasks, as well as (ii) having a shared representation across similar tasks. Here we extend the model-agnostic meta-learning (MAML) framewo…

Adapts DPMM for fast streaming data clustering.

Stochastic methods with coordinate-wise adaptive stepsize (such as RMSprop and Adam) have been widely used in training deep neural networks. Despite their fast convergence, they can generalize worse than stochastic gradient descent. In this paper, by revisiting the design of Adagrad, we propose to split the network par…

In most adaptive signal processing applications, system linearity is assumed and adaptive linear filters are thus used. The traditional class of supervised adaptive filters rely on error-correction learning for their adaptive capability. The kernel method is a powerful nonparametric modeling tool for pattern analysis a…

We consider first order gradient methods for effectively optimizing a composite objective in the form of a sum of smooth and, potentially, non-smooth functions. We present accelerated and adaptive gradient methods, called FLAG and FLARE, which can offer the best of both worlds. They can achieve the optimal convergence …

PDTS improves robustness in sequential decision-making.

AdaBelief optimizes deep learning models with faster convergence and better stability.

We present an approach towards convex optimization that relies on a novel scheme which converts online adaptive algorithms into offline methods. In the offline optimization setting, our derived methods are shown to obtain favourable adaptive guarantees which depend on the harmonic sum of the queried gradients. We furth…

Flexible empirical Bayes for large-scale multiple linear regression.

Meta-learning bounds derived using PAC-Bayes theory for improved generalization.

New approach shows AI can adapt like toddlers by correcting old knowledge.

During recent years there has been an increased interest in stochastic adaptations of limited memory quasi-Newton methods, which compared to pure gradient-based routines can improve the convergence by incorporating second order information. In this work we propose a direct least-squares approach conceptually similar to…

New approach to meta-learning with variational Bayes for unlabeled data.