A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
New findings on representation learning beyond linear functions.
A novel gradient-based method optimizes decision trees for complex tasks.
Gradient-based meta-RL fails with incorrect task distributions, leading to instability and poor performance.
New algorithm estimates task affinities without repeated training, improving model performance and efficiency.
Gradient-based algorithm improves model performance with triage.
Structured Prediction Energy Networks (SPENs) are a simple, yet expressive family of structured prediction models (Belanger and McCallum, 2016). An energy function over candidate structured outputs is given by a deep network, and predictions are formed by gradient-based optimization. This paper presents end-to-end lear…
Proposes efficient model for continual learning that grows model over task-specific parameters.
Neural architecture search has been shown to hold great promise towards the automation of deep learning. However in spite of its potential, neural architecture search remains quite costly. To this point, we propose a novel gradient-based framework for efficient architecture search by sharing information across several …
DSelect-k improves MoE models for multi-task learning with better performance and smoother training.
Gradient-based meta-learning methods leverage gradient descent to learn the commonalities among various tasks. While previous such methods have been successful in meta-learning tasks, they resort to simple gradient descent during meta-testing. Our primary contribution is the {\em MT-net}, which enables the meta-learner…
Paper analyzes dataset distillation for efficient encoding of task-relevant information.
Randomized gradient-based ensemble improves prediction accuracy.
We build a theoretical framework for designing and understanding practical meta-learning methods that integrates sophisticated formalizations of task-similarity with the extensive literature on online convex optimization and sequential prediction algorithms. Our approach enables the task-similarity to be learned adapti…
LEM efficiently models long-term sequences with gradients.
In structured output prediction tasks, labeling ground-truth training output is often expensive. However, for many tasks, even when the true output is unknown, we can evaluate predictions using a scalar reward function, which may be easily assembled from human knowledge or non-differentiable pipelines. But searching th…
Real-world applications often involve domain-specific and task-based performance objectives that are not captured by the standard machine learning losses, but are critical for decision making. A key challenge for direct integration of more meaningful domain and task-based evaluation criteria into an end-to-end gradient…
In this work we study generalization of neural networks in gradient-based meta-learning by analyzing various properties of the objective landscapes. We experimentally demonstrate that as meta-training progresses, the meta-test solutions, obtained after adapting the meta-train solution of the model, to new tasks via few…
NNEinFact fits any nonnegative tensor factorization quickly and accurately.
We introduce Recurrent Predictive State Policy (RPSP) networks, a recurrent architecture that brings insights from predictive state representations to reinforcement learning in partially observable environments. Predictive state policy networks consist of a recursive filter, which keeps track of a belief about the stat…
We consider the task of few shot link prediction on graphs. The goal is to learn from a distribution over graphs so that a model is able to quickly infer missing edges in a new graph after a small amount of training. We show that current link prediction methods are generally ill-equipped to handle this task. They canno…
A new framework predicts links in time-dependent networks using Bernoulli autoregression.
Gradient-based methods improve understanding of deep learning survival models.
Cloak identifies essential features to preserve prediction privacy without provider collaboration.
We study the problem of meta-learning through the lens of online convex optimization, developing a meta-algorithm bridging the gap between popular gradient-based meta-learning and classical regularization-based multi-task transfer methods. Our method is the first to simultaneously satisfy good sample efficiency guarant…
This paper explores evasion attacks against Bayesian models.
Recently, a technique called Layer-wise Relevance Propagation (LRP) was shown to deliver insightful explanations in the form of input space relevances for understanding feed-forward neural network classification decisions. In the present work, we extend the usage of LRP to recurrent neural networks. We propose a specif…
Gradient-based method prunes large models to create transferable architectures.
Adaptive model learns from time series data with changing distributions.
WassersteinGrad improves weather forecasting explanations by addressing geometric misalignment issues.
New SMC method for pBNNs improves scalability and predictive performance.
SAP learns efficient task-specific parameter subspaces for few-shot learning.
Catastrophic forgetting is a problem faced by many machine learning models and algorithms. When trained on one task, then trained on a second task, many machine learning models "forget" how to perform the first task. This is widely believed to be a serious problem for neural networks. Here, we investigate the extent to…
Proposes IGN for scalable Gaussian process networks.
GBML with deep nets converges globally and generalizes well.
Deep Convolutional Neural Networks (DCNN) has shown excellent performance in a variety of machine learning tasks. This manuscript presents Deep Convolutional Neural Fields (DeepCNF), a combination of DCNN with Conditional Random Field (CRF), for sequence labeling with highly imbalanced label distribution. The widely-us…
Combines gradient-based and competitive learning for unsupervised feature extraction.
TTT improves model adaptation to test data, especially for nonlinear models.
Causal structure learning has been a challenging task in the past decades and several mainstream approaches such as constraint- and score-based methods have been studied with theoretical guarantees. Recently, a new approach has transformed the combinatorial structure learning problem into a continuous one and then solv…
Variational inference transforms posterior inference into parametric optimization thereby enabling the use of latent variable models where otherwise impractical. However, variational inference can be finicky when different variational parameters control variables that are strongly correlated under the model. Traditiona…
Proposes a method to quantify and explain deep learning model uncertainties.
Method generates visual explanations for similarity models without classification.
GIT uses gradient estimators to target interventions for causal discovery.
Meta learning works well with overparameterized models, a phenomenon called 'benign overfitting'.
The utility of learning a dynamics/world model of the environment in reinforcement learning has been shown in a many ways. When using neural networks, however, these models suffer catastrophic forgetting when learned in a lifelong or continual fashion. Current solutions to the continual learning problem require experie…
Optimal selection of a subset of items from a given set is a hard problem that requires combinatorial optimization. In this paper, we propose a subset selection algorithm that is trainable with gradient-based methods yet achieves near-optimal performance via submodular optimization. We focus on the task of identifying …
Extends gradient-based optimization to spline functions.
Inferring the most likely configuration for a subset of variables of a joint distribution given the remaining ones - which we refer to as co-generation - is an important challenge that is computationally demanding for all but the simplest settings. This task has received a considerable amount of attention, particularly…
The paper tackles performative policy learning with strategic agents, improving scalability and generalizability.