Multitask learning algorithms are typically designed assuming some fixed, a priori known latent structure shared by all the tasks. However, it is usually unclear what type of latent task structure is the most appropriate for a given multitask learning problem. Ideally, the "right" latent task structure should be learne…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “task structures”
MT-HAL learns features and task associations for multiple tasks with a shared sparse structure.
problem Learning features and task associations for multiple tasks with shared structure.
method Fully nonparametric approach that learns features, samples, and task associations with a shared sparse structure.
result MT-HAL achieves a powerful convergence rate and outperforms other methods across various simulation settings.
Multi-task learning (MTL) aims to improve generalization performance by learning multiple related tasks simultaneously. While sometimes the underlying task relationship structure is known, often the structure needs to be estimated from data at hand. In this paper, we present a novel family of models for MTL, applicable…
GraphToken encodes structured data for LLMs, improving graph reasoning tasks.
problem Efficiently encoding structured data for large language models.
method GraphToken learns an encoding function to extend prompts with explicit structured information.
result Significant improvements across node, edge, and graph-level tasks on the GraphQA benchmark.
Cellular Electron Cryo-Tomography (CECT) is a powerful 3D imaging tool for studying the native structure and organization of macromolecules inside single cells. For systematic recognition and recovery of macromolecular structures captured by CECT, methods for several important tasks such as subtomogram classification a…
In order to learn quickly with few samples, meta-learning utilizes prior knowledge learned from previous tasks. However, a critical challenge in meta-learning is task uncertainty and heterogeneity, which can not be handled via globally sharing knowledge among tasks. In this paper, based on gradient-based meta-learning,…
Method learns graph structure for multi-task learning, revealing interpretable relationships.
problem Learning relationships among tasks in multi-task learning.
method Simultaneously learns graph structure and model parameters, optimizing the graph structure with the model parameters.
result Reduces generalization error and reveals interpretable sparse graph among tasks.
The one-class kernel spectral regression (OC-KSR), the regression-based formulation of the kernel null-space approach has been found to be an effective Fisher criterion-based methodology for one-class classification (OCC), achieving state-of-the-art performance in one-class classification while providing relatively hig…
Transformers can generalize to a large task family with only a few demonstrations.
problem Can learning from a small set of tasks generalize to a large task family?
method Investigating autoregressive compositional structure where each task is a composition of operations, each from a finite family of subtasks.
result Transformers can generalize to tasks with only demonstrations.
Paper tackles multi-task learning for molecular property prediction with limited data.
problem Limited labeled data for each molecular property task in drug discovery.
method Proposes SGNN-EBM method to utilize relation graph between tasks and improve multi-task learning performance.
result Empirical results show the effectiveness of SGNN-EBM.
Meta-learning performance is affected by how task diversity is allocated, not just overall variability.
problem Meta-learning performance degrades when task diversity is unevenly distributed.
method Decomposed task-specific regression effects into structurally informative and orthogonal components.
result Meta-learning prediction degrades when a larger fraction of task variability is orthogonal and non-informative.
Recently, a lot of techniques were developed to sparsify the weights of neural networks and to remove networks' structure units, e.g. neurons. We adjust the existing sparsification approaches to the gated recurrent architectures. Specifically, in addition to the sparsification of weights and neurons, we propose sparsif…
Neural networks have succeeded in many reasoning tasks. Empirically, these tasks require specialized network structures, e.g., Graph Neural Networks (GNNs) perform well on many such tasks, but less structured networks fail. Theoretically, there is limited understanding of why and when a network structure generalizes be…
This paper proposes a method to reveal task relationships in multi-task learning models using sparse graphs.
problem Understanding the underlying task relationships in multi-task learning models.
method Proposes a bilevel formulation of multi-task learning that induces sparse graphs.
result The method improves interpretability of multi-task learning models without sacrificing generalization performance.
Continual Learning is a learning paradigm where learning systems are trained with sequential or streaming tasks. Two notable directions among the recent advances in continual learning with neural networks are () variational Bayes based regularization by learning priors from previous tasks, and, () learning the s…
Multi-task learning (MTL) improves prediction performance in different contexts by learning models jointly on multiple different, but related tasks. Network data, which are a priori data with a rich relational structure, provide an important context for applying MTL. In particular, the explicit relational structure imp…
New method learns shared structures in non-linear tasks.
problem Learning shared linear representations in non-linear tasks.
method Convex optimization with structural assumptions.
result Rank and clustered estimators recover shared structures under certain conditions.
This research improves neural network representation identifiability through task structures.
problem Improving neural network representation identifiability in multi-task settings.
method Analyzing the effects of task distributions and causal structures on latent factors, leading to simpler optimization.
result A straightforward optimization procedure enables better representation recovery in both synthetic and real-world data.
Deep learning exploits latent structure to learn high-dimensional tasks.
problem Statistical intractability of high-dimensional tasks in deep learning.
method Study of locality and compositionality in data, tasks, and neural network representations.
result Neural networks improve generalization with more training examples.
The goal of optimization-based meta-learning is to find a single initialization shared across a distribution of tasks to speed up the process of learning new tasks. Conditional meta-learning seeks task-specific initialization to better capture complex task distributions and improve performance. However, many existing c…
CausalWorld benchmarks robotic manipulation tasks with causal structure for transfer learning.
problem Challenges in transferring learned skills to new robotic manipulation environments.
method Proposes a simulation-based benchmark with a combinatorial family of tasks.
result Demonstrates the feasibility of tasks in the benchmark and provides baseline results.
For many structured learning tasks, the data annotation process is complex and costly. Existing annotation schemes usually aim at acquiring completely annotated structures, under the common perception that partial structures are of low quality and could hurt the learning process. This paper questions this common percep…
Modular neural networks generalize better with less data.
problem Theoretical and practical understanding of how modularity improves neural network generalization.
method Theoretical analysis of sample complexity, development of a novel learning rule.
result Modular networks require fewer samples to generalize compared to nonmodular networks, especially in high-dimensional tasks.
Paper proposes an algorithm for lifelong learning with shared structure.
problem Lifelong learning with shared structure in an online setting.
method Proposes a simple algorithm using multi-task empirical risk minimization.
result Establishes a sample complexity bound based on task-eluder dimension.
Learning Linear Regression with Low-Rank Tasks in-Contextcond-mat.dis-nn
This work analyzes how transformers learn common linear regression tasks.
problem Understanding how in-context learning operates in real-world applications with common task structures.
method Analyzing a linear attention model trained on low-rank regression tasks.
result Statistical fluctuations in finite pre-training data induce an implicit regularization, leading to a sharp phase transition in generalization error.
Study identifies specialist representations from generalist models without parametric constraints.
problem Identify task-relevant latent representations from generalist models.
method Nonparametric, fully unsupervised approach, proving identifiability of task structure and latent representations.
result Identifiability of task structure and latent representations in a nonparametric setting.
Much attention has been devoted recently to the generalization puzzle in deep learning: large, deep networks can generalize well, but existing theories bounding generalization error are exceedingly loose, and thus cannot explain this striking performance. Furthermore, a major hope is that knowledge may transfer across …
Modeling the relationship between chemical structure and molecular activity is a key goal in drug development. Many benchmark tasks have been proposed for molecular property prediction, but these tasks are generally aimed at specific, isolated biomedical properties. In this work, we propose a new cross-modal small mole…
How can neural networks perform so well on compositional tasks even though they lack explicit compositional representations? We use a novel analysis technique called ROLE to show that recurrent neural networks perform well on such tasks by converging to solutions which implicitly represent symbolic structure. This meth…
In order to efficiently learn with small amount of data on new tasks, meta-learning transfers knowledge learned from previous tasks to the new ones. However, a critical challenge in meta-learning is the task heterogeneity which cannot be well handled by traditional globally shared meta-learning methods. In addition, cu…
We study the problem of detecting critical structures using a graph embedding model. Existing graph embedding models lack the ability to precisely detect critical structures that are specific to a task at the global scale. In this paper, we propose a novel graph embedding model, called the Ego-CNNs, that employs the eg…
Paper proposes learnable topological features for efficient phylogenetic inference.
problem Finding appropriate topological structures for phylogenetic inference tasks requires significant design effort and domain expertise.
method Combines raw node features with graph neural networks to automatically adapt to different tasks.
result Demonstrates effectiveness and efficiency on simulated and real data phylogenetic inference tasks.
RNNs solve modular addition tasks using low rank and sparse Fourier structures.
problem Solving modular addition tasks with recurrent neural networks.
method Identified low rank structures and sparse Fourier representations in RNN weights.
result RNNs robust to removing individual frequencies but degrade with more ablation.
Transfer learning for bandits with latent Lipschitz continuity.
problem Learning to transfer structural information from prior tasks to new tasks.
method Proposes a framework to estimate Lipschitz constant from prior tasks and apply it to new tasks.
result Regret bound close to oracle algorithm with full knowledge of Lipschitz constant under mild assumptions.
This work extends HiP-MDPs to robust state abstractions for multi-task and meta-reinforcement learning.
problem Limited observability of state in HiP-MDPs for real-world scenarios with rich observation spaces.
method Inspired by Block MDPs, the work extends HiP-MDPs to enable robust state abstractions for multi-task and meta-reinforcement learning.
result Transfer and generalization bounds based on task and state similarity, and sample complexity bounds that depend on the aggregate number of samples across tasks.
Convolutional networks outperform shallow classifiers on certain tasks due to hierarchical structure.
problem Understanding why convolutional networks outperform shallow classifiers on specific tasks.
method Approximation theory, visual tasks with deterministic scrambling, and network performance evaluation.
result Hierarchical structure is crucial for convolutional networks' performance on certain tasks, but not all.
New method learns graph structure and uncertainty from data.
problem Learning latent graph structures and their uncertainty from data.
method Proposes a sampling-based method to learn latent graph structure and uncertainty simultaneously.
result Proves that suitable loss functions on stochastic model outputs solve both learning latent graph structure and achieving optimal predictions.
PPI uses proxy data to improve inference from limited labels across related tasks.
problem Statistical inference with limited labels across multiple related tasks.
method Prediction-powered inference framework that uses cross-task recalibration to improve power and accuracy.
result Cross-task recalibration can substantially reduce confidence interval widths when labels are scarce.
OCEAN infers online task identities from context variables.
problem Online task inference for compositional tasks with context adaptation.
method Variational inference framework OCEAN models global and local context variables in a joint latent space.
result OCEAN provides more effective task inference with sequential context adaptation.
Algorithm finds latent structure in value functions for improved reinforcement learning.
problem Finding latent structure in value functions for efficient reinforcement learning.
method Proposes a practical algorithm using two posterior distributions over state abstractions and abstract-state values.
result Substantial performance gains in multi-task settings where tasks share a common, low-dimensional representation.
MoEs can efficiently model complex tasks with low-dimensionality and sparsity.
problem Understanding the theoretical foundations of MoEs for complex tasks.
method Systematic study of MoEs with two structural priors: low-dimensionality and sparsity.
result MoEs can approximate functions on low-dimensional manifolds and exhibit exponential structured tasks.
Graph neural networks (GNNs) are shown to be successful in modeling applications with graph structures. However, training an accurate GNN model requires a large collection of labeled data and expressive features, which might be inaccessible for some applications. To tackle this problem, we propose a pre-training framew…
CP-DRL improves curriculum reinforcement learning by leveraging causal relationships.
problem Designing effective task sequences for reinforcement learning.
method Causal-Paced Deep Reinforcement Learning (CP-DRL) that approximates SCM differences based on interaction data.
result CP-DRL outperforms existing methods on benchmarks, achieving faster convergence and higher returns.
Paper tackles high-order inference in structured prediction tasks.
problem Maximizing a score function on the space of labels in high-order Markov random fields.
method Generative model approach with two-stage convex optimization algorithm.
result Success in general high-order inference problems driven by hyperedge expansion properties.
Automated multi-task learning algorithm that optimizes network topology.
problem Over-sharing in multi-task learning leads to over-generalization and suboptimal performance.
method Tree-structured design space with gumbel-softmax sampling for differentiable network splitting.
result End-to-end trainable algorithm that optimizes network topology for multiple objectives across tasks.
Transformers learn causal structure through gradient descent on self-attention mechanisms.
problem Understanding how transformers learn causal structure during training.
method In-context learning task and simplified two-layer transformer model.
result Gradient descent on a simplified transformer learns to encode latent causal graphs.
A new method for efficient structural node embeddings using Von Neumann entropy.
problem Efficiently identifying structurally equivalent nodes in complex networks.
method VNEstruct: a simple approach generating low-dimensional structural node embeddings using Von Neumann entropy.
result VNEstruct achieves robustness on structural role identification and state-of-the-art performance on graph classification tasks.
Paper analyzes dataset distillation for efficient encoding of task-relevant information.
problem Efficiently encoding task-relevant information from gradient-based learning of non-linear tasks.
method Theoretical analysis of dataset distillation applied to two-layer neural networks with gradient-based training.
result Low-dimensional structure of the problem is efficiently encoded into distilled data, reproducing a model with high generalization ability.