Multitask learning algorithms are typically designed assuming some fixed, a priori known latent structure shared by all the tasks. However, it is usually unclear what type of latent task structure is the most appropriate for a given multitask learning problem. Ideally, the "right" latent task structure should be learne…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,932 papers · 148 categories
Trend · papers per month
48 results for “structured tasks”
MT-HAL learns features and task associations for multiple tasks with a shared sparse structure.
problem Learning features and task associations for multiple tasks with shared structure.
method Fully nonparametric approach that learns features, samples, and task associations with a shared sparse structure.
result MT-HAL achieves a powerful convergence rate and outperforms other methods across various simulation settings.
Multi-task learning (MTL) aims to improve generalization performance by learning multiple related tasks simultaneously. While sometimes the underlying task relationship structure is known, often the structure needs to be estimated from data at hand. In this paper, we present a novel family of models for MTL, applicable…
GraphToken encodes structured data for LLMs, improving graph reasoning tasks.
problem Efficiently encoding structured data for large language models.
method GraphToken learns an encoding function to extend prompts with explicit structured information.
result Significant improvements across node, edge, and graph-level tasks on the GraphQA benchmark.
HSML tailors meta-learning knowledge to task clusters.
problem Handling task uncertainty and heterogeneity in meta-learning.
method Gradient-based HSML with hierarchical task clustering.
result HSML achieves state-of-the-art performance in few-shot learning.
Cellular Electron Cryo-Tomography (CECT) is a powerful 3D imaging tool for studying the native structure and organization of macromolecules inside single cells. For systematic recognition and recovery of macromolecular structures captured by CECT, methods for several important tasks such as subtomogram classification a…
Neural networks learn symbolic structure to perform compositional tasks.
problem How neural networks perform well on compositional tasks without explicit representations.
method ROLE analysis to uncover symbolic structure in recurrent neural networks.
result Neural networks converge to solutions that implicitly represent symbolic structure.
Neural networks struggle with reasoning tasks that require specialized structures.
problem Understanding why and when neural network structures generalize better.
method Developing a framework to characterize algorithmic alignment with reasoning tasks.
result Neural networks align better with dynamic programming (DP) for certain reasoning tasks.
New method improves meta-learning performance by task-specific initialization.
problem Difficulties in generalizing and achieving theoretical guarantees in conditional meta-learning.
method Structured prediction approach for task-specific initialization.
result TASML improves performance of existing meta-learning models.
Method learns graph structure for multi-task learning, revealing interpretable relationships.
problem Learning relationships among tasks in multi-task learning.
method Simultaneously learns graph structure and model parameters, optimizing the graph structure with the model parameters.
result Reduces generalization error and reveals interpretable sparse graph among tasks.
Bayesian approach adapts deep network structure for continual learning.
problem Training neural networks with sequential or streaming tasks.
method Bayesian approach to learn deep network structure for each task.
result Model performs comparably or better than recent advances in continual learning.
Extends OC-KSR for multi-task one-class classification.
problem Improving one-class classification performance with shared information.
method Linear and non-linear structure learning mechanisms for multi-task one-class classification.
result Improved performance on multiple one-class problems.
Transformers can generalize to a large task family with only a few demonstrations.
problem Can learning from a small set of tasks generalize to a large task family?
method Investigating autoregressive compositional structure where each task is a composition of operations, each from a finite family of subtasks.
result Transformers can generalize to tasks with only demonstrations.
Paper tackles multi-task learning for molecular property prediction with limited data.
problem Limited labeled data for each molecular property task in drug discovery.
method Proposes SGNN-EBM method to utilize relation graph between tasks and improve multi-task learning performance.
result Empirical results show the effectiveness of SGNN-EBM.
Meta-learning performance is affected by how task diversity is allocated, not just overall variability.
problem Meta-learning performance degrades when task diversity is unevenly distributed.
method Decomposed task-specific regression effects into structurally informative and orthogonal components.
result Meta-learning prediction degrades when a larger fraction of task variability is orthogonal and non-informative.
This paper proposes a method to reveal task relationships in multi-task learning models using sparse graphs.
problem Understanding the underlying task relationships in multi-task learning models.
method Proposes a bilevel formulation of multi-task learning that induces sparse graphs.
result The method improves interpretability of multi-task learning models without sacrificing generalization performance.
The study challenges the notion that partial data annotation is inferior, suggesting it can sometimes outperform complete annotation.
problem The inefficiency and high cost of completely annotating structured data.
method Information theoretic formulation applied to three diverse structured learning tasks.
result Learning from partial structures can sometimes outperform learning from complete ones.
We sparsify gated RNNs by simplifying their structure.
problem Improving efficiency of RNNs by reducing their complexity.
method Adjust existing sparsification techniques to gated RNNs, sparsifying preactivations of gates.
result Simplified LSTM structure improves model performance and efficiency.
Multi-task learning (MTL) improves prediction performance in different contexts by learning models jointly on multiple different, but related tasks. Network data, which are a priori data with a rich relational structure, provide an important context for applying MTL. In particular, the explicit relational structure imp…
New method learns shared structures in non-linear tasks.
problem Learning shared linear representations in non-linear tasks.
method Convex optimization with structural assumptions.
result Rank and clustered estimators recover shared structures under certain conditions.
This research improves neural network representation identifiability through task structures.
problem Improving neural network representation identifiability in multi-task settings.
method Analyzing the effects of task distributions and causal structures on latent factors, leading to simpler optimization.
result A straightforward optimization procedure enables better representation recovery in both synthetic and real-world data.
Ego-CNN detects critical structures in graphs efficiently.
problem Lack of precise detection of critical structures in existing graph embedding models.
method Ego-CNN uses ego-convolutions at each layer and stacks them in an ego-centric way.
result Ego-CNN achieves comparable task performance to state-of-the-art models and can incorporate scale-free priors.
Deep learning exploits latent structure to learn high-dimensional tasks.
problem Statistical intractability of high-dimensional tasks in deep learning.
method Study of locality and compositionality in data, tasks, and neural network representations.
result Neural networks improve generalization with more training examples.
CausalWorld benchmarks robotic manipulation tasks with causal structure for transfer learning.
problem Challenges in transferring learned skills to new robotic manipulation environments.
method Proposes a simulation-based benchmark with a combinatorial family of tasks.
result Demonstrates the feasibility of tasks in the benchmark and provides baseline results.
New task aligns molecular structure with gene expression changes.
problem Modeling the relationship between chemical structure and gene expression changes.
method Developed a cross-modal small molecule retrieval task and a coordinated deep learning approach to align chemical structure and gene expression profiles.
result Demonstrated the feasibility of the new task and highlighted the limitations of current data and systems.
Proposes GTTN for discovering all low-rank structures in deep multi-task learning.
problem Discovering all low-rank structures among tasks in deep multi-task models.
method Introduces GTTN, a convex combination of matrix trace norms of all tensor flattenings, to automatically determine the importance of components.
result Demonstrates the effectiveness of GTTN on real-world datasets.
Meta-trained LSTM Meta-Learners learn tasks faster and differently than DL and RL systems.
problem Understanding the learning dynamics of meta-trained LSTM Meta-Learners compared to traditional DL and RL systems.
method Comparative analysis of learning trajectories on three structured tasks.
result Meta-trained LSTM Meta-Learners learn all task structure concurrently, unlike sample-inefficient DL and RL systems.
Modular neural networks generalize better with less data.
problem Theoretical and practical understanding of how modularity improves neural network generalization.
method Theoretical analysis of sample complexity, development of a novel learning rule.
result Modular networks require fewer samples to generalize compared to nonmodular networks, especially in high-dimensional tasks.
Paper proposes an algorithm for lifelong learning with shared structure.
problem Lifelong learning with shared structure in an online setting.
method Proposes a simple algorithm using multi-task empirical risk minimization.
result Establishes a sample complexity bound based on task-eluder dimension.
Learning Linear Regression with Low-Rank Tasks in-Contextcond-mat.dis-nn
This work analyzes how transformers learn common linear regression tasks.
problem Understanding how in-context learning operates in real-world applications with common task structures.
method Analyzing a linear attention model trained on low-rank regression tasks.
result Statistical fluctuations in finite pre-training data induce an implicit regularization, leading to a sharp phase transition in generalization error.
Study identifies specialist representations from generalist models without parametric constraints.
problem Identify task-relevant latent representations from generalist models.
method Nonparametric, fully unsupervised approach, proving identifiability of task structure and latent representations.
result Identifiability of task structure and latent representations in a nonparametric setting.
Much attention has been devoted recently to the generalization puzzle in deep learning: large, deep networks can generalize well, but existing theories bounding generalization error are exceedingly loose, and thus cannot explain this striking performance. Furthermore, a major hope is that knowledge may transfer across …
ARML learns task relations to improve meta-learning efficiency.
problem Handling task heterogeneity in meta-learning.
method Automatically extracts cross-task relations and constructs a meta-knowledge graph.
result ARML outperforms state-of-the-art baselines in few-shot learning tasks.
A new DNN structure reduces complexity for wireless tasks.
problem Reducing complexity in training deep neural networks for wireless tasks.
method Proposes a DNN with special structure using permutation invariant a priori information.
result The proposed DNN structure reduces training complexity and model parameters.
Unified model for multiple healthcare tasks using graph structure.
problem Multiple healthcare tasks with shared graph structure.
method Graph-Convolutional Network (GCN) with multiple variational autoencoders.
result Boosted performance across all tasks, outperforming state-of-the-art.
HUBERT combines BERT's structure with TPRs to improve NLP task transfer.
problem Improving transferability among NLP tasks.
method Combines BERT's bidirectional Transformer structure with TPRs to learn shared data structures.
result HUBERT outperforms BERT on GLUE and HANS datasets, showing better transferability.
Paper proposes learnable topological features for efficient phylogenetic inference.
problem Finding appropriate topological structures for phylogenetic inference tasks requires significant design effort and domain expertise.
method Combines raw node features with graph neural networks to automatically adapt to different tasks.
result Demonstrates effectiveness and efficiency on simulated and real data phylogenetic inference tasks.
RNNs solve modular addition tasks using low rank and sparse Fourier structures.
problem Solving modular addition tasks with recurrent neural networks.
method Identified low rank structures and sparse Fourier representations in RNN weights.
result RNNs robust to removing individual frequencies but degrade with more ablation.
Transfer learning for bandits with latent Lipschitz continuity.
problem Learning to transfer structural information from prior tasks to new tasks.
method Proposes a framework to estimate Lipschitz constant from prior tasks and apply it to new tasks.
result Regret bound close to oracle algorithm with full knowledge of Lipschitz constant under mild assumptions.
This work extends HiP-MDPs to robust state abstractions for multi-task and meta-reinforcement learning.
problem Limited observability of state in HiP-MDPs for real-world scenarios with rich observation spaces.
method Inspired by Block MDPs, the work extends HiP-MDPs to enable robust state abstractions for multi-task and meta-reinforcement learning.
result Transfer and generalization bounds based on task and state similarity, and sample complexity bounds that depend on the aggregate number of samples across tasks.
Convolutional networks outperform shallow classifiers on certain tasks due to hierarchical structure.
problem Understanding why convolutional networks outperform shallow classifiers on specific tasks.
method Approximation theory, visual tasks with deterministic scrambling, and network performance evaluation.
result Hierarchical structure is crucial for convolutional networks' performance on certain tasks, but not all.
New method learns graph structure and uncertainty from data.
problem Learning latent graph structures and their uncertainty from data.
method Proposes a sampling-based method to learn latent graph structure and uncertainty simultaneously.
result Proves that suitable loss functions on stochastic model outputs solve both learning latent graph structure and achieving optimal predictions.
PPI uses proxy data to improve inference from limited labels across related tasks.
problem Statistical inference with limited labels across multiple related tasks.
method Prediction-powered inference framework that uses cross-task recalibration to improve power and accuracy.
result Cross-task recalibration can substantially reduce confidence interval widths when labels are scarce.
OCEAN infers online task identities from context variables.
problem Online task inference for compositional tasks with context adaptation.
method Variational inference framework OCEAN models global and local context variables in a joint latent space.
result OCEAN provides more effective task inference with sequential context adaptation.
Algorithm finds latent structure in value functions for improved reinforcement learning.
problem Finding latent structure in value functions for efficient reinforcement learning.
method Proposes a practical algorithm using two posterior distributions over state abstractions and abstract-state values.
result Substantial performance gains in multi-task settings where tasks share a common, low-dimensional representation.
MoEs can efficiently model complex tasks with low-dimensionality and sparsity.
problem Understanding the theoretical foundations of MoEs for complex tasks.
method Systematic study of MoEs with two structural priors: low-dimensionality and sparsity.
result MoEs can approximate functions on low-dimensional manifolds and exhibit exponential structured tasks.
This research tackles uncertainty estimation in autoregressive structured prediction tasks.
problem Ensuring safety and robustness of AI systems through accurate uncertainty estimation.
method Develops a unified probabilistic ensemble-based framework for token-level and sequence-level uncertainty estimation.
result Provides baselines for error and out-of-domain detection on translation and speech recognition datasets.
Pre-trains GNN for generic graph features, reducing data need.
problem Lack of labeled data and expressive features for GNN models.
method Synthetic graph tasks: link reconstruction, centrality ranking, cluster preservation.
result Pre-trained GNN achieves high performance with less labeled data.