The paper tackles learning from similar but not identical linear representations, improving performance over single-task learning.
problem Understanding how to learn from tasks with similar but not exactly the same linear representations, especially when dealing with outlier tasks.
method Proposes adaptive and robust penalized empirical risk minimization and spectral methods.
result Both methods outperform single-task learning when representations are similar and perform at least as well otherwise, with minimax optimality demonstrated.
VTAB benchmarks diverse visual tasks to assess representation learning effectiveness.
problem Lack of a unified evaluation for general visual representations.
method Developed VTAB, a benchmark for diverse visual tasks, and evaluated many representation learning algorithms.
result VTAB revealed insights into the effectiveness of various representation learning methods.
CausalCOMRL improves RL task representations by integrating causal relationships, enhancing generalizability.
problem Spurious correlations in context-based offline meta-reinforcement learning.
method CausalCOMRL integrates causal representation learning to uncover and incorporate causal relationships among task components.
result CausalCOMRL achieves better performance on meta-reinforcement learning benchmarks.
Improves shared encoder representations for better multi-task learning performance.
problem Improving quality of shared encoder representations in multi-task learning.
method Dummy Gradient norm Regularization (DGR) to decrease gradient norm of dummy task-specific predictors.
result DGR improves multi-task prediction performances and superior performance compared to existing methods.
Framework learns portable representations for diverse tasks.
problem Creating task-independent abstract representations for diverse environments.
method Autonomously learns portable representations in egocentric space.
result Portable representations enable task-independent planning and transfer.
This work improves sample efficiency in meta-learning for nonlinear tasks.
problem Learning complex tasks efficiently with limited data.
method Subspace-based representations for nonlinear tasks.
result Subspace-based representations can be learned efficiently and improve future task performance.
Study shows how deep network representations can be transferred between datasets and tasks.
problem Transferability of deep network representations across datasets and tasks.
method Examined layer-wise transferability of representations in deep networks across multiple datasets and tasks.
result Interesting empirical observations on layer-wise transferability of representations.
Paper defines and solves a problem in representation learning to ensure fairness with high confidence.
problem Learning fair representations with high confidence guarantees for all downstream tasks.
method Formally defines the problem, introduces FRG framework, proves high probability fairness, and demonstrates effectiveness empirically.
result FRG framework provides high-confidence guarantees for limiting unfairness across all downstream models and tasks.
This research improves multitask learning by creating task-specific pathways.
problem Creating task-specific representations for efficient learning.
method Developed generalization bounds for learning multiple tasks over multiple pathways.
result Proves the superiority of multipath representation over traditional shallow supernets.
ICP separates and competes feature representations to learn diverse information.
problem Learning representations with diversified information.
method Information Competing Process (ICP) separates representations into parts with different mutual information constraints, forcing them to learn independently in a competitive environment.
result ICP facilitates obtaining diversified representations with rich information.
Agents solving multi-task classification learn disentangled representations.
problem Creating interpretable world models from multi-task data.
method Optimal multi-task classification tasks, noise, evidence accumulation.
result Disentangled representations emerge in multi-task learning.
Paper introduces RAS for robust MTL with contamination.
problem Representation-based multi-task learning struggles with contamination.
method Robust and Adaptive Spectral (RAS) method.
result RAS prevents negative transfer and performs well with up to 80% contamination.
New statistical guarantees for transfer learning with diverse tasks.
problem Statistical guarantees for transfer learning across different tasks.
method Formal analysis of t + 1 t+1 t + 1 tasks with shared representation. result Sample complexity reduction for new tasks using shared representation.
Disentangled representations improve abstract visual reasoning tasks.
problem The usefulness of disentangled representations for abstract visual reasoning.
method A large-scale study with 360 state-of-the-art unsupervised disentanglement models and 3600 abstract reasoning models.
result Disentangled representations lead to better down-stream performance in abstract reasoning tasks.
This thesis argues for learning representations from multiple similar tasks for better generalization.
problem Insufficient information in a single task for good generalization.
method Introduces the concept of an environment of a learner and uses a sample from it to learn a representation.
result Learning a representation from n n n tasks reduces the number of examples required for each task by a factor of n n n . A tensor model for meta-learning adapts to task-specific features.
problem Learning shared representations for diverse tasks without task-specific observable information.
method Modeling meta-parameters as an order-3 tensor, estimating through tensor regression and method of moments.
result Tensor-based approach improves meta-learning performance with fewer samples.
This paper introduces a method to select and weight pretext tasks for better self-supervised speech representation learning.
problem Combining pretext tasks for better performance in self-supervised speech representation learning.
method Estimating calibrated weights for partial losses corresponding to pretext tasks during self-supervised training.
result Groups of selected and weighted pretext tasks perform better than classic baselines in automatic speech recognition and speaker/emotion recognition.
STAR improves equivariant and invariant representation learning by routing projection heads.
problem Redundant feature learning in equivariant and invariant representation learning.
method Soft Task-Aware Routing (STAR) for projection heads specialization.
result Lower canonical correlations between invariant and equivariant embeddings.
Disentangled representations naturally emerge in multi-task learning.
problem Finding adaptable representations for multiple tasks.
method Empirical study of neural networks trained on automatically generated supervised tasks.
result Disentanglement naturally occurs during multi-task learning.
New algorithm reduces sample complexity for multi-task bandits.
problem Optimizing representation and predictor pairs for multi-task bandits.
method OSRL-SC algorithm with sample complexity H ( G log ( 1 / δ G ) + X log ( 1 / δ H ) ) H(G\log(1/δ_G)+ X\log(1/δ_H)) H ( G log ( 1/ δ G ) + X log ( 1/ δ H )) . result OSRL-SC algorithm approaches sample complexity lower bounds.
Chemical representations derived from deep learning are emerging as a powerful tool in areas such as drug discovery and materials innovation. Currently, this methodology has three major limitations - the cost of representation generation, risk of inherited bias, and the requirement for large amounts of data. We propose…
New meta-learning method handles nonlinear tasks for faster convergence.
problem Meta-learning with nonlinear representations and tasks.
method Derive theoretical guarantees with regularization in RKHS.
result Improved rates scaling with the number of tasks.
New algorithm REFUEL shows multitask representation learning is more sample-efficient in RL.
problem Understanding the benefit of representation learning in reinforcement learning.
method Developed REFUEL algorithm for multitask low-rank RL, analyzing both upstream and downstream tasks.
result Multitask representation learning is provably more sample-efficient than individual task learning.
Study multi-task learning with low-rank representation in stochastic linear bandits.
problem Transfer learning across multiple linear bandit tasks with shared low-dimensional representation.
method Proposes a greedy policy with trace norm regularization to implicitly learn a low-rank representation without knowing the rank.
result Upper bound on multi-task regret of N d T ( T + d ) r \sqrt{NdT(T+d)r} N d T ( T + d ) r , showing benefit over independent task solving. Better data representations can simplify learning tasks by aligning model distributions with true data distributions.
problem Learning complexity influenced by the alignment of model distributions with true data distributions.
method Analyzed the effect of data representations on learning complexity using a task complexity score and information coding length.
result Better representations can simplify learning tasks by aligning model distributions with true data distributions, improving learning outcomes.
Proposes a new method to learn representations directly optimized for a task.
problem Learning representations optimized for unrelated tasks.
method Jointly learns representation and prediction function for a specific task.
result Learned representations outperform pre-trained ones and are more sample-efficient.
This paper evaluates various representations for robotics tasks, improving performance in lifting, stacking, and pushing.
problem Improving data-efficiency in reinforcement learning for robotics with limited data.
method Systematic evaluation of common representations in three robotics tasks: lifting, stacking, and pushing.
result Some representations can perform as well as simulator states as agent inputs, challenging common intuitions.
Paper proposes a new speech representation benchmark and model.
problem Lack of benchmarks for comparing speech representations.
method Unsupervised triplet-loss objective for training a universal non-semantic speech representation.
result Proposed representation outperforms other models on benchmark and transfer learning tasks.
New method reduces forgetting in neural networks by focusing on representation rather than output accuracy.
problem Catastrophic forgetting in neural networks during sequential learning.
method Developed a novel meta-learning algorithm to reduce representational forgetting, improving representations on new tasks without losing old task representations.
result Meta-learner produces weight updates that maintain old task representations while improving new task learning.
This work shows how disentangled and sparse representations improve multi-task learning.
problem Improving generalization in multi-task learning with disentangled and sparse representations.
method Proved a new identifiability result and proposed a practical approach using sparsity-promoting bi-level optimization.
result Maximally sparse base-predictors yield disentangled representations under certain conditions.
Self-supervised learning improves by predicting known information, reducing labeled data needs.
problem Efficiently learn useful semantic representations without labeled data.
method Develops a mechanism exploiting statistical connections between pretext tasks to learn representations that solve downstream tasks.
result Proves linear layer yields small approximation error and drastically reduces labeled sample complexity.
Study identifies specialist representations from generalist models without parametric constraints.
problem Identify task-relevant latent representations from generalist models.
method Nonparametric, fully unsupervised approach, proving identifiability of task structure and latent representations.
result Identifiability of task structure and latent representations in a nonparametric setting.
A method for a single policy to solve various tasks across diverse agent morphologies.
problem Generalizing a single policy to solve various tasks across diverse agent morphologies.
method Unified representation and behavior distillation using a morphology-task graph and Transformer architecture.
result Improves multi-task performances compared to baselines, suggesting a promising approach.
Representation learning improves EHR data for healthcare tasks.
problem Transforming EHR data into useful representations for machine learning.
method Deep learning and disentangling underlying factors from EHR data.
result Better representations improve machine learning performance in healthcare.
Graph Denoising Policy Network learns robust representations from noisy graphs.
problem Noise sensitivity in graph representation learning.
method Reinforcement learning to select signal neighborhoods and aggregate features.
result Significantly outperforms state-of-the-art methods on node classification tasks.
GeoERM learns shared representations on Riemannian manifolds for multi-task learning.
problem Heterogeneous and adversarial tasks in MTL.
method Geometry-aware MTL framework embedding shared representation on Riemannian manifold, optimizing via manifold operations.
result GeoERM improves estimation accuracy and stability, outperforming alternatives.
Paper introduces automatic learning of internal representations for better generalization.
problem Improving machine learning by biasing hypothesis space.
method Learning internal representation for a learning environment, then using it to bias hypothesis space for future tasks.
result Representation learning can drastically reduce the number of examples needed per task.
Method learns representations invariant to task-irrelevant details in reinforcement learning tasks.
problem Learning representations that are invariant to task-irrelevant details in reinforcement learning.
method Uses bisimulation metrics to learn robust latent representations that encode only task-relevant information.
result Demonstrates SOTA performance in modified visual MuJoCo tasks and a first-person driving task.
Math theory explains how neural networks learn abstract representations.
problem Understanding how neural networks learn abstract representations.
method Mathematical theory reformulating network optimization into mean field optimization over neural preactivations.
result Abstract representations of latent variables are guaranteed to appear in neural networks trained on tasks that depend on these variables.
Paper proposes an algorithm for lifelong learning with shared structure.
problem Lifelong learning with shared structure in an online setting.
method Proposes a simple algorithm using multi-task empirical risk minimization.
result Establishes a sample complexity bound based on task-eluder dimension.
This paper improves few-shot learning by reducing sample complexity using representation learning.
problem Reducing sample complexity for target tasks with limited data.
method Representation learning to pool all source task samples for target task learning.
result Representation learning can achieve substantial sample size reduction, bypassing the $Ω( rac{1}{T})$ barrier.
Meta-learning for bandit tasks using shared representations.
problem Learning new bandit tasks efficiently using shared low-dimensional representations.
method Proposes a greedy policy to learn new bandit tasks leveraging a partially learned low-dimensional representation.
result Upper bound on regret of proposed policy, showing efficiency of learning new tasks.
This research improves neural network representation identifiability through task structures.
problem Improving neural network representation identifiability in multi-task settings.
method Analyzing the effects of task distributions and causal structures on latent factors, leading to simpler optimization.
result A straightforward optimization procedure enables better representation recovery in both synthetic and real-world data.
Proto-value networks improve deep reinforcement learning representations using auxiliary tasks.
problem Improving deep reinforcement learning representations with auxiliary tasks.
method Derived a new family of auxiliary tasks based on the successor measure, combined with off-policy learning rule.
result Proto-value networks produce rich features comparable to established algorithms using only linear approximation and a small number of interactions.
New research shows fair data representations are impossible for different tasks.
problem Achieving fairness in machine learning models trained on various tasks.
method Analyzing the limits of fair data representations.
result No representation can guarantee fairness for different tasks trained on it.
Improved VAE representations lead to better image classification.
problem VAE representations are inferior to non-latent models for image classification.
method Used a decoder that prefers local features, improving global feature capture in latent variables.
result Significant improvement in downstream semantic classification tasks.
Graph convolutional deep kernel machine learns representations for graph tasks.
problem Limited representation learning in infinite-width neural networks.
method Developed a graph convolutional deep kernel machine as an infinite-width limit.
result Representation learning improves performance for heterophilous node classification tasks.
Pretraining method enhances dialogue representation learning across various tasks.
problem Scarce labeled data for specific dialogue tasks.
method Multi-task unsupervised pretraining with natural training objectives.
result Significant improvement in downstream tasks without encoder discrimination.