Current deep learning based text classification methods are limited by their ability to achieve fast learning and generalization when the data is scarce. We address this problem by integrating a meta-learning procedure that uses the knowledge learned across many tasks as an inductive bias towards better natural languag…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “task agnostic”
Task-agnostic data augmentation shows little benefit for pretrained transformers.
problem Evaluating the effectiveness of task-agnostic data augmentation on pretrained transformers.
method Conducted a systematic examination of two data augmentation techniques (Easy Data Augmentation and Back-Translation) across 5 tasks, 6 datasets, and 3 pretrained transformer models.
result Data augmentation techniques previously effective for non-pretrained models fail to consistently improve performance for pretrained transformers, even with limited training data.
TAME learns new tasks without knowing them, outperforming existing methods.
problem Learning new tasks without task identities.
method TAME detects task shifts using loss function deviations and employs online pruning.
result TAME outperforms existing methods on continual learning benchmarks.
Task-agnostic RL tackles exploration in MDPs with multiple tasks.
problem Challenges in reinforcement learning with multiple tasks or conflicting objectives.
method Task-agnostic RL framework, UCBZero algorithm.
result UCBZero finds near-optimal policies for multiple tasks efficiently.
MTL-NAS combines NAS with GP-MTL for task-agnostic multi-task learning.
problem Designing architectures for diverse tasks with varying priors.
method Disentangled GP-MTL networks, hierarchical feature sharing, and gradient-based search.
result General-purpose model trained once can adapt to multiple tasks.
New algorithm learns disjunctions faster than previous methods.
problem Learning Boolean disjunctions in the agnostic PAC model.
method Developed an agnostic learner with complexity .
result First separation between SQ and CSQ models in distribution-free agnostic learning.
TA-VAAL improves active learning by better utilizing task structures and overall data distribution.
problem High labeling cost limits deep learning applications; active learning selects informative samples.
method Task-aware variational adversarial active learning (TA-VAAL) modifies VAAL by relaxing task loss prediction and using ranking loss information.
result TA-VAAL outperforms state-of-the-arts on various datasets, including balanced and imbalanced labels.
Study MAML's generalization in varying tasks, proving bounds on error.
problem Bounding MAML's generalization error across tasks.
method Characterizes MAML's generalization error from two perspectives: recurring and unseen tasks.
result MAML's generalization error depends on the number of tasks and samples per task.
Catastrophic forgetting is the notorious vulnerability of neural networks to the change of the data distribution while learning. This phenomenon has long been considered a major obstacle for allowing the use of learning agents in realistic continual learning settings. A large body of continual learning research assumes…
Task-agnostic data valuation without validation requirements.
problem Valuing data without specific task assumptions.
method Estimating data diversity and relevance through queries without raw data.
result Estimates capture the diversity and relevance of seller's data for the buyer.
Bayesian MAML outperforms MAML in meta learning tasks with theoretical guarantees.
problem Theoretical understanding of Bayesian MAML's superiority over MAML.
method Comparison of meta test risks between Bayesian MAML and MAML in meta linear regression.
result Bayesian MAML has provably lower meta test risks than MAML in both distribution agnostic and linear centroid cases.
Model agnostic feature importance for DNNs in NLP.
problem Characterizing DNNs as black boxes and explaining their decision-making process.
method Phrase-wise feature importance calculation for model agnostic DNNs.
result Robust and generalizable approach to feature importance calculation.
The study examines when MAML's objective has a benign landscape.
problem Understanding when MAML's objective landscape is benign.
method Analyzing the landscape of MAML objective on LQR tasks.
result The benign landscape of the MAML objective depends on task similarities.
New MAML variant prioritizes hardest tasks, improving robustness.
problem Meta-learning's focus on average performance, ignoring worst-case.
method Reformulate MAML to minimize max loss over observed tasks.
result Task-robust model performs well across various task distributions.
New method handles unknown task boundaries in continual learning.
problem Catastrophic forgetting in neural networks.
method Fixed-point equations for online variational Bayes optimization.
result Approximates online Bayes update for non-stationary data.
While model-based deep reinforcement learning (RL) holds great promise for sample efficiency and generalization, learning an accurate dynamics model is often challenging and requires substantial interaction with the environment. A wide variety of domains have dynamics that share common foundations like the laws of clas…
Study the cost of overfitting in noisy KRR models.
problem Cost of overfitting in noisy kernel ridge regression.
method An agnostic view of overfitting cost as a function of sample size for any target function, using Gaussian universality ansatz and task eigenstructure.
result Characterization of benign, tempered, and catastrophic overfitting.
MetaPerturb learns to improve generalization across different tasks and architectures.
problem Improving generalization on unseen data for diverse tasks and architectures.
method MetaPerturb is a meta-learned set-based perturbation function that improves generalization performance across heterogeneous tasks and architectures.
result MetaPerturb significantly outperforms baselines on most tasks and architectures with minimal increase in parameter size and no hyperparameters to tune.
The composition of elementary behaviors to solve challenging transfer learning problems is one of the key elements in building intelligent machines. To date, there has been plenty of work on learning task-specific policies or skills but almost no focus on composing necessary, task-agnostic skills to find a solution to …
Adaptive multi-domain learning reduces parameter count for efficient deep learning.
problem Different domains have varying complexity, leading to inefficient model training.
method Proposes adaptive parameterization to reduce model complexity without sacrificing performance.
result Efficient multi-domain learning solutions with far fewer parameters.
First-order ANIL learns shared representations even with overparametrization.
problem Lack of theoretical evidence for model-agnostic meta-learning (ANIL) learning shared representations.
method First-order ANIL with a linear two-layer network architecture, showing asymptotically low-rank solutions with overparametrization.
result First-order ANIL learns linear shared representations, even with overparametrization, and performs well in adaptation.
DAC-SSM learns domain-agnostic states for better imitation learning.
problem Domain shifts hinder imitation learning in partially observable tasks.
method DAC-SSM uses adversarial training to remove domain-dependent information from states.
result DAC-SSM achieves comparable performance to experts in sparse reward tasks.
New SQ lower bound shows complexity nearly matches known upper bound for smoothed agnostic learning.
problem Smoothed agnostic learning of halfspaces under subgaussian distributions.
method Statistical Query (SQ) lower bound using moment-matching hard distribution and linear programming duality.
result First non-trivial lower bound on complexity nearly matches known upper bound.
Perception Learning: A Formal Separation of Sensory Representation Learning from Decision Learningcs.LG
PeL separates sensory interface optimization from decision learning.
problem Optimizing sensory interfaces without task-specific information.
method Formal separation of perception and decision learning, using metrics for stability, informativeness, and geometry.
result Updates preserving invariants are orthogonal to decision gradients.
Meta-learning has emerged as an important framework for learning new tasks from just a few examples. The success of any meta-learning model depends on (i) its fast adaptation to new tasks, as well as (ii) having a shared representation across similar tasks. Here we extend the model-agnostic meta-learning (MAML) framewo…
This paper proposes a system-agnostic policy for dynamic scheduling.
problem Dynamic scheduling in changing systems is challenging due to system-specific optimal policies.
method Descriptive policy that learns a system-agnostic scheduling principle.
result System-agnostic meta-learning enables adaptation to unseen system characteristics.
Meta-learning approaches have been proposed to tackle the few-shot learning problem.Typically, a meta-learner is trained on a variety of tasks in the hopes of being generalizable to new tasks. However, the generalizability on new tasks of a meta-learner could be fragile when it is over-trained on existing tasks during …
This paper tackles online reinforcement learning for unseen tasks with unknown boundaries.
problem Real-world tasks violate assumptions of task distributions, independence, and clear task delineations.
method A mixture of Gaussian Processes models different dynamics, and a transition prior handles temporal dependencies.
result The approach reliably handles task distribution shifts and outperforms alternatives in non-stationary tasks.
New method prevents forgetting in learning new tasks.
problem Learning new tasks without forgetting old knowledge.
method Hierarchical information-theoretic optimality principle and Mixture-of-Variational-Experts layer.
result Our method can operate task-agnostically and improve performance in various learning problems.
Novel framework for ML-assisted inference valid for any statistical task.
problem Limited validity of existing methods for post-prediction inference.
method Introduces PSPS framework for task-agnostic ML-assisted inference.
result Valid and efficient inference for arbitrary ML models.
Query access significantly speeds up learning Multi-Index Models under Gaussian distribution.
problem Agnostically learning Multi-Index Models (MIMs) under Gaussian distribution.
method Query access for MIMs with complexity under standard regularity assumptions.
result Query access gives significant runtime improvements over random examples for agnostically learning MIMs.
This study optimizes offline reinforcement learning methods for various tasks without rewards.
problem Optimizing offline reinforcement learning for multiple tasks without rewards.
method Designing a new model-based approach with singleton absorbing MDPs to achieve optimal convergence rates.
result Achieved optimal convergence rates for offline reinforcement learning in various settings.
Model-agnostic meta-learners aim to acquire meta-learned parameters from similar tasks to adapt to novel tasks from the same distribution with few gradient updates. With the flexibility in the choice of models, those frameworks demonstrate appealing performance on a variety of domains such as few-shot image classificat…
The paper analyzes MAML's representation using RSA, revealing that feature reuse is not the primary reason for its success.
problem Understanding why model-agnostic meta-learning (MAML) works well in few-shot learning tasks.
method Representation similarity analysis (RSA) applied to MAML's few-shot learning instantiation.
result Feature reuse is not the primary reason for MAML's success; instead, it is the learning task itself that increases representation similarity.
Meta-learning for few-shot learning entails acquiring a prior over previous tasks and experiences, such that new tasks be learned from small amounts of data. However, a critical challenge in few-shot learning is task ambiguity: even when a powerful prior can be meta-learned from a large number of prior tasks, a small d…
While neural networks are powerful function approximators, they suffer from catastrophic forgetting when the data distribution is not stationary. One particular formalism that studies learning under non-stationary distribution is provided by continual learning, where the non-stationarity is imposed by a sequence of dis…
A new dataset distance using optimal transport, agnostic of model and label sets.
problem Quantifying task similarity across datasets without model dependence.
method Optimal transport for model-agnostic dataset comparison.
result The new distance correlates with transfer learning difficulty across various datasets.
Model-agnostic meta-learning (MAML) is a meta-learning technique to train a model on a multitude of learning tasks in a way that primes the model for few-shot learning of new tasks. The MAML algorithm performs well on few-shot learning problems in classification, regression, and fine-tuning of policy gradients in reinf…
Learning to infer Bayesian posterior from a few-shot dataset is an important step towards robust meta-learning due to the model uncertainty inherent in the problem. In this paper, we propose a novel Bayesian model-agnostic meta-learning method. The proposed method combines scalable gradient-based meta-learning with non…
This paper tackles continuous domain adaptation with a new approach.
problem Learning in non-stationary environments, especially domain drift.
method Variational domain-agnostic feature replay, composed of inference, generative, and solver modules.
result Demonstrates the effectiveness of the proposed approach for practical usage.
Meta-learning approach prevents forgetting across tasks.
problem Catastrophic forgetting in deep neural networks.
method Incremental Task-Agnostic Meta-learning (iTAML) with a meta-update rule.
result Significant improvements in class-incremental learning tasks.
Proposes PIC and POIC for measuring task difficulty in RL.
problem Lack of metrics to measure task difficulty in RL.
method Introduces policy information capacity (PIC) and policy-optimal information capacity (POIC) as metrics based on mutual information.
result Empirically shows PIC and POIC correlate with task solvability better than alternatives.
New method learns robot actions from videos without explicit labels.
problem Training robots to perform tasks from few demonstrations.
method Uses images and text for task-agnostic and general representation, synthesizes hallucinated actions, and applies dense correspondences.
result Trains robot policies solely from RGB videos, achieving diverse tasks across different robots and environments.
Proposes a novel model-agnostic training procedure for anomaly detection incorporating known anomalies.
problem Challenges of anomaly detection, especially when only a few anomalous samples are available.
method Reformulates one-class classification as a binary classification problem, using pseudo-anomalous samples drawn from a normalizing flow model.
result Demonstrates comparable or superior performance on tasks with variable amounts of known anomalies.
Continuous word representation (aka word embedding) is a basic building block in many neural network-based models used in natural language processing tasks. Although it is widely accepted that words with similar semantics should be close to each other in the embedding space, we find that word embeddings learned in seve…
New insights into RL with weak function approximation.
problem Statistical complexity of RL with function approximation in large state spaces.
method Agnostic policy learning framework, exploring environment access, coverage, and representational conditions.
result Characterization of fundamental performance bounds and statistical separations.
Study agnostic RL in large state spaces with weak function approximation.
problem Statistical intractability of agnostic policy learning in various environments.
method Investigates agnostic policy learning with different forms of environment access.
result Agnostic policy learning remains statistically intractable with certain forms of environment access.
Proposes a multilingual email segmentation benchmark and model.
problem Lack of multilingual email zoning corpora and models.
method Analysis of existing corpora, development of multilingual benchmark, introduction of OKAPI model.
result OKAPI model achieves state-of-the-art performance in English and generalizes well to unseen languages.