HyperSTAR optimizes deep network hyperparameters for specific tasks.
problem Finding optimal hyperparameters for deep neural networks is computationally expensive and task-agnostic approaches are inefficient.
method HyperSTAR learns a dataset representation and a performance predictor from raw images, ranking and recommending hyperparameters conditioned on a joint dataset-hyperparameter space.
result HyperSTAR reduces HPO time by 50% on large-scale image classification datasets.
TA-VAAL improves active learning by better utilizing task structures and overall data distribution.
problem High labeling cost limits deep learning applications; active learning selects informative samples.
method Task-aware variational adversarial active learning (TA-VAAL) modifies VAAL by relaxing task loss prediction and using ranking loss information.
result TA-VAAL outperforms state-of-the-arts on various datasets, including balanced and imbalanced labels.
Paper introduces CoD for few-shot task-aware knowledge distillation using counterfactual explanations.
problem Lack of data for task-aware distillation in resource-constrained scenarios.
method Counterfactual-explanation-infused Distillation CoD for few-shot task-aware knowledge distillation.
result CoD achieves superior performance with significantly fewer samples than baseline methods.
STAR improves equivariant and invariant representation learning by routing projection heads.
problem Redundant feature learning in equivariant and invariant representation learning.
method Soft Task-Aware Routing (STAR) for projection heads specialization.
result Lower canonical correlations between invariant and equivariant embeddings.
LOFT separates subspace rotation and transformation for orthogonal fine-tuning.
problem Conflating subspace rotation and transformation in orthogonal fine-tuning.
method LOFT explicitly separates subspace rotation and transformation, using task-aware support selection.
result LOFT recovers principal-subspace orthogonal adaptation and improves efficiency-performance trade-off.
In recent years, neural network approaches have been widely adopted for machine learning tasks, with applications in computer vision. More recently, unsupervised generative models based on neural networks have been successfully applied to model data distributions via low-dimensional latent spaces. In this paper, we use…
Improves deep learning robustness by considering task and model.
problem Adversarial attacks on deep learning systems.
method Binary and interval label encoding strategy to redefine classification tasks and design corresponding loss functions.
result Our method enhances robustness without sacrificing accuracy.
We consider the problem of imitation learning from expert demonstrations in partially observable Markov decision processes (POMDPs). Belief representations, which characterize the distribution over the latent states in a POMDP, have been modeled using recurrent neural networks and probabilistic latent variable models, …
Neural architecture search has recently attracted lots of research efforts as it promises to automate the manual design of neural networks. However, it requires a large amount of computing resources and in order to alleviate this, a performance prediction network has been recently proposed that enables efficient archit…
New method for scalable barycenter computation using Wasserstein gradient flows.
problem Scalability and integration of label information in barycenter computation.
method Gradient flows in Wasserstein space, time discretization, mini-batch optimal transport, modular regularization, task-aware functions, supervised information integration.
result Empirically validated new state-of-the-art barycenter solver with labeled barycenters outperforming unlabeled ones.
CVIB uses information theory to learn counterfactuals from MNAR data without RCTs.
problem Debiasing learning from missing-not-at-random (MNAR) data in recommendation systems.
method CVIB, a variational information bottleneck, separates task-aware mutual information into factual and counterfactual parts.
result CVIB significantly enhances both shallow and deep models in recommendation systems.
This paper improves forecast stability without sacrificing accuracy using dynamic loss weighting.
problem Rolling origin forecast instability in time series forecasting.
method Dynamic loss weighting algorithms applied to the N-BEATS model.
result Dynamic loss weighting can further improve forecast stability without compromising accuracy.
Neural architecture search has been shown to hold great promise towards the automation of deep learning. However in spite of its potential, neural architecture search remains quite costly. To this point, we propose a novel gradient-based framework for efficient architecture search by sharing information across several …
STanHop predicts multivariate time series with memory-enhanced capabilities.
problem Predicting multivariate time series with memory-enhanced capabilities.
method Sparse Tandem Hopfield Network (STanHop) with two external memory modules.
result STanHop outperforms dense Hopfield models in memory retrieval error.
Model-agnostic meta-learners aim to acquire meta-learned parameters from similar tasks to adapt to novel tasks from the same distribution with few gradient updates. With the flexibility in the choice of models, those frameworks demonstrate appealing performance on a variety of domains such as few-shot image classificat…
Paper proposes a method to make learned models focus on task-relevant information.
problem Mismatch between model objective and downstream task objective.
method Direct prediction towards task-relevant information, self-supervised.
result Model more effectively models relevant parts of the scene conditioned on the goal.
Proposes a new method for decision-aware learning in optimization.
problem Contextual linear optimization with cost prediction errors.
method Reweighing prediction error by decision regret for decision-aware predictor.
result Improves over predict-then-optimize framework for misspecified models.
Extends neural diffusion processes for multi-task regression.
problem Limited to single-task inference, existing formulations cannot capture dependencies across related tasks.
method Introduces a task encoder to condition diffusion model on low-dimensional representations of context observations.
result Improves predictive performance and uncertainty calibration across related functions.
AMEIR automates recommender system design using NAS.
problem Manual feature and architecture engineering in recommender systems.
method Three-stage search space and tailored three-step pipeline.
result AMEIR outperforms manual and complex NAS methods.
This paper explains how overparameterization aids in meta-learning with few samples.
problem Building a generalizable model with few samples in meta-learning.
method Analyzes the optimal linear representation and sample complexity for meta-learning tasks.
result Overparameterization naturally answers fundamental meta-learning questions, reducing sample complexity.
New framework prevents forgetting in sequential learning tasks.
problem Catastrophic forgetting in Convolutional Neural Networks.
method Conditional channel gating modules and task classifier.
result Consistent improvement over existing methods in continual learning.
Deep learning provides a powerful tool for machine perception when the observations resemble the training data. However, real-world robotic systems must react intelligently to their observations even in unexpected circumstances. This requires a system to reason about its own uncertainty given unfamiliar, out-of-distrib…