REGAIN learns optimal auxiliary directions for forecast reconciliation.
problem Forecast reconciliation from fixed systems; identifying useful auxiliary directions.
method REGAIN learns normalized auxiliary directions, forecasts induced series, and selects directions by loss reduction.
result Gain-selected auxiliary directions improve forecast quality, especially for residual uncertainty.
Dimension reduction of multivariate data supervised by auxiliary information is considered. A series of basis for dimension reduction is obtained as minimizers of a novel criterion. The proposed method is akin to continuum regression, and the resulting basis is called continuum directions. With a presence of binary sup…
Learning with auxiliary tasks can improve the ability of a primary task to generalise. However, this comes at the cost of manually labelling auxiliary data. We propose a new method which automatically learns appropriate labels for an auxiliary task, such that any supervised learning task can be improved without requiri…
Aux-NAS uses auxiliary labels to improve primary task performance without extra inference cost.
problem Improving primary task performance using auxiliary labels without increasing inference cost.
method Architecture-based approach with a flexible asymmetric structure for primary and auxiliary tasks, using Neural Architecture Search (NAS) to evolve networks with only primary-to-auxiliary connections.
result Achieves improved performance on multiple tasks without increasing inference cost.
Paper proposes a method to improve DAG structure reconstruction using auxiliary DAGs.
problem Improving DAG structure reconstruction with limited data.
method Introduces structural similarity measures and a transfer learning framework.
result Significant improvement in DAG reconstruction, even with dissimilar auxiliary DAGs.
Improves deep learning models by blending gradients from training loss and auxiliary objective.
problem Minimizing a single training loss while encouraging desirable model properties.
method Solves a bilevel optimization problem by combining training loss gradients and orthogonal projections of auxiliary gradients.
result Bloop method leads to better performance than other gradient surgery methods without EMA.
New bounds for transfer learning in linear models, improving generalization.
problem Understanding when auxiliary data helps in improving generalization in linear models.
method Derivation of exact error bounds and optimal task weights for linear regression and linear neural networks.
result First non-vacuous sufficient conditions for beneficial auxiliary learning in linear neural networks.
New HMC method uses asymmetrical momentum distributions and improves performance.
problem Rigorous convergence guarantees for HMC with Gaussian momentum distributions.
method New convergence analysis for HMC with general asymmetrical momentum distributions, proposing AD-HMC.
result AD-HMC exhibits geometric convergence in Wasserstein distance under certain conditions.
Learning to drive faithfully in highly stochastic urban settings remains an open problem. To that end, we propose a Multi-task Learning from Demonstration (MT-LfD) framework which uses supervised auxiliary task prediction to guide the main task of predicting the driving commands. Our framework involves an end-to-end tr…
We focus on the problem of search in the multilingual setting. Examining the problems of next-sentence prediction and inverse cloze, we show that at large scale, instance-based transfer learning is surprisingly effective in the multilingual setting, leading to positive transfer on all of the 35 target languages and two…
Two environments are enough to infer causal graphs and counterfactuals.
problem Inferring causal relations from multiple environments, especially for nonlinear mechanisms.
method Using structural causal models and the invariance principle, the study shows that only two auxiliary environments are sufficient for causal graph inference and counterfactual inference.
result Two auxiliary environments are sufficient for identifying causal graphs and counterfactuals.
In this paper, we consider solving a class of nonconvex and nonsmooth problems frequently appearing in signal processing and machine learning research. The traditional alternating direction method of multipliers encounters troubles in both mathematics and computations in solving the nonconvex and nonsmooth subproblem. …
The paper reviews identifiability in linear and nonlinear models, from Gaussian to non-Gaussian.
problem Identifiability issues in latent-variable and structural-equation models, especially in nonlinear cases.
method Review of identifiability theory for linear and nonlinear models, including factor analysis and structural equation models.
result Even nonparametric nonlinear models can be estimated with additional assumptions.
Boosts A/B test precision using auxiliary data from historical users.
problem Small sample sizes and imprecise estimates in A/B tests.
method Coupling design-based causal estimation with machine-learning models of historical user data.
result Effect estimates using auxiliary data are roughly equivalent to increasing sample size by 20%, or up to 50-80% in some cases.
Forward gradients improve neural network training without backpropagation issues.
problem Training neural networks without backpropagation's locking and memorization problems.
method Using directional derivatives in forward differentiation mode, with biased guesses based on feedback from small auxiliary networks.
result Using gradients from a local loss as a candidate direction improves Forward Gradient methods.
SXL embeds spatial autocorrelation into neural networks for better geographic data learning.
problem Difficulties in learning spatial effects for neural networks in geographic data.
method SXL uses auxiliary tasks and autoregressive embeddings to learn spatial autocorrelation.
result SXL improves neural network training in unsupervised and supervised learning tasks.
AuxiLearn combines auxiliary tasks into a single loss function.
problem Improving neural network performance on a main task using auxiliary tasks.
method Implicit differentiation to learn a network that combines auxiliary tasks into a single coherent objective function.
result AuxiLearn consistently outperforms competing methods in various tasks and domains.
New method uses correlated auxiliary feedback to reduce regret in parameterized bandits.
problem Reducing regret in parameterized bandits with correlated auxiliary feedback.
method Develops a reward estimator using auxiliary feedback with tight confidence bounds.
result Shows significant reduction in regret compared to standard methods.
LDF combines neural networks with probabilistic models for data fusion.
problem Combining limited primary data with readily available auxiliary data.
method Neural networks as conjugate mappings of auxiliary data for posterior analysis.
result Efficient inference and compact latent variable posterior distributions.
In-N-Out improves model robustness to out-of-distribution data.
problem Learning robust models with few in-distribution labeled examples.
method Pre-training with auxiliary information and self-training with pseudolabels.
result In-N-Out outperforms auxiliary inputs or outputs alone on both in-distribution and OOD error.
Paper compares GRU and LSTM for predicting wildfire spread direction.
problem Predicting wildfire spread direction with limited data.
method Comparison of Gated Recurrent Unit (GRU) and Long Short-Term Memory (LSTM) networks.
result GRU performs better for longer time series than LSTM.
Researchers identify valid auxiliary functions for extreme value distributions and their max-domains of attraction.
problem Characterize valid auxiliary functions for extreme value distributions and their max-domains of attraction.
method Introduced 'universal' auxiliary functions valid for both VR and vMR representations, identified sets of valid auxiliary functions, and proposed a method for finding appropriate auxiliary functions.
result Characterized valid auxiliary functions for both VR and vMR representations for the entire MDA distribution families.
Auxiliary Tuning adapts pre-trained models for novel tasks efficiently.
problem Adapting pre-trained models for new tasks efficiently.
method Supplementing pre-trained model with an auxiliary model that shifts output distribution.
result Achieved similar results to training from scratch with fewer resources.
The paper improves high-dimensional linear regression prediction and estimation using auxiliary samples.
problem Estimating and predicting high-dimensional linear regression models with auxiliary samples.
method Proposes Trans-Lasso for data-driven transfer learning, establishing optimality for prediction and estimation.
result Knowledge from auxiliary samples can improve learning performance in target problems.
JEPA fails to improve language model performance when fine-tuning.
problem Improving language model performance through latent representation learning.
method Testing various training-time auxiliaries on natural-language-to-regex generation.
result No auxiliary improves cross-entropy gradient cosine with decoder visibility.
Method reweights auxiliary tasks to reduce data need for main task.
problem Limited labeled data for supervised learning.
method Formulates weighted likelihood function as surrogate prior, minimizing divergence to true prior.
result Effective use of limited labeled data with auxiliary tasks, improving performance.
This paper reviews dMTL and methods for selecting auxiliary tasks.
problem Improving model performance for multiple tasks.
method Review of dMTL approaches and methods for selecting auxiliary tasks.
result Methods for selecting auxiliary tasks can improve dMTL performance.
Statistical inference is considered for variables of interest, called primary variables, when auxiliary variables are observed along with the primary variables. We consider the setting of incomplete data analysis, where some primary variables are not observed. Utilizing a parametric model of joint distribution of prima…
Regret analysis is challenging in Multi-Agent Reinforcement Learning (MARL) primarily due to the dynamical environments and the decentralized information among agents. We attempt to solve this challenge in the context of decentralized learning in multi-agent linear-quadratic (LQ) dynamical systems. We begin with a simp…
MFAI uses gradient boosted trees to leverage auxiliary info for scalable Bayesian matrix factorization.
problem Matrix factorization struggles with poor data quality, especially high sparsity and low SNR.
method Integrates gradient boosted trees into probabilistic matrix factorization framework.
result MFAI effectively leverages auxiliary information, improving model performance.
Multi-task learning (MTL) has achieved success over a wide range of problems, where the goal is to improve the performance of a primary task using a set of relevant auxiliary tasks. However, when the usefulness of the auxiliary tasks w.r.t. the primary task is not known a priori, the success of MTL models depends on th…
Proves Bochner's identity on graphs using a new auxiliary graph.
problem Extending Bochner's identity to graph theory.
method Introduces a complete tangent graph to prove the identity.
result Validates Bochner's identity on graphs.
CASTLE learns causal DAG to improve model generalization.
problem Improving model generalization to out-of-sample data.
method CASTLE learns causal relationships via adjacency matrix embedded in neural network input layers, reconstructing only causal features.
result CASTLE leads to better out-of-sample predictions compared to other regularizers.
ScoreFusion fuses multiple diffusion models to enhance generative modeling of a target population.
problem Enhancing generative modeling of a target population with limited data.
method ScoreFusion uses KL barycenters of auxiliary populations and recasts the learning problem as score matching in denoising diffusion.
result ScoreFusion achieves a dimension-free sample complexity bound in total variation distance.
We propose a probabilistic model for refining coarse-grained spatial data by utilizing auxiliary spatial data sets. Existing methods require that the spatial granularities of the auxiliary data sets are the same as the desired granularity of target data. The proposed model can effectively make use of auxiliary data set…
Proposes a method to improve graph neural networks on heterogeneous graphs using meta-paths.
problem Improving graph neural networks on heterogeneous graphs with auxiliary tasks.
method Self-supervised auxiliary learning with meta-paths for heterogeneous graphs.
result Consistently improves link prediction and node classification on heterogeneous graphs.
NeurT-FDR controls FDR by incorporating auxiliary covariates in deep learning.
problem Controlling FDR in complex large-scale problems with indirect relations among covariates.
method NeurT-FDR uses a deep Black-Box framework that parametrizes test-level covariates as a neural network and adjusts auxiliary covariates through a regression framework.
result NeurT-FDR makes substantially more discoveries in real datasets compared to competitive baselines.
We consider the problem of learning from sparse and underspecified rewards, where an agent receives a complex input, such as a natural language instruction, and needs to generate a complex response, such as an action sequence, while only receiving binary success-failure feedback. Such success-failure rewards are often …
We introduce a new family of MCMC samplers that combine auxiliary variables, Gibbs sampling and Taylor expansions of the target density. Our approach permits the marginalisation over the auxiliary variables yielding marginal samplers, or the augmentation of the auxiliary variables, yielding auxiliary samplers. The well…
Deep generative models parameterized by neural networks have recently achieved state-of-the-art performance in unsupervised and semi-supervised learning. We extend deep generative models with auxiliary variables which improves the variational approximation. The auxiliary variables leave the generative model unchanged b…
LRSAO uses RL to dynamically select and unlearn auxiliary objectives for EA optimization.
problem Optimizing complex landscapes with evolutionary algorithms.
method Local Reinforcement Learning (RL) to dynamically select and unlearn auxiliary objectives.
result Improved optimization complexity from O(n2log(n)/ℓ) to Θ(n2/ℓ2+nlog(n)). We propose a technique for increasing the efficiency of gradient-based inference and learning in Bayesian networks with multiple layers of continuous latent vari- ables. We show that, in many cases, it is possible to express such models in an auxiliary form, where continuous latent variables are conditionally determini…
Generalization and reliability of multilingual translation often highly depend on the amount of available parallel data for each language pair of interest. In this paper, we focus on zero-shot generalization---a challenging setup that tests models on translation directions they have not been optimized for at training t…
Enhances machine learning with background knowledge through feature generation.
problem Machine learning struggles with small datasets.
method Generates features from auxiliary datasets not similar to the original task.
result Significant improvement in various learning tasks.
Conventional application of convolutional neural networks (CNNs) for image classification and recognition is based on the assumption that all target classes are equal(i.e., no hierarchy) and exclusive of one another (i.e., no overlap). CNN-based image classifiers built on this assumption, therefore, cannot take into ac…
DARTS- improves robustness by factoring out skip connections' advantage.
problem Performance collapse in DARTS architecture search.
method Factor out skip connections' advantage with an auxiliary skip connection.
result Significantly improved robustness across various datasets.
Learn dynamics of a system using auxiliary data from similar systems.
problem Learning dynamics of a linear system with limited data.
method Weighted least squares approach, incorporating auxiliary data.
result Auxiliary data can help reduce intrinsic error due to noise.
Detecting and aggregating sentiments toward people, organizations, and events expressed in unstructured social media have become critical text mining operations. Early systems detected sentiments over whole passages, whereas more recently, target-specific sentiments have been of greater interest. In this paper, we pres…