A new distance for mixed-variable, hierarchical datasets with meta variables.
problem Heterogeneous datasets limit generalizability and performance in machine learning and optimization.
method Developed a modeling framework for mixed-variable and hierarchical domains with meta variables, and a novel distance function.
result The novel distance function allows comparison of heterogeneous datasets, improving model performance.
Meta-learning performance is affected by how task diversity is allocated, not just overall variability.
problem Meta-learning performance degrades when task diversity is unevenly distributed.
method Decomposed task-specific regression effects into structurally informative and orthogonal components.
result Meta-learning prediction degrades when a larger fraction of task variability is orthogonal and non-informative.
VSML unifies meta learning concepts and enables simple backpropagation.
problem Improving and unifying meta learning concepts for neural networks.
method Unified approach using variable shared meta learning and simple weight-sharing.
result Simple backpropagation can be implemented and meta learned without gradient calculation.
Deep reinforcement learning algorithms require large amounts of experience to learn an individual task. While in principle meta-reinforcement learning (meta-RL) algorithms enable agents to learn new skills from small amounts of experience, several major challenges preclude their practicality. Current methods rely heavi…
ACE models allow flexible conditioning and prediction of latent variables.
problem Lack of flexibility in conditioning and prediction of latent variables in probabilistic models.
method Introduces Amortized Conditioning Engine (ACE) that explicitly represents latent variables and allows runtime conditioning and prediction.
result ACE models outperform existing methods in diverse tasks like image completion, classification, Bayesian optimization, and simulation-based inference.
Meta-learners estimate CATE from multiple environments with partial identification.
problem Estimating CATE from observational data across multiple environments with violations of causal assumptions.
method Adapt IV literature for partial identification, propose model-agnostic meta-learners.
result Meta-learners effectively estimate CATE bounds across various experiments.
This paper introduces Meta-Q-Learning (MQL), a new off-policy algorithm for meta-Reinforcement Learning (meta-RL). MQL builds upon three simple ideas. First, we show that Q-learning is competitive with state-of-the-art meta-RL algorithms if given access to a context variable that is a representation of the past traject…
Meta Additive Model learns auto-weighting for robust sparse learning.
problem Sparse models' sensitivity to atypical data and noise.
method Bilevel optimization framework with meta data-driven weighting.
result MAM outperforms state-of-the-art models under various data corruptions.
This paper presents an automatic approach for selecting optimal meta-models for sensitivity analysis in complex systems.
problem Efficient surrogate models for high-dimensional problems in virtual prototyping.
method Automatic selection of meta-models, variable space reduction, and advanced sensitivity measures.
result Optimal meta-models and subspace identification for accurate probabilistic analysis.
We learn a compact surrogate model for optimization problems to reduce training and inference time.
problem Solving optimization problems with unknown parameters is computationally expensive and may lead to suboptimal solutions.
method We represent the optimization problem in terms of meta-variables and learn a low-dimensional surrogate model end-to-end with the predictive model.
result We achieve a large reduction in training and inference time, and improved performance.
Learning from small data sets is critical in many practical applications where data collection is time consuming or expensive, e.g., robotics, animal experiments or drug design. Meta learning is one way to increase the data efficiency of learning algorithms by generalizing learned concepts from a set of training tasks …
Meta-learning predicts stock trading volumes by learning from each stock's unique patterns.
problem Predicting trading volumes for different stocks using a universal model.
method Dual-process meta-learning framework that learns common patterns with a meta-learner and specific patterns with stock-dependent parameters.
result Improves performance of various baseline models in volume predictions.
MetaVRF learns adaptive kernels for fast few-shot learning.
problem Few-shot learning with limited data.
method MetaVRF with latent variable model and variational inference.
result MetaVRF produces kernels with high representational power and fast adaptation.
While tasks could come with varying the number of instances and classes in realistic settings, the existing meta-learning approaches for few-shot classification assume that the number of instances per task and class is fixed. Due to such restriction, they learn to equally utilize the meta-knowledge across all the tasks…
Current meta-learning approaches focus on learning functional representations of relationships between variables, i.e. on estimating conditional expectations in regression. In many applications, however, we are faced with conditional distributions which cannot be meaningfully summarized using expectation only (due to e…
Existing Bayesian treatments of neural networks are typically characterized by weak prior and approximate posterior distributions according to which all the weights are drawn independently. Here, we consider a richer prior distribution in which units in the network are represented by latent variables, and the weights b…
Probabilistic active meta-learning improves data efficiency in robotics.
problem Data-efficient learning in robotics where data collection is expensive.
method Conceptualizing meta-learning with a probabilistic latent variable model for sequential task selection.
result Improves data efficiency compared to baselines on simulated robotic experiments.
In this paper, a machine learning-based simulation framework of general-purpose multibody dynamics is introduced. The aim of the framework is to generate a well-trained meta-model of multibody dynamics (MBD) systems. To this end, deep neural network (DNN) is employed to the framework so as to construct data-based meta-…
ST-MAML tackles task ambiguity in meta-learning by encoding tasks with stochastic representations.
problem Handling tasks from multiple distributions is challenging for meta-learning due to task ambiguity.
method ST-MAML uses a stochastic neural network module to encode tasks and propagate task representations to revise input variable encoding.
result ST-MAML matches or outperforms state-of-the-art methods on various tasks.
We propose to meta-learn causal structures based on how fast a learner adapts to new distributions arising from sparse distributional changes, e.g. due to interventions, actions of agents and other sources of non-stationarities. We show that under this assumption, the correct causal structural choices lead to faster ad…
New method detects and prevents unfairness in few-shot regression models.
problem Fairness issues in supervised few-shot meta-learning models.
method Causal Bayesian knowledge graph for dependency visualization, risk difference quantification, and fast-adapted bias-control approach.
result Efficiently detects and mitigates unfairness in model predictions.
Meta-DRL improves resource allocation in O-RAN networks.
problem Dynamic resource allocation in O-RAN networks.
method Meta Deep Reinforcement Learning (Meta-DRL) inspired by MAML.
result 19.8% improvement in network management performance.
Deep generative models are effective methods of modeling data. However, it is not easy for a single generative model to faithfully capture the distributions of complex data such as images. In this paper, we propose an approach for boosting generative models, which cascades meta-models together to produce a stronger mod…
Proposes active learning for meta-learning in graph node response prediction.
problem Difficulty in improving performance with meta-learning due to unbalanced observations.
method Combines graph convolutional neural networks and reinforcement learning for both prediction and node selection.
result Can predict responses and select nodes even for unseen response variables.
We introduce a framework based on bilevel programming that unifies gradient-based hyperparameter optimization and meta-learning. We show that an approximate version of the bilevel problem can be solved by taking into explicit account the optimization dynamics for the inner objective. Depending on the specific setting, …
We study 'meta-dependence' in conditional independence tests across different empirical distributions.
problem Understanding the breakdown of conditional independence properties in finite data.
method Geometric intuition and information projections to measure meta-dependence between conditional independences.
result We provide a measure of meta-dependence that consolidates findings across synthetic and real-world data.
Unified framework for modeling hierarchical spaces in design problems.
problem Challenges in modeling hierarchical, conditional, heterogeneous, or tree-structured domains.
method Unified framework combining feature modeling and graph theory, introducing meta and partially-decreed variables.
result Demonstrated effectiveness on complex system design problems, including neural networks and green-aircraft.
Meta-learner estimates heterogeneous DiD effects robustly.
problem Estimating heterogeneous treatment effects in panel data with DiD.
method Doubly robust meta-learner for CATT, using convex risk minimization and auxiliary models.
result Superior performance over existing methods in empirical tests.
Improved meta-learning for dynamics using additional structured knowledge.
problem Meta-learning for dynamics with limited raw observations.
method Extended Neural ODE Process model to use privileged information.
result Improved accuracy and calibration on simulated dynamics tasks.
New method learns priors for Bayesian neural networks from datasets.
problem Lack of prior beliefs in Bayesian deep learning.
method Amortised variational inference to learn priors from datasets.
result Flexible Bayesian neural networks for meta-learning and within-task minibatching.
The performance of gradient-based optimization strategies depends heavily on the initial weights of the parametric model. Recent works show that there exist weight initializations from which optimization procedures can find the task-specific parameters faster than from uniformly random initializations and that such a w…
Computer simulation has become the standard tool in many engineering fields for designing and optimizing systems, as well as for assessing their reliability. To cope with demanding analysis such as optimization and reliability, surrogate models (a.k.a meta-models) have been increasingly investigated in the last decade.…
A new meta-learning method using shared variational inference.
problem Meta-learning with uncertainty over model parameters.
method Shared amortized variational inference network for conditional prior and posterior.
result Prevents collapse of conditional prior to Dirac delta function.
New approach to meta-learning with variational Bayes for unlabeled data.
problem Improving machine learning systems' adaptability to small data.
method Generative meta-learning using AEVB and mean-field variational Bayes.
result Relevant VB updates do not depend on generative neural networks for certain models.
ABMLL adapts LLMs to multiple datasets efficiently.
problem Limited generalization across multiple datasets in LLMs.
method Amortized Bayesian Meta-Learning for LoRA (ABMLL).
result ABMLL supports effective generalization across datasets and scales to large models.
Meta-learning reformulated as Bayesian risk minimization.
problem Learning models to quickly adapt to new tasks from small datasets.
method Formalized meta-learning as Bayesian risk minimization, using a probabilistic framework to compute predictive distributions from posterior distributions of latent variables conditioned on contextual datasets.
result A novel Gaussian approximation for the posterior distribution that converges to maximum likelihood estimates and outperforms Neural Process on benchmark datasets.
S2MAM improves semi-supervised learning by selecting relevant variables and updating similarity metrics.
problem Joint learning from labeled and unlabeled data with geometric structure.
method Bilevel optimization scheme for automatic variable selection and similarity matrix update.
result The proposed S2MAM achieves robust and interpretable predictions.
A new method improves semi-supervised learning by handling tasks with different attribute spaces.
problem Existing methods assume tasks share the same attribute space, limiting their applicability.
method Meta-learning approach that embeds labeled and unlabeled data in task-specific spaces using neural networks.
result Improves test performance on tasks with small labeled data using unlabeled and various task data.
Meta-continual learning improves few-shot learning performance.
problem Limited applicability of deep learning solutions to real-world data.
method Unsupervised meta-continual learning with meta-optimization and self-attention.
result Meta-continual learning achieves competitive performance even without supervision.
A meta-learning approach improves the performance of alternating minimization for non-convex optimization problems.
problem Optimizing non-convex problems with multiple variables using alternating minimization.
method Meta-learning based alternating minimization (MLAM) to replace handcrafted updating rules.
result The proposed MLAM method outperforms traditional AM-based methods in various non-convex optimization problems.
Bayesian meta-learning improves health prediction models across similar diseases.
problem Inter- and intra-task variability in healthcare predictions due to disease heterogeneity and patient differences.
method Bayesian meta-learning approach that models task similarity to mitigate negative transfer and improve generalizability.
result Significant generalizability improvements in stroke prediction tasks using electronic health record data.
Paper introduces a method to infer causal direction from limited data.
problem Inference of causal direction from limited observational data.
method Meta learning combined with causal inference to create a generative model.
result The method accurately infers causal direction across various dataset sizes.
This note complements the inspiring work on dimensional analysis and market microstructure by Kyle and Obizhaeva [18]. Following closely these authors, our main result shows by a similar argument as usually applied in physics the following remarkable fact. If the market impact of a meta-order only depends on four well-…
Meta-Dynamic models learn shared neural dynamics across tasks.
problem Learning latent dynamics from neural recordings across different tasks.
method Captures variabilities on a low-dimensional manifold to meta-learn dynamics.
result Meta-Dynamic models can rapidly learn latent dynamics from new recordings.
Model learns to select relevant clinical variables for disease subtype prediction from small data.
problem Few-shot disease subtype prediction from small genomic data.
method Meta learning Prototypical Network with feature selection and sample reweighting.
result Superior performance in predicting disease subtypes and identifying genes.
Meta-learning model predicts intervention effects from uncertain causal graphs.
problem Estimating intervention effects when causal structures are uncertain.
method Model-Averaged Causal Estimation Transformer Neural Process (MACE-TNP) using meta-learning.
result MACE-TNP outperforms Bayesian baselines in predicting intervention distributions.
Pre-season prediction of crop production outcomes such as grain yields and N losses can provide insights to stakeholders when making decisions. Simulation models can assist in scenario planning, but their use is limited because of data requirements and long run times. Thus, there is a need for more computationally expe…
Embeddings in machine learning are low-dimensional representations of complex input patterns, with the property that simple geometric operations like Euclidean distances and dot products can be used for classification and comparison tasks. The proposed meta-embeddings are special embeddings that live in more general in…