Factored contextual policy search improves data efficiency in robot learning.
problem Scalable robot learning with limited data.
method Factoring contexts into target-type and environment-type contexts, using Bayesian optimization.
result Experience can be generalized over target-type contexts, leading to faster policy generalization.
Optimal binning method for numeric targets using mathematical programming.
problem Optimizing the discretization of numeric variables for classification.
method Mathematical programming formulation for binary, continuous, and multi-class targets with constraints.
result Convex mixed-integer programming formulations for all target types.
We introduce the first unified theory for target tracking using Multiple Hypothesis Tracking, Topological Data Analysis, and machine learning. Our string of innovations are 1) robust topological features are used to encode behavioral information, 2) statistical models are fitted to distributions over these topological …
Study shows partially-typed NER datasets can match fully-typed ones in model performance.
problem Leveraging multiple partially-typed NER datasets for training models without fully-typed annotations.
method Systematic analysis and controlled experiments comparing partially-typed and fully-typed datasets.
result Models trained with partially-typed annotations can achieve similar performance to those trained with fully-typed annotations.
Bayesian fusion improves radar target recognition for UAVs.
problem Improving radar target recognition for UAVs using multistatic radar configurations.
method Proposes a fully Bayesian RATR framework using Optimal Bayesian Fusion (OBF) to aggregate classification probability vectors from multiple radars.
result Empirical results show that the OBF method significantly enhances classification accuracy compared to other fusion methods and single radar configurations.
A new model for context-aware recommendations using LSTM and latent context.
problem Challenges in incorporating context into recommendation models, especially sparsity and dimensionality issues.
method Sequential latent context modeling using LSTM, reducing context dimensions to a compressed latent space.
result The proposed SLCM outperforms state-of-the-art CARS models in empirical analysis.
Enhances activity recognition in wearable computing with context awareness and uncertainty quantification.
problem Context-dependent activity recognition and unknown contexts in wearable computing.
method Developed the α-{eta} network coupled with uncertainty quantification (UQ) based on maximum entropy.
result Improved accuracy and F-score by 10% through high-level context identification.
MLPs can approximate any function in context, challenging the importance of in-context universality.
problem Understanding why transformers are more effective than classical models.
method Proved MLPs with trainable activation functions are universal in context.
result Transformer success is likely due to factors other than in-context universality.
HCFContext predicts mobile context using collaborative filtering and homomorphic encryption.
problem Accurate mobile context determination for enterprise policies.
method Proposes HPContext and HCFContext models using sequential history and collaborative filtering, with privacy-preserving homomorphic encryption.
result HCFContext enhances context prediction by leveraging related users' observations.
Extracts biological context from biomedical texts to associate with events.
problem Identifying biological context and associating it with biochemical events in texts.
method Analyzed an annotated corpus and developed classifiers using syntactic, distance, and frequency features.
result Developed and evaluated classifiers for context-event association.
A new bandit model with stochastic context distributions and UCB algorithm.
problem Learning optimal actions in a stochastic environment with hidden contexts.
method Stochastic contextual bandit model and UCB algorithm.
result Order-optimal high-probability bound on cumulative regret for linear and kernelized reward functions.
New method improves sentence classification using context information.
problem Classifying sentences with limited context information.
method Context-LSTM-CNN method that considers large contexts and long-range dependencies.
result Consistently improves over previous methods on two datasets.
Paper argues context equals environment, improving AI generalization.
problem AI models struggle to generalize in new environments.
method In-Context Risk Minimization (ICRM) algorithm.
result ICRM leads to significant out-of-distribution performance improvements.
Scales attention for long contexts in LLMs.
problem Development of attention mechanisms for long context inference.
method Scale-invariant total attention and sparsity conditions, with a position-dependent transformation of logits.
result Scale-invariant attention scheme improves validation loss and long-context retrieval.
Enhances neural processes to learn from multiple related datasets.
problem Improving predictions from datasets with shared similarities.
method Developed the in-context in-context learning pseudo-token TNP (ICICL-TNP) to condition on both sets of datapoints and sets of datasets.
result Demonstrated the importance and effectiveness of in-context in-context learning.
Transformers can scale both context and task, but MLPs can only scale task.
problem Understanding and scaling In-Context Learning in transformers.
method Simplified transformer architecture, feature map, and MLP combination.
result Simplified transformer can perform ICL and context-scaling but not task-scaling.
Transformers can be hijacked by context, but deeper models are more robust.
problem Robustness of Transformers against context hijacking for linear classification.
method Developed a theoretical analysis on the robustness of linear transformers, considering model depth, training context lengths, and number of hijacking context tokens.
result Deeper transformers are more robust to context hijacking.
This study examines how sequential correlations affect in-context learning in sequence models.
problem Understanding how in-context learning works with sequentially correlated data.
method Extended linear regression model to sequentially correlated data, tested on transformer architectures.
result Sequential correlations alter the effective context length and attention architecture effectiveness.
Paper proposes linear transformers for efficient in-context learning without context length limitations.
problem Quadratic complexity of softmax transformers limits data processing speed.
method Investigates linear transformers under domain generalization, showing they learn mappings from context distributions to response functions.
result Linear transformers achieve in-context learning with a linear complexity in context length, offering a dimension-independent convergence rate.
New method for contextual bandits with corrupted context.
problem Contextual bandits with corrupted context in online settings.
method Combining contextual bandit and multi-armed bandit approaches.
result Improved learning from all iterations, including corrupted ones.
Adapts Thompson Sampling for contextual bandits with limited context.
problem Online problems in clinical trials, recommender systems, and attention modeling.
method Adapts Thompson Sampling to a restricted context setting.
result Empirical advantages of proposed algorithms on real-life datasets.
The paper tackles long-context linear system identification with improved sample complexity bounds.
problem Identifying dynamical systems with long dependencies over fixed context windows.
method Established sample complexity bounds for systems with linear dependencies over a context window of length p.
result The learning process is not hindered by slow mixing properties in extended context windows.
NOTMAD estimates context-specific Bayesian networks without breaking datasets.
problem Non-convexity of acyclic graphs limits sharing information between context-specific estimators.
method NOTMAD models context-specific Bayesian networks as mixtures of archetypal DAGs, estimating structures and parameters jointly.
result NOTMAD shares information between context-specific acyclic graphs, enabling single-sample resolution.
MCPCA analyzes shared factors across multiple data contexts.
problem No tools to recover shared factors across multiple contexts.
method Developed a theoretical and algorithmic framework (MCPCA).
result Reveals shared axes of variation across subsets of contexts.
Thompson Sampling tackles noisy context in stochastic bandits.
problem Designing an action policy for noisy, corrupted contexts in stochastic bandits.
method Introducing a Thompson Sampling algorithm for Gaussian bandits with Gaussian context noise, adopting an information-theoretic analysis.
result Demonstrates the Bayesian regret of the proposed algorithm concerning the oracle's action policy.
Context-aware ZSL improves object recognition by considering object context.
problem Previous ZSL approaches ignore object context, limiting their effectiveness.
method Proposes a new approach that models the conditional likelihood of objects appearing in specific contexts.
result Contextual information significantly improves ZSL performance and is robust to class imbalance.
Study quantifies context dependency in image classification and segmentation models.
problem Understanding how much context affects model predictions in image classification and segmentation.
method Developed a method to quantify and control model sensitivity to visual context by removing selected objects from images.
result Discovered that certain objects (e.g., 'sidewalk') rely heavily on the presence of other objects (e.g., 'cars') in the context.
This work introduces a method to decompose uncertainty in in-context learning for large language models.
problem Understanding the sources of uncertainty in in-context learning for large language models.
method Variational uncertainty decomposition framework without sampling from latent parameter posterior.
result Quantitative and qualitative validation of decomposed epistemic and aleatoric uncertainties.
CaGAT learns context-aware edge representations for graph data.
problem Ignoring edge representation in GNNs.
method Unified Context-aware Adaptive Graph Attention Network (CaGAT) that learns both node and edge representations.
result CaGAT improves performance on semi-supervised learning tasks.
ContextFlow++ improves generative models by conditioning on mixed-variable contexts.
problem Lack of effective methods for context conditioning in flow-based generative models.
method Proposes ContextFlow++ with additive conditioning and mixed-variable architecture.
result ContextFlow++ achieves higher performance metrics and faster training.
AppsPred predicts smartphone app usage based on context.
problem Predicting personalized usage behavior of smartphone apps based on contexts.
method Random Forest machine learning technique considering multi-dimensional contexts.
result AppsPred significantly outperforms other machine learning approaches in predicting smartphone apps.
Transformers learn to perform logistic regression in-context.
problem Understanding how transformers learn to perform specific tasks in-context.
method Constructed multi-layer transformers that perform in-context logistic regression through normalized gradient descent.
result Transformers can be trained to perform in-context logistic regression effectively.
New insights into how depth and width affect in-context learning in deep models.
problem Understanding how various resources impact in-context learning in deep models.
method Analyzed linear regression in a deep linear self-attention model, varying resources like depth, width, context length, and training steps.
result Increasing depth improves in-context learning even at infinite context length, contrary to previous findings.
Enhances neural language processing with a hierarchical context-aware model.
problem Limited context in neural language processing systems.
method Hierarchical recurrent neural network with multi-level context representation.
result Improves semantic error detection by 12.75% relative for unsupervised models and 20.37% relative for supervised models.
New algorithm reduces regret with diverse contexts in bandits.
problem Impact of context diversity on stochastic linear contextual bandits.
method Design of LinUCB-d algorithm and analysis of its regret performance.
result Cumulative expected regret is bounded by a constant under diverse context assumption.
Efficiently selects top-m designs for various contexts using sequential sampling.
problem Optimizing selection of top-m designs across different contexts.
method Formulated as a stochastic dynamic programming problem, developed sequential sampling policy.
result Asymptotically optimal sampling ratios for efficient selection.
Introduces CStrees for modeling context-specific causal models from observational and interventional data.
problem Modeling context-specific causal relationships from mixed data types.
method Introduces CStrees with a novel factorization criterion and graphical characterization for context-specific conditional independence models.
result Derives a graphical characterization of model equivalence for observational CStrees and extends it to CStree models under context-specific interventions.
New method discovers context effects in choice data.
problem Identifying context effects from choice data is challenging.
method Automatic discovery of context effects from observed choices.
result Automatic discovery of context effects from observed choices.
Proposes BehavDT model for context-aware user behavior prediction.
problem Building a context-aware predictive model based on diverse user behavioral activities.
method Introduces BehavDT, a behavioral decision tree that considers user behavior-oriented generalization.
result BehavDT model outperforms traditional machine learning approaches in predicting user diverse behaviors considering multi-dimensional contexts.
Transformers forecast time series in-context, improving efficiency and performance.
problem Overfitting and limited performance in time series forecasting.
method Reformulate time series forecasting as input tokens, aligning with in-context learning mechanisms.
result Consistently better performance across various settings (full-data, few-shot, zero-shot).
A method for policy search with high-dimensional context variables.
problem Learning from high-dimensional context variables like camera images is challenging.
method Model-based relative entropy stochastic search framework with integrated dimensionality reduction.
result The proposed method outperforms naive dimensionality reduction methods.
Unified approach to fair online learning with stochastic contexts.
problem Fairness in online learning with unknown sensitive contexts.
method Adapting Blackwell's approachability theory to handle unknown contexts' distributions.
result Characterization of optimal trade-off between fairness and performance objectives.
CCM improves context for Meta-RL by contrastive learning.
problem Improving context for Meta-RL to enable task generalization.
method CCM framework using contrastive learning for context encoding and information-gain-based trajectory collection.
result CCM outperforms state-of-the-art algorithms in benchmarks and sparse-reward environments.
TL-ANDI distills context from source data to improve transfer learning for TFMs.
problem Limited transfer learning due to context-size constraints and distribution shifts.
method TL-ANDI uses posterior-aware distillation to construct a compact source context and locally distills labels.
result Improves transfer performance by addressing context-size and distribution shifts.
Bayesian optimization for contextual policy search improves robot learning.
problem Scalable robot learning with limited data.
method Factored contextual representation using target and environment contexts.
result Experience can be generalized over target contexts, leading to faster learning and better generalization.
LLMs generate answers under incomplete context, and their uncertainty should scale with missing information.
problem Evaluating the quality of LLM answers under incomplete context.
method A controlled framework with varying context availability, and two uncertainty measures (sampling-based confidence and response entropy) evaluated on SQuAD.
result Response entropy increases with context removal and explains more variance in accuracy than confidence, suggesting it is a more responsive uncertainty measure.
End-to-end TTS learns context features from text input.
problem Lack of understanding of context features learned by end-to-end TTS.
method Evaluated encoder outputs against context criteria derived from parametric TTS.
result Encoder outputs reflect linguistic and phonetic context features.
Fine-tuning harms in-context learning, but restricting updates to the value matrix improves zero-shot performance.
problem Fine-tuning harms in-context learning, reducing zero-shot performance on unseen tasks.
method Theoretical analysis of linear attention models, identifying conditions for degraded few-shot performance.
result Restricting updates to the value matrix improves zero-shot performance while preserving in-context learning.