arXiv research

ContextFlow++ improves generative models by conditioning on mixed-variable contexts.

Enhances neural processes to learn from multiple related datasets.

Scales attention for long contexts in LLMs.

Neural Processes (NPs) (Garnelo et al 2018a;b) approach regression by learning to map a context set of observed input-output pairs to a distribution over regression functions. Each function models the distribution of the output given an input, conditioned on the context. NPs have the benefit of fitting observed data ef…

Optimally explores dynamical systems with varying properties using context inference.

This work introduces RISE to explain LLMs more reliably by distinguishing essential context.

We compress large neural networks for quick adaptation to specific contexts.

Unified approach to fair online learning with stochastic contexts.

CoDA adapts dynamics models to new physical systems by conditioning on context.

Introduces CStrees for modeling context-specific causal models from observational and interventional data.

Paper analyzes risk bounds for in-context learning in multiclass classification.

The paper develops a new framework for detecting distributional drifts conditioned on context.

Transformers can learn Markov processes with constant depth, surprising results.

Generative model for inferring graph from time series data.

An $η$-Einstein condition is introduced in the context of indefinite g.f.f-manifolds, and a few Schur-type lemmas for indefinite S-manifolds are provided.

Word embeddings are a powerful approach for capturing semantic similarity among terms in a vocabulary. In this paper, we develop exponential family embeddings, a class of methods that extends the idea of word embeddings to other types of high-dimensional data. As examples, we studied neural data with real-valued observ…

Paper solves nonparametric contextual bandit with unbounded contexts.

Parallel recordings of neural spike counts have revealed the existence of context-dependent noise correlations in neural populations. Theories of population coding have also shown that such correlations can impact the information encoded by neural populations about external stimuli. Although studies have shown that the…

Transformers learn a mesa-optimizer to implement in-context learning.

GACELA fills long gaps in musical audio with a GAN and context conditioning.

Contextualized ML learns context-dependent effects using deep learning.

Zero-Shot Learning (ZSL) aims at classifying unlabeled objects by leveraging auxiliary knowledge, such as semantic representations. A limitation of previous approaches is that only intrinsic properties of objects, e.g. their visual appearance, are taken into account while their context, e.g. the surrounding objects in …

The conditions for a cuspidal edge, swallowtail and other fundamental singularities are given in the context of Lie sphere geometry. We then use these conditions to study the Lie sphere transformations of a surface.

New algorithm reduces regret in noisy context bandits.

New insights into how depth and width affect in-context learning in deep models.

A method for learning a context latent vector to improve generalization in model-based RL.

CCM improves context for Meta-RL by contrastive learning.

Scarce data is a major challenge to scaling robot learning to truly complex tasks, as we need to generalize locally learned policies over different task contexts. Contextual policy search offers data-efficient learning and generalization by explicitly conditioning the policy on a parametric context space. In this paper…

Generative neural nets learn deep policies conditioned on goals.

The paper proves conditions for zero Gaussian curvature convex hypersurfaces to be hyperplanes.

New method improves Bayesian optimization by considering budget and prior quality.

This paper compares amortized methods for Bayesian posterior estimation.

Understanding how funding and 4H context regulate crypto markets.

Fine-tuning harms in-context learning, but restricting updates to the value matrix improves zero-shot performance.

An analysis is made of reality conditions within the context of noncommutative geometry. We show that if a covariant derivative satisfies a given left Leibniz rule then a right Leibniz rule is equivalent to the reality condition. We show also that the matrix which determines the reality condition must satisfy the Yang-…

Motivated by online recommendation and advertising systems, we consider a causal model for stochastic contextual bandits with a latent low-dimensional confounder. In our model, there are $L$ observed contexts and $K$ arms of the bandit. The observed context influences the reward obtained through a latent confounder var…

Researchers analyze neural process architectures and their representational capacities.

Despite tremendous progress in outlier detection research in recent years, the majority of existing methods are designed only to detect unconditional outliers that correspond to unusual data patterns expressed in the joint space of all data attributes. Such methods are not applicable when we seek to detect conditional …

Study on neural scaling laws for solving linear systems in-context.

Method discovers local independence in systems with continuous variables.

In the algebraic context, we show that null Osserman, spacelike Osserman, and timelike Osserman are equivalent conditions for a model of signature (2,2). We also classify the null Jordan Osserman models of signature (2,2). In the geometric context, we show that a pseudo-Riemannian manifold of signature (2,2) is null Jo…

New algorithm reduces regret in GLM bandits with tighter bounds.

The contextual bandit literature has traditionally focused on algorithms that address the exploration-exploitation tradeoff. In particular, greedy algorithms that exploit current estimates without any exploration may be sub-optimal in general. However, exploration-free greedy algorithms are desirable in practical setti…

Attention-only transformers learn from context via two stages of inference.

Transformer learns context and regularization for ICL in inverse problems.

We extend a result of Patodi for closed Riemannian manifolds to the context of closed contact manifolds by showing the condition that a manifold is an $η$-Einstein Sasakian manifold is spectrally determined. We also prove that the condition that a Sasakian space form has constant $φ$-sectional curvature $c$ is spectral…

MacroHFT uses memory and context-aware reinforcement learning to improve HFT performance.

Log-linear models are the popular workhorses of analyzing contingency tables. A log-linear parameterization of an interaction model can be more expressive than a direct parameterization based on probabilities, leading to a powerful way of defining restrictions derived from marginal, conditional and context-specific ind…