We present an analysis of the problem of identifying biological context and associating it with biochemical events in biomedical texts. This constitutes a non-trivial, inter-sentential relation extraction task. We focus on biological context as descriptions of the species, tissue type and cell type that are associated …
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “context association”
LLMs can be tricked into recalling facts based on context clues.
problem Manipulation of LLMs' factual recall through context changes.
method Mathematical exploration of transformers' associative memory properties.
result Transformers use self-attention and value matrix for associative memory.
This article proposes a biologically inspired neurocomputational architecture which learns associations between words and referents in different contexts, considering evidence collected from the literature of Psycholinguistics and Neurolinguistics. The multi-layered architecture takes as input raw images of objects (re…
Distributional Associations vs In-Context Reasoning: A Study of Feed-forward and Attention Layerscs.LG
The study compares feed-forward and attention layers in language models.
problem Understanding the role of feed-forward and attention layers in language models.
method Empirical and theoretical analysis in a synthetic setting.
result Feed-forward layers learn simple distributional associations, while attention layers focus on in-context reasoning.
Bayesian framework uses context trees for efficient time series modeling.
problem Building flexible and interpretable mixture models for real-valued time series.
method Hierarchical Bayesian modelling with context trees and autoregressive models.
result The methods outperform state-of-the-art techniques on simulated and real-world experiments.
Transformers recall from long distributions with statistical guarantees.
problem Designing Transformers that can recall from arbitrarily long, distributional contexts.
method Recast associative memory as probability measures, decomposing the task into recall and prediction.
result A shallow measure-theoretic Transformer learns the recall-and-predict map under spectral assumptions.
The paper introduces contexture theory to characterize representation learning from contexts.
problem Lack of systematic characterization of representation learning methods.
method Characterizes representation learning as learning from the association between input and context variable.
result Contexture theory shows that representations can be approximated by top singular functions of the context.
Paper tackles binary feedbacks in contextual search learning.
problem Learning underlying mean value function in context with binary feedbacks.
method Tri-section search combined with margin-based active learning.
result Algorithm achieves queries for -estimation accuracy.
The associator of a non-associative algebra is the curvature of the Hochschild quasi-complex. The relationship ``curvature-associator'' is investigated. Based on this generic example, we extend the geometric language of vector fields to a purely algebraic setting, similar to the context of Gerstenhaber algebras. We int…
The paper tackles long-context linear system identification with improved sample complexity bounds.
problem Identifying dynamical systems with long dependencies over fixed context windows.
method Established sample complexity bounds for systems with linear dependencies over a context window of length p.
result The learning process is not hindered by slow mixing properties in extended context windows.
The Giroux correspondence and the notion of a near force-free magnetic field are used to topologically characterize near force-free magnetic fields which describe a variety of physical processes, including plasma equilibrium. As a byproduct, the topological characterization of force-free magnetic fields associated with…
The dissertation establishes a contexture theory to mathematically characterize representation learning.
problem The lack of a scientific understanding of representation learning in foundation models.
method Introduces the contexture theory as a unified framework for analyzing representation learning methods.
result Representation learning is optimal when the association between input and context is neither too strong nor too weak.
MCPCA analyzes shared factors across multiple data contexts.
problem No tools to recover shared factors across multiple contexts.
method Developed a theoretical and algorithmic framework (MCPCA).
result Reveals shared axes of variation across subsets of contexts.
Develops methods to analyze feature-outcome associations in subpopulations.
problem Challenges in understanding feature-outcome associations in high-dimensional data.
method Geometric decomposition framework using gradient flow and co-monotonicity decomposition.
result Identifies context-dependent patterns and improves statistical power and interpretability.
New model uses attention for in-context learning of categorical data.
problem Learning from categorical data in context.
method Attention-based network with self-attention and cross-attention layers, using functional gradient descent.
result Model can perform multi-step inference for categorical observations.
In the framework of Abstract Differential Geometry, we show that to a given principal sheaf and a representation of its stuctural sheaf in , where A is a sheaf of associative, commutative, unital algebras (over R or C), we associate a vector sheaf. Moreover, under some natural assumptions on the compatibility of t…
Enhances neural processes to learn from multiple related datasets.
problem Improving predictions from datasets with shared similarities.
method Developed the in-context in-context learning pseudo-token TNP (ICICL-TNP) to condition on both sets of datapoints and sets of datasets.
result Demonstrated the importance and effectiveness of in-context in-context learning.
Trains word embeddings from music and text data to link music contexts.
problem Varying vocabulary size and musical relevance in word embeddings.
method Combines general text and music-specific data to train word embeddings.
result Trained embeddings better associate music contexts with compositions.
A new method uses gene interaction networks to predict gene functions.
problem Predicting gene functions from gene interactions.
method Context graph kernel approach in a machine learning framework.
result The proposed method outperforms linkage-assumption-based methods.
We consider generalized gradients in the general context of -structures. They are natural first order differential operators acting on sections of vector bundles associated to irreducible -representations. We study their geometric properties and show in particular their conformal invariance.
Framework detects anomalies in industrial processes using deep learning.
problem Detect anomalies in complex industrial processes.
method Causal-based framework with unsupervised deep learning.
result Successfully validated abstract contexts of blast furnace assets.
CoDA adapts dynamics models to new physical systems by conditioning on context.
problem Generalizing to new physical systems with shared dynamics but different contexts.
method Context-informed dynamics adaptation (CoDA) using multiple environments and a hypernetwork.
result State-of-the-art generalization results on nonlinear dynamics.
Despite tremendous progress in outlier detection research in recent years, the majority of existing methods are designed only to detect unconditional outliers that correspond to unusual data patterns expressed in the joint space of all data attributes. Such methods are not applicable when we seek to detect conditional …
A strong interaction is known to exist between edge-colored graphs (which encode PL pseudo-manifolds of arbitrary dimension) and random tensor models (as a possible approach to the study of Quantum Gravity). The key tool is the {\it G-degree} of the involved graphs, which drives the {\it expansion} in the tensor …
The present paper unifies some aspects concerning the vertical Liouville distributions on the tangent (cotangent) bundle of a Finsler (Cartan) space in the context of generalized geometry. More exactly, we consider the big-tangent manifold associated to a Finsler space and of its -du…
The Temporal Dimension of Riskq-fin.PM
Multi-period measures of risk account for the path that the value of an investment portfolio takes. In the context of probabilistic risk measures, the focus has traditionally been on the magnitude of investment loss and not on the dimension associated with the passage of time. In this paper, the concept of temporal pat…
RNNs struggle with in-context retrieval, while Transformers excel.
problem In-context retrieval capability of RNNs.
method Theoretical analysis and experimental techniques (CoT, RAG, Transformer layer).
result Enhancing RNNs with techniques improves their in-context retrieval capability, closing the representation gap with Transformers.
In this paper, we introduce a novel layer designed to be used as the output of pre-trained neural networks in the context of classification. Based on Associative Memories, this layer can help design Deep Neural Networks which support incremental learning and that can be (partially) trained in real time on embedded devi…
We examine universal curvature identities for pseudo-Riemannian manifolds with boundary. We determine the Euler-Lagrange equations associated to the Chern-Gauss-Bonnet formula and show that they are given solely in terms of curvature {and the second fundamental form and do not involve covariant derivatives thus general…
In this paper we develop a general conceptual approach to the problem of existence of action-angle variables for dynamical systems, which establishes and uses the fundamental conservation property of associated torus actions: anything which is preserved by the system is also preserved by the associated torus actions. T…
Home entertainment systems feature in a variety of usage scenarios with one or more simultaneous users, for whom the complexity of choosing media to consume has increased rapidly over the last decade. Users' decision processes are complex and highly influenced by contextual settings, but data supporting the development…
TabPFN's internal geometry topology correlates with dataset reliability.
problem Understanding TabPFN's behavior on structurally difficult tabular geometries.
method Using zigzag persistent homology, studying TabPFN's internal representations on synthetic tabular tasks with known topology.
result Topology of TabPFN's internal representation geometry is strongly associated with dataset-level reliability.
We use the theory of Berezin-Toeplitz operators of Ma and Marinescu to study the quantum Hamiltonian dynamics associated with classical Hamiltonian flows over closed prequantized symplectic manifolds in the context of geometric quantization of Kostant and Souriau. We express the associated evolution operators via paral…
We outline the notions and concepts of the calculus of variational multivectors within the Poisson formalism over the spaces of infinite jets of mappings from commutative (non)graded smooth manifolds to the factors of noncommutative associative algebras over the equivalence under cyclic permutations of the letters in t…
Reformulates divergence map for Turaev cobracket in non-commutative geometry.
problem Algebraic description of Turaev cobracket on surfaces.
method Non-commutative geometry, flat connection, associative algebras, Lie operad.
result Algebraic description of Turaev cobracket on surfaces.
Let be the Lebesgue space of square-integrable functions on the unit circle. We show that the injectivity problem for Toeplitz operators is linked to the existence of geodesics in the Grassmann manifold of . We also investigate this connection in the context of restricted Grassmann manifolds associated to $p…
LLMs compress financial texts, but distort decision-making.
problem LLMs compress financial texts, altering decision-making.
method Analyzed two diagnostic patterns: decontextualization and model dependency. Proposed Agentic Context Compression.
result LLM-compressed financial texts alter decision-making.
GOAT learns multiple node representations from graph structure alone.
problem Context-free graph representation learning limits model performance.
method Inspired by gossip and mutual attention, GOAT learns multiple node representations.
result GOAT outperforms 12 SOTA baselines on link prediction and clustering tasks.
We derive the Chern-Gauss-Bonnet Theorem for manifolds with smooth non-degenerate boundary in the pseudo-Riemannian context from the corresponding result in the Riemannian setting by examining the Euler-Lagrange equations associated to the Pfaffian of a complex "metric" on the tangent space and then applying analytic c…
In the context of generalized geometry we first show how the Courant bracket helps to define connections with skew torsion and then investigate a five-dimensional invariant functional and its associated geometry. A Hamiltonian flow arising from this corresponds to a version of the Nahm equations using the Courant brack…
New Thompson Sampling for partially observed context bandits reduces regret logarithmically with time.
problem Improving Thompson Sampling for partially observed context bandits.
method Proposed a Thompson Sampling algorithm for partially observable contextual multi-armed bandits with theoretical performance guarantees.
result Regret scales logarithmically with time and the number of arms, and linearly with the dimension.
On overfitting and asymptotic bias in batch reinforcement learning with partial observabilitystat.ML
This paper provides an analysis of the tradeoff between asymptotic bias (suboptimality with unlimited data) and overfitting (additional suboptimality due to limited data) in the context of reinforcement learning with partial observability. Our theoretical analysis formally characterizes that while potentially increasin…
Large language models based on transformers have achieved great empirical successes. However, as they are deployed more widely, there is a growing need to better understand their internal mechanisms in order to make them more reliable. These models appear to store vast amounts of knowledge from their training data, and…
Covariantly constant forms on torsionful geometries from world-sheet and spacetime perspectiveshep-th
The symmetries of two-dimensional supersymmetric sigma models on target spaces with covariantly constant forms associated to special holonomy groups are analysed. It is shown that each pair of such forms gives rise to a new one, called a Nijenhuis form, and that there may be further reductions of the structure group. I…
New setting combines state evolution and corrupted context for better decision-making.
problem Decision-making in a changing state with unreliable context.
method Proposes a new algorithm using a referee to dynamically combine contextual bandit and multi-armed bandit policies.
result Improved empirical performance compared to existing algorithms.
Diffusion models learn hierarchical composition rules from data.
problem How many samples do generative models need to learn hierarchical composition rules?
method Theoretical and empirical investigation of diffusion models on probabilistic context-free grammars.
result Diffusion models learn hierarchical composition rules with sample complexity scaling polynomially with context size.
Paper proposes semi-supervised learning with triplet Markov chains.
problem Lack of labels in training data.
method Variational Bayesian inference for semi-supervised learning.
result Derives semi-supervised algorithms for various sequential models.
Consider a nonparametric contextual multi-arm bandit problem where each arm is associated to a nonparametric reward function mapping from contexts to the expected reward. Suppose that there is a large set of arms, yet there is a simple but unknown structure amongst the arm reward…