We consider partial matchings, which are finite graphs consisting of edges and vertices of degree zero or one. We consider transformations between two states of partial matchings. We introduce a method of presenting a transformation between partial matchings. We introduce the notion of the lattice presentation of a par…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “transformer matching”
GMC benchmark isolates retrieval in Transformers, revealing max-margin alignment.
problem Understanding how Transformers develop match-and-copy behavior on natural data.
method Introducing Gaussian Match-and-Copy (GMC) as a minimalist benchmark.
result Gradient descent drives parameters to diverge while aligning with max-margin separator.
The strength of association between a pair of data vectors is represented by a nonnegative real number, called matching weight. For dimensionality reduction, we consider a linear transformation of data vectors, and define a matching error as the weighted sum of squared distances between transformed vectors with respect…
The paper provides convergence guarantees for ODE-based generative models using transformers.
problem Theoretical guarantees for ODE-based generative models.
method A pre-trained autoencoder maps inputs to a latent space, and a transformer predicts the velocity field.
result The distribution of samples generated via estimated ODE flow converges to the target distribution in Wasserstein-2 distance.
Match van Stockum dust to vacuum metrics with a single parameter.
problem Matching van Stockum dust to vacuum metrics.
method 1-parametric family of non-static Papapetrou vacuum metrics, Ehlers and Kramer--Neugebauer transformations.
result Explicit examples of matching, including Bonnor metric and Lanczos--van Stockum dust metric.
A new method matches measures across different spaces using cost-regularized optimal transport.
problem Matching measures in different spaces without aligned data.
method Cost-regularized optimal transport formulation to match measures across two Euclidean spaces.
result Demonstrated applicability to single-cell spatial transcriptomics/multiomics matching tasks.
A simple log-transform fixes heavy-tailed data for generative models.
problem Standard generative models struggle with heavy-tailed data.
method Apply the soft-log transform to data before training and exponentiate samples after generation.
result Log-FM outperforms specialized baselines on multivariate benchmarks.
GDT improves reinforcement learning by matching future state information efficiently.
problem Efficient learning of multi-task policies from trajectory data.
method Generalized Decision Transformer (GDT) for offline hindsight information matching.
result GDT enables effective offline multi-task state-marginal matching and imitation learning.
Transformers can interpolate between arbitrary measures.
problem Understanding the expressive power of Transformers as measure-to-measure maps.
method Provided an explicit choice of parameters for a single Transformer to match N arbitrary input measures to N arbitrary target measures.
result A single Transformer can interpolate between arbitrary measures.
SMURF-THP improves Transformer Hawkes process models by providing uncertainty quantification.
problem Uncertainty quantification for Transformer Hawkes process predictions.
method Score matching for learning the score function of event arrival times.
result SMURF-THP outperforms likelihood-based methods in confidence calibration.
Paper introduces infinite-dimensional generative models using Doob's h-transform.
problem Defining generative models in infinite dimensions.
method Using Doob's h-transform to force a reference diffusion towards a target distribution.
result The forced process can be approximated by minimising a score-matching objective.
ZeroS improves Transformers by adding negative weights, matching or beating softmax attention.
problem Limited performance of linear attention methods, especially in long context sequences.
method Proposes Zero-Sum Linear Attention (ZeroS) that removes the zero-order term and reweights zero-sum softmax residuals.
result ZeroS matches or exceeds standard softmax attention across various benchmarks, theoretically expanding representable functions.
The study enhances financial rule matching using NLP without datasets.
problem Performing semantic matching between financial rules and policies.
method Outperforming pre-trained models with NLP techniques using free resources.
result Improved semantic matching between financial rules and policies.
Matching datasets of multiple modalities has become an important task in data analysis. Existing methods often rely on the embedding and transformation of each single modality without utilizing any correspondence information, which often results in sub-optimal matching performance. In this paper, we propose a nonlinear…
The paper tackles matching a desired mean in causal systems through shift interventions.
problem Matching a desired mean in causal systems.
method Defining Markov equivalence classes, proposing active learning strategies, deriving lower bounds.
result Proposed active learning strategies require fewer interventions than previous approaches, especially for certain graph classes.
New algorithm learns bridged diffusion processes without time-reversals.
problem Learning bridged diffusion processes efficiently and accurately.
method Score matching with Doob's h-transform, avoiding time-reversals.
result Outperforms existing methods in learning bridged diffusion processes.
DFM models are analyzed for generating distributions with provable convergence.
problem Training DFM models to generate distributions that match true data.
method Theoretical analysis decomposes error into approximation and estimation errors.
result DFM models converge to true data distribution as training set size increases.
The paper matches features in images using centro-affine invariants and heat flow.
problem Feature matching in images with invariant algorithms.
method Developed an invariant algorithm using centro-affine invariants and heat flow.
result The algorithm compares favorably with existing feature matching methods.
Variational autoencoder is a powerful deep generative model with variational inference. The practice of modeling latent variables in the VAE's original formulation as normal distributions with a diagonal covariance matrix limits the flexibility to match the true posterior distribution. We propose a new transformation, …
TVM improves generative modeling by matching terminal velocities.
problem Creating high-fidelity one- and few-step generative models.
method TVM generalizes flow matching, modeling transitions between diffusion timesteps and regularizing terminal behavior.
result TVM achieves state-of-the-art FID scores with minimal architectural changes and fused attention kernel.
Increasing availability of vehicle GPS data has created potentially transformative opportunities for traffic management, route planning and other location-based services. Critical to the utility of the data is their accuracy. Map-matching is the process of improving the accuracy by aligning GPS data with the road netwo…
Flow matching KL divergence bound derived for smooth distributions.
problem Estimating smooth distributions efficiently.
method Deterministic upper bound on KL divergence derived from flow-matching loss.
result Flow matching achieves nearly minimax-optimal efficiency under TV distance.
DiffusionBlocks trains neural networks by breaking them into independent blocks, reducing memory usage.
problem Memory bottlenecks in end-to-end neural network training.
method Transforming transformer-based networks into independent trainable blocks via a denoising process.
result Independent block-wise training matches end-to-end training performance while reducing memory requirements.
The problem of learning a correspondence relationship between nodes of two networks has drawn much attention of the computer science community and recently that of statisticians. The unseeded version of this problem, in which we do not know any part of the true correspondence, is a long-standing challenge. For low-rank…
Transformers achieve near-optimal dynamic regret in non-stationary reinforcement learning.
problem Understanding and handling non-stationary environments in reinforcement learning.
method Demonstrated that transformers can achieve nearly optimal dynamic regret bounds in non-stationary settings.
result Transformers can approximate and learn strategies for non-stationary environments, matching or outperforming existing expert algorithms.
Any generic closed curve in the plane can be transformed into a simple closed curve by a finite sequence of local transformations called homotopy moves. We prove that simplifying a planar closed curve with self-crossings requires homotopy moves in the worst case. Our algorithm improves the best previou…
CDF Transform-and-Shift: An effective way to deal with datasets of inhomogeneous cluster densitiescs.LG
The problem of inhomogeneous cluster densities has been a long-standing issue for distance-based and density-based algorithms in clustering and anomaly detection. These algorithms implicitly assume that all clusters have approximately the same density. As a result, they often exhibit a bias towards dense clusters in th…
VAELLS learns latent manifold structure to improve VAE model accuracy.
problem VAEs struggle with mismatched latent structure and global structure.
method Integrates learnable manifold model into latent space of VAE.
result Improves model accuracy by matching prior to data manifold structure.
This work interprets diffusion score matching using normalizing flows for better model training and evaluations.
problem Limitations of diffusion score matching when dealing with certain types of distributions.
method The approach involves interpreting the diffusion matrix using normalizing flows to provide better interpretation and usage of diffusion score matching.
result Diffusion score matching is equivalent to the original score matching evaluated in the transformed space defined by the normalizing flow.
New method simulates diffusion bridges using score matching.
problem Simulating diffusion bridges for statistical inference.
method Backward time representation, variational formulation, score matching.
result Effective approximation of diffusion bridges.
GenSBI offers JAX-native SBI methods for natural sciences.
problem Lack of native SBI libraries in JAX for natural sciences.
method Flow and diffusion generative models in JAX.
result Near-ideal mean C2ST scores on SBIBM tasks.
DFM simplifies CNF training without interpolants.
problem Efficiently training CNFs with computationally expensive ODE solving.
method DFM optimizes dual vector fields for bijective transformations.
result DFM outperforms CNF trained with FM or ML objectives.
Reflected Flow Matchingstat.ML
RFM improves CNFs by adding a boundary constraint term and matching velocity fields.
problem Flow matching on constrained domains leads to unnatural samples.
method RFM adds a boundary constraint term and matches velocity fields in a simulation-free manner.
result RFM achieves comparable or better results on standard image benchmarks and produces high-quality samples.
We present the Compressive Transformer, an attentive sequence model which compresses past memories for long-range sequence learning. We find the Compressive Transformer obtains state-of-the-art language modelling results in the WikiText-103 and Enwik8 benchmarks, achieving 17.1 ppl and 0.97 bpc respectively. We also fi…
CR-UOT improves matching of heterogeneous single-cell omics profiles.
problem Matching nonnegative finite Radon measures across heterogeneous spaces.
method Cost-regularized unbalanced optimal transport (CR-UOT) framework.
result CR-UOT improves alignment of heterogeneous single-cell omics profiles.
Proposes a measure to predict generalization in non-matching environments.
problem Characterizing and comparing generalization of machine learning models in non-matching environments.
method Neighborhood invariance measure, calculating invariance as the largest fraction of transformed points classified into the same class.
result Strong and robust correlation between neighborhood invariance and actual out-of-domain generalization.
New theory explains signal propagation in normalization-free transformers.
problem Understanding signal propagation in normalization-free transformers.
method Deriving recurrence relations for activation statistics and APJNs across layers.
result Transformers with elementwise tanh-like nonlinearities exhibit subcritical signal propagation.
Paper learns data-driven organ matching rules from observational data.
problem Tackles organ transplantation compatibility using observational data.
method Representation learning to cluster donors and apply recipient transformations.
result Model outperforms human experts in predicting transplant outcomes.
Fast approximate inference for non-Gaussian data.
problem Efficient inference for non-Gaussian data.
method Laplace Matching for fast approximate inference in latent Gaussian models.
result Achieves high approximation quality with low computational cost.
Method matches noisy remote sensing images robustly.
problem Matching noisy remote sensing images.
method Combining attention mechanism with feature enhancement.
result More efficient and accurate matches achieved.
Flow Matching enables robust training of CNFs with various probability paths.
problem Training Continuous Normalizing Flows (CNFs) at large scales.
method Flow Matching (FM) is a simulation-free approach for training CNFs by regressing vector fields of conditional probability paths.
result Flow Matching with diffusion paths yields more robust and stable training compared to diffusion-based methods.
Graph matching consists of aligning the vertices of two unlabeled graphs in order to maximize the shared structure across networks; when the graphs are unipartite, this is commonly formulated as minimizing their edge disagreements. In this paper, we address the common setting in which one of the graphs to match is a bi…
Algorithmic Task Capture, Computational Complexity, and Inductive Bias of Infinite Transformerscs.LG
Transformers capture combinatorial tasks with bounded error and logarithmic sample dependence.
problem Capturing complex combinatorial tasks with bounded error and sample efficiency.
method Formal definition of algorithmic capture, empirical analysis of infinite-width transformers, upper bounds on computational complexity.
result Transformers exhibit an inductive bias favoring simpler algorithmic procedures over higher complexity ones.
SDM Policy accelerates inference for robotic tasks while maintaining high action quality.
problem Prolonged inference times in diffusion-based policies for high-frequency control tasks.
method Two-stage optimization: score matching and distribution matching; dual-teacher mechanism.
result 6x inference speedup with state-of-the-art action quality.
L-GATr transforms high-energy physics data using geometric algebra and Lorentz symmetry.
problem Extracting scientific understanding from particle-physics experiments with high precision and efficiency.
method L-GATr, a geometric algebra Transformer, representing data in 4D space-time and being equivariant under Lorentz transformations.
result L-GATr achieves performance comparable to or better than domain-specific baselines on regression, classification, and generative tasks.
Deep generative networks have been widely used for learning mappings from a low-dimensional latent space to a high-dimensional data space. In many cases, data transformations are defined by linear paths in this latent space. However, the Euclidean structure of the latent space may be a poor match for the underlying lat…
New methods use vector search and nearest-neighbor matching for policy learning in causal inference.
problem Learning optimal policies in causal inference with limited data.
method RAG-based policy learning with vector search and nearest-neighbor matching.
result The methods bound the within-candidate choice regret and evaluate the one-step method directly as a policy.
Introduces Causal Energy Minimization to understand Transformer layers.
problem Empirical parameterization of Transformer blocks remains largely unexplored.
method Causal Energy Minimization framework that recasts Transformer layers as optimization steps on conditional energy functions.
result Identifies design space for Transformer layers including weight sharing and energy-based interpretations.