JEPAs learn data density by predicting perturbed samples, enabling density estimation.
problem Representation collapse in latent spaces.
method Combines latent-space prediction and anti-collapse terms to estimate data density.
result JEPAs can estimate sample probabilities efficiently and in closed-form.
SiamJEPA uses Siamese student encoders to improve JEPA-based representation learning.
problem Improving self-supervised representation learning in JEPA models.
method Proposes SiamJEPA with masked Siamese student encoders and EMA teacher network.
result Siamese student encoders improve representation separability and learning speed.
UWM-JEPA predicts future scenarios in belief space, improving accuracy in partially observed environments.
problem Predicting future scenarios in partially observed environments with uncertainty.
method Introduces UWM-JEPA, a JEPA world model with a density-matrix latent and learned unitary predictor.
result UWM-JEPA achieves 0.77 accuracy on a hidden-velocity indicator task, outperforming LSTM-JEPA.
T-JEPA learns tabular data representations without augmentations, outperforming traditional methods.
problem Challenges in self-supervised learning for tabular data due to lack of data augmentations.
method T-JEPA uses a Joint Embedding Predictive Architecture (JEPA) to predict latent representations of different subsets of features within the same sample.
result Significant improvement in classification and regression tasks, outperforming traditional methods.
LeJEPA provides a scalable, theory-driven approach to self-supervised learning.
problem Lack of practical guidance and theory in JEPAs.
method Identified optimal Gaussian distribution and introduced SIGReg objective.
result LeJEPA achieves state-of-the-art performance with minimal hyperparameters and heuristics.
JEPA fails to improve language model performance when fine-tuning.
problem Improving language model performance through latent representation learning.
method Testing various training-time auxiliaries on natural-language-to-regex generation.
result No auxiliary improves cross-entropy gradient cosine with decoder visibility.
SPLICE generates accurate time-series imputations with reliable prediction intervals.
problem Lack of reliability guarantees in time-series imputation models.
method Modular framework combining latent generative imputation with distribution-free prediction intervals.
result SPLICE achieves lowest mean Load-only MSE and best CRPS on various datasets.
This work improves AI's ability to solve physical tasks by optimizing world models in abstracted spaces.
problem Developing AI agents capable of solving diverse physical tasks and generalizing to new environments.
method Investigates and optimizes a family of joint-embedding predictive world models (JEPA-WMs) for efficient planning in abstracted spaces.
result Proposes a model that outperforms two established baselines in both navigation and manipulation tasks.
LaT-PFN model predicts time series with zero-shot capability.
problem Zero-shot time series forecasting.
method In-context latent space learning with JEPA and PFN integration.
result Superior zero-shot predictions compared to baselines.
Future autonomous systems need reliable world models and complex action sequences.
problem Current automated systems lack reliable world models and complex action sequences.
method Introduce energy-based and latent variable models combined in a hierarchical joint embedding predictive architecture (H-JEPA).
result Combining energy-based and latent variable models in H-JEPA can lead to reliable world models and complex action sequences.
A new approach to protein language models combines latent space prediction with masked language modeling.
problem Improving protein language models by predicting amino acid identities at masked positions.
method A variant of masked language modeling that predicts latent targets only at masked positions, retaining the MLM cross-entropy.
result The new approach outperforms pure masked language modeling on 11 out of 16 downstream tasks.
Symbolic grounding in causal dynamics achieves near-infinite temporal consistency.
problem Achieving linear identifiability in non-Gaussian physical systems.
method Physics-Grounded Symbolic Architecture (PGSA)
result PGSA achieves exact linear identifiability for all physical regimes.
Proposes a new neural network architecture inspired by biology to improve learning and information flow.
problem Improving artificial neural networks to match biological neuron properties like multidirectional propagation and probabilistic modeling.
method Extends KAN approach with joint distribution neurons that can propagate values and distributions, including variance and higher-order moments.
result Proposed architecture can predict and propagate distributions, including expected values and variances.
ST-STORM separates semantic and appearance features for robust representation learning.
problem Traditional SSL methods fail to capture appearance cues in critical applications.
method Hybrid SSL framework with two latent streams, Content and Style, disentangled through gating mechanisms.
result The Style branch effectively isolates complex appearance phenomena without degrading semantic performance.
Embeddings are ubiquitous in machine learning, appearing in recommender systems, NLP, and many other applications. Researchers and developers often need to explore the properties of a specific embedding, and one way to analyze embeddings is to visualize them. We present the Embedding Projector, a tool for interactive v…
Proposes QQE for transforming and embedding data distributions.
problem Transforming and embedding data distributions for better representation or visualization.
method Quantile-Quantile Embedding (QQE) using quantile-quantile plot concept.
result QQE allows for better discrimination of classes in some cases.
Maps can be embedded in higher dimensions if they lift to embeddings in product spaces.
problem Embedding maps in higher dimensions without self-intersections.
method Lifting maps to embeddings in product spaces.
result Maps can be embedded in higher dimensions if they lift to embeddings in product spaces.
New embeddings for manifolds using heat kernels.
problem Constructing canonical conformal embeddings for manifolds.
method Employing heat kernel embedding from Bérard-Besson-Gallot'94 to find canonical conformal embeddings.
result Intrinsic construction of canonical conformal embeddings with dimensions growing exponentially with t. Introduces PELP for graph-enhanced word embeddings.
problem Combining graph side-information into static word embeddings.
method Probabilistic embeddings using Laplacian priors.
result Unified and flexible approach to various embedding methods.
Curvature regularization prevents distortion in graph embeddings.
problem Graph topology patterns distort in Euclidean space, making detection difficult.
method Proposes curvature regularization to enforce flatness in embedding manifolds.
result Significant improvements in five embedding methods on open graph datasets.
Word embeddings are a powerful approach for unsupervised analysis of language. Recently, Rudolph et al. (2016) developed exponential family embeddings, which cast word embeddings in a probabilistic framework. Here, we develop dynamic embeddings, building on exponential family embeddings to capture how the meanings of w…
Proposes cone embedding for better graph hierarchical structure representation.
problem Lack of natural and interpretable hierarchical indicators in graph embeddings.
method Metric cone embedding method to capture hierarchical structure.
result Extracts hierarchical structure from other graph embedding outputs.
Classifies linear embeddings of grassmannians and ind-grassmannians.
problem Understanding linear embeddings of grassmannians and ind-grassmannians.
method Classification through isomorphism of Picard groups and direct limits.
result Most linear embeddings of grassmannians are equivariant.
BC-Aligner maintains backward compatibility of embeddings after frequent updates.
problem Updating embeddings without requiring consumer teams to retrain their models.
method Learning backward compatible embeddings through BC-Aligner.
result BC-Aligner maintains backward compatibility with existing unintended tasks after multiple model version updates.
Embedding calculus proves convergence for surfaces.
problem Proving convergence of embedding calculus for surfaces.
method Goodwillie-Weiss' embedding calculus for spaces of embeddings into a manifold of dimension at most two.
result Relates Johnson filtration of mapping class group to embedding calculus.
Unified framework for word embedding models using noise examples.
problem Improving word embedding models with negative sampling.
method Formulated a Word-Context Classification (WCC) framework that generalizes SkipGram word embedding models.
result The best noise distribution is the data distribution, improving both performance and training speed.
Models use embeddings and attention for better claim severity prediction.
problem Improving predictive modeling of claim severity with categorical data.
method Developed neural networks and attention-based architectures with embeddings.
result Attention-based models enhance predictive performance with contextual augmentation.
A fast graph embedding method for large graphs.
problem Efficiently embedding large graphs for various applications.
method One-hot graph encoder embedding with linear complexity.
result Graph encoder embedding is approximately normally distributed and converges to its mean.
Paper proves impossibility of three desirable properties in node embedding.
problem Understanding limitations of node embedding methods.
method Axiomatic approach to node embedding, proving impossibility of three properties.
result No node embedding method can satisfy all three desirable properties simultaneously.
The study characterizes and verifies equivariant embeddings of symmetric Kählerian manifolds.
problem Characterizing and verifying equivariant embeddings of symmetric Kählerian manifolds.
method Investigation motivated by Cartan and Wallach's theorem on symmetric spaces, focusing on CPn and parallel plurimean curvature. result If an equivariant embedding has parallel plurimean curvature, it is the extrinsically symmetric one.
Proves uniqueness of embedding complex manifold into infinite-dimensional space.
problem Balanced embedding of non-compact complex manifold into infinite-dimensional projective space.
method Fine estimates of asymptotics of a balanced embedding.
result Uniqueness of embedding proven.
The paper defines invariants for almost graph embeddings and explores their properties.
problem Understanding the properties and limitations of almost graph embeddings in the plane.
method Introducing and analyzing integer invariants (winding number, Wu numbers) for almost embeddings.
result Some values of invariants are realizable for almost embeddings but not for embeddings.
Recently, click-through rate (CTR) prediction models have evolved from shallow methods to deep neural networks. Most deep CTR models follow an Embedding\&MLP paradigm, that is, first mapping discrete id features, e.g. user visited items, into low dimensional vectors with an embedding module, then learn a multi-layer pe…
This work analyzes PPR-based node embeddings and their topological information.
problem Understanding and interpreting PPR-based node embeddings.
method Unified framework and two methods for topology recovery.
result PPR-based embeddings maintain more topological information than random walk-based embeddings.
Network representation learning in low dimensional vector space has attracted considerable attention in both academic and industrial domains. Most real-world networks are dynamic with addition/deletion of nodes and edges. The existing graph embedding methods are designed for static networks and they cannot capture evol…
For leveled spatial graphs, we find a surface embedding that allows cellular embedding.
problem Finding a surface embedding for general spatial graphs is not always possible.
method Define leveled property, decompose graph into subgraphs, and construct surface.
result For leveled spatial graphs with a small number of levels, a surface can always be found.
Long spacelike embeddings can be approximated by isometric ones.
problem Approximating long embeddings to isometric embeddings in Lorentzian spaces.
method Proving approximation by constructing C1 isometric embeddings. result Long spacelike embeddings can be C0-approximated by C1 isometric embeddings. A {\it wrinkled embedding} f:Vn→Wm is a topological embedding which is a smooth embedding everywhere on V except a set of (n−1)-dimensional spheres, where f has cuspidal corners. In this paper we prove that any rotation of the tangent plane field TV⊂TW of a {\it smoothly embedded} submanifold $V\s…
Graph embeddings have become a key and widely used technique within the field of graph mining, proving to be successful across a broad range of domains including social, citation, transportation and biological. Graph embedding techniques aim to automatically create a low-dimensional representation of a given graph, whi…
Graphs embeddable on torus and linklessly in 3D can be embedded linklessly in standard torus.
problem Embedding linklessly in a standard torus for graphs embeddable on torus and in 3D.
method Analyzing graphs of order 9 and below, showing linkless embedding in standard torus.
result For graphs of order 9 and below, linkless embedding in standard torus is possible.
Geometrically transforms word embeddings into a common space for better comparison.
problem Comparing embeddings from different sources is challenging.
method Applies orthogonal rotations and Mahalanobis scaling to transform embeddings into a shared latent space.
result The method improves word similarity and analogy tasks.
Obtaining continuous representations of structural data such as directed acyclic graphs (DAGs) has gained attention in machine learning and artificial intelligence. However, embedding complex DAGs in which both ancestors and descendants of nodes are exponentially increasing is difficult. Tackling in this problem, we de…
The paper proves nonexistence and existence results for minimal surfaces in R^4.
problem Proving nonexistence and existence of minimal surfaces in R^4.
method Analyzing complete minimal immersions with finite total curvature and embedded planar ends.
result Existence of embedded minimal spheres in R^4 with 3 embedded planar ends.
Recent works reveal that network embedding techniques enable many machine learning models to handle diverse downstream tasks on graph structured data. However, as previous methods usually focus on learning embeddings for a single network, they can not learn representations transferable on multiple networks. Hence, it i…
MCE reduces embedding instability in nonlinear dimensionality reduction.
problem Embedding instability caused by random initialization.
method Median of multiple embeddings (MCE) based on large deviation theory.
result MCE achieves consistency at an exponential rate and effectively mitigates instability.
Natural language processing has improved tremendously after the success of word embedding techniques such as word2vec. Recently, the same idea has been applied on source code with encouraging results. In this survey, we aim to collect and discuss the usage of word embedding techniques on programs and source code. The a…
Paper uses JIVE to decompose word embeddings, improving sentiment analysis performance.
problem Improving sentiment analysis performance on word embeddings.
method Joint and individual variance explained (JIVE) method for decomposition.
result Mapping word embeddings into joint components improves sentiment analysis performance.
Well-quasi-orders proved on embedded planar graphs.
problem Proving well-quasi-orders on embedded planar graphs.
method Careful analysis and extensions of classical methods for embedded minor relations.
result Embedded minor relations are well-quasi-orders on various classes of embedded planar graphs.