Proposes a method for coarse graph alignment using sparse partial least squares.
problem Aligning graphs with community structures when there's no natural one-to-one mapping.
method Sparse partial least squares method incorporating observed graph structures and imposing sparsity.
result Demonstrates effectiveness in simulations.
The study explains how neural networks align their kernels to target functions during training.
problem Understanding how neural networks align their kernels to target functions during training.
method Theoretical analysis of kernel evolution in toy models and deep networks.
result Kernel alignment naturally emerges during training to accelerate convergence and improve generalization.
New method for robust distribution alignment using log-likelihood ratio and normalizing flows.
problem Distribution alignment challenges in deep learning.
method Log-likelihood ratio statistic and normalizing flows.
result Minimizing the proposed objective yields robust domain alignment.
Robustly aligns datasets with partial GW distance to handle contamination.
problem Aligning contaminated datasets using Gromov-Wasserstein distances.
method Proposes a partial GW distance estimator to minimize distortion from outliers.
result The partial GW distance estimator is minimax optimal and near-optimal in finite samples.
New method separates graph structure from node attributes to recover lost signal.
problem Standard representation learning on attributed graphs merges incompatible metric spaces, leading to geometrically flawed alignment.
method Custom variational autoencoder that separates manifold learning from structural alignment.
result Transforms geometric conflict into interpretable structural descriptor, uncovering connectivity patterns and anomalies.
Two semi-supervised manifold alignment methods improve cross-domain classification.
problem Aligning data from multiple sources for better analysis.
method SPUD and MASH methods using graph integration and diffusion.
result SPUD and MASH methods outperform existing methods in cross-domain classification.
Study finds no significant alignment between whitepaper claims and market structure.
problem Correlation between cryptocurrency whitepaper narratives and market behavior.
method Developed a contamination-aware pipeline for measuring structural correspondence, combining NLP classification and market statistics.
result No significant claims-market alignment detected in the sample.
New method aligns brain data across individuals for better brain decoding.
problem Inter-individual variability in brain response patterns limits decoder generalization.
method SpectralOT method that embeds cortical geometry into Laplace-Beltrami eigenmodes.
result SpectralOT strikes balance between aligning functional features and preserving anatomical structure.
In many machine learning applications, it is necessary to meaningfully aggregate, through alignment, different but related datasets. Optimal transport (OT)-based approaches pose alignment as a divergence minimization problem: the aim is to transform a source dataset to match a target dataset using the Wasserstein dista…
This paper improves GNN robustness by aligning feature and adjacency matrix learning.
problem Improving robustness of graph neural networks (GNN) in noisy graph data.
method Proposes a novel regularized GSL approach that aligns feature information and graph information, incorporating sparse dimensional reduction.
result Demonstrates superior performance in noisy graph structures compared to competitive baselines.
Graph-Relational Domain Adaptation (GRDA) adapts domains based on their graph structure.
problem Uniform alignment of domains ignores topological structures.
method Uses a domain graph to encode adjacency and a novel graph discriminator.
result Empirically shows improved generalization and domain information incorporation.
Neural networks have succeeded in many reasoning tasks. Empirically, these tasks require specialized network structures, e.g., Graph Neural Networks (GNNs) perform well on many such tasks, but less structured networks fail. Theoretically, there is limited understanding of why and when a network structure generalizes be…
A novel method for comparing graphs of different sizes using Wasserstein distance.
problem Comparing non-aligned graphs of varying sizes.
method Optimal transport in graph comparison framework, solving a one-to-many assignment problem.
result Significant improvements in graph alignment and classification tasks.
Rewiring GNNs to optimize community and feature alignment improves their performance.
problem Improving GNNs' performance by addressing over-squashing and generalization issues.
method Three rewiring strategies: ComMa, FeaSt, and ComFy, targeting community structure, node labels, and their alignment.
result Rewiring strategies enhance GNNs' performance by optimizing label-community alignment.
This paper improves cross-domain learning using random forests for manifold alignment.
problem Improving cross-domain learning and feature integration.
method Semi-supervised manifold alignment using random forest proximities.
result Random forest proximities enhance downstream classification accuracy.
New study reveals task alignment is key to ICL performance.
problem Understanding how task alignment affects generalization in in-context learning.
method Derived an exact expression for ICL generalization error in high dimensions under task covariance mismatch.
result Identified train-test task alignment as a key determinant of generalization in ICL.
SMAI framework tests and integrates single-cell data alignability.
problem Lack of a rigorous statistical test for alignability and distortion during alignment.
method Spectral manifold alignment and inference (SMAI) framework.
result SMAI outperforms existing methods in alignability testing and integration.
The paper addresses rigid alignment of noisy patches, providing a polynomial time algorithm and convergence conditions.
problem Finding a rigid alignment of overlapping local views (patches) that minimizes alignment error in a noisy setting.
method Characterizes non-degeneracy based on kernel and positivity of a matrix, provides polynomial time algorithm for testing non-degeneracy, and uses Riemannian gradient descent for alignment.
result The algorithm converges locally linearly to a non-degenerate perfect alignment under certain conditions.
Alignment of neural network representations is influenced by SNR and sample size.
problem Understanding how neural network representations align across different conditions.
method Controlled training of neural networks on perturbed datasets, analyzing alignment and generalization.
result Alignment varies monotonically with SNR but non-monotonically with sample size, with minimal alignment near the interpolation threshold.
Study Einstein metrics on aligned homogeneous spaces with maximal third Betti number.
problem Existence and classification of Einstein metrics on specific homogeneous spaces.
method Analysis of isotropy representation and computation of Ricci curvature.
result Computation of Ricci curvature formulas for aligned homogeneous spaces.
MALI aligns distinct domains using labeled data.
problem Aligning multi-domain data for machine learning.
method MALI learns manifold structure via diffusion and uses labeled data to guide alignment.
result MALI outperforms state-of-the-art methods across multiple datasets.
Gromov-Wasserstein (GW) is a powerful tool to compare probability measures whose supports are in different metric spaces. GW suffers however from a computational drawback since it requires to solve a complex non-convex quadratic program. We consider in this work a specific family of cost metrics, namely \textit{tree me…
Linear attention in Transformers can be interpreted as dynamic VAR models.
problem Misalignment between Transformers and autoregressive forecasting objectives.
method Interpreting linear attention as VAR, rearranging MLP, attention, and flow.
result SAMoVAR improves performance, interpretability, and efficiency.
A new method for aligning datasets without known correspondences.
problem Aligning datasets from different domains without labeled correspondences.
method Integrates MDS and Wasserstein Procrustes for joint optimization of embeddings and correspondences.
result Maps datasets to a common low-dimensional space without labeled correspondences.
DTA aligns multimodal data with prior correspondence knowledge.
problem Challenges in aligning data from different domains.
method Semi-supervised manifold alignment method exploiting prior correspondence knowledge.
result DTA outperforms other methods in aligning multimodal data.
DFA trains deep networks by aligning weights then memorizing data.
problem Understanding why DFA works for some networks but not others.
method Two-step learning process: alignment followed by memorization.
result DFA aligns weights to maximize gradient alignment, breaking degeneracy.
A novel federated learning framework resolves structural misalignment in model fusion.
problem Structural misalignment in model fusion due to chaotic information distribution.
method Feature-oriented regulation method (Ψ-Net) to ensure feature information allocation and dedicated collaboration schemes. result Effective enhancement of federated learning applicability to heterogeneous settings with improved convergence speed, accuracy, and efficiency.
Two-dimensional embeddings remain the dominant approach to visualize high dimensional data. The choice of embeddings ranges from highly non-linear ones, which can capture complex relationships but are difficult to interpret quantitatively, to axis-aligned projections, which are easy to interpret but are limited to biva…
Direct Density Ratio Optimization aligns LLMs with human preferences without assuming specific models.
problem Statistical inconsistency in aligning LLMs with human preferences.
method Direct Density Ratio Optimization (DDRO) estimates density ratio directly.
result DDRO is statistically consistent, converging to true human preferences as data grows.
Two impossibility theorems show formal alignment certification is impossible for AI systems.
problem Formal certification of AI alignment over open-ended domains is impossible.
method Two independent impossibility theorems: Semantic and Statistical barriers.
result No procedure can simultaneously satisfy soundness, completeness, and tractability.
Geometric method captures rare topics and temporal alignment in co-author networks.
problem Missing rare topics and smooth temporal alignment in topic modeling.
method Integrates multimodal text and co-author network data using Hellinger distances and Ward's linkage.
result Effective identification of rare topics and visualization of topic drift over time.
Classifies surfaces supporting alignable nets with geodesic and conjugate properties.
problem Classifying surfaces with specific geometric properties.
method Cartan's theory of moving frames, coordinate-free classification, explicit immersion formulas.
result Two classes of alignable Voss surfaces, each with two two-parameter families, including one with an isothermal-conjugate geodesic net.
Unified framework simplifies DPO algorithms for LLM alignment.
problem Vast number of DPO variants complicates model alignment.
method Mutual information inspired unifying framework with flexible priors.
result Many DPO variants can be derived from the new framework.
Zero-shot learning (ZSL) aims at recognizing unseen classes with knowledge transferred from seen classes. This is typically achieved by exploiting a semantic feature space (FS) shared by both seen and unseen classes, i.e., attributes or word vectors, as the bridge. However, due to the mutually disjoint of training (see…
G-CREWE efficiently aligns large networks using node embeddings and compression.
problem Efficiently aligning large networks for various applications.
method Uses node embeddings and compression to align networks at fine and coarse resolutions.
result G-CREWE achieves efficient and accurate network alignment, twice as fast as existing methods.
High-dimensional models pose both safety benefits and risks.
problem Emergent problems in safety alignment due to high-dimensional representations.
method Detailed visualizations and lower-dimensional subspace projections.
result Dimensional reduction preserves safety alignment while avoiding linear jailbreaking.
In this paper, we develop a new aligned vertex convolutional network model to learn multi-scale local-level vertex features for graph classification. Our idea is to transform the graphs of arbitrary sizes into fixed-sized aligned vertex grid structures, and define a new vertex convolution operation by adopting a set of…
Geometric framework for aligning fiber tracts across subjects.
problem Challenges in finding direct tract correspondence across multiple individuals.
method Geometric framework using intrinsic mean and deformation fields, parallel transport for registration.
result Bundle alignment results on 43 healthy adult subjects.
Proposes a new method for manifold alignment using geometry-regularized twin autoencoders.
problem Traditional MA methods lack out-of-sample extension and real-world applicability.
method Guided representation learning with geometry-regularized twin autoencoders.
result Improves cross-domain generalization and robustness while maintaining alignment fidelity.
New statistical model improves protein alignment accuracy.
problem Improving accuracy in protein sequence alignment.
method Inferred a time-dependent Markov process from protein sequences.
result Developed a new optimal matrix (MMLSUM) for protein alignment.
Paper tackles hidden game problem in AI alignment and language games.
problem Hidden game problem in AI alignment and language games.
method Developed a composition of regret minimization techniques to discover and exploit hidden structures.
result Achieved optimal external and swap regret bounds for rapid convergence to correlated equilibria.
Improved algorithm for network alignment with binary diffing application.
problem Network alignment problem and binary diffing.
method Inspired by Bayati et al. [2], the algorithm uses message passing with modifications for speed and convergence.
result Our model outperforms state-of-the-art solvers in experiments.
We propose a novel framework for combining datasets via alignment of their intrinsic geometry. This alignment can be used to fuse data originating from disparate modalities, or to correct batch effects while preserving intrinsic data structure. Importantly, we do not assume any pointwise correspondence between datasets…
Proposes a method to compare noisy high-dimensional datasets with low-dimensional manifolds.
problem Comparing distributions on manifolds in noisy high-dimensional datasets.
method Linking low-rank structure to manifold geometry, developing a scale-invariant distance measure.
result Superior robustness and statistical power compared to existing methods.
Proposes EOT eigenmaps for aligning and embedding multiple datasets.
problem Aligning and embedding multiple datasets with shared structures but individual distortions.
method Entropic Optimal Transport (EOT) eigenmaps, leveraging leading singular vectors of EOT plan matrix.
result Proves theoretical guarantees and favorable properties for aligning and embedding datasets.
Gradient descent aligns neural feature matrices with pre-activation tangent features.
problem Understanding neural feature learning mechanisms.
method Analytical proof of alignment between weight matrices and pre-activation tangent features.
result Derivative alignment occurs almost surely in high-dimensional settings.
Statistical-computational gap found in aligning multiple Gaussian graphs.
problem Aligning multiple Gaussian graphs with unknown signals.
method Generalized informational threshold and computational barrier analysis.
result Existence of a statistical-computational gap in multiple Gaussian graph alignment.
iREPA shows spatial structure, not global semantic, drives generation performance in REPA.
problem Understanding what aspect of the target representation matters for generation.
method Empirical analysis of 27 vision encoders, two modifications to REPA.
result Spatial structure, not global semantic, drives generation performance.