We study alignment in linear neural networks and its relation to gradient descent.
problem Understanding alignment in linear neural networks and its impact on training.
method Defined alignment for fully connected networks, analyzed alignment under gradient descent, and compared gradient descent to projected gradient descent for layer-constrained networks.
result Gradient descent can converge linearly to a global minimum when alignment is invariant, and alignment is impossible with large datasets in layer-constrained networks.
New graph kernel scales well with graph size and number, achieving state-of-the-art performance.
problem Graph kernels lose structure information when representing graphs.
method Proposes a positive-definite global alignment graph kernel using random features and random graph embeddings.
result Achieves quasi-linear scalability with respect to graph size and number.
We present Manifold Alignment Determination (MAD), an algorithm for learning alignments between data points from multiple views or modalities. The approach is capable of learning correspondences between views as well as correspondences between individual data-points. The proposed method requires only a few aligned exam…
This study presents the results of a series of simulation experiments that evaluate and compare four different manifold alignment methods under the influence of noise. The data was created by simulating the dynamics of two slightly different double pendulums in three-dimensional space. The method of semi-supervised fea…
Global existence and convergence proved for Kazdan-Warner equation with non-negative prescribed function.
problem Existence and convergence of solutions to the Kazdan-Warner equation on a closed Riemann surface.
method Global existence and convergence proved using additional assumptions on the prescribed function and the geometry of the surface.
result Global existence and convergence of solutions proved under specific conditions.
Study tests how U.S. equity prices align with global asset frequencies using financial variables.
problem Testing whether U.S. equity prices align with global asset frequencies using financial variables.
method Examines SPX and RUT gaps, uses OIS-based funding, volatility, trading-friction, financial-condition variables, and residual information.
result Gains in fit survive broad-dollar neutralization, alternative blocks, PCA, residualization, and nested horizon selection, supporting reduced-form P-Q alignment.
iREPA shows spatial structure, not global semantic, drives generation performance in REPA.
problem Understanding what aspect of the target representation matters for generation.
method Empirical analysis of 27 vision encoders, two modifications to REPA.
result Spatial structure, not global semantic, drives generation performance.
A new deep generative model captures global dependencies without supervision.
problem Global modeling in deep generative models.
method Non-i.i.d. variational autoencoders with mixture model and global Gaussian latent variable.
result Captures interpretable disentangled representations and domain alignment.
New method aligns states for better imitation learning.
problem Different dynamics models between imitator and expert.
method State alignment-based reinforcement learning.
result Superior performance on various imitation learning settings.
A novel distance measure aligns time series with feature and temporal variability.
problem Measuring similarity between time series with different features and dynamics.
method Learn a latent global transformation and temporal alignment in a joint optimization problem.
result Framework robustly aligns time series across various invariance classes.
New framework formalizes RLHF trilemma: improving safety, fairness, and robustness is computationally infeasible.
problem Aligning large language models with diverse human values while maintaining computational feasibility and robustness.
method Complexity-theoretic analysis integrating statistical learning theory and robust optimization.
result Achieving both representativeness (epsilon <= 0.01) and robustness (delta <= 0.001) for global-scale populations requires super-polynomial operations.
Longitudinal patient data has the potential to improve clinical risk stratification models for disease. However, chronic diseases that progress slowly over time are often heterogeneous in their clinical presentation. Patients may progress through disease stages at varying rates. This leads to pathophysiological misalig…
Foundation models fail to preserve continuous geometry, identified as the Geometric Alignment Tax.
problem Continuous geometry is lost in foundation models due to discrete categorical bottlenecks.
method Controlled ablations on synthetic systems and evaluation of 14 biological models using rate-distortion theory and MINE.
result Replacing cross-entropy with a continuous head reduces geometric distortion by up to 8.5x.
A new method for unsupervised domain adaptation using manifold learning.
problem Leveraging rich source domain information to target domain without labeled data.
method Discriminative Manifold Embedding and Alignment framework.
result Consistent transferability and discriminability achieved through manifold metric alignment.
The paper addresses rigid alignment of noisy patches, providing a polynomial time algorithm and convergence conditions.
problem Finding a rigid alignment of overlapping local views (patches) that minimizes alignment error in a noisy setting.
method Characterizes non-degeneracy based on kernel and positivity of a matrix, provides polynomial time algorithm for testing non-degeneracy, and uses Riemannian gradient descent for alignment.
result The algorithm converges locally linearly to a non-degenerate perfect alignment under certain conditions.
Early alignment in neural networks leads to sparse representations but hinders convergence.
problem The implicit bias of gradient descent during early training phases.
method Quantitative description of early alignment phase in small initialisation, one hidden layer networks.
result Early alignment induces a sparse representation but also hinders convergence to global minima.
HawkesLLM models text generation with temporal influence, improving semantic alignment under limited memory.
problem Path-dependent uncertainty in agentic text-simulation systems.
method HawkesLLM framework separates temporal influence modeling from text generation, using a multivariate Hawkes process and a language model.
result HawkesLLM improves late-stage semantic alignment under a compact prompt-memory budget.
Two semi-supervised manifold alignment methods improve cross-domain classification.
problem Aligning data from multiple sources for better analysis.
method SPUD and MASH methods using graph integration and diffusion.
result SPUD and MASH methods outperform existing methods in cross-domain classification.
Two-layer ReLU networks often converge to simpler solutions, improving generalization.
problem Understanding generalization in overparametrized neural networks, especially for complex tasks.
method Theoretical analysis of two-layer ReLU networks, focusing on the early alignment phase.
result Two-layer ReLU networks often converge to simpler solutions rather than interpolating the training data, leading to better generalization.
New framework assesses neural sensitivity to small perturbations.
problem Comparing neural representations' sensitivity to small changes.
method Local decodable information, Fisher information, and projected pullback/Fisher metric.
result Reveals differences in neural sensitivity not captured by activation alignment.
Geometric method captures rare topics and temporal alignment in co-author networks.
problem Missing rare topics and smooth temporal alignment in topic modeling.
method Integrates multimodal text and co-author network data using Hellinger distances and Ward's linkage.
result Effective identification of rare topics and visualization of topic drift over time.
Recent unsupervised approaches to domain adaptation primarily focus on minimizing the gap between the source and the target domains through refining the feature generator, in order to learn a better alignment between the two domains. This minimization can be achieved via a domain classifier to detect target-domain feat…
This paper improves GNNs' generalization by adding a Low-Rank Global Attention module.
problem Improving the generalization power of Graph Neural Networks (GNNs).
method Incorporating a Low-Rank Global Attention (LRGA) module into GNNs.
result Augmenting GNNs with LRGA aligns them with a powerful graph isomorphism test, 2-Folklore Weisfeiler-Lehman (2-FWL).
This paper studies nonlinear representation learning dynamics beyond the NTK regime.
problem Efficient reasoning and inference in raw sensory data representations.
method Identifies common model structure assumption and data-architecture alignment condition for global convergence and optimality.
result Theoretical framework explains network size effects and provides practical model structure guidelines.
NetOTC compares and aligns directed or undirected networks via random walk transitions.
problem Comparing and aligning networks of different types and sizes.
method NetOTC uses a transport-based approach to find optimal transition couplings of random walks.
result NetOTC quantifies network differences and provides vertex and edge alignments.
Paper resolves bias in ALFT training using generalized alignment games.
problem Systematic bias in estimating logarithmic rewards from small batches.
method Generalized Distributional Alignment Games, U-statistics, minimax polynomial estimators, Variance-Optimal Augmented Polynomial Optimization Program (AQP) Estimator.
result Proves optimal bias and accelerated convergence in ALFT training.
TMDA aligns subdomain data distribution discrepancies across domains using manifold representations.
problem Transfer learning challenges due to domain divergence.
method TMDA uses low-dimensional manifolds to represent subdomains and aligns local data distribution discrepancies across domains using M3D.
result TMDA is a promising method for various transfer learning tasks.
This work shows how exploiting gradient alignment can improve distributed and federated learning performance.
problem Misalignment of gradients across clients in distributed and federated learning.
method Utilizing implicit regularization through a novel GradAlign algorithm that induces gradient alignment with large mini-batches.
result Improvements in test accuracies and generalization performance.
SAIL-RevKL improves SAIL's convergence by regularizing the objective function.
problem Convergence of self-improving online LLM alignment algorithms.
method Proposed SAIL-RevKL, a regularized objective function to improve optimization landscape.
result Proved SAIL-RevKL satisfies the Polyak-Lojasiewicz (PL) condition with near-linear sample complexity.
Rewiring GNNs to optimize community and feature alignment improves their performance.
problem Improving GNNs' performance by addressing over-squashing and generalization issues.
method Three rewiring strategies: ComMa, FeaSt, and ComFy, targeting community structure, node labels, and their alignment.
result Rewiring strategies enhance GNNs' performance by optimizing label-community alignment.
Proposes a framework to improve domain adaptation without labeled data.
problem Improving adaptability and preserving intrinsic data structure in unsupervised domain adaptation.
method Discriminative Manifold Propagation framework using soft labels and manifold metric alignment.
result The method achieves better transferability and discriminability compared to existing approaches.
HSACC improves multi-view clustering of incomplete data.
problem Challenges in clustering incomplete multi-view data.
method Hierarchical Semantic Alignment and Cooperative Completion framework.
result HSACC outperforms state-of-the-art methods on benchmark datasets.
The alignment of a set of objects by means of transformations plays an important role in computer vision. Whilst the case for only two objects can be solved globally, when multiple objects are considered usually iterative methods are used. In practice the iterative methods perform well if the relative transformations b…
The recent advances in deep transfer learning reveal that adversarial learning can be embedded into deep networks to learn more transferable features to reduce the distribution discrepancy between two domains. Existing adversarial domain adaptation methods either learn a single domain discriminator to align the global …
Unified framework improves cross-corpus EEG emotion recognition by aligning prototypes and refining decision boundaries.
problem Cross-corpus EEG emotion recognition suffers from performance degradation due to physiological variability and device inconsistencies.
method Prototype-driven Adversarial Alignment (PAA) framework with three configurations: local, contrastive, and boundary-aware.
result State-of-the-art performance improvements across four cross-corpus evaluation protocols.
Enhances stock return prediction using LLMs and hybrid models.
problem Insufficient use of semantic information and alignment of LLMs with stock features.
method LG model with three strategies for global information modeling and SCRL for embedding alignment.
result Superior performance in Rank Information Coefficient and returns compared to models relying only on stock features.
Language is dynamic, constantly evolving and adapting with respect to time, domain or topic. The adaptability of language is an active research area, where researchers discover social, cultural and domain-specific changes in language using distributional tools such as word embeddings. In this paper, we introduce the gl…
A geometric theory explains loss functions for robust representation learning.
problem Treats robustness, domain adaptation, and sensor drift as separate literatures.
method Estimates covariance Sigma_task and uses it to pin Jacobian penalties.
result Proves optimality and necessity of range coverage for penalty matrices.
This paper explores two techniques to improve the performance of text-dependent speaker verification systems based on deep neural networks. Firstly, we propose a general alignment mechanism to keep the temporal structure of each phrase and obtain a supervector with the speaker and phrase information, since both are rel…
The paper studies 1-equivariant harmonic map flow behavior from R² to S².
problem Examining the long-term dynamics of 1-equivariant harmonic map flow.
method Constructing and analyzing global solutions for the flow equation.
result Global solutions exhibit trichotomy in long-time asymptotic behavior.
DTS improves inference-time alignment of diffusion models with less compute.
problem Inference-time alignment of diffusion models suffers from inaccurate value estimation and inefficient reuse of past computations.
method Diffusion Tree Sampling (DTS) uses a tree-based approach to propagate terminal rewards and iteratively refine value estimates.
result DTS produces asymptotically exact samples and matches the FID of best-performing baselines with up to 10x less compute.
Proposes a Carbon Equivalence Principle for financial products to align incentives and drive sustainability.
problem Align financial market incentives with carbon emissions to limit global warming.
method Introduces a Carbon Equivalence Principle requiring financial products to describe equivalent carbon flows alongside cash flows.
result Transparency of carbon flows in financial products can align incentives and reduce future costs, necessitating project re-structuring and financial net-zero designs.
Proposes methods to recover labels from shuffled networks using graph averages.
problem Recovering labels from a shuffled network using graph averages.
method Cluster networks into classes, then match the new graph to cluster-averages, minimizing the graph matching objective function.
result Higher fidelity matching performance when clustering networks into different classes.
New string kernels discover global properties through random feature maps, avoiding quadratic complexity.
problem Existing string kernels struggle with capturing long patterns, maintaining positive definiteness, and handling large datasets efficiently.
method Proposes a new class of global string kernels using random feature maps to discover global properties through global alignments, ensuring positive definiteness and linear computational cost.
result Random String Embeddings (RSE) achieve better or comparable accuracy to state-of-the-art methods, especially for longer strings.
We present a novel framework based on optimal transport for the challenging problem of comparing graphs. Specifically, we exploit the probabilistic distribution of smooth graph signals defined with respect to the graph topology. This allows us to derive an explicit expression of the Wasserstein distance between graph s…
This paper concerns rigidity results to Serrin's overdetermined problem in an epigraph {Δu+f(u)=0, in Ω={(x′,xn):xn>φ(x′)},u>0, in Ω,u=0, on ∂Ω,∣∇u∣=const.on∂Ω.. We prove that up to isometry the ep…
EoS selectively shapes learning, affecting some groups more than others.
problem EoS affects learning differently across the data distribution.
method Branching intervention to enter or exit EoS regime, controlled perturbation to isolate mechanisms.
result EoS redistributes learning, amplifying progress on some groups and suppressing others.
PPO algorithm converges to global optimality in multi-agent reinforcement learning.
problem Designing statistical guarantees for policy optimization methods in multi-agent reinforcement learning.
method Leveraging a multi-agent performance difference lemma, a localized action value function is used as a descent direction for each local policy, leading to a multi-agent PPO algorithm.
result The multi-agent PPO algorithm converges to the globally optimal policy at a sublinear rate under standard regularity conditions.