SGD updates align with a low-rank subspace but do not lead to further loss reduction.
problem Understanding the training dynamics of deep neural networks, particularly the role of the dominant subspace.
method Exploring whether neural networks can be trained within the dominant subspace of the loss Hessian.
result SGD updates, when projected onto the dominant subspace, do not decrease the training loss further, suggesting spurious alignment.
Faster WIND accelerates iterative BOND for LLM alignment.
problem Iterative BOND is inefficient in practice due to sample and computation inefficiency.
method Unified game-theoretic connection to self-play alignment, WIND framework with efficient algorithms.
result WIND variant achieves superior sample efficiency and faster computation.
AOT aligns LLMs on distributional preferences via optimal transport.
problem Current LLM alignment techniques lack distributional level alignment.
method Alignment via Optimal Transport (AOT) aligns LLMs on unpaired preference data.
result AOT enables alignment by penalizing reward distribution violations.
Study improves understanding and performance of FA learning rules in neural networks.
problem Lack of theoretical understanding and limited applications of Feedback Alignment (FA) methods.
method Introduces a unified framework linking synaptic weight changes to implicit regularization, providing convergence conditions and empirical evidence.
result Better alignment can enhance FA performance on complex multi-class tasks.
GENIE balances domain-invariant feature learning and gradient alignment for improved DG performance.
problem Domain Generalization (DG) overfitting to domain-specific features
method GENIE (Generalization-ENhancing Iterative Equalizer) optimizer
result Prevents a small subset of parameters from dominating optimization, promoting domain-invariant feature learning
Sequence-to-sequence text-to-speech (TTS) is dominated by soft-attention-based methods. Recently, hard-attention-based methods have been proposed to prevent fatal alignment errors, but their sampling method of discrete alignment is poorly investigated. This research investigates various combinations of sampling methods…
The paper improves alignment methods for deep neural networks using geometric and spectral analysis.
problem Improving alignment methods for deep neural networks.
method Geometric and spectral analysis of residual Jacobian chains.
result Deterministic and margin-verified results on the transport of dominant singular subspaces across layers.
Unified framework for aligning LLMs from human feedback.
problem Lack of strong theoretical justification for RLHF and inability to compare methods.
method Reframed alignment as distribution learning from pairwise preferences, proposing three principled objectives.
result Proposed objectives achieve strong non-asymptotic convergence to target LM.
Research on time-series similarity measures has emphasized the need for elastic methods which align the indices of pairs of time series and a plethora of non-parametric have been proposed for the task. On the other hand, deep learning approaches are dominant in closely related domains, such as learning image and text s…
Two-dimensional embeddings remain the dominant approach to visualize high dimensional data. The choice of embeddings ranges from highly non-linear ones, which can capture complex relationships but are difficult to interpret quantitatively, to axis-aligned projections, which are easy to interpret but are limited to biva…
New method accelerates neural network training by focusing on flat directions.
problem Improving neural network training speed and stability.
method Bulk-SGD, interpolated gradient methods.
result Updates along the Dominant subspace can accelerate convergence but compromise stability.
Time series data analytics has been a problem of substantial interests for decades, and Dynamic Time Warping (DTW) has been the most widely adopted technique to measure dissimilarity between time series. A number of global-alignment kernels have since been proposed in the spirit of DTW to extend its use to kernel-based…
Weibull weight-scale parameter λ evolves during AdamW training, with alignment, injection, and decay forces driving its growth and relaxation.
problem Understanding the evolution of the Weibull weight-scale parameter λ during AdamW training. method Deriving a leading-order three-force decomposition of the squared weight norm from AdamW updates.
result The alignment force dominates the rise phase, contributing 88-94% of the absolute force budget across four random seeds.
Study finds monthly SIPs outperform first-day SIPs in Nifty 50 by 0.5-2.5% annually.
problem Underexplored impact of SIP timing in India's equity market.
method 22-year analysis using multi-layered statistical framework (non-parametric tests, effect size metrics, SSD).
result Monthly SIPs (EXP-SIP) outperform first-day SIPs (FTD-SIP) by 0.5-2.5% annually over short-to-medium-term horizons.
Conformer encoder reverses sequence in time dimension, affecting decoder training.
problem Reversal of sequence in Conformer encoder impacts decoder training.
method Analyzed initial behavior of decoder cross-attention and proposed methods to avoid flipping.
result Self-attention module of Conformer starts dominating, allowing only reversed information to pass.
Method embeds numeric tabular datasets into a shared vector space for similarity and retrieval.
problem Lack of meaningful representation for numeric tabular datasets in large language models.
method Structured exploratory data analysis descriptors, sentence transformer embedding, CCA for cross-dataset alignment.
result Total P@1 score of 0.9 across 15 datasets, robust nearest-neighbor retrieval and cluster structure.
SpecGD mitigates misalignment in phase retrieval models with anisotropic inputs.
problem Misalignment during gradient descent in phase retrieval models with anisotropic inputs.
method Spectral gradient descent modifies gradient updates to preserve directional information and remove spike amplification.
result SpecGD removes spike amplification, leading to stable alignment and accelerated noise contraction.
A-GPS learns to generate Pareto sets efficiently with user preferences.
problem Online discrete multi-objective optimization with user preferences.
method Generative model with class probability estimator (CPE) for non-dominance and preference alignment.
result Amortized generative model for efficient Pareto set approximation.
SC-InfoNCE improves InfoNCE for feature clustering in contrastive learning.
problem Lack of theoretical understanding of InfoNCE's feature clustering mechanism.
method Introduced a transition probability matrix to model data augmentation dynamics and optimize feature similarity.
result SC-InfoNCE achieves strong performance across diverse domains, aligning feature similarity with downstream data.
New graph feedback model for bandits with improved regret bounds.
problem Understanding how graph structure affects regret in bandit problems.
method Introduced fractional weak domination number and k-packing independence number to capture upper and lower bounds on regret. Used strong duality theorem to derive upper and lower bounds. result Proved general upper and lower bounds on regret for various graph structures, showing tightness up to a logarithmic factor.
Paper formalizes anti-discrimination law in automated systems.
problem Algorithmic discrimination in legal contexts.
method Decision-theoretic framework grounded in UK anti-discrimination law.
result Introduced 'conditional estimation parity' metric for ML fairness.
As demonstrated during the recent financial crisis, regulators require additional analytical tools to assess systemic risk in the financial sector. This paper describes one such tool; namely a novel market modeling and analysis capability. Our model builds upon two leading market models: one which emphasizes market mic…
The paper reveals low-rank structure in neural network gradients, influenced by data and model parameters.
problem Investigating low-rank structure in gradients of neural networks under relaxed assumptions.
method Spiked data model, relaxation of isotropy assumptions, analysis of mean-field and neural-tangent-kernel scalings.
result Gradient of input weights is approximately low rank, dominated by two rank-one terms.
This work tackles the challenge of aligning generative models without explicit reward signals.
problem Aligning generative models without explicit reward signals.
method A Bilevel Optimization framework where the reward function is treated as the optimization variable of an outer-level problem.
result Theoretical analysis and insights generalize to tabular classification and model-based reinforcement learning.
Study analyzes accuracy of tensor deflation in noisy conditions.
problem Analyzing accuracy of tensor deflation in noisy conditions.
method Asymptotic study of Hotelling-type tensor deflation in large tensor dimensions.
result Characterization of estimated singular values and singular vector alignments.
Paper solves graph matching problem using convex relaxation to the simplex.
problem Finding the best alignment between two graphs.
method Introduces a new convex relaxation onto the unit simplex and uses mirror descent scheme.
result Shows exact recovery of ground truth permutation with high probability.
The paper develops methods to reduce deployment risk under dynamic covariate shifts.
problem Reduction of deployment risk under dynamic covariate shifts.
method Time-domain Poincare inequality and Jacobian-velocity theorem to identify and control directional tangent energy.
result Drift-aligned tangent regularization (DTR) reduces risk volatility and directional gain in low-rank drift regimes.
New benchmark predicts cardiometabolic risk from accelerometer data, with varying accuracy.
problem Lack of accurate tabular benchmarks for cardiometabolic risk from accelerometer data.
method Tabular learning methods (ridge regression, XGBoost, TabPFN v2) applied to NHANES data.
result TabPFN v2 achieves best performance, but triglycerides remain largely unpredictable.
Study on RL on volatility surfaces, proving no free lunch for law-seeking methods.
problem Aligning RL agents with no-arbitrage laws in volatile markets.
method Built a law manifold, defined penalties, and used a Goodhart decomposition.
result No free lunch theorem: Law-seeking RL cannot outperform baselines.
Connected domination numbers found for plane triangulations up to 13 vertices.
problem Finding connected domination numbers for plane triangulations.
method Analyzing triangulations of up to 13 vertices and proving the difference between connected and regular domination numbers can be arbitrarily large.
result Connected domination numbers for triangulations up to 13 vertices and upper bound for larger triangulations.
A geometric theory explains loss functions for robust representation learning.
problem Treats robustness, domain adaptation, and sensor drift as separate literatures.
method Estimates covariance Sigma_task and uses it to pin Jacobian penalties.
result Proves optimality and necessity of range coverage for penalty matrices.
Manifolds can be dominated by hypersurfaces in a sphere.
problem Dominating manifolds with hypersurfaces.
method Proving any smooth, closed, oriented manifold can be dominated by a codimension 1 submanifold of the sphere.
result Any smooth, closed, oriented manifold can be dominated by a codimension 1 submanifold of the sphere.
Understanding how funding and 4H context regulate crypto markets.
problem Analyzing the chaotic appearance of financial markets.
method Observing interactions between market context and capital conditions in the 4H timeframe.
result Ranges in crypto markets are strategic positioning by informed participants, not indecision.
New method ranks multivariate distributions in SMOOP using q-dominance.
problem Lack of reliable methods to rank multivariate distributions in SMOOP.
method Introduces center-outward q-dominance and develops empirical test procedures.
result Proves q-dominance implies FSD and establishes a sample size threshold.
Generative classifiers show surprising human-like performance.
problem Comparing generative and discriminative models for object recognition.
method Built on recent advances in generative modeling to create classifiers and compared them to discriminative models.
result Generative classifiers outperform discriminative models in several key areas, including shape bias and out-of-distribution accuracy.
We show that non-domination results for targets that are not dominated by products are stable under Cartesian products.
New framework for ranking distributions using variable fractional parameters.
problem Ordering distributions with varying steepness and local non-concavities.
method Introducing a function γ:Ro[0,1] to replace the fixed parameter in fractional SD. result Enables ranking of a broader range of distributions and incorporates dynamic greediness.
Research examines when 4-manifolds are dominated by geometric ones.
problem When is an orientable closed 4-manifold dominated by another?
method Focuses on geometric or fibred cases.
result Characterizes conditions for domination.
3-manifolds can virtually dominate others with positive simplicial volume.
problem Domination of 3-manifolds with positive simplicial volume.
method Proving existence of finite covers with degree-1 maps.
result Virtual domination of 3-manifolds with positive simplicial volume.
Neighborhood sampling affects graph neural network training outcomes.
problem Understanding the impact of neighborhood sampling on graph neural network training.
method Theoretical analysis using neural tangent kernels and Gaussian processes.
result Posterior covariance differs for different neighborhood sampling approaches, indicating no dominant approach.
Study domination between non-Fuchsian surface group representations and anti-de Sitter geometry.
problem Domination problem between non-Fuchsian representations of closed surface groups.
method Analysis of branched harmonic immersions and construction of anti-de Sitter 3-manifolds.
result Found that representations admitting branched harmonic immersions dominate other representations, and constructed large families of branched anti-de Sitter 3-manifolds.
GROOVE learns representations for weakly paired multimodal data.
problem Learning representations for high-content perturbation data with weakly paired samples.
method GroupCLIP contrastive loss integrated with an autoencoder framework.
result GROOVE performs on par with or outperforms existing approaches for cross-modal tasks.
Dominant knots have isomorphic Seifert and Tait graphs.
problem Understanding knot dominance through graph isomorphism.
method Examined alternating knots and their Seifert and Tait graphs.
result Isomorphic Seifert and Tait graphs indicate dominant knots.
New homology theory connects graph domination to subtle algebraic structures.
problem Understanding graph domination through algebraic homology.
method Interpreting überhomology as poset homology and showing its functorial properties.
result The Euler characteristic of bold homology equals the evaluation of the connected domination polynomial.
3-manifolds can be virtually dominated by maps of degree 8.
problem Understanding virtual domination of 3-manifolds.
method Proving existence of finite covers with specific map properties.
result Virtual 8-dominance of 3-manifolds.
Odd-dimensional manifolds have contact maps of non-zero degree.
problem Contact domination in odd-dimensional manifolds.
method Proving existence of maps from tight contact manifolds.
result Existence of non-zero degree maps from Liouville-fillable but not Weinstein-fillable contact manifolds.
Unified framework for efficient Frank-Wolfe optimization of Dominant Set Clustering.
problem Optimizing Dominant Set Clustering with various Frank-Wolfe algorithms.
method Unified framework for pairwise, standard, and away-steps Frank-Wolfe algorithms, with explicit convergence rates.
result Explicit convergence rates for Frank-Wolfe methods in Dominant Set Clustering.
New EI strategies using OWA and SSD for excess return.
problem Selecting EI portfolios that stochastically dominate a benchmark.
method Proposes a new OWA-based EI model and introduces a new SSD criterion.
result OWA-based EI portfolios stochastically dominate a benchmark and generate excess return.