This study presents the results of a series of simulation experiments that evaluate and compare four different manifold alignment methods under the influence of noise. The data was created by simulating the dynamics of two slightly different double pendulums in three-dimensional space. The method of semi-supervised fea…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “noise alignment”
Alignment of neural network representations is influenced by SNR and sample size.
problem Understanding how neural network representations align across different conditions.
method Controlled training of neural networks on perturbed datasets, analyzing alignment and generalization.
result Alignment varies monotonically with SNR but non-monotonically with sample size, with minimal alignment near the interpolation threshold.
Hölder-DPO aligns models robustly with noisy human feedback.
problem No existing alignment methods can handle severe label noise.
method Proposes Hölder-DPO, a principled alignment loss with provable redescending property.
result Hölder-DPO enables scalable human feedback valuation and improves model alignment.
The paper analyzes how noise geometry influences the performance of SGD in machine learning.
problem Understanding how noise geometry affects the performance of stochastic gradient descent.
method Developed two metrics to quantify noise alignment strength and analyzed their effects on loss and subspace projection dynamics.
result Noise geometry can be used to guarantee alignment under certain conditions, aiding SGD's ability to escape from sharp minima.
Traditional pairwise sequence alignment is based on matching individual samples from two sequences, under time monotonicity constraints. However, in many application settings matching subsequences (segments) instead of individual samples may bring in additional robustness to noise or local non-causal perturbations. Thi…
AlignFlow improves FGMs by optimizing noise and data alignment.
problem Optimal Transport methods for FGMs are limited by scalability issues.
method Introduces Semi-Discrete Optimal Transport (SDOT) to enhance FGM training.
result AlignFlow scales well to large datasets and model architectures.
The paper explores how symmetries and noise in SGD influence parameter dynamics.
problem Understanding the dynamics of parameter updates in SGD with symmetries.
method Proved the existence of noise equilibria and showed their role in balancing gradient noise.
result Gradient noise creates a systematic motion of parameters to a unique fixed point, called noise equilibria.
Proposes a method to compare noisy high-dimensional datasets with low-dimensional manifolds.
problem Comparing distributions on manifolds in noisy high-dimensional datasets.
method Linking low-rank structure to manifold geometry, developing a scale-invariant distance measure.
result Superior robustness and statistical power compared to existing methods.
The paper improves alignment methods for deep neural networks using geometric and spectral analysis.
problem Improving alignment methods for deep neural networks.
method Geometric and spectral analysis of residual Jacobian chains.
result Deterministic and margin-verified results on the transport of dominant singular subspaces across layers.
RESTA defends LLMs against jailbreaking attacks by adding random noise to embeddings.
problem Vulnerability of LLMs to jailbreaking attacks that generate harmful outputs.
method Adds random noise to embedding vectors and aggregates during token generation.
result RESTA achieves superior robustness versus utility tradeoffs compared to baseline defenses.
Study analyzes Hotelling-type tensor deflation for spiked tensors, providing insights into signal and noise.
problem Characterizing singular values and alignments in Hotelling-type tensor deflation.
method Asymptotic study of Hotelling-type tensor deflation in large dimensional regime using random tensor theory.
result Characterization of singular values and alignments at each step of the deflation procedure.
We analyze why some models resist unlearning using linear stability theory.
problem Understanding and predicting when machine learning models resist unlearning.
method Linear stability theory applied to machine learning models, focusing on data coherence and optimization dynamics.
result Data coherence and signal-to-noise ratio (SNR) influence unlearning resistance; lower SNR makes unlearning easier.
New method designs joint initial noises for diffusion models to improve diversity and alignment.
problem Independent initial noises limit diversity in generated images.
method Coupling of initial noises, maintaining Gaussian distribution while allowing dependence.
result Repulsive Gaussian coupling improves diversity without increasing sampling cost.
Latent Noise Injection improves synthetic data generation for privacy and statistical alignment.
problem Slow convergence of generative models in high-dimensional settings.
method Latent Noise Injection using Masked Autoregressive Flows (MAF).
result Synthetic data closely reflects the underlying distribution, especially in high-dimensional settings.
Improved graph matching algorithm robust to noise.
problem Finding a bijection between vertex sets of two graphs.
method Uses multistage signature vectors to match vertices.
result Recover matching exactly with high probability for .
We examine the influence of input data representations on learning complexity. For learning, we posit that each model implicitly uses a candidate model distribution for unexplained variations in the data, its noise model. If the model distribution is not well aligned to the true distribution, then even relevant variati…
The alignment of a set of objects by means of transformations plays an important role in computer vision. Whilst the case for only two objects can be solved globally, when multiple objects are considered usually iterative methods are used. In practice the iterative methods perform well if the relative transformations b…
Training More Robust Classification Model via Discriminative Loss and Gaussian Noise Injectionstat.ML
Enhances deep learning robustness to noise without sacrificing clean data accuracy.
problem Robustness of deep neural networks to input noise.
method Discriminative loss at penultimate layer and class-wise feature alignment with Gaussian noise.
result Improves robustness to various perturbations without degrading clean data accuracy.
This paper improves GNN robustness by aligning feature and adjacency matrix learning.
problem Improving robustness of graph neural networks (GNN) in noisy graph data.
method Proposes a novel regularized GSL approach that aligns feature information and graph information, incorporating sparse dimensional reduction.
result Demonstrates superior performance in noisy graph structures compared to competitive baselines.
PCA++ improves robustness to background noise in contrastive learning.
problem Recovering shared signal subspaces from positive pairs in high-dimensional data with structured background noise.
method PCA++ uses hard uniformity-constrained contrastive learning to enforce identity covariance on projected features.
result PCA++ outperforms standard PCA and alignment-only PCA+ in simulations and real-world datasets.
In machine learning the best performance on a certain task is achieved by fully supervised methods when perfect ground truth labels are available. However, labels are often noisy, especially in remote sensing where manually curated public datasets are rare. We study the multi-modal cadaster map alignment problem for wh…
LAAT detects multiple low-density manifolds in noisy data.
problem Detecting multiple low-density manifolds in noisy data.
method Locally Aligned Ant Technique (LAAT) based on Ant Colony Optimization.
result LAAT recovers multiple manifolds in extremely noisy data.
Proposes EOT eigenmaps for aligning and embedding multiple datasets.
problem Aligning and embedding multiple datasets with shared structures but individual distortions.
method Entropic Optimal Transport (EOT) eigenmaps, leveraging leading singular vectors of EOT plan matrix.
result Proves theoretical guarantees and favorable properties for aligning and embedding datasets.
Unbalanced COOT improves feature alignment robustly to outliers.
problem Optimal transport methods are sensitive to outliers in real-world data.
method COOT infers alignment between features and samples, unbalanced COOT adds robustness.
result Unbalanced COOT is robust to noise in real-world datasets.
We introduce BilBOWA (Bilingual Bag-of-Words without Alignments), a simple and computationally-efficient model for learning bilingual distributed representations of words which can scale to large monolingual datasets and does not require word-aligned parallel training data. Instead it trains directly on monolingual dat…
The paper addresses rigid alignment of noisy patches, providing a polynomial time algorithm and convergence conditions.
problem Finding a rigid alignment of overlapping local views (patches) that minimizes alignment error in a noisy setting.
method Characterizes non-degeneracy based on kernel and positivity of a matrix, provides polynomial time algorithm for testing non-degeneracy, and uses Riemannian gradient descent for alignment.
result The algorithm converges locally linearly to a non-degenerate perfect alignment under certain conditions.
SGD in DLNs reveals feature learning dynamics.
problem Understanding SGD dynamics in DLNs during saddle-to-saddle training.
method Stochastic Langevin dynamics with anisotropic, state-dependent noise; one-dimensional per-mode SDEs; Boltzmann distribution approximation.
result SGD noise encodes feature learning progression but does not alter saddle-to-saddle dynamics.
New framework aligns latent representations over-the-air using intelligent metasurfaces.
problem Heterogeneous transmitter-receiver models produce misaligned latent representations in semantic communication.
method Intelligent metasurfaces (SIM) emulate supervised and zero-shot semantic aligners directly in the wave domain.
result SIMs achieve up to 90% task accuracy in high SNR regimes, robust to low SNR.
Researchers prove it's impossible to partially recover graph alignments in certain conditions.
problem Recovering vertex correspondence between two random graphs with correlated edges.
method Used the probabilistic method to build automorphisms between tree components of a subcritical Erdös-Rényi graph.
result Proved an impossibility result for partial recovery in the sparse regime with constant average degree and correlation.
Study analyzes accuracy of tensor deflation in noisy conditions.
problem Analyzing accuracy of tensor deflation in noisy conditions.
method Asymptotic study of Hotelling-type tensor deflation in large tensor dimensions.
result Characterization of estimated singular values and singular vector alignments.
Training a source model optimally for its own task is suboptimal for downstream transfer.
problem The optimality of a source model for its own task hinders downstream transfer performance.
method Analyzes L2-SP ridge regression, characterizes transfer-optimal source penalty, and identifies alignment-dependent effects.
result Transfer benefits from stronger source regularization when aligned imperfectly, and from weaker regularization when aligned perfectly.
Demon aligns diffusion models without retraining or backpropagation.
problem Aligning diffusion models with user preferences.
method Stochastic optimization to control noise distribution.
result Significantly improves aesthetics scores for text-to-image generation.
Partial soft-matching distance improves neural representation comparison by allowing some neurons to remain unmatched.
problem Neural representations are noisy and contain outliers, making traditional matching methods unreliable.
method Extends soft-matching distance to a partial optimal transport setting, allowing some neurons to remain unmatched.
result Partial soft-matching provides robust correspondences that are more reliable under noise and outliers.
Study on estimating signals from shifted and noisy copies in high dimensions, revealing a phase transition.
problem Estimating a signal in high-dimensional space from its circularly-shifted and noisy copies.
method Analysis of sample complexity in the high-dimensional regime, focusing on the parameter α.
result A phase transition phenomenon governed by α, with different sample complexities based on α values.
The alignment property of SGD noise and how it helps select flat minima: A stability analysisstat.ML
SGD noise helps select flat minima by concentrating in sharp directions and being proportional to loss value.
problem Understanding the implicit regularization of SGD and selecting flat minima in over-parameterized models.
method Relating SGD's linear stability to the Frobenius norm of the Hessian and analyzing the alignment property of SGD noise.
result Flat minima are linearly stable for SGD, and their sharpness is bounded independently of model size and sample size.
New method flattens decision boundary by targeting shortcut-aligned axes in disentangled latent space.
problem Shortcut learning in neural networks, leading to poor out-of-distribution generalization.
method Injects targeted anisotropic noise to regularize classifier sensitivity along shortcut-aligned axes.
result Achieves state-of-the-art OOD performance without shortcut labels or conflicting samples.
The paper analyzes deflation for estimating a low-rank spike in large tensors with noise.
problem Estimating a low-rank symmetric spike in large tensors with additive Gaussian noise.
method Characterization of deflation performance in terms of vector alignments and weights.
result Understanding deflation mechanism in noisy conditions and designing more efficient methods.
This work explores how neural network architecture affects robustness to noisy labels.
problem The impact of neural network architecture on robustness to noisy labels.
method Formal framework connecting robustness to architecture alignments, measured by predictive power in representations.
result Network robustness to noisy labels improves when its architecture is more aligned with the target function.
Enhances AI models with human feedback for noisy data.
problem Improving AI model alignment with human feedback in noisy environments.
method Two-stage SL+LHF framework connecting machine learning with human feedback.
result The LNCA ratio identifies conditions for SL+LHF superiority over pure SL.
Adversarial training makes logistic regression weight loss landscapes sharper.
problem Understanding why adversarial training sharpens the weight loss landscape in logistic regression.
method Theoretical analysis of linear logistic regression model with L2 norm constraints, and experiments on ResNet18.
result Adversarial training sharpens the weight loss landscape in linear logistic regression models.
EBRM improves robustness and generalization of language model rewards.
problem Challenges in capturing complex human preferences and generalizing to unseen data in reward models.
method Energy-Based Reward Model (EBRM) that models reward distribution explicitly, using conflict-aware data filtering, label-noise-aware contrastive training, and hybrid initialization.
result Significant improvements in robustness and generalization, up to 5.97% improvement in safety-critical alignment tasks.
Label noise in adversarial training leads to robust overfitting, explained and mitigated.
problem Label noise in adversarial training causes robust overfitting.
method Proposed a method to automatically calibrate labels.
result Consistent performance improvements across various models and datasets.
The presence of noise is common in signal processing regardless the signal type. Deep neural networks have shown good performance in noise removal, especially on the image domain. In this work, we consider deep neural networks as a denoising tool where our focus is on one dimensional signals. We introduce an encoder-de…
Novel PO algorithms improve LLM alignment tasks.
problem Evaluating PO algorithms on LLM alignment is costly and noisy.
method Designed a diagnostic suite of MuJoCo tasks and datasets, used evolutionary strategies to discover specialized PO algorithms.
result Proposed MPO algorithms significantly outperform existing PO algorithms in LLM alignment.
This work optimizes signal estimation for sparse MRA with collision-free signals.
problem Recovering an unknown signal from repeated observations under cyclic isometries with high noise.
method Investigates minimax optimality for collision-free signals in the MRA model.
result The minimax optimal rate of estimation is \( \sigma^2/\sqrt{n} \) for sparse MRA.
Grad-TTS models speech from text using diffusion probabilistic techniques.
problem Creating high-quality speech from text input.
method Score-based decoder with stochastic differential equations for noise-to-speech transformation.
result Grad-TTS produces mel-spectrograms from text input with competitive quality.
Understanding the behavior of stochastic gradient descent (SGD) in the context of deep neural networks has raised lots of concerns recently. Along this line, we study a general form of gradient based optimization dynamics with unbiased noise, which unifies SGD and standard Langevin dynamics. Through investigating this …
Spectral Gradient Descent Mitigates Anisotropy-Driven Misalignment: A Case Study in Phase Retrievalstat.ML
SpecGD mitigates misalignment in phase retrieval models with anisotropic inputs.
problem Misalignment during gradient descent in phase retrieval models with anisotropic inputs.
method Spectral gradient descent modifies gradient updates to preserve directional information and remove spike amplification.
result SpecGD removes spike amplification, leading to stable alignment and accelerated noise contraction.