The paper proposes noise-invariant distances and features for robust testing and learning.
problem Testing and learning on distributions with irrelevant noise.
method Kernel embeddings, Maximum Mean Discrepancy, distances invariant to additive symmetric noise.
result Noise-invariant distances and features for robust testing and learning.
Paper tackles label noise in transfer learning, proposing a new framework.
problem Label noise in source domain affects learning of invariant representations and correcting label shift.
method Proposes Denoising Conditional Invariant Component (DCIC) framework to handle noisy labels.
result Proposed framework ensures extraction of invariant representations and unbiased estimation of target domain labels.
New auto-encoder handles varying noise levels without retraining.
problem Auto-encoders degrade in noisy conditions.
method Formalized auto-encoders as transform learning, derived new architecture.
result Models generalize well to different noise levels.
New algorithm for signal estimation in noisy matrix models.
problem Signal estimation in rectangular spiked matrix models with rotationally invariant noise.
method Orthogonal Approximate Message Passing (OAMP) algorithm for signal estimation.
result Optimal OAMP algorithm minimizes mean-squared error and achieves Bayes-optimal performance.
SGD tends to favor simpler subnetworks, improving generalization.
problem SGD's tendency to favor simpler subnetworks over complex ones.
method Identifying invariant sets and analyzing SGD's behavior around them.
result SGD collapses networks to simpler subnetworks, improving generalization.
Study analyzes perturbations in singular subspaces under random noise.
problem Understanding singular vector and subspace changes in signal-plus-noise models.
method Generalized Davis-Kahan-Wedin theorem for any unitarily invariant norm, considering ℓ∞ and ℓ2,∞ bounds. result Fine-grained insights into singular vector and subspace perturbations, including ℓ∞ and ℓ2,∞ bounds. New algorithms improve rank one signal estimation from noisy data.
problem Estimating a rank one signal matrix from corrupted data with rotationally invariant noise.
method Developed approximate message-passing algorithms exploiting eigenvalues and iterates denoisers.
result Achieves optimal asymptotic estimation error among iterative algorithms.
Study examines the training process of an unsupervised learning model for detecting gravitational-wave transient noise.
problem Transient noise in gravitational-wave detector data causes instability and signal overlap.
method Unsupervised deep learning with variational autoencoder and invariant information clustering applied to the Gravity Spy dataset.
result Training process of the unsupervised learning architecture is examined and reported.
In nonlinear latent variable models or dynamic models, if we consider the latent variables as confounders (common causes), the noise dependencies imply further relations between the observed variables. Such models are then closely related to causal discovery in the presence of nonlinear confounders, which is a challeng…
Bayes-optimal limits in PCA with structured noise are determined.
problem Analyzing statistical dependencies in measurement noise for high-dimensional inference.
method Study of spiked matrix model with low-order polynomial orthogonal noise, providing Bayes-optimal limits and proposing a novel AMP.
result A novel AMP algorithm reaches the information-theoretic limits for more general priors.
The distribution of Chern-Simons invariants on 3-manifolds resembles quadratic residues.
problem Distribution of Chern-Simons invariants on 3-manifolds.
method Analyzing the values of the Chern-Simons function on conjugacy classes of representations.
result Chern-Simons invariants tend to become equidistributed on the circle with white noise fluctuations.
Noise2Self removes noise from data without clean signals or noise estimates.
problem Removing noise from high-dimensional data without prior signal or noise information.
method Noise independence across dimensions allows self-supervised estimation of denoising performance.
result General framework calibrates denoising algorithms from noisy data alone.
Rotation invariant algorithms fail on sparse problems even with noise.
problem Rotation invariant algorithms' suboptimality in sparse linear problems with noise.
method Lower bounds and trajectory analysis of optimization algorithms.
result Rotation invariant algorithms are suboptimal even with noise and many examples.
Study non-asymptotic bounds on correlation in high-dimensional linear systems, revealing invariant subspaces and bottlenecks.
problem Understanding correlation and mixing in high-dimensional linear systems with Gaussian noise.
method Sampling from sub-trajectories, using Talagrand's inequality, and analyzing invariant subspaces.
result Large discrepancy between algebraic and geometric multiplicity leads to bottlenecks between invariant subspaces.
Spectral clustering performance depends on eigenvector fluctuations, shown to be Gaussian.
problem Predicting the performance of spectral clustering.
method General spike random matrix model and rotational invariance of noise.
result Fluctuations of eigenvector entries are Gaussian in large-dimensional regime.
The paper proposes a method to create robust speech recognition models that work well under noisy conditions.
problem Creating robust speech recognition models that perform well under various noise conditions.
method The approach involves learning invariant feature representations using ideas from image generation and domain adaptation.
result The proposed method generalizes better than standard training methods, especially in noisy conditions.
Discover conservation laws from trajectories using a neural network.
problem Finding invariants and conservation laws from large-scale data without prior knowledge.
method ConservNet, a neural network trained with noise-variance loss to discover hidden invariants in grouped multi-dimensional observables.
result Successfully discovers underlying invariants from simulated and real-world systems.
Study on financial systems using perturbed unimodal maps with heteroscedastic noise.
problem Analyzing systemic risk in financial systems using mathematical models.
method Investigation of one-dimensional unimodal maps perturbed by heteroscedastic noise, proving stability, convergence, and Lyapunov exponent continuity.
result Continuous dependence of average Lyapunov exponent on Markov chain parameters, and Gumbel's law for extreme values.
Unsupervised learning classifies transient noise in gravitational wave detectors.
problem Transient noise interferes with gravitational wave signals, causing instability.
method Combines variational autoencoder and invariant information clustering.
result Consistent classification with Gravity Spy project labels.
New method identifies causal graphs with limited data and noise.
problem Identifying causal graphs from observational data is generally impossible.
method Using additional data from two environments with different noise statistics, and assuming Gaussian noise.
result The entire causal graph can be uniquely identified with a constant number of environments.
Holographic Invariant Storage uses vector architectures to ensure LLM safety at design time.
problem Mitigating context drift in large language models (LLMs) during deployment.
method Introduces Holographic Invariant Storage (HIS) protocol that combines known properties of bipolar Vector Symbolic Architectures into a design-time safety contract.
result Closed-form guarantees for single-signal recovery fidelity, continuous-noise robustness, and multi-signal capacity degradation are provided and validated.
Combines PCA and AMP for better signal estimation in noisy data.
problem Estimating a rank-1 signal in rotationally invariant noise.
method Combines PCA and AMP, with PCA initialization at the start of AMP.
result Rigorous asymptotic characterization of the new estimator's performance.
Gradient descent with noise converges to a unique optimum in nonconvex matrix factorization.
problem Gradient descent with noise converges to a unique optimum in nonconvex matrix factorization.
method A perturbed form of gradient descent with arbitrary initialization.
result Gradient descent with noise converges to a unique optimum.
Improved prediction and noise removal for dynamical time series.
problem Effective prediction and noise removal for noisy dynamical time series.
method Combination of kernel based regression and smooth splines for denoising and prediction.
result Combination of kernel based regression and smooth splines yields more accurate predictors by a factor of 2 or more.
Estimating signals with linear recurrence relations under Gaussian noise is nearly as hard as sparse signals.
problem Estimating discrete-time signals with unknown linear recurrence relations in Gaussian noise.
method Analyzing shift-invariant subspaces and their Fourier coefficients as reproducing filters.
result The statistical complexity is nearly the same as for s-sparse signals, and the estimator is tractable. Privacy-preserving crypto exchanges adjust prices based on Gaussian noise.
problem Ensuring fair pricing in privacy-preserving cryptocurrency exchanges.
method Derive Kyle equilibrium with Gaussian noise perturbation, rescaling price-impact and strategy factors.
result Identify a privacy subsidy as a transfer from LP pool to traders, invariant to noise.
Safety filter for unknown discrete-time systems with learned models and noise covariance.
problem Ensuring safety for unknown discrete-time linear systems with Gaussian noise.
method Develops a learning-based safety filter using empirical model and noise covariance, optimizing control actions to stay within safety constraints.
result Minimally modifies nominal control actions to ensure safety with high probability, tightening constraints as more data is collected.
Proposes a new CSC model for handling unknown noise.
problem Existing CSC methods can only model Gaussian noise, which is restrictive.
method Uses Gaussian mixture model for unknown noise and EM algorithm for optimization.
result Effective modeling of complicated unknown noise with high-quality filters and representation.
DACL tackles domain-specific contrastive learning by using Mixup noise.
problem Domain-specific contrastive learning methods rely on data augmentation techniques that require domain knowledge.
method DACL uses Mixup noise to create similar and dissimilar examples without domain-specific data augmentation.
result DACL outperforms other domain-agnostic noising methods and combines well with domain-specific methods.
Proposes DRIG for robust predictions using noise interventions.
problem Developing robust prediction models against distribution shifts.
method Distributional Robustness via Invariant Gradients (DRIG) exploiting general noise interventions.
result DRIG yields robust predictions among a data-dependent class of distribution shifts.
Improved speech separation and enhancement using neural beamforming.
problem Challenging speech separation and enhancement in reverberant environments.
method Sequential neural beamforming combining spectral and spatial separation methods.
result Average improvement of 2.75 dB in scale-invariant signal-to-noise ratio and 14.2% absolute reduction in speech recognition metric.
New method selects causal features from diverse data types.
problem Discovering causal relationships from non-continuous data types.
method Transformation-Model (TRAM) based Invariant Causal Prediction (TRAM-ICP) with TRAM-GCM and TRAM-Wald tests.
result Improved power and type I error control for diverse response types.
Proposes a method to compare noisy high-dimensional datasets with low-dimensional manifolds.
problem Comparing distributions on manifolds in noisy high-dimensional datasets.
method Linking low-rank structure to manifold geometry, developing a scale-invariant distance measure.
result Superior robustness and statistical power compared to existing methods.
Generative models improve for multiscale scientific data with new noise and interpolation techniques.
problem Numerical challenges in generating high-fidelity samples for multiscale scientific data.
method Design of noise distributions and interpolation schedules in function space to ensure Lipschitz regularity and finite noise roughness.
result Scale-adaptive noise and interpolation schedules improve numerical efficiency and fidelity of generated samples.
PCA++ improves robustness to background noise in contrastive learning.
problem Recovering shared signal subspaces from positive pairs in high-dimensional data with structured background noise.
method PCA++ uses hard uniformity-constrained contrastive learning to enforce identity covariance on projected features.
result PCA++ outperforms standard PCA and alignment-only PCA+ in simulations and real-world datasets.
Proposes a new loss function for robust training of deep neural networks against noisy labels.
problem Training deep neural networks with noisy labels, especially instance-independent noise.
method Introduces a novel information-theoretic loss function, L_DMI, based on Determinant based Mutual Information (DMI).
result L_DMI is the first provably robust loss function to instance-independent label noise, without requiring auxiliary information.
Develops structured noise for more accurate graph classifier robustness certificates.
problem Isotropic noise limits robustness certificates for graph classifiers.
method Randomized smoothing with anisotropic noise distribution.
result Structured-aware robustness certificates provide more accurate predictions.
Noise titration benchmarks time series forecasting models rigorously.
problem Evaluation of time series forecasting models is often flawed due to lack of interventionist methods.
method Interventionist benchmarking using Gaussian noise titration of dynamical systems.
result Fern model outperforms state-of-the-art models in non-stationary conditions.
Neural networks can approximate positive homogeneous functions, especially with multiple hidden layers.
problem Approximating positive homogeneous functions with neural networks.
method Using scale-invariant ReLU networks with multiple hidden layers.
result Approximation of positive homogeneous functions is possible with neural networks, especially with two hidden layers.
PS-IG improves feature attribution by reducing noise and variance.
problem Improving feature attribution in machine learning models.
method Path-sampled integrated gradients (PS-IG) computes expected value over sampled baselines.
result PS-IG reduces attribution variance by a factor of 1/3 under uniform sampling.
SC-Net learns interpretable filters for inverse problems, achieving optimal convergence and super-resolution.
problem Solving ill-posed inverse problems with effective regularization and interpretability.
method SC-Net operates in the spectral domain, learning a pointwise adaptive filter function based on signal-to-noise ratio.
result SC-Net achieves optimal convergence rate and zero-shot super-resolution, matching theoretical bounds.
A new method detects small holes in noisy data.
problem Detecting small holes in high-density regions from noise.
method Robust Density-Aware Distance (RDAD) filtration, incorporating distance-to-measure concept.
result The RDAD filtration prolongs the persistences of small holes, making them distinguishable from noise.
Unsupervised beamforming improves ASR in noisy conditions.
problem Improving ASR in unknown noisy environments.
method Multichannel NMF-Informed Beamforming (MVDR) for noise-robust ASR.
result The proposed method outperformed DNN-based beamforming in unknown environments.
New algorithm identifies causal structures using invariant regression across environments.
problem Identifying causal relationships in multi-environment settings with varying noise distributions.
method Introducing invariance of functional relations of variables to their causes across environments to infer causal structures.
result Proposed algorithm outperforms existing methods in terms of computational and sample complexity.
Corrupting the input and hidden layers of deep neural networks (DNNs) with multiplicative noise, often drawn from the Bernoulli distribution (or 'dropout'), provides regularization that has significantly contributed to deep learning's success. However, understanding how multiplicative corruptions prevent overfitting ha…
MDMs train to decode tokens in a random order, which affects performance; we show they can be optimized for a favorable order.
problem Performance of MDMs is affected by the random order in which tokens are decoded.
method We show that MDMs can be optimized for a favorable decoding order by equipping their continuous-time variational objective with multivariate noise schedules.
result MDMs can be decomposed into a weighted auto-regressive losses over orders, making them auto-regressive models with learnable orders.
A new sorting method using R2 values improves causal discovery from noisy data.
problem Improving causal discovery from noisy observational data.
method Introducing R2-sortability and an algorithm, R2-SortnRegress, to find causal order. result Sorting variables by increasing R2 yields a close-to-causal order. Noise-cleaning fMRI brain activity matrices for better precision estimation.
problem Denoise precision matrices of fMRI time series to estimate true matrices.
method Comparison of various noise-cleaning algorithms on synthetic and real fMRI data.
result Optimal Rotationally Invariant Estimator outperforms others in fMRI data.