IEBN normalizes noise by enhancing instance-specific information, improving deep learning performance.
problem Improving deep learning performance by regulating noise in batch normalization.
method Integrates self-attention mechanism to recalibrate channel information in BN.
result IEBN outperforms BN with improved generalization and stability.
Paper explores whether gradient normalization can replace clipping for SGD in heavy-tailed noise.
problem Ensuring convergence of SGD in heavy-tailed noise.
method Revisits gradient clipping and normalization, proving their sufficiency and effectiveness.
result Gradient normalization alone is sufficient for nonconvex SGD convergence under smoothness assumptions.
Doubly-stochastic normalization improves robustness to heteroskedastic noise.
problem Robustness to heteroskedastic noise in affinity matrix construction.
method Doubly-stochastic normalization of the Gaussian kernel.
result Doubly-stochastic normalization converges to clean matrix with rate m−1/2 under heteroskedastic noise. RS-NSGD improves SGD convergence for heavy-tailed noise.
problem Nonconvex optimization with heavy-tailed noise.
method Integrates direction normalization into subspace updates.
result Achieves better oracle complexity than full-dimensional normalized SGD.
Method estimates mixture models without normalization.
problem Estimating mixture models with intractable normalization.
method Extends noise contrastive estimation (NCE) for mixture models.
result Probabilistic clustering using deep representations.
Examines WENDy-IRLS algorithm's noise robustness and efficiency in various differential equations.
problem Noise robustness and efficiency of WENDy-IRLS algorithm.
method Studied coverage and bias properties of WENDy-IRLS algorithm's estimators in various differential equations and noise distributions.
result WENDy-IRLS algorithm shows notable noise robustness and computational efficiency.
Test the robustness of quantum-enhanced phase estimation under various noise conditions.
problem Evaluate the robustness of quantum-enhanced adaptive phase estimation (QEAPE) in noisy conditions.
method Simulated QEAPE under four phase-noise models and compared resource usage of evolutionary and Bayesian control policies.
result Demonstrated the effectiveness of both evolutionary and Bayesian control policies in noisy conditions.
Normalizing flows improve density estimation from noisy data.
problem Estimating underlying density from noisy samples.
method Use normalizing flows for density estimation with arbitrary noise distributions, using amortized variational inference.
result Normalizing flows can outperform Gaussian mixtures for density deconvolution.
New theory explains why normalization is preferred in SGD under heavy-tailed noise.
problem Understanding why normalization is preferred in stochastic gradient descent (SGD) under heavy-tailed noise.
method Developed a worst-case complexity theory for stochastically preconditioned SGD and its variants.
result Normalization guarantees convergence at optimal rates, while clipping may fail in the worst case.
Paper proves convergence of bi-stochastically normalized graph Laplacian to manifold Laplacian and robustness to outlier noise.
problem Convergence of bi-stochastically normalized graph Laplacian to manifold Laplacian and robustness to outlier noise.
method Proves convergence of bi-stochastically normalized graph Laplacian to manifold Laplacian with rates, and proposes an approximate and constrained matrix scaling problem to achieve the same consistency rate.
result Graph Laplacian consistency rate matches the rate for clean manifold data plus an additional term proportional to the boundedness of the inner-products of the noise vectors.
Paper proposes BCNMCC for system identification with noisy input.
problem System identification with noisy input and impulsive output noise.
method Introduces BCV to NMCC algorithm to compensate for input noise bias.
result BCNMCC outperforms other algorithms in noisy input conditions.
Proposes NCMN to remove feature correlation in multiplicative noise.
problem High feature correlation in multiplicative noise reduces network performance.
method Exploits batch normalization to remove correlation effect.
result Significantly improves performance on image classification tasks.
Proposes a method to improve classification robustness against label noise.
problem Improving classification robustness against label noise.
method Extends a simple regression approach to classification, modeling labels as samples from a logistic-normal distribution.
result The method improves robustness against label noise in classification.
Study shows how network width affects SGD hyper-parameters and generalization.
problem Understanding how network width impacts SGD hyper-parameters and generalization.
method Generated model families by increasing network width, performed hyper-parameter search.
result Wider networks achieve higher test accuracy and optimal normalized noise scale.
Robustly infers manifold density and geometry under high-dimensional noise.
problem Inaccurate kernel density estimation under high-dimensional noise.
method Doubly stochastic normalization of Gaussian kernel.
result Robust tools for density estimation, noise magnitude estimation, and distance approximation.
A new method inflates and deflates data manifolds to estimate densities without losing universality.
problem Density estimation on low-dimensional manifolds with non-Euclidean support.
method Inflation-deflation approach using Normalizing Flows with added noise.
result Exact estimation of densities on manifolds with sufficient conditions and Gaussian noise approximation.
MN-PCA models structured noise in data and feature spaces.
problem Complex and structured noise in real-world data.
method Matrix normal distribution for structured noise modeling, generalized Mahalanobis distance approximation.
result MN-PCA obtains a low-rank data representation and structured noise simultaneously.
SkewD robustly discovers causal relationships in skewed noise models.
problem Distinguishing cause from effect in skewed noise models.
method SkewD extends normal-distribution framework to skew-normal setting for reliable inference.
result SkewD remains robust under high skewness, improving reliability.
Optimized normalization layers improve domain generalization.
problem Improving model generalization across different domains.
method Learning separate normalization parameters per domain using multiple normalization methods (batch and instance).
result State-of-the-art accuracy on domain generalization benchmarks.
We solve image inverse problems using a flow-based noise model.
problem Image inverse problems with complex noise patterns.
method Normalizing flow prior for maximum a posteriori estimation.
result Empirical validation on various inverse problems.
Non-normal RNNs outperform orthogonal ones in sequential tasks.
problem Vanishing/exploding gradients in RNNs training.
method Investigate non-normal RNNs with non-normal recurrent connectivity matrix.
result Non-normal RNNs outperform orthogonal ones in various benchmarks.
Paper derives normal approximations for singular subspaces with i.i.d. noise.
problem Normal approximation of singular subspaces under i.i.d. noise.
method Explicit representation formula, expected projection distance calculation, non-asymptotic normal approximation, bias corrections.
result Non-asymptotic normal approximation with optimal SNR condition and comprehensive simulation results.
A method predicts GNS of transformer layers using normalization layer norms.
problem Estimating gradient noise scale with minimal variance.
method Simultaneously compute per-example gradient norms and parameter gradients.
result Total GNS is predicted well by normalization layer GNS.
Researchers improve NCE by addressing its flat loss landscape issues.
problem NCE's poor performance due to an ill-behaved loss landscape.
method Introduced eNCE with an exponential loss and normalized gradient descent.
result Proven that landscape issues arise from inappropriate noise distribution.
Optimized method tackles convex optimization with heavy-tailed noise.
problem Convex optimization problems with noisy gradients.
method Vanilla stochastic proximal subgradient method without gradient clipping or normalization.
result Achieves optimal complexity for various convex optimization types under heavy-tailed noise.
Estimates parameters in max-linear Bayesian networks with noise.
problem Causal inference in extreme-value settings with noise parameters.
method Max-plus algebra and logarithm transformation, normal distribution estimation, EM algorithm and quadratic optimization.
result An estimator of a parameter for each edge in a DAG is normally distributed.
RECLAIM discovers causal graphs in cyclic, noisy systems.
problem Discovering causal relationships in cyclic, noisy systems.
method RECLAIM uses EM with residual normalizing flows to handle cycles and noise.
result RECLAIM effectively discovers causal graphs in both synthetic and real-world datasets.
Antithetic noise improves diffusion models' uncertainty quantification.
problem Improving uncertainty quantification in diffusion models.
method Pairing each noise sample with its negation, leading to strong negative correlation.
result Substantially more reliable uncertainty quantification with up to 90% narrower confidence intervals.
New methods solve optimization problems with heavy-tailed noise, improving upon existing complexity bounds.
problem Optimization problems with heavy-tailed noise and weakly average smoothness.
method Normalized stochastic first-order methods with Polyak, multi-extrapolated, and recursive momentum.
result First-order oracle complexity results for finding approximate stochastic stationary points under heavy-tailed noise.
SGLD improves sampling for Bayesian models on large datasets.
problem Poor convergence and mixing times in SGLD for models with scale and correlation variations.
method Preconditioning noise vector, comparing normalization approaches.
result Preconditioning improves mixing times and regularization effects.
Study shows short rate can explode to infinity in HJM model, impacting Eurodollar futures.
problem Exploding short rate in HJM model affecting Eurodollar futures.
method Small-noise deterministic limit analysis.
result Explicit explosion criteria derived for short rate under mild assumptions.
Unified framework for distributed compressed SGD under (L0,L1)-smoothness.
problem Understanding the joint effect of batch noise, adaptivity, and compression in distributed stochastic optimization.
method Developed a unified theoretical framework using SDEs that incorporate curvature-dependent terms.
result Normalizing updates in DCSGD stabilizes convergence, with normalization degree determined by noise structure and landscape regularity.
Improves efficiency of simulators that fail to return.
problem Computational inefficiency in simulators that don't return for certain inputs.
method Trains a conditional normalizing flow to propose perturbations.
result Increased computational efficiency of simulators.
Geometric analysis improves noise injection in GANs.
problem Unclear mechanism of noise injection in GANs.
method Geometric framework based on Riemannian geometry.
result A new strategy for noise injection is devised.
GAN normalizes CT scans for consistent radiomic feature values.
problem Variations in dose levels and slice thickness affect radiomic features sensitivity.
method Used a 3D generative adversarial network (GAN) to normalize reduced dose, thick slice images to normal dose, thinner slice images.
result GAN-based approach led to significantly smaller error in radiomic features.
Study shows sample complexity for logistic regression with normal covariates.
problem Estimating parameters of logistic regression with normal design.
method Analyzes sample complexity in terms of dimension and inverse temperature.
result Shows two change-points in sample complexity curve based on inverse temperature.
New photometric stereo method using learned dictionaries for robustness.
problem Estimating object normals from varying lighting conditions.
method Adaptive dictionary learning for image preprocessing and direct regularization of normal vectors.
result State-of-the-art performance in noisy conditions.
Paper uses VAEs to detect radar targets in complex noise.
problem Detecting radar targets in compound clutter and thermal noise.
method Proposes a VAE architecture to distinguish radar targets from various noise types.
result The VAE outperforms classical detectors in challenging noise conditions.
A new method avoids noise amplification when subtracting or dividing stochastic signals.
problem Noise amplification when subtracting or dividing stochastic signals.
method Normalizing flows to approximate the distribution of the signal of interest.
result Normalizing flows can generate an approximation of the probability distribution over the signal of interest, avoiding subtraction or division.
Most real life systems have a random component: the multitude of endogenous and exogenous factors influencing them result in stochastic fluctuations of the parameters determining their dynamics. These empirical systems are in many cases subject to noise of multiplicative nature. The special properties of multiplicative…
Dropout-based regularization methods can be regarded as injecting random noise with pre-defined magnitude to different parts of the neural network during training. It was recently shown that Bayesian dropout procedure not only improves generalization but also leads to extremely sparse neural architectures by automatica…
Develops NPMC method for noisy labels, improving multiclass classification accuracy.
problem Asymmetric misclassification costs and label noise in multiclass classification.
method Empirical likelihood approach using exponential tilting density ratio model.
result Root n consistent and asymptotically normal estimators for clean labels and noise mechanism.
Paper analyzes NCE method for unnormalized models, reducing asymptotic variance.
problem Estimating parameters of unnormalized models with high asymptotic variance.
method Proposes a method to reduce asymptotic variance by estimating auxiliary distribution parameters and analyzing objective function forms.
result NCE estimator is consistent and asymptotically normal, with reduced variance.
Developed criteria for selecting non-normalized models using NCE and score matching.
problem No information criteria for non-normalized models estimated by NCE or score matching.
method Developed information criteria based on discrepancy measures for non-normalized models estimated by NCE or score matching.
result The proposed criteria enable selection of the appropriate non-normalized model in a data-driven manner.
Improved physics-integrated generative models with noise robustness and fidelity.
problem Enhancing generative models to produce outputs that comply with physical laws and improve generalization.
method Integrating variational autoencoder with planar normalizing flow and attention mechanisms to learn latent posterior distributions and mitigate noise.
result Significant improvement in reconstruction quality and robustness against noise.
The paper analyzes how different regularizers affect neural network complexity under input noise.
problem Understanding the impact of input noise on neural network complexity and regularization.
method Analysis of various regularization schemes (L2, DARC1, Jacobian, Spectral normalization) on neural network complexity under Gaussian input noise.
result Spectral normalization achieves the highest test set accuracies for both shallow and deep architectures.
Bayesian neural networks improve uncertainty calibration without sacrificing accuracy.
problem Bayesian neural networks struggle with uncertainty calibration and high-dimensional geometry.
method Model uncertainty only in weight directions using a von Mises-Fisher posterior on the unit sphere, deriving a compact KL term.
result A lightweight, dimension-aware variational unit improves calibration without sacrificing accuracy.
Iterative BP-CNN improves channel decoding under correlated noise.
problem Channel decoding under correlated noise.
method Concatenates CNN with BP decoder, iteratively improving SNR.
result Iterative BP-CNN achieves better BER with lower complexity.