UNIQ method quantizes neural networks adaptively, reducing BOPS.
problem Efficiently quantizing neural networks to reduce computational cost.
method Adapts to parameter distribution, using uniform noise injection.
result Shows advantages in low computational budget scenarios.
SmoothOut improves deep learning by smoothing out sharp minima, enhancing generalization.
problem Sharp minima in deep neural networks lead to poor generalization.
method SmoothOut framework that perturbs multiple copies of the DNN by noise injection and averages them, improving generalization.
result SmoothOut eliminates sharp minima and improves generalization in both small-batch and large-batch training.
Paper presents faster, robust adversarial training methods.
problem Increasing neural network robustness against adversarial attacks.
method Integrates FGSM with Pixelwise Noise Injection Layer (PNIL) and uniform noise.
result Achieves comparable results to PGD-based adversarial training but faster.
This dissertation shows that careful injection of noise into sample data can substantially speed up Expectation-Maximization algorithms. Expectation-Maximization algorithms are a class of iterative algorithms for extracting maximum likelihood estimates from corrupted or incomplete data. The convergence speed-up is an e…
Geometric analysis improves noise injection in GANs.
problem Unclear mechanism of noise injection in GANs.
method Geometric framework based on Riemannian geometry.
result A new strategy for noise injection is devised.
Smooths metrics on manifolds with curvature bounds and injectivity radius constraints.
problem Smooth metrics on manifolds with curvature and injectivity constraints.
method Bi-Lipschitz smoothing with controlled smoothing and volume lower bounds.
result Proves existence of smooth metrics with curvature bounds and injectivity radius constraints.
This paper connects noise injection to Bayesian inference for neural networks, improving model uncertainty.
problem Improving the reliability and confidence of neural network predictions through uncertainty quantification.
method Introducing noise into neural network parameters during training and inference to estimate prediction uncertainty.
result The MCNI method outperforms baseline models in regression and classification tasks.
Uniform curvature bounds for regularized metrics with bounds on Ricci tensor and injectivity radius.
problem Bounding curvature of regularized metrics with constraints on Ricci tensor and injectivity radius.
method Mollification of riemannian metrics, uniform W2,p-harmonic radius bounds, Ricci tensor bounds, injectivity radius bounds. result Uniform estimate on the change of sectional curvature for regularized metrics.
Dropout-based regularization methods can be regarded as injecting random noise with pre-defined magnitude to different parts of the neural network during training. It was recently shown that Bayesian dropout procedure not only improves generalization but also leads to extremely sparse neural architectures by automatica…
Noise improves model quality in non-linear neural networks during decentralized training.
problem Improving generalization of locally trained neural networks.
method Injecting noise into the weights of neural networks during decentralized training.
result Noise injection improves model quality for non-linear neural networks, but not for linear models.
Noisy EM speeds up convergence in mixture models.
problem Speeding up convergence of EM algorithms in mixture models.
method Noisy Expectation Maximization (NEM) algorithm with various noise injection methods.
result NEM algorithm speeds up convergence to local maxima in mixture models.
Noise Injection probes deep learning dynamics during training phases.
problem Understanding the learning mechanism of deep neural networks.
method Noise Injection Nodes (NINs) are used to perturb DNN architectures without altering the optimization algorithm.
result Distinct training phases are observed based on the scale of injected noise.
Noise Injection Node Regularization improves robustness in neural networks.
problem Improving robustness of neural networks against various perturbations.
method Injecting structured noise into neural networks during training.
result Significant improvement in robustness against data perturbations.
Label smoothing improves model performance even with noisy labels.
problem Mitigating label noise in deep learning models.
method Examined label smoothing as a technique to cope with label noise and compared it to loss-correction methods.
result Label smoothing is competitive with loss-correction techniques under label noise and beneficial for distillation from noisy data.
Noise injection before gradient steps helps in regularization for neural networks.
problem Improving generalization in overparametrized neural networks.
method Injecting small noise perturbations before computing gradient steps, especially in layer-wise fashion.
result Small noise perturbations can explicitly regularize neural networks without variance explosion.
Uniform eigenvalue bounds for Hodge Laplacian on manifolds with Ricci curvature and injectivity radius bounds.
problem Establishing uniform bounds for eigenvalues of the Hodge Laplacian on manifolds with specific geometric constraints.
method Using uniform bounds on Ricci curvature, injectivity radius, and diameter, we derive eigenvalue estimates for the Hodge Laplacian.
result Uniform upper bounds for eigenvalues of the Hodge Laplacian on differential forms on manifolds with given geometric constraints.
GNIs induce a regulariser that penalizes high-frequency components in neural network activations.
problem Understanding the regularizing effect of Gaussian noise injections on neural network activations.
method Deriving the explicit regularizer by marginalizing out injected noise and analyzing its effect in the Fourier domain.
result GNIs induce a regularizer that produces calibrated classifiers with large margins.
S-SGD adds symmetrical noise to weights to avoid sharp minima in deep learning.
problem SGD does not always converge to a flat minimum, leading to poor generalization.
method Symmetrical weight noise injection in SGD.
result S-SGD outperforms conventional SGD and weight-noise injection methods in large batch training.
SymNoise improves language model fine-tuning by 6.7% over NEFTune, using symmetric noise.
problem Improving performance of language models through noise-based fine-tuning.
method Introducing SymNoise, a new fine-tuning method using symmetric noise in embeddings.
result SymNoise increases model performance by 69.04% on AlpacaEval compared to NEFTune's 64.69%.
Noise injection improves inference privacy in DNN models.
problem Malicious servers can infer sensitive attributes from input data.
method Adaptive Noise Injection (ANI) using a lightweight DNN on the client.
result Significant improvement in privacy (up to 48.5% degradation in sensitive-task accuracy with <1% degradation in primary accuracy).
GNIs induce asymmetric heavy-tailed noise in SGD, affecting network performance.
problem The effect of Gaussian noise injections on SGD dynamics and network performance.
method Developed a Langevin-like SDE driven by asymmetric heavy-tailed noise to model the modified SGD dynamics.
result GNIs induce an implicit bias that varies with noise heaviness and asymmetry, affecting network performance.
The paper establishes a uniform Lipschitz bound on the square root of the systole function in Teichmüller space.
problem Uniform Lipschitz bounds on geometric functions in Teichmüller space.
method Injectivity radius analysis and Lipschitz bounds on systole function.
result Uniform Lipschitz constant for the square root of the systole function on Teichmüller space.
Noise in RNNs promotes flatter minima and more stable dynamics.
problem Understanding and optimizing the training of RNNs with noise.
method Formalizing RNNs as stochastic differential equations and analyzing the effect of noise in the hidden states.
result Noise injection in RNNs leads to flatter minima, more stable dynamics, and improved robustness.
Uniform proof of π1-injectivity for certain maps in low dimensions.
problem Characterizing π1-injective maps in low-dimensional manifolds. method Uniform group-theoretic proof based on residual finiteness and numerical invariants.
result Uniform determination of when maps induce π1-isomorphisms. Unified stability bounds for noisy SGD across convex and non-convex losses.
problem Deriving generalization bounds for noisy stochastic gradient descent.
method Unified approach using Lyapunov functions and applied probability.
result Time-uniform stability bounds for SGD on various loss functions.
Uniformly positive scalar curvature implies a lower bound on injectivity radius.
problem Bounding curvature and scalar curvature in three-manifolds.
method Analyzing bounded sectional curvature and uniformly positive scalar curvature properties.
result Uniform lower bound on injectivity radius.
Proposes a new noise injection method for neural networks that improves accuracy and representation clarity.
problem Improving neural network performance and representation clarity.
method Adaptive Structured Noise Injection (ASNI) for shallow and deep neural networks.
result Boosts the accuracy of neural networks and disentangles hidden layer representations.
Anti-correlated noise improves machine learning model generalization.
problem Improving machine learning model generalization.
method Injecting anticorrelated noise into gradient descent steps.
result Anti-correlated noise leads to better model generalization than uncorrelated noise.
Paper develops robust neural network sensors for fuel injection quantities.
problem Adversarial noise increases error in standard neural network models for fuel injection measurements.
method Apply provable robust network learning and verification methods to fuel injection measurements.
result Provable robust model reduces mean relative error to 16.5% under sensor noise.
Noise injection regularizes Hessian, improving neural network training and generalization.
problem Regularizing over-parameterized neural networks with nonconvex and nonlinear geometry.
method Injecting isotropic Gaussian noise into weight matrices and designing a two-point estimate of the Hessian penalty.
result Effective regularization of Hessian improves generalization, achieving up to 2.4% test accuracy increase.
STAG injects noise into graph neural networks to improve performance.
problem Graph neural networks suffer from over-smoothing and limited discrimination.
method Introduces a stochastic aggregation framework (STAG) with adaptive noise injection.
result STAG models correct both over-smoothing and discrimination issues.
Adapts RL regularization techniques to prevent overfitting.
problem Preventing reinforcement learning agents from overfitting to limited training environments.
method Selective Noise Injection (SNI) and Information Bottleneck (IB) techniques.
result Significantly improved generalization performance on Coinrun benchmark.
Uniform bounds for eigenvalues of Hodge Laplacian on manifolds with lower Ricci curvature.
problem Establishing bounds for eigenvalues of Hodge Laplacian under lower Ricci curvature.
method Using geometric assumptions including lower Ricci curvature, injectivity radius, and diameter bounds.
result Uniform eigenvalue bounds for the Hodge Laplacian and connection Laplacian.
New method for certified unlearning reduces noise injection.
problem Achieving formal unlearning guarantees with adaptive noise calibration.
method Adaptive per-instance noise calibration based on individual data point sensitivities.
result Derivation of high-probability per-instance sensitivity bounds for ridge regression.
Improves RNN performance under noisy computations.
problem Power and speed limitations in deep learning with noisy analog circuits.
method Deep Noise Injection training to robustify RNN weights/biases.
result Trained RNNs show more consistent performance under noisy inference.
Proposes a differentially private bandit algorithm reducing noise over time.
problem Privacy concerns in interactive recommendation systems.
method Tree-based mechanism to add Laplace or Gaussian noise to model parameters, focusing on dynamic global sensitivity.
result Demonstrates (ε,δ)-differential privacy with reduced noise and improved regret. Uniform proof for Ricci flows on complete manifolds.
problem Proving short-time existence, uniqueness, and continuous dependence for Ricci flows.
method Using Koch-Lamm framework and tensor heat kernel estimates, with a new continuous dependence estimate.
result Uniform proof of short-time existence, uniqueness, and continuous dependence for Ricci flows.
The Fridman function is bounded by the injectivity radius for certain hyperbolic manifolds.
problem Bounding the Fridman function for hyperbolic manifolds.
method Analyzing the relationship between the Fridman function and the injectivity radius function.
result The Fridman function is bounded above by the injectivity radius function for certain hyperbolic manifolds.
Generative method avoids function estimation for data generation.
problem Challenges in function estimation for generative models.
method Deterministic point transport with gradient descent.
result Data generation possible without function estimation.
Interpolating label noise makes models vulnerable to adversarial attacks.
problem Adversarial vulnerability of models trained on noisy labels.
method Theoretical analysis of label noise and adversarial risk relationship.
result Uniform label noise induces adversarial risk similar to worst-case poisoning.
Colored noise improves neural network robustness against adversarial attacks.
problem Vulnerability of neural networks to adversarial perturbations.
method Injection of colored noise into network weights and activations during adversarial training.
result Our approach outperforms previous methods in terms of adversarial accuracy on CIFAR-10 and CIFAR-100 datasets.
Study on classifier robustness to uniform and Gaussian noise.
problem Understanding how classifiers handle random noise.
method Characterized robustness to uniform ℓp and Gaussian noise. result Distance to decision boundary predicts robustness.
Paper shows how to integrate quantization into neural compression models.
problem Integrating quantization into neural compression models.
method Integrates uniform noise channel at test time using universal quantization.
result Eliminates mismatch between training and test phases while maintaining differentiability.
Paper finds dropout noise approximation invalid for logistic regression and neural networks.
problem Invalidity of dropout noise approximation for logistic regression and neural networks.
method Derived equivalence between dropout noise injection and L2 regularisation using divergent Taylor expansion. result Approximation approach is invalid for robust constraints and general neural network topologies.
We establish a uniform estimate for the injectivity radius of the past null cone of a point in a general Lorentzian manifold foliated by spacelike hypersurfaces and satisfying an upper curvature bound. Precisely, our main assumptions are, on one hand, upper bounds on the null curvature of the spacetime and the lapse fu…
FCNv2 robustness tested under noise and random initial conditions.
problem Assessing AI weather forecasting model robustness to input noise.
method Two experiments with varying noise levels and random initial conditions.
result FCNv2 preserves hurricane features under low to moderate noise, but underestimates intensity and persistence.
Geometry-aware noise improves model generalization on complex manifolds.
problem Improving model generalization on highly curved data manifolds.
method Add geometry-aware noise to input space, projecting Gaussian noise onto tangent space of manifold and mapping it via geodesic curve.
result Geometry-aware noise leads to improved generalization and robustness on highly curved manifolds.
The paper is concerned with regularity properties of boundaries of causal pasts of points in a 3+1-dimensional Einstein-vacuum spacetime. In a Lorentzian manifold such boundaries play crucial role in propagation of linear and nonlinear waves. We prove a uniform lower bound on the radius of injectivity of these null bou…