Optimizes SGLD noise structure for better generalization bounds.
problem Improving generalization bounds for large models trained with SGLD.
method Manipulates the noise structure in SGLD to optimize information-theoretical bounds.
result Optimal noise covariance is the square root of the expected gradient covariance under certain constraints.
New insights into noise distribution for self-supervised learning.
problem Challenges the assumption that optimal noise should match data distribution.
method Turns to Noise-Contrastive Estimation (NCE) to define optimality of noise distribution.
result Optimal noise distribution is different from data distribution, challenging GANs assumption.
Optimized method tackles convex optimization with heavy-tailed noise.
problem Convex optimization problems with noisy gradients.
method Vanilla stochastic proximal subgradient method without gradient clipping or normalization.
result Achieves optimal complexity for various convex optimization types under heavy-tailed noise.
Classical scaling is shown to be optimal under various noisy conditions.
problem Consistency of classical scaling under general noise conditions.
method Established using finite fourth moments of noise, derived convergence rates, and matching minimax lower bounds.
result Classical scaling achieves minimax optimality in recovering true configuration from noisy dissimilarities.
Study uses Bayesian Optimization to analyze noise effects in materials research.
problem Optimizing materials with many variables and experimental noise.
method Batch Bayesian Optimization with synthetic data analysis.
result Noise sensitivity varies by problem landscape, impacting optimization outcomes.
New findings show optimal noise in contrastive learning is not the same as data distribution.
problem The optimal noise distribution in contrastive learning is not the same as the data distribution.
method Empirical and theoretical analysis of contrastive learning methods.
result Deviation from the assumption of equal noise and data distribution leads to better statistical estimators.
Unified approach for first-order methods with Markovian noise in stochastic optimization and variational inequalities.
problem Stochastic optimization problems with Markovian noise.
method Unified theoretical analysis of first-order gradient methods using randomized batching and multilevel Monte Carlo.
result Optimal (linear) dependence on the mixing time of the noise sequence, eliminating previous limiting assumptions.
Paper improves SLCB regret bound for bounded noise.
problem Stochastic linear contextual bandits with bounded noise.
method Set-membership estimation (SME) and optimism in the face of uncertainty (OFU).
result Improved regret bound of O ( log T ) O(\log T) O ( log T ) . New BO method optimizes multiple objectives under input noise.
problem Optimizing multiple performance metrics in manufacturing processes subject to random input noise.
method Formalizes optimization of multivariate value-at-risk (MVaR) using random scalarizations.
result Significantly outperforms alternative methods in identifying robust designs.
New method uses SLL to create masks for PX in noisy optimization problems.
problem Effective optimization in noisy problems with hidden variable dependencies.
method Statistical Linkage Learning (SLL) for decomposition and mask construction.
result Proposed method maintains effectiveness in noisy conditions and outperforms state-of-the-art.
AlignFlow improves FGMs by optimizing noise and data alignment.
problem Optimal Transport methods for FGMs are limited by scalability issues.
method Introduces Semi-Discrete Optimal Transport (SDOT) to enhance FGM training.
result AlignFlow scales well to large datasets and model architectures.
Noise makes learning linear thresholds hard, but algorithms can still learn near-optimal thresholds.
problem Learning linear thresholds in noisy data.
method Exploiting natural assumptions on data-generating process.
result Efficient learning of near-optimal linear thresholds is still possible with small data even in the presence of noise.
Study optimizes resource allocation in noisy systems for better control.
problem Limited attention in stochastic systems with multiplicative noise.
method Analytical and numerical methods for optimal attention allocation.
result Effective resource allocation enhances noise estimation and control decisions.
Paper analyzes convergence of stochastic methods under heavy-tailed noise.
problem Analyzing convergence of stochastic methods under heavy-tailed noise.
method Investigates vanilla and clipped stochastic subgradient descent methods.
result Demonstrates convergence properties under sub-Weibull and p-BCM noise assumptions.
SGLBO optimizes quantum circuits with fewer measurements, improving accuracy and noise resilience.
problem Efficiently optimizing parameterized quantum circuits with reduced measurement shots and noise.
method Developed SGLBO combining SGD and BO, with adaptive measurement-shot strategy and suffix averaging.
result Significantly reduces measurement-shot cost while improving accuracy and noise resilience.
Paper tackles robustness in adversarial noise with a meta-optimizer.
problem Sensitivity to adversarial noise hinders machine learning deployment.
method Meta-optimizer learns to robustly optimize models using adversarial examples.
result Meta-optimizer transfers adversarial knowledge to new models without generating new examples.
Improved sample complexity for learning halfspaces with malicious noise.
problem Efficiently learning halfspaces in the presence of malicious noise.
method New analysis of Awasthi et al. algorithm with matrix Chernoff inequality and localization schemes.
result Achieved near-optimal sample complexity of i l d e O ( d ) ilde{O}(d) i l d e O ( d ) for isotropic log-concave distributions. A new framework for PPLS combines noise estimation, optimization, and calibration.
problem Probabilistic PLS models need interpretable latent factors and calibrated uncertainty.
method End-to-end pipeline combining noise estimation, constrained optimization, and prediction calibration.
result Achieves near-nominal coverage and native calibrated uncertainty across benchmarks.
New algorithm for signal estimation in noisy matrix models.
problem Signal estimation in rectangular spiked matrix models with rotationally invariant noise.
method Orthogonal Approximate Message Passing (OAMP) algorithm for signal estimation.
result Optimal OAMP algorithm minimizes mean-squared error and achieves Bayes-optimal performance.
Bayes-optimal limits in PCA with structured noise are determined.
problem Analyzing statistical dependencies in measurement noise for high-dimensional inference.
method Study of spiked matrix model with low-order polynomial orthogonal noise, providing Bayes-optimal limits and proposing a novel AMP.
result A novel AMP algorithm reaches the information-theoretic limits for more general priors.
Existence of strong randomized equilibria in mean-field games with common noise.
problem Existence of strong solutions in mean-field games of optimal stopping.
method Connection with Bank-El Karoui's representation problem and continuity assumptions.
result Existence of strong randomized mean-field equilibrium under certain conditions.
This work reveals symmetries in quantum circuits and develops a noise-aware optimization method.
problem Understanding and optimizing the cost landscape of parametrized quantum circuits.
method Analytical proof of symmetries and their resilience to noise, followed by the development of SYMH optimization method.
result Symmetries in PQCs lead to degeneracy in the cost landscape and can be exploited to improve optimization under noise.
Study on online regression with noise, achieving near-optimal regret bounds.
problem Online generalized linear regression with stochastic noise.
method Sharp analysis of FTRL algorithm for stochastic label noise.
result Achieved near-optimal regret bounds for O ( σ 2 d log T ) + o ( log T ) O(σ^2 d \log T) + o(\log T) O ( σ 2 d log T ) + o ( log T ) . CWGD measures gradient diversity weighted by curvature, improving SGD convergence.
problem Gradient noise in high-curvature directions is underestimated by standard methods.
method CWGD weights gradient diversity by the inverse square root of the Hessian.
result CWGD-Cosine reduces optimization error by up to 20% compared to standard cosine annealing.
Stochastic gradient methods can converge in expectation under heavy-tailed noise.
problem Convergence of stochastic gradient methods under heavy-tailed noise.
method Comprehensive study of stochastic optimization under heavy-tailed noise for e x t s f S G D extsf{SGD} e x t s f S G D , e x t s f S M D extsf{SMD} e x t s f S M D , e x t s f A S M D extsf{ASMD} e x t s f A S M D , e x t s f S G D M extsf{SGDM} e x t s f S G D M in convex and nonconvex optimization. result Established in-expectation convergence results for various stochastic gradient methods.
New method designs joint initial noises for diffusion models to improve diversity and alignment.
problem Independent initial noises limit diversity in generated images.
method Coupling of initial noises, maintaining Gaussian distribution while allowing dependence.
result Repulsive Gaussian coupling improves diversity without increasing sampling cost.
Ridge regression performs optimally in noisy environments with heavy-tailed distributions.
problem Performance of ridge regression in noisy environments with heavy-tailed noise.
method Established excess risk bounds using integral operator framework and Fuk-Nagaev inequality.
result Ridge regression achieves optimal convergence rates under heavy-tailed noise, demonstrating robustness.
Optimal Gaussian noise mechanisms achieve nearly optimal error in unbiased mean estimation.
problem Efficiently estimating the mean of high-dimensional data while preserving privacy.
method Differential privacy mechanisms with Gaussian noise, focusing on optimal covariance.
result Gaussian noise mechanisms achieve nearly optimal error among all private unbiased mean estimation mechanisms.
The paper describes a method to infer the signal-to-noise ratio in portfolio optimization.
problem Estimating the signal-to-noise ratio in portfolio optimization problems.
method A statistic similar to the Sharpe Ratio Information Criterion is used for inference.
result The method works well for reasonable sample and asset universe sizes.
Nonlinear SGD achieves high-probability rates in non-convex optimization with heavy-tailed noise.
problem Optimization in non-convex problems with heavy-tailed noise.
method General nonlinear framework for SGD, including symmetrization techniques.
result Achieves O ~ ( t − 1 / 2 ) \widetilde{\mathcal{O}}(t^{-1/2}) O ( t − 1/2 ) rate for heavy-tailed noise. Study revisits AdaGrad convergence with relaxed noise assumptions.
problem Non-convex smooth optimization problems with general noise.
method General noise model with function value gap and gradient magnitude control.
result Probabilistic convergence rate of ( ilde{\mathcal{O}}(1/\sqrt{T})) under general noise.
Optimizes control of noisy discrete systems without system matrix knowledge.
problem Optimal control of discrete-time systems with additive and multiplicative noises.
method Stochastic Lyapunov and Riccati equations, model-free reinforcement learning.
result Model-free reinforcement learning algorithm converges to optimal control policy.
Method reweights instances and classes to improve robustness in noisy data.
problem Improving deep learning performance in the presence of label noise.
method Formulates constrained optimization problems to assign importance weights to instances and class labels.
result Significant performance gains observed in benchmark datasets with label noise.
Kernel-based bandit is an extensively studied black-box optimization problem, in which the objective function is assumed to live in a known reproducing kernel Hilbert space. While nearly optimal regret bounds (up to logarithmic factors) are established in the noisy setting, surprisingly, less is known about the noise-f…
Stochastic optimization's success linked to heavy-tailed noise.
problem Understanding stochastic optimization's success mechanisms.
method Modeling stochastic optimization as random recurrence relations, analyzing multiplicative noise and heavy-tailed behavior.
result Multiplicative noise leads to heavy-tailed stationary behavior in optimization parameters.
Paper provides tail bounds for stochastic mirror descent in heavy-tailed noise.
problem Optimizing convex and Lipschitz functions with heavy-tailed noise.
method Develops tail bounds for optimization error of Stochastic Mirror Descent.
result Tail bounds extend to heavier-tailed noise regimes without diameter constraints.
SignSGD analysis quantifies its effects in high dimensions.
problem Understanding signSGD's effects in high-dimensional settings.
method High-dimensional analysis of signSGD, deriving SDE and ODE for risk.
result Quantification of signSGD's effects: effective learning rate, noise compression, diagonal preconditioning, gradient noise reshaping.
Study efficient active learning for halfspaces with Tsybakov noise using non-convex optimization.
problem Efficiently learn halfspaces with Tsybakov noise under structured unlabeled data.
method Non-convex optimization approach to find approximate first-order stationary points.
result Designs an algorithm with improved label complexity compared to previous methods.
New method certifies neural network robustness under random input noise.
problem Certifying neural network robustness against random input noise.
method Chance-constrained optimization problem reformulated with input-output samples, convex conditions developed.
result Proposed method certifies robustness against various input noise regimes over larger uncertainty regions.
We study Principal Component Analysis (PCA) in a setting where a part of the corrupting noise is data-dependent and, as a result, the noise and the true data are correlated. Under a bounded-ness assumption on the true data and the noise, and a simple assumption on data-noise correlation, we obtain a nearly optimal samp…
Noise Injection probes deep learning dynamics during training phases.
problem Understanding the learning mechanism of deep neural networks.
method Noise Injection Nodes (NINs) are used to perturb DNN architectures without altering the optimization algorithm.
result Distinct training phases are observed based on the scale of injected noise.
A new noise model for preferential Bayesian optimization using user anchors.
problem Inadequate assumption of homoscedastic noise in human-in-the-loop settings.
method Proposes a heteroscedastic noise model with anchors and a KDE uncertainty map.
result Risk-adjusted performance improvement and clarified anchor placement effects.
Variational hybrid quantum-classical optimization represents one of the most promising avenue to show the advantage of nowadays noisy intermediate-scale quantum computers in solving hard problems, such as finding the minimum-energy state of a Hamiltonian or solving some machine-learning tasks. In these devices noise is…
Framework for robust control in cooperative systems with uncertain common noise.
problem Optimizing collective behavior of agents in the presence of uncertain common noise.
method Proposes a robust mean-field control framework and proves existence of optimal controls.
result Existence of optimal open-loop controls linked to a lifted robust Markov decision problem.
Sparsity promoting norms are frequently used in high dimensional regression. A limitation of such Lasso-type estimators is that the optimal regularization parameter depends on the unknown noise level. Estimators such as the concomitant Lasso address this dependence by jointly estimating the noise level and the regressi…
Omega method mitigates noise in stochastic game optimization.
problem Noise sensitivity and convergence issues in stochastic game optimization.
method Omega method incorporates EMA of historic gradients in its update rule.
result Omega method outperforms optimistic gradient method in stochastic games.
Improved SGD bounds for machine learning models with Markovian noise.
problem Uniform high-probability bounds for SGD under PL condition with Markovian noise.
method Combining Poisson equation for Markovian noise and probabilistic induction for almost-sure bounds.
result Matching 1 / k 1/k 1/ k decay rate for expected suboptimality. Improved BO algorithms reduce prediction error under Gaussian noise.
problem Reducing prediction error in Bayesian optimization with Gaussian noise.
method Established new prediction error bounds for Gaussian process under frequentist setting.
result Proved improved convergence rates of cumulative regret for GP-UCB and GP-TS.