New approach improves model generalization through distributionally robust learning.
problem Improving model generalization in machine learning.
method Stochastic gradient descent applied to the outer minimization problem, with gradient estimation through multi-level Monte Carlo randomization.
result Our approach yields significant benefits over previous work in numerical experiments.
New robust estimators for risk minimization in statistical models.
problem Robust estimation in statistical models, especially in heavy-tailed settings.
method Novel robust gradient descent approach for convex risk minimization.
result Proves robustness of estimators in general statistical models and specific cases.
Gradient descent solves robust mean estimation in high dimensions.
problem High-dimensional robust mean estimation in the presence of adversarial outliers.
method Gradient descent with a structural lemma showing near-optimal solutions.
result Gradient descent can solve the robust mean estimation problem directly.
Robust CG methods avoid data corruption and solve structured statistical estimation problems.
problem Data corruption and heavy-tailed data in structured statistical estimation.
method Robustification of Conditional Gradient (CG) type methods using Huber's corruption model and robust mean gradient estimation.
result Robust CG methods converge linearly with correct sample complexity, even for high-dimensional problems.
Robust estimation methods find global minima efficiently via quasi-gradients.
problem Efficiently solving robust estimation problems with non-convex optimization.
method Identifying generalized quasi-gradients to guarantee low-regret algorithms.
result Generalized quasi-gradients ensure efficient approximation of global minima.
Paper introduces robust learning methods using coordinate gradient descent.
problem Supervised learning with corrupted features and labels.
method Coordinate gradient descent combined with robust estimators of partial derivatives.
result Robust learning methods with nearly identical numerical complexity to non-robust ones.
Faster policy learning via continuous-time gradients.
problem Efficiently estimating policy gradients for continuous-time systems.
method Approximating continuous-time gradients directly, using adaptive discretization.
result More efficient policy gradient estimator leads to faster learning.
New methods estimate policy value and gradients for deterministic policies from off-policy data.
problem Estimating policy value and gradients for deterministic policies from off-policy data.
method Proposed new doubly robust estimators based on kernelization approaches.
result Demonstrated a rate independent of horizon length for policy value and gradient estimation.
A method for estimating the median of gradients in stochastic optimization.
problem Robust gradient estimation in stochastic optimization for various applications.
method Stochastic Proximal Point Method for median gradient estimation.
result The proposed method can converge even under heavy-tailed, state-dependent noise.
New method improves off-policy critic evaluation in reinforcement learning.
problem High variance and instability in off-policy policy evaluation.
method Doubly robust estimators applied to actor-critic algorithms.
result Doubly robust estimation significantly improves performance in continuous control tasks.
Paper tackles robust M-estimation for high-dimensional data with heavy tails or arbitrary corruption.
problem Sparsity-constrained M-estimation with heavy-tailed or corrupted data. method Defines Robust Descent Condition (RDC) and uses Robust Hard Thresholding (IHT) with gradient estimators.
result Robust Hard Thresholding is minimax optimal for k-sparse high-dimensional linear and logistic regression with heavy tails or arbitrary corruption. New method improves neural network robustness to adversarial attacks.
problem Improving adversarial robustness of neural networks.
method Inspired by adaptive control theory, the approach uses persistency of excitation to constrain gradient descent updates.
result Networks trained with the PoE-motivated learning rate schedule are significantly more robust to adversarial attacks.
New method uses robust estimators for Newton's method in empirical risk minimization.
problem Improving robustness in empirical risk minimization.
method Robust Newton's method with gradient and Hessian replaced by robust estimators.
result Faster convergence rates in high-dimensional settings.
Robust multivariate linear regression methods for online and offline use.
problem Estimating parameters of multivariate Gaussian linear regression models robustly.
method Robust versions of least-square criterion with online and offline algorithms.
result Asymptotic normality of robust estimates proved under weak assumptions.
Robust boosting improves regression accuracy in noisy data.
problem Handling outliers in non-parametric regression.
method Two-stage approach: robust residual scale minimization followed by bounded loss optimization.
result Robust boosting outperforms standard methods in outlier-prone data.
Accelerated optimization methods improve robustness and privacy in estimation.
problem Improving robustness and privacy in estimation methods.
method Accelerated gradient methods based on Frank-Wolfe and projected gradient descent, with tailored learning rates and Nesterov's momentum.
result Reduction in iteration complexity, leading to stronger statistical guarantees.
Paper tackles efficient policy gradient estimation from off-policy data.
problem Estimating policy gradients from off-policy data is challenging and inefficient.
method Derives asymptotic lower bounds, proposes a meta-algorithm with 3-way robustness, and establishes convergence guarantees.
result Meta-algorithm achieves the lower bound on mean-squared error without parametric assumptions.
New algorithms robustly estimate mean with near-optimal error rates.
problem Outlier robust mean estimation in high-dimensional data.
method Stability condition and iterative filtering algorithms.
result Optimal error rates with subgaussian rates for robust mean estimation.
LEGO estimates tangent spaces more robustly than LPCA in noisy data.
problem Estimating tangent spaces in high-noise settings.
method Spectral method using graph Laplacian eigenvectors and gradient orthogonization.
result LEGO yields more robust tangent space estimates than LPCA.
Proposes SDRG to adjust missingness in machine learning models.
problem Systemic missingness in observational data leads to biased parameter estimation.
method Introduces SDRG using two models: weight-corrected gradients and per-covariate control variates.
result Empirically demonstrates convergence in training image classifiers with missing data.
Private minimum Hellinger distance estimators maintain robustness and efficiency while ensuring privacy.
problem Ensuring privacy in robust statistical estimation.
method Derive private minimum Hellinger distance estimators satisfying Hellinger differential privacy.
result Private minimum Hellinger distance estimators retain robustness and efficiency under privacy constraints.
New PG method derived from IS, improving variance reduction.
problem Improving policy gradient methods for better performance.
method Starting from the DR estimator, a new PG method is derived.
result Empirically shows the new DR-PG estimator is effective.
PPO's gradients are heavy-tailed, affecting learning; a robust estimator improves performance.
problem Heavy-tailedness of PPO gradients causing learning issues.
method Characterized heavy-tailed gradients, identified likelihood ratios and advantages as sources, proposed GMOM as a robust estimator.
result GMOM improves PPO performance without clipping tricks.
Paper develops a robust PP distributed quasi-Newton estimation for Byzantine machines.
problem Byzantine machines in distributed computing under Privacy Protection constraints.
method Robust PP distributed quasi-Newton estimation method that transmits only five vectors.
result Reduces privacy budgeting and transmission cost compared to gradient descent and Newton iteration.
New algorithms optimize risk for large datasets, improving efficiency.
problem Optimizing risk for large datasets with robust methods.
method Proposed algorithms for distributionally robust optimization with CVaR and χ² divergence uncertainty sets.
result Algorithms require independent gradient evaluations of training set size and parameters, suitable for large-scale applications.
Paper proposes a new method for robust modal regression.
problem Estimating the global mode of conditional density functions robustly.
method Directly approximates the gradient of modal regression risk using kernelized and neural-network-based log-density derivative estimators.
result Proposed methods achieve superior performance on various datasets.
Develops algorithm to escape saddle points in Byzantine settings.
problem Byzantine workers create fake local minima near saddle points.
method ByzantinePGD, a robust first-order algorithm.
result Converges to approximate true local minimizer with low iteration complexity.
Gradient boosting estimates Riesz representer for causal inference.
problem Estimating causal quantities using traditional methods is challenging and prone to variance issues.
method Gradient boosting algorithm to directly estimate Riesz representer.
result Gradient boosting performs similarly or better than traditional methods in estimating causal quantities.
Improved robust regression algorithms with faster runtime and better estimation rates.
problem Statistical regression problems under strong contamination model.
method Nearly-linear time algorithms using robust gradient descent and Sever framework.
result Improved estimation rates and runtime compared to state-of-the-art.
Unified view of stochastic optimization methods with improved convergence and robustness.
problem Stochastic convex composite optimization with noise.
method Estimate sequence approach, accelerated algorithms, robust strategies.
result Optimal complexity accelerated SVRG algorithm robust to noise.
Paper proposes MMD-Bayes for robust Bayesian estimation in misspecified models.
problem Inconsistent estimates in Bayesian statistics due to model misspecification.
method Uses Maximum Mean Discrepancy (MMD) to create a robust pseudo-likelihood.
result MMD-Bayes posterior is consistent and robust to model misspecification.
New method improves robust sparse association estimation.
problem Outliers in high-dimensional data.
method Splitting robust estimation into optimization phases, using augmented Lagrangian and adaptive gradient descent.
result Improved precision over existing methods.
New neural network architecture preserves gradient norms to approximate Lipschitz functions.
problem Training neural networks with strict Lipschitz constraints to ensure robustness and generalization.
method Identified gradient norm preservation as a necessary property, combined with norm-constrained weight matrices and GroupSort activation function.
result Norm-constrained GroupSort architectures can approximate Lipschitz functions and achieve tighter Wasserstein distance estimates.
New method to recover over-parameterized models corrupted during estimation.
problem Recovering statistical models corrupted after initial estimation.
method Robust estimation using over-parameterized models and redundancy.
result Stochastic gradient descent is well-suited for model repair, but sparsity is generally not repairable.
Novel algorithm reduces privacy noise in machine learning.
problem High privacy noise in machine learning algorithms.
method Robust statistics, specifically median and trimmed mean, to bound sensitivity of SGD iterates.
result Improved privacy-utility trade-off with reduced noise and computational efficiency.
SGLBO optimizes quantum circuits with fewer measurements, improving accuracy and noise resilience.
problem Efficiently optimizing parameterized quantum circuits with reduced measurement shots and noise.
method Developed SGLBO combining SGD and BO, with adaptive measurement-shot strategy and suffix averaging.
result Significantly reduces measurement-shot cost while improving accuracy and noise resilience.
New method improves MMD estimation without convexity assumptions.
problem Lack of theoretical guarantees for MMD estimation algorithms.
method Preconditioned gradient descent (PGD) scheme for MMD optimization.
result PGD scheme converges globally under specific conditions.
Improved linear regression with privacy and robustness guarantees.
problem Private and robust linear regression with adversarial corruption.
method Differentially private stochastic gradient descent with full-batch gradient descent and adaptive clipping.
result Near optimal sample complexity for both private and robust linear regression.
PaloBoost improves robustness against overfitting in TreeBoost.
problem Overfitting in TreeBoost models.
method Uses out-of-bag samples for gradient-aware pruning and adaptive learning rates.
result PaloBoost is robust to overfitting and less sensitive to parameters.
New approaches improve adversarial robustness of DEQs.
problem Adversarial vulnerability of DEQs.
method Developed approaches to estimate intermediate gradients and integrate them into attacking pipelines.
result Demonstrated adversarial robustness of DEQs competitive with deep networks.
New method provides tighter robustness guarantees for adversarial attacks.
problem Ensuring robustness against adversarial attacks in machine learning models.
method Developed a Second-order Smoothing (SoS) robustness certificate using Gaussian random smoothing.
result SoS certificates are tighter and provide improved robustness on high-dimensional datasets.
Zeno++ improves robustness of asynchronous SGD in fully asynchronous settings.
problem Byzantine failures in fully asynchronous SGD.
method Estimates descent of loss after applying candidate gradient.
result Proves convergence for non-convex problems under Byzantine failures.
AR-DAE approximates entropy gradient for machine learning models.
problem Intractable computation of entropy gradient for continuous distributions.
method Amortized residual denoising autoencoder (AR-DAE) to approximate entropy gradient.
result AR-DAE provides an unbiased gradient approximation for entropy.
A new algorithm reduces bias and variance in distributionally robust optimization.
problem Distributionally robust optimization with bias and variance issues.
method Prospect, a stochastic gradient-based algorithm that reduces hyperparameter tuning.
result Prospect achieves linear convergence and 2-3x faster convergence on various benchmarks.
This paper tackles robust control of LQR systems with multiplicative noise using policy gradient methods.
problem Robustness in reinforcement learning control of complex systems with multiplicative noise.
method Policy gradient algorithms with gradient domination property for non-convex cost functions.
result Global convergence of policy gradient algorithms to the globally optimum control policy.
Unified perspective on score matching and new estimators designed.
problem Infeasibility of maximum likelihood estimation in complex models.
method Minimum Stein discrepancy estimators, diffusion kernel Stein discrepancy (DKSD), diffusion score matching (DSM).
result Consistency, asymptotic normality, and robustness of DKSD and DSM estimators.
We develop methods for parameter estimation in settings with large-scale data sets, where traditional methods are no longer tenable. Our methods rely on stochastic approximations, which are computationally efficient as they maintain one iterate as a parameter estimate, and successively update that iterate based on a si…
New method prevents gradient attenuation in Lipschitz constrained convolutional networks.
problem Gradient norm attenuation in Lipschitz constrained convolutional networks.
method Block Convolution Orthogonal Parameterization (BCOP) to train scalable, expressive, provably Lipschitz convolutional networks.
result Empirically, BCOP parameterization is competitive with existing approaches to provable adversarial robustness and Wasserstein distance estimation.