Robust learning mixtures of linear regressions improve robustness.
problem Improving robustness in learning mixtures of linear regressions.
method Connecting mixtures of linear regressions and mixtures of Gaussians with thresholding for a quasi-polynomial time algorithm.
result The algorithm has significantly better robustness than previous results.
This paper studies robust regression in the settings of Huber's ε ε ε -contamination models. We consider estimators that are maximizers of multivariate regression depth functions. These estimators are shown to achieve minimax rates in the settings of ε ε ε -contamination models for various regression problems including nonpa…
Improved linear regression with privacy and robustness guarantees.
problem Private and robust linear regression with adversarial corruption.
method Differentially private stochastic gradient descent with full-batch gradient descent and adaptive clipping.
result Near optimal sample complexity for both private and robust linear regression.
Study improves robustness and sparsity in linear regression with adversarial outliers and heavy-tailed noise.
problem Outliers and heavy-tailed noise in linear regression coefficients.
method Sharp concentration inequalities and generic chaining.
result Sharper error bounds under weaker assumptions.
Robustly estimates linear regression coefficients with adversarial and noisy data.
problem Estimating robust linear regression coefficients with adversarial and noisy data.
method Adversarial robust weighted Huber regression with polynomial computational complexity.
result Derives an estimation error bound that depends on the stable rank and condition number of the covariance matrix.
The generalized linear model (GLM) plays a key role in regression analyses. In high-dimensional data, the sparse GLM has been used but it is not robust against outliers. Recently, the robust methods have been proposed for the specific example of the sparse GLM. Among them, we focus on the robust and sparse linear regre…
Efficiently estimates sparse linear regression with heavy-tailed data and outliers.
problem Sparse estimation of linear regression coefficients with heavy-tailed covariates and noises, including outliers.
method Efficient computation of robust estimator with nearly optimal error bound.
result Nearly optimal error bound for robust sparse estimation.
Improved estimator reduces bias in statistical learning models.
problem Asymptotic bias in classic WDRO estimator.
method Adjusted Wasserstein distributionally robust estimator.
result Asymptotic unbiased estimator with smaller MSE.
This work precisely characterizes and improves the tradeoff between robustness and accuracy in linear regression.
problem Tradeoff between robustness and accuracy in adversarial training.
method Characterizes the effect of augmentation on standard error in linear regression; proves RST improves robust error without sacrificing standard error.
result RST improves both standard and robust error for neural networks under various perturbations.
The paper explores robustness in linear regression models under adversarial attacks.
problem The impact of test-time adversarial attacks on linear regression models.
method Quantitative estimates and phase transitions analysis.
result Precise characterization of tradeoffs between adversarial robustness and accuracy.
New method for robust linear regression in nearly linear time.
problem High-dimensional robust linear regression with adversarial corruption.
method Proposes estimators for two settings with near linear time complexity.
result Achieves optimal sample complexities and recovery guarantees.
Efficiently performs robust and sparse kernel regression.
problem Robust and sparse kernel regression.
method Sign gradient descent and early stopping.
result Sign gradient descent achieves robust and sparse kernel regression efficiently.
Improved SGD for robust linear and ReLU regression with adversarial corruptions.
problem Robust regression with adversarial corruptions in streaming data.
method Stochastic gradient descent (SGD-exp) with exponentially decaying step size.
result Nearly linear convergence to true parameter with up to 50% Massart corruption rate.
The study examines robustness auditing for linear regression, improving existing methods and identifying computational challenges.
problem Detecting small subsets of data that can reverse regression coefficients.
method Empirical study of mixed integer quadratically constrained optimization and exact greedy methods, combined with a spectral algorithm.
result Existing methods largely outperform state of the art, but computational bottlenecks remain, especially for higher dimensions.
Robust Bayesian models are appealing alternatives to standard models, providing protection from data that contains outliers or other departures from the model assumptions. Historically, robust models were mostly developed on a case-by-case basis; examples include robust linear regression, robust mixture models, and bur…
Paper develops robust Bayesian models for linear regression under adversarial perturbations.
problem Ensuring reliable machine learning models under data perturbations.
method Formulates adversarial Bregman divergence loss, computes adversarial perturbation, introduces adversarially robust posteriors, derives generalization certificates.
result Derives first rigorous generalization certificates for adversarially robust Bayesian linear regression.
Study on robustness in linear regression models, focusing on adversarial perturbations.
problem Understanding and improving robustness in linear regression models to adversarial perturbations.
method Developed a two-stage adversarial learning framework that incorporates model structure information.
result Proved the consistency and developed the Bahadur representation of the adversarially robust estimator.
Robust multivariate linear regression methods for online and offline use.
problem Estimating parameters of multivariate Gaussian linear regression models robustly.
method Robust versions of least-square criterion with online and offline algorithms.
result Asymptotic normality of robust estimates proved under weak assumptions.
Paper proposes a new method for selective inference in robust regression.
problem Statistical inference after removing outliers identified by robust methods.
method Conditional SI using piecewise-linear homotopy continuation.
result Proposed method is applicable to a wide class of robust regression and outlier detection methods.
Improved robust regression algorithms with faster runtime and better estimation rates.
problem Statistical regression problems under strong contamination model.
method Nearly-linear time algorithms using robust gradient descent and Sever framework.
result Improved estimation rates and runtime compared to state-of-the-art.
Estimates GLMs robustly against label corruptions.
problem Learning GLMs under adversarial label corruptions.
method Iterative trimmed maximum likelihood estimator.
result Achieves minimax near-optimal risk.
Enhances GBDT robustness with one-hot encoding and regularization.
problem Low robustness of GBDT models against covariate perturbation.
method One-hot encoding to linear framework, risk decomposition, L 1 L_1 L 1 or L 2 L_2 L 2 regularization. result Regularization enhances GBDT robustness.
Paper studies IPG method's robustness in noisy distributed linear regression.
problem Distributed linear regression with noise.
method Iteratively Pre-conditioned Gradient-descent (IPG) method.
result IPG method's robustness against noise compared favorably to state-of-the-art algorithms.
Unified framework SVAM learns GLMs robustly to adversarial label corruption.
problem Learning GLMs under adversarial label corruption.
method SVAM framework based on variance reduction technique.
result Provable model recovery guarantees superior to state-of-the-art.
Study robust linear regression without distributional assumptions for heavy-tailed responses.
problem Linear regression with heavy-tailed responses and no distributional assumptions.
method Combining truncated least squares, median-of-means, and aggregation theory to construct a non-linear estimator.
result Achieves excess risk of order d / n d/n d / n with optimal sub-exponential tail. New algorithm for robust regression with subgaussian error bound.
problem Linear regression in the presence of outliers and finite moments.
method Adaptation of spectral method to linear regression problem.
result Optimal sub-gaussian error bound for robust regression.
Gradient boosting algorithms construct a regression predictor using a linear combination of ``base learners''. Boosting also offers an approach to obtaining robust non-parametric regression estimators that are scalable to applications with many explanatory variables. The robust boosting algorithm is based on a two-stag…
Algorithm solves robust linear regression with block Lewis weights.
problem Group distributionally robust least squares problem.
method Algorithm based on geometric construction and block Lewis weights, using accelerated proximal methods.
result Improves over known methods for moderate accuracy regimes and matches state-of-the-art guarantees.
Ability for accurate hospital case cost modelling and prediction is critical for efficient health care financial management and budgetary planning. A variety of regression machine learning algorithms are known to be effective for health care cost predictions. The purpose of this experiment was to build an Azure Machine…
We study the problem of robust linear regression with response variable corruptions. We consider the oblivious adversary model, where the adversary corrupts a fraction of the responses in complete ignorance of the data. We provide a nearly linear time estimator which consistently estimates the true regression vector, e…
Study robustness of early-stopping GD for linear regression attacks.
problem Robustness of gradient-descent methods to adversarial attacks.
method Early-stopping strategies, gradient-descent, Mahalanobis attacks, feature-dependent learning rates, data transformations.
result Early-stopped GD is optimally robust to Euclidean-norm attacks but sub-optimal for Mahalanobis attacks.
Study enhances robustness of In-CVaR based regression models under perturbation and contamination.
problem Enhancing robustness of nonlinear regression models under perturbation and contamination.
method Introduces interval conditional value-at-risk (In-CVaR) and rigorously analyzes its robustness properties under both perturbation and contamination.
result The In-CVaR based estimator is qualitatively robust in terms of the Prokhorov metric if and only if the largest portion of losses is trimmed.
Overparameterized MLR fits hyper-curves, improving model robustness.
problem Improper predictors degrade model generalizability.
method Parameterizing with a scalar and monomial basis, fitting hyper-curves.
result Hyper-curve approach yields robust predictions for noisy data.
New algorithm for robust high-dimensional linear regression is both fast and statistically optimal.
problem Challenges in high-dimensional linear regression under heavy-tailed noise or outliers.
method Projected sub-gradient descent algorithm for sparse and low-rank regression problems.
result Algorithm achieves linear convergence and statistical optimality under various noise conditions.
New robust estimator improves variable selection and coefficient estimation in linear regression with heavy-tailed errors and outliers.
problem Heavy-tailed errors and anomalous predictors in high-dimensional regression.
method Adaptive PENSE estimator for robust variable selection and estimation.
result Adaptive PENSE estimator provides reliable results even under very heavy-tailed errors and aberrant predictors.
We consider the task of robust non-linear regression in the presence of both inlier noise and outliers. Assuming that the unknown non-linear function belongs to a Reproducing Kernel Hilbert Space (RKHS), our goal is to estimate the set of the associated unknown parameters. Due to the presence of outliers, common techni…
Paper proposes robust tensor regression method for tensor data analysis.
problem Outliers in tensor data analysis can make existing methods sensitive.
method Nonconvex relaxation of tensor tubal rank in optimization framework.
result Global convergence of proposed estimation algorithm under mild assumptions.
Improved efficient robust regression with near-linear time and subquadratic samples.
problem Robust linear regression with unknown covariance matrix under Gaussian covariates.
method Near-linear time algorithm using subquadratic samples, complemented by SQ and polynomial lower bounds.
result Achieves prediction error O ( ε κ ) O(\sqrt{εκ}) O ( ε κ ) for ε κ ≲ 1 εκ\lesssim 1 ε κ ≲ 1 , improving over prior works. Mix-IRLS solves imbalanced mixed linear regression problems efficiently.
problem Imbalanced mixed linear regression problems.
method Sequential robust regression approach.
result Mix-IRLS outperforms other methods on imbalanced mixtures and real-world datasets.
Corrects mismatch in consistency of nuisance estimators for doubly robust methods.
problem Mismatch in consistency of nuisance estimators in doubly robust methods.
method Calibrated debiased machine learning (calibrated DML) with isotonic regression adjustment.
result Calibrated DML yields doubly robust asymptotic normality with slower convergence of nuisance estimators.
ADA augments data using AR replicas for robust regression.
problem Improving robustness in nonlinear over-parametrized regression.
method Extends Anchor regression (AR) for data augmentation, using replicas of modified samples.
result ADA provides more robust regression predictions compared to state-of-the-art solutions.
Study on linear regression robustness to adversarial attacks.
problem Adversarial attacks on linear regression models.
method Analysis of prediction error bounds, asymptotic results, convex optimization.
result Adversarial error can grow to infinity with more features, while test error goes to zero.
We provide a novel -- and to the best of our knowledge, the first -- algorithm for high dimensional sparse regression with constant fraction of corruptions in explanatory and/or response variables. Our algorithm recovers the true sparse parameters with sub-linear sample complexity, in the presence of a constant fractio…
c-lasso is a Python tool for robust and sparse regression with linear constraints.
problem Sparse and robust linear regression with linear constraints.
method Estimates coefficients and scale under linear constraints using perspective M-estimators.
result Provides estimators for various loss functions with linear constraints.
New findings show some designs can't be robustly recovered in linear regression.
problem Robust linear regression with adversarial corruption.
method Investigated well-spreadness of design matrices for robust recovery.
result Certified well-spreadness of random matrices efficiently, but hardness for small observations.
Near-optimal algorithms for mean estimation and linear regression with Gaussian covariates and Huber contamination.
problem Gaussian mean estimation and linear regression with Gaussian covariates in the presence of Huber contamination.
method Near-optimal algorithms with optimal error guarantees, achieving sample complexity n = i l d e O ( d / ε 2 ) n = ilde{O}(d/ε^2) n = i l d e O ( d / ε 2 ) and almost linear runtime. result First sample near-optimal and almost linear-time algorithms with optimal error guarantees for both problems.
Improved robustness in multivariate regression and classification with DRO under Wasserstein metric.
problem Outliers in covariates and responses.
method Distributionally Robust Optimization (DRO) with Wasserstein metric ambiguity set and regularization.
result Significant improvement in predictive error and robustness.
In high-dimensional data, many sparse regression methods have been proposed. However, they may not be robust against outliers. Recently, the use of density power weight has been studied for robust parameter estimation and the corresponding divergences have been discussed. One of such divergences is the γ γ γ -divergence a…