Proposes a method to create fair, robust predictors that remain consistent across different scenarios.
problem Creating fair and robust machine learning models that behave consistently across different scenarios.
method Graphical criteria and a model-agnostic framework called CIP based on HSCIC.
result Demonstrates the effectiveness of CIP in enforcing counterfactual invariance across various datasets.
Study robust learning without knowing perturbation sets, using interactions with attackers.
problem Learning robust predictors against unknown adversarial perturbations.
method Examined different interaction models with adversarial attackers, derived bounds on sample complexity and interactions.
result Upper bounds on sample complexity and lower bounds on interactions in various models.
Bayes predictor remains robust to ignorable missingness shifts.
problem Challenges in prediction with missing covariates and shifts in missingness reasons.
method Bayesian approach and different prediction methods.
result Bayes predictor remains unchanged by ignorable shifts, but robust prediction requires disregarding missingness for non-ignorable shifts.
Proposes new method for calibrating treatment effect predictors.
problem Calibrating predictors of heterogeneous treatment effects.
method Causal isotonic calibration and cross-calibration.
result Achieves fast calibration rates under weak conditions.
This work precisely characterizes and improves the tradeoff between robustness and accuracy in linear regression.
problem Tradeoff between robustness and accuracy in adversarial training.
method Characterizes the effect of augmentation on standard error in linear regression; proves RST improves robust error without sacrificing standard error.
result RST improves both standard and robust error for neural networks under various perturbations.
Overparameterized MLR fits hyper-curves, improving model robustness.
problem Improper predictors degrade model generalizability.
method Parameterizing with a scalar and monomial basis, fitting hyper-curves.
result Hyper-curve approach yields robust predictions for noisy data.
New minimax optimal learner for robust predictors against adversarial examples.
problem Learning robust predictors against adversarial examples.
method Global perspective and new algorithmic ideas.
result Characterizes classes of predictors that are robustly learnable.
New theory explains how noisy, high-dimensional data can still lead to robust predictions.
problem Modern machine learning models achieve high performance with noisy, high-dimensional data.
method Synthesizes principles from Information Theory, Latent Factor Models, and Psychometrics to clarify predictive robustness.
result Predictive robustness arises from data architecture and model capacity, not just data cleanliness.
Causal predictors don't generalize better across domains than non-causal predictors.
problem How well do causal predictors generalize across different domains?
method 16 prediction tasks on tabular datasets, selecting causal features.
result Causal predictors do not outperform non-causal predictors in domain generalization.
While adversarial training can improve robust accuracy (against an adversary), it sometimes hurts standard accuracy (when there is no adversary). Previous work has studied this tradeoff between standard and robust accuracy, but only in the setting where no predictor performs well on both objectives in the infinite data…
New robust estimator improves variable selection and coefficient estimation in linear regression with heavy-tailed errors and outliers.
problem Heavy-tailed errors and anomalous predictors in high-dimensional regression.
method Adaptive PENSE estimator for robust variable selection and estimation.
result Adaptive PENSE estimator provides reliable results even under very heavy-tailed errors and aberrant predictors.
This paper proposes a new algorithmic framework, predictor-verifier training, to train neural networks that are verifiable, i.e., networks that provably satisfy some desired input-output properties. The key idea is to simultaneously train two networks: a predictor network that performs the task at hand,e.g., predicting…
New framework improves adversarial robustness in one-stage L2D.
problem Adversarial robustness in one-stage Learning-to-Defer (L2D).
method Formalizes attacks, proposes cost-sensitive adversarial surrogate losses, establishes theoretical guarantees.
result Improves robustness against untargeted and targeted attacks while preserving clean performance.
This thesis explores robust machine learning against adversarial examples.
problem How to create machine learning systems robust to adversarial examples.
method Theoretical exploration and development of new learning algorithms with robustness guarantees.
result Developed new learning algorithms with provable robustness guarantees.
New methods for CI testing under model misspecification.
problem Challenges in CI testing with misspecified models.
method Proposes new approximations and upper bounds for testing errors of regression-based CI tests.
result Introduces the Rao-Blackwellized Predictor Test (RBPT) robust against misspecified inductive biases.
This paper presents Sparse Partitioning, a Bayesian method for identifying predictors that either individually or in combination with others affect a response variable. The method is designed for regression problems involving binary or tertiary predictors and allows the number of predictors to exceed the size of the sa…
Robust TOT regression method handles outliers in tensor data.
problem Outliers in tensor data affect standard TOT regression.
method ROTOT method using a single loss function for outliers and robust MPCA for predictor.
result ROTOT method reduces influence of both casewise and cellwise outliers.
New method tackles OOD robustness with a single additional variable.
problem Out-of-distribution generalization with unobserved confounders.
method Identifiability assumptions using a single additional variable.
result Superior empirical performance on benchmark tasks.
Paper introduces a method to create robust representations against covariate shifts.
problem Distribution shift between training and testing data in machine learning.
method Introduces a variational objective with two components: discriminative representation and invariant support.
result Optimal representations ensure robustness to covariate shifts, improving performance on DomainBed.
A new method accounts for predictor dependencies in XAI feature rankings.
problem Assumption of predictor independence in XAI methods leads to unreliable feature rankings.
method Proposes a proxy method to modify XAI outcomes considering predictor dependencies.
result Allows more accurate feature ranking in models with correlated predictors.
To improve accuracy and speed of regressions and classifications, we present a data-based prediction method, Random Bits Regression (RBR). This method first generates a large number of random binary intermediate/derived features based on the original input matrix, and then performs regularized linear/logistic regressio…
Method improves treatment effect prediction robust to unknown covariate shifts.
problem Estimating heterogeneous treatment effects for different populations.
method Post-processing CATE T-learners with multi-accurate predictors to handle unknown covariate shifts.
result Improves bias and mean squared error in simulations with covariate shifts.
Boosts barely robust learners to be more adversarially robust.
problem Learning predictors robust to small perturbations on a small fraction of data.
method Oracle-efficient algorithm for robustness with larger perturbation set.
result Qualitative and quantitative equivalence between strongly robust and barely robust learning.
Method constructs confidence regions for linear models with arbitrary predictors.
problem Constructing confidence regions for linear models with non-linear predictors.
method Mixed Integer Linear Programming for constraints.
result Empty confidence regions for hypothesis testing.
CreDRO learns credal ensembles via distributionally robust optimization, improving EU quantification.
problem Quantifying predictive epistemic uncertainty in credal models.
method Distributionally robust optimization to capture EU from training randomness and potential distribution shifts.
result Empirically, CreDRO outperforms existing credal methods on various tasks.
Proposes two-stage robust and sparse distributed inference for large-scale data.
problem Statistical inference in large-scale, high-dimensional, and outlier-contaminated data.
method Two-stage approach: model selection with robust Lasso, fusion of local selections, and bootstrap methods for inference.
result Robust and computationally efficient inference procedures for variable selection, confidence intervals, and standard deviation approximations.
A new estimator improves financial econometrics by providing reliable inference.
problem Poor performance of standard regression methods in financial economics with thick-tailed predictors.
method Developed an unbiased, consistent, and asymptotically normal estimator for linear regression.
result The new method delivers reliable inference under heteroskedasticity and quantile regression.
UM-GNN improves GNN robustness against poisoning attacks.
problem Vulnerability of GNNs to poisoning attacks.
method UM-GNN uses epistemic uncertainties from message passing to build a surrogate predictor.
result UM-GNN achieves significantly improved robustness against poisoning attacks.
Proposes a new probabilistic framework for domain generalization.
problem Learning predictors robust to unseen domain shifts.
method Quantile Risk Minimization (QRM) and Empirical QRM (EQRM) algorithms.
result Empirical QRM outperforms state-of-the-art baselines on various datasets.
In order to identify important variables that are involved in making optimal treatment decision, Lu et al. (2013) proposed a penalized least squared regression framework for a fixed number of predictors, which is robust against the misspecification of the conditional mean model. Two problems arise: (i) in a world of ex…
Study highlights robustness issues in healthcare diagnostic models due to distribution shifts.
problem Robustness of diagnostic models in healthcare is compromised by distribution shifts.
method Theoretical analysis and simulation studies to understand and mitigate shortcuts learned by models.
result Ignoring covariates or using invariant learning approaches leads to non-robust predictors.
MAP perturbation models have emerged as a powerful framework for inference in structured prediction. Such models provide a way to efficiently sample from the Gibbs distribution and facilitate predictions that are robust to random noise. In this paper, we propose a provably polynomial time randomized algorithm for learn…
MEC improves efficiency and robustness in semi-supervised inference.
problem Efficient inference with limited labeled data and robust uncertainty quantification.
method Machine-Learning-Assisted Generalized Entropy Calibration (MEC) using cross-fitted, calibration-weighted PPI.
result MEC achieves semiparametric efficiency bounds under weaker assumptions and provides near-nominal coverage.
We consider new formulations and methods for sparse quantile regression in the high-dimensional setting. Quantile regression plays an important role in many applications, including outlier-robust exploratory analysis in gene selection. In addition, the sparsity consideration in quantile regression enables the explorati…
Study shows how to improve contextual bandits with loss predictors.
problem Improving minimax regret in contextual bandits with loss predictors.
method Developed novel algorithmic techniques for upper bounds and lower bounds in various settings.
result Optimal regret is O ( min { T , E T 1 4 } ) \mathcal{O}(\min\{\sqrt{T}, \sqrt{\mathcal{E}}T^\frac{1}{4}\}) O ( min { T , E T 4 1 }) when E \mathcal{E} E is known, and O ( E T 1 3 ) \mathcal{O}(\sqrt{\mathcal{E}}T^\frac{1}{3}) O ( E T 3 1 ) if E \mathcal{E} E is unknown. New method detects out-of-distribution samples in regression tasks.
problem Detecting instances far from training data in regression models.
method Estimating predictor entropy based on nearest neighbors and generative models.
result A new method for robust OOD detection in regression tasks.
While machine learning systems show high success rate in many complex tasks, research shows they can also fail in very unexpected situations. Rise of machine learning products in safety-critical industries cause an increase in attention in evaluating model robustness and estimating failure probability in machine learni…
Least squares estimator fails to achieve optimal risk in bounded distributions, but non-linear predictors can.
problem Optimal risk in bounded distributions for constrained least squares.
method Comparison of least squares and non-linear predictors.
result Non-linear predictors can achieve optimal risk O ( d / n ) O(d/n) O ( d / n ) in bounded distributions. New trade-off found between accuracy and adversarial robustness in regression.
problem Finding a balance between accuracy and robustness in regression models.
method Deriving a fundamental trade-off between standard and adversarial risk in regression with polynomial ridge functions.
result A necessary condition for achieving adversarial robustness without significant accuracy loss.
Study robustness of global feature effect explanations in machine learning models.
problem Vulnerability of global feature effect explanations to data and model perturbations.
method Theoretical bounds and experimental evaluation of partial dependence plots and accumulated local effects.
result Quantifies the gap between best and worst-case scenarios of misinterpreting machine learning predictions globally.
Modular neural causal models outperform other models in generalization and adaptation.
problem Robust out-of-distribution generalization and fast adaptation in machine learning.
method Factorizing data generating process into modules using only causal parents as predictors.
result Modular neural causal models offer robust generalization and fast adaptation, especially in low data regimes.
New truthful calibration errors improve model ranking in multiclass prediction.
problem Non-truthful calibration errors can mislead model comparisons.
method Introduced perfectly truthful calibration errors for multiclass predictions.
result Truthful calibration errors preserve decision-theoretic dominance and stabilize model rankings.
Deep neural networks improve AFT model for non-linear predictors.
problem Nonlinearity in predictors of AFT models.
method Apply DNNs to fit AFT models using Gehan-type loss and sub-sampling.
result DeepR-AFT outperforms parametric and semiparametric models.
Develops a SAS approach for high-dimensional risk prediction using unlabeled data.
problem Challenges in risk modeling with EHR data due to lack of direct disease outcomes and high dimensionality.
method Surrogate Assisted Semi-supervised Learning (SAS) approach leveraging unlabeled and labeled data.
result Valid inference for predicted risk even when underlying model is dense and mis-specified.
The paper connects counterfactual fairness to robust prediction and group fairness using causal context.
problem The challenge of ensuring fairness in AI systems when counterfactuals cannot be directly observed.
method Using causal context to bridge counterfactual fairness, robust prediction, and group fairness.
result Counterfactual fairness is equivalent to group fairness metrics in specific contexts.
Study shows semi-supervised learning can be more robust with fewer labeled examples.
problem Learning robust predictors in semi-supervised PAC model with minimal labeled data.
method Characterizes the minimal labeled and unlabeled data required for robust learning.
result Proves nearly matching upper and lower bounds on labeled sample complexity.
Develops robust learning framework under distributional perturbations.
problem Learning robust to data distributional changes.
method Distributionally Robust Optimization (DRO) under Wasserstein metric.
result Establishes performance guarantees and tractable formulations.
New approach for algorithms that learn predictors to improve performance.
problem Improving algorithm performance by learning better predictions.
method Identify functional dependence and apply online learning techniques.
result Improved results in various online settings with simpler analysis.