This paper studies confidence regions for robust estimators using Wasserstein distance.
problem Developing robust estimators against model misspecification.
method Wasserstein distributionally robust optimization.
result Asymptotic normality and optimal confidence regions for distributionally robust estimators.
Unified framework for adversarial attacks using min-max optimization.
problem Improving adversarial robustness and attack generation.
method General framework of min-max optimization over multiple domains.
result Substantial attack improvement and robustness improvement.
This study reveals a Min-Max property in LeNet's convolutional layers, enhancing adversarial robustness.
problem Uncertainty in the connection weights of convolutional layers in neural networks.
method Demonstrates the Min-Max property through back propagation-based training and a simplified convolution formulation.
result The Min-Max property improves adversarial robustness, indicating a stronger uncertainty in the model parameters.
Proposes active sampling for improving fairness in machine learning.
problem Improving fairness in machine learning models, especially for disadvantaged groups.
method Simple active sampling and reweighting strategies for min-max fairness.
result Proves the rate of convergence to a min-max fair solution for convex problems.
Boosted decision stumps and trees are made robust against adversarial attacks efficiently.
problem Efficiently making boosted decision stumps and trees robust against adversarial attacks.
method Exact min-max robust loss and test error computation for decision stumps, upper bound optimization for trees.
result State-of-the-art robust test error rates for boosted trees on various datasets.
Improves model generalization by minimizing loss sharpness.
problem Overparameterized models often fail to generalize well despite low training loss.
method Sharpness-Aware Minimization (SAM) minimizes both loss value and sharpness.
result SAM improves model generalization across various datasets and models.
Develops fair classifiers robust to training distribution perturbations.
problem Ensuring fairness in classifiers robust to training data perturbations.
method Formulates a min-max objective function to minimize distributionally robust training loss while maintaining fairness for perturbed distributions. Uses an iterative online learning algorithm to find a fair and robust classifier.
result Our classifier maintains fairness and accuracy for a wide range of perturbations compared to state-of-the-art fair classifiers.
ALPS improves neural network robustness and generalization.
problem Challenges in designing effective regularization schemes for adversarial robustness.
method Adversarial Labelling of Perturbed Samples (ALPS) using synthetic samples and min-max formulation.
result ALPS achieves state-of-the-art regularization performance and adversarial robustness.
New MAML variant prioritizes hardest tasks, improving robustness.
problem Meta-learning's focus on average performance, ignoring worst-case.
method Reformulate MAML to minimize max loss over observed tasks.
result Task-robust model performs well across various task distributions.
New DAM method improves AUC scores in medical image classification.
problem Maximizing AUC in large-scale medical image classification.
method Proposes AUC margin loss for robust optimization, conducts extensive empirical studies.
result Improves performance on four medical image classification tasks, achieving 1st place on Stanford CheXpert.
New methods solve min-max problems on manifolds using Riemannian Hamiltonians.
problem Min-max optimization on Riemannian manifolds.
method Riemannian Hamiltonian methods (RHM) to minimize the Hamiltonian function.
result RHM leads to correct search directions and global optimality in min-max problems.
Optimizes deep learning training by treating it as an optimal control problem.
problem Fragility of deep neural networks to adversarial inputs.
method Formulates adversarial training as a min-max optimization problem and interprets it as an optimal control problem.
result Provides the first convergence analysis of adversarial training algorithm.
SAWAR improves SA models by making them more robust to data uncertainties.
problem Improving survival analysis models' robustness to data uncertainties.
method Adversarial robustness through Min-Max optimization with CROWN-IBP.
result SAWAR outperforms baseline methods and SOTA models across various metrics.
Study on deep learning IDS resistance against adversarial attacks.
problem Vulnerabilities in deep learning-based IDS against adversarial attacks.
method Apply min-max optimization to train IDS against adversarial samples.
result Adversarial attack methods can be used in continuous domains and boost IDS robustness.
Robust VB framework handles contamination using min-max median aggregation.
problem Handling contamination and outliers in datasets.
method Partition data into subsets, formulate robust optimization problem, use min-max median KL divergence.
result Min-max median formulation improves robustness and statistical rates.
Survey of advances in non-convex min-max optimization for applications.
problem Finding optimal solutions in non-convex, non-concave min-max problems.
method Selective review of theoretical and algorithmic advances.
result Exciting recent advances in solving non-convex min-max problems.
New algorithms improve DRSL for large-scale problems.
problem Distributionally robust learning for real-world applications.
method Variance-reduced stochastic extra-gradient algorithms for min-max optimization.
result Provable faster convergence rates than existing approaches.
Develops robust learning methods for datasets with sub-populations.
problem Robust performance and generalization to unseen testing populations in datasets with sub-populations.
method Min-max-regret (MMR) formulation for distribution-free robust hierarchical model.
result Empirical MMR enjoys regret guarantees on training and unseen testing populations.
The paper tackles robust statistical methods using Wasserstein DRO formulations.
problem Distributional uncertainty in learning from limited samples.
method Min-max distributionally robust optimization with Wasserstein DRO formulations.
result Error bounds free from the curse of dimensionality.
A framework for federated adversarial learning with convergence analysis.
problem Unique vulnerabilities to adversarial attacks in federated learning.
method Formulates a general federated adversarial learning framework with inner and outer loops for client-side adversarial training and server-side model aggregation.
result The minimum loss under the proposed algorithm can converge to ε with chosen learning rate and communication rounds.
Our work proves convergence to low robust training loss for polynomial width ReLU networks.
problem Understanding why adversarial training leads to low robust training loss in over-parameterized neural nets.
method Extending convergence theory for standard supervised training to adversarial training, using tools from online learning and showing ReLU networks can approximate the step function.
result Convergence to low robust training loss for polynomial width ReLU networks under natural assumptions.
This work proposes SDI regularization to improve adversarial robustness.
problem Improving adversarial robustness of deep neural networks.
method SDI regularization term to complement adversarial training.
result Combining SDI with AT variants enhances robustness and generalization.
Paper tackles constrained robust optimization without gradients.
problem Constrained robust (min-max) optimization in a black-box setting.
method Integrates ZO gradient estimator with alternating projected SGD-ascent method.
result Proposed ZO-Min-Max framework converges sub-linearly under mild conditions.
We propose a robust estimator to improve maximum likelihood in probabilistic models.
problem Overfitting and sensitivity to noise in maximum likelihood estimation.
method Distributionally robust maximum likelihood estimator that minimizes worst-case expected log-loss.
result The robust estimator is statistically consistent and performs well in regression and classification tasks.
Paper proposes robust reinforcement learning method for better real-world application.
problem Overfitting in reinforcement learning algorithms limits their applicability to real-world scenarios.
method Formalizes robust reinforcement learning as a min-max game with a Wasserstein constraint and proposes an efficient solver.
result Significant gains in performance on high-dimensional MuJuCo environments compared to standard and robust algorithms.
New approach improves deep learning robustness in medical imaging.
problem Deep learning models are vulnerable to adversarial examples in medical imaging.
method Propose a min-max learning scheme to generate adversarial examples and filter them out.
result Proposed method significantly improves robustness of deep learning models in medical imaging.
Bayesian optimization methods improved for min max optimization problems.
problem Min-max optimization for unknown functions.
method Extended Bayesian optimization to min-max problems with new acquisition functions.
result Improved acquisition functions lead to better solutions.
SRPO improves AI alignment with human preferences through self-improvement and task-independent optimization.
problem AI models trained with RLHF lack self-correction mechanisms and struggle with task generalization.
method SRPO formulates the preference learning problem as a min-max objective, optimizing a self-improvement policy and a generative policy in an adversarial fashion, making the solution task-independent.
result SRPO outperforms existing methods, achieving 90% AI Win-Rate on XSum and 56% on Arena-Hard prompts after a single revision.
Study shows fast rates for inverse reinforcement learning with linear rewards.
problem Entropy-regularized min-max inverse reinforcement learning in finite-horizon MDPs.
method Structural and statistical analysis of Min-Max-IRL with pseudo-self-concordance.
result Both trajectory-level KL divergence and parameter error decay at O(n−1). A new algorithm reduces the cost of training robust deep neural networks.
problem High computational cost in training robust deep neural networks.
method Iterative descent-ascent algorithm based on saddle-point dynamical system.
result The algorithm converges to robust optimal solution under adversarial constraints.
A wide array of machine learning problems are formulated as the minimization of the expectation of a convex loss function on some parameter space. Since the probability distribution of the data of interest is usually unknown, it is is often estimated from training sets, which may lead to poor out-of-sample performance.…
New method for robustly estimating barycenters in data aggregation.
problem Outliers and noise in data measures hinder traditional OT barycenter estimation.
method Proposes a novel scalable approach using semi-unbalanced neural optimal transport.
result Demonstrates robustness to outliers and class imbalance.
The paper explains how simple methods can converge to optimal solutions in complex neural games.
problem Finding optimal solutions in neural games with non-convex objectives.
method Theoretical framework using hidden convexity and overparameterization, with path-length bounds and PŁ conditions.
result Simple gradient methods can converge to Nash equilibria in non-convex min-max games under certain conditions.
Study evaluates adversarial training for deep learning IDSs against various attacks.
problem Evasion attacks against deep learning-based IDSs.
method Investigated adversarial training using min-max approach on CNN and RNN.
result Adversarial training improves robustness against five attack methods.
Riemannian algorithms converge at Euclidean rates for geodesically convex-concave problems.
problem Min-max optimization on Riemannian manifolds.
method RCEG method and RGDA for geodesically strongly-convex-concave problems.
result RCEG achieves linear convergence rate in geodesically strongly-convex-concave cases.
The study uncovers the breakdown of Gaussian universality in high-dimensional empirical risk minimization.
problem Understanding the breakdown of Gaussian universality in high-dimensional empirical risk minimization.
method Extending the Convex Gaussian Min-Max Theorem to non-Gaussian settings, deriving asymptotic min-max characterizations, and proving asymptotic equivalence of regularizers.
result The projection of the ERM estimator onto a test covariate approximately follows a Gaussian convolution under certain conditions.
We use the theory of rectifiable metric spaces to define a Dirichlet energy of Lipschitz functions defined on the support of integral currents. This energy is obtained by integration of the square of the norm of the tangential derivative, or equivalently of the approximate local dilatation, of the Lipschitz functions. …
New method improves PCA robustness using Wasserstein distances.
problem Uncertainty in probability distribution affects PCA robustness.
method Distributionally robust optimization with Wasserstein distances.
result Explicit reformulation leads to efficient smoothing algorithm.
Framework for worst-case generation using Wasserstein space optimization.
problem Evaluating robustness and stress-testing systems under distribution shifts.
method Min-max optimization over continuous probability distributions in Wasserstein space.
result Global convergence guarantees for the proposed Gradient Descent Ascent scheme.
Proposes an efficient alternative to nonconvex-nonconcave min-max optimization.
problem Min-max optimization challenges in nonconvex-nonconcave settings.
method Introduces ε-greedy adversarial equilibrium model and proves its existence.
result Existence of ε-greedy adversarial equilibrium for smooth bounded functions.
Paper tackles fast convergence for non-convex strongly-concave min-max problems.
problem Non-convex strongly-concave min-max problems in deep learning.
method Proximal stage-based method with PL condition for faster convergence.
result Established fast convergence in primal objective gap and duality gap.
AIB method improves robustness against adversarial perturbations.
problem Optimizing the IB principle for better robustness and understanding compression effects.
method Proposes adversarial information bottleneck (AIB) method to optimize IB principle without explicit distribution assumptions.
result Demonstrates effectiveness in learning more invariant representations and mitigating adversarial perturbations.
A new method for deep learning imbalance or noise, ABSGD, improves efficiency and effectiveness.
problem Data imbalance or label noise in deep learning.
method A modification of momentum SGD with individual-level weights proportional to loss values.
result Guaranteed convergence to stationary points of DRO problems, capturing class diversity.
Unified method for deep active learning improves performance and efficiency.
problem Improving deep active learning performance and efficiency.
method Unified and principled approach using Wasserstein distance for querying and training.
result Consistently better empirical performance and time-efficient query strategy compared to baselines.
A new active learning method considers both uncertainty and diversity to minimize labeling and decision costs.
problem Classical AL approaches fail to capture data distribution in unlabeled data, leading to mislabeling of outliers.
method CBAL considers classification uncertainty and instance diversity, using a min-max approach to minimize labeling and decision costs.
result Extensive experiments show CBAL outperforms state-of-the-art AL approaches.
This paper presents a novel clustering concept that is based on jointly learned nonlinear transforms (NTs) with priors on the information loss and the discrimination. We introduce a clustering principle that is based on evaluation of a parametric min-max measure for the discriminative prior. The decomposition of the pr…
We propose a novel data-driven method to learn a mixture of multiple kernels with random features that is certifiabaly robust against adverserial inputs. Specifically, we consider a distributionally robust optimization of the kernel-target alignment with respect to the distribution of training samples over a distributi…
Model-free approach to hedge path-dependent options using min-max optimization.
problem Hedging path-dependent options with maturity T using a static portfolio of vanilla options.
method Model-free approach based on primal-dual Martingale Optimal Transport (MOT) problem, solving a min-max optimization problem.
result Provides theoretical bounds on hedging error at maturity T.