Monotone adversarial corruptions degrade optimal learning algorithms.
problem Optimal learning algorithms' reliance on exchangeability and independence is challenged.
method Introduces a monotone adversarial corruption model where an adversary adds monotone corruptions to a clean dataset.
result Optimal learning algorithms achieve suboptimal expected error on new test points.
New method tackles adversarial sign-corrupted isotonic regression, estimating monotonic signals under heavy dependence.
problem Estimating monotonic signals when responses are sign-corrupted and adversarially designed to violate monotonicity.
method Developed ASCIFIT, a three-step estimation procedure using PAVA with pre- and post-processing corrections.
result Theoretical guarantees of sharp high probability upper bounds and minimax lower bounds for ASCIFIT.
A new algorithm finds minimizers in dueling optimization with a monotone adversary.
problem Finding minimizers in dueling optimization with a monotone adversary.
method Introduces and studies dueling optimization with a monotone adversary, designs an efficient randomized algorithm.
result Efficient algorithm incurs cost O(d) and iteration complexity O(dlog(1/ε)2), asymptotically optimal. Paper tackles ranking items with a semi-random comparison graph and a monotone adversary.
problem Ranking items based on pairwise comparisons from a semi-random comparison graph with a monotone adversary.
method Developed a weighted maximum likelihood estimator (MLE) and an SDP-based approach to reweight the semi-random graph.
result Achieves near-optimal sample complexity, up to a log^2(n) factor, for identifying the top-K preferred items.
Improves GANs training through game theory.
problem Hard training of GANs due to antagonistic networks.
method Rewrote GAN training as a variational inequality and introduced a stochastic relaxed forward-backward algorithm.
result Algorithm converges to an exact solution or a neighborhood of it under monotonicity.
Study reveals how neural network smoothness affects their vulnerability to adversarial attacks.
problem Understanding adversarial vulnerability in deep learning networks.
method Analysis of manifold smoothness and generalization capability of deep neural networks trained with local errors.
result High generalization accuracy requires a fast power-law decay of eigen-spectrum of hidden representations.
New algorithms reduce regret for online submodular maximization under various conditions.
problem Online optimization of submodular functions with adversarial or random utilities.
method Characterized strongly DR-submodular functions and derived bounds for different utility classes.
result Logarithmic regret bounds for adversarial strongly DR-submodular functions and submodular functions with random order.
There is a growing body of literature showing that deep neural networks are vulnerable to adversarial input modification. Recently this work has been extended from image classification to malware classification over boolean features. In this paper we present several new methods for training restricted networks in this …
Extends conformal prediction for controlling expected risk of monotone loss functions.
problem Controlling expected risk of monotone loss functions.
method Generalizes split conformal prediction with coverage guarantee, extending to distribution shift, quantile risk, multiple, adversarial, and expectations of U-statistics.
result Tight up to an O(1/n) factor, with worked examples in computer vision and natural language processing. Adversarial training leads to large generalization gap, decomposed into bias and variance.
problem Understanding the large generalization gap in adversarially trained models.
method Bias-Variance decomposition of test risk as a function of adversarial perturbation radius.
result Bias increases monotonically with adversarial perturbation radius and is dominant in test risk.
It is becoming increasingly important to understand the vulnerability of machine learning models to adversarial attacks. In this paper we study the feasibility of robust learning from the perspective of computational learning theory, considering both sample and computational complexity. In particular, our definition of…
New method for conditional sampling using M-GANs, likely-free inference.
problem Conditional sampling of probability measures.
method Developed a novel computational approach called M-GANs based on block triangular transport.
result Accurate sampling of conditional measures in various applications.
ThiopheneIV is a new solver for implied volatility with proven monotonicity.
problem Efficiently solving implied volatility in financial models.
method Monotone core with Euler-Chebyshev and Halley steps, exact arithmetic proof, practical boundary handling.
result ThiopheneIV agrees closely with multiprecision Black reference prices at low latency.
New method for robust learning from batches, even adversarial ones.
problem Learning from batches that may be corrupt or adversarial.
method General framework for robust learning, derived from optimal robust algorithms.
result First robust agnostic learning algorithms for various distributions.
In this paper, we consider first-order convergence theory and algorithms for solving a class of non-convex non-concave min-max saddle-point problems, whose objective function is weakly convex in the variables of minimization and weakly concave in the variables of maximization. It has many important applications in mach…
Study robustness of split conformal prediction under adversarial attacks.
problem Ensuring distribution-free coverage guarantees in CP under adversarial conditions.
method Theoretical analysis and extensive experiments on split conformal prediction robustness.
result Prediction coverage varies with calibration-time attack strength, enabling control over coverage under adversarial tests.
The safety and robustness of learning-based decision-making systems are under threats from adversarial examples, as imperceptible perturbations can mislead neural networks to completely different outputs. In this paper, we present an adaptive view of the issue via evaluating various test-time smoothing defense against …
Study learns optimal auctions from corrupted or perturbed bidder valuation samples.
problem Learning revenue-optimal auctions from corrupted or perturbed samples.
method Proves upper bounds, proposes algorithms for learning near-optimal auctions.
result Proves tight upper bounds and proposes algorithms for near-optimal auctions.
New approaches improve adversarial robustness of DEQs.
problem Adversarial vulnerability of DEQs.
method Developed approaches to estimate intermediate gradients and integrate them into attacking pipelines.
result Demonstrated adversarial robustness of DEQs competitive with deep networks.
This work examines how adversarial vulnerability changes with the dimensionality of the subspace of perturbations.
problem Understanding adversarial vulnerability in constrained input spaces.
method Investigates adversarial vulnerability in subspace V of the input space X with varying dimensions, using PGD attacks and analyzing the dependence on ε and dim(V)/dim(X). result Adversarial success of PGD attacks is a monotonically increasing function of $ε( rac{dim(V)}{dim(X)})^{ rac{1}{q}}$.
Applications in machine learning, optimization, and control require the sequential selection of a few system elements, such as sensors, data, or actuators, to optimize the system performance across multiple time steps. However, in failure-prone and adversarial environments, sensors get attacked, data get deleted, and a…
Adaptive learning rate improves FTRL's performance in online learning.
problem Optimizing FTRL's learning rate for competitive regret in online learning.
method Formulated as a sequential decision-making problem, introduced competitive analysis framework, and proposed stability-penalty matching update rules.
result Achieved a constant competitive ratio under specific conditions, enabling Best-Of-Both-Worlds algorithms.
New approach optimizes policies in adversarial MDPs using adversarial learning.
problem Optimizing policies in adversarial Markov decision processes.
method Adversarial learning on advantage functions, extending previous reductions.
result Stronger regret criteria and performance guarantees for policy optimization.
Study on adversarial training dynamics in high dimensions using SGD.
problem Analyzing adversarial training of models in high-dimensional settings.
method Deriving deterministic equivalents for SGD iterates under Gaussian mixtures.
result No constant learning rate guarantees monotone descent in adversarial training.
The paper tackles robust submodular maximization under matroid constraints, providing approximation algorithms for summary extraction.
problem Maximizing submodular functions while ensuring high value even after deletions.
method Constant-factor approximation algorithms for centralized and streaming settings, considering both non-monotone and monotone objectives.
result Approximation algorithms with space complexity depending on matroid rank and deleted elements, achieving improved factors in monotone cases.
The paper calculates how much data is needed to learn decision lists in the presence of evasion attacks.
problem Quantifying sample complexity for robust learning of decision lists against evasion attacks.
method PAC learning framework, Lipschitz condition on distributions, lower and upper bounds on sample complexity.
result Upper and lower bounds on sample complexity for robust learning of decision lists, showing exponential vs polynomial dependence on adversary's budget.
Efficient algorithm detects communities in noisy graphs with robustness.
problem Community detection in stochastic block models with adversarial corruptions.
method Efficient algorithm using semidefinite programming for robust clustering.
result Achieves error O(ε)+e−2C(1±o(1)) for ε-fraction of corruptions. Save for some special cases, current training methods for Generative Adversarial Networks (GANs) are at best guaranteed to converge to a `local Nash equilibrium` (LNE). Such LNEs, however, can be arbitrarily far from an actual Nash equilibrium (NE), which implies that there are no guarantees on the quality of the found…
The paper addresses monotonicity in machine learning models for fairness and accountability.
problem Ensuring fairness and accountability in transparent machine learning models.
method Study of three types of monotonicity (individual, weak pairwise, strong pairwise) and propose monotonic groves of neural additive models.
result Monotonic groves of neural additive models maintain transparency, accountability, and fairness.
New parameterization of neural networks with Lipschitz bounds for robustness.
problem Developing robust neural networks with Lipschitz bounds.
method Introducing a new parameterization that admits a Lipschitz bound during training without requiring projections or barrier functions.
result The new parameterization improves robustness to adversarial attacks in image classification.
Applications of safety, security, and rescue in robotics, such as multi-robot target tracking, involve the execution of information acquisition tasks by teams of mobile robots. However, in failure-prone or adversarial environments, robots get attacked, their communication channels get jammed, and their sensors may fail…
Generative Adversarial Networks (GAN) have become one of the most successful frameworks for unsupervised generative modeling. As GANs are difficult to train much research has focused on this. However, very little of this research has directly exploited game-theoretic techniques. We introduce Generative Adversarial Netw…
MonoFlow rethinks GANs using Wasserstein gradient flows.
problem Inconsistencies between GAN theory and practice.
method Unified generative modeling framework based on Wasserstein gradient flows.
result Adversarial training can be seen as particle flow optimization.
Probit Monotone BART estimates binary outcomes using monotonic functions.
problem Estimating conditional mean functions for binary outcomes with monotonicity constraints.
method Proposes a new BART variant that incorporates monotonicity constraints for binary outcomes.
result Allows for more precise estimation of monotonic functions in binary outcome models.
Monotone neural networks can approximate and interpolate functions efficiently.
problem Understanding the efficiency and expressiveness of monotone neural networks.
method Solving the monotone interpolation problem using depth-4 networks and comparing size bounds with arbitrary networks.
result Monotone neural networks can approximate and interpolate functions efficiently, but may require exponential size in high dimensions.
We study adversarial perturbations when the instances are uniformly distributed over {0,1}n. We study both "inherent" bounds that apply to any problem and any classifier for such a problem as well as bounds that apply to specific problems and specific hypothesis classes. As the current literature contains multiple…
Improves k-NN for monotonic data with robustness against noise.
problem Class noise in real-life data violates monotonic constraints in k-NN.
method Monotonic Fuzzy k-NN (MonFkNN) with new fuzzy membership calculation.
result Significant accuracy improvements and robustness against monotonic noise.
Study examines explainable machine learning for monotonic models, finding Integrated gradients better for strong monotonicity.
problem Applying explainable machine learning to science-informed models.
method Proposed axioms for monotonicity, tested Shapley value and Integrated gradients methods.
result Integrated gradients provides better explanations for strong monotonicity.
Despite the improved accuracy of deep neural networks, the discovery of adversarial examples has raised serious safety concerns. In this paper, we study two variants of pointwise robustness, the maximum safe radius problem, which for a given input sample computes the minimum distance to an adversarial example, and the …
Bayesian approach improves online prediction accuracy without distributional assumptions.
problem Online construction of confidence sets for black-box models.
method Combines empirical distribution with Bayesian regularization to predict quantiles.
result Adaptive algorithm with low regret and correct coverage probability for iid data.
Formula proves monotonicity for anisotropic minimal hypersurfaces.
problem Understanding anisotropic minimal hypersurfaces.
method Proved a monotonicity formula under a sign assumption on the Minkowski norm.
result Monotonicity formula for anisotropic minimal hypersurfaces.
New example of manifolds with monotonic heat kernels found.
problem Understanding monotonicity of heat kernels on manifolds.
method Analyzing new examples and classifying flat tori.
result Generic metrics fail monotonicity at large times.
Learning performance can show non-monotonic behavior. That is, more data does not necessarily lead to better models, even on average. We propose three algorithms that take a supervised learning model and make it perform more monotone. We prove consistency and monotonicity with high probability, and evaluate the algorit…
The paper develops algorithms to restore monotonicity in non-monotone functions.
problem Non-monotone solutions from heuristic algorithms need to be corrected.
method Develops algorithms to restore monotonicity with limited queries.
result Restores monotonicity while degrading the function value by at most ε.
Nonnegative matrix factorization (NMF) factorizes a non-negative matrix into product of two non-negative matrices, namely a signal matrix and a mixing matrix. NMF suffers from the scale and ordering ambiguities. Often, the source signals can be monotonous in nature. For example, in source separation problem, the source…
In [S. Basu, A. Gabrielov, N. Vorobjov, Semi-monotone sets. arXiv:1004.5047v2 (2011)] we defined semi-monotone sets, as open bounded sets, definable in an o-minimal structure over the reals, and having connected intersections with all translated coordinate cones in R^n. In this paper we develop this theory further by d…
COMET learns monotonic neural networks by incorporating counterexamples.
problem Enforcing monotonicity constraints in neural networks for real-world tasks.
method Counterexample-guided learning technique for ReLU neural networks.
result COMET achieves state-of-the-art results and improves model quality.
Monotonic differentiable sorting networks improve upon previous methods.
problem Non-monotonicity in differentiable sorting networks.
method Relaxation of conditional swap operations using sigmoid functions to ensure monotonicity.
result Monotonic differentiable sorting networks improve upon previous methods.