arXiv research

We provide recovery guarantees for compressible signals that have been corrupted with noise and extend the framework introduced in \cite{bafna2018thwarting} to defend neural networks against $\ell_0$-norm, $\ell_2$-norm, and $\ell_{\infty}$-norm attacks. Our results are general as they can be applied to most unitary tr…

AdamW optimizes a constrained loss with $\ell_\infty$ norm constraint.

Paper finds a fast method for a matrix norm proximal operator.

The paper tackles multi-armed bandits with vector losses, focusing on minimizing the $\ell^\infty$-norm of relative losses.

This paper tackles robustness of ensemble stumps and trees under general ℓ_p norm perturbations.

New neural network design resists small $\ell_\infty$-norm adversarial perturbations.

This work improves robustness guarantees for neural networks using low rank representations.

Proposes a new method for joint sample and feature selection in multi-view data.

Uniformly finite homology is a coarse homology theory, defined via chains that satisfy a uniform boundedness condition. By construction, uniformly finite homology carries a canonical $\ell^\infty$-semi-norm. We show that, for uniformly discrete spaces of bounded geometry, this semi-norm on uniformly finite homology in …

Feature hashing and other random projection schemes are commonly used to reduce the dimensionality of feature vectors. The goal is to efficiently project a high-dimensional feature vector living in $\mathbb{R}^n$ into a much lower-dimensional space $\mathbb{R}^m$, while approximately preserving Euclidean norm. These sc…

We consider the problem of Graphical lasso with an additional $\ell_{\infty}$ element-wise norm constraint on the precision matrix. This problem has applications in high-dimensional covariance decomposition such as in \citep{Janzamin-12}. We propose an ADMM algorithm to solve this problem. We also use a continuation st…

This paper assesses Gaussian and Exponential mechanisms for certifying adversarial robustness.

Improved bounds for discrete probability distribution estimation under the ℓ∞ norm.

CNN layers with large norms are still robust to adversarial attacks.

This paper tackles the problem of defending a neural network against adversarial attacks crafted with different norms (in particular $\ell_\infty$ and $\ell_2$ bounded adversarial examples). It has been observed that defense mechanisms designed to protect against one type of attacks often offer poor performance against…

Sparse coding techniques for image processing traditionally rely on a processing of small overlapping patches separately followed by averaging. This has the disadvantage that the reconstructed image no longer obeys the sparsity prior used in the processing. For this purpose convolutional sparse coding has been introduc…

Enhances robustness of AT frameworks to multiple perturbations without increasing training complexity.

Study shows momentum-based optimizers like Muon and MomentumGD bias towards KKT points in smooth homogeneous models.

We discuss the problem of adaptive discrete-time signal denoising in the situation where the signal to be recovered admits a "linear oracle" -- an unknown linear estimate that takes the form of convolution of observations with a time-invariant filter. It was shown by Juditsky and Nemirovski (2009) that when the $\ell_2…

Study analyzes perturbations in singular subspaces under random noise.

This study shows how optimizer choice affects adversarial robustness in neural networks.

This paper explores adversarial training limits and improves model robustness against norm-bounded perturbations.

Adam's bias shifts from full-batch to max-margin of different norms for separable data.

In this paper, we study the problem of estimating the covariance matrix under differential privacy, where the underlying covariance matrix is assumed to be sparse and of high dimensions. We propose a new method, called DP-Thresholding, to achieve a non-trivial $\ell_2$-norm based error bound, which is significantly bet…

Motivated by the study of $Q$-learning algorithms in reinforcement learning, we study a class of stochastic approximation procedures based on operators that satisfy monotonicity and quasi-contractivity conditions with respect to an underlying cone. We prove a general sandwich relation on the iterate error at each time,…

Adversarial examples are carefully perturbed in-puts for fooling machine learning models. A well-acknowledged defense method against such examples is adversarial training, where adversarial examples are injected into training data to increase robustness. In this paper, we propose a new attack to unveil an undesired pro…

Neural networks have been shown to be vulnerable against minor adversarial perturbations of their inputs, especially for high dimensional data under $\ell_\infty$ attacks. To combat this problem, techniques like adversarial training have been employed to obtain models which are robust on the training set. However, the …

Unified signSGD and gradient descent analysis for neural networks.

Active sampling algorithm for linear regression with various norms and improved query complexity.

New method optimizes offline linear bandits using different confidence sets.

Generative models improve adversarial robustness by adding synthetic data.

New method improves subspace iteration for eigenvectors in machine learning.

Using sparse-inducing norms to learn robust models has received increasing attention from many fields for its attractive properties. Projection-based methods have been widely applied to learning tasks constrained by such norms. As a key building block of these methods, an efficient operator for Euclidean projection ont…

One-pass algorithm finds small subset for $\ell_p$ subspace approximation with additive error.

We give improved algorithms for the $\ell_{p}$-regression problem, $\min_{x} \|x\|_{p}$ such that $A x=b,$ for all $p \in (1,2) \cup (2,\infty).$ Our algorithms obtain a high accuracy solution in $\tilde{O}_{p}(m^{\frac{|p-2|}{2p + |p-2|}}) \le \tilde{O}_{p}(m^{\frac{1}{3}})$ iterations, where each iteration requires s…

Novel framework improves randomized smoothing for various norms.

Data augmentation improves robustness in adversarial training.

Researchers redefine $\ell^\infty$-cohomology for groups and spaces, linking it to amenability, hyperbolicity, and algorithmic undecidability.

We apply the OSCAR (octagonal selection and clustering algorithms for regression) in recovering group-sparse matrices (two-dimensional---2D---arrays) from compressive measurements. We propose a 2D version of OSCAR (2OSCAR) consisting of the $\ell_1$ norm and the pair-wise $\ell_{\infty}$ norm, which is convex but non-d…

Deep neural networks perform well on real world data but are prone to adversarial perturbations: small changes in the input easily lead to misclassification. In this work, we propose an attack methodology not only for cases where the perturbations are measured by $\ell_p$ norms, but in fact any adversarial dissimilarit…

New algorithm achieves optimal privacy and efficiency in non-Euclidean convex optimization.

We propose practical algorithms for entrywise $\ell_p$-norm low-rank approximation, for $p = 1$ or $p = \infty$. The proposed framework, which is non-convex and gradient-based, is easy to implement and typically attains better approximations, faster, than state of the art. From a theoretical standpoint, we show that th…

A new PCA method using T$\ell_1$-norm outperforms existing methods.

We characterize the effectiveness of a classical algorithm for recovering the Markov graph of a general discrete pairwise graphical model from i.i.d. samples. The algorithm is (appropriately regularized) maximum conditional log-likelihood, which involves solving a convex program for each node; for Ising models this is …

New approach to certifiably robust neural networks using Boolean function perspective.

Paper tackles image reconstruction from limited data using polyhedral norms and convex regularizers.

New method accelerates steepest descent for convex optimization.

Study on curvature bounds and geodesic dimension in sub-Finsler Heisenberg groups.