arXiv research

We provide recovery guarantees for compressible signals that have been corrupted with noise and extend the framework introduced in \cite{bafna2018thwarting} to defend neural networks against $\ell_0$-norm, $\ell_2$-norm, and $\ell_{\infty}$-norm attacks. Our results are general as they can be applied to most unitary tr…

A new PCA method using T$\ell_1$-norm outperforms existing methods.

In this paper, we propose $\ell_p$-norm regularized models to seek near-optimal sparse portfolios. These sparse solutions reduce the complexity of portfolio implementation and management. Theoretical results are established to guarantee the sparsity of the second-order KKT points of the $\ell_p$-norm regularized models…

Signal estimation problems with smoothness and sparsity priors can be naturally modeled as quadratic optimization with $\ell_0$-"norm" constraints. Since such problems are non-convex and hard-to-solve, the standard approach is, instead, to tackle their convex surrogates based on $\ell_1$-norm relaxations. In this paper…

AdamW optimizes a constrained loss with $\ell_\infty$ norm constraint.

Generalizes inequality for complete manifolds involving homology classes.

The paper studies the minimum ℓ₁-norm interpolator's risk behavior in over-parameterized settings.

This paper tackles robustness of ensemble stumps and trees under general ℓ_p norm perturbations.

Advances robust principal component analysis with transformed ℓ1 regularization.

State-of-the-art subspace clustering methods are based on expressing each data point as a linear combination of other data points while regularizing the matrix of coefficients with $\ell_1$, $\ell_2$ or nuclear norms. $\ell_1$ regularization is guaranteed to give a subspace-preserving affinity (i.e., there are no conne…

CNN layers with large norms are still robust to adversarial attacks.

Paper tackles low-rank matrix recovery with column $\ell_{2,0}$-norm regularization.

In many applications, high-dimensional data points can be well represented by low-dimensional subspaces. To identify the subspaces, it is important to capture a global and local structure of the data which is achieved by imposing low-rank and sparseness constraints on the data representation matrix. In low-rank sparse …

This paper assesses Gaussian and Exponential mechanisms for certifying adversarial robustness.

In compressed sensing, in order to recover a sparse or nearly sparse vector from possibly noisy measurements, the most popular approach is $\ell_1$-norm minimization. Upper bounds for the $\ell_2$- norm of the error between the true and estimated vectors are given in [1] and reviewed in [2], while bounds for the $\ell_…

New algorithm solves $\ell_0$-norm constrained multilinear logistic regression for tensor data.

The paper proposes a new method for dictionary learning using $\ell_p$-norm maximization.

Feature hashing and other random projection schemes are commonly used to reduce the dimensionality of feature vectors. The goal is to efficiently project a high-dimensional feature vector living in $\mathbb{R}^n$ into a much lower-dimensional space $\mathbb{R}^m$, while approximately preserving Euclidean norm. These sc…

New method for factor analysis using nuclear and $\ell_0$ norms.

We give improved algorithms for the $\ell_{p}$-regression problem, $\min_{x} \|x\|_{p}$ such that $A x=b,$ for all $p \in (1,2) \cup (2,\infty).$ Our algorithms obtain a high accuracy solution in $\tilde{O}_{p}(m^{\frac{|p-2|}{2p + |p-2|}}) \le \tilde{O}_{p}(m^{\frac{1}{3}})$ iterations, where each iteration requires s…

Adversarial training is a principled approach for training robust neural networks. Despite of tremendous successes in practice, its theoretical properties still remain largely unexplored. In this paper, we provide new theoretical insights of gradient descent based adversarial training by studying its computational prop…

Proximal operators are of particular interest in optimization problems dealing with non-smooth objectives because in many practical cases they lead to optimization algorithms whose updates can be computed in closed form or very efficiently. A well-known example is the proximal operator of the vector $\ell_1$ norm, whic…

The paper tackles multi-armed bandits with vector losses, focusing on minimizing the $\ell^\infty$-norm of relative losses.

Paper develops algorithms for sparse linear regression with generalized elastic net penalty.

Uniformly finite homology is a coarse homology theory, defined via chains that satisfy a uniform boundedness condition. By construction, uniformly finite homology carries a canonical $\ell^\infty$-semi-norm. We show that, for uniformly discrete spaces of bounded geometry, this semi-norm on uniformly finite homology in …

We study the robustness properties of $\ell_1$ norm minimization for the classical linear regression problem with a given design matrix and contamination restricted to the dependent variable. We perform a fine error analysis of the $\ell_1$ estimator for measurements errors consisting of outliers coupled with noise. We…

A data filtering method for cluster analysis is proposed, based on minimizing a least squares function with a weighted $\ell_0$-norm penalty. To overcome the discontinuity of the objective function, smooth non-convex functions are employed to approximate the $\ell_0$-norm. The convergence of the global minimum points o…

This work improves robustness guarantees for neural networks using low rank representations.

Paper refines cross-lingual word embeddings using Manhattan norm.

The paper proposes a novel MKL approach for OCC using $\ell_p$-norm constraints.

Proposes an efficient method for sparse index tracking with $\ell_0$-norm constraints.

We discuss the problem of adaptive discrete-time signal denoising in the situation where the signal to be recovered admits a "linear oracle" -- an unknown linear estimate that takes the form of convolution of observations with a time-invariant filter. It was shown by Juditsky and Nemirovski (2009) that when the $\ell_2…

Proposes a new method for joint sample and feature selection in multi-view data.

Verifying robustness of neural networks given a specified threat model is a fundamental yet challenging task. While current verification methods mainly focus on the $\ell_p$-norm threat model of the input instances, robustness verification against semantic adversarial attacks inducing large $\ell_p$-norm perturbations,…

An elementary proof found for the double bubble problem in a specific norm.

Study tightens bounds for interpolating noisy data using minimum l1-norm.

Unified analysis of parameter norms in overparameterized linear models, revealing scaling laws and thresholds.

We establish a theoretical link between adversarial training and operator norm regularization for deep neural networks. Specifically, we prove that $\ell_p$-norm constrained projected gradient ascent based adversarial training with an $\ell_q$-norm loss on the logits of clean and perturbed inputs is equivalent to data-…

We study the column subset selection problem with respect to the entrywise $\ell_1$-norm loss. It is known that in the worst case, to obtain a good rank-$k$ approximation to a matrix, one needs an arbitrarily large $n^{Ω(1)}$ number of columns to obtain a $(1+ε)$-approximation to the best entrywise $\ell_1$-norm low ra…

We show how to turn any classifier that classifies well under Gaussian noise into a new classifier that is certifiably robust to adversarial perturbations under the $\ell_2$ norm. This "randomized smoothing" technique has been proposed recently in the literature, but existing guarantees are loose. We prove a tight robu…

In this paper, we study the problem of estimating the covariance matrix under differential privacy, where the underlying covariance matrix is assumed to be sparse and of high dimensions. We propose a new method, called DP-Thresholding, to achieve a non-trivial $\ell_2$-norm based error bound, which is significantly bet…

We simplify Thurston norm computation for 2-bridge link complements.

This paper tackles the problem of defending a neural network against adversarial attacks crafted with different norms (in particular $\ell_\infty$ and $\ell_2$ bounded adversarial examples). It has been observed that defense mechanisms designed to protect against one type of attacks often offer poor performance against…

Improved bounds for discrete probability distribution estimation under the ℓ∞ norm.

We consider the empirical risk minimization problem for linear supervised learning, with regularization by structured sparsity-inducing norms. These are defined as sums of Euclidean norms on certain subsets of variables, extending the usual $\ell_1$-norm and the group $\ell_1$-norm by allowing the subsets to overlap. T…

We propose $\ell_1$ norm regularized quadratic surface support vector machine models for binary classification in supervised learning. We establish their desired theoretical properties, including the existence and uniqueness of the optimal solution, reduction to the standard SVMs over (almost) linearly separable data s…

New neural network design resists small $\ell_\infty$-norm adversarial perturbations.

We consider the problem of Graphical lasso with an additional $\ell_{\infty}$ element-wise norm constraint on the precision matrix. This problem has applications in high-dimensional covariance decomposition such as in \citep{Janzamin-12}. We propose an ADMM algorithm to solve this problem. We also use a continuation st…