arXiv research

The paper studies the minimum ℓ₁-norm interpolator's risk behavior in over-parameterized settings.

Study tightens bounds for interpolating noisy data using minimum l1-norm.

A data filtering method for cluster analysis is proposed, based on minimizing a least squares function with a weighted $\ell_0$-norm penalty. To overcome the discontinuity of the objective function, smooth non-convex functions are employed to approximate the $\ell_0$-norm. The convergence of the global minimum points o…

The study analyzes robustness of estimators in linear models with adversarial errors.

Deep linear networks can closely approximate interpolants without improving risk.

The paper analyzes boosting and minimum-$\ell_1$-norm classifiers in high dimensions.

Develops exact convex optimization formulations for neural networks.

Lower bounds show OLS outperforms basis pursuit in overparameterized linear regression.

The paper tackles multi-armed bandits with vector losses, focusing on minimizing the $\ell^\infty$-norm of relative losses.

We give improved algorithms for the $\ell_{p}$-regression problem, $\min_{x} \|x\|_{p}$ such that $A x=b,$ for all $p \in (1,2) \cup (2,\infty).$ Our algorithms obtain a high accuracy solution in $\tilde{O}_{p}(m^{\frac{|p-2|}{2p + |p-2|}}) \le \tilde{O}_{p}(m^{\frac{1}{3}})$ iterations, where each iteration requires s…

Unified analysis of parameter norms in overparameterized linear models, revealing scaling laws and thresholds.

This paper considers the fundamental problem of learning a complete (orthogonal) dictionary from samples of sparsely generated signals. Most existing methods solve the dictionary (and sparse representations) based on heuristic algorithms, usually without theoretical guarantees for either optimality or complexity. The r…

Sparse optimization refers to an optimization problem involving the zero-norm in objective or constraints. In this paper, nonconvex approximation approaches for sparse optimization have been studied with a unifying point of view in DC (Difference of Convex functions) programming framework. Considering a common DC appro…

New model leads to optimal test loss in sparse linear regression.

We study the problem of globally recovering a dictionary from a set of signals via $\ell_1$-minimization. We assume that the signals are generated as i.i.d. random linear combinations of the $K$ atoms from a complete reference dictionary $D^*\in \mathbb R^{K\times K}$, where the linear combination coefficients are from…

Inflating the minimum norm interpolator improves linear regression generalization error.

We provide recovery guarantees for compressible signals that have been corrupted with noise and extend the framework introduced in \cite{bafna2018thwarting} to defend neural networks against $\ell_0$-norm, $\ell_2$-norm, and $\ell_{\infty}$-norm attacks. Our results are general as they can be applied to most unitary tr…

A new PCA method using T$\ell_1$-norm outperforms existing methods.

Statistical analysis of regularization in continual learning tasks.

Characterizes inductive bias in multi-channel linear CNNs with bounded weight norm.

New method estimates robust mean in high dimensions with minimized outliers.

Transfer learning improves MNI's performance in high-dimensional linear regression.

Our paper characterizes how ReLU affects GD's implicit bias in high-dimensional neural networks.

This study shows how optimizer choice affects adversarial robustness in neural networks.

We derive an upper bound on the local Rademacher complexity of $\ell_p$-norm multiple kernel learning, which yields a tighter excess risk bound than global approaches. Previous local approaches aimed at analyzed the case $p=1$ only while our analysis covers all cases $1\leq p\leq\infty$, assuming the different feature …

New method approximates complex kernel norms with random features, making learning tractable.

We introduce a financial portfolio optimization framework that allows us to automatically select the relevant assets and estimate their weights by relying on a sorted $\ell_1$-Norm penalization, henceforth SLOPE. Our approach is able to group constituents with similar correlation properties, and with the same underlyin…

Paper improves regret bounds for distributed experts problem.

New tensor formulation reveals gradient flow's bias in linear neural networks.

In this paper, we propose $\ell_p$-norm regularized models to seek near-optimal sparse portfolios. These sparse solutions reduce the complexity of portfolio implementation and management. Theoretical results are established to guarantee the sparsity of the second-order KKT points of the $\ell_p$-norm regularized models…

Signal estimation problems with smoothness and sparsity priors can be naturally modeled as quadratic optimization with $\ell_0$-"norm" constraints. Since such problems are non-convex and hard-to-solve, the standard approach is, instead, to tackle their convex surrogates based on $\ell_1$-norm relaxations. In this paper…

The paper characterizes functions of shallow ReLU NN denoisers under minimal norm constraints.

The paper analyzes the robustness of a minimum $\ell_2$ interpolator in high-dimensional linear regression.

AdamW optimizes a constrained loss with $\ell_\infty$ norm constraint.

Generalizes inequality for complete manifolds involving homology classes.

Minimum-norm solutions generalize well in over-parametrized neural networks.

A study on a surprising phase transition in model generalization error as parameters approach sample size.

Study shows how networks converge to minimum norm solutions with regularization.

This paper tackles robustness of ensemble stumps and trees under general ℓ_p norm perturbations.

Advances robust principal component analysis with transformed ℓ1 regularization.

State-of-the-art subspace clustering methods are based on expressing each data point as a linear combination of other data points while regularizing the matrix of coefficients with $\ell_1$, $\ell_2$ or nuclear norms. $\ell_1$ regularization is guaranteed to give a subspace-preserving affinity (i.e., there are no conne…

CNN layers with large norms are still robust to adversarial attacks.

Paper tackles low-rank matrix recovery with column $\ell_{2,0}$-norm regularization.

In many applications, high-dimensional data points can be well represented by low-dimensional subspaces. To identify the subspaces, it is important to capture a global and local structure of the data which is achieved by imposing low-rank and sparseness constraints on the data representation matrix. In low-rank sparse …

This paper assesses Gaussian and Exponential mechanisms for certifying adversarial robustness.

In compressed sensing, in order to recover a sparse or nearly sparse vector from possibly noisy measurements, the most popular approach is $\ell_1$-norm minimization. Upper bounds for the $\ell_2$- norm of the error between the true and estimated vectors are given in [1] and reviewed in [2], while bounds for the $\ell_…

New algorithm solves $\ell_0$-norm constrained multilinear logistic regression for tensor data.

The study examines how shallow neural nets converge to training samples or manifold points during diffusion.