The paper tackles multi-armed bandits with vector losses, focusing on minimizing the ℓ∞-norm of relative losses.
problem Minimizing the ℓ∞-norm of relative losses in multi-armed bandits with multiple losses. method Defines relative loss vector, derives lower bounds, and provides matching algorithms for both fixed-confidence best-arm identification and regret minimization.
result Derives problem-dependent sample complexity lower bound and matching algorithms for fixed-confidence best-arm identification.
We provide recovery guarantees for compressible signals that have been corrupted with noise and extend the framework introduced in \cite{bafna2018thwarting} to defend neural networks against ℓ0-norm, ℓ2-norm, and ℓ∞-norm attacks. Our results are general as they can be applied to most unitary tr…
AdamW optimizes a constrained loss with ℓ∞ norm constraint.
problem Understanding the optimization behavior of AdamW with ℓ∞ norm constraint. method Analyzing AdamW as a smoothed version of SignGD and connecting it to Frank-Wolfe optimization.
result AdamW implicitly performs constrained optimization with ℓ∞ norm constraint. We discuss the problem of adaptive discrete-time signal denoising in the situation where the signal to be recovered admits a "linear oracle" -- an unknown linear estimate that takes the form of convolution of observations with a time-invariant filter. It was shown by Juditsky and Nemirovski (2009) that when the $\ell_2…
This paper tackles robustness of ensemble stumps and trees under general ℓ_p norm perturbations.
problem The vulnerability of ensemble stumps and trees to small input perturbations under the ℓ_∞ norm.
method Developed dynamic programming algorithms for robustness verification and certified defense under general ℓ_p norm perturbations.
result First certified defense method for ensemble stumps and trees under ℓ_p norm perturbations.
New neural network design resists small ℓ∞-norm adversarial perturbations.
problem Vulnerability of neural networks to small ℓ∞-norm adversarial perturbations. method Designing ℓ∞-dist neurons and constructing ℓ∞-dist nets, proving their 1-Lipschitz property and expressive power. result Certified robustness of ℓ∞-dist nets with state-of-the-art performance on various datasets. This work improves robustness guarantees for neural networks using low rank representations.
problem Certified robustness to adversarial perturbations in neural networks.
method Low rank representations to provide improved robustness guarantees.
result Improved robustness guarantees for ℓ∞ perturbations using natural low rank representations. Proposes a new method for joint sample and feature selection in multi-view data.
problem Cannot detect latent subsets of samples and remove outliers.
method Weighted Sparse Partial Least Squares (ℓ∞/ℓ0-wsPLS) method for joint sample and feature selection. result Developed globally convergent algorithm and iterative algorithms for multi-view data fusion.
Uniformly finite homology is a coarse homology theory, defined via chains that satisfy a uniform boundedness condition. By construction, uniformly finite homology carries a canonical ℓ∞-semi-norm. We show that, for uniformly discrete spaces of bounded geometry, this semi-norm on uniformly finite homology in …
Proximal operators are of particular interest in optimization problems dealing with non-smooth objectives because in many practical cases they lead to optimization algorithms whose updates can be computed in closed form or very efficiently. A well-known example is the proximal operator of the vector ℓ1 norm, whic…
Feature hashing and other random projection schemes are commonly used to reduce the dimensionality of feature vectors. The goal is to efficiently project a high-dimensional feature vector living in Rn into a much lower-dimensional space Rm, while approximately preserving Euclidean norm. These sc…
We consider the problem of Graphical lasso with an additional ℓ∞ element-wise norm constraint on the precision matrix. This problem has applications in high-dimensional covariance decomposition such as in \citep{Janzamin-12}. We propose an ADMM algorithm to solve this problem. We also use a continuation st…
This paper assesses Gaussian and Exponential mechanisms for certifying adversarial robustness.
problem Certifying adversarial robustness using randomized smoothing mechanisms.
method Proposes a generic framework to assess the appropriateness of randomized smoothing mechanisms.
result Gaussian mechanism is an appropriate option for certifying both ℓ2-norm and ℓ∞-norm robustness. Improved bounds for discrete probability distribution estimation under the ℓ∞ norm.
problem Estimating discrete probability distributions under the ℓ∞ norm with improved bounds.
method Minimax bounds in expectation and high-probability tail bounds.
result Resolved open questions posed in Kontorovich and Painsky (JMLR, 2025), including a fully empirical tightest risk bound and identifying the worst-case extremal distribution.
CNN layers with large norms are still robust to adversarial attacks.
problem Understanding the relationship between layer norms and adversarial robustness in CNNs.
method Theoretical analysis of ℓ1 and ℓ∞ norms, norm decay method, adversarial training frameworks. result Adversarially robust CNNs can have comparable or larger layer norms than non-adversarially robust ones.
This paper tackles the problem of defending a neural network against adversarial attacks crafted with different norms (in particular ℓ∞ and ℓ2 bounded adversarial examples). It has been observed that defense mechanisms designed to protect against one type of attacks often offer poor performance against…
Sparse coding techniques for image processing traditionally rely on a processing of small overlapping patches separately followed by averaging. This has the disadvantage that the reconstructed image no longer obeys the sparsity prior used in the processing. For this purpose convolutional sparse coding has been introduc…
Enhances robustness of AT frameworks to multiple perturbations without increasing training complexity.
problem Defending against the union of multiple perturbations in adversarial training.
method SNAP technique that augments a network with shaped noise to enhance robustness.
result 14%-to-20% improvement in adversarial accuracy for ResNet-18 on CIFAR-10.
Active sampling algorithm for linear regression with various norms and improved query complexity.
problem Efficiently querying a few entries of a target vector for near optimal minimizers of linear regression.
method Lewis weight sampling and active sampling algorithms for different p norms. result Optimal query complexity for p∈(0,1), 1<p<2, and 2<p<∞. This study connects Jacobian regularization to adversarial robustness and improves generalization.
problem Adversarial attacks make deep neural networks vulnerable.
method Developed a connection between Jacobian regularization and adversarial training, and established robust generalization gaps.
result Jacobian norms are related to both standard and robust generalization.
Study shows momentum-based optimizers like Muon and MomentumGD bias towards KKT points in smooth homogeneous models.
problem Understanding the implicit bias of momentum-based optimizers on smooth homogeneous models.
method Analysis of Muon, MomentumGD, Signum, and Adam optimizers under decaying learning rate schedules.
result Momentum-based optimizers approximate steepest descent trajectories and bias towards KKT points of margin maximization problems.
We give improved algorithms for the ℓp-regression problem, minx∥x∥p such that Ax=b, for all p∈(1,2)∪(2,∞). Our algorithms obtain a high accuracy solution in O~p(m2p+∣p−2∣∣p−2∣)≤O~p(m31) iterations, where each iteration requires s…
Study analyzes perturbations in singular subspaces under random noise.
problem Understanding singular vector and subspace changes in signal-plus-noise models.
method Generalized Davis-Kahan-Wedin theorem for any unitarily invariant norm, considering ℓ∞ and ℓ2,∞ bounds. result Fine-grained insights into singular vector and subspace perturbations, including ℓ∞ and ℓ2,∞ bounds. This study shows how optimizer choice affects adversarial robustness in neural networks.
problem Understanding and improving adversarial robustness in neural networks.
method Revisiting known results linking robust classifiers and minimum norm solutions, combining them with recent optimizer bias findings.
result Achieving both perfect standard accuracy and robustness with certain optimizers under specific conditions.
Unified algorithm for minimizing composite functions with flexible design.
problem Minimizing composite functions with specific structural properties.
method Unified accelerated algorithm for complementary composite minimization.
result Near-optimal algorithms for various optimization problems.
This paper explores adversarial training limits and improves model robustness against norm-bounded perturbations.
problem Understanding and improving adversarial robustness of deep neural networks.
method Systematic study of adversarial training with various factors, including model size, activation functions, and unlabeled data.
result Training robust models that go beyond state-of-the-art results by combining larger models, Swish/SiLU activations, and model weight averaging.
Adam's bias shifts from full-batch to max-margin of different norms for separable data.
problem Understanding Adam's implicit bias in the incremental batch setting.
method Analyzing incremental Adam on linearly separable data, constructing datasets, and using a proxy algorithm.
result Incremental Adam can converge to different max-margin classifiers depending on the dataset and batching scheme.
New algorithm achieves optimal privacy and efficiency in non-Euclidean convex optimization.
problem Optimizing convex functions while maintaining privacy in non-Euclidean settings.
method Developed a linear-time algorithm for ℓp-setups, leveraging geometric properties. result Optimal excess risk achieved in linear time for 1<p≤2. New method encodes manifold homotopy types into algebra structures, extending previous bounds.
problem Encoding the real homotopy type of compact manifolds into algebraic structures.
method Homotopy transfer of unital DGCA structure from de Rham algebra to cohomology.
result Multiplication vanishes for all k ≥ ℓ-1 in minimal unital C∞-algebra for certain dimensions.
Paper tackles low-rank matrix recovery with column ℓ2,0-norm regularization.
problem Low-rank matrix recovery problems with column sparsity constraints.
method Developed alternating majorization-minimization (AMM) methods with extrapolation and hybrid AMM.
result Global convergence analysis and superior performance in matrix completion problems.
In this paper, we study the problem of estimating the covariance matrix under differential privacy, where the underlying covariance matrix is assumed to be sparse and of high dimensions. We propose a new method, called DP-Thresholding, to achieve a non-trivial ℓ2-norm based error bound, which is significantly bet…
Motivated by the study of Q-learning algorithms in reinforcement learning, we study a class of stochastic approximation procedures based on operators that satisfy monotonicity and quasi-contractivity conditions with respect to an underlying cone. We prove a general sandwich relation on the iterate error at each time,…
We study the robustness properties of ℓ1 norm minimization for the classical linear regression problem with a given design matrix and contamination restricted to the dependent variable. We perform a fine error analysis of the ℓ1 estimator for measurements errors consisting of outliers coupled with noise. We…
An elementary proof found for the double bubble problem in a specific norm.
problem Finding the shapes that minimize perimeter for given volumes in a specific norm.
method Direct comparison to a small family of parameterized sets for analysis.
result Existence of minimizing sets for any volume ratio parameter.
Adversarial examples are carefully perturbed in-puts for fooling machine learning models. A well-acknowledged defense method against such examples is adversarial training, where adversarial examples are injected into training data to increase robustness. In this paper, we propose a new attack to unveil an undesired pro…
This paper is a continuation of the work by the same authors on the Cartan group equipped with the sub-Finsler ℓ∞ norm. We start by giving a detailed presentation of the structure of bang-bang extremal trajectories. Then we prove upper bounds on the number of switchings on bang-bang minimizers. We prove that…
New framework for private convex optimization in arbitrary norms.
problem Private optimization of convex functions in non-Euclidean settings.
method Regularized exponential mechanism based on localization tools from convex geometry.
result First optimal privacy-utility tradeoffs for ℓp norms and Schatten-p norms. New method for factor analysis using nuclear and ℓ0 norms.
problem Finding a low-rank plus sparse decomposition from noisy covariance matrix.
method Formulated an optimization problem with nuclear norm, ℓ0 norm, and KL divergence. Used alternating minimization algorithm. result Algorithm effectively decomposes covariance matrices in synthetic and real datasets.
The paper constructs stable minimal hypersurfaces with specific singularities.
problem Creating minimal hypersurfaces with controlled singularities.
method Constructing hypersurfaces with a given singular set in a modified Euclidean space.
result Embedded minimal hypersurfaces with stable properties and specified singularities.
Neural networks have been shown to be vulnerable against minor adversarial perturbations of their inputs, especially for high dimensional data under ℓ∞ attacks. To combat this problem, techniques like adversarial training have been employed to obtain models which are robust on the training set. However, the …
Unified signSGD and gradient descent analysis for neural networks.
problem Performance of sign-based optimization methods in neural networks.
method Unified analysis of separable smoothness and ℓ∞-smoothness, isolating geometric properties affecting performance. result Sign-based methods are preferable over gradient descent under specific Hessian properties in deep networks.
Given i.i.d. observations of a random vector X∈Rp, we study the problem of estimating both its covariance matrix Σ∗, and its inverse covariance or concentration matrix {Θ∗=(Σ∗)−1.} We estimate Θ∗ by minimizing an ℓ1-penalized log-determinant Bregman divergence; in the multivariate G…
New method optimizes offline linear bandits using different confidence sets.
problem Optimizing offline learning for linear contextual bandits.
method Introduces a family of pessimistic learning rules based on ℓp confidence sets. result The π^∞ rule achieves minimax performance and strictly dominates other predictors. Generative models improve adversarial robustness by adding synthetic data.
problem Improving robustness in machine learning models trained on limited data.
method Using synthetic data generated from a large dataset to augment the original training set.
result Generative models can significantly reduce the robust-accuracy gap compared to models trained with additional real data.
New method improves subspace iteration for eigenvectors in machine learning.
problem Computing eigenvectors for large-scale problems in machine learning.
method Subspace iteration with ℓ2o∞ norm convergence analysis. result Deterministic bounds and practical stopping criterion for improved performance.
New algorithm solves ℓ0-norm constrained multilinear logistic regression for tensor data.
problem Non-convex and nonsmooth ℓ0-norm constraints in multilinear logistic regression. method APALM+ method for globally convergent optimization. result APALM+ ensures convergence to a first-order critical point. Using sparse-inducing norms to learn robust models has received increasing attention from many fields for its attractive properties. Projection-based methods have been widely applied to learning tasks constrained by such norms. As a key building block of these methods, an efficient operator for Euclidean projection ont…
One-pass algorithm finds small subset for ℓp subspace approximation with additive error.
problem Finding a small subset of data points for ℓp subspace approximation. method One-pass subset selection with additive approximation guarantee for p∈[1,∞). result First one-pass algorithm with additive error for ℓp subspace approximation.