arXiv research

Paper establishes lower bounds for non-stationary kernelized bandits.

Unified analysis of MPLE for Ising models with bounded operator norm or infinity norm.

We consider the problem of minimizing the sum of submodular set functions assuming minimization oracles of each summand function. Most existing approaches reformulate the problem as the convex minimization of the sum of the corresponding Lovász extensions and the squared Euclidean norm, leading to algorithms requiring …

New method improves tensor completion by selectively preserving important elements.

A new algorithm solves sparse optimization problems on measures efficiently.

We prove that for a so-called sticky process $S$ there exists an equivalent probability $Q$ and a $Q$-martingale $\tilde{S}$ that is arbitrarily close to $S$ in $L^p(Q)$ norm. For continuous $S$, $\tilde{S}$ can be chosen arbitrarily close to $S$ in supremum norm. In the case where $S$ is a local martingale we may choo…

We augment adversarial training (AT) with worst case adversarial training (WCAT) which improves adversarial robustness by 11% over the current state-of-the-art result in the $\ell_2$ norm on CIFAR-10. We obtain verifiable average case and worst case robustness guarantees, based on the expected and maximum values of the…

Study variations of metrics on Riemannian submersions to preserve fiber geometry.

This work shows how penalising bias terms in norm regularisation leads to sparse solutions.

Characterizing the phase transitions of convex optimizations in recovering structured signals or data is of central importance in compressed sensing, machine learning and statistics. The phase transitions of many convex optimization signal recovery methods such as $\ell_1$ minimization and nuclear norm minimization are…

Data clustering is a fundamental problem with a wide range of applications. Standard methods, eg the $k$-means method, usually require solving a non-convex optimization problem. Recently, total variation based convex relaxation to the $k$-means model has emerged as an attractive alternative for data clustering. However…

Deep neural networks perform well on real world data but are prone to adversarial perturbations: small changes in the input easily lead to misclassification. In this work, we propose an attack methodology not only for cases where the perturbations are measured by $\ell_p$ norms, but in fact any adversarial dissimilarit…

Introduces HTV to measure function complexity in learning schemes.

We investigate the statistical complexity of estimating the parameters of a discrete-state Markov chain kernel from a single long sequence of state observations. In the finite case, we characterize (modulo logarithmic factors) the minimax sample complexity of estimation with respect to the operator infinity norm, while…

Paper addresses Byzantine attacks in decentralized optimization over networks.

Revisits shallow neural networks using Lipschitz norms and measures.

Proposes a new model for image restoration combining deep learning and total variation.

Motivated by some applications in signal processing and machine learning, we consider two convex optimization problems where, given a cone $K$, a norm $\|\cdot\|$ and a smooth convex function $f$, we want either 1) to minimize the norm over the intersection of the cone and a level set of $f$, or 2) to minimize over the…

The paper uses Banach spaces to analyze neural networks.

Currents represent generalized surfaces studied in geometric measure theory. They range from relatively tame integral currents representing oriented compact manifolds with boundary and integer multiplicities, to arbitrary elements of the dual space of differential forms. The flat norm provides a natural distance in the…

Paper analyzes mistake and generalization of MNIC classifiers.

We propose a systematic construction of native Banach spaces for general spline-admissible operators ${\rm L}$. In short, the native space for ${\rm L}$ and the (dual) norm $\|\cdot\|_{\mathcal{X}'}$ is the largest space of functions $f: \mathbb{R}^d \to \mathbb{R}$ such that $\|{\rm L} f\|_{\mathcal{X}'}<\infty$, subj…

In this work we study input gradient regularization of deep neural networks, and demonstrate that such regularization leads to generalization proofs and improved adversarial robustness. The proof of generalization does not overcome the curse of dimensionality, but it is independent of the number of layers in the networ…

Study on free-boundary CMC hypersurfaces in upper hemisphere, proving Morse index and eigenvalue bounds.

Ideas from the image processing literature have recently motivated a new set of clustering algorithms that rely on the concept of total variation. While these algorithms perform well for bi-partitioning tasks, their recursive extensions yield unimpressive results for multiclass clustering tasks. This paper presents a g…

The paper studies curves in Riemannian manifolds using total variation flow.

Optimal pre-processing reduces disparate impact by minimizing total variation distance.

We prove that some Riemannian manifolds with boundary under an explicit integral pinching are spherical space forms. Precisely, we show that 3-dimensional Riemannian manifolds with totally geodesic boundary, positive scalar curvature and an explicit integral pinching between the $L^2$-norm of their scalar curvature and…

We show a very simple and general total second variation formula for Perelman's $\mathcal{W}$-functional at arbitrary points in the space of Riemannian metrics. Moreover we perform a study of the properties of the variations of Kähler structures. We deduce a quite simple and general total second variation formula for P…

This paper accelerates TV regularization algorithms by unrolling proximal gradient descent.

We consider the problem of estimating the parameters of a $d$-dimensional rectified Gaussian distribution from i.i.d. samples. A rectified Gaussian distribution is defined by passing a standard Gaussian distribution through a one-layer ReLU neural network. We give a simple algorithm to estimate the parameters (i.e., th…

We derive variational formulas for the total Q-prime curvature under the deformation of strictly pseudoconvex domains in a complex manifold. We also show that the total Q-prime curvature agrees with the renormalized volume of such domains with respect to the complete Einstein-Kähler metric. In the appendix, by Rod Gove…

We consider the problem of estimating a function defined over $n$ locations on a $d$-dimensional grid (having all side lengths equal to $n^{1/d}$). When the function is constrained to have discrete total variation bounded by $C_n$, we derive the minimax optimal (squared) $\ell_2$ estimation error rate, parametrized by …

We study the theoretical properties of image denoising via total variation penalized least-squares. We define the total vatiation in terms of the two-dimensional total discrete derivative of the image and show that it gives rise to denoised images that are piecewise constant on rectangular sets. We prove that, if the t…

Paper tackles Byzantine attacks in distributed learning with a new ADMM method.

The total variation distance is a core statistical distance between probability measures that satisfies the metric axioms, with value always falling in $[0,1]$. This distance plays a fundamental role in machine learning and signal processing: It is a member of the broader class of $f$-divergences, and it is related to …

We consider a class of sparsity-inducing regularization terms based on submodular functions. While previous work has focused on non-decreasing functions, we explore symmetric submodular functions and their \lova extensions. We show that the Lovasz extension may be seen as the convex envelope of a function that depends …

Through the direct study of the analysis estimator we derive oracle inequalities with fast and slow rates by adapting the arguments involving projections by Dalalyan, Hebiri and Lederer (2017). We then extend the theory to the square root analysis estimator. Finally, we focus on (square root) total variation regularize…

The real homology of a compact, n-dimensional Riemannian manifold M is naturally endowed with the stable norm. The stable norm of a homology class is the minimal Riemannian volume of its representatives. If M is orientable the stable norm on H_{n-1}(M,R) is a homogenized version of the Riemannian (n-1)-volume. We study…

SaR-SVM-STV improves hyperspectral image classification with shape-adaptive reconstruction and denoising.

Study variational properties of curves in half-plane with area constraints.

The $\ell$-1 norm based optimization is widely used in signal processing, especially in recent compressed sensing theory. This paper studies the solution path of the $\ell$-1 norm penalized least-square problem, whose constrained form is known as Least Absolute Shrinkage and Selection Operator (LASSO). A solution path …

Study variations of Riemannian submersions to maintain geodesic fibers and positive curvatures.

In 1986, W. Thurston introduced a (possibly degenerate) norm on the first cohomology group of a 3-manifold. Inspired by this definition, Turaev introduced in 2002 a analogous norm on the first cohomology group of a finite 2-complex. We show that if N is the exterior of a link in a rational homology sphere, then the Thu…

We examine the total mixed scalar curvature of a fixed distribution as a functional of a pseudo-Riemannian metric. We develop variational formulas for quantities of extrinsic geometry of the distribution to find the critical points of this action. Together with the arbitrary variations of the metric, we consider also v…

Estimates parameters of interconnected linear systems using total variation penalization.

We study \emph{TV regularization}, a widely used technique for eliciting structured sparsity. In particular, we propose efficient algorithms for computing prox-operators for $\ell_p$-norm TV. The most important among these is $\ell_1$-norm TV, for whose prox-operator we present a new geometric analysis which unveils a …

Sharp inequality between TV and Hellinger distances for Gaussian mixtures.