arXiv research

Entropic regularization is quickly emerging as a new standard in optimal transport (OT). It enables to cast the OT computation as a differentiable and unconstrained convex optimization problem, which can be efficiently solved using the Sinkhorn algorithm. However, entropy keeps the transportation plan strictly positive…

Data-driven optimization improves mean-variance portfolios by penalizing norms.

Differential privacy is concerned about the prediction quality while measuring the privacy impact on individuals whose information is contained in the data. We consider differentially private risk minimization problems with regularizers that induce structured sparsity. These regularizers are known to be convex but they…

We theoretically and experimentally investigate tensor-based regression and classification. Our focus is regularization with various tensor norms, including the overlapped trace norm, the latent trace norm, and the scaled latent trace norm. We first give dual optimization methods using the alternating direction method …

Article provides polytopes as dual unit balls of Thurston norms on 3-manifolds.

One of the popular approaches for low-rank tensor completion is to use the latent trace norm regularization. However, most existing works in this direction learn a sparse combination of tensors. In this work, we fill this gap by proposing a variant of the latent trace norm that helps in learning a non-sparse combinatio…

New theorem for generalized group sparsity improves consistency and convergence rates.

There is growing body of learning problems for which it is natural to organize the parameters into matrix, so as to appropriately regularize the parameters under some matrix norm (in order to impose some more sophisticated prior knowledge). This work describes and analyzes a systematic method for constructing such matr…

The study solves the isoperimetric problem for Heisenberg group norms.

In this paper, we discuss how a suitable family of tensor kernels can be used to efficiently solve nonparametric extensions of $\ell^p$ regularized learning methods. Our main contribution is proposing a fast dual algorithm, and showing that it allows to solve the problem efficiently. Our results contrast recent finding…

Paper tackles image reconstruction from limited data using polyhedral norms and convex regularizers.

Study shows horofunction compactification's topology matches dual norm's unit ball.

Sparse learning has recently received increasing attention in many areas including machine learning, statistics, and applied mathematics. The mixed-norm regularization based on the l1q norm with q>1 is attractive in many applications of regression and classification in that it facilitates group sparsity in the model. T…

Study on Santaló point for convex bodies in normed spaces.

The Euler class conjecture links geometric structures to integral points on the Thurston norm ball.

We introduce Primal-Dual Wasserstein GAN, a new learning algorithm for building latent variable models of the data distribution based on the primal and the dual formulations of the optimal transport (OT) problem. We utilize the primal formulation to learn a flexible inference mechanism and to create an optimal approxim…

We study incompressible surfaces constructed by Culler-Shalen theory in the context of twisted Alexander polynomials. For a $1$st cohomology class of a $3$-manifold the coefficients of twisted Alexander polynomials induce regular functions on the $SL_2(\mathbb{C})$-character variety. We prove that if an ideal point giv…

This paper proposes a mechanism to produce equivalent Lipschitz surrogates for zero-norm and rank optimization problems by means of the global exact penalty for their equivalent mathematical programs with an equilibrium constraint (MPECs). Specifically, we reformulate these combinatorial problems as equivalent MPECs by…

Convex optimization method infers latent structure in random dot product graphs.

Estimates the dual Thurston norm for foliations on negative curvature 3-manifolds.

Canonical correlation analysis (CCA) is a multivariate statistical technique for finding the linear relationship between two sets of variables. The kernel generalization of CCA named kernel CCA has been proposed to find nonlinear relations between datasets. Despite their wide usage, they have one common limitation that…

This paper deals with supervised classification and feature selection in high dimensional space. A classical approach is to project data on a low dimensional space and classify by minimizing an appropriate quadratic cost. A strict control on sparsity is moreover obtained by adding an $\ell_1$ constraint, here on the ma…

A new kernel for probability measures based on optimal transport.

Proposes an efficient method for sparse index tracking with $\ell_0$-norm constraints.

We relate the Gromov norm on homology classes to the harmonic norm on the dual cohomology and obtain double sided bounds in terms of the volume and other geometric quantities of the underlying manifold. Along the way, we provide comparisons to other related norms and quantities as well.

Intersection norms are integer norms on the first homology group of a surface. In this article, we prove that there are some polytopes which are not dual unit balls of such norms. By the way, we investigate the set of collections of curves on $Σ$2 whose complement is a disk.

Study one-shot strategic classification under unknown costs, improving worst-case accuracy.

We present a general regularization-based framework for Multi-task learning (MTL), in which the similarity between tasks can be learned or refined using $\ell_p$-norm Multiple Kernel learning (MKL). Based on this very general formulation (including a general loss function), we derive the corresponding dual formulation …

Sparse coding consists in representing signals as sparse linear combinations of atoms selected from a dictionary. We consider an extension of this framework where the atoms are further assumed to be embedded in a tree. This is achieved using a recently introduced tree-structured sparse regularization norm, which has pr…

Construct dual F-manifolds for regular F-manifolds.

Efficiently regularizes deep learning models using Jacobian nuclear norm.

Dual-sPLS improves feature selection and prediction in high-dimensional data.

New proof shows norms can't explain deep learning's implicit regularization.

Exact spectral norm regularization improves neural network generalization.

In this note we introduce a natural Finsler structure on convex surfaces, referred to as the projective Finsler structure, which is dual in a sense to the obvious inclusion of a convex surface in a normed space. It has an associated projective girth, which is similar to the notion of girth defined by Schäffer. We prove…

Study shows how networks converge to minimum norm solutions with regularization.

We study a regularizer which is defined as a parameterized infimum of quadratics, and which we call the box-norm. We show that the k-support norm, a regularizer proposed by [Argyriou et al, 2012] for sparse vector prediction problems, belongs to this family, and the box-norm can be generated as a perturbation of the fo…

We use distributionally-robust optimization for machine learning to mitigate the effect of data poisoning attacks. We provide performance guarantees for the trained model on the original data (not including the poison records) by training the model for the worst-case distribution on a neighbourhood around the empirical…

Dual optimization connects ERM-fDR to normalization function.

In this paper, we study randomized reduction methods, which reduce high-dimensional features into low-dimensional space by randomized methods (e.g., random projection, random hashing), for large-scale high-dimensional classification. Previous theoretical results on randomized reduction methods hinge on strong assumptio…

Dual regularized graph Laplacian improves spectral clustering for community detection.

For every finite collection of curves on a surface, we define an associated (semi-)norm on the first homology group of the surface. The unit ball of the dual norm is the convex hull of its integer points. We give an interpretation of these points in terms of certain coorientations of the original collection of curves. …

Revisits shallow neural networks using Lipschitz norms and measures.

In this paper, we introduce a powerful technique based on Leave-one-out analysis to the study of low-rank matrix completion problems. Using this technique, we develop a general approach for obtaining fine-grained, entrywise bounds for iterative stochastic procedures in the presence of probabilistic dependency. We demon…

New method accelerates convergence for entropy-regularized reinforcement learning problems.

New method calculates Thurston norm for 3-manifolds with toroidal boundaries.

We investigate the generalizability of deep learning based on the sensitivity to input perturbation. We hypothesize that the high sensitivity to the perturbation of data degrades the performance on it. To reduce the sensitivity to perturbation, we propose a simple and effective regularization method, referred to as spe…

The $k$-support norm is a regularizer which has been successfully applied to sparse vector prediction problems. We show that it belongs to a general class of norms which can be formulated as a parameterized infimum over quadratics. We further extend the $k$-support norm to matrices, and we observe that it is a special …