arXiv research

New duality concept for vector spaces and groupoids.

The aim of this paper is to obtain on the dual 1-jet space J^{1*}(R;M) the main geometrical objects used in the dual jet geometry of time-dependent Hamiltonians. We talk about distinguished (d-) tensors, time-dependent semisprays, nonlinear connections and their mathematical connections.

Dual-objective GANs reduce training instabilities with tunable α-loss parameters.

A new method for distributed optimization reduces communication rounds without minibatches.

Compact theorem for $SO(3)$ anti-self-dual equations on cylindrical manifolds.

Construct dual F-manifolds for regular F-manifolds.

FeDualEx tackles saddle point optimization in federated learning with composite objectives.

The minimization of convex objectives coming from linear supervised learning problems, such as penalized generalized linear models, can be formulated as finite sums of convex functions. For such problems, a large set of stochastic first-order solvers based on the idea of variance reduction are available and combine bot…

DFM simplifies CNF training without interpolants.

We study natural lifting operations from a bundle E over R to the dual bundle of its first-jet bundle. The main purpose is to define a complete lift of a type (1,1) tensor field on E and to understand all features of its construction. Various other lifting operations of tensorial objects on E are needed for that purpos…

A large number of objectives have been proposed to train latent variable generative models. We show that many of them are Lagrangian dual functions of the same primal optimization problem. The primal problem optimizes the mutual information between latent and visible variables, subject to the constraints of accurately …

Unified algorithm solves convex optimization problems with optimal rates.

We relate the minimax game of generative adversarial networks (GANs) to finding the saddle points of the Lagrangian function for a convex optimization problem, where the discriminator outputs and the distribution of generator outputs play the roles of primal variables and dual variables, respectively. This formulation …

Random extrapolation speeds up coordinate descent for sparse and dense data.

Multi-objective optimization (MOO) is a well-studied problem for several important recommendation problems. While multiple approaches have been proposed, in this work, we focus on using constrained optimization formulations (e.g., quadratic and linear programs) to formulate and solve MOO problems. This approach can be …

Proposes an online method for solving non-convex DRO with KL regularization.

This paper analyzes privacy-preserving methods for sparse model optimization.

We develop a general duality between neural networks and compositional kernels, striving towards a better understanding of deep learning. We show that initial representations generated by common random initializations are sufficiently rich to express all functions in the dual kernel space. Hence, though the training ob…

We present a primal-dual algorithmic framework to obtain approximate solutions to a prototypical constrained convex optimization problem, and rigorously characterize how common structural assumptions affect the numerical efficiency. Our main analysis technique provides a fresh perspective on Nesterov's excessive gap te…

Introduces cofrontal mappings and studies their properties.

In this paper we propose a randomized primal-dual proximal block coordinate updating framework for a general multi-block convex optimization model with coupled objective function and linear constraints. Assuming mere convexity, we establish its $O(1/t)$ convergence rate in terms of the objective value and feasibility m…

Dual-sPLS improves feature selection and prediction in high-dimensional data.

We present a theory and applications of discrete exterior calculus on simplicial complexes of arbitrary finite dimension. This can be thought of as calculus on a discrete space. Our theory includes not only discrete differential forms but also discrete vector fields and the operators acting on these objects. This allow…

We propose a doubly stochastic primal-dual coordinate optimization algorithm for empirical risk minimization, which can be formulated as a bilinear saddle-point problem. In each iteration, our method randomly samples a block of coordinates of the primal and dual solutions to update. The linear convergence of our method…

Scaff-PD improves fairness and robustness in federated learning with reduced communication.

Iterative Hard Thresholding (IHT) is a class of projected gradient descent methods for optimizing sparsity-constrained minimization models, with the best known efficiency and scalability in practice. As far as we know, the existing IHT-style methods are designed for sparse minimization in primal form. It remains open t…

We define a symmetric monoidal (4,3)-category with duals whose objects are certain enriched multi-fusion categories. For every modular tensor category $\mathcal{C}$, there is a self enriched multi-fusion category $\mathfrak{C}$ giving rise to an object of this symmetric monoidal (4,3)-category. We conjecture that the e…

Dual-based algorithms optimize distributed convex problems over networks.

The paper explores conditions for manifolds to have specific geometric structures.

In this paper we study some geometrical objects (d-tensors, multi-time semisprays of polymomenta and nonlinear connections) on the dual 1-jet vector bundle $J^{1*}(\cal{T}, M)\to \cal{T}\times M$. Some geometrical formulas, which connect the last two geometrical objects, are also derived. Finally, a canonical nonlinear…

Generalized dual discriminator GANs improve upon traditional GANs by using two discriminators and a flexible loss function.

We study a stochastic and distributed algorithm for nonconvex problems whose objective consists of a sum of $N$ nonconvex $L_i/N$-smooth functions, plus a nonsmooth regularizer. The proposed NonconvEx primal-dual SpliTTing (NESTT) algorithm splits the problem into $N$ subproblems, and utilizes an augmented Lagrangian b…

New algorithm reduces regret in CMDPs without cancellation of errors.

This work reveals a primal-dual relationship between GANs and Autoencoders, improving their theoretical understanding.

Study on lightlike geometry in indefinite Sasakian statistical manifolds.

New algorithms optimize without knowing problem parameters.

New method trains normalizing flows using entropy-regularized transport.

Block Coordinate Update (BCU) methods enjoy low per-update computational complexity because every time only one or a few block variables would need to be updated among possibly a large number of blocks. They are also easily parallelized and thus have been particularly popular for solving problems involving large-scale …

Evolutionary forms, as well as exterior forms, are skew-symmetric differential forms. But in contrast to the exterior forms, the basis of evolutionary forms is deforming manifolds (with unclosed metric forms). Such forms possess a peculiarity, namely, the closed inexact exterior forms are obtained from that. The closur…

We establish the existence and characterization of a primal and a dual facelift - discontinuity of the value function at the terminal time - for utility-maximization in incomplete semimartingale-driven financial markets. Unlike in the lower- and upper-hedging problems, and somewhat unexpectedly, a facelift turns out to…

Affine hamiltonians are defined in the paper and their study is based especially on the fact that in the hyperregular case they are dual objects of lagrangians defined on affine bundles, by mean of natural Legendre maps. The variational problems for affine hamiltonians and lagrangians of order $k\geq 2$ are studied, re…

Study allocates resources to strategic agents while balancing cost and incentives.

Study on volumes of quasifuchsian manifolds, focusing on similarities and proximities.

Gradient descent reveals the exact implicit bias via dual optimization for linearly separable data.

We consider the diffeological pseudo-bundles of exterior algebras, and the Clifford action of the corresponding Clifford algebras, associated to a given finite-dimensional and locally trivial diffeological vector pseudo-bundle, as well as the behavior of the former three constructions (exterior algebra, Clifford action…

Algorithm optimizes constrained reinforcement learning with dual variables.

We consider a wide range of regularized stochastic minimization problems with two regularization terms, one of which is composed with a linear function. This optimization model abstracts a number of important applications in artificial intelligence and machine learning, such as fused Lasso, fused logistic regression, a…

We classify all fusion categories for a given set of fusion rules with three simple object types. If a conjecture of Ostrik is true, our classification completes the classification of fusion categories with three simple object types. To facilitate the discussion we describe a convenient, concrete and useful variation o…