arXiv research

Unified algorithm for minimizing composite functions with flexible design.

We consider the problem of minimizing the composition of a smooth (nonconvex) function and a smooth vector mapping, where the inner mapping is in the form of an expectation over some random variable or a finite sum. We propose a stochastic composite gradient method that employs an incremental variance-reduced estimator…

Many machine learning, statistical inference, and portfolio optimization problems require minimization of a composition of expected value functions (CEVF). Of particular interest is the finite-sum versions of such compositional optimization problems (FS-CEVF). Compositional stochastic variance reduced gradient (C-SVRG)…

Dual averaging-type methods are widely used in industrial machine learning applications due to their ability to promoting solution structure (e.g., sparsity) efficiently. In this paper, we propose a novel accelerated dual-averaging primal-dual algorithm for minimizing a composite convex function. We also derive a stoch…

Develops consistent approximations for composite optimization problems.

Model estimates foreign exchange reserve compositions of undisclosed central banks.

New DP-CD method outperforms DP-SGD in solving composite DP-ERM problems.

Paper proposes iLPA for solving DC composite optimization problems, with applications to matrix completion with outliers.

Regret minimization is a powerful tool for solving large-scale problems; it was recently used in breakthrough results for large-scale extensive-form game solving. This was achieved by composing simplex regret minimizers into an overall regret-minimization framework for extensive-form game strategy spaces. In this paper…

We present a graphical criterion for reading dependencies from the minimal directed independence map G of a graphoid p when G is a polytree and p satisfies composition and weak transitivity. We prove that the criterion is sound and complete. We argue that assuming composition and weak transitivity is not too restrictiv…

Classical stochastic gradient methods are well suited for minimizing expected-value objective functions. However, they do not apply to the minimization of a nonlinear function involving expected values or a composition of two expected-value functions, i.e., problems of the form $\min_x \mathbf{E}_v [f_v\big(\mathbf{E}_…

Proposes a new framework for learning image augmentations to improve classification performance.

Improved subgradient method tackles ill-conditioned composite optimization problems.

We consider smooth bounded surfaces with a smooth boundary and a prescribed background metric g_0. We now consider all metrics g conformal to g_0 which have a prescribed volume M. We now minimize the first eigenvalue of the Laplace operator of g over the metrics conformal to g_0 and having the prescribed volume. We sho…

We propose an algorithmic framework for convex minimization problems of a composite function with two terms: a self-concordant function and a possibly nonsmooth regularization term. Our method is a new proximal Newton algorithm that features a local quadratic convergence rate. As a specific instance of our framework, w…

A class of spiral minimal surfaces in E^3 is constructed using a symmetry reduction. The new surfaces are invariant with respect to the composition of rotation and dilatation. The solutions are obtained in closed form %through the Legendre transformation and their asymptotic behaviour is described.

Since its inception, the modus operandi of multi-task learning (MTL) has been to minimize the task-wise mean of the empirical risks. We introduce a generalized loss-compositional paradigm for MTL that includes a spectrum of formulations as a subfamily. One endpoint of this spectrum is minimax MTL: a new MTL formulation…

Minimal Kaehler submanifolds up to codimension four are studied.

We consider in this work a system of two stochastic differential equations named the perturbed compositional gradient flow. By introducing a separation of fast and slow scales of the two equations, we show that the limit of the slow motion is given by an averaged ordinary differential equation. We then demonstrate that…

This paper addresses measurement errors in high-dimensional compositional data using a log-contrast model calibration approach.

This paper studies robust estimation methods in high dimensions, comparing model-averaged and composite quantile estimators.

The paper accelerates ISTA and FISTA algorithms for composite optimization problems.

In many applications one may acquire a composition of several signals that may be corrupted by noise, and it is a challenging problem to reliably separate the components from one another without sacrificing significant details. Adding to the challenge, in a compressive sensing framework, one is given only an undersampl…

In this paper by reduction we construct a family of conformally flat Hamiltonian-minimal Lagrangian tori in $\mathbb{CP}^3$ as the image of the composition of the Hopf map $\mathcal{H}: \mathbb{S}^7\to \mathbb{CP}^3$ and a map $ψ:\mathbb{R}^3 \to \mathbb{S}^7$ with certain conditions.

We study the minimal crossing number $c(K_{1}\# K_{2})$ of composite knots $K_{1}\# K_{2}$, where $K_1$ and $K_2$ are prime, by relating it to the minimal crossing number of spatial graphs, in particular the $2n$-theta curve $θ_{K_{1},K_{2}}^n$ that results from tying $n$ of the edges of the planar embedding of the $2n…

This research proves guarantees on sequence models' generalization to longer and novel sequences.

In this paper, we extend the geometric descent method recently proposed by Bubeck, Lee and Singh to tackle nonsmooth and strongly convex composite problems. We prove that our proposed algorithm, dubbed geometric proximal gradient method (GeoPG), converges with a linear rate $(1-1/\sqrtκ)$ and thus achieves the optimal …

We consider the minimization of composite objective functions composed of the expectation of quadratic functions and an arbitrary convex function. We study the stochastic dual averaging algorithm with a constant step-size, showing that it leads to a convergence rate of O(1/n) without strong convexity assumptions. This …

New method for causal inference with complex treatment compositions.

Paper solves robust convex problems with heavy-tailed noise.

Optimal search for change point anomaly in multiple processes.

We generalize Newton-type methods for minimizing smooth functions to handle a sum of two convex functions: a smooth function and a nonsmooth function with a simple proximal mapping. We show that the resulting proximal Newton-type methods inherit the desirable convergence behavior of Newton-type methods for minimizing s…

New algorithm solves complex optimization problems without needing projections.

New sampling methods for constrained and composite distributions.

The paper describes fitting submanifolds to data using Sussmann's orbit theorem.

Robots learn actions and language through curiosity-driven self-exploration.

In this paper we develop a randomized block-coordinate descent method for minimizing the sum of a smooth and a simple nonsmooth block-separable convex function and prove that it obtains an $ε$-accurate solution with probability at least $1-ρ$ in at most $O(\tfrac{n}ε \log \tfrac{1}ρ)$ iterations, where $n$ is the numbe…

On some specified convex supporting sets of spheres, we find a generalized longitude function whose level sets are totally geodesic. Given an arbitrary (weakly) harmonic map into spheres, the composition of the generalized longitude function and harmonic map satisfies an elliptic equation of divergence type. With the a…

We consider a composite convex minimization problem associated with regularized empirical risk minimization, which often arises in machine learning. We propose two new stochastic gradient methods that are based on stochastic dual averaging method with variance reduction. Our methods generate a sparser solution than the…

New algorithm tackles complex optimization problems with inexact and stochastic methods.

This work is an analytical and numerical study of the composition of several fractals into one and of the relation between the composite dimension and the dimensions of the component fractals. In the case of composition of standard IFS with segments of equal size, the composite dimension can be expressed as a function …

We propose a variable metric framework for minimizing the sum of a self-concordant function and a possibly non-smooth convex function, endowed with an easily computable proximal operator. We theoretically establish the convergence of our framework without relying on the usual Lipschitz gradient assumption on the smooth…

Paper analyzes convergence of PAM method for low-rank factorization models.

Develops minibatch stochastic proximal gradient for large-scale learning models.

A flat virtual link is a finite collection of oriented closed curves $\mathfrak L$ on an oriented surface $M$ considered up to virtual homotopy, i.e., a composition of elementary stabilizations, destabilizations, and homotopies. Specializing to a pair of curves $(L_1,L_2)$, we show that the minimal number of intersecti…

In this paper, we present a simple analysis of {\bf fast rates} with {\it high probability} of {\bf empirical minimization} for {\it stochastic composite optimization} over a finite-dimensional bounded convex set with exponential concave loss functions and an arbitrary convex regularization. To the best of our knowledg…

Lawson-Osserman constructed three types of non-parametric minimal cones of high codimensions based on Hopf maps between spheres, which correspond to Lipschitz but non-differentiable solutions to the minimal surface equations, thereby making sharp contrast to the regularity theorem for minimal graphs of codimension 1. I…

This paper studies the question of whether minimal genus Heegaard splittings of exterior spaces of knots which are connected sums are weakly reducible or not. Furthermore it is shown that the Heegaard splittings of the knots used by Morimoto to show that tunnel number can be sub-additive are all strongly irreducible. T…