arXiv research

Study curvature and torsion in Gaussian distribution's dual coordinate system.

Stochastic dual coordinate ascent (SDCA) is an effective technique for solving regularized loss minimization problems in machine learning. This paper considers an extension of SDCA under the mini-batch setting that is often used in practice. Our main contribution is to introduce an accelerated mini-batch version of SDC…

Construct dual F-manifolds for regular F-manifolds.

We explain that spectral networks are a unifying framework that incorporates both shear (Fock-Goncharov) and length-twist (Fenchel-Nielsen) coordinate systems on moduli spaces of flat SL(2,C) connections, in the following sense. Given a spectral network W on a punctured Riemann surface C, we explain the process of "abe…

The equations governing anti-self-dual and Einstein-Weyl conformal geometries can be regarded as `master dispersionless systems' in four and three dimensions respectively. Their integrability by twistor methods has been established by Penrose and Hitchin. In this note we present, in specially adapted coordinate systems…

Analytic plane curves determine unique conformal coordinates.

PURE-CD algorithm proves complexity bounds for convex-concave problems.

Stochastic Gradient Descent (SGD) has become popular for solving large scale supervised machine learning optimization problems such as SVM, due to their strong theoretical guarantees. While the closely related Dual Coordinate Ascent (DCA) method has been implemented in various software packages, it has so far lacked go…

We propose a doubly stochastic primal-dual coordinate optimization algorithm for empirical risk minimization, which can be formulated as a bilinear saddle-point problem. In each iteration, our method randomly samples a block of coordinates of the primal and dual solutions to update. The linear convergence of our method…

Study of meromorphic connections and their spectral duals in $\mathfrak{gl}_3(\mathbb{C})$.

We introduce a proximal version of dual coordinate ascent method. We demonstrate how the derived algorithmic framework can be used for numerous regularized loss minimization problems, including $\ell_1$ regularization and structured output SVM. The convergence rates we obtain match, and sometimes improve, state-of-the-…

Machine learning with big data often involves large optimization models. For distributed optimization over a cluster of machines, frequent communication and synchronization of all model parameters (optimization variables) can be very costly. A promising solution is to use parameter servers to store different subsets of…

Accelerated coordinate descent is widely used in optimization due to its cheap per-iteration cost and scalability to large-scale problems. Up to a primal-dual transformation, it is also the same as accelerated stochastic gradient descent that is one of the central methods used in machine learning. In this paper, we imp…

The closed string model in the background gravity field is considered as a bi-Hamiltonian system in assumption that string model is the integrable model for particular kind of the background fields. The dual nonlocal Poisson brackets(PB), depending of the background fields and of their derivatives, are obtained. The in…

We propose a new randomized coordinate descent method for a convex optimization template with broad applications. Our analysis relies on a novel combination of four ideas applied to the primal-dual gap function: smoothing, acceleration, homotopy, and coordinate descent with non-uniform sampling. As a result, our method…

Capacity control, the bias/variance dilemma, and learning unknown functions from data, are all concerned with identifying effective and consistent fits of unknown geometric loci to random data points. A geometric locus is a curve or surface formed by points, all of which possess some uniform property. A geometric locus…

Random extrapolation speeds up coordinate descent for sparse and dense data.

We study primal-dual type stochastic optimization algorithms with non-uniform sampling. Our main theoretical contribution in this paper is to present a convergence analysis of Stochastic Primal Dual Coordinate (SPDC) Method with arbitrary sampling. Based on this theoretical framework, we propose Optimality Violation-ba…

The stochastic dual coordinate-ascent (S-DCA) technique is a useful alternative to the traditional stochastic gradient-descent algorithm for solving large-scale optimization problems due to its scalability to large data sets and strong theoretical guarantees. However, the available S-DCA formulation is limited to finit…

This paper introduces AdaSDCA: an adaptive variant of stochastic dual coordinate ascent (SDCA) for solving the regularized empirical risk minimization problems. Our modification consists in allowing the method adaptively change the probability distribution over the dual variables throughout the iterative process. AdaSD…

We introduce a proximal version of the stochastic dual coordinate ascent method and show how to accelerate the method using an inner-outer iteration procedure. We analyze the runtime of the framework and obtain rates that improve state-of-the-art results for various key machine learning optimization problems including …

New geometric structures defined on SPD matrices for better understanding.

Starting from a bundle E over R, the dual of the first jet bundle, which is a co-dimension 1 sub-bundle of the cotangent bundle of E, is the appropriate manifold for the geometric description of time-dependent Hamiltonian systems. Based on previous work, we recall properties of the complete lifts of a type (1,1) tensor…

In this paper we study G-surfaces, a rather unknown surface class originally defined by Calapso, and show that the coordinate surfaces of a Guichard net are G-surfaces. Based on this observation, we present distinguished Combescure transformations that provide a duality for Guichard nets. Another class of special Combe…

The paper explores moduli space of heterotic system using two deformation paths.

We discuss bases of the space of holomorphic quadratic differentials that are dual to the differentials of Fenchel-Nielsen coordinates and hence appear naturally when considering functions on the set of hyperbolic metrics which are invariant under pull-back by diffeomorphisms, such as eigenvalues of the Laplacian. The …

In modern large-scale machine learning applications, the training data are often partitioned and stored on multiple machines. It is customary to employ the "data parallelism" approach, where the aggregated training loss is minimized without moving data across machines. In this paper, we introduce a novel distributed du…

New findings on Kähler manifolds restrict orthogonal coordinates existence.

We consider a generic convex optimization problem associated with regularized empirical risk minimization of linear predictors. The problem structure allows us to reformulate it as a convex-concave saddle point problem. We propose a stochastic primal-dual coordinate (SPDC) method, which alternates between maximizing ov…

We consider convex-concave saddle point problems with a separable structure and non-strongly convex functions. We propose an efficient stochastic block coordinate descent method using adaptive primal-dual updates, which enables flexible parallel optimization for large-scale problems. Our method shares the efficiency an…

This work tackles resource allocation in asynchronous and stochastic systems.

While convergence of the Alternating Direction Method of Multipliers (ADMM) on convex problems is well studied, convergence on nonconvex problems is only partially understood. In this paper, we consider the Gaussian phase retrieval problem, formulated as a linear constrained optimization problem with a biconvex objecti…

Marginal MAP inference involves making MAP predictions in systems defined with latent variables or missing information. It is significantly more difficult than pure marginalization and MAP tasks, for which a large class of efficient and convergent variational algorithms, such as dual decomposition, exist. In this work,…

In this paper, we consider the problem of recovering a sparse signal based on penalized least squares formulations. We develop a novel algorithm of primal-dual active set type for a class of nonconvex sparsity-promoting penalties, including $\ell^0$, bridge, smoothly clipped absolute deviation, capped $\ell^1$ and mini…

We consider a generic convex-concave saddle point problem with separable structure, a form that covers a wide-ranged machine learning applications. Under this problem structure, we follow the framework of primal-dual updates for saddle point problems, and incorporate stochastic block coordinate descent with adaptive st…

The Dynnikov coordinate system puts global coordinates on the boundary of Teichmüller space of an $n$--punctured disk. We survey the Dynnikov coordinate system, and investigate how we use this coordinate system to study pseudo--Anosov braids making use of results from Thurston's theory on surface homeomorphisms.

New solver MPLP++ outperforms existing solvers for dense graph models.

Unified algorithm solves convex optimization problems with optimal rates.

We provide theoretical complexity analysis for new algorithms to compute the optimal transport (OT) distance between two discrete probability distributions, and demonstrate their favorable practical performance over state-of-art primal-dual algorithms and their capability in solving other problems in large-scale, such …

New DCD and BDCD methods for K-SVM and K-RR reduce communication costs.

New taxonomy and improved solvers for discrete energy minimization.

Coordinate descent methods employ random partial updates of decision variables in order to solve huge-scale convex optimization problems. In this work, we introduce new adaptive rules for the random selection of their updates. By adaptive, we mean that our selection rules are based on the dual residual or the primal-du…

We propose a new stochastic dual coordinate ascent technique that can be applied to a wide range of regularized learning problems. Our method is based on Alternating Direction Multiplier Method (ADMM) to deal with complex regularization functions such as structured regularizations. Although the original ADMM is a batch…

Method constructs orthogonal curvilinear coordinates in constant curvature spaces.

We study the limiting case of the Krichever construction of orthogonal curvilinear coordinate systems when the spectral curve becomes singular. We show that the case when the curve is reducible and all its irreducible components are rational curves the construction procedure reduces to solving systems of linear equatio…

In this paper we generalize the framework of the feasible descent method (FDM) to a randomized (R-FDM) and a coordinate-wise random feasible descent method (RC-FDM) framework. We show that the famous SDCA algorithm for optimizing the SVM dual problem, or the stochastic coordinate descent method for the LASSO problem, f…

Paper finds local normal forms for wavefronts in flat coordinates.

In integrable hydrodynamic systems, coordinates exist where generators and symmetries are simple.