arXiv research

Sequential coordinate ascent is more robust in high-dimensional linear regression.

We study transformations of coordinates on a Lorentzian Einstein manifold with a parallel distribution of null lines and show that the general Walker coordinates can be simplified. In these coordinates, the full Lorentzian Einstein equation is reduced to equations on a family of Einstein Riemannian metrics.

SySCD improves SCD scalability and speeds up training.

We design a randomised parallel version of Adaboost based on previous studies on parallel coordinate descent. The algorithm uses the fact that the logarithm of the exponential loss is a function with coordinate-wise Lipschitz continuous gradient, in order to define the step lengths. We provide the proof of convergence …

We present a generic framework for parallel coordinate descent (CD) algorithms that includes, as special cases, the original sequential algorithms Cyclic CD and Stochastic CD, as well as the recent parallel Shotgun algorithm. We introduce two novel parallel algorithms that are also special cases---Thread-Greedy CD and …

Study finds holonomy algebras for Lorentzian Weyl spin manifolds with specific spinors.

Study parallel waves in spacetimes, focusing on causality and open questions.

We propose and analyze a new parallel coordinate descent method---`NSync---in which at each iteration a random subset of coordinates is updated, in parallel, allowing for the subsets to be chosen non-uniformly. We derive convergence rates under a strong convexity assumption, and comment on how to assign probabilities t…

We propose a new stochastic coordinate descent method for minimizing the sum of convex functions each of which depends on a small number of coordinates only. Our method (APPROX) is simultaneously Accelerated, Parallel and PROXimal; this is the first time such a method is proposed. In the special case when the number of…

Many powerful machine learning models are based on the composition of multiple processing layers, such as deep nets, which gives rise to nonconvex objective functions. A general, recent approach to optimise such "nested" functions is the method of auxiliary coordinates (MAC). MAC introduces an auxiliary coordinate for …

This paper focuses on coordinate update methods, which are useful for solving problems involving large or high-dimensional datasets. They decompose a problem into simple subproblems, where each updates one, or a small block of, variables while fixing others. These methods can deal with linear and nonlinear mappings, sm…

In this paper we consider planar sections and visual contours of co-dimension one affine immersions. The main theorem says that the third order Taylor expansion of the difference between the visual contour and planar section functions is exactly the cubic form. We also consider parameterizations on two dimensional affi…

We consider convex-concave saddle point problems with a separable structure and non-strongly convex functions. We propose an efficient stochastic block coordinate descent method using adaptive primal-dual updates, which enables flexible parallel optimization for large-scale problems. Our method shares the efficiency an…

This monograph presents a class of algorithms called coordinate descent algorithms for mathematicians, statisticians, and engineers outside the field of optimization. This particular class of algorithms has recently gained popularity due to their effectiveness in solving large-scale optimization problems in machine lea…

Geometrically revisits Dupin cyclidic systems using evolving circles and cyclides.

This study develops methods to coordinate travel routes to reduce congestion.

Periods of parallel exterior forms define natural coordinates on the deformation space of complete affine structures on the two-torus. These coordinates define a differentiable structure on this deformation space, under which it is diffeomorphic to $R^2$. The action of the mapping class group of $T^2$ is equivalent in …

New solver MPLP++ outperforms existing solvers for dense graph models.

Large-scale L1-regularized loss minimization problems arise in high-dimensional applications such as compressed sensing and high-dimensional supervised learning, including classification and regression problems. High-performance algorithms and implementations are critical to efficiently solving these problems. Building…

Recent several years have witnessed the surge of asynchronous (async-) parallel computing methods due to the extremely big data involved in many modern applications and also the advancement of multi-core machines and computer clusters. In optimization, most works about async-parallel methods are on unconstrained proble…

Coordinate ascent variational inference is an important algorithm for inference in probabilistic models, but it is slow because it updates only a single variable at a time. Block coordinate methods perform inference faster by updating blocks of variables in parallel. However, the speed and stability of these algorithms…

In this paper we develop and analyze Hydra: HYbriD cooRdinAte descent method for solving loss minimization problems with big data. We initially partition the coordinates (features) and assign each partition to a different node of a cluster. At every iteration, each node picks a random subset of the coordinates from tho…

We provide a coordinate-free version of the local classification, due to A. G. Walker [Quart. J. Math. Oxford (2) 1, 69 (1950)], of null parallel distributions on pseudo-Riemannian manifolds. The underlying manifold is realized, locally, as the total space of a fibre bundle, each fibre of which is an affine principal b…

Study of bound states in quantum layers with confining potentials.

Stochastic dual coordinate ascent (SDCA) is an effective technique for solving regularized loss minimization problems in machine learning. This paper considers an extension of SDCA under the mini-batch setting that is often used in practice. Our main contribution is to introduce an accelerated mini-batch version of SDC…

We consider learning problems over training sets in which both, the number of training examples and the dimension of the feature vectors, are large. To solve these problems we propose the random parallel stochastic algorithm (RAPSA). We call the algorithm random parallel because it utilizes multiple parallel processors…

Develops DP-SCD for stochastic coordinate descent, making it differentially private.

We develop parallel and distributed Frank-Wolfe algorithms; the former on shared memory machines with mini-batching, and the latter in a delayed update framework. Whenever possible, we perform computations asynchronously, which helps attain speedups on multicore machines as well as in distributed environments. Moreover…

This paper provides a block coordinate descent algorithm to solve unconstrained optimization problems. In our algorithm, computation of function values or gradients is not required. Instead, pairwise comparison of function values is used. Our algorithm consists of two steps; one is the direction estimate step and the o…

Randomized block-diagonal preconditioning improves parallel learning convergence.

In this work we show that randomized (block) coordinate descent methods can be accelerated by parallelization when applied to the problem of minimizing the sum of a partially separable smooth convex function and a simple separable convex function. The theoretical speedup, as compared to the serial method, and referring…

Finding a fixed point to a nonexpansive operator, i.e., $x^*=Tx^*$, abstracts many problems in numerical linear algebra, optimization, and other areas of scientific computing. To solve fixed-point problems, we propose ARock, an algorithmic framework in which multiple agents (machines, processors, or cores) update $x$ i…

In modern large-scale machine learning applications, the training data are often partitioned and stored on multiple machines. It is customary to employ the "data parallelism" approach, where the aggregated training loss is minimized without moving data across machines. In this paper, we introduce a novel distributed du…

Geodesic coordinates derived for a specific metric in surface group representations.

We develop randomized (block) coordinate descent (CD) methods for linearly constrained convex optimization. Unlike most CD methods, we do not assume the constraints to be separable, but let them be coupled linearly. To our knowledge, ours is the first CD method that allows linear coupling constraints, without making th…

We consider a generic convex-concave saddle point problem with separable structure, a form that covers a wide-ranged machine learning applications. Under this problem structure, we follow the framework of primal-dual updates for saddle point problems, and incorporate stochastic block coordinate descent with adaptive st…

We consider a class (M, g, q) of four-dimensional Riemannian manifolds M, where besides the metric g there is an additional structure q, whose fourth power is the unit matrix. We use the existence of a local coordinate system such that there the coordinates of g and q are circulant matrices. In this system q has consta…

We prove that a plane domain which is almost isoperimetric (with respect to the $L^1$ metric) is close to a square whose sides are parallel to the coordinates axis. Closeness is measured either by $L^\infty$ Haussdorf distance or Fraenkel asymmetry. In the first case, we determine the extremal domains.

mlr3mbo is a modular R toolbox for Bayesian optimization.

In this paper, we use two conformal non-homogeneous coordinate systems, modeled on the de Sitter space ${\mathbb S}^{m+1}_1$, to cover the conformal space ${\mathbb Q}^{m+1}_1$, so that the conformal geometry of regular space-like hypersurfaces in $\mathbb{Q}^{m+1}_1$ is treated as that of hypersurfaces in ${\mathbb S}…

New proof shows affine manifolds with parallel volume are Riemannian-flat.

Novel ML approach solves complex warehouse routing problem.

Model predicts web page parallelism for improved browser performance and energy.

We study the performance of a family of randomized parallel coordinate descent methods for minimizing the sum of a nonsmooth and separable convex functions. The problem class includes as a special case L1-regularized L1 regression and the minimization of the exponential loss ("AdaBoost problem"). We assume the input da…

In this paper, we propose a convergent parallel best-response algorithm with the exact line search for the nondifferentiable nonconvex sparsity-regularized rank minimization problem. On the one hand, it exhibits a faster convergence than subgradient algorithms and block coordinate descent algorithms. On the other hand,…

The study examines parallel forms on manifolds, focusing on specific dimensions and forms.

We introduce a stochastic model for noisy vector fields on manifolds.

Humans can only interact with part of the surrounding environment due to biological restrictions. Therefore, we learn to reason the spatial relationships across a series of observations to piece together the surrounding environment. Inspired by such behavior and the fact that machines also have computational constraint…