arXiv research

A new method for high-dimensional Bayesian optimization.

Coordinate descent methods usually minimize a cost function by updating a random decision variable (corresponding to one coordinate) at a time. Ideally, we would update the decision variable that yields the largest decrease in the cost function. However, finding this coordinate would require checking all of them, which…

We propose a new stochastic coordinate descent method for minimizing the sum of convex functions each of which depends on a small number of coordinates only. Our method (APPROX) is simultaneously Accelerated, Parallel and PROXimal; this is the first time such a method is proposed. In the special case when the number of…

New framework improves EM algorithm convergence under log-Sobolev inequality.

In this paper we analyze the randomized block-coordinate descent (RBCD) methods proposed in [8,11] for minimizing the sum of a smooth convex function and a block-separable convex function. In particular, we extend Nesterov's technique developed in [8] for analyzing the RBCD method for minimizing a smooth convex functio…

We provide upper bounds of the expected Wasserstein distance between a probability measure and its empirical version, generalizing recent results for finite dimensional Euclidean spaces and bounded functional spaces. Such a generalization can cover Euclidean spaces with large dimensionality, with the optimal dependence…

Develops DP-SCD for stochastic coordinate descent, making it differentially private.

Knowledge graph embedding, which aims to represent entities and relations as low dimensional vectors (or matrices, tensors, etc.), has been shown to be a powerful technique for predicting missing links in knowledge graphs. Existing knowledge graph embedding models mainly focus on modeling relation patterns such as symm…

Uniform sampling of training data has been commonly used in traditional stochastic optimization algorithms such as Proximal Stochastic Gradient Descent (prox-SGD) and Proximal Stochastic Dual Coordinate Ascent (prox-SDCA). Although uniform sampling can guarantee that the sampled stochastic quantity is an unbiased estim…

RL optimizes resource allocation in MG by balancing experience and exploration.

sEM uses optimal transport to improve EM algorithm for better convergence and avoiding local optima.

Machine learning often needs to model density from a multidimensional data sample, including correlations between coordinates. Additionally, we often have missing data case: that data points can miss values for some of coordinates. This article adapts rapid parametric density estimation approach for this purpose: model…

This work uses a scalable approach to identify partially observed nonlinear systems.

DECAF optimizes molecular graphs for ensemble properties, improving drug design accuracy.

Randomized block-diagonal preconditioning improves parallel learning convergence.

New algorithm for robust circular coordinates in recurrent time series data.

In this paper we investigate the relation between complexified Fenchel-Nielsen coordinates and spectral network coordinates on Seiberg-Witten moduli space. The main technique is the comparison of exact expressions for the expectation value of 't Hooft defects in certain 4D $SU(2)$ $\mathcal{N}=2$ gauge theories. We der…

New adaptive stepsize method for stochastic approximation converges to target point.

We consider the problem of estimating the arithmetic average of a finite collection of real vectors stored in a distributed fashion across several compute nodes subject to a communication budget constraint. Our analysis does not rely on any statistical assumptions about the source of the vectors. This problem arises as…

Differentially private random block coordinate descent improves utility in machine learning.

We present criteria for establishing a triangulation of a manifold. Given a manifold M, a simplicial complex A, and a map H from the underlying space of A to M, our criteria are presented in local coordinate charts for M, and ensure that H is a homeomorphism. These criteria do not require a differentiable structure, or…

Concept Factorization (CF) and its variants may produce inaccurate representation and clustering results due to the sensitivity to noise, hard constraint on the reconstruction error and pre-obtained approximate similarities. To improve the representation ability, a novel unsupervised Robust Flexible Auto-weighted Local…

New action-angle coordinates found for singular symplectic manifolds.

Proposes a neural network method to correct residual distortions in coordinate transformations.

Simplifies neural network models by explicitly enforcing constraints in Cartesian coordinates.

Accelerated coordinate descent is widely used in optimization due to its cheap per-iteration cost and scalability to large-scale problems. Up to a primal-dual transformation, it is also the same as accelerated stochastic gradient descent that is one of the central methods used in machine learning. In this paper, we imp…

We study fractional Sobolev and Besov spaces on noncompact Riemannian manifolds with bounded geometry. Usually, these spaces are defined via geodesic normal coordinates which, depending on the problem at hand, may often not be the best choice. We consider a more general definition subject to different local coordinates…

We study the limit of quasilocal mass defined in [4] and [5] for a family of spacelike 2-surfaces in spacetime. In particular, we show the limit coincides with the ADM mass at spatial infinity. The limit for coordinate spheres of a boosted slice of the Schwarzchild solution is computed explicitly and shown to give the …

This paper is concerned with improving the empirical convergence speed of block-coordinate descent algorithms for approximate nonnegative tensor factorization (NTF). We propose an extrapolation strategy in-between block updates, referred to as heuristic extrapolation with restarts (HER). HER significantly accelerates t…

Estimates hybrid dynamical systems with polynomial expansions and Markovian switching.

Paper proposes a method to improve circular coordinate representation for detecting changes in high-dimensional datasets.

Typical spoken language understanding systems provide narrow semantic parses using a domain-specific ontology. The parses contain intents and slots that are directly consumed by downstream domain applications. In this work we discuss expanding such systems to handle compound entities and intents by introducing a domain…

Improves spline quality and accuracy in computational microscopy.

Enhanced model predicts chaotic systems with improved long-term accuracy.

We introduce a proximal version of dual coordinate ascent method. We demonstrate how the derived algorithmic framework can be used for numerous regularized loss minimization problems, including $\ell_1$ regularization and structured output SVM. The convergence rates we obtain match, and sometimes improve, state-of-the-…

The pathwise coordinate optimization is one of the most important computational frameworks for high dimensional convex and nonconvex sparse learning problems. It differs from the classical coordinate optimization algorithms in three salient features: {\it warm start initialization}, {\it active set updating}, and {\it …

A general class of Newton algorithms on Graßmann and Lagrange-Graßmann manifolds is introduced, that depends on an arbitrary pair of local coordinates. Local quadratic convergence of the algorithm is shown under a suitable condition on the choice of coordinate systems. Our result extends and unifies previous convergenc…

We develop parallel and distributed Frank-Wolfe algorithms; the former on shared memory machines with mini-batching, and the latter in a delayed update framework. Whenever possible, we perform computations asynchronously, which helps attain speedups on multicore machines as well as in distributed environments. Moreover…

This paper develops a path-first theory using signatures and jump lifts for self-exiting processes.

Intrinsically motivated reinforcement learning aims to address the exploration challenge for sparse-reward tasks. However, the study of exploration methods in transition-dependent multi-agent settings is largely absent from the literature. We aim to take a step towards solving this problem. We present two exploration m…

PURE-CD algorithm proves complexity bounds for convex-concave problems.

How do groups of individuals achieve consensus in movement decisions? Do individuals follow their friends, the one predetermined leader, or whomever just happens to be nearby? To address these questions computationally, we formalize "Coordination Strategy Inference Problem". In this setting, a group of multiple individ…

Sequential coordinate ascent is more robust in high-dimensional linear regression.

In Physics and in Mathematics $\mathbb{Z}_2^n$-gradings, $n \geq 2$, do appear quite frequently. The corresponding sign rules are determined by the `scalar product' of the involved $\mathbb{Z}_2^n$-degrees. The present paper is the first of a series on $\mathbb{Z}_2^n$-Supergeometry. The new theory exhibits challenging…

A multi-neck spacetime wormhole is constructed with a simple metric tensor.

The enumeration of normal surfaces is a crucial but very slow operation in algorithmic 3-manifold topology. At the heart of this operation is a polytope vertex enumeration in a high-dimensional space (standard coordinates). Tollefson's Q-theory speeds up this operation by using a much smaller space (quadrilateral coord…

We construct from a real affine manifold with singularities (a tropical manifold) a degeneration of Calabi-Yau manifolds. This solves a fundamental problem in mirror symmetry. Furthermore, a striking feature of our approach is that it yields an explicit and canonical order-by-order description of the degeneration via f…

This study considers that the collective route choices of travelers en route represent a resolution of their competition on network routes. Well understanding this competition and coordinating their route choices help mitigate urban traffic congestion. Even though existing studies have developed such mechanisms (e.g., …