The pathwise coordinate optimization is one of the most important computational frameworks for high dimensional convex and nonconvex sparse learning problems. It differs from the classical coordinate optimization algorithms in three salient features: {\it warm start initialization}, {\it active set updating}, and {\it …
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,051 papers · 148 categories
Trend · papers per month
48 results for “optimal coordination”
A new method for high-dimensional Bayesian optimization.
problem Challenges in extending BO to high dimensions.
method Expected Coordinate Improvement (ECI) criterion for high-dimensional Bayesian optimization.
result Significantly better results than standard BO and competitive results with state-of-the-art methods.
This monograph presents a class of algorithms called coordinate descent algorithms for mathematicians, statisticians, and engineers outside the field of optimization. This particular class of algorithms has recently gained popularity due to their effectiveness in solving large-scale optimization problems in machine lea…
Harmonic coordinates for Finsler manifolds prove a theorem but not optimal regularity.
problem Proving the Myers--Steenrod theorem for Finsler manifolds.
method Existence of harmonic coordinates for nonlinear Finsler Laplacian.
result Partial results on optimal regularity for Berwald metrics.
Coordinate descent (CD) algorithms have become the method of choice for solving a number of optimization problems in machine learning. They are particularly popular for training linear models, including linear support vector machine classification, LASSO regression, and logistic regression. We consider general CD with …
Accelerated Randomized Coordinate Descent Algorithms for Stochastic Optimization and Online Learningcs.LG
We propose accelerated randomized coordinate descent algorithms for stochastic optimization and online learning. Our algorithms have significantly less per-iteration complexity than the known accelerated gradient algorithms. The proposed algorithms for online learning have better regret performance than the known rando…
Stochastic Coordinate Minimization with Progressive Precision for Stochastic Convex Optimizationstat.ML
Optimizes CM for stochastic convex optimization with progressive precision.
problem Stochastic nature of objective function in convex optimization.
method Iterative coordinate minimization with optimal precision control.
result Order-optimal regret performance for strongly convex and nonsmooth functions.
New algorithm optimizes Bayesian network learning from Gaussian data.
problem Learning Bayesian networks from Gaussian observational data.
method Proposes a coordinate descent algorithm for -penalized maximum likelihood estimation.
result The algorithm converges to a coordinate-wise minimum and achieves optimal objective value as sample size increases.
We propose and analyze a new parallel coordinate descent method---`NSync---in which at each iteration a random subset of coordinates is updated, in parallel, allowing for the subsets to be chosen non-uniformly. We derive convergence rates under a strong convexity assumption, and comment on how to assign probabilities t…
Optimized GPRNN reduces model complexity and overfitting, improving performance.
problem Overfitting in neural networks and high model complexity.
method Gaussian Process Regression - Neural Network hybrid with optimized redundant coordinates.
result Optimized GPRNN achieves lower test set error with fewer terms/neurons.
Optimal Coordinate Ascent (OCA) improves feature selection in machine learning.
problem Efficiently selecting features in machine learning models.
method Optimal Coordinate Ascent (OCA) for feature selection.
result OCA outperforms previous methods in feature selection and model performance.
Cyclic coordinate descent identifies models in finite time and converges linearly.
problem Model identification in composite nonsmooth optimization problems.
method Cyclic coordinate descent for a wide class of functions.
result Explicit local linear convergence rates for coordinate descent.
New method speeds up optimization over probability measures.
problem High computational overhead in optimizing probability measures.
method Randomized coordinate descent on Wasserstein space.
result Significant speedups over full-gradient methods.
DP-SGD can update fewer coordinates while maintaining privacy.
problem How to update fewer coordinates in DP-SGD without losing optimization signal.
method TP-TopK (Two-Phase TopK DP-SGD), a two-phase method for coordinate-sparse private training.
result Private training can update fewer coordinates without losing optimization signal, scaling noise with active dimension \(k\) instead of full dimension \(d\).
A new method uses Coordinate Descent to optimize ResNet networks for private inference.
problem Reducing ReLU count in ResNet networks for private inference.
method Directly optimizing in the discrete domain using Coordinate Descent.
result Our method yields a sparse solution and is state-of-the-art.
RL optimizes resource allocation in MG by balancing experience and exploration.
problem Optimal resource allocation in competitive scenarios.
method Introduced RL to MG, allowing dynamic strategy adjustment based on experience and expected rewards.
result Achieves optimal resource coordination by balancing exploitation and exploration.
Two popular examples of first-order optimization methods over linear spaces are coordinate descent and matching pursuit algorithms, with their randomized variants. While the former targets the optimization by moving along coordinates, the latter considers a generalized notion of directions. Exploiting the connection be…
CAVI converges for log-concave measures via optimal transport.
problem Finding the closest product measure to a log-concave measure via CAVI.
method Adapting coordinate descent techniques from Euclidean space to optimal transport for log-concave densities.
result Proves convergence of CAVI for log-concave densities and provides rates of convergence under additional conditions.
We propose a new randomized coordinate descent method for a convex optimization template with broad applications. Our analysis relies on a novel combination of four ideas applied to the primal-dual gap function: smoothing, acceleration, homotopy, and coordinate descent with non-uniform sampling. As a result, our method…
New method reveals insights about stochastic optimization methods using modified equations.
problem Understanding the qualitative behavior of stochastic optimization algorithms.
method Developed a class of stochastic differential equations to approximate the dynamics of stochastic optimization methods.
result Mean-square stability of the modified equation provides qualitative insights about stochastic coordinate descent.
Some results on existence of global Chebyshev coordinates on a Riemannian manifold or, more generally, on Aleksandrov surface are proved. For instance, if the positive and the negative parts of integral curvature of a Riemannian manifold M are less than 2πeach, then there exist global Chebyshev coordinates on M. These …
We develop a coordinate-free approach to natural gradient descent for scalable neural networks.
problem First-order optimization methods are sensitive to model parameterization.
method We construct a coordinate-free natural gradient and analyze its invariance properties for K-FAC.
result K-FAC's natural gradient matches the coordinate-free update, maintaining invariance to affine transformations.
Task Embedded Coordinate Update: A Realizable Framework for Multivariate Non-convex Optimizationcs.LG
Proposes TECU framework for efficient non-convex optimization.
problem Multivariate non-convex optimization problems with coupled objective functions.
method Embeds task-specific strategies into coordinate descent update schemes.
result Demonstrates improved efficiency and effectiveness in solving practical problems.
New algorithm learns coordinated decisions in loosely-coupled multi-agent systems.
problem Learning coordinated decisions in multi-agent systems with sparse interactions.
method Multi-Agent Thompson Sampling (MATS) for multi-agent multi-armed bandits.
result MATS achieves sublinear regret and outperforms MAUCE on synthetic and real benchmarks.
Random scan CAVI converges linearly under log-concave assumptions.
problem Analyzing the convergence rate of random scan Coordinate Ascent Variational Inference (CAVI) under log-concave conditions.
method Building on previous work, we analyze the random scan version of CAVI using optimal transport geometry.
result We obtain tight linear convergence rates for the random scan version of CAVI.
Optimized coordinate system improves sparse grid regression performance.
problem Sparse grid methods struggle with skewed and rotated coordinates.
method Proposes an optimized coordinate system to reduce effective dimensionality.
result Adaptive sparse grid least squares algorithm benefits from preprocessing.
Greedy coordinate descent achieves linear convergence for non-smooth composite problems.
problem Optimization of non-smooth composite problems.
method Greedy selection of subgradients for optimization.
result Linear convergence rates independent of problem dimension .
Paper proposes a new method to optimize feature coordinates for better image classification.
problem Improving feature extraction for better machine learning classification.
method Mutual-energy inner product optimization method.
result The method enhances low-frequency features and suppresses high-frequency noise, leading to better classification results.
CD methods tackle nonconvex optimization with three terms, achieving critical points.
problem Minimizing nonconvex functions with specific structure.
method Developed randomized CD, randomly permuted CD, and accelerated CD methods.
result CD methods converge to critical points with sublinear complexity.
We propose a randomized block-coordinate variant of the classic Frank-Wolfe algorithm for convex optimization with block-separable constraints. Despite its lower iteration cost, we show that it achieves a similar convergence rate in duality gap as the full Frank-Wolfe algorithm. We also show that, when applied to the d…
There has been significant recent work on the theory and application of randomized coordinate descent algorithms, beginning with the work of Nesterov [SIAM J. Optim., 22(2), 2012], who showed that a random-coordinate selection rule achieves the same convergence rate as the Gauss-Southwell selection rule. This result su…
This paper focuses on coordinate update methods, which are useful for solving problems involving large or high-dimensional datasets. They decompose a problem into simple subproblems, where each updates one, or a small block of, variables while fixing others. These methods can deal with linear and nonlinear mappings, sm…
We propose a doubly stochastic primal-dual coordinate optimization algorithm for empirical risk minimization, which can be formulated as a bilinear saddle-point problem. In each iteration, our method randomly samples a block of coordinates of the primal and dual solutions to update. The linear convergence of our method…
Ballistic Convergence in Hit-and-Run Monte Carlo and a Coordinate-free Randomized Kaczmarz Algorithmmath.PR
Paper analyzes Hit-and-Run's convergence rates and applies similar methods to randomized Kaczmarz.
problem Quantifying advantages of Hit-and-Run's coordinate-free property.
method Sharp estimates via coupling methods and mixing time bounds.
result Ballistic and superdiffusive convergence rates in certain settings.
Study uses reinforcement learning to optimize metachronal paddling at low Reynolds number.
problem Optimizing metachronal paddling strategies for efficient swimming at low Reynolds numbers.
method Applied reinforcement learning to a swimmer model with varying paddle spacings.
result The reinforcement learning algorithm selects a back-to-front metachronal wave-like stroke as the most efficient, regardless of the number of paddles.
Differentially private random block coordinate descent improves utility in machine learning.
problem Lack of privacy in classical CD methods when handling sensitive information.
method Proposes a differentially private random block coordinate descent method using sketch matrices and importance sampling.
result Demonstrates improved convergence rates and utility guarantees compared to non-private methods.
We use differential equations based approaches to provide some {\it \textbf{physics}} insights into analyzing the dynamics of popular optimization algorithms in machine learning. In particular, we study gradient descent, proximal gradient descent, coordinate gradient descent, proximal coordinate gradient, and Newton's …
MERL uses evolutionary and gradient-based methods to optimize sparse team-based and dense agent-specific rewards in multiagent coordination.
problem Training multiagent reinforcement learning policies on sparse team-based rewards is difficult and relying solely on agent-specific rewards is sub-optimal.
method MERL employs a split-level training platform with an evolutionary algorithm and a gradient-based optimizer, transferring skills between the two processes.
result MERL significantly outperforms state-of-the-art methods on coordination benchmarks.
Efficient CD algorithms on matrix manifolds for optimization problems.
problem Optimization on Riemannian manifolds with computational efficiency.
method Developed coordinate descent algorithms for various matrix manifolds, updating only a few variables at each iteration.
result Proposed algorithms achieve low cost per iteration and a more efficient variant via first-order approximation.
This paper introduces a new method for optimizing large-scale problems using Markov chain block updates.
problem Optimizing large-scale problems with efficient and natural block selection.
method Markov chain block coordinate descent (BCD) for optimization.
result The method converges for minimizing Lipschitz differentiable functions, with sublinear and linear convergence rates for convex and strongly convex functions, respectively.
In this paper we study the fundamental problems of maximizing a continuous non-monotone submodular function over the hypercube, both with and without coordinate-wise concavity. This family of optimization problems has several applications in machine learning, economics, and communication systems. Our main result is the…
Optimizes coordinate charts for smooth elliptic structures.
problem Achieving optimal regularity for coordinate charts of smooth elliptic structures.
method Generalizing Malgrange's proof of the Newlander-Nirenberg Theorem to this setting.
result Optimal regularity for coordinate charts of smooth elliptic structures.
Stochastic Primal Dual Coordinate Method with Non-Uniform Sampling Based on Optimality Violationsstat.ML
We study primal-dual type stochastic optimization algorithms with non-uniform sampling. Our main theoretical contribution in this paper is to present a convergence analysis of Stochastic Primal Dual Coordinate (SPDC) Method with arbitrary sampling. Based on this theoretical framework, we propose Optimality Violation-ba…
DSCOVR: Randomized Primal-Dual Block Coordinate Algorithms for Asynchronous Distributed Optimizationmath.OC
Machine learning with big data often involves large optimization models. For distributed optimization over a cluster of machines, frequent communication and synchronization of all model parameters (optimization variables) can be very costly. A promising solution is to use parameter servers to store different subsets of…
Isometry pursuitstat.ML
Isometry pursuit identifies orthonormal submatrices from wide matrices.
problem Identifying isometric embeddings from wide matrices.
method A convex algorithm combining normalization and multitask basis pursuit.
result The method identifies isometric embeddings from interpretable dictionaries.
This study develops methods to coordinate travel routes to reduce congestion.
problem Coordination of travel routes to reduce urban traffic congestion.
method Developed mathematical approaches to quantify coordination potential and adaptive centroid-based clustering algorithm (ACCA).
result ACCA efficiently forms proper coordination groups for CB-CRM, improving efficiency with minimal performance loss.
New DP-CD method outperforms DP-SGD in solving composite DP-ERM problems.
problem Privacy-preserving machine learning with differential privacy.
method Differentially Private proximal Coordinate Descent (DP-CD) for composite Empirical Risk Minimization (ERM).
result DP-CD outperforms DP-SGD due to larger step sizes and better gradient exploitation.
Sequential coordinate ascent is more robust in high-dimensional linear regression.
problem Behavior difference between sequential and parallel coordinate ascent in variational inference.
method Comparison of sequential and parallel coordinate ascent algorithms in high-dimensional linear regression.
result Sequential algorithm converges under more relaxed conditions than parallel algorithm.