arXiv research

LocalKMeans parallelizes Lloyd's algorithm for distributed data.

Develops local curvature estimates for mean curvature flow.

Study efficient power iteration for tensor models, proving convergence under specific conditions.

We provide a sufficient condition for the local stability of closed Einstein manifolds of positive Ricci curvature under the Ricci iteration in terms of the spectrum of the Lichnerowicz Laplacian acting on divergence-free tensor fields. We use this result to consider the stability of several Einstein manifolds under th…

Local Linear embedding (LLE) is a popular dimension reduction method. In this paper, we first show LLE with nonnegative constraint is equivalent to the widely used Laplacian embedding. We further propose to iterate the two steps in LLE repeatedly to improve the results. Thirdly, we relax the kNN constraint of LLE and p…

In this paper we consider $l_0$ regularized convex cone programming problems. In particular, we first propose an iterative hard thresholding (IHT) method and its variant for solving $l_0$ regularized box constrained convex programming. We show that the sequence generated by these methods converges to a local minimizer.…

Byzantine-resilient federated learning with local iterations and robust mean estimation.

New iterative schemes solve Yamabe-type equations on closed manifolds.

The L1 loss landscape of neural nets near local minima behaves differently, revealing exponential decay and increased vertex density.

LocalNewton reduces communication in distributed learning.

Efficient local planning with linear approximations for agents with limited simulator access.

Sublinear LSVI via LSH reduces runtime to sublinear in actions.

To accelerate the training of machine learning models, distributed stochastic gradient descent (SGD) and its variants have been widely adopted, which apply multiple workers in parallel to speed up training. Among them, Local SGD has gained much attention due to its lower communication cost. Nevertheless, when the data …

Paper improves a method for fast global and local convergence in optimization.

In this paper, we propose a new adaptive stochastic gradient Langevin dynamics (ASGLD) algorithmic framework and its two specialized versions, namely adaptive stochastic gradient (ASG) and adaptive gradient Langevin dynamics(AGLD), for non-convex optimization problems. All proposed algorithms can escape from saddle poi…

The paper estimates curvature for a specific flow on manifolds.

Improved ADMM for convex distributed learning with differential privacy.

This paper analyzes a simplified strategy for nonlinear control using local linear models and iLQR updates.

Alternating direction method of multiplier (ADMM) is a popular method used to design distributed versions of a machine learning algorithm, whereby local computations are performed on local data with the output exchanged among neighbors in an iterative fashion. During this iterative process the leakage of data privacy a…

Paper proposes a new descriptor for early trajectory characterization in matrix iterations.

LES optimizes designs by sampling descent sequences, achieving strong sample efficiency.

LARA forecasts financial asset trends by refining noisy labels and extracting profitable samples.

Power iteration has been generalized to solve many interesting problems in machine learning and statistics. Despite its striking success, theoretical understanding of when and how such an algorithm enjoys good convergence property is limited. In this work, we introduce a new class of optimization problems called scale …

This paper analyzes Local SGD for federated learning, achieving both statistical and communication efficiency.

Study smooth mappings between manifolds and their properties.

We prove the transversality result necessary for defining local Morse chain complexes with finite cyclic group symmetry. Our arguments use special regularized distance functions constructed using classical covering lemmas, and an inductive perturbation process indexed by the strata of the isotropy set. A global existen…

Paper proposes a pre-conditioning method to speed up gradient descent in multi-agent optimization.

New method for sampling from complex distributions using stochastic localization.

Paper analyzes solutions to quasilinear elliptic equations on manifolds using Nash-Moser iteration.

We study robust distributed learning that involves minimizing a non-convex loss function with saddle points. We consider the Byzantine setting where some worker machines have abnormal or even arbitrary and adversarial behavior. In this setting, the Byzantine machines may create fake local minima near a saddle point tha…

Secure federated learning framework resists adversarial users.

Multi-view spectral clustering, which aims at yielding an agreement or consensus data objects grouping across multi-views with their graph laplacian matrices, is a fundamental clustering problem. Among the existing methods, Low-Rank Representation (LRR) based method is quite superior in terms of its effectiveness, intu…

We study distributed computing of the truncated singular value decomposition problem. We develop an algorithm that we call \texttt{LocalPower} for improving communication efficiency. Specifically, we uniformly partition the dataset among $m$ nodes and alternate between multiple (precisely $p$) local power iterations an…

By using the De Giorgi iteration method we will give a new simple proof of the recent result of B.Kotschwar, O.Munteanu, J.Wang [KMW] and N.Sesum [S] on the local boundedness of the Riemmanian curvature tensor of solutions of Ricci flow in terms of its inital value on a given ball and a local uniform bound on the Ricci…

In a recent series of papers it has been established that variants of Gradient Descent/Ascent and Mirror Descent exhibit last iterate convergence in convex-concave zero-sum games. Specifically, \cite{DISZ17, LiangS18} show last iterate convergence of the so called "Optimistic Gradient Descent/Ascent" for the case of \t…

Multiple gossip steps improve decentralized optimization convergence.

In this paper, we study the efficiency of a {\bf R}estarted {\bf S}ub{\bf G}radient (RSG) method that periodically restarts the standard subgradient method (SG). We show that, when applied to a broad class of convex optimization problems, RSG method can find an $ε$-optimal solution with a lower complexity than the SG m…

Paper analyzes adaptive ISTA with MAD for LASSO problem.

Policy gradient converges linearly with Hadamard parameterization in tabular settings.

We present a distributed proximal-gradient method for optimizing the average of convex functions, each of which is the private local objective of an agent in a network with time-varying topology. The local objectives have distinct differentiable components, but they share a common nondifferentiable component, which has…

Improved API to achieve optimal error bound and query complexity in local planning.

AutoStep MCMC adapts step size locally for better sampling efficiency.

This paper presents the first theoretical results showing that stable identification of overcomplete $μ$-coherent dictionaries $Φ\in \mathbb{R}^{d\times K}$ is locally possible from training signals with sparsity levels $S$ up to the order $O(μ^{-2})$ and signal to noise ratios up to $O(\sqrt{d})$. In particular the di…

We introduce a collaborative learning framework allowing multiple parties having different sets of attributes about the same user to jointly build models without exposing their raw data or model parameters. In particular, we propose a Federated Stochastic Block Coordinate Descent (FedBCD) algorithm, in which each party…

Early stopping of iterative algorithms is a widely-used form of regularization in statistics, commonly used in conjunction with boosting and related gradient-type algorithms. Although consistency results have been established in some settings, such estimators are less well-understood than their analogues based on penal…

Paper develops Gaussian approximations and bootstrap for federated LSA with trade-off bounds.

New averaging technique speeds up Newton method convergence.

ParK efficiently solves kernel ridge regression for large datasets.