New algorithm for weighted low rank approximation with provable guarantees.
problem Weighted low rank approximation (WLRA) is computationally hard.
method Reweights the low rank solution using the weight matrix itself.
result Provably optimal approximation guarantees for WLRA.
Paper tackles low sample and communication complexities in decentralized bilevel optimization.
problem Decentralized bilevel optimization problems with limited computation and communication capabilities.
method Proposes INTERACT and SVR-INTERACT algorithms to achieve low sample and communication complexities.
result Achieves both low sample and communication complexities for solving decentralized bilevel optimization problems.
Low complexity decentralized neural net with centralized performance.
problem Training large neural networks in distributed nodes without data sharing.
method Layer-wise learning using ADMM for low complexity and centralized performance.
result Equivalent learning performance to centralized training in distributed nodes.
New framework reduces private mean estimation error with optimal efficiency.
problem Locally private mean estimation of high-dimensional vectors.
method ProjUnit framework: random projections, normalization, and optimal algorithm execution in lower dimensions.
result Optimal error up to a 1+o(1)-factor with computational efficiency and low communication complexity.
Federated learning supports exact support recovery with minimal communication.
problem Learning the exact support of sparse linear regression in federated learning.
method One-shot communication algorithm for exact support recovery without optimization.
result Polynomial sample complexity and logarithmic number of clients required.
To solve the big topic modeling problem, we need to reduce both time and space complexities of batch latent Dirichlet allocation (LDA) algorithms. Although parallel LDA algorithms on the multi-processor architecture have low time and space complexities, their communication costs among processors often scale linearly wi…
Statistical estimates can often be improved by fusion of data from several different sources. One example is so-called ensemble methods which have been successfully applied in areas such as machine learning for classification and clustering. In this paper, we present an ensemble method to improve community detection by…
Distributed learning techniques such as federated learning have enabled multiple workers to train machine learning models together to reduce the overall training time. However, current distributed training algorithms (centralized or decentralized) suffer from the communication bottleneck on multiple low-bandwidth worke…
STL-SGD accelerates Local SGD by gradually increasing communication periods.
problem Reducing communication complexity in distributed SGD.
method Gradually increasing communication periods while decreasing learning rate.
result Significant reduction in communication complexity with improved convergence rates.
A new hybrid-ordered SGD method reduces communication and complexity for non-convex optimization.
problem Balancing communication, computational complexity, and convergence rate in distributed non-convex optimization.
method Hybrid-ordered distributed SGD with pre-shared scalers and periodic vector communication.
result Order-wise faster convergence compared to existing methods.
This paper provides new algorithms for distributed clustering for two popular center-based objectives, k-median and k-means. These algorithms have provable guarantees and improve communication complexity over existing approaches. Following a classic approach in clustering by \cite{har2004coresets}, we reduce the proble…
Federated Q-Learning achieves linear regret speedup with low communication cost.
problem Achieving linear regret speedup in federated reinforcement learning without high communication costs.
method Proposed two federated Q-Learning algorithms: FedQ-Hoeffding and FedQ-Bernstein, using event-triggered synchronization, novel step size selection, and concentration inequalities.
result Total regrets achieve linear speedup compared to single-agent counterparts with logarithmic communication cost.
We investigate the properties of correlation based networks originating from economic complex systems, such as the network of stocks traded at the New York Stock Exchange (NYSE). The weaker links (low correlation) of the system are found to contribute to the overall connectivity of the network significantly more than t…
This paper tackles energy-efficient machine learning on low-power devices.
problem Energy consumption in machine learning due to data communication.
method Dynamic averaging for integer exponential families on low-power processors.
result Achieves comparable model quality with significantly less communication and energy.
New algorithm learns low-rank matrices with linear number of samples.
problem Learning low-rank matrices efficiently in latent-variable applications.
method Proposed algorithm that uses linear number of samples in high dimension.
result Learning kimesk, rank-r, matrices requires $Ω( rac{kr}{ε^2})$ samples. Modern distributed training of machine learning models suffers from high communication overhead for synchronizing stochastic gradients and model parameters. In this paper, to reduce the communication complexity, we propose \emph{double quantization}, a general scheme for quantizing both model parameters and gradients. …
This paper considers a new framework to detect communities in a graph from the observation of signals at its nodes. We model the observed signals as noisy outputs of an unknown network process, represented as a graph filter that is excited by a set of unknown low-rank inputs/excitations. Application scenarios of this m…
New evidence shows computational barriers in graphon estimation using low-degree polynomials.
problem Estimating graphons efficiently and accurately.
method Low-degree polynomials to analyze computational limits.
result Low-degree polynomial estimators cannot significantly outperform USVT in graphon estimation.
Study quantile reward identification with 1-bit feedback constraints.
problem Best arm identification with quantile reward and 1-bit communication.
method Proposes an algorithm using noisy binary search for quantile reward estimation.
result Derives upper and lower bounds on sample complexity for 1-bit feedback.
FedLoRU improves FL efficiency by using low-rank updates.
problem Communication inefficiency and performance reduction in Federated Learning.
method Proposes FedLoRU, a low-rank update framework for FL, which reduces communication costs while maintaining performance.
result FedLoRU achieves convergence rates similar to FedAvg and is robust to heterogeneous and large numbers of clients.
The sum-product or belief propagation (BP) algorithm is a widely-used message-passing algorithm for computing marginal distributions in graphical models with discrete variables. At the core of the BP message updates, when applied to a graphical model with pairwise interactions, lies a matrix-vector product with complex…
Study recovers community structure from coarse graph measurements.
problem Community recovery from low-resolution graph measurements.
method Formalized coarsening process of graph measurements, developed conditions for perfect recovery.
result Simple and closed-form asymptotic conditions for perfect recovery of coarse graph communities.
We propose an efficient meta-algorithm for Bayesian estimation problems that is based on low-degree polynomials, semidefinite programming, and tensor decomposition. The algorithm is inspired by recent lower bound constructions for sum-of-squares and related to the method of moments. Our focus is on sample complexity bo…
Motivated by the ever-increasing demands for limited communication bandwidth and low-power consumption, we propose a new methodology, named joint Variational Autoencoders with Bernoulli mixture models (VAB), for performing clustering in the compressed data domain. The idea is to reduce the data dimension by Variational…
We present a Communication-efficient Surrogate Likelihood (CSL) framework for solving distributed statistical inference problems. CSL provides a communication-efficient surrogate to the global likelihood that can be used for low-dimensional estimation, high-dimensional regularized estimation and Bayesian inference. For…
Study shows it's impossible to count communities without finding them.
problem Determining the number and sizes of communities in random graph models.
method Hypothesis testing between models with different community structures, using low-degree polynomial framework.
result Testing between two different planted distributions is as hard as finding the communities.
This paper analyzes DeepWalk and node2vec for community detection in large networks.
problem Community detection in large, sparse networks.
method Low-dimensional network embedding algorithms (DeepWalk and node2vec) applied to random walk segments.
result The performance of DeepWalk and node2vec in recovering communities depends on the length of random walk segments and sparsity of the network.
This paper reviews low voltage load forecasting methods and applications.
problem Reliable forecasting for low voltage networks is needed for decarbonization.
method Comprehensive survey of current approaches, challenges, and trends.
result Established an open list of low voltage datasets for further research.
A new federated learning algorithm improves on existing methods by exploiting data smoothness.
problem Federated learning optimization with smooth loss functions.
method Federated Low Rank Gradient Descent (FedLRGD) algorithm.
result FedLRGD outperforms Federated Averaging (FedAve) in federated oracle complexity under certain conditions.
Paper proves MDS NP-hard and provides a PTAS.
problem Theoretical limitations of MDS objective function.
method Proves NP-hardness and provides a PTAS approximation algorithm.
result Minimizing Kamada-Kawai objective is NP-hard.
Distributed-OMP recovers sparse vectors with low communication costs.
problem High-dimensional sparse linear regression with limited computation and communication.
method Distributed orthogonal matching pursuit (OMP) scheme.
result Support of the regression vector can be recovered with linear communication per machine and logarithmic in dimension.
Agents collaborate to reduce regret in a multi-agent linear bandit problem with side information.
problem Reducing regret in a multi-agent stochastic linear bandit with side information.
method A decentralized algorithm where agents communicate subspace indices and each plays a projected LinUCB on the corresponding low-dimensional subspace.
result Per-agent finite-time regret is much smaller when agents communicate compared to non-communicating case.
PowerGossip compresses model differences for decentralized deep learning with low-rank linear compressors.
problem Communication bottleneck in decentralized deep learning models.
method Low-rank linear compressors applied on model differences using power iteration steps.
result Asymptotically independent of network and compression, faster convergence, and comparable performance to tuned compression algorithms.
Mixed-precision CA-SGD for generalized linear models on GPUs
problem SGD communication bottleneck
method Mixed-precision CA-SGD
result Matches FP32 SGD loss within 0.5% on various problems
Optimal joint separation condition for radar and communications channels in dual-blind deconvolution.
problem Recovering information from overlaid radar and communications signals with unknown channels.
method Extremal functions from Beurling-Selberg interpolation theory for joint separation, nuclear norm minimization for matrix retrieval, and MUSIC for parameter estimation.
result Guaranteed well-conditioned Vandermonde matrix for MUSIC, validating theoretical findings.
Recently, researchers proposed various low-precision gradient compression, for efficient communication in large-scale distributed optimization. Based on these work, we try to reduce the communication complexity from a new direction. We pursue an ideal bijective mapping between two spaces of gradient distribution, so th…
Physics-based deep learning improves fiber-optic communication efficiency.
problem Improving signal propagation in fiber-optic communication systems.
method Parameterizing the split-step method of solving the nonlinear Schrödinger equation as a deep neural network.
result Filters can be pruned to as few as 3 taps/step without sacrificing performance.
New algorithm reduces sample and communication complexities in federated Q-learning.
problem Optimal Q-function learning in federated Q-learning with limited communication.
method Introduced Fed-DVR-Q algorithm for order-optimal sample and communication complexities.
result Complete characterization of sample-communication complexity trade-off.
A new parallel algorithm for learning optimal policies in MDPs with low communication costs.
problem Learning optimal policies for infinite-horizon MDPs.
method Primal-Dual Stochastic Mirror Descent for convex programming problems with inexact constraints.
result First parallel algorithm for average-reward MDPs with generative model and low communication costs.
Paper identifies sparse structures and communities in heterogeneous graphical models.
problem Detecting community structures in graphical models.
method Novel decomposition into sparse and low-rank parts, three-stage estimation procedure.
result Consistent model selection for adaptive ℓ1 penalized estimator. This paper investigates the autonomous control of massive unmanned aerial vehicles (UAVs) for mission-critical applications (e.g., dispatching many UAVs from a source to a destination for firefighting). Achieving their fast travel and low motion energy without inter-UAV collision under wind perturbation is a daunting c…
New findings support a new community recovery threshold for Stochastic Block Model with many communities.
problem Recovering communities in Stochastic Block Model with more than sqrt(n) communities.
method Counting specific motifs to achieve polynomial-time community recovery above a new threshold.
result LDP fails below the new threshold, but polynomial-time recovery is possible above it.
Community detection improves stock market portfolio optimization.
problem Improving portfolio optimization in financial markets.
method Community detection in correlation-based networks of worldwide stock markets.
result Portfolios constructed using community detection outperform traditional methods.
Study hypothesis testing under quantized samples with communication constraints, achieving near-optimal sample complexity.
problem Optimizing hypothesis testing with quantized samples and communication constraints.
method Developed a polynomial-time algorithm achieving near-optimal sample complexity under communication constraints.
result Achieved near-optimal sample complexity under communication constraints, with a logarithmic factor increase over unconstrained setting.
Novel algorithm resists Byzantine attacks in federated learning for PCA and LRCS.
problem Byzantine attacks in federated learning for PCA and LRCS.
method Subspace-Median algorithm for federated PCA, altGDmin for LRCS.
result Provably Byzantine-resilient communication-efficient and sample-efficient algorithms.
Communication-efficient SGD algorithms, which allow nodes to perform local updates and periodically synchronize local models, are highly effective in improving the speed and scalability of distributed SGD. However, a rigorous convergence analysis and comparative study of different communication-reduction strategies rem…
New method reduces distributed non-convex optimization rounds and bits.
problem Efficiently optimizing non-convex models in distributed systems.
method Introduces permutation compressors to reduce communication complexity.
result PermK compressors lead to significant communication complexity improvements.
Develops accelerated methods for optimization using low-dimensional projected-gradient information.
problem Optimization with low-dimensional projected-gradient information and Nesterov acceleration.
method Randomized-subspace Nesterov accelerated gradient methods for smooth convex and strongly convex optimization.
result Established accelerated oracle-complexity guarantees and unified basis for comparing sketch families.