Study hypothesis testing under quantized samples with communication constraints, achieving near-optimal sample complexity.
problem Optimizing hypothesis testing with quantized samples and communication constraints.
method Developed a polynomial-time algorithm achieving near-optimal sample complexity under communication constraints.
result Achieved near-optimal sample complexity under communication constraints, with a logarithmic factor increase over unconstrained setting.
We study distributed estimation methods under communication constraints in a distributed version of the nonparametric random design regression model. We derive minimax lower bounds and exhibit methods that attain those bounds. Moreover, we show that adaptive estimation is possible in this setting.
Study binary hypothesis testing with privacy and communication constraints.
problem Binary hypothesis testing under local differential privacy and communication constraints.
method Qualifies results as minimax or instance optimal, develops instance-optimal algorithms.
result Achieves minimum possible sample complexity under both privacy and communication constraints.
New framework for distributed nonparametric estimation under slow communication.
problem Efficiently estimate nonparametric models across multiple nodes with limited communication.
method Developed a general framework for nonparametric estimation under communication constraints.
result Derived minimax lower and upper bounds for various models.
Study on efficient estimation of Gaussian mean with limited communication.
problem Estimating Gaussian mean under communication constraints.
method Decomposition into localization and refinement stages, development of communication-efficient and statistically optimal procedures.
result Established minimax rates of convergence and developed optimal procedures.
This work proposes ACTC for adaptive distributed learning under communication constraints.
problem Adaptive distributed learning in networks with communication constraints.
method ACTC (Adapt-Compress-Then-Combine) strategy with diffusion exchange of compressed updates.
result ACTC iterates converge to the optimizer with significant bit savings.
Optimal distributed testing under communication constraints with shared randomness.
problem Signal detection in a distributed system with limited communication.
method Derivation of minimax testing errors, distributed testing algorithms, and theoretical lower bounds.
result Consistent nonparametric distributed testing is possible even with minimal communication.
Distributed sensors compress and send features to a fusion center for linear regression.
problem Efficiently compress and transmit features from distributed sensors to a fusion center under varying communication constraints.
method Designs a distributed and adaptive feature compression scheme using optimal quantizers and simple adaptive strategies.
result Demonstrates improved inference performance through simulated experiments.
Study quantile reward identification with 1-bit feedback constraints.
problem Best arm identification with quantile reward and 1-bit communication.
method Proposes an algorithm using noisy binary search for quantile reward estimation.
result Derives upper and lower bounds on sample complexity for 1-bit feedback.
SHIFT method optimally estimates heterogeneous discrete distributions with limited communication.
problem Collaborative learning of discrete distributions under heterogeneity and communication constraints.
method Two-stage method: First, users learn a central distribution; then, fine-tune this to estimate individual distributions.
result SHIFT is minimax optimal in the model of heterogeneity and under communication constraints.
We break dimension dependence in sparse distribution estimation with communication constraints.
problem Estimating sparse distributions with limited communication.
method Novel localization schemes and tree-based estimation.
result Achieve dimension-free convergence rate independent of dimension d. A decentralized policy achieves logarithmic regret for multi-agent MAB problems with communication constraints.
problem Decentralized policy for multi-agent MAB problems with option availability and communication constraints.
method Upper Confidence Bound (UCB) algorithms with non-stationary stochastic communication protocol.
result Guaranteed logarithmic regret for non-fully connected spatial graphs with communication constraints.
Paper addresses privacy and communication in distributed learning, achieving optimal performance.
problem Balancing privacy, communication, and accuracy in distributed learning and estimation.
method Developed novel encoding and decoding mechanisms for mean and frequency estimation under local differential privacy and communication constraints.
result Achieved optimal privacy and communication efficiency in mean and frequency estimation.
New method for distributed online learning with communication constraints reduces joint regret.
problem Joint regret minimization in a distributed online learning setting with communication constraints.
method Adaptive graph partitioning and comparator-adaptive online convex optimization with delayed gradient information.
result Optimal graph partition selection for adversarial activations and gradients reduces joint regret.
Many machine learning approaches are characterized by information constraints on how they interact with the training data. These include memory and sequential access constraints (e.g. fast first-order methods to solve stochastic optimization problems); communication constraints (e.g. distributed learning); partial acce…
Study on distributed nonparametric function estimation with optimal rate and cost of adaptation.
problem Optimal rate of convergence and cost of adaptation in distributed nonparametric function estimation.
method Distributed minimax estimation and adaptive estimation under communication constraints for Gaussian sequence model and white noise model.
result Established minimax rate of convergence and exact communication cost for adaptation.
This study analyzes communication constraints in MoE architectures using information theory.
problem Communication constraints in Mixture-of-Experts (MoE) architectures.
method Developed a rate-distortion characterization of finite-rate gating in MoE architectures using information theory.
result Yielded capacity-aware limits for communication-constrained MoE systems.
New algorithm tackles batched stochastic linear bandits with 1-bit communication constraints.
problem Stochastic linear bandits with 1-bit communication constraints.
method Phased-elimination algorithms based on G-optimal designs and 1-bit mean estimation.
result Achieves near-optimal regret bounds for broad scaling regimes.
Paper studies distributed learning with limited communication bits, achieving optimal error exponents.
problem Distributed hypothesis testing with constant communication bits.
method Geometric approach in distribution spaces, encoding empirical distributions to transmission bits.
result Optimal achievable error exponents and coding schemes for various communication constraints.
Network agents solve adaptive regression problems with compressed signals.
problem Solving regression problems in networks with communication constraints.
method ACTC diffusion strategy with randomized differential compression.
result Optimized resource allocation improves performance.
Algorithm reduces regret in multi-agent bandits through gossiping.
problem Minimizing cumulative regret in decentralized multi-agent bandits.
method Insert-Eliminate algorithm with gossip communication.
result Regret significantly reduced with minimal collaboration.
As convolutional neural networks (CNNs) enable state-of-the-art computer vision applications, their high energy consumption has emerged as a key impediment to their deployment on embedded and mobile devices. Towards efficient image classification under hardware constraints, prior work has proposed adaptive CNNs, i.e., …
A central result in statistical theory is Pinsker's theorem, which characterizes the minimax rate in the normal means model of nonparametric estimation. In this paper, we present an extension to Pinsker's theorem where estimation is carried out under storage or communication constraints. In particular, we place limits …
A new approach for cooperative multi-agent reinforcement learning with limited communication, reducing the number of communication rounds.
problem Limited communication in decentralized MARL systems leads to outdated information and unstable learning.
method Base policy prediction technique to estimate gradients and collect samples for a sequence of base policies.
result The proposed algorithm converges to an ε-Nash equilibrium with significantly fewer communication rounds and samples.
Paper develops an efficient mean estimator for 1-bit communication constraints.
problem Mean estimation under 1-bit communication constraints.
method Adaptive mean estimator based on randomized threshold queries.
result Order-optimal sample complexity in various tail regimes.
We develop model free PAC performance guarantees for multiple concurrent MDPs, extending recent works where a single learner interacts with multiple non-interacting agents in a noise free environment. Our framework allows noisy and resource limited communication between agents, and develops novel PAC guarantees in this…
This work addresses the instability in asynchronous data parallel optimization. It does so by introducing a novel distributed optimizer which is able to efficiently optimize a centralized model under communication constraints. The optimizer achieves this by pushing a normalized sequence of first-order gradients to a pa…
In many real-world applications of machine learning, data are distributed across many clients and cannot leave the devices they are stored on. Furthermore, each client's data, computational resources and communication constraints may be very different. This setting is known as federated learning, in which privacy is a …
In this paper, learning of tree-structured Gaussian graphical models from distributed data is addressed. In our model, samples are stored in a set of distributed machines where each machine has access to only a subset of features. A central machine is then responsible for learning the structure based on received messag…
Centralised training with decentralised execution is an important setting for cooperative deep multi-agent reinforcement learning due to communication constraints during execution and computational tractability in training. In this paper, we analyse value-based methods that are known to have superior performance in com…
The paper analyzes how disturbances affect the convergence of algorithms in complex systems.
problem Analyzing the impact of disturbances on algorithm convergence in complex systems.
method Leveraging converse Lyapunov theorems, the paper derives stability bounds and convergence rates in the presence of disturbances.
result Key inequalities quantify the impact of disturbances on algorithmic performance.
This paper studies the problem of nonparametric estimation of a smooth function with data distributed across multiple machines. We assume an independent sample from a white noise model is collected at each machine, and an estimator of the underlying true function needs to be constructed at a central machine. We place l…
We formulate the notion of minimax estimation under storage or communication constraints, and prove an extension to Pinsker's theorem for nonparametric estimation over Sobolev ellipsoids. Placing limits on the number of bits used to encode any estimator, we give tight lower and upper bounds on the excess risk due to qu…
The goal of decentralized optimization over a network is to optimize a global objective formed by a sum of local (possibly nonsmooth) convex functions using only local computation and communication. It arises in various application domains, including distributed tracking and localization, multi-agent co-ordination, est…
A communication-efficient method controls FDR in network settings.
problem Controlling FDR in networks with limited communication.
method Sample-and-Forward: a flexible procedure for multihop networks.
result Nodes can control FDR without sharing p-values, achieving power and FDR control.
Distributed Thompson sampling improves regret convergence in constrained communication networks.
problem Maximizing a black-box function with multi-agent Bayesian optimization under communication constraints.
method Distributed Thompson sampling using Gaussian processes, with theoretical bounds on regret convergence.
result Theoretical bounds on Bayesian average and simple regret depend on communication graph structure and are applicable in constrained networks.
Optimization results are one method for understanding neural computation from Nature's perspective and for defining the physical limits on neuron-like engineering. Earlier work looks at individual properties or performance criteria and occasionally a combination of two, such as energy and information. Here we make use …
Paper proposes a distributed sampling method for Bayesian inference.
problem Privacy and communication constraints in spatially distributed datasets.
method Alternating Direction Method of Multipliers for distributed sampling.
result Algorithm converges to target distribution in Wasserstein distance.
Unified approach for federated learning using MM optimization.
problem Scaling stochastic optimization to federated learning.
method Unified Majorize-Minimize (MM) framework for stochastic optimization, extended to federated learning.
result Unified algorithm \QSMM\ for federated learning that aggregates surrogate majorizing functions.
A new discrete privacy mechanism for federated learning.
problem Differentially private federated learning with communication constraints.
method Skellam mechanism based on Poisson distributions.
result Skellam mechanism provides similar privacy-accuracy trade-offs as Gaussian mechanism.
MARL algorithm uses regularization to avoid explicit structures, improving performance.
problem Lack of effective reinforcement learning methods for multi-agent systems.
method MARQ uses regularization to promote structured exploration without explicit centralized structures.
result MARQ outperforms existing methods in multi-agent environments.
In many real-world settings, a team of agents must coordinate their behaviour while acting in a decentralised way. At the same time, it is often possible to train the agents in a centralised fashion in a simulated or laboratory setting, where global state information is available and communication constraints are lifte…
Asynchronous federated modeling improves spatial data sharing without centralizing raw data.
problem Privacy and bandwidth constraints in distributed spatial data.
method Asynchronous federated modeling using low-rank Gaussian process approximations with block-wise optimization and adaptive strategies.
result Asynchronous federated modeling achieves synchronous performance and outperforms it in heterogeneous settings.
FLIX simplifies federated learning with efficient communication.
problem Handling constraints specific to federated learning.
method Introduces FLIX, a new framework for federated learning that addresses communication and personalization challenges.
result FLIX achieves dissimilarity regularization similar to local methods without requiring local steps.
We consider distributed statistical optimization in one-shot setting, where there are m machines each observing n i.i.d. samples. Based on its observed samples, each machine sends a B-bit-long message to a server. The server then collects messages from all machines, and estimates a parameter that minimizes an exp…
Local differential privacy (LDP) is a model where users send privatized data to an untrusted central server whose goal it to solve some data analysis task. In the non-interactive version of this model the protocol consists of a single round in which a server sends requests to all users then receives their responses. Th…
New algorithms for Bayesian inference in decentralized learning.
problem Bayesian inference in decentralized learning settings.
method Decentralized SGLD and Decentralized SGHMC.
result Convergence of iterates to target distribution in 2-Wasserstein distance.
Compressed Federated Distillation reduces communication in federated learning.
problem Communication constraints in Federated Learning.
method Compressed Federated Distillation (CFD) leverages soft labels and quantization techniques.
result Reduces communication by more than 4 orders of magnitude compared to Federated Averaging.