The paper gives bounds for how long it takes for gossip protocols to spread information in networks.
problem Understanding the diffusion time in asynchronous gossip protocols.
method Provides non-asymptotic bounds for the number of messages needed for consensus in asynchronous gossip protocols.
result Explicit formula and approximation for the number of messages needed for consensus in different types of graphs.
Distributing Neural Network training is of particular interest for several reasons including scaling using computing clusters, training at data sources such as IOT devices and edge servers, utilizing underutilized resources across heterogeneous environments, and so on. Most contemporary approaches primarily address sca…
Floating Gossip improves continuous machine learning in a decentralized manner.
problem Continuous learning in infrastructure-less environments.
method Mean field approach to analyze the impact of communication and computing in Floating Gossip.
result Floating Gossip can effectively incorporate large amounts of data into machine learning models.
Paper proposes robust gossip algorithms for mean and trimmed mean estimation.
problem Vulnerability of mean-based gossip algorithms to malicious nodes.
method Developed extsc{GoRank} for rank estimation and extsc{GoTrim} for trimmed mean estimation.
result Established convergence rates for rank and trimmed mean estimation.
AsylADMM improves gossip-based learning for non-smooth objectives.
problem Efficient and robust decentralized learning on edge devices.
method Asynchronous gossip algorithm for non-smooth optimization.
result AsylADMM converges faster on non-smooth problems.
New gossip algorithms improve robustness of rank-based statistics in decentralized systems.
problem Ensuring robustness in decentralized AI and edge intelligence systems, especially in the presence of corrupted or adversarial data.
method Developed asynchronous gossip algorithms for computing rank-based statistics.
result First convergence rate bound for asynchronous gossip-based rank estimation.
Multiple gossip steps improve decentralized optimization convergence.
problem Efficiently optimizing large-scale machine learning models with limited communication.
method Integrates multiple gossip steps between gradient descent iterations in compressed decentralized optimization.
result Convergence to within ε of the optimal value for smooth non-convex objectives.
Continuized Nesterov acceleration accelerates stochastic gradient descent and gossip algorithms.
problem Improving the convergence rate of stochastic gradient descent and gossip algorithms.
method Introducing a continuized variant of Nesterov acceleration, which mixes variables continuously and takes gradient steps at random times.
result The continuized Nesterov acceleration achieves convergence rates similar to Nesterov's original acceleration but with random parameters.
This paper develops efficient federated learning and unlearning methods in Bayesian models.
problem Managing epistemic uncertainty and legal right to be forgotten in decentralized networks.
method Develops federated variational inference solutions based on decentralized local free energy minimization.
result Demonstrates efficient unlearning mechanisms in federated learning and unlearning.
Decentralized ranking consensus via gossip for robust and scalable systems.
problem Achieving reliable and resilient consensus on collective rankings in a decentralized setting.
method Random gossip communication for decentralized computation of global rankings.
result Robust and scalable consensus on collective rankings achieved through decentralized, local interactions.
We consider decentralized stochastic optimization with the objective function (e.g. data samples for machine learning task) being distributed over n machines that can only communicate to their neighbors on a fixed communication graph. To reduce the communication bottleneck, the nodes compress (e.g. quantize or sparsi…
Algorithm reduces regret in multi-agent bandits through gossiping.
problem Minimizing cumulative regret in decentralized multi-agent bandits.
method Insert-Eliminate algorithm with gossip communication.
result Regret significantly reduced with minimal collaboration.
Consider a network of agents connected by communication links, where each agent holds a real value. The gossip problem consists in estimating the average of the values diffused in the network in a distributed manner. We develop a method solving the gossip problem that depends only on the spectral dimension of the netwo…
This paper proposes a gossip-based algorithm for distributed bilevel optimization over networks.
problem Distributed bilevel optimization over networks with limited communication.
method Gossip-based distributed bilevel learning algorithm.
result Achieves optimal sample complexities for general and strongly convex objectives.
We address the issue of speeding up the training of convolutional neural networks by studying a distributed method adapted to stochastic gradient descent. Our parallel optimization setup uses several threads, each applying individual gradient descents on a local variable. We propose a new way of sharing information bet…
Multi-simulator training has contributed to the recent success of Deep Reinforcement Learning by stabilizing learning and allowing for higher training throughputs. We propose Gossip-based Actor-Learner Architectures (GALA) where several actor-learners (such as A2C agents) are organized in a peer-to-peer communication t…
Unified framework for Byzantine robust gossip algorithms with guaranteed performance.
problem Vulnerability of decentralized machine learning to misbehaving devices.
method Introduces F-RG framework and CS+ robust aggregation rule for Byzantine resilience.
result CS+-RG has near-optimal breakdown tolerance and outperforms existing methods.
We analyse the learning performance of Distributed Gradient Descent in the context of multi-agent decentralised non-parametric regression with the square loss function when i.i.d. samples are assigned to agents. We show that if agents hold sufficiently many samples with respect to the network size, then Distributed Gra…
We address the issue of speeding up the training of convolutional networks. Here we study a distributed method adapted to stochastic gradient descent (SGD). The parallel optimization setup uses several threads, each applying individual gradient descents on a local variable. We propose a new way to share information bet…
In the era of big data, an important weapon in a machine learning researcher's arsenal is a scalable Support Vector Machine (SVM) algorithm. SVMs are extensively used for solving classification problems. Traditional algorithms for learning SVMs often scale super linearly with training set size which becomes infeasible …
Paper tightens lower bounds on decentralized training complexity.
problem Understanding and optimizing iteration complexity in decentralized training.
method Proved a tight lower bound on iteration complexity and proposed DeTAG algorithm.
result DeTAG achieves the theoretical lower bound with only a logarithmic gap.
The emerging concern about data privacy and security has motivated the proposal of federated learning, which allows nodes to only synchronize the locally-trained models instead their own original data. Conventional federated learning architecture, inherited from the parameter server design, relies on highly centralized…
GOAT learns multiple node representations from graph structure alone.
problem Context-free graph representation learning limits model performance.
method Inspired by gossip and mutual attention, GOAT learns multiple node representations.
result GOAT outperforms 12 SOTA baselines on link prediction and clustering tasks.
Efficient and robust algorithms for decentralized estimation in networks are essential to many distributed systems. Whereas distributed estimation of sample mean statistics has been the subject of a good deal of attention, computation of U-statistics, relying on more expensive averaging over pairs of observations, is…
In decentralized networks (of sensors, connected objects, etc.), there is an important need for efficient algorithms to optimize a global cost function, for instance to learn a global model from the local data collected by each computing unit. In this paper, we address the problem of decentralized minimization of pairw…
A new approach corrects bias in federated learning due to varying communication links.
problem Bias in federated learning due to non-uniform and time-varying communication links.
method Proposes Federated Postponed Broadcast (FedPBC) to correct bias in Federated Average (FedAvg).
result FedPBC converges to a stationary point of the global objective, overcoming bias caused by varying communication links.
In this paper, we determine the optimal convergence rates for strongly convex and smooth distributed optimization in two settings: centralized and decentralized communications over a network. For centralized (i.e. master/slave) algorithms, we show that distributing Nesterov's accelerated gradient descent is optimal and…
Novel algorithm for decentralized optimization in time-varying networks with delays.
problem Decentralized optimization in networks with communication delays.
method DT-GO algorithm, applicable to general directed graphs, converges to same complexity as centralized SGD.
result Algorithm DT-GO achieves convergence rates for convex and non-convex objectives, similar to centralized SGD.
Bayesian algorithms minimize cumulative regret in decentralized multi-agent bandits.
problem Minimizing cumulative regret in a decentralized multi-agent multi-armed bandit problem.
method Proposed decentralized Bayesian multi-armed bandit framework, including Thompson Sampling and Bayes-UCB algorithms.
result Regret scales logarithmically with constants matching those of an optimal centralized agent.
RelaySum improves decentralized deep learning by uniformly distributing data across workers.
problem Handling data heterogeneity in decentralized deep learning.
method RelaySum uses spanning trees to distribute information exactly uniformly across all workers with finite delays.
result RelaySum is independent of data heterogeneity and scales to many workers, enabling highly accurate decentralized deep learning.
In this paper, we introduce a distributed version of the classical stochastic Multi-Arm Bandit (MAB) problem. Our setting consists of a large number of agents n that collaboratively and simultaneously solve the same instance of K armed MAB to minimize the average cumulative regret over all agents. The agents can co…
Maker Protocol manages Dai stable coin on Ethereum blockchain.
problem Managing decentralized finance applications on blockchain.
method Analyzes Maker Protocol's components and governance.
result Maker Protocol is a significant decentralized finance application.
Compound Finance optimizes risk metrics for V3 protocol using Chainrisk simulations.
problem Optimizing systemic risks in Compound V3 protocol.
method Millions of Chainrisk simulations to evaluate VaR and LaR, providing parameter adjustments.
result Optimization framework enhances protocol stability.
This paper examines interest rates and market efficiency in DeFi loanable funds protocols.
problem Equilibrium of supply and demand for loanable funds in DeFi protocols.
method Review of interest rate mechanisms in Compound, Aave, and dYdX; empirical analysis of market efficiency and inter-connectedness.
result Interest rate rules in DeFi protocols do not always equilibrate supply and demand.
We study the power of interactivity in local differential privacy. First, we focus on the difference between fully interactive and sequentially interactive protocols. Sequentially interactive protocols may query users adaptively in sequence, but they cannot return to previously queried users. The vast majority of exist…
New federated learning protocols resist Byzantine failures and offer privacy guarantees.
problem Resisting Byzantine failures in federated learning.
method Proposes robust federated learning protocols with optimal statistical rates and privacy guarantees.
result Achieves nearly optimal statistical rates and tight rate in terms of all parameters for strongly convex losses.
The paper analyzes how mutable blockchain protocols affect miner behavior and strategic stability.
problem The mutability of blockchain protocols undermines long-term planning and cooperative equilibria.
method Integrates Austrian capital theory with repeated game theory to examine miner behavior under different institutional conditions.
result Effective time preference increases when protocol rules are mutable, leading to political rent-seeking and undermining strategic coherence.
New algorithm for decentralized online convex optimization with unknown delays.
problem Decentralized online convex optimization with unknown, time-varying feedback delays.
method Proposes a novel algorithm that incorporates adaptive learning rates and gossip-based delay estimation.
result Achieves improved regret bounds of O(N √d tot + N √T (1-σ^2)^(1/4)) and O(N δmax ln T α) for different settings.
This paper examines ADMM for network averaging, revealing its convergence rates and network topology impacts.
problem Efficiently averaging local information over a network using ADMM.
method Comparative analysis of ADMM and other algorithms on a canonical distributed averaging problem.
result Characterization of ADMM convergence and optimal parameter tuning based on network spectral properties.
Protocol minimizes disclosure in classification tasks.
problem Ensuring minimal disclosure in classification protocols.
method Developed a protocol for multi-party classification that minimizes non-responsive document disclosure.
result Guarantees minimal disclosure of non-responsive documents.
This study measures liquidity risks in Aave, a blockchain lending protocol.
problem Liquidity risks in lending protocols, especially in Aave.
method Measurements of liquidity risks using Aave as a case study, focusing on available liquidity and market concentration.
result Liquidity risks in Aave are volatile and affect the protocol negatively, especially for repeat borrowers.
Paper presents a risk management framework for blockchain protocols.
problem Blockchain protocol risks affecting DLT and digital assets.
method Developed a comprehensive risk management framework using traditional taxonomy.
result Structured approach to identify, measure, monitor and report blockchain protocol risks.
Study analyzes risk management in Aave and Compound lending protocols, finding v3 better than v2.
problem Risk management in decentralized lending protocols.
method Cross-version and cross-chain analysis using fixed effects model.
result v3 protocols have better risk management, with stronger impact on L2 blockchains.
Paper explores how to design federated learning protocols that benefit all participants while maintaining privacy.
problem Privacy concerns undermine the accuracy benefits of federated learning in privacy-sensitive domains.
method The paper provides conditions for mutually beneficial federated learning protocols and designs protocols that maximize total utility and accuracy.
result The paper demonstrates that federated learning can be designed to be mutually beneficial, striking a balance between privacy and model accuracy.
FEET protocol evaluates foundation models across three scenarios.
problem Lack of standardized evaluation protocols for foundation models.
method Structured evaluation protocol across three use cases: frozen, few-shot, and fine-tuned embeddings.
result Demonstrates comprehensive assessment of foundation models' effectiveness.
Stochastic gradient descent is a simple approach to find the local minima of a cost function whose evaluations are corrupted by noise. In this paper, we develop a procedure extending stochastic gradient descent algorithms to the case where the function is defined on a Riemannian manifold. We prove that, as in the Eucli…
A scalable protocol for federated averaging with privacy and correctness guarantees.
problem Privacy and correctness in federated learning from multiple parties.
method Scalable protocol using correlated and independent Gaussian noise, analyzed for differential privacy and graph topology.
result Nearly matches trusted curator model's utility with minimal communication.
Survey of yield farming protocols in DeFi.
problem Understanding and evaluating yield farming mechanisms in DeFi.
method Analyzed smart contracts, performed simulations, reviewed literature.
result Characterized major yield aggregators and identified risks.