Paper introduces SNEP for efficient Bayesian learning and posterior server for distributed data.
problem Efficient Bayesian learning with distributed data.
method SNEP (stochastic natural gradient expectation propagation) and posterior server architecture.
result SNEP converges and improves Bayesian learning efficiency.
Bayesian approach reduces FL communication cost by one-shot.
problem High communication cost in optimization-based FL for high-dimensional models.
method Bayesian pseudocoresets and function-space inference for one-shot FL.
result Achieves prediction performance competitive to state-of-the-art with up to 2 orders of magnitude reduction in communication cost.
EP-GFlowNets parallelize GFlowNet training for large-scale Bayesian inference.
problem Prohibitive repeated evaluations of unnormalized distributions for large-scale posterior sampling.
method Divide-and-conquer approach with server learning from local models.
result EP-GFlowNets enable efficient parallel and federated Bayesian inference.
Federated learning framework with uncertainty estimation.
problem Uncertainty estimation in federated learning models.
method Infers and distills posterior predictive distributions from clients to server.
result Improved accuracy and reliability in predictions and tasks like OOD detection.
Simplifies RCA for anomalies with separable likelihoods.
problem Challenges in accurate and friendly root cause analysis.
method Bayesian framework with separable likelihoods under certain restrictions.
result Framework successfully applied to web server error logs.
We study Bayesian optimal control of a general class of smoothly parameterized Markov decision problems. Since computing the optimal control is computationally expensive, we design an algorithm that trades off performance for computational efficiency. The algorithm is a lazy posterior sampling method that maintains a d…
New perspective on federated learning as posterior inference, improving optimization.
problem Optimizing global models in distributed learning settings.
method Formulated as posterior inference problem, using MCMC for approximate inference and federated averaging for refinement.
result Federated posterior averaging (FedPA) outperforms existing methods on benchmarks.
FMP sampling improves model calibration without sharing data.
problem Difficulties in specifying priors for modern neural networks.
method FMP sampling replaces prior and likelihood with a predictive distribution, running the sampler centrally.
result FMP sampling significantly improves calibration over baseline methods.
Improved speech recognition for voice assistants by analyzing speech data.
problem Reducing false triggers in speech-enabled assistants.
method Post-processing LVCSR hypothesis lattice with a Bidirectional Lattice Recurrent Neural Network (LatticeRNN).
result LatticeRNN significantly improves detection accuracy over traditional methods.
Lapse improves parameter servers by dynamically allocating parameters, achieving near-linear scaling.
problem Efficiently managing distributed training with reduced communication overhead.
method Integrate dynamic parameter allocation into parameter servers, proposing Lapse.
result Lapse provides near-linear scaling and can be orders of magnitude faster than existing parameter servers.
A new scheme for private computation splits client data into shares for server operations.
problem Private computation between client and server without revealing data.
method Stochastic scheme splitting client data into privatized shares.
result Server performs operations on privatized shares without learning raw data.
Efficient Bayesian FL method improves predictive accuracy and uncertainty estimates.
problem Federated Learning with model and predictive uncertainty and personalization.
method Second-order optimization approach for Bayesian FL.
result Improved predictive accuracies and uncertainty estimates.
New algorithm for federated learning without a central server.
problem Federated learning in unidirectional trust social networks.
method Central Server Free Federated Learning (OPS) method.
result Users can benefit from communication with trusted users in federated learning.
Paper speeds up DGD by allowing multiple transmissions from non-straggling servers.
problem Limited parallel execution due to straggling servers in DGD.
method Allowing multiple transmissions from each CS at each iteration.
result Significant reduction in average completion time per iteration.
Proposes a Bayesian federated learning method for diverse tasks.
problem Current federated learning approaches focus on homogeneous tasks, ignoring task diversity.
method Integrates multi-task learning with MOGP at the local level and federated learning at the global level.
result Demonstrates superior predictive performance and uncertainty calibration on diverse tasks.
Algorithm stabilizes queues in asymmetric systems with unknown service rates.
problem Stabilizing queues in multi-class multi-server systems with unknown service rates.
method Proposes UCB and Thompson Sampling algorithms to stabilize queues while learning service rates.
result Achieves system stability with an average queue length bound of \(O(\min\{N,K\}/ε)\) for large time horizon \(T\).
Corella protects client data privacy in multi-server learning with correlated queries.
problem Protecting client data privacy in multi-server machine learning.
method Proposes a private multi-server learning approach using correlated queries and strong noise.
result Mitigates client data leakage with high accuracy and minimal computational effort.
LiuBei is a resilient ML algorithm that tolerates Byzantine workers and servers without trusting any component.
problem Byzantine failures in distributed ML solutions.
method Byzantine-resilient ML algorithm that aggregates gradients and replicates parameter servers, using a filtering mechanism and scatter/gather protocol.
result LiuBei achieves Byzantine resilience to both servers and workers and guarantees convergence, with an accuracy loss of around 5% and a 24% convergence overhead.
New method speeds up distributed linear regression.
problem Efficiently solve distributed linear regression problems.
method Iteratively Pre-conditioned Stochastic Gradient Descent (IPSG)
result Converges linearly in expectation to the solution.
A new method combines MCMC results to avoid failures in parallel computing.
problem Parallel MCMC's sensitivity to subposterior sampling issues leads to failures.
method Parallel Active Inference (PAI) uses Gaussian Process (GP) surrogate modeling and active learning.
result PAI successfully combines MCMC results where previous methods fail.
Paper introduces an asynchronous optimization algorithm for parameter servers.
problem Solving optimization problems with asynchrony and general regularizers.
method Asynchronous incremental aggregated gradient algorithm in a parameter server framework.
result Established linear convergence rate and step-size choices for strong convex data loss.
Paper explores differential privacy in high-dimensional federated learning, tackling server trustworthiness and estimation.
problem Maintaining privacy in distributed environments with high-dimensional data.
method Investigates scenarios with untrusted and trusted central servers, introduces novel federated estimation algorithms for linear regression models.
result Tight minimax rates depend on high-dimensionality even with sparsity assumptions, and novel algorithms handle slight variations among distributed models.
Asynch-SGBDT speeds up GBDT training on Parameter Server.
problem Training GBDT on Parameter Server is slow due to computational resources and time.
method Developed asynch-SGBDT, an asynchronous parallel method for GBDT training.
result Asynch-SGBDT achieves linear speedup in asynchronous parallel training.
A smart device optimizes server selection for energy and latency in dynamic networks.
problem Optimizing server selection for mobile devices in edge computing with uncertainty and dynamic changes.
method Formulated as a budget-limited multi-armed bandit problem, a policy is proposed to minimize regret.
result The proposed method outperforms existing solutions in terms of energy and latency.
New robust aggregation rules for SGD in the presence of Byzantine failures.
problem Byzantine failures in distributed SGD.
method Three new aggregation rules for robust SGD under a general Byzantine failure model.
result Proven Byzantine resilience of the proposed aggregation rules.
New algorithm solves distributed ML with no trusted central server.
problem Tackles Byzantine failures in distributed machine learning.
method ByzSGD algorithm using Scatter/Gather, DMC, and MDA.
result Solves general Byzantine-resilient distributed ML problem.
CSE-FSL reduces communication and storage costs in federated learning.
problem High communication and storage costs in federated learning.
method CSE-FSL uses an auxiliary network to locally update client models and sends only selected epochs' smashed data.
result Significant communication reduction with state-of-the-art convergence and model accuracy.
DSVGD improves federated learning with fewer communication rounds.
problem Federated learning scalability and trustworthiness.
method Distributed Stein Variational Gradient Descent (DSVGD) for non-parametric Bayesian inference.
result DSVGD achieves comparable accuracy and scalability to other methods, with well-calibrated predictions.
A new privacy-preserving ML framework using random sampling.
problem Achieving differential privacy in client-distributed ML.
method Draw and Discard: random model sampling for scalability and privacy.
result Demonstrated improved model quality and privacy guarantees.
Enhanced federated learning reduces communication costs and improves model accuracy.
problem Reducing communication costs in federated learning.
method Asynchronous model update and temporally weighted aggregation.
result The proposed algorithm outperforms baseline in terms of communication cost and model accuracy.
Hybrid approach reduces computation time and decoding complexity.
problem Straggling servers in distributed computing.
method Coded partial gradient computation (CPGC) that balances gradient accuracy and completion time.
result Reduces both computation time and decoding complexity.
New method reduces FL communication cost by leveraging server's side information.
problem High communication cost in federated learning.
method Exploits server's side information to compress client updates.
result Up to 82 times smaller bitrate with higher accuracy.
Detects backdoors in outsourced models by replicating training steps across multiple servers.
problem Detecting backdoors in models trained on cloud providers without prior knowledge.
method Replicate training steps across multiple servers to identify deviations and malicious updates.
result 99.6% accuracy in identifying backdoored models out of 50% malicious providers.
Stanza separates convolutional and fully connected layers for faster deep learning training.
problem Heavy data transfer between workers and servers in distributed deep learning.
method Layer separation: most nodes train convolutional layers, others train fully connected layers only.
result Significant acceleration of training time (1.34x--13.9x) over current systems.
This paper improves privacy in federated learning without a trusted server.
problem Privacy in federated learning with silos that distrust each other.
method Introduces Inter-Silo Record-Level Differential Privacy (ISRL-DP) and accelerated algorithms for convex and smooth losses.
result Achieves optimal privacy and accuracy tradeoffs in federated learning.
BrainTorrent uses peer-to-peer FL for medical image segmentation without a central server.
problem Lack of sufficient annotated data for personalized medical models.
method Peer-to-peer federated learning framework without a central server.
result BrainTorrent outperforms traditional server-based FL and achieves similar performance to pooled data training.
This work provides bounds on generalization error and privacy leakage in federated learning.
problem Bounding generalization error and privacy leakage in federated learning.
method Information-theoretic framework for classical, distributed, and federated learning.
result Upper and lower bounds on generalization error and privacy leakage.
Paper tackles federated linear bandit learning with AirComp for noisy channels.
problem Minimize cumulative regret in federated linear bandit learning.
method Proposes a federated linear bandits scheme using over-the-air computation (AirComp) over noisy fading channels.
result Determines the regret bound of the proposed scheme.
Client-based machine learning uses mobile devices for computation, improving privacy and reducing data upload.
problem Exploiting mobile devices for machine learning tasks to protect privacy and reduce data upload.
method Leveraging local hardware and data on mobile devices for computation-intensive tasks, only uploading results.
result Client-based machine learning can relieve server burdens and protect user privacy.
Paper tackles nonparametric classification with privacy constraints, achieving optimal accuracy.
problem Nonparametric classification under distributed differential privacy constraints.
method Minimax and adaptive transfer learning, considering privacy, sample sizes, and heterogeneity.
result Developed an adaptive classifier achieving optimal misclassification rate with privacy constraints.
Paper proposes a GPU-based system for training massive deep learning models in ads systems.
problem Training massive deep learning models with terabyte-scale parameters in ads systems.
method Hierarchical GPU parameter server with 3-layer storage (GPU High-Bandwidth Memory, CPU main memory, SSD).
result 4-node hierarchical GPU parameter server trains a model 2X faster than a 150-node in-memory system.
DFedAvgM is a decentralized FedAvg with momentum for privacy and communication efficiency.
problem Efficiently train models with privacy and communication efficiency in federated learning.
method Decentralized Federated Averaging with Momentum (DFedAvgM) on clients connected by an undirected graph, using stochastic gradient descent with momentum and quantization.
result DFedAvgM converges under trivial assumptions and can be improved with the PŁ property, numerically verified.
Paper improves communication in distributed optimization, reducing worker-to-server data exchanges.
problem Efficiency in server-to-worker communication in distributed optimization.
method MARINA-P, a novel downlink compression method using correlated compressors; M3, combining MARINA-P with uplink compression.
result MARINA-P achieves provably superior server-to-worker communication complexity with increasing number of workers.
New strategies reduce FL's impact on client resources, enabling larger models and more users.
problem Communication bottleneck in Federated Learning on heterogeneous edge networks.
method Lossy compression and Federated Dropout to reduce client-to-server communication and local computation.
result Up to 14x reduction in server-to-client communication, 1.7x reduction in local computation, and 28x reduction in upload communication.
Byzantine-resilient federated learning with local iterations and robust mean estimation.
problem Byzantine clients disrupt federated learning with local iterations.
method Local SGD iterations, robust mean estimation, and matrix concentration result.
result Convergence analysis for strongly-convex and non-convex smooth objectives in heterogeneous data settings.
Hybrid-FL improves ML model accuracy in non-IID data environments.
problem Performance degradation in FL due to non-IID data.
method Hybrid-FL combines client and server learning, selecting optimal clients and data for aggregation.
result 13.5% higher classification accuracy than previous methods.
Mix2FLD improves FL accuracy with FD, reducing convergence time.
problem Uplink-downlink capacity asymmetry in federated learning.
method Two-way mixup of local samples and model parameters, preserving privacy.
result Achieves up to 16.7% higher test accuracy with reduced convergence time.
Paper studies IPG method's robustness in noisy distributed linear regression.
problem Distributed linear regression with noise.
method Iteratively Pre-conditioned Gradient-descent (IPG) method.
result IPG method's robustness against noise compared favorably to state-of-the-art algorithms.