Turbo-Aggregate reduces secure aggregation time from quadratic to nearly linear.
problem Quadratic overhead in secure model aggregation for federated learning.
method Multi-group circular strategy, additive secret sharing, and coding techniques.
result Achieves O ( N log N ) O(N\log{N}) O ( N log N ) overhead, compared to O ( N 2 ) O(N^2) O ( N 2 ) , for up to 50% user dropout. This paper improves federated learning efficiency by auto-tuning secure aggregation parameters.
problem Efficient communication in federated learning with quantized updates.
method Auto-tuning secure aggregation parameters based on random rotation properties.
result Improved communication efficiency in federated learning with secure aggregation.
BAFFLE uses blockchain to federate learning without a central aggregator.
problem Infeasibility of orchestrating a centralized aggregator in federated learning.
method Blockchain-driven, decentralized coordination via Smart Contracts.
result Significant reduction in gas costs and scalability for federated learning on blockchain.
Fishnets improve set and graph learning with scalable, robust aggregation.
problem Learning informative embeddings for sets and graphs with scalable and robust aggregation.
method Proposes Fishnets, a new aggregation strategy for set-based learning.
result Fishnets achieve state-of-the-art performance on graph datasets with fewer parameters and faster training.
Non-affine aggregation rules cannot preserve monotonicity in convex learning.
problem Designing non-affine aggregation rules that maintain monotonicity in convex learning.
method Proving that monotonicity of aggregated gradients is preserved only if the aggregation rule is positively affine.
result Non-affine aggregation prevents steady convergence and substantially degrades algorithmic stability.
A robust aggregation method improves federated learning's accuracy in corrupted settings.
problem Making federated learning robust to corrupted updates from devices.
method Robust aggregation oracle based on geometric median for constant iterations of non-robust averaging.
result The robust aggregation oracle outperforms classical methods in high corruption levels.
We consider the forecast aggregation problem in repeated settings, where the forecasts are done on a binary event. At each period multiple experts provide forecasts about an event. The goal of the aggregator is to aggregate those forecasts into a subjective accurate forecast. We assume that experts are Bayesian; namely…
Federated learning is protected against adversarial attacks with residual-based reweighting.
problem Adversarial attacks on federated learning's aggregation process.
method Residual-based reweighting combined with iteratively reweighted least squares.
result Our aggregation algorithm outperforms other methods in label-flipping and backdoor attacks.
New federated learning algorithms improve model aggregation robustness.
problem Improving model aggregation in federated learning.
method Complete mathematical convergence analysis and novel aggregation algorithms.
result Derived novel algorithms that modify model architecture based on client contributions.
BOA improves financial forecasting by combining expert models.
problem Challenges in choosing between multiple machine learning models for financial forecasting.
method Online aggregation of expert models using Bernstein Online Aggregation (BOA) procedure.
result BOA leads to better portfolio performance, higher Sharpe Ratio, and lower shortfall.
Optimized Q-learning reduces regret in state-aggregated MDPs.
problem Reducing regret in reinforcement learning with state aggregation.
method Optimistic Q-learning applied to fixed-horizon episodic MDPs with aggregated states.
result Regret bound of i l d e O ( H 5 M K + ε H K ) ilde{\mathcal{O}}(\sqrt{H^5 M K} + εHK) i l d e O ( H 5 M K + ε H K ) , independent of states and actions. Adaptive Bayesian learning aggregates experts to improve performance.
problem Bayesian online learning's performance depends on inferential choices.
method Treat Bayesian update rules as experts and aggregate them based on sequential predictive losses.
result The aggregate competes with the best expert in hindsight at a low aggregation cost.
Recently, it has been shown that many functions on sets can be represented by sum decompositions. These decompositons easily lend themselves to neural approximations, extending the applicability of neural nets to set-valued inputs---Deep Set learning. This work investigates a core component of Deep Set architecture: ag…
We propose a clustering-based iterative algorithm to solve certain optimization problems in machine learning, where we start the algorithm by aggregating the original data, solving the problem on aggregated data, and then in subsequent steps gradually disaggregate the aggregated data. We apply the algorithm to common m…
Federated edge learning improves with CSIT-free model aggregation using RIS.
problem Lack of CSIT in federated edge learning systems.
method Use RIS to align channel coefficients for model aggregation without CSIT, optimize RIS and receiver jointly.
result Achieves similar learning accuracy as CSIT-based methods without CSIT.
MEVA aggregates model predictions to improve accuracy without needing model details.
problem Improving model accuracy by combining multiple models.
method Non-intrusive, data-driven framework that treats models as black boxes and optimizes aggregation methods.
result MVA outperforms MEA in estimating aggregated predictions, enhancing robustness and accuracy.
This work analyzes two methods for combining multiple binary labels in bipartite ranking.
problem Combining multiple binary labels for optimal bipartite ranking.
method Loss aggregation vs. label aggregation approaches.
result Label aggregation is preferable to loss aggregation due to label dictatorship issues.
New aggregation methods improve robustness and efficiency in distributed learning.
problem Outliers and malicious agents compromise traditional averaging in distributed learning.
method Developed statistically efficient and robust aggregation schemes based on median and trimmed mean variations.
result Achieved higher sample efficiency compared to traditional robust aggregation schemes.
FedNNNN improves FL by adjusting model update vector norms.
problem Slow convergence and low prediction accuracy in FL.
method Norm-Normalized Neural Network Aggregation (FedNNNN) with momentum control.
result Up to 5.4% accuracy improvement on multiple datasets.
U-aggregation combines multiple models without labels for better risk prediction.
problem Challenges in selecting best model for new populations due to limited data and lack of true labels.
method U-aggregation, an unsupervised model aggregation method that integrates pre-trained models without observed labels.
result U-aggregation improves genetic risk prediction of complex traits using publicly available models.
Aggregation defenses improve deep learning models' robustness against data poisoning attacks.
problem Data poisoning attacks manipulate deep learning models with malicious training samples.
method Deep Partition Aggregation, efficiency improvements, data-to-complexity ratio, poisoning overfitting phenomenon.
result Aggregation defenses boost poisoning robustness through the poisoning overfitting phenomenon.
This paper compares two loss functions for learning from aggregated responses and introduces an interpolating estimator.
problem Learning from aggregated responses in privacy-sensitive settings.
method Investigates bag-level and instance-level loss functions, and introduces an interpolating estimator.
result Instance-level loss can be seen as a regularized form of bag-level loss, leading to improved estimators.
GmCN adapts GCN feature aggregation to improve graph learning.
problem Fixed neighborhood graph aggregation in GCNs is biased and can be affected by graph structure noises.
method GmCN allows nodes to adaptively select optimal neighbors for feature aggregation.
result GmCN improves graph learning effectiveness through adaptive neighbor selection.
Proposes a novel MTL approach based on bias-variance analysis.
problem Improving multi-task learning performance through shared knowledge.
method Two-phase iterative aggregation of targets and features using bias-variance analysis.
result Validation on synthetic and real-world datasets demonstrates the effectiveness of the proposed method.
Proposes a method for private aggregation in heterogeneous federated learning.
problem Ensuring resilience to Byzantine clients and maintaining client data privacy in federated learning with heterogeneous data.
method Careful co-design of verifiable secret sharing, secure aggregation, and private information retrieval scheme.
result Achieves information-theoretic privacy guarantees and Byzantine resilience under data heterogeneity.
The paper quantifies uncertainty in aggregated machine learning metrics.
problem Uncertainty in summarizing model performance across multiple tasks.
method Statistical methodologies including bootstrapping and Bayesian modeling.
result Insights into model performance dominance for specific tasks.
In this paper we discuss policy iteration methods for approximate solution of a finite-state discounted Markov decision problem, with a focus on feature-based aggregation methods and their connection with deep reinforcement learning schemes. We introduce features of the states of the original problem, and we formulate …
A new aggregation strategy improves GNN performance and learning dynamics.
problem Improving expressivity and learning dynamics of GNNs.
method Proposes a variance-preserving aggregation function (VPA) for GNNs.
result VPA leads to increased predictive performance and improved learning dynamics.
Policy-GNN optimizes GNN aggregation for diverse node iterations.
problem Optimizing GNN performance by varying aggregation iterations for different nodes.
method Policy-GNN uses a meta-policy framework with deep reinforcement learning to adaptively determine the number of aggregations for each node.
result Policy-GNN significantly outperforms state-of-the-art alternatives on real-world datasets.
New aggregation method improves GNN robustness to structural perturbations.
problem Graph Neural Networks (GNNs) are vulnerable to adversarial attacks that manipulate graph structure.
method Proposes a robust aggregation function with a breakdown point of 0.5, inspired by robust statistics.
result Improves GNN robustness by a factor of 3 on Cora ML and 5.5 on Citeseer, and 8 for low-degree nodes.
Improved aggregation methods learn from all ICU events without preprocessing for better patient risk analysis.
problem Lack of efficient methods to dynamically assess patient status in ICU.
method Improved aggregation methods for a deep learning architecture that learns from all events without preprocessing.
result Models achieve strong performance (AUROC 0.87) in patient mortality classification.
The paper compares aggregated data labels in curated and random bags for machine learning models.
problem Protecting user privacy in machine learning systems with aggregated data.
method Examined curated and random bags for training machine learning models and compared their performance.
result Gradient-based learning can be performed on aggregated data without performance degradation.
Study aggregation of statistical evidence under unknown dependence using group-invariance.
problem Aggregating statistical evidence under unknown and complex dependence structures.
method Develops a framework using group-invariance and permutation-based constructions to aggregate evidence across transformed datasets.
result Shows uniform improvement in critical values for single-batch aggregation over deterministic calibrations, adapting to unknown dependence structures.
Aggregates models from different datasets using shared latent structures.
problem Aggregating models from heterogeneous datasets with shared latent structures.
method Bayesian nonparametrics for identifying correspondences among local model parameterizations.
result Framework successfully aggregates various model types across different applications.
Paper studies fundamental limits of communication in distributed learning.
problem Communication efficiency in model aggregation for distributed learning.
method Rate-Distortion approach to model aggregation as a vector Gaussian CEO problem.
result Derives rate region bound and sum-rate-distortion function for model aggregation.
Active learning method reduces labeling cost for regression models with aggregated data.
problem Reducing labeling cost for training regression models with aggregated data.
method Sequentially selects sets to be labeled using mutual information quantifying model parameter uncertainty.
result Achieves better predictive performance with fewer labeled sets.
Paper tackles learning win-win solutions in aggregation systems.
problem Learning approximate equilibrium solutions in aggregation systems to keep suppliers.
method Multi-Agent Reinforcement Learning (MARL) with variance reduction mechanism.
result Approach reduces value variance and moves joint solution closer to Nash Equilibrium.
Federated learning leaks participant dataset quality even with secure aggregation.
problem Leakage of participant dataset quality in federated learning with secure aggregation.
method Image recognition experiments to infer and attribute dataset quality.
result Relative quality ordering of participants can be inferred and used for various purposes.
We develop a secure aggregation protocol for federated learning that reduces communication and computation costs.
problem Expensive communication and privacy concerns in federated learning.
method Adapting compression-based federated techniques to additive secret sharing.
result Our protocol achieves high accuracy with low communication costs and is more efficient than prior work.
MULTIPOLAR aggregates diverse source policies for efficient transfer RL.
problem Efficiently transfer knowledge between different environmental dynamics.
method Adaptive action aggregation and residual prediction network.
result Demonstrated effectiveness across diverse simulated environments.
Improved local explainer aggregation for interpretable machine learning models.
problem Improving the interpretability of black box machine learning models.
method Non-convex optimization and integer optimization framework for local explainer aggregation.
result Our method outperforms existing methods in terms of coverage and fidelity, particularly in multi-class settings.
Paper analyzes sparse aggregation in GLMs with Kullback-Leibler risk bounds.
problem Sparse aggregation in GLMs for parameter approximation.
method Exponential weighted aggregation scheme with Kullback-Leibler risk bounds.
result Sharp oracle inequality for Kullback-Leibler risk with leading constant 1 and minimax-optimal rate of aggregation.
AEA dynamically aggregates ensemble targets for actor-critic learning.
problem Static ensemble aggregation methods struggle with overestimation bias and variance.
method Adaptive Ensemble Aggregation (AEA) dynamically constructs ensemble-based targets.
result AEA converges to optimal variance reduction and maximal Fisher information.
Study improves fair opinion aggregation by balancing voter attributes.
problem Aggregation of opinions can be biased by voter attributes.
method Combines majority voting and D&S model with fairness options.
result Effective combination of Soft D&S and fairness options for different data types.
Method learns dynamics from aggregate data without full trajectories.
problem Learning nonlinear dynamics from aggregate data where individual trajectories are not available.
method Combines weak Fokker-Planck Equation description with Wasserstein GAN.
result Successfully learns nonlinear dynamics from aggregate data.
A new procedure aggregates models to predict data from multiple clusters.
problem Predicting data from multiple clusters with different underlying models.
method Three-step procedure: clustering, model fitting, and aggregation.
result The method outperforms existing models in various prediction problems.
Boosting weak learners to strong ones from aggregate labels is possible for LLP but not for MIL.
problem Boosting weak learners to strong ones from aggregate labels in learning from label proportions (LLP).
method Using a weak learner on large enough bags to obtain a strong learner for small bags in polynomial time.
result Boosting is possible for LLP but not for MIL.
This work analyzes aggregation strategies for Bayesian deep learning models in federated learning.
problem Improper aggregation of Bayesian deep learning models in federated learning leads to sub-optimal performance.
method Six aggregation strategies for Bayesian deep learning models are analyzed using CIFAR-10 dataset and a fully variational ResNet-20 architecture.
result Aggregation strategy is a key hyperparameter affecting accuracy, calibration, uncertainty quantification, training stability, and client compute requirements.