Unified approach to aggregating models and preferences.
problem Consistent aggregation of models and preferences.
method Formal definition and weighted averaging of models and preferences.
result All rational aggregation rules are weighted averages of highest-ranked models/experts.
In order to scale standard Gaussian process (GP) regression to large-scale datasets, aggregation models employ factorized training process and then combine predictions from distributed experts. The state-of-the-art aggregation models, however, either provide inconsistent predictions or require time-consuming aggregatio…
Federated learning is a distributed framework for training machine learning models over the data residing at mobile devices, while protecting the privacy of individual users. A major bottleneck in scaling federated learning to a large number of users is the overhead of secure model aggregation across many users. In par…
U-aggregation combines multiple models without labels for better risk prediction.
problem Challenges in selecting best model for new populations due to limited data and lack of true labels.
method U-aggregation, an unsupervised model aggregation method that integrates pre-trained models without observed labels.
result U-aggregation improves genetic risk prediction of complex traits using publicly available models.
Estimates variance function using aggregation methods in regression models.
problem Estimating variance function in regression models.
method Two-step procedure involving model selection or convex aggregation, using two independent samples.
result Consistency of the proposed method in L2 error for MS and C aggregations.
MEVA aggregates model predictions to improve accuracy without needing model details.
problem Improving model accuracy by combining multiple models.
method Non-intrusive, data-driven framework that treats models as black boxes and optimizes aggregation methods.
result MVA outperforms MEA in estimating aggregated predictions, enhancing robustness and accuracy.
New method aggregates nodes in sparse graphical models.
problem Estimating edge-sparse graphical models.
method Tree-aggregated graphical lasso (tag-lasso) method.
result Aggregates nodes in a data-driven fashion using a tree.
SACP aggregates nonconformity scores from multiple predictors to create more efficient uncertainty sets.
problem Combining predictive uncertainties from multiple models for efficient and reliable uncertainty quantification.
method SACP (Symmetric Aggregated Conformal Prediction) aggregates nonconformity scores using a flexible symmetric aggregation function.
result SACP consistently improves efficiency and often outperforms state-of-the-art model aggregation baselines.
New algorithm for aggregate inference in HMMs with continuous observations.
problem Inference in large populations with indistinguishable individuals and continuous measurements.
method Continuous observation collective forward-backward algorithm extending existing discrete case algorithm.
result Efficacy demonstrated through numerical experiments.
Study improves fair opinion aggregation by balancing voter attributes.
problem Aggregation of opinions can be biased by voter attributes.
method Combines majority voting and D&S model with fairness options.
result Effective combination of Soft D&S and fairness options for different data types.
Paper analyzes sparse aggregation in GLMs with Kullback-Leibler risk bounds.
problem Sparse aggregation in GLMs for parameter approximation.
method Exponential weighted aggregation scheme with Kullback-Leibler risk bounds.
result Sharp oracle inequality for Kullback-Leibler risk with leading constant 1 and minimax-optimal rate of aggregation.
BOA improves financial forecasting by combining expert models.
problem Challenges in choosing between multiple machine learning models for financial forecasting.
method Online aggregation of expert models using Bernstein Online Aggregation (BOA) procedure.
result BOA leads to better portfolio performance, higher Sharpe Ratio, and lower shortfall.
The paper proposes criteria and methods for evaluating and aggregating feature-based model explanations.
problem Lack of quantitative evaluation criteria for feature-based model explanations.
method Developed quantitative evaluation criteria (low sensitivity, high faithfulness, low complexity), devised a framework for aggregation, and derived a new aggregate Shapley value explanation function.
result A new aggregate Shapley value explanation function that minimizes sensitivity.
Model infers functions for attributes using multi-aggregate datasets with knowledge transfer.
problem Modeling aggregate data with varying granularities and spatial supports.
method Multi-output Gaussian process (MoGP) with linear mixing of independent latent GPs, aggregation process, and prior distribution of mixing weights.
result Proposed model outperforms in refining coarse-grained aggregate data.
New federated learning algorithms improve model aggregation robustness.
problem Improving model aggregation in federated learning.
method Complete mathematical convergence analysis and novel aggregation algorithms.
result Derived novel algorithms that modify model architecture based on client contributions.
This paper solves aggregation of Pareto optimal models by using Bayesian priors and weighted averaging.
problem How to rationally aggregate Pareto optimal models while preserving Pareto efficiency.
method Four logical steps: 1) Bayesian models, 2) Prior as preference ranking, 3) Consistent aggregation, 4) Weighted average of priors.
result All rational/consistent aggregation rules follow a generalized hierarchical Bayesian model.
PNA improves GNNs for graph data with multiple aggregators.
problem Capturing continuous features in graph neural networks.
method Combines multiple aggregators with degree-scalers.
result PNA outperforms existing models on graph theory and real-world tasks.
Aggregates predictions from multiple regression models using random projections and kernel methods.
problem Combining predictions from multiple regression models to improve accuracy.
method Random projection of high-dimensional feature space, followed by kernel-based consensual aggregation.
result The aggregation scheme performs similarly to using the original high-dimensional features, with high probability.
A new algorithm estimates aggregate marginals from noisy data in an online manner.
problem Estimating aggregate marginals of a Markov chain from noisy aggregate observations.
method Sliding window Sinkhorn belief propagation (SW-SBP) algorithm.
result Demonstrated improved performance on inferring population flow.
Aggregates probability models using Wasserstein space and variational approach.
problem Model aggregation in the Wasserstein space of distributions.
method Data-driven calibration framework based on Γ-convergence. result Empirical minimizers converge to the minimizers of the actual problem.
Federated edge learning improves with CSIT-free model aggregation using RIS.
problem Lack of CSIT in federated edge learning systems.
method Use RIS to align channel coefficients for model aggregation without CSIT, optimize RIS and receiver jointly.
result Achieves similar learning accuracy as CSIT-based methods without CSIT.
Federated learning is the centralized training of statistical models from decentralized data on mobile devices while preserving the privacy of each device. We present a robust aggregation approach to make federated learning robust to settings when a fraction of the devices may be sending corrupted updates to the server…
Early-stopped aggregation improves computational efficiency in adaptive statistical inference.
problem Computational inefficiency in model selection and aggregation for adaptive inference.
method Early-stopped aggregation (ESA) framework: compute only a few simpler models using early-stopping criteria.
result ESA achieves optimal adaptive contraction rates in variational Bayes and frequentist settings.
Aggregation defenses improve deep learning models' robustness against data poisoning attacks.
problem Data poisoning attacks manipulate deep learning models with malicious training samples.
method Deep Partition Aggregation, efficiency improvements, data-to-complexity ratio, poisoning overfitting phenomenon.
result Aggregation defenses boost poisoning robustness through the poisoning overfitting phenomenon.
STAG injects noise into graph neural networks to improve performance.
problem Graph neural networks suffer from over-smoothing and limited discrimination.
method Introduces a stochastic aggregation framework (STAG) with adaptive noise injection.
result STAG models correct both over-smoothing and discrimination issues.
State aggregation is a popular model reduction method rooted in optimal control. It reduces the complexity of engineering systems by mapping the system's states into a small number of meta-states. The choice of aggregation map often depends on the data analysts' knowledge and is largely ad hoc. In this paper, we propos…
The paper quantifies uncertainty in aggregated machine learning metrics.
problem Uncertainty in summarizing model performance across multiple tasks.
method Statistical methodologies including bootstrapping and Bayesian modeling.
result Insights into model performance dominance for specific tasks.
Given a finite family of functions, the goal of model selection aggregation is to construct a procedure that mimics the function from this family that is the closest to an unknown regression function. More precisely, we consider a general regression model with fixed design and measure the distance between functions by …
Relational probabilistic models have the challenge of aggregation, where one variable depends on a population of other variables. Consider the problem of predicting gender from movie ratings; this is challenging because the number of movies per user and users per movie can vary greatly. Surprisingly, aggregation is not…
FedNNNN improves FL by adjusting model update vector norms.
problem Slow convergence and low prediction accuracy in FL.
method Norm-Normalized Neural Network Aggregation (FedNNNN) with momentum control.
result Up to 5.4% accuracy improvement on multiple datasets.
Improved local explainer aggregation for interpretable machine learning models.
problem Improving the interpretability of black box machine learning models.
method Non-convex optimization and integer optimization framework for local explainer aggregation.
result Our method outperforms existing methods in terms of coverage and fidelity, particularly in multi-class settings.
LightSecAgg reduces secure aggregation complexity in FL.
problem Complexity in secure aggregation protocols for FL systems.
method One-shot aggregate-mask reconstruction via mask encoding/decoding.
result Significantly reduces overhead for resiliency against dropped users.
This paper compares two loss functions for learning from aggregated responses and introduces an interpolating estimator.
problem Learning from aggregated responses in privacy-sensitive settings.
method Investigates bag-level and instance-level loss functions, and introduces an interpolating estimator.
result Instance-level loss can be seen as a regularized form of bag-level loss, leading to improved estimators.
Revises GNN neighborhood aggregation for more accurate node classification.
problem Flaws in benchmark GNN models for node classification.
method Statistical signal processing approach to neighborhood aggregation.
result Novel insights for designing more efficient GNN models.
A key aspect of Federated Learning (FL) is the requirement of a centralized aggregator to maintain and update the global model. However, in many cases orchestrating a centralized aggregator might be infeasible due to numerous operational constraints. In this paper, we introduce BAFFLE, an aggregator free, blockchain dr…
The paper examines risk aggregation under mixtures of marginals, finding that more homogeneous distributions lead to larger uncertainty.
problem Investigating the impact of mixing on risk aggregation uncertainty.
method Analyzes ordering relations and inequalities for aggregation sets under distribution and quantile mixtures.
result More homogeneous marginals result in larger aggregation sets, indicating greater model uncertainty.
Paper studies fundamental limits of communication in distributed learning.
problem Communication efficiency in model aggregation for distributed learning.
method Rate-Distortion approach to model aggregation as a vector Gaussian CEO problem.
result Derives rate region bound and sum-rate-distortion function for model aggregation.
A new method for faster prediction in distributed Gaussian processes.
problem Inefficient aggregation of distributed Gaussian processes with correlations.
method Proposes a novel approach for aggregated prediction in distributed GPs that incorporates correlations among experts.
result Results in more stable predictions in less time compared to state-of-the-art methods.
Federated Learning enables mobile devices to collaboratively learn a shared inference model while keeping all the training data on a user's device, decoupling the ability to do machine learning from the need to store the data in the cloud. Existing work on federated learning with limited communication demonstrates how …
The paper uses tensor decompositions to improve neural network models for tree data.
problem Encoding structural knowledge from tree-structured data efficiently.
method Introduces new aggregation functions using Canonical and Tensor-Train decompositions.
result Proposed models outperform traditional methods on tree classification tasks.
Improved defense against data poisoning attacks by aggregating smaller subsets.
problem Mitigating the impact of poisoned data on model robustness.
method Finite Aggregation method that combines duplicates of smaller disjoint subsets for training.
result Consistent improvement in certified robustness bounds, up to 4.77% on GTSRB.
FedBE aggregates local models into a robust global model via Bayesian inference.
problem Challenges in aggregating non-i.i.d. local models into a global model in federated learning.
method FedBE uses Bayesian inference to sample and combine higher-quality global models from local models.
result FedBE leads to more robust aggregation of local models into a global model, especially when data is non-i.i.d.
Productivity and credit limits affect aggregate production in non-monotonic ways.
problem Understanding how aggregate production is influenced by individual characteristics and financial constraints.
method Analytical proof of non-monotonic effects of productivity and credit limits on aggregate production in a general equilibrium model.
result Equilibrium aggregate production can be non-monotonic in both individual productivity and credit limit.
New aggregation methods improve robustness and efficiency in distributed learning.
problem Outliers and malicious agents compromise traditional averaging in distributed learning.
method Developed statistically efficient and robust aggregation schemes based on median and trimmed mean variations.
result Achieved higher sample efficiency compared to traditional robust aggregation schemes.
CARE improves LLM aggregation by accounting for shared confounders.
problem LLM judges' correlated errors due to shared latent confounders.
method CARE explicitly models judges' scores as true quality and confounders.
result CARE reduces aggregation error by up to 26.8% across various benchmarks.
Policy-GNN optimizes GNN aggregation for diverse node iterations.
problem Optimizing GNN performance by varying aggregation iterations for different nodes.
method Policy-GNN uses a meta-policy framework with deep reinforcement learning to adaptively determine the number of aggregations for each node.
result Policy-GNN significantly outperforms state-of-the-art alternatives on real-world datasets.
In order to improve forecasts, a decisionmaker often combines probabilities given by various sources, such as human experts and machine learning classifiers. When few training data are available, aggregation can be improved by incorporating prior knowledge about the event being forecasted and about salient properties o…
Partial fusion combines neural networks to balance accuracy and efficiency.
problem Balancing accuracy and computational cost in neural networks.
method Extending weight aggregation methods based on neuron-level similarity, using partial optimal transport to match similar neurons.
result Achieves a flexible tradeoff between computational cost and performance.