Partial model averaging improves Federated Learning performance.
problem Periodic model averaging causes significant model discrepancy in Federated Learning.
method Proposes a partial model averaging framework that encourages local models to stay close to each other.
result Partial averaging achieves up to 2.2% higher validation accuracy than full averaging.
A new distributed SGD algorithm reduces communication by infrequent global reduction.
problem Reducing communication in large-scale machine learning training.
method Hierarchical averaging stochastic gradient descent (Hier-AVG) with infrequent global reduction.
result Hier-AVG achieves comparable training speed with better test accuracy.
Geometric proof for averaging theorem on Riemannian manifolds.
problem Averaging theorem for perturbed dynamical systems on Riemannian manifolds.
method Geometric proof using a free coordinate approach.
result Generalization of averaging procedure to any open domain with compact closure.
New perspective on federated learning as posterior inference, improving optimization.
problem Optimizing global models in distributed learning settings.
method Formulated as posterior inference problem, using MCMC for approximate inference and federated averaging for refinement.
result Federated posterior averaging (FedPA) outperforms existing methods on benchmarks.
Efficient algorithm for global optimization of multivariate Lipschitz functions.
problem Global optimization of multivariate Lipschitz continuous functions.
method Proposes an efficient minimax optimal algorithm using a predetermined query creation rule.
result Achieves an average regret bound of O ( L n T − 1 n ) O(L\sqrt{n}T^{-\frac{1}{n}}) O ( L n T − n 1 ) , minimax optimal. A new method averages neural network parameters to rank features robustly.
problem Neural networks' sensitivity to random initialization affects feature ranking robustness.
method Parameter averaging of multiple shallow networks trained with different random seeds.
result The averaged model discovers ground-truth feature importance consistently.
DSPI connects natural policy gradient to policy iteration, proving global convergence.
problem Optimizing policies in reinforcement learning.
method DSPI framework, combining smoothed policy iteration and natural policy gradient.
result DSPI achieves geometric convergence and optimal complexity for policy optimization.
PDA method optimizes neural networks with global convergence rate analysis.
problem Quantitative convergence rate for neural network optimization in mean field regime.
method Particle dual averaging (PDA) method, combining Langevin algorithm and outer loop optimization.
result Established quantitative global convergence for two-layer mean field neural networks.
Convolutional neural networks converge quickly with gradient descent.
problem Learning efficient image classifiers with over-parameterized networks.
method Gradient descent for training over-parametrized CNNs with global average-pooling.
result Gradient descent quickly reduces the misclassification risk of CNNs.
Researchers develop a method to measure treatment effects in settings with shared states.
problem Measuring treatment effects in settings with shared states like prices, recommendations, or social signals.
method Double machine learning (DML) theorem with conditions for efficient inference under shared-state interference.
result Efficient estimation of average direct effect (ADE) and global average treatment effect (GATE) in various models.
Federated learning technique improves convergence speed with communication delays.
problem Communication delays between edge nodes and aggregator in federated learning.
method Developed FedDelAvg, a technique that generalizes federated averaging to incorporate a weighting between current local model and delayed global model.
result FedDelAvg achieves a significant improvement in convergence speed, especially when optimizing the weighting scheme to account for delays.
Study on averaging models in parallel SGD for better convergence.
problem Understanding when and how averaging models in parallel SGD improves convergence.
method Examined two scenarios: convex and non-convex objectives, and analyzed the effect of averaging frequency on convergence.
result The benefit of frequent averaging depends on gradient variance and the presence of multiple optimal points for non-convex objectives.
New method uses LP to achieve optimal sample complexity in multi-agent reinforcement learning.
problem Achieving global optimality in multi-agent reinforcement learning with average-cost criterion.
method Randomized Linear Programming and Stochastic Primal-Dual Methods for multi-agent saddle point problems.
result Sample complexity matches tight dependencies on state and action spaces, and scales with network size.
Averaged SGD achieves optimal convergence rate for neural networks in the NTK regime.
problem Convergence analysis of averaged stochastic gradient descent for neural networks.
method Analyzed convergence of averaged stochastic gradient descent for overparameterized two-layer neural networks.
result Achieved minimax optimal convergence rate with global convergence guarantee.
New algorithm optimizes Hölder continuous functions efficiently.
problem Optimizing Hölder continuous multivariate functions.
method Uses a query creation rule for global optimization, avoiding proxy functions.
result Achieves an average regret bound of $O(T^{- racα{n}})$ for Hölder exponent α α α . Tensor regression networks improve neural network compression and regularization.
problem Improving neural network compression and regularization with low-rank tensor approximations.
method Investigating various low-rank tensor approximations in tensor regression networks.
result Tensor regression networks with Global Average Pooling layer outperformed in deep CNNs, while shallow CNNs with tensor regression and dropout achieved lower test error.
GWN improves multimodal data fusion accuracy for chronic pain patients.
problem Dynamic and unspecified uncertainties in multimodal data fusion.
method Inspired by Global Workspace Theory, GWN is a neural network architecture that dynamically attends to multiple modalities.
result GWN achieved higher F1 scores (0.92 and 0.75) for multimodal discrimination and classification tasks.
We introduce a new graph kernel combining local and global properties.
problem Graph kernels focusing on local properties often fail on large graphs.
method Weisfeiler-Lehman algorithm with stochastic approximation.
result Our kernel outperforms state-of-the-art on graph classification benchmarks.
Bayesian model predicts drip-line locations in heavy calcium isotopes.
problem Determining the neutron drip line in the Ca region of heavy nuclei.
method Bayesian model averaging with Gaussian-process-based extrapolations.
result Predicted posterior probabilities for drip-line locations in heavy calcium isotopes.
New gossip algorithms solve decentralized optimization of pairwise functions.
problem Efficiently optimize global cost functions in decentralized networks.
method Gossip dual averaging algorithms for synchronous and asynchronous settings.
result Preserves convergence rate of centralized dual averaging with an additive bias.
This study examined how the correlation and network structure of 30 global indices and 145 local Korean indices belonging to the KOSPI 200 have changed during the 13-year period, 2000-2012. The correlations among the indices were calculated. The results showed that although the average correlations of the global indice…
One-Shot Federated Learning learns a global model in one round.
problem Training a global model in federated learning networks.
method Ensemble learning and knowledge aggregation.
result Achieves 51.5% relative gain in AUC over local baselines.
New geometric metrics improve Bayesian optimization performance evaluation.
problem Current metrics lack geometric insights and cannot compare algorithms effectively.
method Proposed four geometric metrics: precision, recall, average degree, and average distance.
result Proposed metrics provide more detailed evaluation of Bayesian optimization.
Paper connects 3D gravity averages to 2D CFT correlators.
problem Understanding 3D gravity partition functions.
method 3D topological field theories and mapping class group averages.
result Established a correspondence between 3D gravity averages and 2D CFT correlators.
New framework improves restless bandit policies for large numbers of arms.
problem Efficiently compute policies for large numbers of arms in restless bandit problems.
method Follow-the-Virtual-Advice framework, converting single-armed policies to N-armed policies.
result Achieves an O(1/\sqrt{N}) optimality gap in both discrete and continuous settings.
We show that there are not pure C 5 \mathcal{C}^5 C 5 regular y-global Landsberg surfaced. The proof is based on the averaged connection associated with the linear Chern's connection and the classification of irreducibles holonomies of torsion-free affine connections. The structure consists on exausting all the possible case…
GSP improves global average pooling for deep metric learning by learning weights and selecting semantic entities.
problem Improving global average pooling for deep metric learning.
method Generalized Sum Pooling (GSP) method that learns weights and selects semantic entities.
result GSP improves metric learning performance on 4 popular benchmarks.
Paper proposes quantum methods for optimizing machine learning functions.
problem Machine learning optimization problem
method Average approach and Partial Swap Test Cut-off method (PSTC)
result Potential to improve PSTC to O ( ∣ Θ ∣ ⋅ s u b l i n e a r N ) O(\sqrt{|Θ|} \cdot sublinear \ N) O ( ∣Θ∣ ⋅ s u b l in e a r N ) Improved portfolio optimization method yields better risk-adjusted returns.
problem Optimizing global minimum variance portfolios with reduced risk.
method k-fold boosted k − k- k − BAHC covariance cleaning procedure for correlation matrices. result Our method outperforms other filtering methods in Sharpe ratios, despite higher turnover.
A new method normalizes flow mixtures for better inference across different data types.
problem Inference failure across diverse posterior geometries in normalizing flows.
method Introduces a two-stage framework with a stable global weighting mechanism based on sEMA.
result Achieves consistent NLL improvements and stable weight trajectories over baselines.
New methods optimize functions faster with less gradient accuracy needed.
problem Optimizing complex functions with limited gradient accuracy.
method Hessian averaging and adaptive gradient sampling methods.
result Improved convergence rates for various function types.
Study compares local and global models for hierarchical forecasting accuracy.
problem Challenges in hierarchical time series forecasting, especially in accuracy and information utilisation.
method Developed and evaluated local and global forecasting models (GFMs) to exploit cross-series and cross-hierarchies information.
result Global Forecasting Models (GFMs) outperform local models in hierarchical forecasting accuracy and computational efficiency.
Meta algorithm solves multivariate optimization using univariate optimizers.
problem Multivariate global optimization problems.
method Meta algorithm combining univariate global optimizers.
result Meta algorithm provides robust regret guarantees.
FedMA improves federated learning for neural nets, matching and averaging model elements.
problem Federated learning's communication burden and model performance.
method Layer-wise construction of shared model by matching and averaging hidden elements.
result FedMA outperforms state-of-the-art algorithms on deep CNN and LSTM architectures.
New method for estimating global and local parameters using regularized Riesz representers.
problem Estimating global and local parameters in complex models robustly.
method Adaptive inference methods based on ℓ1 regularization, including Riesz representer as a nuisance parameter.
result Non-asymptotic and asymptotic uniform validity for honest confidence bands.
This study optimizes model averaging for personalized collaborative learning.
problem Differences in data or objectives between nodes in federated learning.
method Weighted averaging between local and global models for scalar mean estimation.
result There is always some positive model averaging that reduces expected squared error.
FedAvg converges linearly to global minimum in federated learning with partial participation.
problem Challenges in federated learning with partial client participation.
method Federated averaging (FedAvg) method for over-parameterized neural networks.
result FedAvg converges to global minimum at a linear rate after t iterations.
This work tackles risk-sensitive deep RL by optimizing policies with variance constraints.
problem Risk and aleatoric uncertainty in deep reinforcement learning.
method Lagrangian and Fenchel dualities to transform the problem into an unconstrained saddle-point policy optimization problem, and an actor-critic algorithm to iteratively update policy, Lagrange multiplier, and Fenchel dual variable.
result The proposed actor-critic algorithm finds a globally optimal policy at a sublinear rate.
Proposes methods to recover labels from shuffled networks using graph averages.
problem Recovering labels from a shuffled network using graph averages.
method Cluster networks into classes, then match the new graph to cluster-averages, minimizing the graph matching objective function.
result Higher fidelity matching performance when clustering networks into different classes.
Introduces joint Shapley values to measure feature importance in models.
problem Measuring the importance of feature sets in machine learning models.
method Extends Shapley's axioms to measure a set of features' average contribution to a model's prediction.
result Joint Shapley values provide unique insights and are more consistent with local intuitions.
Novel approach combines local and global brain changes for AD prediction.
problem Detecting Alzheimer's disease through local and global brain changes.
method Patch-based 3D-CNNs combined with global topological features for multi-scale brain tissue connectivity.
result Average precision score of 0.95 for classifying cognitively normal subjects and AD patients (prevalence ~55%).
FGPR uses averaging and SGD for federated G P \mathcal{GP} G P regression, excelling in personalization and multi-fidelity modeling.
problem Privacy-preserving multi-fidelity data modeling and personalization.
method Federated Gaussian process framework with averaging and SGD for local computations.
result FGPR converges to a critical point of the full log-likelihood function, excels in personalization and multi-fidelity modeling.
A modified VDCNN model reduces size and latency for mobile platforms.
problem Memory and processing constraints on mobile platforms.
method Temporal Depthwise Separable Convolutions and Global Average Pooling.
result The squeezed model (SVDCNN) is 10x-20x smaller with minimal accuracy loss.
GEMS learns global models by intersecting local 'good-enough' models.
problem Aggregating data across multiple nodes with limited communication.
method GEMS framework that intersects 'good-enough' models from each node.
result GEMS improves model accuracy by 15 points on average.
Federated Learning can benefit from Model Agnostic Meta Learning for personalized models.
problem Personalizing models for diverse devices in Federated Learning.
method Interpreted Federated Averaging as a meta learning algorithm and applied MAML principles.
result Personalized models trained with Federated Averaging are easier to fine-tune and achieve higher accuracy.
A new system learns entity representations to improve local entity disambiguation.
problem Local entity disambiguation in text.
method Entity-ELMo (E-ELMo) approach for contextual entity representation.
result Outperforms state-of-the-art models by 0.5% on AIDA test-b.
Improved sequential tests detect anomalies faster in multi-stream auditing.
problem Efficiently auditing machine learning systems across multiple data streams.
method Developed new sequential tests using merging martingales and averaging/products rules.
result Balanced tests achieve optimal stopping times in sparse and dense alternatives.
Bayesian framework mixes imperfect models for improved predictions.
problem Improving predictions of complex computational models in unknown domains.
method Local Bayesian Dirichlet mixing of imperfect models using the Dirichlet distribution.
result Global and local mixtures of models achieve excellent performance in prediction accuracy and uncertainty quantification.