On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,695 papers · 148 categories

Trend · papers per month

count % of corpus

48 results for “federated learning with NAG”

Federated Learning with Nesterov Accelerated Gradientcs.LG

FedNAG improves federated learning accuracy and reduces training time.

problem Efficiency of federated learning with gradient descent.

method Nesterov Accelerated Gradient (NAG) applied to federated learning (FL) with additional momentum and model aggregation.

result FedNAG increases learning accuracy by 3-24% and reduces training time by 11-70%.

federated learning federated learning with NAG Nesterov Accelerated Gradient gradient descent accuracy improvement

2020-09-18abs ↗pdf ↗

Improved Learning Rates for Stochastic Optimizationcs.LG

Paper improves learning rates for SGD and NAG.

problem Generalization performance of stochastic optimization algorithms.

method Establishes new learning rates for SGD and NAG.

result Improved guarantees in some settings or comparable rates under weaker assumptions.

stochastic optimization learning rates SGD NAG

2021-07-19abs ↗pdf ↗

Lyapunov Analysis For Monotonically Forward-Backward Accelerated Algorithmsmath.OC

This paper analyzes and improves monotonic accelerated algorithms like M-NAG and M-FISTA.

problem Establishing linear convergence of M-NAG and M-FISTA under strong convexity.

method Lyapunov analysis and modified Lyapunov functions.

result Linear convergence of M-NAG and M-FISTA is guaranteed without full NAG iterates.

Lyapunov analysis Monotonic algorithms Accelerated gradient methods M-NAG M-FISTA

2024-12-18abs ↗pdf ↗

Nesterov acceleration in benignly non-convex landscapesmath.OC

Improved optimization guarantees for deep learning models with Nesterov acceleration.

problem Optimization in non-convex deep learning landscapes.

method Analysis of Nesterov acceleration in benignly non-convex landscapes.

result Identical guarantees can be obtained in optimization problems with weak geometric assumptions, especially in overparametrized deep learning.

Nesterov acceleration non-convex optimization deep learning optimization guarantees benign non-convexity

2024-10-10abs ↗pdf ↗

Understanding the Acceleration Phenomenon via High-Resolution Differential Equationsmath.OC

Gradient-based optimization algorithms can be studied from the perspective of limiting ordinary differential equations (ODEs). Motivated by the fact that existing ODEs do not distinguish between two fundamentally different algorithms---Nesterov's accelerated gradient method for strongly convex functions (NAG-SC) and Po…

2018-10-21abs ↗pdf ↗

Accelerated Quasi-Newton Proximal Extragradient: Faster Rate for Smooth Convex Optimizationmath.OC

New method solves convex optimization faster than NAG.

problem Unconstrained smooth convex optimization problems.

method Accelerated quasi-Newton proximal extragradient (A-QPNE) method.

result Achieves a faster convergence rate of

{O}\bigl(\min\{\frac{1}{k^2}, \frac{\sqrt{d\log k}}{k^{2.5}}\}\bigr)

convex optimization quasi-Newton proximal extragradient accelerated gradient online learning

2023-06-03abs ↗pdf ↗

Scheduled Restart Momentum for Accelerated Stochastic Gradient Descentcs.LG

SRSGD improves DNN training speed and accuracy.

problem Training deep neural networks is computationally expensive and slow.

method Scheduled Restart SGD (SRSGD) combines NAG momentum with momentum reset.

result SRSGD achieves better error rates with fewer training epochs.

SGD NAG Momentum Deep Learning Convergence

2020-02-24abs ↗pdf ↗

Optimizing Time Series Forecasting: A Comparative Study of Adam and Nesterov Accelerated Gradient on LSTM and GRU networks Using Stock Market dataq-fin.ST

GRU models with Adam optimizer outperform other combinations in stock market forecasting.

problem Comparing optimization techniques for time series forecasting in LSTM and GRU networks.

method Examined Adam and Nesterov Accelerated Gradient (NAG) on LSTM and GRU models for stock market forecasting.

result GRU models with Adam optimizer produced the lowest RMSE and outperformed other combinations.

Time-series Forecasting Neural Network LSTM GRU Adam Optimizer

2024-09-28abs ↗pdf ↗

On the insufficiency of existing momentum schemes for Stochastic Optimizationcs.LG

Momentum based stochastic gradient methods such as heavy ball (HB) and Nesterov's accelerated gradient descent (NAG) method are widely used in practice for training deep networks and other supervised learning models, as they often provide significant improvements over stochastic gradient descent (SGD). Rigorously speak…

2018-03-15abs ↗pdf ↗

Variational Optimization on Lie Groups, with Examples of Leading (Generalized) Eigenvalue Problemscs.LG

The article considers smooth optimization of functions on Lie groups. By generalizing NAG variational principle in vector space (Wibisono et al., 2016) to Lie groups, continuous Lie-NAG dynamics which are guaranteed to converge to local optimum are obtained. They correspond to momentum versions of gradient flow on Lie …

2020-01-27abs ↗pdf ↗

Linear convergence of forward-backward accelerated algorithms without knowledge of the modulus of strong convexitymath.OC

New insights show NAG and FISTA converge linearly without knowing strong convexity modulus.

problem Understanding linear convergence of NAG and FISTA without strong convexity modulus knowledge.

method High-resolution ODE framework, dynamically adapting kinetic energy coefficient.

result NAG and FISTA demonstrate linear convergence without requiring strong convexity modulus knowledge.

Nesterov's Accelerated Gradient Descent FISTA Linear Convergence Strong Convexity Dynamically Adapting Coefficient

2023-06-16abs ↗pdf ↗

Momentum-based gradient descent methods for Lie groupsmath.OC

Proposes momentum methods for Lie groups, improving on classical algorithms.

problem Optimization on nonlinear spaces, especially Lie groups.

method Generalizes Nesterov's Accelerated Gradient method to Lie groups.

result Demonstrates faster convergence for NAG-like methods on Lie groups.

Lie groups optimization momentum methods Nesterov's Accelerated Gradient Polyak's Heavy Ball

2024-04-14abs ↗pdf ↗

Quantitative Convergences of Lie Group Momentum Optimizerscs.LG

This paper analyzes two Lie group momentum optimization algorithms and their convergence rates.

problem Optimizing functions on Lie groups using momentum-based dynamics.

method Investigates Lie Heavy-Ball and Lie NAG-SC algorithms, quantifying their convergence rates under smoothness and convexity assumptions.

result Lie NAG-SC accelerates optimization over the momentumless case, while Lie Heavy-Ball does not.

Lie groups momentum optimization convergence rates Riemannian manifolds acceleration

2024-05-30abs ↗pdf ↗

Hessian-Free High-Resolution Nesterov Acceleration for Samplingcs.LG

This research accelerates sampling methods using Nesterov's Acceleration.

problem Improving sampling efficiency in MCMC methods.

method Developed a Hessian-Free High-Resolution ODE reformulation of NAG-SC, injected noise, and discretized the diffusion process.

result Quantified acceleration beyond underdamped Langevin in

W_2

distance for log-strongly-concave targets.

Nesterov Acceleration Sampling MCMC Hessian-Free High-Resolution

2020-06-16abs ↗pdf ↗

A Family of Controllable Momentum Coefficients for Forward-Backward Accelerated Algorithmsmath.OC

A new family of momentum coefficients improves the convergence rate of accelerated algorithms.

problem Improving the convergence rate of accelerated gradient methods for strongly convex functions.

method Introducing a family of controllable momentum coefficients for forward-backward accelerated methods.

result Established a controllable $O\left(1/k^{2α} ight)$ convergence rate for the NAG-

α

method.

accelerated gradient methods momentum coefficients forward-backward methods convergence rates strongly convex functions

2025-01-17abs ↗pdf ↗

Stability and Convergence Trade-off of Iterative Optimization Algorithmsstat.ML

The overall performance or expected excess risk of an iterative machine learning algorithm can be decomposed into training error and generalization error. While the former is controlled by its convergence analysis, the latter can be tightly handled by algorithmic stability. The machine learning community has a rich his…

2018-04-04abs ↗pdf ↗

NeCPD: An Online Tensor Decomposition with Optimal Stochastic Gradient Descentcs.LG

NeCPD improves online tensor decomposition using SGD with Hessian analysis and NAG.

problem Efficiently decompose multi-way tensors in online data processing.

method NeCPD solver based on SGD with Hessian analysis and NAG.

result NeCPD provides more accurate results than existing methods.

tensor decomposition online data stochastic gradient descent Hessian analysis NAG

2020-03-18abs ↗pdf ↗

Towards the Theory of Unsupervised Federated Learning: Non-asymptotic Analysis of Federated EM Algorithmsstat.ML

This paper develops a federated EM algorithm for unsupervised learning of mixture models.

problem Theoretical foundations of unsupervised federated learning are lacking.

method Introduces a federated gradient EM algorithm (FedGrEM) for unsupervised learning of mixture models.

result Theoretical analysis shows FedGrEM outperforms local single-task learning.

federated learning unsupervised learning federated EM mixture models finite-sample theory

2023-10-23abs ↗pdf ↗

Practical One-Shot Federated Learning for Cross-Silo Settingcs.LG

A practical one-shot federated learning algorithm for cross-silo setting.

problem Limited applicability of existing one-shot federated learning algorithms due to specific model support and lack of privacy guarantees.

method FedKT, a one-shot federated learning algorithm that supports any classification models and provides differential privacy guarantees.

result FedKT significantly outperforms other state-of-the-art federated learning algorithms with a single communication round.

federated learning one-shot cross-silo privacy differential privacy

2020-10-02abs ↗pdf ↗

The OARF Benchmark Suite: Characterization and Implications for Federated Learning Systemscs.LG

Paper introduces OARF benchmark suite for federated learning systems.

problem Limited diversity in federated learning benchmarks.

method Characterizes OARF benchmark suite with diverse data and applications.

result Federated learning can effectively increase end-to-end throughput.

federated learning benchmark suite federated machine learning throughput convergence time

2020-06-14abs ↗pdf ↗

Adaptive Federated Optimizationcs.LG

Adaptive optimizers improve federated learning performance.

problem Inadequate optimization methods in federated learning.

method Federated versions of adaptive optimizers (Adagrad, Adam, Yogi).

result Adaptive optimizers enhance federated learning performance.

federated learning adaptive optimization federated Averaging non-convex optimization client heterogeneity

2020-02-29abs ↗pdf ↗

A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protectioncs.LG

Federated learning has been a hot research topic in enabling the collaborative training of machine learning models among different organizations under the privacy restrictions. As researchers try to support more machine learning models with different privacy-preserving approaches, there is a requirement in developing s…

2019-07-23abs ↗pdf ↗

Improving Infinitely Deep Bayesian Neural Networks with Nesterov's Accelerated Gradient Methodstat.ML

Improved SDE-BNN model reduces NFEs and accelerates convergence.

problem High computational cost and convergence instability in SDE-BNNs.

method Nesterov's Accelerated Gradient (NAG) method integrated into SDE-BNN framework.

result Significantly reduced number of function evaluations (NFEs) and improved predictive accuracy.

Bayesian neural networks Nesterov's Accelerated Gradient Stochastic Differential Equations Convergence stability Function evaluations reduction

2026-03-26abs ↗pdf ↗

Convergence guarantees for RMSProp and ADAM in non-convex optimization and an empirical comparison to Nesterov accelerationcs.LG

RMSProp and ADAM continue to be extremely popular algorithms for training neural nets but their theoretical convergence properties have remained unclear. Further, recent work has seemed to suggest that these algorithms have worse generalization properties when compared to carefully tuned stochastic gradient descent or …

2018-07-18abs ↗pdf ↗

Communication-Efficient Federated Distillationcs.LG

Compressed Federated Distillation reduces communication in federated learning.

problem Communication constraints in Federated Learning.

method Compressed Federated Distillation (CFD) leverages soft labels and quantization techniques.

result Reduces communication by more than 4 orders of magnitude compared to Federated Averaging.

Federated Learning Communication Efficiency Federated Distillation Soft Labels Quantization

2020-12-01abs ↗pdf ↗

Federated

f

-Differential Privacystat.ML

New federated $f$ -differential privacy for collaborative learning.

problem Privacy in federated learning.

method Introducing federated

f

-differential privacy and proposing a generic private federated learning framework.

result Proves federated

f

-differential privacy provides privacy guarantee on each record of one client's data.

federated learning privacy differential privacy f-differential privacy collaborative learning

2021-02-22abs ↗pdf ↗

pFedGame -- Decentralized Federated Learning using Game Theory in Dynamic Topologystat.ML

pFedGame uses game theory for decentralized federated learning in dynamic networks.

problem Performance bottlenecks, data bias, model convergence issues, and model poisoning attacks in federated learning.

method pFedGame employs game theory to decentralize federated learning, avoiding a central aggregation server and addressing dynamic network challenges.

result pFedGame achieves higher accuracy (over 70%) in heterogeneous data compared to existing methods.

federated learning decentralized game theory dynamic networks model convergence

2024-10-05abs ↗pdf ↗

Federated Asymptotics: a model to compare federated learning algorithmscs.LG

A framework to compare federated learning algorithms in high-dimensional settings.

problem Comparing the performance of federated learning algorithms in high-dimensional settings.

method Formulating federated learning as a multi-criterion objective and analyzing a linear regression model.

result Federated Averaging with simple client fine-tuning achieves the same asymptotic risk as more intricate approaches and outperforms without personalization.

federated learning asymptotics personalization information sharing meta-learning

2021-08-16abs ↗pdf ↗

Federated Learning as a Mean-Field Gamestat.ML

Federated learning linked to mean-field games for large-scale learning.

problem Large-scale distributed and privacy-preserving learning algorithms.

method Established a connection between federated learning and mean-field games, presenting federated learning as a differential game.

result Properties of the equilibrium of the federated learning game were discussed.

federated learning mean-field games machine learning game theory control theory

2021-07-08abs ↗pdf ↗

Provable Acceleration of Nesterov's Accelerated Gradient for Rectangular Matrix Factorization and Linear Neural Networkscs.LG

GD and NAG accelerate matrix factorization and neural networks.

problem Optimizing rectangular matrix factorization and linear neural networks.

method Gradient descent and Nesterov's accelerated gradient with specific initialization.

result NAG achieves the best-known iteration complexity for these problems.

matrix factorization Nesterov's acceleration rectangular matrices linear neural networks convergence rate

2024-10-12abs ↗pdf ↗

Hierarchically Fair Federated Learningcs.LG

A fair reward system boosts participation in federated learning.

problem Fairness in federated learning among competitive agents with siloed data.

method Hierarchically fair federated learning (HFFL) framework with proportional rewards based on contribution levels.

result Efficacy of HFFL in maintaining fairness and facilitating federated learning in competitive settings.

federated learning fairness competitive agents hierarchical reward proportional rewards

2020-04-22abs ↗pdf ↗

Federated Multi-Mini-Batch: An Efficient Training Approach to Federated Learning in Non-IID Environmentscs.LG

Federated multi-mini-batch improves efficiency in non-IID environments.

problem Performance and communication efficiency challenges in federated learning with non-IID data.

method Introduces federated multi-mini-batch approach to balance performance and communication.

result Federated multi-mini-batch outperforms federated averaging in non-IID settings.

Federated Learning Non-IID Data Mini-Batch Communication Efficiency

2020-11-13abs ↗pdf ↗

Towards Fair Federated Learning with Zero-Shot Data Augmentationcs.CV

This work improves fairness in federated learning by using zero-shot data augmentation.

problem Statistical heterogeneity leads to biased and less uniform accuracy across clients in federated learning.

method Proposes a federated learning system with zero-shot data augmentation to mitigate statistical heterogeneity and improve fairness.

result Empirical results show improved test accuracy and fairness across clients.

federated learning zero-shot data augmentation statistical heterogeneity fairness accuracy

2021-04-27abs ↗pdf ↗

Variational Federated Multi-Task Learningcs.LG

In federated learning, a central server coordinates the training of a single model on a massively distributed network of devices. This setting can be naturally extended to a multi-task learning framework, to handle real-world federated datasets that typically show strong statistical heterogeneity among devices. Despite…

2019-06-14abs ↗pdf ↗

Federated Learning and Differential Privacy: Software tools analysis, the Sherpa.ai FL framework and methodological guidelines for preserving data privacycs.LG

Sherpa.ai framework combines federated learning and differential privacy for edge AI services.

problem Protecting data privacy in edge AI services.

method Holistic federated learning and differential privacy approach with methodological guidelines.

result Demonstrated through classification and regression use cases.

federated learning differential privacy Sherpa.ai edge AI privacy protection

2020-07-02abs ↗pdf ↗

What Do We Mean by Generalization in Federated Learning?cs.LG

Federated learning studies separate client data and distribution gaps.

problem Understanding performance differences in federated learning across different datasets.

method Proposed a framework to disentangle out-of-sample and participation gaps.

result Dataset synthesis strategy is crucial for realistic simulations of federated learning generalization.

Federated Learning Generalization Client Data Distribution Gaps Synthetic Data

2021-10-27abs ↗pdf ↗

Dynamic Defense Against Byzantine Poisoning Attacks in Federated Learningcs.LG

Dynamic defense against Byzantine poisoning in federated learning.

problem Byzantine poisoning attacks on federated learning models.

method Dynamic federated aggregation operator to filter out adversarial clients.

result Dynamic selection of clients improves global learning model performance.

federated learning byzantine poisoning aggregation operator adversarial attacks deep learning

2020-07-29abs ↗pdf ↗

Communication-Efficient Adaptive Federated Learningcs.LG

A new federated learning method reduces communication costs and improves adaptivity.

problem Large communication overhead and lack of adaptivity in federated learning.

method FedCAMS: A novel communication-efficient adaptive federated learning method with theoretical guarantees.

result FedCAMS achieves the same convergence rate as non-compressed federated learning methods.

federated learning communication efficiency adaptive optimization federated adaptive convergence analysis

2022-05-05abs ↗pdf ↗

Pretraining Federated Text Models for Next Word Predictioncs.LG

Research improves federated text models for next word prediction.

problem Training models on distributed devices efficiently and effectively.

method Employ transfer learning in federated learning for next word prediction.

result Enhancements to current baselines with pretrained embeddings and whole model pretraining.

federated learning next word prediction transfer learning word embeddings model pretraining

2020-05-11abs ↗pdf ↗

Federated Learning with Communication Delay in Edge Networkscs.LG

Federated learning technique improves convergence speed with communication delays.

problem Communication delays between edge nodes and aggregator in federated learning.

method Developed FedDelAvg, a technique that generalizes federated averaging to incorporate a weighting between current local model and delayed global model.

result FedDelAvg achieves a significant improvement in convergence speed, especially when optimizing the weighting scheme to account for delays.

federated learning communication delays edge networks federated averaging convergence speed

2020-08-21abs ↗pdf ↗

Federated Covariate Shift Adaptation for Missing Target Output Valuesstat.ML

Federated learning method improves covariate shift adaptation for missing target values.

problem Missing target values in federated learning.

method Federated covariate shift adaptation algorithm for missing target output values.

result Asymptotically unbiased and efficient algorithm for federated learning.

federated learning covariate shift missing target asymptotic variance

2023-02-28abs ↗pdf ↗

Privacy Threats Against Federated Matrix Factorizationcs.CR

This paper analyzes privacy threats in federated matrix factorization.

problem Privacy threats in federated matrix factorization models.

method Categorizes federated matrix factorization into three types and analyzes privacy threats.

result This is the first study of privacy threats in federated matrix factorization.

federated learning matrix factorization privacy threats privacy-preserving approaches federated matrix factorization types

2020-07-03abs ↗pdf ↗

FedCluster: Boosting the Convergence of Federated Learning via Cluster-Cyclingcs.LG

FedCluster accelerates federated learning convergence by cycling device groups.

problem Federated learning convergence issues with device-level data heterogeneity.

method FedCluster groups devices into clusters that cycle through learning rounds, boosting convergence with meta-updates.

result FedCluster achieves faster convergence in nonconvex optimization compared to FedAvg.

federated learning convergence cycling meta-updates device heterogeneity

2020-09-22abs ↗pdf ↗

Federated Residual Learningcs.LG

New federated learning framework reduces model complexity and improves performance.

problem Slow convergence in traditional federated learning due to non-i.i.d. data.

method Clients train personalized local models, server trains shared model, addressing heterogeneity.

result Substantial performance gains over baselines, robust to non-i.i.d. data.

federated learning joint training model complexity performance gains non-i.i.d. data

2020-03-28abs ↗pdf ↗

Interpret Federated Learning with Shapley Valuescs.LG

Federated Learning is introduced to protect privacy by distributing training data into multiple parties. Each party trains its own model and a meta-model is constructed from the sub models. In this way the details of the data are not disclosed in between each party. In this paper we investigate the model interpretation…

2019-05-11abs ↗pdf ↗

Federated Continual Learning with Weighted Inter-client Transfercs.LG

A new framework for federated continual learning reduces interference and improves performance.

problem Learning from a sequence of tasks with limited data from each client.

method Federated Weighted Inter-client Transfer (FedWeIT) framework.

result FedWeIT significantly outperforms existing methods with reduced communication cost.

federated learning continual learning knowledge transfer interference weighted combination

2020-03-06abs ↗pdf ↗

Stochastic Smoothed Gradient Descent Ascent for Federated Minimax Optimizationstat.ML

New algorithm solves federated minimax optimization problems.

problem Federated minimax optimization challenges.

method Federated Stochastic Smoothed Gradient Descent Ascent (FESS-GDA).

result FESS-GDA uniformly solves federated minimax problems.

federated learning minimax optimization smoothed gradient ascent FESS-GDA convergence

2023-11-02abs ↗pdf ↗

FedSPDnet: Geometry-Aware Federated Deep Learning with SPDnetstat.ML

FedSPDnet improves federated learning for SPD matrices, outperforming existing methods.

problem Federated learning for SPD matrices with orthogonality constraints.

method Two efficient aggregation strategies: ProjAvg and RLAvg, preserving geometric structure.

result FedSPDnet outperforms federated EEGnet in F1 score and robustness to federation and partial participation.

federated learning SPD matrices Stiefel manifold orthogonality signal processing

2026-04-24abs ↗pdf ↗