Accelerates Riemannian gradient methods with extrapolation.
problem Optimizing functions on manifolds efficiently.
method Extrapolating iterates in Riemannian gradient descent.
result Achieves optimal convergence rate and computational advantage.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,291 papers · 148 categories
Trend · papers per month
48 results for “gradient extrapolation”
Accelerates coordinate descent methods for machine learning problems.
problem Slowness of coordinate descent methods in machine learning.
method Extrapolation-based accelerated coordinate descent.
result Significant speed-up in practice compared to existing methods.
Paper accelerates NMF algorithms using extrapolation.
problem Efficiently solving nonnegative matrix factorization problems.
method Inspired by convex optimization, uses extrapolation for two-block exact coordinate descent algorithms.
result Significantly accelerates NMF algorithms on various data sets.
Implicit Bias of Policy Gradient in Linear Quadratic Control: Extrapolation to Unseen Initial Statescs.LG
This paper studies how gradient descent in control systems can perform well on unseen data.
problem The extent of a learned controller's ability to extrapolate to unseen initial states.
method Theoretical study of policy gradient in Linear Quadratic Regulator (LQR) problems, focusing on the role of exploration.
result The performance of a learned controller on unseen initial states depends on the degree of exploration induced by the system.
NSR enables neural networks to reason with continuous numbers and extrapolate.
problem Quantitative reasoning and extrapolation in neural networks.
method Proposes Neural Status Register (NSR) for continuous number reasoning.
result NSR achieves extrapolation to numbers many orders of magnitude larger than training data.
A new method combines extrapolation and line search for solving nonconvex, nonsmooth optimization problems.
problem Nonconvex, nonsmooth optimization problems in machine learning and image processing.
method Proximal gradient method with extrapolation and line search (PGels).
result The method reduces to existing algorithms under proper parameter choices and converges to stationary points.
The paper analyzes SGD and its convergence using Markov chain theory.
problem Analyzing the convergence of SGD with constant step-size.
method Integrates Markov chain theory into SGD analysis and uses Richardson-Romberg extrapolation.
result Provides an explicit asymptotic expansion of SGD iterates moments.
The paper tackles extrapolation in generative models by enforcing independence of mechanisms.
problem How to make generative models extrapolate to new, unseen environments?
method Developed a theoretical framework for independence of mechanisms, demonstrated on toy examples and real-world data.
result Extrapolation capabilities of generative models can be improved by enforcing independence of mechanisms explicitly during training.
Learnable token perturbations boost extrapolation in LLMs.
problem Limited flexibility of current discrete perturbations in large language models.
method Learnable continuous latent vector transformations in embedding space, unbiased estimating equations, stochastic gradient descent optimization.
result Significant gains in out-of-domain settings over state-of-the-art methods.
In this paper, we study the proximal gradient algorithm with extrapolation for minimizing the sum of a Lipschitz differentiable function and a proper closed convex function. Under the error bound condition used in [19] for analyzing the convergence of the proximal gradient algorithm, we show that there exists a thresho…
Probabilistic models predict neural network performance across varying hyperparameters.
problem Predicting neural network performance with different hyperparameters.
method Probabilistic models based on random forests and Bayesian recurrent neural networks.
result Models outperform state-of-the-art hyperparameter optimization methods.
The Strength of Nesterov's Extrapolation in the Individual Convergence of Nonsmooth Optimizationcs.LG
Nesterov's extrapolation improves convergence in nonsmooth optimization.
problem Improving convergence rate in nonsmooth convex optimization.
method Nesterov's extrapolation applied to projected subgradient methods.
result Nesterov's extrapolation optimizes individual convergence for nonsmooth problems.
A new method uses extrapolation to train deep learning models with larger batch sizes.
problem Persistent degradation in performance when increasing batch size for deep learning.
method Proposes extrapolation to stabilize optimization trajectory and improve generalization.
result Demonstrates scaling to larger batch sizes while maintaining or surpassing state-of-the-art accuracy.
Nonasymptotic Analysis of Stochastic Gradient Descent with the Richardson-Romberg Extrapolationmath.OC
The paper analyzes SGD with Richardson-Romberg extrapolation for convex optimization problems.
problem Solving strongly convex and smooth minimization problems efficiently.
method Combining SGD with Polyak-Ruppert averaging and Richardson-Romberg extrapolation.
result An expansion of the mean-squared error of the estimator with respect to the number of iterations.
Quantized Variational Inference improves ELBO optimization with fast convergence.
problem Maximizing Evidence Lower Bound (ELBO) for variational inference.
method Optimal Voronoi Tesselation for variance-free gradients, Richardson extrapolation for asymptotic improvement.
result Quantized Variational Inference leads to fast convergence with comparable computational cost.
Neural networks extrapolate poorly in simple tasks but succeed in complex ones.
problem Understanding neural networks' extrapolation capabilities and conditions for success.
method Analyzing ReLU MLPs and GNNs, connecting to neural tangent kernel.
result ReLU MLPs learn linear functions but not most nonlinear ones, while GNNs succeed in complex tasks due to task-specific non-linearities.
Faster matrix completion and extrapolation via kernel regression.
problem Matrix completion and extrapolation with prior information.
method Kernel ridge regression in reproducing kernel Hilbert spaces.
result Novel algorithm performs faster and reduces recovery error.
FedExProx's performance is no better than GD for quadratic optimization.
problem Improving convergence of parallel proximal algorithms.
method Developed a novel analysis framework to establish tighter convergence rates.
result FedExProx can outperform GD in non-strongly convex quadratic problems.
New method finds Nash equilibria faster in multiplayer games.
problem Finding Nash equilibria in multi-player games, especially in noisy environments.
method Extra-gradient with player sampling and variance reduction.
result Proves better convergence rate than full extra-gradient for noisy games.
New algorithm reduces complexity for distributed and stochastic optimization.
problem Optimization of distributed and stochastic problems with multiple agents.
method Random Gradient Extrapolation Method (RGEM)
result Achieves optimal complexity bounds for both distributed and stochastic optimization.
Adaptive learning rate improves SGD performance in federated and private learning.
problem Improving SGD performance in federated and private learning.
method Adaptively tunes learning rate using extrapolation of gradient flow.
result Adaptive algorithm outperforms manually tuned optimizers in federated and private learning.
Interpolatron accelerates deep neural network optimization faster than existing methods.
problem Accelerating nonconvex optimization for deep neural networks.
method Proposes Interpolatron, a new interpolation scheme to accelerate nonconvex optimization.
result Interpolatron converges much faster than state-of-the-art methods on DNNs of great depths.
Proposes BMME for optimizing nonsmooth nonconvex problems with block structure.
problem Optimizing nonsmooth nonconvex problems with block structure.
method Block Alternating Bregman Majorization Minimization with Extrapolation (BMME).
result Subsequential convergence to a first-order stationary point under mild assumptions, global convergence under stronger conditions.
Linear Convergence of Accelerated Stochastic Gradient Descent for Nonconvex Nonsmooth Optimizationmath.OC
Accelerated SGD method converges linearly to local minima of nonconvex problems.
problem Nonconvex nonsmooth optimization problems.
method Combining variance reduction and Nesterov's extrapolation for accelerated SGD.
result Linear convergence to a stationary point of the nonconvex optimization problem.
RTE enables extrapolation to new tasks by learning task transformations.
problem Learning systems struggle to generalize to unseen tasks.
method Relational Task Extrapolator (RTE) learns task transformations to enable extrapolation.
result RTE substantially outperforms existing approaches on extrapolation tasks.
We extend nonparametric models to handle extrapolation, providing bounds for inference.
problem Challenges in nonparametric statistical inference when evaluating outside the conditioning variable's support.
method Introduced a class of extrapolation assumptions and a consistent estimation procedure to handle extrapolation.
result Validated extrapolation-aware conclusions through various applications and real-world data.
New method reduces bias in incomplete data using deliberate missingness.
problem Systematic gradient biases in incomplete data for stochastic learning.
method Richardson-SGD debiasing procedure with deliberate missingness.
result Reduces gradient bias from to .
Paper analyzes \FedAvg's convergence and introduces a new algorithm to reduce bias.
problem Analyzing convergence and bias in Federated Averaging.
method Markov property, first-order bias expansion, Richardson-Romberg extrapolation.
result Bias in \FedAvg can be decomposed into noise and client heterogeneity components.
Binary encoding enables neural networks to extrapolate periodic functions.
problem Extrapolating periodic functions without prior knowledge of their form.
method Normalized Base-2 Encoding (NB2E) for continuous numerical values.
result MLPs using NB2E can successfully extrapolate diverse periodic signals.
Optimizing tail risks using an importance sampling based extrapolation for heavy-tailed objectivesstat.ME
New method approximates CVaR with less data for heavy-tailed risks.
problem Lack of data for accurate CVaR approximation in heavy-tailed distributions.
method Importance sampling based extrapolation for heavy-tailed distributions.
result Statistically consistent approximations with reduced data requirements.
Method controls extrapolation in prediction profiles for statistical and machine learning models.
problem Avoiding invalid predictions due to extrapolation in prediction profiles.
method Genetic algorithm optimization over constrained factor regions.
result Optimal factor settings without constraint are often invalid and extrapolated.
Neural networks struggle with extrapolation, but a new framework allows them to learn counterfactual invariances.
problem Neural networks' inability to extrapolate beyond training data distribution.
method Introduces a learning framework that allows neural networks to extrapolate over group transformations based on counterfactual invariances.
result Neural networks can learn counterfactual invariances from a single environment, overcoming their limitations in extrapolation.
New method optimizes language model performance for test-time strategies.
problem Mismatch between training objectives and test-time deployment of large language models.
method Tail-Extrapolated estimators to approximate best-of-N performance from limited training rollouts.
result Improved performance of best-of-N deployment across various models and datasets.
Uncertainty-biased molecular dynamics for learning uniformly accurate interatomic potentialsphysics.comp-ph
New method uses biased MD to create accurate MLIPs.
problem Creating a comprehensive data set for MLIPs.
method Bias MD by MLIP's energy uncertainty, using gradient-based uncertainties.
result Develops uniformly accurate MLIPs with lower computational cost.
Study shows LLMs can extrapolate rules from out-of-distribution prompts.
problem Understanding LLMs' ability to generalize from unexpected inputs.
method Formal languages and rule-based scenarios to evaluate LLMs' OOD behavior.
result LLMs can extrapolate rules from out-of-distribution prompts, even in complex scenarios.
New methods reduce extrapolation errors in feature importance.
problem Flawed feature importance methods using unrestricted permutations lead to extrapolation errors.
method Three new approaches: conditional model reliance, Knockoffs with Gaussian transformation, and restricted ALE plot designs.
result Theoretical and numerical results show our strategies reduce/eliminate extrapolation.
This work reformulates GAN training as variational inequalities to improve training stability.
problem Training GANs is difficult due to the saddle point optimization problem.
method Casts GAN optimization as variational inequalities and applies optimization techniques.
result Proposed techniques improve GAN training stability and sample quality.
New models extrapolate false alarms in ASV without new data.
problem Reliable extrapolation of false alarm rates in ASV without new speaker data.
method Generative models in ASV score space for arbitrary systems.
result Models accurately extrapolate false alarm rates for large speaker populations.
A new principle for extrapolating regression outside training data.
problem Regression extrapolation when predictions are outside training data range.
method Data-adaptive marginal transformation and simple relationship assumption.
result Progression method offers guarantees on approximation error beyond training data range.
Asymmetric Nonlinear Return Extrapolation and Optimal Portfolio Choice under Stochastic Volatilityq-fin.PM
We extend return extrapolation to incorporate asymmetry and saturation, finding that asymmetric nonlinear extrapolation leads to lower welfare loss.
problem Optimal portfolio choice under stochastic volatility
method Smooth, nonlinear extrapolation function with sentiment and variance hedging
result Lower welfare loss with asymmetric nonlinear extrapolation
Logical neural networks solve mazes by filling dead ends, but not all methods generalize well.
problem Understanding how logical neural networks extrapolate solutions to mazes.
method Examined recurrent and implicit neural networks trained on maze-solving tasks.
result Models fail to generalize well to diverse maze sizes, suggesting limitations in learning scalable algorithms.
The paper explores how to make neural networks extrapolate longer sequences.
problem Neural networks struggle to extrapolate beyond seen data, especially for longer sequences.
method The authors propose a model with separate content- and location-based attention mechanisms.
result Models with the proposed attention mechanisms are better at extrapolating longer sequences.
Improves reinforcement learning extrapolation in Gridworlds.
problem Generalizing to unseen states in reinforcement learning.
method Avoiding deterministic action choice, using ego-centric representation, incorporating symmetry, and adding an entropy term.
result Significant improvement in extrapolation performance in Gridworld environments.
New methods improve neural network extrapolation to long sequences.
problem Neural networks struggle with extrapolation to very long or adversarial sequences.
method Activation binning and localized differentiable memory architecture.
result No extrapolation errors detected within memory constraints.
Concept modulation models unify identifiability and extrapolation in conditional latent variable models.
problem Reliable generalization in conditional latent variable models
method Concept modulation models (CMMs) with structure
result Lifts identifiability to conditional settings and controls extrapolation through attribute potentials.
New technique reduces bias in CSO problems, improving sample complexity.
problem Reducing bias in conditional stochastic optimization problems.
method Introducing a stochastic extrapolation technique combined with variance reduction.
result Achieved significantly better sample complexity for nonconvex smooth objectives.
New framework shows ERM is optimal for both interpolation and extrapolation in domain generalization.
problem Formalizing and solving the challenges of domain generalization.
method Reformulated domain generalization as an online game between a risk-minimizing player and an adversary.
result ERM is minimax-optimal for both interpolation and extrapolation in domain generalization.
Asymmetric Nonlinear Return Extrapolation and Optimal Portfolio Choice under Stochastic Volatilityq-fin.PM
Extends return extrapolation to nonlinear, asymmetric functions under stochastic volatility.
problem Behavioral anomalies in portfolio choice under stochastic volatility.
method Smooth, nonlinear, asymmetric extrapolation function; CRRA investor; Heston stochastic volatility; Hamilton-Jacobi-Bellman equation; Numerical solutions (finite-difference ADI, deep learning-driven iterative).
result Saturation acts as an endogenous correction mechanism, reducing welfare loss.