On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,657 papers · 148 categories

Trend · papers per month

count % of corpus

7 results for “hyper-gradient”

Bilevel reinforcement learning via the development of hyper-gradient without lower-level convexitymath.OC

New method circumvents non-convexity in bilevel RL via hyper-gradient.

problem Non-convexity in lower-level RL problems in bilevel reinforcement learning.

method Characterizing hyper-gradient via fully first-order information, circumventing convexity assumption.

result Developed model-based and model-free algorithms with convergence rate

O(ε^{-1})

bilevel reinforcement learning hyper-gradient non-convex optimization reinforcement learning fixed point equation

2024-05-30abs ↗pdf ↗

Decentralized Hyper-Gradient Computation over Time-Varying Directed Networksstat.ML

Paper tackles hyper-gradient estimation in decentralized FL over time-varying networks.

problem Excessive communication costs and inability to use robust networks.

method Introduces an optimality condition and uses Push-Sum for averaging model parameters and gradients over time-varying directed networks.

result Derives a hyper-gradient estimator that operates over time-varying directed networks and converges to the true hyper-gradient.

Decentralized Federated Learning Hyper-Gradient Estimation Push-Sum Time-Varying Networks Influence Estimation

2022-10-05abs ↗pdf ↗

LancBiO: dynamic Lanczos-aided bilevel optimization via Krylov subspacemath.OC

A new method tackles bilevel optimization using Lanczos process for efficient hyper-gradient computation.

problem Efficiently solving large-scale bilevel optimization problems with gradient-based methods.

method Constructing low-dimensional approximate Krylov subspaces with the Lanczos process to approximate the Hessian inverse vector product.

result Demonstrates a

\mathcal{O}(ε^{-1})

convergence rate and efficiency in synthetic and deep learning tasks.

bilevel optimization Lanczos process hyper-gradient Krylov subspaces machine learning

2024-04-04abs ↗pdf ↗

Faster Gradient Methods for Highly-Smooth Stochastic Bilevel Optimizationmath.OC

Paper improves stochastic bilevel optimization methods for highly-smooth problems.

problem Finding

ε

-stationary points in stochastic bilevel optimization.

method Proposes F

{}^2

SA-

p

methods using

p

th-order finite differences for hyper-gradient approximation.

result Achieves upper complexity bound of

ilde{\mathcal{O}}(p ε^{-4-p/2})

for

p

th-order smooth problems.

stochastic bilevel optimization hyper-gradient approximation first-order methods higher-order smoothness complexity analysis

2025-09-03abs ↗pdf ↗

Shuffling Gradient-Based Methods for Nonconvex-Concave Minimax Optimizationmath.OC

Develops shuffling gradient-based methods for nonconvex-concave minimax optimization.

problem Nonconvex-concave minimax optimization problems.

method Two shuffling gradient-based algorithms for nonconvex-linear and nonconvex-strongly concave settings.

result Achieves state-of-the-art oracle complexity in nonconvex optimization and best-known complexity bounds for nonconvex-strongly concave setting.

minimax optimization nonconvex optimization shuffling gradient oracle complexity hyper-gradient

2024-10-29abs ↗pdf ↗

Blockwise Stochastic Variance-Reduced Methods with Parallel Speedup for Multi-Block Bilevel Optimizationmath.OC

New algorithms tackle complex multi-block optimization problems in machine learning.

problem Non-convex multi-block bilevel optimization with hierarchical sampling challenges.

method Blockwise stochastic variance-reduced methods with parallel speedup.

result Achieves matching complexity to single-block problems with parallel speedup.

bilevel optimization stochastic methods parallel speedup Hessian estimation machine learning

2023-05-30abs ↗pdf ↗

Generalization Guarantees for Neural Architecture Search with Train-Validation Splitstat.ML

This paper explores how train-validation splits help in NAS to prevent overfitting.

problem NAS overfits with train-validation splits and needs better generalization guarantees.

method Established refined properties of validation loss and risk for NAS.

result NAS with train-validation splits can select the most generalizable model.

Neural Architecture Search NAS Train-Validation Split Generalization Overfitting

2021-04-29abs ↗pdf ↗