A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Two multifidelity trust-region methods use low-fidelity models for efficient optimization.
TRSVR combines SVRG with trust-region for faster optimization.
Bayesian optimization tackles constrained high-dimensional problems with penalties and trust regions.
Two new algorithms solve nonconvex-strongly concave problems efficiently.
AdaScale-TuRBO improves high-dimensional Bayesian optimization by dynamically scaling the GP lengthscale.
Proposes a new algorithm for solving optimization problems with stochastic objectives and equality constraints.
TROLL improves RL for LLMs by replacing clipping with a trust region projection.
Simplified trust region method reduces representation change during fine-tuning.
We propose to improve trust region policy search with normalizing flows policy. We illustrate that when the trust region is constructed by KL divergence constraints, normalizing flows policy generates samples far from the 'center' of the previous policy iterate, which potentially enables better exploration and helps av…
TREGO improves EGO for global optimization of high-dimensional problems.
We propose a trust region method for policy optimization that employs Quasi-Newton approximation for the Hessian, called Quasi-Newton Trust Region Policy Optimization QNTRPO. Gradient descent is the de facto algorithm for reinforcement learning tasks with continuous controls. The algorithm has achieved state-of-the-art…
JAXFit speeds up curve fitting on GPUs.
Stochastic variational inference allows for fast posterior inference in complex Bayesian models. However, the algorithm is prone to local optima which can make the quality of the posterior approximation sensitive to the choice of hyperparameters and initialization. We address this problem by replacing the natural gradi…
Proximal policy optimization (PPO) is one of the most popular deep reinforcement learning (RL) methods, achieving state-of-the-art performance across a wide range of challenging tasks. However, as a model-free RL method, the success of PPO relies heavily on the effectiveness of its exploratory policy search. In this pa…
The paper analyzes and improves a deep learning optimization technique using matrix gradient orthogonality.
We target the problem of finding a local minimum in non-convex finite-sum minimization. Towards this goal, we first prove that the trust region method with inexact gradient and Hessian estimation can achieve a convergence rate of order as long as those differential estimations are sufficientl…
Trust is a collective, self-fulfilling phenomenon that suggests analogies with phase transitions. We introduce a stylized model for the build-up and collapse of trust in networks, which generically displays a first order transition. The basic assumption of our model is that whereas trust begets trust, panic also begets…
Building upon the recent success of deep reinforcement learning methods, we investigate the possibility of on-policy reinforcement learning improvement by reusing the data from several consecutive policies. On-policy methods bring many benefits, such as ability to evaluate each resulting policy. However, they usually d…
TRM improves long-horizon LLM RL by masking divergent sequences.
TRM improves long-horizon reinforcement learning for LLMs by masking divergent sequences.
Trust-region methods have yielded state-of-the-art results in policy search. A common approach is to use KL-divergence to bound the region of trust resulting in a natural gradient policy update. We show that the natural gradient and trust region optimization are equivalent if we use the natural parameterization of a st…
New method solves stochastic optimization problems with random models.
Mirror descent method improved RL algorithms.
New sampling strategy improves TR algorithms for stochastic optimization.
New formulas for geodesics on Stiefel and flag manifolds using trust-region method.
Develops ODRPO to improve RL algorithms with better performance and stability.
Adaptive replication improves stochastic function optimization.
MF-TRPO optimizes MFGs with finite sample guarantees.
New algorithm improves deep learning stability with limited data.
Improves inference-time alignment for diffusion models without updating weights.
Nowadays stochastic approximation methods are one of the major research direction to deal with the large-scale machine learning problems. From stochastic first order methods, now the focus is shifting to stochastic second order methods due to their faster convergence and availability of computing resources. In this pap…
Reinforcement Learning(RL) with sparse rewards is a major challenge. We propose \emph{Hindsight Trust Region Policy Optimization}(HTRPO), a new RL algorithm that extends the highly successful TRPO algorithm with \emph{hindsight} to tackle the challenge of sparse rewards. Hindsight refers to the algorithm's ability to l…
Trust region policy optimization (TRPO) is a popular and empirically successful policy search algorithm in Reinforcement Learning (RL) in which a surrogate problem, that restricts consecutive policies to be 'close' to one another, is iteratively solved. Nevertheless, TRPO has been considered a heuristic algorithm inspi…
New method solves optimization problems with stochastic objectives and constraints.
Enhances BO in high dimensions with Newton methods.
New method improves Gaussian Mixture Model fitting speed.
End-to-end pipeline for data-driven decision making in mixed-integer optimization.
In data-limited settings, stochastic policies can outperform deterministic ones in bandit problems.
Proximal policy optimization (PPO) is one of the most successful deep reinforcement-learning methods, achieving state-of-the-art performance across a wide range of challenging tasks. However, its optimization behavior is still far from being fully understood. In this paper, we show that PPO could neither strictly restr…
Safe RL for autonomous vehicles using PCPO with trust regions and parallel learners.
Deep Neural Networks are quite vulnerable to adversarial perturbations. Current state-of-the-art adversarial attack methods typically require very time consuming hyper-parameter tuning, or require many iterations to solve an optimization based adversarial attack. To address this problem, we present a new family of trus…
The paper introduces a method for fitting complex models using simulation and optimization.
The estimation of multivariate GARCH time series models is a difficult task mainly due to the significant overparameterization exhibited by the problem and usually referred to as the "curse of dimensionality". For example, in the case of the VEC family, the number of parameters involved in the model grows as a polynomi…
Recent advances in policy gradient methods and deep learning have demonstrated their applicability for complex reinforcement learning problems. However, the variance of the performance gradient estimates obtained from the simulation is often excessive, leading to poor sample efficiency. In this paper, we apply the stoc…
Discovering the underlying physical behavior of complex systems is a crucial, but less well-understood topic in many engineering disciplines. This study proposes a finite-difference inspired convolutional neural network framework to learn hidden partial differential equations from given data and iteratively estimate fu…
Success conditioning optimizes policies by imitating successful trajectories, solving a trust-region optimization problem.
A new method improves SVI for high-dimensional, poorly-conditioned distributions.
An algorithm is proposed for solving stochastic and finite sum minimization problems. Based on a trust region methodology, the algorithm employs normalized steps, at least as long as the norms of the stochastic gradient estimates are within a specified interval. The complete algorithm---which dynamically chooses whethe…