arXiv research

Two new algorithms solve nonconvex-strongly concave problems efficiently.

TRSVR combines SVRG with trust-region for faster optimization.

Proposes a new algorithm for solving optimization problems with stochastic objectives and equality constraints.

Two multifidelity trust-region methods use low-fidelity models for efficient optimization.

We propose a trust region method for policy optimization that employs Quasi-Newton approximation for the Hessian, called Quasi-Newton Trust Region Policy Optimization QNTRPO. Gradient descent is the de facto algorithm for reinforcement learning tasks with continuous controls. The algorithm has achieved state-of-the-art…

TREGO improves EGO for global optimization of high-dimensional problems.

Mirror descent method improved RL algorithms.

Stochastic variational inference allows for fast posterior inference in complex Bayesian models. However, the algorithm is prone to local optima which can make the quality of the posterior approximation sensitive to the choice of hyperparameters and initialization. We address this problem by replacing the natural gradi…

Building upon the recent success of deep reinforcement learning methods, we investigate the possibility of on-policy reinforcement learning improvement by reusing the data from several consecutive policies. On-policy methods bring many benefits, such as ability to evaluate each resulting policy. However, they usually d…

JAXFit speeds up curve fitting on GPUs.

The paper analyzes and improves a deep learning optimization technique using matrix gradient orthogonality.

Bayesian optimization tackles constrained high-dimensional problems with penalties and trust regions.

Develops ODRPO to improve RL algorithms with better performance and stability.

New sampling strategy improves TR algorithms for stochastic optimization.

AdaScale-TuRBO improves high-dimensional Bayesian optimization by dynamically scaling the GP lengthscale.

We target the problem of finding a local minimum in non-convex finite-sum minimization. Towards this goal, we first prove that the trust region method with inexact gradient and Hessian estimation can achieve a convergence rate of order $\mathcal{O}(1/{k^{2/3}})$ as long as those differential estimations are sufficientl…

Reinforcement Learning(RL) with sparse rewards is a major challenge. We propose \emph{Hindsight Trust Region Policy Optimization}(HTRPO), a new RL algorithm that extends the highly successful TRPO algorithm with \emph{hindsight} to tackle the challenge of sparse rewards. Hindsight refers to the algorithm's ability to l…

TROLL improves RL for LLMs by replacing clipping with a trust region projection.

Simplified trust region method reduces representation change during fine-tuning.

MF-TRPO optimizes MFGs with finite sample guarantees.

New algorithm improves deep learning stability with limited data.

The paper introduces a method for fitting complex models using simulation and optimization.

We propose to improve trust region policy search with normalizing flows policy. We illustrate that when the trust region is constructed by KL divergence constraints, normalizing flows policy generates samples far from the 'center' of the previous policy iterate, which potentially enables better exploration and helps av…

Trust region policy optimization (TRPO) is a popular and empirically successful policy search algorithm in Reinforcement Learning (RL) in which a surrogate problem, that restricts consecutive policies to be 'close' to one another, is iteratively solved. Nevertheless, TRPO has been considered a heuristic algorithm inspi…

Proximal policy optimization (PPO) is one of the most successful deep reinforcement-learning methods, achieving state-of-the-art performance across a wide range of challenging tasks. However, its optimization behavior is still far from being fully understood. In this paper, we show that PPO could neither strictly restr…

An algorithm is proposed for solving stochastic and finite sum minimization problems. Based on a trust region methodology, the algorithm employs normalized steps, at least as long as the norms of the stochastic gradient estimates are within a specified interval. The complete algorithm---which dynamically chooses whethe…

Safe RL for autonomous vehicles using PCPO with trust regions and parallel learners.

New algorithms solve complex minimax problems efficiently.

New method improves Gaussian Mixture Model fitting speed.

Proximal policy optimization (PPO) is one of the most popular deep reinforcement learning (RL) methods, achieving state-of-the-art performance across a wide range of challenging tasks. However, as a model-free RL method, the success of PPO relies heavily on the effectiveness of its exploratory policy search. In this pa…

In data-limited settings, stochastic policies can outperform deterministic ones in bandit problems.

End-to-end pipeline for data-driven decision making in mixed-integer optimization.

Study shows code-level optimizations significantly impact deep RL algorithms.

We introduce TrustVI, a fast second-order algorithm for black-box variational inference based on trust-region optimization and the reparameterization trick. At each iteration, TrustVI proposes and assesses a step based on minibatches of draws from the variational distribution. The algorithm provably converges to a stat…

Trust is a collective, self-fulfilling phenomenon that suggests analogies with phase transitions. We introduce a stylized model for the build-up and collapse of trust in networks, which generically displays a first order transition. The basic assumption of our model is that whereas trust begets trust, panic also begets…

TRM improves long-horizon LLM RL by masking divergent sequences.

TRM improves long-horizon reinforcement learning for LLMs by masking divergent sequences.

Trust-region methods have yielded state-of-the-art results in policy search. A common approach is to use KL-divergence to bound the region of trust resulting in a natural gradient policy update. We show that the natural gradient and trust region optimization are equivalent if we use the natural parameterization of a st…

New method solves stochastic optimization problems with random models.

Paper presents a unique method to recover signals from their bispectrum.

New formulas for geodesics on Stiefel and flag manifolds using trust-region method.

Adaptive replication improves stochastic function optimization.

A new method for joint noise removal and trend estimation from sparse signals.

Improves inference-time alignment for diffusion models without updating weights.

Nowadays stochastic approximation methods are one of the major research direction to deal with the large-scale machine learning problems. From stochastic first order methods, now the focus is shifting to stochastic second order methods due to their faster convergence and availability of computing resources. In this pap…

VaR-CPO optimizes VaR-constrained RL problems with conservative policy updates.

New method solves optimization problems with stochastic objectives and constraints.

Paper enhances RL policies using trust region optimization for offline data.