arXiv research

Two multifidelity trust-region methods use low-fidelity models for efficient optimization.

TRSVR combines SVRG with trust-region for faster optimization.

Bayesian optimization tackles constrained high-dimensional problems with penalties and trust regions.

Proposes a Quasi-Newton trust region method for policy optimization in reinforcement learning.

Simplified trust region method reduces representation change during fine-tuning.

AdaScale-TuRBO improves high-dimensional Bayesian optimization by dynamically scaling the GP lengthscale.

New algorithm finds local minima in non-convex problems efficiently.

Proposes a new algorithm for solving optimization problems with stochastic objectives and equality constraints.

TROLL improves RL for LLMs by replacing clipping with a trust region projection.

TRPO adapts trust region methods for faster RL convergence.

TREGO improves EGO for global optimization of high-dimensional problems.

Proximal policy optimization (PPO) is one of the most popular deep reinforcement learning (RL) methods, achieving state-of-the-art performance across a wide range of challenging tasks. However, as a model-free RL method, the success of PPO relies heavily on the effectiveness of its exploratory policy search. In this pa…

Two new algorithms solve nonconvex-strongly concave problems efficiently.

Stochastic variational inference allows for fast posterior inference in complex Bayesian models. However, the algorithm is prone to local optima which can make the quality of the posterior approximation sensitive to the choice of hyperparameters and initialization. We address this problem by replacing the natural gradi…

JAXFit speeds up curve fitting on GPUs.

The paper analyzes and improves a deep learning optimization technique using matrix gradient orthogonality.

Building upon the recent success of deep reinforcement learning methods, we investigate the possibility of on-policy reinforcement learning improvement by reusing the data from several consecutive policies. On-policy methods bring many benefits, such as ability to evaluate each resulting policy. However, they usually d…

New method solves stochastic optimization problems with random models.

TRM improves long-horizon LLM RL by masking divergent sequences.

Trust-region method improves Gaussian mixture models for complex distributions.

We propose to improve trust region policy search with normalizing flows policy. We illustrate that when the trust region is constructed by KL divergence constraints, normalizing flows policy generates samples far from the 'center' of the previous policy iterate, which potentially enables better exploration and helps av…

TRM improves long-horizon reinforcement learning for LLMs by masking divergent sequences.

Trust-region methods have yielded state-of-the-art results in policy search. A common approach is to use KL-divergence to bound the region of trust resulting in a natural gradient policy update. We show that the natural gradient and trust region optimization are equivalent if we use the natural parameterization of a st…

Enhanced PPO improves stability and performance.

FD-Net predicts future dynamics from data using Hessian-Free TRCG method.

Nowadays stochastic approximation methods are one of the major research direction to deal with the large-scale machine learning problems. From stochastic first order methods, now the focus is shifting to stochastic second order methods due to their faster convergence and availability of computing resources. In this pap…

Adaptive replication improves stochastic function optimization.

Adaptive norms improve neural network optimization with second-order methods.

New method solves optimization problems with stochastic objectives and constraints.

Mirror descent method improved RL algorithms.

New method improves Gaussian Mixture Model fitting speed.

New formulas for geodesics on Stiefel and flag manifolds using trust-region method.

Enhances BO in high dimensions with Newton methods.

New algorithm improves deep learning stability with limited data.

Improves inference-time alignment for diffusion models without updating weights.

End-to-end pipeline for data-driven decision making in mixed-integer optimization.

Develops ODRPO to improve RL algorithms with better performance and stability.

Deep Neural Networks are quite vulnerable to adversarial perturbations. Current state-of-the-art adversarial attack methods typically require very time consuming hyper-parameter tuning, or require many iterations to solve an optimization based adversarial attack. To address this problem, we present a new family of trus…

The paper introduces a method for fitting complex models using simulation and optimization.

Trust is a collective, self-fulfilling phenomenon that suggests analogies with phase transitions. We introduce a stylized model for the build-up and collapse of trust in networks, which generically displays a first order transition. The basic assumption of our model is that whereas trust begets trust, panic also begets…

Recent advances in policy gradient methods and deep learning have demonstrated their applicability for complex reinforcement learning problems. However, the variance of the performance gradient estimates obtained from the simulation is often excessive, leading to poor sample efficiency. In this paper, we apply the stoc…

HTRPO tackles sparse rewards in RL with improved stability and performance.

The estimation of multivariate GARCH time series models is a difficult task mainly due to the significant overparameterization exhibited by the problem and usually referred to as the "curse of dimensionality". For example, in the case of the VEC family, the number of parameters involved in the model grows as a polynomi…

A new method improves SVI for high-dimensional, poorly-conditioned distributions.

New sampling strategy improves TR algorithms for stochastic optimization.

Paper enhances RL policies using trust region optimization for offline data.

An algorithm is proposed for solving stochastic and finite sum minimization problems. Based on a trust region methodology, the algorithm employs normalized steps, at least as long as the norms of the stochastic gradient estimates are within a specified interval. The complete algorithm---which dynamically chooses whethe…

We consider variants of trust-region and cubic regularization methods for non-convex optimization, in which the Hessian matrix is approximated. Under mild conditions on the inexact Hessian, and using approximate solution of the corresponding sub-problems, we provide iteration complexity to achieve $ε$-approximate seco…