arXiv research

Two multifidelity trust-region methods use low-fidelity models for efficient optimization.

TRSVR combines SVRG with trust-region for faster optimization.

Simplified trust region method reduces representation change during fine-tuning.

Bayesian optimization tackles constrained high-dimensional problems with penalties and trust regions.

AdaScale-TuRBO improves high-dimensional Bayesian optimization by dynamically scaling the GP lengthscale.

We propose a trust region method for policy optimization that employs Quasi-Newton approximation for the Hessian, called Quasi-Newton Trust Region Policy Optimization QNTRPO. Gradient descent is the de facto algorithm for reinforcement learning tasks with continuous controls. The algorithm has achieved state-of-the-art…

Proximal policy optimization (PPO) is one of the most popular deep reinforcement learning (RL) methods, achieving state-of-the-art performance across a wide range of challenging tasks. However, as a model-free RL method, the success of PPO relies heavily on the effectiveness of its exploratory policy search. In this pa…

TROLL improves RL for LLMs by replacing clipping with a trust region projection.

Proposes a new algorithm for solving optimization problems with stochastic objectives and equality constraints.

Stochastic variational inference allows for fast posterior inference in complex Bayesian models. However, the algorithm is prone to local optima which can make the quality of the posterior approximation sensitive to the choice of hyperparameters and initialization. We address this problem by replacing the natural gradi…

JAXFit speeds up curve fitting on GPUs.

TREGO improves EGO for global optimization of high-dimensional problems.

Building upon the recent success of deep reinforcement learning methods, we investigate the possibility of on-policy reinforcement learning improvement by reusing the data from several consecutive policies. On-policy methods bring many benefits, such as ability to evaluate each resulting policy. However, they usually d…

The paper analyzes and improves a deep learning optimization technique using matrix gradient orthogonality.

We target the problem of finding a local minimum in non-convex finite-sum minimization. Towards this goal, we first prove that the trust region method with inexact gradient and Hessian estimation can achieve a convergence rate of order $\mathcal{O}(1/{k^{2/3}})$ as long as those differential estimations are sufficientl…

Two new algorithms solve nonconvex-strongly concave problems efficiently.

New method solves stochastic optimization problems with random models.

TRM improves long-horizon LLM RL by masking divergent sequences.

TRM improves long-horizon reinforcement learning for LLMs by masking divergent sequences.

Nowadays stochastic approximation methods are one of the major research direction to deal with the large-scale machine learning problems. From stochastic first order methods, now the focus is shifting to stochastic second order methods due to their faster convergence and availability of computing resources. In this pap…

New method solves optimization problems with stochastic objectives and constraints.

Adaptive replication improves stochastic function optimization.

We propose to improve trust region policy search with normalizing flows policy. We illustrate that when the trust region is constructed by KL divergence constraints, normalizing flows policy generates samples far from the 'center' of the previous policy iterate, which potentially enables better exploration and helps av…

New method improves Gaussian Mixture Model fitting speed.

Mirror descent method improved RL algorithms.

Enhances BO in high dimensions with Newton methods.

New formulas for geodesics on Stiefel and flag manifolds using trust-region method.

New algorithm improves deep learning stability with limited data.

Improves inference-time alignment for diffusion models without updating weights.

Deep Neural Networks are quite vulnerable to adversarial perturbations. Current state-of-the-art adversarial attack methods typically require very time consuming hyper-parameter tuning, or require many iterations to solve an optimization based adversarial attack. To address this problem, we present a new family of trus…

Develops ODRPO to improve RL algorithms with better performance and stability.

Proximal policy optimization (PPO) is one of the most successful deep reinforcement-learning methods, achieving state-of-the-art performance across a wide range of challenging tasks. However, its optimization behavior is still far from being fully understood. In this paper, we show that PPO could neither strictly restr…

Recent advances in policy gradient methods and deep learning have demonstrated their applicability for complex reinforcement learning problems. However, the variance of the performance gradient estimates obtained from the simulation is often excessive, leading to poor sample efficiency. In this paper, we apply the stoc…

End-to-end pipeline for data-driven decision making in mixed-integer optimization.

Trust region policy optimization (TRPO) is a popular and empirically successful policy search algorithm in Reinforcement Learning (RL) in which a surrogate problem, that restricts consecutive policies to be 'close' to one another, is iteratively solved. Nevertheless, TRPO has been considered a heuristic algorithm inspi…

The estimation of multivariate GARCH time series models is a difficult task mainly due to the significant overparameterization exhibited by the problem and usually referred to as the "curse of dimensionality". For example, in the case of the VEC family, the number of parameters involved in the model grows as a polynomi…

A new method improves SVI for high-dimensional, poorly-conditioned distributions.

Discovering the underlying physical behavior of complex systems is a crucial, but less well-understood topic in many engineering disciplines. This study proposes a finite-difference inspired convolutional neural network framework to learn hidden partial differential equations from given data and iteratively estimate fu…

Paper enhances RL policies using trust region optimization for offline data.

We consider variants of trust-region and cubic regularization methods for non-convex optimization, in which the Hessian matrix is approximated. Under mild conditions on the inexact Hessian, and using approximate solution of the corresponding sub-problems, we provide iteration complexity to achieve $ε$-approximate seco…

The paper introduces a method for fitting complex models using simulation and optimization.

An algorithm is proposed for solving stochastic and finite sum minimization problems. Based on a trust region methodology, the algorithm employs normalized steps, at least as long as the norms of the stochastic gradient estimates are within a specified interval. The complete algorithm---which dynamically chooses whethe…

We investigate the use of regularized Newton methods with adaptive norms for optimizing neural networks. This approach can be seen as a second-order counterpart of adaptive gradient methods, which we here show to be interpretable as first-order trust region methods with ellipsoidal constraints. In particular, we prove …

Trust-region methods have yielded state-of-the-art results in policy search. A common approach is to use KL-divergence to bound the region of trust resulting in a natural gradient policy update. We show that the natural gradient and trust region optimization are equivalent if we use the natural parameterization of a st…

VaR-CPO optimizes VaR-constrained RL problems with conservative policy updates.

New sampling strategy improves TR algorithms for stochastic optimization.

The goal of tensor completion is to fill in missing entries of a partially known tensor (possibly including some noise) under a low-rank constraint. This may be formulated as a least-squares problem. The set of tensors of a given multilinear rank is known to admit a Riemannian manifold structure, thus methods of Rieman…

A new method for joint noise removal and trend estimation from sparse signals.