The paper tackles denoising of function samples modulo 1.
problem Recover smooth estimates of a function's modulo 1 samples from noisy data.
method Formulates and solves a quadratically constrained quadratic program relaxation.
result Demonstrates robustness of the approach to noise.
New methods for non-convex optimization using inexact Hessian approximations.
problem Optimization of non-convex functions with inexact Hessian information.
method Trust-region and cubic regularization methods with inexact Hessian approximations.
result Iteration complexity to achieve ε-approximate second-order optimality.
Estimates smooth function modulo 1 samples robustly from noisy data.
problem Estimating smooth function modulo 1 samples from noisy mod 1 samples.
method Formulates and solves a smoothness regularized least-squares problem over the unit circle.
result Proves robustness to noise for adversarial, Gaussian, and Bernoulli noise models.
Two multifidelity trust-region methods use low-fidelity models for efficient optimization.
problem Efficiently solving complex optimization problems with limited data.
method Sketched Trust-Region (STR) and SVD Trust-Region (SVDTR) methods using low-fidelity models.
result Potential gain in efficiency demonstrated through numerical examples.
BMM algorithm improves convergence for nonconvex optimization problems.
problem Constrained nonsmooth nonconvex optimization problems.
method Block majorization-minimization with diminishing radius.
result Improved convergence rate for nonconvex optimization problems.
Normalizing flows policy improves trust region policy optimization.
problem Improving exploration and avoiding local optima in policy optimization.
method Constructing trust region with KL divergence constraints and using normalizing flows policy.
result Normalizing flows policy significantly improves policy optimization, especially on high-dimensional tasks.
TRSVR combines SVRG with trust-region for faster optimization.
problem Unconstrained nonconvex optimization problems.
method Adaptive stochastic trust-region method with variance reduction.
result Converges to first-order stationary points with SVRG.
Proposes a Quasi-Newton trust region method for policy optimization in reinforcement learning.
problem Lack of stepsize selection criterion and slow convergence in gradient descent for policy optimization.
method Uses a trust region method with Quasi-Newton approximation for the Hessian.
result Demonstrates improved performance and efficiency in continuous control tasks.
Two new algorithms solve nonconvex-strongly concave problems efficiently.
problem Solving nonconvex-strongly concave minimax problems.
method Proposed MINIMAX-TR and MINIMAX-TRACE algorithms.
result Find ( ε , ε ) (ε, \sqrtε) ( ε , ε ) -second order stationary points within O ( ε − 1.5 ) \mathcal{O}(ε^{-1.5}) O ( ε − 1.5 ) iterations. AdaScale-TuRBO improves high-dimensional Bayesian optimization by dynamically scaling the GP lengthscale.
problem Inappropriate lengthscale design in TuRBO's local GP model causes suboptimal performance in high dimensions.
method Proposes AdaScale-TuRBO, which scales the GP lengthscale with both problem dimension and trust region size.
result AdaScale-TuRBO robustly outperforms standard TuRBO and other methods on synthetic and real-world tasks.
Proposes a new algorithm for solving optimization problems with stochastic objectives and equality constraints.
problem Optimization problems with stochastic objectives and deterministic equality constraints.
method Trust-region stochastic sequential quadratic programming (TR-StoSQP) with adaptive relaxation techniques.
result Established a global almost sure convergence guarantee for TR-StoSQP.
TROLL improves RL for LLMs by replacing clipping with a trust region projection.
problem Clipping in RL for LLMs causes instability and suboptimal performance.
method TROLL uses a discrete differentiable trust region projection to replace clipping, balancing computational cost and effectiveness.
result TROLL consistently outperforms PPO-like clipping in training speed, stability, and final success rates.
Bayesian optimization tackles constrained high-dimensional problems with penalties and trust regions.
problem Constrained optimization in high-dimensional black-box settings with expensive evaluations and complex feasibility regions.
method Penalty formulation, surrogate model, trust region strategy, Expected Improvement acquisition function.
result The proposed Trust Region method identifies high-quality feasible solutions with fewer evaluations and maintains stable performance.
Simplified trust region method reduces representation change during fine-tuning.
problem Stability and representational collapse in fine-tuning pre-trained models.
method Replaces adversarial objectives with parametric noise in trust region theory.
result Matches or exceeds previous trust region methods in performance and speed.
TREGO improves EGO for global optimization of high-dimensional problems.
problem Efficient Global Optimization struggles with high dimensions and lacks theoretical guarantees.
method TREGO alternates between EGO steps and local steps within a trust region.
result TREGO outperforms EGO and other methods in black-box optimization problems.
TRPO adapts trust region methods for faster RL convergence.
problem Improving RL convergence rates in regularized MDPs.
method Adaptive scaling in TRPO for faster convergence rates.
result First RL result showing faster rates with regularization.
TRGPPO improves PPO's exploration, leading to better performance.
problem PPO's exploration is insufficient, especially with bad initialization.
method TRGPPO uses a trust region to adaptively adjust the clipping range.
result TRGPPO achieves better performance bounds and improved exploration.
New algorithm finds local minima in non-convex problems efficiently.
problem Finding local minima in non-convex finite-sum minimization problems.
method Stochastic Trust Region (STR) algorithm combining inexact gradient and Hessian estimation.
result STR finds ( ε , ε ) (ε, \sqrtε) ( ε , ε ) -approximate local minimum with improved efficiency. JAXFit speeds up curve fitting on GPUs.
problem Nonlinear least squares curve fitting problems.
method Trust region method on GPU with automatic differentiation.
result Significantly faster than CPU and other GPU libraries.
Improves on-policy RL by reusing data from multiple policies.
problem Lack of reuse of data from previous policies in on-policy RL.
method Adapts replay buffer concept to combine on- and off-policy learning.
result Method outperforms state-of-the-art on-policy RL algorithms.
Efficient trust region based attacks reduce adversarial perturbation time significantly.
problem Vulnerability of neural networks to adversarial perturbations requiring time-consuming tuning.
method Trust region optimization for efficient adversarial attack computation.
result Significant speed up of up to 37 times for VGG-16 model on a Titan Xp GPU.
Trust-region methods and natural gradients are equivalent in certain policy search scenarios.
problem Improving policy search methods in continuous control tasks.
method Introducing compatible policy search (COPOS) that uses natural parameterization and compatible value function approximation to control entropy loss.
result COPOS yields state-of-the-art results in challenging tasks and reduces entropy loss.
Stochastic variational inference allows for fast posterior inference in complex Bayesian models. However, the algorithm is prone to local optima which can make the quality of the posterior approximation sensitive to the choice of hyperparameters and initialization. We address this problem by replacing the natural gradi…
Chunking is a significant CL problem, accounting for half of performance drop, and current methods don't address it.
problem Chunking of data in continual learning.
method Analyzing and addressing the chunking sub-problem in continual learning.
result Current CL algorithms perform poorly on chunking, only as well as plain SGD training when there is no distribution shift.
The paper analyzes and improves a deep learning optimization technique using matrix gradient orthogonality.
problem Improving deep learning training through more effective optimization methods.
method Develops a stochastic non-Euclidean trust-region gradient method for deep learning optimization.
result Proves state-of-the-art convergence results for the proposed algorithm in various scenarios.
This work simplifies IRL by using potential-based reward shaping.
problem Computational inefficiency in inverse reinforcement learning.
method Potential-based reward shaping to reduce RL sub-problems.
result Reduces computational burden of inverse reinforcement learning.
Proposes STRON method for large-scale machine learning problems.
problem Large-scale machine learning problems.
method Stochastic Trust Region Inexact Newton (STRON) method using CG to solve trust region subproblem with progressive subsampling.
result Empirical results show efficacy of STRON method.
TRM improves long-horizon LLM RL by masking divergent sequences.
problem Long-horizon reinforcement learning with LLMs suffers from off-policy mismatch and approximation errors.
method Derives and applies trust region bounds to control divergence, proposing Trust Region Masking.
result First non-vacuous monotonic improvement guarantees for long-horizon LLM-RL.
TRM improves long-horizon reinforcement learning for LLMs by masking divergent sequences.
problem Long-horizon reinforcement learning for LLMs suffers from off-policy mismatch and approximation errors.
method Derives and applies trust region bounds to control divergence, proposing Trust Region Masking.
result First non-vacuous monotonic improvement guarantees for long-horizon LLM-RL.
HTRPO tackles sparse rewards in RL with improved stability and performance.
problem Sparse rewards in reinforcement learning.
method HTRPO extends TRPO with hindsight and QKL for better policy update stability.
result HTRPO consistently outperforms TRPO and HPG in various sparse reward tasks.
Improved sample efficiency in policy gradient methods using SVRG.
problem Excessive variance in policy gradient estimates.
method Applying SVRG to model-free policy gradient methods.
result Significantly better performance on robotic control tasks.
New trust-region algorithms improve ML performance with indefinite Hessians.
problem Training machine learning models with indefinite Hessians.
method Quasi-Newton trust-region framework for indefinite Hessians.
result Proposed methods outperform traditional BFGS and Hessian-free methods.
Enhanced PPO improves stability and performance.
problem Performance instability in PPO.
method Introduces Truly PPO with new clipping function and trust region constraint.
result Improves sample efficiency and performance.
New method solves stochastic optimization problems with random models.
problem Optimizing stochastic objectives with deterministic constraints.
method Trust-Region Sequential Quadratic Programming with random model.
result Global convergence guarantees for first- and second-order stationary points.
Mirror descent method improved RL algorithms.
problem Improving RL algorithms for better performance.
method Mirror descent method applied to RL, solving trust-region problems.
result MDPO outperforms or matches other RL algorithms in continuous control tasks.
The paper analyzes inexact variants of iterative methods for solving optimization problems.
problem Solving optimization problems with inexact sub-problems.
method Inexact variants of stochastic gradient descent, Newton, proximal point, and subspace ascent methods.
result Iteration complexity results for inexact variants of various iterative methods.
Algorithm improves stochastic gradient optimization with normalized steps.
problem Stochastic and finite sum minimization problems.
method Trust region algorithm with normalized steps.
result Algorithm converges similarly to traditional stochastic gradient under certain conditions.
New sampling strategy improves TR algorithms for stochastic optimization.
problem Derivative-free stochastic optimization with Monte Carlo estimates.
method Stratified adaptive sampling to optimize MC sample size.
result Reduced sample complexity and superior efficiency confirmed.
FD-Net predicts future dynamics from data using Hessian-Free TRCG method.
problem Discovering hidden partial differential equations from data.
method Finite-difference inspired convolutional neural network with Hessian-Free TRCG method.
result FD-Net predicts future dynamics efficiently using few trainable parameters.
New formulas for geodesics on Stiefel and flag manifolds using trust-region method.
problem Computing geodesics and logarithms on Stiefel and flag manifolds.
method Closed-form geodesic formulas, trust-region solver, Fréchet derivatives.
result Efficient computation of geodesic distance and logarithm map.
A meta-learning approach improves the performance of alternating minimization for non-convex optimization problems.
problem Optimizing non-convex problems with multiple variables using alternating minimization.
method Meta-learning based alternating minimization (MLAM) to replace handcrafted updating rules.
result The proposed MLAM method outperforms traditional AM-based methods in various non-convex optimization problems.
Develops ODRPO to improve RL algorithms with better performance and stability.
problem RL algorithms converge to sub-optimal solutions due to limited policy representation.
method Integrates DRO approach to solve trust region constrained optimization problem without parameterizing policies.
result Achieves globally optimal policy update and higher sample efficiency.
Trust-region method improves Gaussian mixture models for complex distributions.
problem Learning accurate approximations of complex, multimodal distributions.
method Information-geometric trust regions for principled exploration, lower bound optimization, online component adaptation.
result Improved GMM approximations with better quality and efficiency.
Adaptive replication improves stochastic function optimization.
problem Challenges in accurately estimating functions with high variance.
method Trust-region-based Bayesian optimization with adaptive replication.
result Adaptive replication substantially improves solution accuracy and efficiency.
Many problems in sequential decision making and stochastic control often have natural multiscale structure: sub-tasks are assembled together to accomplish complex goals. Systematically inferring and leveraging hierarchical structure, particularly beyond a single level of abstraction, has remained a longstanding challen…
MF-TRPO optimizes MFGs with finite sample guarantees.
problem Computing approximate Nash equilibria in MFGs.
method Extends TRPO to MFGs, providing convergence guarantees.
result Theoretical guarantees on MF-TRPO's convergence.
New algorithm improves deep learning stability with limited data.
problem Stability and robustness in reinforcement learning with scarce data.
method Uncertainty-aware trust region approach to policy optimization.
result Stable policy updates adapt to uncertainty levels during learning.
Improves inference-time alignment for diffusion models without updating weights.
problem Aligning diffusion models without updating weights for high-reward outputs.
method Trust-Region Iterative Twisted Sequential Monte Carlo (TRI-TSMC) for variance reduction and efficiency.
result Improves primary alignment objectives on text generation tasks.