arXiv research

Global optimization in Bayesian inference yields little additional benefit.

Paper proves global optimality of a simple optimization scheme for learning DAG models.

Single-timescale actor-critic finds globally optimal policy.

Meta algorithm solves multivariate optimization using univariate optimizers.

TREGO improves EGO for global optimization of high-dimensional problems.

Understanding the global optimality in deep learning (DL) has been attracting more and more attention recently. Conventional DL solvers, however, have not been developed intentionally to seek for such global optimality. In this paper we propose a novel approximation algorithm, BPGrad, towards optimizing deep models glo…

Improves Bayesian optimization using Gaussian process Thompson sampling.

HALO uses local Lipschitz constants to optimize functions efficiently.

New method for global optimization of Gaussian processes reduces computational time.

Combines global and local search for efficient global optimization with Gaussian processes.

New method optimizes Bayesian optimization for high-dimensional posterior samples.

Proposes glocal hypergradient estimation for hyperparameter optimization.

Optimizers find approximate global minima in non-convex problems.

Paper solves globally optimal k-means for low dimensional data.

Accelerates convergence in global non-convex optimization with reversible diffusion.

Global optimization finds applications in a wide range of real world problems. The multi-start methods are a popular class of global optimization techniques, which are based on the ideas of conducting local searches at multiple starting points. In this work we propose a new multi-start algorithm where the starting poin…

Postprocessing reduces Bayesian optimization steps for global optima.

Bayesian optimization has recently emerged as a popular method for the sample-efficient optimization of expensive black-box functions. However, the application to high-dimensional problems with several thousand observations remains challenging, and on difficult problems Bayesian optimization is often not competitive wi…

Policy gradient methods with actor-critic schemes demonstrate tremendous empirical successes, especially when the actors and critics are parameterized by neural networks. However, it remains less clear whether such "neural" policy gradient methods converge to globally optimal policies and whether they even converge at …

Bayesian optimization is a sample-efficient method for finding a global optimum of an expensive-to-evaluate black-box function. A global solution is found by accumulating a pair of query point and its function value, repeating these two procedures: (i) modeling a surrogate function; (ii) maximizing an acquisition funct…

ECP optimizes expensive functions without knowing Lipschitz constant.

Generative adversarial imitation learning (GAIL) demonstrates tremendous success in practice, especially when combined with neural networks. Different from reinforcement learning, GAIL learns both policy and reward function from expert (human) demonstration. Despite its empirical success, it remains unclear whether GAI…

Study optimal transport on globally hyperbolic spacetimes, focusing on weak Kantorovich potentials' regularity.

Gradient descent with preconditioning finds global optima in overparameterized nonconvex factorization.

Policy gradient converges to globally optimal policy in nearly linear-quadratic systems.

The ultimate goal of optimization is to find the minimizer of a target function.However, typical criteria for active optimization often ignore the uncertainty about the minimizer. We propose a novel criterion for global optimization and an associated sequential active learning strategy using Gaussian processes.Our crit…

We propose an SDP relaxation for the Gromov-Wasserstein distance, providing globally optimal solutions.

We study the error landscape of deep linear and nonlinear neural networks with the squared error loss. Minimizing the loss of a deep linear neural network is a nonconvex problem, and despite recent progress, our understanding of this loss surface is still incomplete. For deep linear networks, we present necessary and s…

Proximal policy optimization and trust region policy optimization (PPO and TRPO) with actor and critic parametrized by neural networks achieve significant empirical success in deep reinforcement learning. However, due to nonconvexity, the global convergence of PPO and TRPO remains less understood, which separates theor…

Softmax policy gradient achieves global optimality in wide neural networks with entropy regularization.

Develop intrinsic consensus-based optimization framework on Riemannian manifolds with bounded curvature.

ProGO optimizes non-convex functions without gradients, outperforming existing methods.

AGGLIO optimizes non-convex functions with local convexity guarantees.

Gradients help find global optima in complex functions.

New method trains quantized neural networks to global optimality.

Efficient algorithm for global optimization of multivariate Lipschitz functions.

We consider the problem of maximizing an unknown function over a compact and convex set using as few observations as possible. We observe that the optimization of the function essentially relies on learning the induced bipartite ranking rule of f. Based on this idea, we relate global optimization to bipartite ranking w…

Develops a novel global pooling framework using optimal transport.

We solve the optimization of two-layer ReLU networks using convex math.

SGD converges to global minimum for certain non-convex functions.

Global optimization problems whose objective function is expensive to evaluate can be solved effectively by recursively fitting a surrogate function to function samples and minimizing an acquisition function to generate new samples. The acquisition step trades off between seeking for a new optimization vector where the…

In this paper, a new sequential surrogate-based optimization (SSBO) algorithm is developed, which aims to improve the global search ability and local search efficiency for the global optimization of expensive black-box models. The proposed method involves three basic sub-criteria to infill new samples asynchronously to…

Optimized AIS scheme reduces bias and MSE for general proposals.

The paper connects PSO and CBO methods using stochastic modeling and mean-field limits.

Contemporary global optimization algorithms are based on local measures of utility, rather than a probability measure over location and value of the optimum. They thus attempt to collect low function values, not to learn about the optimum. The reason for the absence of probabilistic global optimizers is that the corres…

The paper guarantees global stability for stochastic subgradient methods in nonsmooth nonconvex optimization.

Defines globalness measure for explainers using optimal transport.

New framework for DNN training guarantees convergence to global minimum.