Local Bayesian optimization shows strong performance and converges well, contrary to folklore.
problem Understanding the behavior and convergence of local Bayesian optimization methods.
method Studied the behavior of local optimization strategies and rigorously analyzed a specific algorithm.
result Local Bayesian optimization algorithms converge well and perform strongly, contrary to the folklore.
Modified K-means ensures local optimality with same complexity.
problem Lack of rigorous analysis on local optimality guarantees of K-means.
method Proposed modifications to K-means ensuring local optimality.
result Proposed methods provide improved locally optimal solutions.
Paper characterizes optimization landscape of Tucker decomposition.
problem Finding exact Tucker decomposition is a nonconvex optimization problem.
method Characterized the optimization landscape and provided a local search algorithm.
result All local minima are globally optimal if tensor has an exact Tucker decomposition.
Local SGD outperforms minibatch SGD for quadratic objectives.
problem Theoretical foundations of local SGD are lacking.
method Proved local SGD strictly dominates minibatch SGD for quadratic objectives and accelerated local SGD is minimax optimal.
result Local SGD does not dominate minibatch SGD in general convex objectives.
New insights into optimizing Local SGD's outer optimizer for faster convergence.
problem Understanding the impact of outer optimizer and its hyperparameters in Local SGD.
method Analyzing convergence guarantees with new outer learning rates and momentum.
result Tuning the outer learning rate can improve convergence and handle inner learning rate ill-tuning.
Optimize black-box simulators with local generative models.
problem Optimizing non-differentiable, stochastic simulators with intractable likelihoods.
method Differentiable local surrogate models based on deep generative models.
result Local surrogates enable gradient-based optimization, faster than baseline methods.
Adversarial online nonparametric regression achieves optimal rates with locally adaptive learning.
problem Adversarial online nonparametric regression with general convex losses.
method Parameter-free learning algorithm leveraging chaining trees to compete against H{ö}lder functions, dynamically tracking and adapting to local smoothness variations.
result First computationally efficient algorithm with locally adaptive optimal rates for online regression in an adversarial setting.
Distributed optimization often consists of two updating phases: local optimization and inter-node communication. Conventional approaches require working nodes to communicate with the server every one or few iterations to guarantee convergence. In this paper, we establish a completely different conclusion that each node…
HALO uses local Lipschitz constants to optimize functions efficiently.
problem Efficiently solving global optimization problems with complex objective functions.
method Hybrid Adaptive Lipschizian Optimization (HALO) algorithm that estimates local Lipschitz constants and balances global and local information.
result HALO outperforms other global optimization algorithms on numerous test functions.
Paper proposes a method for optimizing local policies for trajectory-centric reinforcement learning.
problem Challenges in global policy optimization for non-linear systems and poor performance of open-loop trajectory optimization.
method Formulates trajectory optimization and local policy synthesis as a single optimization problem and solves it as a nonlinear programming instance.
result Demonstrates improved performance of the proposed technique under simplifying assumptions.
We develop the first Bayesian Optimization algorithm, BLOSSOM, which selects between multiple alternative acquisition functions and traditional local optimization at each step. This is combined with a novel stopping condition based on expected regret. This pairing allows us to obtain the best characteristics of both lo…
LOAD discovers optimal adjustments locally for scalable causal inference.
problem Scalable causal inference for unknown causal graphs.
method Local Optimal Adjustments Discovery (LOAD) method.
result LOAD combines local and global approaches for efficient and accurate causal effect estimation.
LES optimizes designs by sampling descent sequences, achieving strong sample efficiency.
problem Optimizing large, complex design spaces is infeasible and unnecessary.
method LES uses Bayesian optimization to target solutions reachable by iterative optimizers.
result LES achieves strong sample efficiency compared to existing methods.
AGGLIO optimizes non-convex functions with local convexity guarantees.
problem Optimizing non-convex functions with local convexity.
method Stage-wise, graduated optimization technique for locally convex functions.
result Global convergence to the global optimum for non-convex and locally convex objectives.
New method escapes local optima in neural architecture optimization.
problem Escaping local optima in neural architecture optimization.
method Signed neural splitting in steepest descent framework.
result Escapes local optima, leading to better performance.
Optimizing proper loss yields calibrated models under specific conditions.
problem Understanding when optimizing proper loss functions leads to calibrated predictions.
method Local optimality condition and Lipschitz functions.
result Predictors with local optimality are nearly calibrated and nearly locally optimal.
Optimizers find approximate global minima in non-convex problems.
problem Understanding why local methods solve non-convex optimization problems.
method Formalizing the hypothesis that many local minima are approximately global minima.
result Most local minima of practical non-convex objectives are approximately global minima.
Optimizes bounds for threefold singularity volumes.
problem Bounding local volumes of threefold singularities.
method Analyzes Gorenstein canonical non-hypersurface threefold singularities.
result Establishes optimal upper bound for local volumes.
Global optimization finds applications in a wide range of real world problems. The multi-start methods are a popular class of global optimization techniques, which are based on the ideas of conducting local searches at multiple starting points. In this work we propose a new multi-start algorithm where the starting poin…
Optimal lower bounds for eigenvalues of Dirac-Witten operator on certain submanifolds.
problem Estimating eigenvalues of the Dirac-Witten operator on specific submanifolds.
method Optimal lower bounds derived using intrinsic and extrinsic expressions.
result Limiting-cases of eigenvalues studied and optimal bounds obtained.
Paper proposes faster method to find local minima in nonconvex optimization.
problem Escaping saddle points and finding local minima in nonconvex optimization.
method LENA (Last stEp shriNkAge) framework for faster perturbed stochastic gradient methods.
result LENA finds (ε,εH)-approximate local minima within ildeO(ε−3+εH−6) evaluations. Paper shows local SGD outperforms mini-batch SGD under certain conditions.
problem Proving local SGD's superiority in distributed learning with heterogeneous data.
method New lower and upper bounds for local SGD under first-order heterogeneity assumptions.
result Local SGD is min-max optimal under certain conditions, resolving understanding of distributed optimization.
Local adaptive methods in FL can accelerate convergence but introduce bias, which is corrected.
problem The effect of using adaptive optimization methods for local updates in federated learning.
method Proposed correction techniques to overcome the bias introduced by local adaptive methods.
result Correction techniques can achieve faster convergence and higher test accuracy than baseline methods.
Inverse optimal control, also known as inverse reinforcement learning, is the problem of recovering an unknown reward function in a Markov decision process from expert demonstrations of the optimal policy. We introduce a probabilistic inverse optimal control algorithm that scales gracefully with task dimensionality, an…
The article calculates the near horizon limit of Wang--Yau quasi-local mass.
problem Calculating the near horizon limit of quasi-local mass.
method Utilizing the properties of the mean curvature vector and optimal embedding equation.
result Existence and uniqueness of optimal embedding and continuity of quasi-local mass.
It is well-known that normal extremals in sub-Riemannian geometry are curves which locally minimize the energy functional. Most proofs of this fact do not make, however, an explicit use of relations between local optimality and the geometry of the problem. In this paper, we provide a new proof of that classical result,…
Novel method for high-dimensional BO using CMA to define local regions.
problem Challenges in applying BO to high-dimensional optimization problems.
method CMA strategy to learn search distribution and define local regions.
result Our method outperforms existing techniques on various benchmarks.
Minibatch SGD outperforms Local SGD in heterogeneous distributed learning.
problem Optimizing a combined convex objective with stochastic gradient estimates from different machines.
method Analysis of Minibatch SGD and Local SGD in a heterogeneous distributed setting.
result Minibatch SGD dominates Local SGD in the heterogeneous distributed setting.
Local PBO methods improve preferential BO in high-dimensional problems.
problem Efficiently optimizing preferential BO in high-dimensional settings.
method Adapting high-dimensional BO techniques to preferential feedback.
result Local PBO methods reduce cumulative regret compared to global baselines.
Combines global and local search for efficient global optimization with Gaussian processes.
problem Difficulties in building accurate GP models and getting stuck in suboptimal regions.
method Adopting AGLGP model combining global and local GP models, dividing space into regions, and switching between global and local searches.
result Efficiently locates the global optimum with benefits of both global and local search.
Lo-Hp decouples weight generation into local and global policies to improve flexibility and efficiency.
problem Over-coupling and long-horizon issues in current optimization methods.
method Hybrid-Policy Sub-Trajectory Balance objective.
result Learning local optimization policies addresses long-horizon issues and enhances global weight generation.
Study local control in a 7D quaternionic Heisenberg group.
problem Optimizing geodesics in a 7D quaternionic Heisenberg group.
method Matrix representation and analysis of sub-Riemannian structure symmetries.
result Impact of symmetries on geodesic optimality.
Improved local explainer aggregation for interpretable machine learning models.
problem Improving the interpretability of black box machine learning models.
method Non-convex optimization and integer optimization framework for local explainer aggregation.
result Our method outperforms existing methods in terms of coverage and fidelity, particularly in multi-class settings.
Proposes glocal hypergradient estimation for hyperparameter optimization.
problem Combining reliability and efficiency in hyperparameter optimization.
method Uses Koopman operator theory to approximate global hypergradients from local ones.
result Achieves both reliability and efficiency in hyperparameter optimization.
New method solves complex constrained optimization problems.
problem Constrained nonconvex-nonconcave minimax optimization problems.
method Inexact proximal gradient method using sequential convex programming.
result Established complexity guarantees for approximate stationary points.
Proposes a new optimization method for local graph clustering.
problem Local graph clustering and community detection.
method Convex optimization with p-norm network flow.
result Achieves quadratic approximation of conductance for p=2 and constant factor approximation for p→∞.
New insights into bias-variance tradeoff for data-driven optimization under local misspecification.
problem Understanding the relative performance of SAA, IEO, and ETO under local misspecification.
method Developed a local misspecification perspective using contiguity theory in statistics.
result Explicit expressions for decision bias and geometric understanding of variance.
Differentially private hyperparameter tuning improves privacy in machine learning.
problem Hyperparameter tuning leaks private information through selected configurations.
method Local Bayesian optimization using Gaussian Process surrogate for private gradient approximation.
result DP-GIBO converges to locally optimal hyperparameters with polynomial dimensional dependence.
BOinG optimizes HPO problems by focusing on promising local regions.
problem Expensive black-box optimization problems, especially hyperparameter optimization.
method Two-stage approach: global surrogate model followed by local model.
result BOinG exploits the structure of typical HPO problems and performs well on mid-sized problems.
Novel method for shape optimization of non-smooth PDEs.
problem Optimizing shapes governed by non-smooth PDEs.
method Functional variational approach and sensitivity analysis.
result Necessary conditions for locally optimal shapes.
LCBO tackles constrained optimization in high dimensions, offering a polynomial convergence rate.
problem Bayesian optimization for high-dimensional constrained problems.
method LCBO uses local descent and uncertainty-driven exploration, proving polynomial convergence rate.
result LCBO achieves a polynomial convergence rate for KKT residuals in high dimensions.
Gradients help find global optima in complex functions.
problem Finding global optima in functions with many local minima.
method A principle for generating search directions from non-local quadratic approximants based on gradients.
result The proposed algorithm and CMA-ES perform better than random reinitialized BFGS.
Develops a functional generalization of Eldan's stochastic localization for optimization and privacy.
problem Sampling under non-Euclidean geometries and optimization in differential privacy.
method Functional generalization of Eldan's stochastic localization, incorporating log-Laplace transform.
result Improves query complexities in zeroth-order differential private convex optimization.
Bayesian optimization is a sample-efficient method for finding a global optimum of an expensive-to-evaluate black-box function. A global solution is found by accumulating a pair of query point and its function value, repeating these two procedures: (i) modeling a surrogate function; (ii) maximizing an acquisition funct…
In federated distributed learning, the goal is to optimize a global training objective defined over distributed devices, where the data shard at each device is sampled from a possibly different distribution (a.k.a., heterogeneous or non i.i.d. data samples). In this paper, we generalize the local stochastic and full gr…
Paper develops a new local convexity condition for non-isolated minima in non-convex optimization.
problem Lack of theory for non-isolated minima in non-convex optimization.
method Formulates a new local convexity condition and studies SGD convergence under this condition.
result Shows SGD can converge locally under the new condition.
This paper analyzes saddle points and minimax points in non-convex smooth games.
problem Understanding local optimal points in non-convex smooth games.
method Comprehensive analysis of local minimax points, including their optimality conditions and stability.
result Local saddle points are uniformly local minimax points under mild continuity assumptions.
Gradient-based optimization methods are the most popular choice for finding local optima for classical minimization and saddle point problems. Here, we highlight a systemic issue of gradient dynamics that arise for saddle point problems, namely the presence of undesired stable stationary points that are no local optima…