Combines global and local search for efficient global optimization with Gaussian processes.
problem Difficulties in building accurate GP models and getting stuck in suboptimal regions.
method Adopting AGLGP model combining global and local GP models, dividing space into regions, and switching between global and local searches.
result Efficiently locates the global optimum with benefits of both global and local search.
Global optimization finds applications in a wide range of real world problems. The multi-start methods are a popular class of global optimization techniques, which are based on the ideas of conducting local searches at multiple starting points. In this work we propose a new multi-start algorithm where the starting poin…
Local PBO methods improve preferential BO in high-dimensional problems.
problem Efficiently optimizing preferential BO in high-dimensional settings.
method Adapting high-dimensional BO techniques to preferential feedback.
result Local PBO methods reduce cumulative regret compared to global baselines.
In this paper, a new sequential surrogate-based optimization (SSBO) algorithm is developed, which aims to improve the global search ability and local search efficiency for the global optimization of expensive black-box models. The proposed method involves three basic sub-criteria to infill new samples asynchronously to…
BO limits search to lower dimensions with LGPR, improving efficiency and accuracy.
problem High computational costs in BO for high-dimensional optimization.
method Limits search region to lower dimensions and uses LGPR for local Gaussian process regression.
result Search efficiencies improved by about 69% and 40% for Ackley and Rosenbrock functions.
LSH methods extend to function spaces for efficient similarity search.
problem Efficient similarity search in function spaces.
method Locality-sensitive hashing (LSH) extended to Lp spaces using function approximation or Monte Carlo techniques. result An LSH family for Wasserstein distance over continuous probability distributions.
Unified framework for combinatorial and rounding algorithms in experimental design.
problem Designing and analyzing combinatorial and rounding algorithms for experimental design problems.
method Local search framework for combinatorial algorithms and regret minimization framework for rounding algorithms.
result Unified approach to match and improve all known results in D/A/E-design and obtain new results in unknown settings.
Improves diffusion model performance and efficiency through classical search.
problem Tackles inference-time control in diffusion models.
method Proposes a framework combining local and global search for efficient navigation.
result Significant gains in performance and efficiency across various domains.
Paper addresses linear regression with partially mismatched data using local search with theoretical guarantees.
problem Linear regression with partially mismatched data.
method Optimization formulation and greedy local search algorithm with theoretical guarantees.
result Local search algorithm converges to nearly-optimal solution at a linear rate under certain conditions.
Local search algorithms applied to optimization problems often suffer from getting trapped in a local optimum. The common solution for this deficiency is to restart the algorithm when no progress is observed. Alternatively, one can start multiple instances of a local search algorithm, and allocate computational resourc…
LES optimizes designs by sampling descent sequences, achieving strong sample efficiency.
problem Optimizing large, complex design spaces is infeasible and unnecessary.
method LES uses Bayesian optimization to target solutions reachable by iterative optimizers.
result LES achieves strong sample efficiency compared to existing methods.
Paper characterizes optimization landscape of Tucker decomposition.
problem Finding exact Tucker decomposition is a nonconvex optimization problem.
method Characterized the optimization landscape and provided a local search algorithm.
result All local minima are globally optimal if tensor has an exact Tucker decomposition.
Local search improves GFlowNets' ability to generate high-reward samples.
problem GFlowNets struggle with over-exploration in high-reward space.
method Local search focusing on high-reward samples via backtracking and reconstruction.
result Significant performance improvement in biochemical tasks.
Improved causal discovery methods for large graphs without strict assumptions.
problem Sub-optimal solutions due to faithfulness assumption violations.
method Super-structure estimation and local search strategies.
result The proposed method scales to hundreds of nodes with high accuracy.
LA-MCTS learns search space partition for black-box optimization using Monte Carlo Tree Search.
problem High-dimensional black-box optimization challenges.
method LA-MCTS recursively splits search space into regions with high/low function values, learns nonlinear partition and local models online.
result LA-MCTS achieves strong performance in black-box optimization and reinforcement learning benchmarks, especially for high-dimensional problems.
New method speeds up k-means clustering for large k by improving nearest-neighbor search.
problem Efficiently clustering large datasets with high-dimensional points.
method Seeded Approximate Nearest-Neighbor Search methods to improve Lloyd's algorithm.
result Significantly faster k-means clustering for large k values.
Hill-climbing is a powerful baseline for NAS, even with reduced noise.
problem Noise in evaluating neural architectures.
method Hill-climbing algorithm, denoising training pipeline.
result Hill-climbing outperforms state-of-the-art NAS algorithms with reduced noise.
XGES improves GES by favoring early edge deletion, outperforming GES in finite data settings.
problem Learning directed acyclic graphs from finite data.
method Extremely Greedy Equivalent Search (XGES) improves GES by favoring early edge deletion.
result XGES consistently outperforms GES in recovering the correct graphs, and is 10 times faster.
Novel method for high-dimensional BO using CMA to define local regions.
problem Challenges in applying BO to high-dimensional optimization problems.
method CMA strategy to learn search distribution and define local regions.
result Our method outperforms existing techniques on various benchmarks.
New local-search methods close the gap in sparse tensor PCA.
problem Sparse tensor PCA underperforms compared to other methods.
method Proposes new local-search methods including greedy and random-threshold variants.
result Proves local-search methods close the gap to best known polynomial-time procedures.
Bayesian optimisation tackles high-dimensional categorical and mixed search spaces.
problem Bayesian optimisation on high-dimensional categorical and mixed search spaces is challenging.
method Combining local optimisation with a tailored kernel design.
result Empirically outperforms current baselines in performance and computational costs.
Advocates a local feedback approach for RL in unknown systems.
problem Finding optimal feedback laws in unknown nonlinear dynamical systems.
method Searches over a local feedback representation consisting of an open-loop sequence and an optimal linear feedback law.
result Results in highly efficient training and superior performance compared to global methods.
MBExplainer provides explanations for models combining graph embeddings and tabular features.
problem Explaining models using a mix of graph embeddings and tabular features.
method Model-agnostic approach using Shapley values and Monte Carlo Tree Search.
result MBExplainer efficiently finds human-readable explanations for model predictions.
Inverted file and asymmetric distance computation (IVFADC) have been successfully applied to approximate nearest neighbor search and subsequently maximum inner product search. In such a framework, vector quantization is used for coarse partitioning while product quantization is used for quantizing residuals. In the ori…
Study quantifies how COVID-19 spread affects US stock markets.
problem Impact of COVID-19 on US stock market during pandemic.
method Developed a novel temporal complex network approach using econometric and ML models.
result Local spread of COVID-19 and Google searches impact abnormal stock prices.
Gradients help find global optima in complex functions.
problem Finding global optima in functions with many local minima.
method A principle for generating search directions from non-local quadratic approximants based on gradients.
result The proposed algorithm and CMA-ES perform better than random reinitialized BFGS.
Addressing the issue of SVMs parameters optimization, this study proposes an efficient memetic algorithm based on Particle Swarm Optimization algorithm (PSO) and Pattern Search (PS). In the proposed memetic algorithm, PSO is responsible for exploration of the search space and the detection of the potential regions with…
Globally normalized neural sequence models are considered superior to their locally normalized equivalents because they may ameliorate the effects of label bias. However, when considering high-capacity neural parametrizations that condition on the whole input sequence, both model classes are theoretically equivalent in…
The search space of Bayesian Network structures is usually defined as Acyclic Directed Graphs (DAGs) and the search is done by local transformations of DAGs. But the space of Bayesian Networks is ordered by DAG Markov model inclusion and it is natural to consider that a good search policy should take this into account.…
Graph Neural Networks and Guided Local Search improve TSP solutions.
problem Finding optimal solutions to the Traveling Salesperson Problem quickly.
method Hybrid approach combining Graph Neural Networks and Guided Local Search.
result Significant reduction in optimality gap for TSP solutions.
We consider the task of mapping pseudocode to long programs that are functionally correct. Given test cases as a mechanism to validate programs, we search over the space of possible translations of the pseudocode to find a program that passes the validation. However, without proper credit assignment to localize the sou…
Differentiable NAS frameworks grow networks wider and deeper, revealing biases in wiring evolution.
problem Understanding the evolution of neural architecture wiring in differentiable NAS methods.
method Unified view on searching algorithms, local cost minimization, empirical and theoretical analyses.
result Implicit inductive biases cause observed searching patterns in differentiable NAS methods.
FLOP algorithm speeds up causal structure learning for linear models.
problem Efficiently learning causal structures from discrete data.
method FLOP algorithm combines fast parent selection and iterative score updates.
result FLOP finds highly accurate causal structures with near-perfect recovery.
Recently similarity graphs became the leading paradigm for efficient nearest neighbor search, outperforming traditional tree-based and LSH-based methods. Similarity graphs perform the search via greedy routing: a query traverses the graph and in each vertex moves to the adjacent vertex that is the closest to this query…
Bayesian optimization offers a flexible framework to optimize an objective function that is expensive to be evaluated. A Bayesian optimizer iteratively queries the function values on its carefully selected points. Subsequently, it makes a sensible recommendation about where the optimum locates based on its accumulated …
Gradient-based methods are often used for policy optimization in deep reinforcement learning, despite being vulnerable to local optima and saddle points. Although gradient-free methods (e.g., genetic algorithms or evolution strategies) help mitigate these issues, poor initialization and local optima are still concerns …
This paper proposes and evaluates the k-greedy equivalence search algorithm (KES) for learning Bayesian networks (BNs) from complete data. The main characteristic of KES is that it allows a trade-off between greediness and randomness, thus exploring different good local optima. When greediness is set at maximum, KES co…
Improves BN graph learning with splines for scalability.
problem Learning accurate BN graph structures from data.
method Score-and-search approach with MARS for CPD modeling.
result Improves BN graph accuracy and scalability.
CAGES optimizes expensive RL problems by efficiently learning gradients from multiple sources.
problem Optimizing expensive-to-evaluate functions in high-dimensional spaces.
method Cost-Aware Gradient Entropy Search (CAGES) for multi-fidelity Bayesian optimization.
result Significant performance improvements on synthetic and RL benchmark problems.
Differentially private hyperparameter tuning improves privacy in machine learning.
problem Hyperparameter tuning leaks private information through selected configurations.
method Local Bayesian optimization using Gaussian Process surrogate for private gradient approximation.
result DP-GIBO converges to locally optimal hyperparameters with polynomial dimensional dependence.
Two novel search strategies reduce complexity for target localization with size-dependent noise.
problem Target localization with varying measurement noise based on query region size.
method Proposes dyaPM and hiePM strategies with low complexity and connected query geometry. result Unified analysis shows dyaPM asymptotically optimal in search time, hiePM near-optimal in rate. A real-time federated neural architecture search approach reduces costs and improves performance.
problem High communication and computational demands in federated learning for large models.
method Evolutionary approach with double-sampling technique to optimize model performance and reduce costs.
result Effective real-time federated neural architecture search for deep models on edge devices.
Scientific discovery is limited by hypothesis redundancy, and hybrid methods can exploit non-local exploration.
problem Limitation of scientific discovery due to hypothesis redundancy.
method Hybrid discovery systems combining structured local search with LLM-generated non-local proposals.
result Hybrid methods can exploit non-local exploration when three geometric conditions co-occur.
This work proposes searching for optimal operation distribution in neural architecture search.
problem Finding optimal neural architecture with specific operations and connections.
method Search for the optimal operation distribution, providing a stochastic and approximate solution.
result Operation distribution holds enough discriminating power to reliably identify a solution and is easier to optimise than traditional encodings.
Quantum algorithm improves portfolio construction accuracy.
problem Efficiently constructing portfolios with real-world constraints.
method Sampling-based CVaR Variational Quantum Algorithm (VQA) combined with local-search post-processing.
result Achieved a relative solution error of 0.49% on IBM Heron processors.
Bayesian optimization improves policy search in reinforcement learning.
problem Finding optimal policies with high variance estimates from random samples.
method Develops an algorithm combining Bayesian optimization and policy gradients.
result Improves sample complexity and reduces variance in empirical evaluations.
The paper refines NOTEARS for learning Bayesian networks, improving accuracy and efficiency.
problem Learning Bayesian networks from continuous optimization.
method Generalized algebraic characterizations and Karush-Kuhn-Tucker (KKT) conditions for optimization.
result Local search post-processing improves structural Hamming distance by a factor of 2 or more.
Optimizes neural architecture search to generate novel lightweight models.
problem Over-reliance on expert knowledge limits NAS to local optima, preventing architectural breakthroughs.
method Casts NAS as an optimization problem, introduces a hierarchical graph-based search space, and uses Bayesian optimization.
result Generates extremely lightweight yet competitive models on six benchmark datasets.