Bayesian optimization for multi-objective and multi-point search.
problem Efficiently optimize unknown multi-objective functions with high evaluation costs.
method Proposes a Bayesian optimization algorithm that considers both multi-objective and multi-point search problems, using an acquisition function and an efficient gradient estimation method.
result Shows comparable or superior performance to heuristic methods via numerical experiments.
Multi-objective optimization aims at finding trade-off solutions to conflicting objectives. These constitute the Pareto optimal set. In the context of expensive-to-evaluate functions, it is impossible and often non-informative to look for the entire set. As an end-user would typically prefer a certain part of the objec…
Optimizes neural networks training with multi-point optimization.
problem Training efficiency and loss landscape diversity of neural networks.
method Multi-point optimization technique for simultaneous training of multiple models.
result Loss landscapes of neural networks are diverse and intricate, and batch normalization makes them smoother.
Deep neural networks' loss surfaces contain every low-dimensional pattern.
problem Finding arbitrary low-dimensional patterns in neural network loss surfaces.
method Empirical and theoretical analysis of loss landscapes of deep neural networks.
result Deep universal approximators exhibit a property where arbitrary smooth patterns exist in their loss surfaces.
mlr3mbo is a modular R toolbox for Bayesian optimization.
problem Efficiently solving optimization problems with multiple objectives and constraints.
method Bayesian optimization with support for multi-objective, multi-point proposals, parallelization, and custom algorithms.
result mlr3mbo performs competitively with state-of-the-art optimizers and robustly handles various optimization regimes.
A new Latent Diffusion Model generates realistic reservoir facies.
problem Creating accurate reservoir facies from limited measurements.
method Proposes a Latent Diffusion Model for conditional facies generation.
result Significantly outperforms GAN-based alternatives in fidelity and realism.
Efficient boosting method for regression with limited feedback.
problem Online boosting for regression tasks with noisy multi-point bandit feedback.
method Efficient regret minimization method with online boosting algorithm and projection-free online convex optimization.
result Improved state-of-the-art guarantees in efficiency.
A key drawback of the current generation of artificial decision-makers is that they do not adapt well to changes in unexpected situations. This paper addresses the situation in which an AI for aerial dog fighting, with tunable parameters that govern its behavior, will optimize behavior with respect to an objective func…
We present a braid-theoretic approach to combinatorially computing knot Floer homology. To a knot or link K, which is braided about the standard disk open book decomposition for (S^3,ξ_std), we associate a corresponding multi-pointed nice Heegaard diagram. We then describe an explicit algorithm for computing the associ…
New dynamics for SGD in small learning rate regime.
problem Improving stochastic gradient descent in small learning rate regime.
method Introducing stochastic modified flows and distribution dependent stochastic modified flows.
result Captures fluctuating dynamics of SGD in small learning rate - infinite width scaling regime.
We show that all versions of Heegaard Floer homology, link Floer homology, and sutured Floer homology are natural. That is, they assign concrete groups to each based 3-manifold, based link, and balanced sutured manifold, respectively. Furthermore, we functorially assign isomorphisms to (based) diffeomorphisms, and show…
This work improves molecular design by efficiently selecting diverse candidate molecules.
problem Designing molecules that satisfy multiple conflicting objectives.
method A modular 'generate-then-optimize' framework using generative models and a novel acquisition function.
result Significant improvements in sample efficiency across synthetic and application-driven tasks.
Jets from boosted heavy particles have a typical angular scale which can be used to distinguish them from QCD jets. We introduce a machine learning strategy for jet substructure analysis using a spectral function on the angular scale. The angular spectrum allows us to scan energy deposits over the angle between a pair …
Paper analyzes algorithms for nonstationary saddle-point optimization problems.
problem Nonstationary saddle-point optimization problems in game theory, reinforcement learning, and machine learning.
method Proposes extragradient and Frank-Wolfe algorithms for online and bandit settings.
result Establishes sub-linear regret bounds for the proposed algorithms.
We present mlrMBO, a flexible and comprehensive R toolbox for model-based optimization (MBO), also known as Bayesian optimization, which addresses the problem of expensive black-box optimization by approximating the given objective function through a surrogate regression model. It is designed for both single- and multi…
Unlike other industries in which intellectual property is patentable, the financial industry relies on trade secrecy to protect its business processes and methods, which can obscure critical financial risk exposures from regulators and the public. We develop methods for sharing and aggregating such risk exposures that …
This work studies how an AI-controlled dog-fighting agent with tunable decision-making parameters can learn to optimize performance against an intelligent adversary, as measured by a stochastic objective function evaluated on simulated combat engagements. Gaussian process Bayesian optimization (GPBO) techniques are dev…
Paper tackles LDP bandits learning with improved results and sub-linear regret.
problem Contextual bandits learning with LDP privacy constraints.
method Simple black-box reduction frameworks for context-free bandits, extended to GLB.
result First result for BCO with multi-point feedback under LDP, sub-linear regret for GLB.
New knot homology theory from symplectic geometry.
problem Developing a symplectic counterpart to instanton knot homology.
method Using symplectic character varieties and Floer homology, a new invariant for knots in 3-manifolds is constructed.
result Symplectic instanton knot homology (SIK) is a new invariant of knots and links in 3-manifolds.
Paper reduces turbomachinery CFD simulations by identifying key dimensions.
problem Reducing computational cost in turbomachinery 3D CFD simulations.
method Statistical sufficient dimension reduction methods and polynomial variable projection.
result Polynomial variable projection accurately identifies dimension reducing subspaces at lower cost.
A new language for neural architecture search decouples search spaces and algorithms.
problem Current neural architecture search methods are limited to specific use-cases and lack general-purpose constructs.
method Proposes a formal language for encoding search spaces over general computational graphs, allowing modular, composable, and reusable encodings.
result The language enables easy experimentation with different search spaces and algorithms without reinventing the wheel.
This paper compares Grid Search, Random Search, and Genetic Algorithm for NAS.
problem Hyperparameter optimization for neural architecture search.
method Comparison of Grid Search, Random Search, and Genetic Algorithm.
result Genetic Algorithm outperforms Grid Search and Random Search in terms of accuracy and execution time.
Efficient search methods can outperform random search on challenging tasks.
problem Comparing the performance of efficient and random search methods in neural architecture search.
method Comparison of weight sharing and random search methods on progressively larger search spaces for image classification and detection.
result Efficient search methods can provide substantial gains over random search on large, realistic tasks.
A new framework generates large hierarchical search spaces for neural architectures.
problem Discovering neural architectures from simple blocks is hard.
method Context-free grammars for a unified, scalable search space.
result Efficiently searches over complete architectures, outperforming existing methods.
MICO uses mutual information co-training to improve selective search efficiency.
problem Efficiently search and route unseen queries in large-scale search systems.
method Mutual Information Co-training framework for selective search with minimal supervision.
result Significantly improves performance on multiple metrics of selective search.
This work recommends personalized search stories to users based on their interests.
problem Personalized search story recommendation within search engines.
method Deep reinforcement learning architecture trained by imitation learning and reinforcement learning.
result Empirically demonstrated effectiveness on real-world data sets.
A method to reduce memory usage in NAS by pruning the search space.
problem High GPU memory consumption in One-Shot NAS techniques.
method Utilising Zero-Shot NAS to prune the search space before applying One-Shot NAS.
result Reduces memory consumption by 81% while maintaining accuracy.
Paper uses CMAB to improve NAS efficiency and accuracy.
problem Improving efficiency and accuracy of NAS for DNNs.
method Formulated NAS as CMAB, used Nested Monte-Carlo Search.
result Discovered cell structure achieves comparable accuracy to state-of-the-art, 20x faster.
New agent learns from previous search spaces to improve NAS efficiency.
problem NAS requires restarting learning from scratch between different search spaces.
method Transformer-based agent for joint training and knowledge transfer.
result Efficient knowledge transfer between search spaces improves NAS performance.
We consider parallel global optimization of derivative-free expensive-to-evaluate functions, and propose an efficient method based on stochastic approximation for implementing a conceptual Bayesian optimization algorithm proposed by Ginsbourger et al. (2007). At the heart of this algorithm is maximizing the information…
PGS uses neural networks to improve policies online without search trees.
problem Limited scalability of Monte Carlo Tree Search (MCTS) for high branching factor games.
method Adapts a neural network simulation policy via policy gradient updates, avoiding search trees.
result PGS achieves comparable performance to MCTS and defeats strong Hex agents.
New method reduces NAS search time and complexity.
problem High computational cost and complexity in NAS.
method Differentiable search space with annealing and pruning.
result Achieves 1.68% error on CIFAR-10 with 0.2 GPU days.
Approaches to learning Bayesian networks from data typically combine a scoring function with a heuristic search procedure. Given a Bayesian network structure, many of the scoring functions derived in the literature return a score for the entire equivalence class to which the structure belongs. When using such a scoring…
FiGS searches over a larger space of architectures for efficient mobile models.
problem Designing small, efficient deep networks for mobile devices.
method Differentiable search method using sparse regularization and Logistic-Sigmoid distribution.
result FiGS produces state-of-the-art parameter-efficient models on ImageNet and improves object detection performance.
This work proposes searching for optimal operation distribution in neural architecture search.
problem Finding optimal neural architecture with specific operations and connections.
method Search for the optimal operation distribution, providing a stochastic and approximate solution.
result Operation distribution holds enough discriminating power to reliably identify a solution and is easier to optimise than traditional encodings.
CrossBeam learns to search more efficiently in program synthesis.
problem Efficiently searching through vast program spaces.
method Trains a neural model to guide program synthesis, combining previously explored programs.
result CrossBeam explores much smaller portions of the program space compared to state-of-the-art methods.
Deep-n-Cheap automates deep learning model search for low complexity.
problem Finding efficient deep learning models for various datasets.
method Automated search framework for architecture and hyperparameters, including search transfer.
result Models offer comparable performance to state-of-the-art but are faster to train.
Paper presents a new trie for integer sketches to improve similarity searches.
problem Efficient similarity searches on integer sketches.
method Introduces a novel b-bit sketch trie that leverages succinct data structures. result Significantly improves search time and space-efficiency of similarity searches.
Improves diffusion model performance and efficiency through classical search.
problem Tackles inference-time control in diffusion models.
method Proposes a framework combining local and global search for efficient navigation.
result Significant gains in performance and efficiency across various domains.
einspace expands NAS search space to include diverse neural architectures.
problem NAS results are often limited to existing structures; new designs are rare.
method einspace uses a probabilistic context-free grammar to create a versatile search space.
result einspace discovers novel and improved architectures, including convolutions and attention.
We consider a framework for structured prediction based on search in the space of complete structured outputs. Given a structured input, an output is produced by running a time-bounded search procedure guided by a learned cost function, and then returning the least cost output uncovered during the search. This framewor…
Method predicts disease outbreaks using search logs, overcoming instability.
problem Predicting disease outbreaks from search logs is challenging due to short-term and long-term instability.
method Seasonal-adjustment method decomposes logs into seasonal, trend, and irregular components; feature selection method selects relevant search terms.
result Proposed method outperforms comparative methods in prediction accuracy for seven of ten diseases.
SGAS improves neural architecture search by choosing and pruning operations greedily.
problem NAS often fails to generalize in final evaluation.
method Divides search into sub-problems and chooses/prunes candidate operations greedily.
result SGAS finds state-of-the-art architectures with minimal computational cost.
UNAS combines DNAS and RL for efficient architecture search.
problem Discovering high accuracy or low latency neural architectures.
method Unified framework combining differentiable and reinforcement learning approaches.
result UNAS achieves state-of-the-art accuracy on CIFAR-10, CIFAR-100, and ImageNet datasets.
Efficient neural architecture search by sampling structure and operations.
problem Efficiently searching for optimal neural architectures.
method Decouples structure and operation search, using reinforcement learning with policy vectors.
result Significantly improved efficiency compared to traditional methods.
NeuralArTS categorizes neural ops in a type system for NAS.
problem Manual optimization of search spaces for NAS is inefficient.
method Developed NeuralArTS, a type system for categorizing network ops.
result NeuralArTS can be applied to convolutional layers.
Paper addresses bias in search intent affecting click behavior.
problem Bias in user search intent affects click behavior and relevance.
method Proposes a search intent bias hypothesis to improve click models.
result Click models can better interpret user clicks and improve retrieval performance.
Paper tackles NAS problem by modeling it as a sparse supernet.
problem Neural Architecture Search (NAS) problem, particularly Mixed-Path Search.
method Model NAS as a sparse supernet with sparsity constraints. Use hierarchical accelerated proximal gradient algorithm for optimization.
result Proposed method finds compact, general, and powerful neural architectures.