Bayesian optimization tackles mixed discrete-continuous problems with Gaussian processes.
problem Optimizing problems with both discrete and continuous variables using costly simulations.
method Relaxing discrete variables into continuous latent variables, using Bayesian optimization, and incorporating compatibility constraints with Lagrangians.
result Comparative analysis of different mixed Bayesian optimization approaches.
Bayesian optimization tackles expensive discrete and mixed parameter spaces.
problem Optimizing expensive functions with discrete and mixed parameters.
method Probabilistic reparameterization to maximize expectation of AF over continuous parameters.
result Our approach provably converges to a maximizer of the AF and enjoys the same regret bounds as standard BO.
Tree ensemble kernels improve Bayesian optimization for mixed features and constraints.
problem Optimizing over mixed-feature spaces with known constraints.
method Kernel interpretation of tree ensembles as Gaussian Process prior, compatible optimization formulation for acquisition function, integration of known constraints.
result Framework outperforms state-of-the-art methods for mixed-feature spaces and constraints.
A game theory study on optimal hiding and searching strategies in discrete locations.
problem Optimal hiding and searching strategies in a two-person zero-sum game between a hider and a searcher.
method Proved the existence of optimal strategies, developed an algorithm to compute them, and compared with a simple strategy.
result Optimal hiding strategy involves hiding in each location with nonzero probability, and optimal searching strategy can be constructed with up to n simple sequences.
Machine learning pipeline potentially consists of several stages of operations like data preprocessing, feature engineering and machine learning model training. Each operation has a set of hyper-parameters, which can become irrelevant for the pipeline when the operation is not selected. This gives rise to a hierarchica…
Bayesian optimisation tackles high-dimensional categorical and mixed search spaces.
problem Bayesian optimisation on high-dimensional categorical and mixed search spaces is challenging.
method Combining local optimisation with a tailored kernel design.
result Empirically outperforms current baselines in performance and computational costs.
We present a mixed multinomial logit (MNL) model, which leverages the truncated stick-breaking process representation of the Dirichlet process as a flexible nonparametric mixing distribution. The proposed model is a Dirichlet process mixture model and accommodates discrete representations of heterogeneity, like a laten…
Decision trees are flexible models that are well suited for many statistical regression problems. In a Bayesian framework for regression trees, Markov Chain Monte Carlo (MCMC) search algorithms are required to generate samples of tree models according to their posterior probabilities. The critical component of such an …
Two Bayesian optimization methods tackle dynamic design spaces with mixed variables.
problem Optimizing complex systems with varying numbers and types of variables and constraints.
method Two Bayesian optimization approaches: budget allocation and kernel function.
result Both methods converge faster and more consistently than standard approaches.
This paper proposes new search algorithms for counterfactual explanations based upon mixed integer programming. We are concerned with complex data in which variables may take any value from a contiguous range or an additional set of discrete states. We propose a novel set of constraints that we refer to as a "mixed pol…
Develops a new method for learning discrete distributions without embedding them in a continuous space.
problem Challenges in learning discrete distributions using current methodologies.
method Introduces a MAD invertible map and a mixed variational flow (MAD Mix) for discrete distributions.
result MAD Mix produces more reliable approximations than continuous-embedding flows.
MiLeNAS improves neural architecture search by reducing approximation errors and achieving better accuracy.
problem Improving efficiency and accuracy in neural architecture search (NAS).
method Mixed-level reformulation (MiLeNAS) to optimize efficiently and reliably.
result MiLeNAS achieves lower validation error and higher accuracy than bilevel optimization methods.
Short introduction to discrete flat fronts in hyperbolic space with a Weierstrass representation proof.
problem Understanding discrete flat fronts in hyperbolic space.
method Proving a Weierstrass representation for discrete flat fronts.
result Any discrete flat front in the mixed area sense admits a Weierstrass representation.
Hybrid Policy Optimization tackles reinforcement learning in hybrid spaces, improving performance over PPO.
problem Credit assignment issues and biased gradients in hybrid discrete-continuous action spaces.
method Mixed gradient estimator combining pathwise and score-function gradients, reformulating problems in hybrid form.
result HPO substantially outperforms PPO on inventory control and switched systems, with performance gaps increasing with continuous action dimension.
A hybrid model for Bayesian optimization handles mixed variables using MCTS for categorical and GP for continuous.
problem Optimizing functions with mixed variable types (continuous, integer, categorical).
method Merges MCTS for categorical and GP for continuous variables, integrates UCTS search strategy, and dynamically selects kernels.
result Hybrid models outperform traditional methods in Bayesian optimization.
Paper tackles NAS problem by modeling it as a sparse supernet.
problem Neural Architecture Search (NAS) problem, particularly Mixed-Path Search.
method Model NAS as a sparse supernet with sparsity constraints. Use hierarchical accelerated proximal gradient algorithm for optimization.
result Proposed method finds compact, general, and powerful neural architectures.
A hybrid algorithm combines optimization and enumeration for symbolic regression.
problem Finding any function from a set of operators without prior specification.
method Mixed-integer nonlinear optimization with explicit enumeration and constraints.
result The hybrid algorithm is competitive with state-of-the-art methods.
A Ph.D. thesis proposes an efficient approach for optimizing aircraft eco-design with high-dimensional mixed integer variables.
problem Optimizing aircraft eco-design with high-dimensional mixed integer variables.
method EGO optimization approach based on sequential enrichment of adaptive surrogate models, adapted for high-dimensional and mixed variable spaces.
result Effective parameterization tools significantly reduce the number of design variables, enabling optimization of complex aircraft concepts.
We define a family of kernels for mixed continuous/discrete hierarchical parameter spaces and show that they are positive definite.
New bounds for MCMC on discrete spaces without dimension dependence.
problem High-dimensional statistical convergence analysis of MCMC methods.
method Combining multicommodity flow and single-element drift conditions.
result Informed Metropolis-Hastings algorithms achieve relaxation times independent of dimension.
SOBER optimizes and quadrates efficiently in parallel for diverse tasks.
problem Scalability of batch Bayesian optimization and quadrature for expensive functions.
method Reformulates batch selection as a quadrature problem, balancing exploitation and exploration.
result SOBER outperforms 11 baselines on 12 tasks.
Library learns Bayesian networks from mixed data without discretization.
problem Learning Bayesian networks from mixed data (discrete and continuous variables).
method Proposes an algorithm for structural and parameter learning of Bayesian networks from mixed data using a mixed MI score function and Gaussian approximation. Offers two graph structure enumeration algorithms.
result Advantages in solving approximation and gap recovery problems on synthetic and real datasets.
SDE automatically recovers interpretable discrete distributions.
problem Limited interpretable discrete probability laws.
method Unsupervised framework using symbolic density estimation.
result Accurately recovers interpretable discrete distributions.
We show that the discrete principal nets in quadrics of constant curvature that have constant mixed area mean curvature can be characterized by the existence of a Königs dual in a concentric quadric.
Unified Bayesian Optimisation for mixed variables improves performance.
problem Efficient optimisation of problems with both categorical and continuous variables.
method Derive value proposals from the Expected Improvement criterion to optimise both categorical and continuous variables under a single acquisition metric.
result Unified approach significantly outperforms existing methods across mixed-variable tasks.
Efficient Bitwidth Search optimizes neural network quantization for better performance.
problem Finding optimal bitwidth for weights and activations of each layer efficiently.
method EBS algorithm reusing meta weights and binary decomposition for efficient mixed precision convolution.
result Mixed precision QNN outperforms uniform bitwidth and other techniques on CIFAR10 and ImageNet.
EBMs trained on discrete data using heat equations on graph structures.
problem Training EBMs on discrete or mixed data.
method Heat equations on graph structures for data perturbation.
result Efficacy demonstrated in various applications.
HiSS sampling overcomes local mode traps in rugged discrete spaces.
problem Sampling multimodal discrete distributions with gradient-based methods.
method Integrates Metropolis-within-Gibbs framework with logistic convolution.
result HiSS outperforms alternatives on various tasks, including Ising models and binary neural networks.
FLOP algorithm speeds up causal structure learning for linear models.
problem Efficiently learning causal structures from discrete data.
method FLOP algorithm combines fast parent selection and iterative score updates.
result FLOP finds highly accurate causal structures with near-perfect recovery.
This paper tackles convex-submodular minimax problems in mixed continuous-discrete domains.
problem Convex-submodular minimax problems in mixed continuous-discrete domains.
method Introduces new notions of optimality and proposes iterative algorithms combining discrete and continuous optimization.
result Characterizes convergence rates, computational complexity, and quality of solutions for convex and monotone-submodular minimax problems.
New algorithm approximates maximum of certain distributions on subsets.
problem Finding maximum of distributions on subsets.
method Connection between sampling and optimization via exchange inequalities and local random walks.
result Simple nearly-optimal approximation algorithm for MAP inference.
The optimization of expensive to evaluate, black-box, mixed-variable functions, i.e. functions that have continuous and discrete inputs, is a difficult and yet pervasive problem in science and engineering. In Bayesian optimization (BO), special cases of this problem that consider fully continuous or fully discrete doma…
Two new algorithms improve neural architecture search efficiency.
problem Optimizing neural architecture search for faster and more accurate models.
method Introduces NASGD and NASAGD using accelerated gradient descent on a semi-discrete space.
result Achieves comparable accuracy with 40x fewer architectures in 12 hours.
If X is a proper CAT(-1)-space and Γ a non-elementary discrete group of isometries acting properly discontinuously on X, it is shown that the geodesic flow on the quotient space Y=X/Γ is topologically mixing, provided that the generalized Busemann function has zeros on the boundary ∂X and the non-wanderin…
Framework uses optimal transport for neural architecture search.
problem Optimizing neural architectures in deep learning.
method Semi-discrete optimization using optimal transport.
result Gradient flow and minimizing movement scheme converge to reaction-diffusion equations.
The paper analyzes convergence rates of Langevin dynamics and Proximal Sampler using Φ-divergence.
problem Analyzing convergence rates of Langevin dynamics and Proximal Sampler.
method Extending mixing time analyses to Φ-divergence, using strong data processing inequalities. result Convergence of Φ-divergence to 0 exponentially fast along Unadjusted Langevin Algorithm and Proximal Sampler. Automatic methods for Neural Architecture Search (NAS) have been shown to produce state-of-the-art network models. Yet, their main drawback is the computational complexity of the search process. As some primal methods optimized over a discrete search space, thousands of days of GPU were required for convergence. A rece…
Mixed finite element methods solve a PDE using two or more variables. The theory of Discrete Exterior Calculus explains why the degrees of freedom associated to the different variables should be stored on both primal and dual domain meshes with a discrete Hodge star used to transfer information between the meshes. We s…
Improved density estimation for mixed discrete-continuous data.
problem Inconsistent density estimation for mixtures of continuous and discrete data.
method Modification of existing nonparametric density estimation methods to handle mixed discrete-continuous data.
result Improved consistency and empirical performance for mixed discrete-continuous data.
This paper studies convergence behavior of latent mixing measures that arise in finite and infinite mixture models, using transportation distances (i.e., Wasserstein metrics). The relationship between Wasserstein distances on the space of mixing measures and f-divergence functionals such as Hellinger and Kullback-Leibl…
HMQ improves quantization for edge devices with mixed precision.
problem Efficient quantization for edge devices with uniform, power-of-two thresholds.
method Introduces HMQ, a mixed precision quantization block that repurposes Gumbel-Softmax for searching over quantization schemes.
result Achieves competitive and state-of-the-art results on ImageNet despite restrictions.
Recently, randomly mapping vectorial data to strings of discrete symbols (i.e., sketches) for fast and space-efficient similarity searches has become popular. Such random mapping is called similarity-preserving hashing and approximates a similarity metric by using the Hamming distance. Although many efficient similarit…
The neural architecture search (NAS) algorithm with reinforcement learning can be a powerful and novel framework for the automatic discovering process of neural architectures. However, its application is restricted by noncontinuous and high-dimensional search spaces, which result in difficulty in optimization. To resol…
We propose a unified framework to address a family of classical mixed-integer optimization problems with logically constrained decision variables, including network design, facility location, unit commitment, sparse portfolio selection, binary quadratic optimization, sparse principal analysis and sparse learning proble…
Deep neuroevolution, that is evolutionary policy search methods based on deep neural networks, have recently emerged as a competitor to deep reinforcement learning algorithms due to their better parallelization capabilities. However, these methods still suffer from a far worse sample efficiency. In this paper we invest…
Driven by the need for parallelizable hyperparameter optimization methods, this paper studies \emph{open loop} search methods: sequences that are predetermined and can be generated before a single configuration is evaluated. Examples include grid search, uniform random search, low discrepancy sequences, and other sampl…
Materials design can be cast as an optimization problem with the goal of achieving desired properties, by varying material composition, microstructure morphology, and processing conditions. Existence of both qualitative and quantitative material design variables leads to disjointed regions in property space, making the…
Transformer model with mixed-frequency data improves stock volatility prediction.
problem Improving stock volatility prediction using mixed-frequency data.
method Transformer model trained on mixed-frequency data (GARCH-MIDAS model for frequency alignment).
result Transformer model reduces mean square error from 1.00 to 0.86.