Efficient optimisation of black-box problems that comprise both continuous and categorical inputs is important, yet poses significant challenges. We propose a new approach, Continuous and Categorical Bayesian Optimisation (CoCaBO), which combines the strengths of multi-armed bandits and Bayesian optimisation to select …
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “Continuous and Categorical Inputs”
A new method for Gaussian Processes handles mixed continuous and categorical inputs.
problem Modeling cross-correlations between continuous and categorical data.
method Low-Rank Correlation (LRC) method for Gaussian Processes with flexible rank approximation.
result LRC outperforms existing methods in estimating cross-correlations and predicting response surfaces.
VAIOM models financial returns using continuous input and categorical output.
problem Modeling continuous, noisy, and heterogeneous financial data.
method VAIOM is a decoder-only Transformer that separates input representation from output likelihood.
result VAIOM models outperform fixed single-bar LightGBM baseline in both Test halves.
Unified Bayesian Optimisation for mixed variables improves performance.
problem Efficient optimisation of problems with both categorical and continuous variables.
method Derive value proposals from the Expected Improvement criterion to optimise both categorical and continuous variables under a single acquisition metric.
result Unified approach significantly outperforms existing methods across mixed-variable tasks.
WE constructs GP kernels for mixed inputs using weighted EDMs.
problem Limitation of standard GP models in handling categorical variables.
method WEGP constructs kernel function using weighted EDMs for categorical inputs.
result WEGP improves GP model accuracy in both synthetic and real-world optimization problems.
Gradient-based adversarial attacks on categorical sequence models via traversing an embedded worldcs.LG
New methods tackle adversarial attacks on categorical sequences, improving model security.
problem Adversarial attacks on categorical sequence models, especially for money transactions and medical fraud.
method Two black-box adversarial attacks: Monte-Carlo and continuous relaxation methods.
result Generated adversarial sequences fool machine learning models but remain close to original ones.
This study compares and evaluates categorical kernels for Gaussian process regression.
problem Challenges in designing effective categorical kernels for Gaussian process regression.
method Reproducible comparative study of existing kernels, new evaluation metrics, and clustering-based nested kernels.
result Nested kernels outperform other methods, especially when group structure is unknown or unknown.
Paper reduces hyperparameters in mixed-categorical Gaussian processes for green aircraft optimization.
problem High-dimensional mixed-categorical Gaussian processes with many hyperparameters.
method Innovative dimension reduction algorithm using partial least squares regression.
result Significant reduction in fuel consumption (439 kg) for a green aircraft.
Kendall transformation converts continuous data into categorical vectors for robust information theory.
problem Handling small number of observations and preserving ranking in continuous data.
method Kendall transformation converts ordered features into categorical vectors of pairwise order relations.
result Kendall transformation makes information theory methods applicable to continuous data robustly.
We propose Dirichlet Process mixtures of Generalized Linear Models (DP-GLM), a new method of nonparametric regression that accommodates continuous and categorical inputs, and responses that can be modeled by a generalized linear model. We prove conditions for the asymptotic unbiasedness of the DP-GLM regression mean fu…
Natural Language Processing models help encode categorical process inputs.
problem Encoding categorical variables in industrial process modeling.
method Using NLP models for categorical variable encoding, combined with dimensionality reduction.
result Meaningful embeddings of categorical variables improve feature importance.
Paper introduces Categorical Normalizing Flows for better handling of categorical data.
problem Limited application of normalizing flows on categorical data due to lack of intrinsic order.
method Categorical Normalizing Flows use continuous transformations to model latent relations in categorical data, optimizing both continuous representation and model likelihood.
result GraphCNF, a permutation-invariant generative model, outperforms state-of-the-art on molecule generation.
Neural framework for conditional OT maps learns from categorical and continuous variables.
problem Learning conditional optimal transport maps between complex distributions.
method Hypernetwork generates adaptive transport layer parameters based on conditioning variables.
result Our method outperforms simpler conditioning methods in comprehensive ablation studies.
Researchers study the normalizing constant of a continuous categorical distribution.
problem Understanding the normalizing constant of the continuous categorical distribution.
method Characterize numerical behavior and present theoretical and methodological advances.
result The normalizing constant can be written in closed form using elementary functions.
Many real-world functions are defined over both categorical and category-specific continuous variables and thus cannot be optimized by traditional Bayesian optimization (BO) methods. To optimize such functions, we propose a new method that formulates the problem as a multi-armed bandit problem, wherein each category co…
Interprets neural network classifiers for categorical inputs.
problem Neural networks' interpretability in human-sensitive applications.
method Mapping to physical energy model, expansion of neural network layers.
result Each layer's contribution to classification can be analyzed.
We solve the ANOVA decomposition for categorical inputs.
problem Lack of a closed-form expression for ANOVA decomposition with categorical dependent variables.
method Bridge functional analysis with discrete Fourier analysis to derive a closed-form decomposition.
result Closed-form decomposition for categorical inputs without assumptions.
Scalable GP model tackles big data, categorical factors, and multiple responses.
problem Handling big datasets, categorical inputs, and multiple responses in Gaussian processes.
method Latent variable Gaussian process (LVGP) with variational inference for scalability and interpretability.
result The method scales well for large datasets and outperforms state-of-the-art methods.
New GP kernel handles mixed-categorical data, improving model accuracy.
problem Improving Gaussian process models for mixed-categorical data.
method Extends continuous exponential kernels to handle mixed-categorical variables.
result The proposed GP model gives higher likelihood and smaller residual error.
Bayesian optimisation tackles high-dimensional categorical and mixed search spaces.
problem Bayesian optimisation on high-dimensional categorical and mixed search spaces is challenging.
method Combining local optimisation with a tailored kernel design.
result Empirically outperforms current baselines in performance and computational costs.
Multivariate categorical data occur in many applications of machine learning. One of the main difficulties with these vectors of categorical variables is sparsity. The number of possible observations grows exponentially with vector length, but dataset diversity might be poor in comparison. Recent models have gained sig…
Paper introduces new methods for modeling categorical data.
problem Training generative models on categorical data like text and segmentation.
method Argmax Flows and Multinomial Diffusion models.
result Models outperform existing methods in log-likelihood.
AdaCat improves density estimation and planning in autoregressive models.
problem Efficiently modeling sharp density changes in continuous data.
method Adaptive Categorical Discretization (AdaCat) for autoregressive models.
result Improves density estimation and planning in various data types.
Bayesian Optimization (BO) methods are useful for optimizing functions that are expen- sive to evaluate, lack an analytical expression and whose evaluations can be contaminated by noise. These methods rely on a probabilistic model of the objective function, typically a Gaussian process (GP), upon which an acquisition f…
CcGAN tackles conditional image generation for continuous labels.
problem Mathematical challenges in conditioning on continuous, scalar labels.
method Proposes novel empirical losses and label input methods for continuous conditional GANs.
result CcGAN generates diverse, high-quality images from continuous labels.
Bayesian Optimization over High-Dimensional Combinatorial Spaces via Dictionary-based Embeddingscs.LG
Bayesian optimization for high-dimensional combinatorial spaces using embeddings.
problem Optimizing expensive functions over large, complex input spaces.
method Dictionary-based ordinal embeddings for high-dimensional combinatorial structures, using Gaussian process models.
result The proposed method outperforms state-of-the-art BO methods on diverse real-world benchmarks.
Simplex-valued data appear throughout statistics and machine learning, for example in the context of transfer learning and compression of deep networks. Existing models for this class of data rely on the Dirichlet distribution or other related loss functions; here we show these standard choices suffer systematically fr…
Differentiable language model attacks improve adversarial examples for categorical sequence classifiers.
problem Challenges in adversarial attacks for categorical sequence models due to non-differentiability.
method Fine-tuning a language model as a generator of adversarial examples, using a differentiable loss function that combines a surrogate classifier score and approximate edit distance.
result Semantically better and more resistant adversarial examples.
A method for representing and comparing categorical trajectories using multivariate functional principal components.
problem Statistical description and comparison of categorical trajectories.
method Transforming categorical trajectories into binary indicator functions and applying multivariate functional principal components analysis.
result Consistent estimators of mean trajectories and covariance functions are obtained under weak regularity assumptions.
A new method for Gaussian process regression with categorical inputs.
problem Challenges in building a predictive and computationally efficient Gaussian process with categorical inputs.
method Distributional encoding based on maximum mean discrepancy and Wasserstein distance.
result State-of-the-art predictive performance on various datasets.
Many learning algorithms require categorical data to be transformed into real vectors before it can be used as input. Often, categorical variables are encoded as one-hot (or dummy) vectors. However, this mode of representation can be wasteful since it adds many low-signal regressors, especially when the number of uniqu…
New gradient estimators for discrete variables improve model training.
problem Training models with discrete latent variables is challenging due to high gradient variance.
method Introduced novel gradient estimators based on importance sampling and statistical couplings, extending to categorical variables.
result Proposed gradient estimators outperform previous methods in systematic experiments.
Applied Data Scientists throughout various industries are commonly faced with the challenging task of encoding high-cardinality categorical features into digestible inputs for machine learning algorithms. This paper describes a Bayesian encoding technique developed for WeWork's lead scoring engine which outputs the pro…
Bayesian optimization adapted for discrete spaces using random mappings.
problem Global optimization of expensive black-box functions with discrete variables.
method Embeds discrete space into a convex polytope, performs optimization in continuous space.
result Method outperforms existing methods in large combinatorial spaces.
Study categorizes mutual funds using natural language processing from unstructured data.
problem Categorizing mutual funds using unstructured data for financial analysis.
method Used natural language processing models to classify mutual funds from their investment strategy descriptions.
result High accuracy in categorizing mutual funds using NLP from unstructured data.
Researchers propose better probabilistic models for deep learning.
problem Using cross-entropy loss for non-categorical data.
method Introducing continuous-categorical distribution and proposing probabilistic alternatives.
result Potential for outperformance in deep learning models with proper probabilistic treatment.
Categorical d-separation criterion simplifies probability graph analysis.
problem Detecting causal relationships in probability distributions.
method Introducing categorical definitions for causal models and d-separation.
result Abstract version of d-separation criterion applies to various probability theories.
A hybrid model for Bayesian optimization handles mixed variables using MCTS for categorical and GP for continuous.
problem Optimizing functions with mixed variable types (continuous, integer, categorical).
method Merges MCTS for categorical and GP for continuous variables, integrates UCTS search strategy, and dynamically selects kernels.
result Hybrid models outperform traditional methods in Bayesian optimization.
A new method for optimizing models with categorical variables using diffusion.
problem Optimizing models with categorical variables, especially in discrete distributions.
method Introducing ReDGE, a diffusion-based soft reparameterization method for categorical distributions.
result ReDGE consistently matches or outperforms existing gradient-based methods in experiments.
Earth observation embeddings can convert discrete biome maps into continuous representations that better capture ecological variation.
problem Biome maps impose categorical boundaries that compress continuous variation in biotic communities.
method Fit a linear classifier on Earth observation embeddings to predict biome labels.
result Continuous biome representation outperforms discrete biome labels for predicting species occurrence.
Gryffin optimizes categorical variables in materials design, leveraging expert knowledge.
problem Optimizing categorical variables in complex design choices like molecule selection.
method Bayesian optimization with smooth approximations to categorical distributions, incorporating expert knowledge.
result Gryffin accelerates discovery of promising molecules and materials, highlighting relevant correlations.
We present a technique for clustering categorical data by generating many dissimilarity matrices and averaging over them. We begin by demonstrating our technique on low dimensional categorical data and comparing it to several other techniques that have been proposed. Then we give conditions under which our method shoul…
nTreeClus clusters categorical sequences using tree-based learners and k-mers.
problem Challenges in clustering categorical and sequential data.
method nTreeClus uses Tree-based Learners, k-mers, and autoregressive models for categorical time series.
result nTreeClus outperformed baseline methods in various validation metrics.
A new method clusters mixed-type data efficiently.
problem Clustering mixed-type data with continuous and categorical variables.
method Extends Information Bottleneck principle to heterogeneous data using generalised product kernels.
result DIBmix outperforms four established methods in various scenarios.
We propose a method to train generative adversarial networks on mutivariate feature vectors representing multiple categorical values. In contrast to the continuous domain, where GAN-based methods have delivered considerable results, GANs struggle to perform equally well on discrete data. We propose and compare several …
A new method for categorical variational inference using discrete normalizing flows.
problem Challenges in optimizing variational approximations for discrete latent variables.
method Differentiable reparameterization using a mixture of discrete normalizing flows.
result Improves optimization of evidence lower bound and reduces sensitivity to hyperparameters.
Three bootstrap tests compare categorical time series generating processes.
problem Testing equality of generating processes in categorical time series.
method Proposes three tests using dissimilarity measures and bootstrap methods.
result Advantages and disadvantages of each bootstrap method discussed.
Ordinal Causal Discoverystat.ME
New method uniquely identifies causal structure from ordinal data.
problem Challenges in causal discovery for categorical data, especially direction of relationships.
method Exploits ordinal information to uniquely identify causal structure.
result Favorable and robust performance compared to state-of-the-art methods.