StructureBoost improves gradient boosting for complex categorical variables efficiently.
problem Efficiently handling complex categorical variables with known structure.
method Two methods to overcome computational obstacles in SCDT enumeration for structured categorical variables.
result StructureBoost outperforms existing packages on complex categorical problems.
Develops a new method for decision trees using categorical variable structure.
problem Lack of structure in treating categorical variables as predictors.
method Introduces a mathematical framework to represent categorical structure and generalizes decision trees to utilize this structure.
result Improves prediction accuracy on weather data using the new method.
Categorical variables are a natural choice for representing discrete structure in the world. However, stochastic neural networks rarely use categorical latent variables due to the inability to backpropagate through samples. In this work, we present an efficient gradient estimator that replaces the non-differentiable sa…
This study compares and evaluates categorical kernels for Gaussian process regression.
problem Challenges in designing effective categorical kernels for Gaussian process regression.
method Reproducible comparative study of existing kernels, new evaluation metrics, and clustering-based nested kernels.
result Nested kernels outperform other methods, especially when group structure is unknown or unknown.
Bayesian model improves categorization of explosions from sparse data.
problem Challenges in categorizing explosions from limited data.
method Bayesian update to Event Categorization Matrix model with Bayesian Decision Theory.
result Consistent gains in overall accuracy and lower false negative rates.
We study the problem of recovering the latent ground truth labeling of a structured instance with categorical random variables in the presence of noisy observations. We present a new approximate algorithm for graphs with categorical variables that achieves low Hamming error in the presence of noisy vertex and edge obse…
Develops 2-categorical methods for multi-parameter persistence.
problem Fundamental limitations of traditional persistence modules.
method 2-categorical structures to capture hierarchical interactions.
result New invariants effectively characterize multidimensional topological features.
cGAP visualizes high-dimensional categorical data with interpretable geometric structure.
problem Lack of visualization tools for high-dimensional categorical data.
method cGAP uses Homogeneity Analysis (HOMALS) to embed data in a 3D space and maps it to colors.
result cGAP reveals coherent clusters, outliers, and local-to-global structure in categorical data.
cGAP visualizes high-dimensional categorical data with interpretable geometric structure.
problem Lack of visualization tools for high-dimensional categorical data.
method cGAP uses Homogeneity Analysis (HOMALS) to embed data in a 3D space and maps it to colors for visualization.
result cGAP reveals coherent clusters, outliers, and local-to-global structure in categorical data.
Reduces high granularity and dimensionality in hierarchical categorical variables.
problem Overfitting and estimation issues in predictive models due to high granularity and dimensionality.
method Entity embedding and top-down clustering algorithm to reduce granularity and dimensionality.
result The reduced hierarchy improves model fit and complexity balance.
Study categorizes mutual funds using natural language processing from unstructured data.
problem Categorizing mutual funds using unstructured data for financial analysis.
method Used natural language processing models to classify mutual funds from their investment strategy descriptions.
result High accuracy in categorizing mutual funds using NLP from unstructured data.
Method reduces categorical data to lower dimensions using density matrices.
problem Dimensionality reduction for categorical data.
method Density-matrix construction from class-conditional frequencies; spectral embedding.
result Low-dimensional spectral embeddings with controlled rank.
New method uniquely identifies causal structure from ordinal data.
problem Challenges in causal discovery for categorical data, especially direction of relationships.
method Exploits ordinal information to uniquely identify causal structure.
result Favorable and robust performance compared to state-of-the-art methods.
Unified CI test for categorical and ordinal data maintains power in high dimensions.
problem Rapid degradation of statistical power in existing CI tests for high-dimensional conditioning variables.
method Unified CI test for categorical and ordinal data, maintaining reasonable calibration and power in high dimensions.
result Our test outperforms existing baselines in model testing and structure learning for dense directed graphical models.
This work redefines data-centric AI by unifying categorical and cochain notions.
problem The need to rethink data notions for data-centric AI.
method Proposes unifying principles from categorical and cochain notions of data.
result Unified definition of data impacts machine learning development, implementation, and utilization.
A novel graph spectral method for mixed categorical and numerical data.
problem Feature learning for mixed data types (numerical and categorical).
method Graph spectral decomposition of the graph Laplacian to model probabilistic dependence structure.
result Increased separability and clusterability of observations in the transformed feature space.
We propose a generalization of quantization as a categorical way. For a fixed Poisson algebra quantization categories are defined as subcategories of R-module category with the structure of classical limits. We construct the generalized quantization categories including matrix regularization, strict deformation quantiz…
We propose a method to train generative adversarial networks on mutivariate feature vectors representing multiple categorical values. In contrast to the continuous domain, where GAN-based methods have delivered considerable results, GANs struggle to perform equally well on discrete data. We propose and compare several …
CARMS improves gradient estimation for categorical variables.
problem Accurately backpropagating gradients through categorical variables.
method CARMS combines REINFORCE with antithetic sampling to create unbiased gradient estimators.
result CARMS outperforms competing methods on various tasks.
For a principal bundle P→M equipped with a connection Aˉ, we study an infinite dimensional bundle PAˉdecP over the space of paths on M, with the points of PAˉdecP being horizontal paths on P decorated with elements of a second structure group. We co…
We solve the ANOVA decomposition for categorical inputs.
problem Lack of a closed-form expression for ANOVA decomposition with categorical dependent variables.
method Bridge functional analysis with discrete Fourier analysis to derive a closed-form decomposition.
result Closed-form decomposition for categorical inputs without assumptions.
Paper reduces hyperparameters in mixed-categorical Gaussian processes for green aircraft optimization.
problem High-dimensional mixed-categorical Gaussian processes with many hyperparameters.
method Innovative dimension reduction algorithm using partial least squares regression.
result Significant reduction in fuel consumption (439 kg) for a green aircraft.
WE constructs GP kernels for mixed inputs using weighted EDMs.
problem Limitation of standard GP models in handling categorical variables.
method WEGP constructs kernel function using weighted EDMs for categorical inputs.
result WEGP improves GP model accuracy in both synthetic and real-world optimization problems.
New method learns disentangled discrete representations using categorical variational autoencoders.
problem Learning disentangled representations from discrete latent spaces.
method Replaced standard Gaussian VAE with a categorical VAE to mitigate rotational invariance.
result Categorical distributions improve learning of disentangled representations.
In this paper we introduce evidence transfer for clustering, a deep learning method that can incrementally manipulate the latent representations of an autoencoder, according to external categorical evidence, in order to improve a clustering outcome. By evidence transfer we define the process by which the categorical ou…
Extends linear representation hypothesis to categorical and hierarchical concepts in LLMs.
problem Representing concepts without natural contrasts in large language models.
method Formalizes linear representation hypothesis for categorical and hierarchical concepts, proving relationships between concept hierarchy and representation geometry.
result Validated theoretical results on large language models, estimating representations for 900+ concepts.
Cluster analysis is one of the essential tasks in data mining and knowledge discovery. Each type of data poses unique challenges in achieving relatively efficient partitioning of the data into homogeneous groups. While the algorithms for numeric data are relatively well studied in the literature, there are still challe…
The study analyzes decision trees on real and categorical features, deriving bounds on their VC dimension and proposing improved pruning algorithms.
problem Understanding the generalization properties of decision trees on different types of features.
method Introducing partitioning functions, relating them to growth functions and VC dimension, and deriving bounds for decision stumps and trees of various structures.
result Exact VC dimension of decision stumps and improved pruning algorithms for binary trees.
The Gumbel-max trick and its extensions simplify sampling from categorical distributions in machine learning.
problem Sampling from categorical distributions with unnormalized probabilities.
method Extensions of the Gumbel-max trick for various applications.
result Simplified and efficient methods for sampling and gradient estimation.
New framework uses cohomology to analyze probabilistic distortions and arbitrage.
problem Analyzing probabilistic distortions and arbitrage in categorical filtrations.
method Transport cohomological framework, simplicial structure, loop effects, holonomy.
result Nontrivial probabilistic distortions and obstructions generated by loops.
New algorithms detect categorical structures in high-dimensional data.
problem Detecting categorical structures in high-dimensional data.
method Low coordinate degree functions (LCDF) applied to categorical and stochastic block models.
result Unified analysis of LCDF performance for various SBMs and tight lower bounds.
New methods tackle adversarial attacks on categorical sequences, improving model security.
problem Adversarial attacks on categorical sequence models, especially for money transactions and medical fraud.
method Two black-box adversarial attacks: Monte-Carlo and continuous relaxation methods.
result Generated adversarial sequences fool machine learning models but remain close to original ones.
Categorical bundles provide a natural framework for gauge theories involving multiple gauge groups. Unlike the case of traditional bundles there are distinct notions of triviality, and hence also of local triviality, for categorical bundles. We study categorical principal bundles that are product bundles in the categor…
Probabilistic hash embeddings improve online learning of categorical features.
problem Online learning of categorical features with changing vocabulary.
method Probabilistic hash embedding (PHE) with Bayesian online learning.
result PHE mitigates forgetting and maintains high performance in online settings.
VAIOM models financial returns using continuous input and categorical output.
problem Modeling continuous, noisy, and heterogeneous financial data.
method VAIOM is a decoder-only Transformer that separates input representation from output likelihood.
result VAIOM models outperform fixed single-bar LightGBM baseline in both Test halves.
In this work we present Discrete Attend Infer Repeat (Discrete-AIR), a Recurrent Auto-Encoder with structured latent distributions containing discrete categorical distributions, continuous attribute distributions, and factorised spatial attention. While inspired by the original AIR model andretaining AIR model's capabi…
NeuralArTS categorizes neural ops in a type system for NAS.
problem Manual optimization of search spaces for NAS is inefficient.
method Developed NeuralArTS, a type system for categorizing network ops.
result NeuralArTS can be applied to convolutional layers.
Bayesian context trees capture complex dependencies in categorical sequences.
problem Complex, long-range dependencies in categorical sequences are not well captured by simple models.
method Parsimonious Bayesian context trees with model-based agglomerative clustering for efficient inference.
result The proposed framework outperforms existing models on real-world data.
The paper proposes a method to analyze categorical feature interactions in large datasets using graph covariance and LLMs.
problem Analyzing complex datasets with numerous categorical features and timestamps.
method Binarization of categorical features using one-hot encoding, computation of graph covariance, identifying significant feature pairs, and using LLMs to generate explanations.
result The method identifies meaningful feature pairs and potential data stories underlying categorical feature interactions.
The paper defines and studies the category of Z-graded manifolds, including their intrinsic structure and formal properties.
problem Understanding the categorical properties and intrinsic structure of Z-graded manifolds.
method Describing local models, explaining formality, and formulating analogues of theorems.
result Proper definitions of objects and morphisms in the category of Z-graded manifolds, and formulation of Batchelor's theorem.
New method identifies causal effects with categorical unobserved confounders.
problem Estimating causal effects in the presence of unobserved confounders.
method Mixture learning and tensor decomposition for consistent estimation.
result Causal effects are identifiable with categorical unobserved confounders under suitable conditions.
Bayesian optimization for high-dimensional combinatorial spaces using embeddings.
problem Optimizing expensive functions over large, complex input spaces.
method Dictionary-based ordinal embeddings for high-dimensional combinatorial structures, using Gaussian process models.
result The proposed method outperforms state-of-the-art BO methods on diverse real-world benchmarks.
UNTIE learns representations of coupled categorical data.
problem Challenges in learning from unlabeled categorical data with complex couplings.
method UNTIE approach for unsupervised representation learning of heterogeneous couplings.
result UNTIE significantly improves categorical data representations on 25 diverse datasets.
New categorical actions link topological and algebraic structures.
problem Understanding relationships between topological and algebraic structures.
method Categorical actions of type B braid group on homotopy categories.
result Proves Rouquier's conjecture on faithfulness of Type B 2-braid group.
Paper introduces Categorical Normalizing Flows for better handling of categorical data.
problem Limited application of normalizing flows on categorical data due to lack of intrinsic order.
method Categorical Normalizing Flows use continuous transformations to model latent relations in categorical data, optimizing both continuous representation and model likelihood.
result GraphCNF, a permutation-invariant generative model, outperforms state-of-the-art on molecule generation.
This paper proposes a method to reduce complexity in GLMs with categorical predictors.
problem Wasteful, hard-to-interpret, and prone to overfitting of traditional one-hot encoding for high-cardinality categorical predictors.
method Clustering categories of categorical predictors through a numerical method that preserves or improves accuracy while reducing the number of coefficients.
result Clustering categories of categorical predictors reduces complexity substantially without harming accuracy.
Differentiable language model attacks improve adversarial examples for categorical sequence classifiers.
problem Challenges in adversarial attacks for categorical sequence models due to non-differentiability.
method Fine-tuning a language model as a generator of adversarial examples, using a differentiable loss function that combines a surrogate classifier score and approximate edit distance.
result Semantically better and more resistant adversarial examples.
A new model estimates mixed memberships for categorical data with weighted responses.
problem Limited applicability of existing GoM model to weighted categorical data.
method Proposes Weighted Grade of Membership (WGoM) model, relaxing distribution constraints.
result WGoM can describe any response matrix with finite distinct elements.