Extends linear representation hypothesis to categorical and hierarchical concepts in LLMs.
problem Representing concepts without natural contrasts in large language models.
method Formalizes linear representation hypothesis for categorical and hierarchical concepts, proving relationships between concept hierarchy and representation geometry.
result Validated theoretical results on large language models, estimating representations for 900+ concepts.
Cluster analysis is one of the essential tasks in data mining and knowledge discovery. Each type of data poses unique challenges in achieving relatively efficient partitioning of the data into homogeneous groups. While the algorithms for numeric data are relatively well studied in the literature, there are still challe…
VCML learns concepts and metaconcepts from images and questions.
problem Learning concepts and metaconcepts from visual data.
method Bidirectional connection between visual concepts and metaconcepts.
result VCML can generalize from limited data and noisy inputs.
Second part of a series on higher coverings of racks and quandles.
problem Characterizing higher-dimensional centrality conditions in racks and quandles.
method Applying higher categorical Galois theory to racks and quandles.
result Identification and characterization of higher coverings, trivial coverings, and normal coverings.
Paper develops a finite dimensional approximation scheme for Riemannian manifolds.
problem Integration on Riemannian manifolds.
method New finite dimensional approximation scheme motivated by categorical colimit.
result Establishes a generalization for L1-functionals on Riemannian manifolds. Enhanced reinforcement learning using ensemble methods.
problem Improving reinforcement learning performance.
method Distributional reinforcement learning with ensemble group-aided training.
result Ensemble methods lead to more robust and efficient learning.
Paper defines XAI concepts using category theory.
problem Lack of precise mathematical definitions for XAI.
method Uses Category theory to define XAI concepts rigorously.
result Establishes a theoretical foundation for XAI.
We discuss the concept of Galois structure and Galois epimorphism in a general setting. Namely, a Galois structure for an epimorphism π:M→B in some category C is the action of a group object that gives to M the structure of principal homogeneous space in the relative category CB.
The purpose of this contribution is to point out connections between recent ideas about gerbes and gerbal actions (as higher categorical extension of representation theory) and old discussion in quantum field theory on commutator anomalies, gauge group extensions, and 3-cocycles. The unifying concept is the classical o…
Proposes novel losses for fine-grained categorical domain adaptation.
problem Fine-grained alignment of categories across domains in unsupervised domain adaptation.
method Joint category-domain classifier with adversarial training losses for both domain and category levels, and vicinal domain adaptation.
result Achieves state-of-the-art performance on benchmark datasets.
CA connects visual concepts to neural network representations.
problem Global association of human interpretable concepts with neural network internals.
method Established a taxonomy for CA methods and reviewed state-of-the-art research.
result Unified understanding of CA methods and datasets.
COMET learns concepts for few-shot learning, improving performance.
problem Few-shot learning challenges in machine learning.
method Meta-learning with human-interpretable concept dimensions.
result Significant improvement in 1-shot learning tasks.
Surveying deep meta-learning to improve quick concept learning.
problem Limitation of deep neural networks in learning new concepts quickly.
method Categorizes deep meta-learning techniques into metric, model, and optimization-based approaches.
result Unified overview of current deep meta-learning techniques.
One-hot CNN (convolutional neural network) has been shown to be effective for text categorization (Johnson & Zhang, 2015). We view it as a special case of a general framework which jointly trains a linear model with a non-linear feature generator consisting of `text region embedding + pooling'. Under this framework, we…
A concise review of recent few-shot meta-learning methods.
problem Mimicking human fast adaptation to new concepts based on prior knowledge.
method Categorized into four branches based on technical characteristics.
result Current challenges and future prospects identified.
Text documents can be described by a number of abstract concepts such as semantic category, writing style, or sentiment. Machine learning (ML) models have been trained to automatically map documents to these abstract concepts, allowing to annotate very large text collections, more than could be processed by a human in …
We construct a flat (and fake-flat) 2-connection in the configuration space of n indistinguishable particles in the complex plane, which categorifies the sl(2,C)-Knizhnik-Zamolodchikov connection obtained from the adjoint representation of sl(2,C). This will be done by considering the adjoint categorical represen…
Analyzes the concept of fields in classical and quantum physics.
problem Challenges in defining fields in classical and quantum physics.
method Uses groupoid description of quantum mechanics and categorical language.
result Fields as functors among groupoids of test particles and intrinsic system nature.
StructureBoost improves gradient boosting for complex categorical variables efficiently.
problem Efficiently handling complex categorical variables with known structure.
method Two methods to overcome computational obstacles in SCDT enumeration for structured categorical variables.
result StructureBoost outperforms existing packages on complex categorical problems.
New bandit model for e-commerce with ordered categories.
problem Optimizing customer experience in e-commerce with unknown preferences.
method Introducing three types of ordering between categories and proving lower bounds on cumulative regret.
result Proved that algorithms can fully leverage the structure of the model with theoretical guarantees.
Categorical bundles provide a natural framework for gauge theories involving multiple gauge groups. Unlike the case of traditional bundles there are distinct notions of triviality, and hence also of local triviality, for categorical bundles. We study categorical principal bundles that are product bundles in the categor…
Machine learning has made major advances in categorizing objects in images, yet the best algorithms miss important aspects of how people learn and think about categories. People can learn richer concepts from fewer examples, including causal models that explain how members of a category are formed. Here, we explore the…
New approach predicts under latent shifts using high-dimensional images.
problem Prediction under latent subgroup shifts with high-dimensional observations.
method Recognition-parametrised model (RPM) for identifying causal latent structure.
result Successfully adapts predictions for high-dimensional image data.
Significant pattern mining, the problem of finding itemsets that are significantly enriched in one class of objects, is statistically challenging, as the large space of candidate patterns leads to an enormous multiple testing problem. Recently, the concept of testability was proposed as one approach to correct for mult…
Survey on concept factorization methods for better feature learning.
problem Understanding the quality of learned features in representation learning.
method Categorizing and summarizing recent advances in concept factorization methods.
result Provides insights into theoretical basis and current developments in concept factorization.
Bayesian model improves categorization of explosions from sparse data.
problem Challenges in categorizing explosions from limited data.
method Bayesian update to Event Categorization Matrix model with Bayesian Decision Theory.
result Consistent gains in overall accuracy and lower false negative rates.
UNTIE learns representations of coupled categorical data.
problem Challenges in learning from unlabeled categorical data with complex couplings.
method UNTIE approach for unsupervised representation learning of heterogeneous couplings.
result UNTIE significantly improves categorical data representations on 25 diverse datasets.
Paper introduces Categorical Normalizing Flows for better handling of categorical data.
problem Limited application of normalizing flows on categorical data due to lack of intrinsic order.
method Categorical Normalizing Flows use continuous transformations to model latent relations in categorical data, optimizing both continuous representation and model likelihood.
result GraphCNF, a permutation-invariant generative model, outperforms state-of-the-art on molecule generation.
This paper proposes a method to reduce complexity in GLMs with categorical predictors.
problem Wasteful, hard-to-interpret, and prone to overfitting of traditional one-hot encoding for high-cardinality categorical predictors.
method Clustering categories of categorical predictors through a numerical method that preserves or improves accuracy while reducing the number of coefficients.
result Clustering categories of categorical predictors reduces complexity substantially without harming accuracy.
Novel algorithm SAODE improves high-dimensional stream classification in seasonal data.
problem Handling seasonal concept drift in high-dimensional stream classification.
method SAODE classifier that includes time as a super parent to handle seasonal drift.
result SAODE consistently outperforms other methods in stream and concept drift classification.
The paper shows how integrating categorical semantics can enhance unsupervised domain translation.
problem Improving unsupervised domain translation between perceptually different domains.
method Learning invariant categorical semantic features in an unsupervised manner and conditioning them on the style encoder.
result Conditioning the style encoder on learned categorical semantics improves translation and stylization.
Categorical variables are a natural choice for representing discrete structure in the world. However, stochastic neural networks rarely use categorical latent variables due to the inability to backpropagate through samples. In this work, we present an efficient gradient estimator that replaces the non-differentiable sa…
Study categorizes mutual funds using natural language processing from unstructured data.
problem Categorizing mutual funds using unstructured data for financial analysis.
method Used natural language processing models to classify mutual funds from their investment strategy descriptions.
result High accuracy in categorizing mutual funds using NLP from unstructured data.
Develops 2-categorical methods for multi-parameter persistence.
problem Fundamental limitations of traditional persistence modules.
method 2-categorical structures to capture hierarchical interactions.
result New invariants effectively characterize multidimensional topological features.
Transforms classical connections using pushforwards and gauge transformations.
problem Transforming classical connections in categorical settings.
method Constructing pushforwards and applying gauge transformations to decorated path spaces.
result Combines traditional gauge transformation with affine translation.
The paper defines a new concept of approximability for Lagrangian submanifolds.
problem Understanding the approximability of Lagrangian submanifolds.
method Introducing a new notion of categorical approximability for metric spaces, showing it applies to specific types of Lagrangian submanifolds.
result Examples of Lagrangian submanifolds are found that are approximable but not precompact.
CADM proposes a cluster-specific distance metric for categorical data clustering.
problem Inadequate distance metrics for categorical data, especially varying within clusters.
method Cluster-customized adaptive distance metric for categorical data.
result Achieved competitive performance in categorical data clustering.
Develops a new method for decision trees using categorical variable structure.
problem Lack of structure in treating categorical variables as predictors.
method Introduces a mathematical framework to represent categorical structure and generalizes decision trees to utilize this structure.
result Improves prediction accuracy on weather data using the new method.
Categorical d-separation criterion simplifies probability graph analysis.
problem Detecting causal relationships in probability distributions.
method Introducing categorical definitions for causal models and d-separation.
result Abstract version of d-separation criterion applies to various probability theories.
Real-valued word representations have transformed NLP applications; popular examples are word2vec and GloVe, recognized for their ability to capture linguistic regularities. In this paper, we demonstrate a {\em very simple}, and yet counter-intuitive, postprocessing technique -- eliminate the common mean vector and a f…
nTreeClus clusters categorical sequences using tree-based learners and k-mers.
problem Challenges in clustering categorical and sequential data.
method nTreeClus uses Tree-based Learners, k-mers, and autoregressive models for categorical time series.
result nTreeClus outperformed baseline methods in various validation metrics.
A new gradient estimator for categorical distributions reduces bias and variance.
problem Intractability of gradients for categorical distributions in discrete latent variable models.
method CatLog-Derivative trick and IndeCateR gradient estimator.
result IndeCateR reduces bias and variance of gradients for categorical distributions.
Bayesian optimisation tackles high-dimensional categorical and mixed search spaces.
problem Bayesian optimisation on high-dimensional categorical and mixed search spaces is challenging.
method Combining local optimisation with a tailored kernel design.
result Empirically outperforms current baselines in performance and computational costs.
Human categorization is one of the most important and successful targets of cognitive modeling in psychology, yet decades of development and assessment of competing models have been contingent on small sets of simple, artificial experimental stimuli. Here we extend this modeling paradigm to the domain of natural images…
Efficient optimisation of black-box problems that comprise both continuous and categorical inputs is important, yet poses significant challenges. We propose a new approach, Continuous and Categorical Bayesian Optimisation (CoCaBO), which combines the strengths of multi-armed bandits and Bayesian optimisation to select …
The article compares predictor importance in classification problems with categorical outcomes.
problem Comparing predictor importance in classification problems with categorical response variables.
method The approach is based on the categorical Gini correlation (CGC) and tests differences in CGCs across predictor groups.
result The proposed methodology accommodates predictors of arbitrary and unequal dimensions and allows for dependence between predictor groups.
Paper introduces new methods for modeling categorical data.
problem Training generative models on categorical data like text and segmentation.
method Argmax Flows and Multinomial Diffusion models.
result Models outperform existing methods in log-likelihood.
CPML efficiently learns new metrics for categorical data.
problem Metric learning for categorical data.
method CPML (categorical projected metric learning) using Schatten p-norms.
result CPML provides efficient metric learning with improved accuracy.