This study compares machine learning methods for high-cardinality categorical variables.
problem Machine learning struggles with high-cardinality categorical variables.
method Empirical comparison of tree-boosting, deep neural networks, and linear mixed effects models.
result Tree-boosting with random effects outperforms deep neural networks with random effects.
EFDM models spatial point processes with variable cardinality using existence variables.
problem Challenges in extending diffusion models to variable-cardinality spatial point processes.
method Existence-field diffusion model (EFDM) that jointly models spatial locations and cardinality without discrete transitions.
result EFDM achieves improved modeling capability on datasets with varying cardinality.
Structured high-cardinality data arises in many domains, and poses a major challenge for both modeling and inference. Graphical models are a popular approach to modeling structured data but they are unsuitable for high-cardinality variables. The count-min (CM) sketch is a popular approach to estimating probabilities in…
This work restricts hidden cardinality in causal models to infer causal relations.
problem Causal relations between variables with a common unobserved cause cannot be directly inferred.
method Derive inequality constraints from d-separation in causal models with known cardinalities of unobserved variables.
result Inference of causal relations is possible with additional assumptions about cardinalities.
Paper proposes a uniqueness Shapley measure to compare variable importance.
problem Comparing the importance of different variables in identifying subjects.
method Uses Shapley value to combine reductions in log cardinality due to revealing variables.
result Demonstrates speedup in calculating variable importance.
Cardinality potentials are a generally useful class of high order potential that affect probabilities based on how many of D binary variables are active. Maximum a posteriori (MAP) inference for cardinality potential models is well-understood, with efficient computations taking O(DlogD) time. Yet efficient marginalizat…
New methods encode high-cardinality string variables efficiently.
problem Efficient encoding of high-cardinality string categorical variables.
method Two approaches: Gamma-Poisson matrix factorization and min-hash encoder.
result Improves supervised learning with high-cardinality categorical variables.
The cardinality constraint is an intrinsic way to restrict the solution structure in many domains, for example, sparse learning, feature selection, and compressed sensing. To solve a cardinality constrained problem, the key challenge is to solve the projection onto the cardinality constraint set, which is NP-hard in ge…
Regularized target encoding beats traditional methods for high cardinality features in ML.
problem Efficiently encoding high cardinality categorical variables for ML algorithms.
method Regularized target encoding compared to traditional encodings like integer and one-hot encoding.
result Regularized target encoding consistently provided the best results in a large-scale benchmark experiment.
Safe screening rules reduce ℓ0-regression computation by fixing 76% of variables.
problem Efficiently solving ℓ0-regression problems with large datasets. method Convex relaxation and safe screening rules to eliminate variables.
result 76% of variables can be fixed to their optimal values, reducing computational burden.
Novel approximation hierarchy for sparse quadratic programs.
problem Sparse Quadratic Programs with Cardinality Constraints.
method Exploits rank-dominating eigenvectors for min-max optimization over binary variables.
result Efficient screening of nonzero elements with scalable optimization algorithms.
Novel GLMMNet model tackles high-cardinality categorical features in actuarial applications.
problem Inadequate encoding methods for high-cardinality categorical features in actuarial data.
method Generalised Linear Mixed Model Neural Network (GLMMNet) integrating a generalised linear mixed model in a deep learning framework.
result GLMMNet often outperforms or performs comparably with entity embedded neural networks, providing transparency.
Method evaluates disentanglement in DLVMs, including those not aligned with latent axes.
problem Evaluate disentanglement in DLVMs, especially those not aligned with latent axes.
method Proposes a statistical method to discover generative factors of a dataset.
result Empirically demonstrates the advantage of the method on two datasets.
Combines boosting with Gaussian process and mixed effects models.
problem Model misspecifications and independence assumptions in boosting.
method Relaxes zero or linearity assumption in Gaussian process and mixed effects models, and independence assumption in boosting.
result Increased prediction accuracy compared to existing approaches.
Proposes a new DNN framework for count data with high-cardinality features.
problem Real-world data often have correlations and high-cardinality categorical features that traditional DNNs overlook.
method Introduces a hierarchical likelihood learning framework with gamma random effects for Poisson DNNs.
result Improves prediction performance by capturing nonlinear effects and subject-specific cluster effects.
For statistical learning, categorical variables in a table are usually considered as discrete entities and encoded separately to feature vectors, e.g., with one-hot encoding. "Dirty" non-curated data gives rise to categorical variables with a very high cardinality but redundancy: several categories reflect the same ent…
HAMD optimizes cubic portfolios without quadratization, achieving better results.
problem Optimizing higher-order portfolio models with reduced distortion.
method Hybrid pipeline combining continuous Hamiltonian search, cardinality-preserving projection, and iterated local search.
result HAMD achieves significantly lower native cubic objective values than classical heuristics.
TCMI assesses mutual dependence of continuous variables without parametric assumptions.
problem Estimating mutual information from continuous distributions.
method TCMI extends mutual information to continuous variables using cumulative distributions.
result TCMI facilitates feature selection and ranking of variable sets.
Sparse methods for supervised learning aim at finding good linear predictors from as few variables as possible, i.e., with small cardinality of their supports. This combinatorial selection problem is often turned into a convex optimization problem by replacing the cardinality function by its convex envelope (tightest c…
Study improves top-k set prediction with low cardinality.
problem Improving top-k set prediction accuracy with low cardinality.
method Introduces new target loss function and surrogate losses.
result Demonstrates effectiveness of cardinality-aware algorithms.
Unified clustering model handles both pairwise and cardinality constraints for better performance.
problem Clustering with specific constraints (pairwise and cardinality) to improve clustering quality.
method Unified integer programming formulation, binary and quadratic constraints, reformulated as continuous constraints, solved using ADMM.
result Unified model outperforms single category constraints and achieves better clustering performance.
A cardinality-constrained portfolio caps the number of stocks to be traded across and within groups or sectors. These limitations arise from real-world scenarios faced by fund managers, who are constrained by transaction costs and client preferences as they seek to maximize return and limit risk. We develop a new appro…
A serious problem in learning probabilistic models is the presence of hidden variables. These variables are not observed, yet interact with several of the observed variables. Detecting hidden variables poses two problems: determining the relations to other variables in the model and determining the number of states of …
We study the cardinality of the set of manifolds homotopy equivalent to a given manifold M and compare it to the cardinality of the structure set of M.
We investigate the application of two heuristic methods, genetic algorithms and tabu/scatter search, to the optimisation of realistic portfolios. The model is based on the classical mean-variance approach, but enhanced with floor and ceiling constraints, cardinality constraints and nonlinear transaction costs which inc…
Study on loops on non-orientable surfaces, determining cardinality and order.
problem Determining the cardinality and order of maximal complete 1-systems of loops on non-orientable surfaces.
method Proved the cardinality of maximal systems of arcs pairwise-intersecting at most once on a non-orientable surface is 2∣χ∣(∣χ∣+1), and used this to determine the cardinality of maximal complete 1-systems of loops. result Exact cardinality of maximal complete 1-systems of loops on punctured projective planes is determined.
In this paper we address cardinality estimation problem which is an important subproblem in query optimization. Query optimization is a part of every relational DBMS responsible for finding the best way of the execution for the given query. These ways are called plans. The execution time of different plans may differ b…
A GPU framework speeds up BnB for discrete optimization problems.
problem Optimizing large-scale discrete problems with GPU limitations.
method Parallel BnB nodes in GPU batches, using padding and custom kernels.
result One to two orders of magnitude speedup and zero optimality gap.
StructureBoost improves gradient boosting for complex categorical variables efficiently.
problem Efficiently handling complex categorical variables with known structure.
method Two methods to overcome computational obstacles in SCDT enumeration for structured categorical variables.
result StructureBoost outperforms existing packages on complex categorical problems.
Paper solves high-order portfolio optimization with cardinality constraint.
problem Solving non-convex cardinality constrained high-order portfolio optimization.
method Transformed cardinality constraint into penalty term, proposed pDCA, pDCAe, and SCA algorithms.
result Proposed algorithms achieve high utility and sparse solutions efficiently.
We describe discrete restricted Boltzmann machines: probabilistic graphical models with bipartite interactions between visible and hidden discrete variables. Examples are binary restricted Boltzmann machines and discrete naive Bayes models. We detail the inference functions and distributed representations arising in th…
Develops a new framework for causal models on cyclic graphs, solving unique solvability issues.
problem Challenges in specifying unique probability distributions for cyclic functional causal models.
method Introduces a new probability rule and graph-separation property (p-separation) for cyclic fCMs.
result Proves p-separation is sound and complete for all consistent cyclic fCMs, recovering d-separation for DAGs.
Learning rule consistency tied to non-existence of real-valued measurable cardinals.
problem Consistency of k-NN learning rule in metric spaces.
method Analyzing separable subspaces and density conditions.
result The k-NN classifier's consistency depends on the absence of real-valued measurable cardinals.
Quantum computing tackles non-convex portfolio optimization with cardinality constraints.
problem Non-convex portfolio optimization problems in asset management.
method Application of quantum annealing with non-linear cardinality constraints.
result Quantum portfolio optimization yields smaller, more profitable portfolios.
CardiCat generates synthetic data for high-cardinality tabular datasets.
problem Learning complexities of high-cardinality categorical features in tabular data.
method Substitutes one-hot encoding with regularized dual encoder-decoder embedding layers.
result Generates high-quality synthetic data with a smaller parameter space.
Unified comparison of gradient boosting algorithms for insurance claims.
problem Improving predictive accuracy and computational efficiency in insurance claim prediction.
method Unified notation and comprehensive numerical study comparing 12 gradient boosting algorithms on 5 datasets.
result No trade-off between model adequacy and predictive accuracy.
Homotopy cardinality counts augmentations of Legendrian knots.
problem Counting augmentations of Legendrian knots.
method Assigning ruling polynomials and proving homotopy cardinality results.
result Homotopy cardinality of representation categories is a multiple of ruling polynomials.
We show how to use a variational approximation to the logistic function to perform approximate inference in Bayesian networks containing discrete nodes with continuous parents. Essentially, we convert the logistic function to a Gaussian, which facilitates exact inference, and then iteratively adjust the variational par…
Improved classification model for high-cardinality categorical predictors.
problem Handling high-cardinality categorical predictors and non-linear data.
method Data-driven binning of spline functions and shrinkage estimators.
result Improved classification precision with interpretable predictors.
The paper optimizes asset selection for index trackers and enhanced trackers with varying cardinality constraints.
problem Optimizing asset selection for index trackers and enhanced trackers with cardinality constraints.
method Divided into two steps: asset pre-selection and asset weight estimation. Used eight pre-selection procedures with different combinations of selection methods and regression types.
result Out-of-sample tracking errors are roughly proportional to 1/sqrt(cardinality). OLS is more effective than LAD, BE marginally more effective than FS, and (n) marginally more effective than (c).
Paper explores variable skipping to speed up range density estimation.
problem Efficiently estimating range densities over high-dimensional data.
method Variable skipping technique to accelerate range density estimation.
result 10-100x efficiency improvements in challenging high-quantile error metrics.
We introduce a notion of cardinality for the augmentation category associated to a Legendrian knot or link in standard contact R^3. This `homotopy cardinality' is an invariant of the category and allows for a weighted count of augmentations, which we prove to be determined by the ruling polynomial of the link. We prese…
Paper proposes CIV estimator for categorical instruments in small sample settings.
problem Estimation with categorical instruments in settings with few observations per category.
method CIV estimator leveraging regularization assumption for latent categorical variable.
result CIV estimator is asymptotically normal, efficient, and semiparametrically efficient under homoskedasticity.
The aim of this paper is a characterization of great antipodal sets of complex Grassmannian manifolds as certain designs with the smallest cardinalities.
We develop a new statistical test for comparing variables with varying scales.
problem Comparing variables with different scales in multidimensional spaces.
method Order based on expectations of random variables, generalized stochastic dominance (GSD) order, regularized statistical test, linear optimization, imprecise probability models.
result Validated through multidimensional data from various fields.
Paper uses deep learning for accurate, monotonic cardinality estimation.
problem Accurate and monotonic cardinality estimation for similarity selection.
method Feature extraction to Hamming space, followed by deep learning regression.
result Demonstrates improved query optimizer performance.
Quandles can be regarded as generalizations of symmetric spaces. Among symmetric spaces, two-point homogeneous Riemannian manifolds would be the most fundamental ones. In this paper, we define two-point homogeneous quandles analogously, and classify those with prime cardinality.
With any non necessarily orientable unpunctured marked surface (S,M) we associate a commutative algebra, called quasi-cluster algebra, equipped with a distinguished set of generators, called quasi-cluster variables, in bijection with the set of arcs and one-sided simple closed curves in (S,M). Quasi-cluster variables a…