In this paper we address cardinality estimation problem which is an important subproblem in query optimization. Query optimization is a part of every relational DBMS responsible for finding the best way of the execution for the given query. These ways are called plans. The execution time of different plans may differ b…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “cardinality estimation”
Structured high-cardinality data arises in many domains, and poses a major challenge for both modeling and inference. Graphical models are a popular approach to modeling structured data but they are unsuitable for high-cardinality variables. The count-min (CM) sketch is a popular approach to estimating probabilities in…
Paper uses deep learning for accurate, monotonic cardinality estimation.
problem Accurate and monotonic cardinality estimation for similarity selection.
method Feature extraction to Hamming space, followed by deep learning regression.
result Demonstrates improved query optimizer performance.
We present a study on reinforcement learning (RL) from human bandit feedback for sequence-to-sequence learning, exemplified by the task of bandit neural machine translation (NMT). We investigate the reliability of human bandit feedback, and analyze the influence of reliability on the learnability of a reward estimator,…
Cardinal scores (numeric ratings) collected from people are well known to suffer from miscalibrations. A popular approach to address this issue is to assume simplistic models of miscalibration (such as linear biases) to de-bias the scores. This approach, however, often fares poorly because people's miscalibrations are …
Asset pre-selection for a cardinality constrained index tracking portfolio with optional enhancementq-fin.PM
The paper optimizes asset selection for index trackers and enhanced trackers with varying cardinality constraints.
problem Optimizing asset selection for index trackers and enhanced trackers with cardinality constraints.
method Divided into two steps: asset pre-selection and asset weight estimation. Used eight pre-selection procedures with different combinations of selection methods and regression types.
result Out-of-sample tracking errors are roughly proportional to 1/sqrt(cardinality). OLS is more effective than LAD, BE marginally more effective than FS, and (n) marginally more effective than (c).
Novel GLMMNet model tackles high-cardinality categorical features in actuarial applications.
problem Inadequate encoding methods for high-cardinality categorical features in actuarial data.
method Generalised Linear Mixed Model Neural Network (GLMMNet) integrating a generalised linear mixed model in a deep learning framework.
result GLMMNet often outperforms or performs comparably with entity embedded neural networks, providing transparency.
Study improves top-k set prediction with low cardinality.
problem Improving top-k set prediction accuracy with low cardinality.
method Introduces new target loss function and surrogate losses.
result Demonstrates effectiveness of cardinality-aware algorithms.
In this paper we propose and discuss different 0-1 linear models in order to solve the cardinality constrained portfolio problem by using factor models. Factor models are used to build portfolios to track indexes, together with other objectives, also need a smaller number of parameters to estimate than the classical Ma…
Subject-specific Deep Neural Networks for Count Data with High-cardinality Categorical Featurescs.LG
Proposes a new DNN framework for count data with high-cardinality features.
problem Real-world data often have correlations and high-cardinality categorical features that traditional DNNs overlook.
method Introduces a hierarchical likelihood learning framework with gamma random effects for Poisson DNNs.
result Improves prediction performance by capturing nonlinear effects and subject-specific cluster effects.
We prove that on a punctured oriented surface with Euler characteristic chi < 0, the maximal cardinality of a set of essential simple arcs that are pairwise non-homotopic and intersecting at most once is 2|chi|(|chi|+1). This gives a cubic estimate in |chi| for a set of curves pairwise intersecting at most once on a cl…
A cardinality-constrained portfolio caps the number of stocks to be traded across and within groups or sectors. These limitations arise from real-world scenarios faced by fund managers, who are constrained by transaction costs and client preferences as they seek to maximize return and limit risk. We develop a new appro…
When eliciting judgements from humans for an unknown quantity, one often has the choice of making direct-scoring (cardinal) or comparative (ordinal) measurements. In this paper we study the relative merits of either choice, providing empirical and theoretical guidelines for the selection of a measurement scheme. We pro…
We study the cardinality of the set of manifolds homotopy equivalent to a given manifold M and compare it to the cardinality of the structure set of M.
Study on loops on non-orientable surfaces, determining cardinality and order.
problem Determining the cardinality and order of maximal complete 1-systems of loops on non-orientable surfaces.
method Proved the cardinality of maximal systems of arcs pairwise-intersecting at most once on a non-orientable surface is , and used this to determine the cardinality of maximal complete 1-systems of loops.
result Exact cardinality of maximal complete 1-systems of loops on punctured projective planes is determined.
The cardinality constraint is an intrinsic way to restrict the solution structure in many domains, for example, sparse learning, feature selection, and compressed sensing. To solve a cardinality constrained problem, the key challenge is to solve the projection onto the cardinality constraint set, which is NP-hard in ge…
Paper solves high-order portfolio optimization with cardinality constraint.
problem Solving non-convex cardinality constrained high-order portfolio optimization.
method Transformed cardinality constraint into penalty term, proposed pDCA, pDCAe, and SCA algorithms.
result Proposed algorithms achieve high utility and sparse solutions efficiently.
This study compares machine learning methods for high-cardinality categorical variables.
problem Machine learning struggles with high-cardinality categorical variables.
method Empirical comparison of tree-boosting, deep neural networks, and linear mixed effects models.
result Tree-boosting with random effects outperforms deep neural networks with random effects.
Learning rule consistency tied to non-existence of real-valued measurable cardinals.
problem Consistency of k-NN learning rule in metric spaces.
method Analyzing separable subspaces and density conditions.
result The k-NN classifier's consistency depends on the absence of real-valued measurable cardinals.
Quantum computing tackles non-convex portfolio optimization with cardinality constraints.
problem Non-convex portfolio optimization problems in asset management.
method Application of quantum annealing with non-linear cardinality constraints.
result Quantum portfolio optimization yields smaller, more profitable portfolios.
Cardinality potentials are a generally useful class of high order potential that affect probabilities based on how many of D binary variables are active. Maximum a posteriori (MAP) inference for cardinality potential models is well-understood, with efficient computations taking O(DlogD) time. Yet efficient marginalizat…
CardiCat generates synthetic data for high-cardinality tabular datasets.
problem Learning complexities of high-cardinality categorical features in tabular data.
method Substitutes one-hot encoding with regularized dual encoder-decoder embedding layers.
result Generates high-quality synthetic data with a smaller parameter space.
Homotopy cardinality counts augmentations of Legendrian knots.
problem Counting augmentations of Legendrian knots.
method Assigning ruling polynomials and proving homotopy cardinality results.
result Homotopy cardinality of representation categories is a multiple of ruling polynomials.
We introduce a notion of cardinality for the augmentation category associated to a Legendrian knot or link in standard contact R^3. This `homotopy cardinality' is an invariant of the category and allows for a weighted count of augmentations, which we prove to be determined by the ruling polynomial of the link. We prese…
Great antipodal sets of complex Grassmannian manifolds as designs with the smallest cardinalitiesmath.CO
The aim of this paper is a characterization of great antipodal sets of complex Grassmannian manifolds as certain designs with the smallest cardinalities.
Improves treatment effect estimation by reducing sample size needed.
problem Estimating causal treatment effects from observational data requires many covariates, increasing sample size.
method Proposes a nonconvex joint sparsity regularization objective function to recover a sparse subset of covariates.
result Improves sample complexity to scale with the size of the sparse subset and log of the total covariates.
Quandles can be regarded as generalizations of symmetric spaces. Among symmetric spaces, two-point homogeneous Riemannian manifolds would be the most fundamental ones. In this paper, we define two-point homogeneous quandles analogously, and classify those with prime cardinality.
We propose a stochastic variance reduced optimization algorithm for solving sparse learning problems with cardinality constraints. Sufficient conditions are provided, under which the proposed algorithm enjoys strong linear convergence guarantees and optimal estimation accuracy in high dimensions. We further extend the …
CASP improves portfolio optimization by considering asset covariance.
problem Infeasibility in cardinality-constrained portfolio optimization.
method CASP uses volatility-normalized selection and covariance-aware projection.
result CASP-Basic delivers lower portfolio variance than standard Euclidean repair.
Study on top- classification with new loss functions and algorithms.
problem Improving multi-class classification accuracy and cardinality trade-off.
method Introducing cardinality-aware loss functions and deriving their consistency bounds.
result New cardinality-aware algorithms for top- classification.
Random shuffle method boosts HF dataset size 10-21 times.
problem Narrow dataset cardinality in HF clinical studies.
method Random shuffle method to enhance dataset size.
result Enhanced HF dataset size 10-21 times.
Data in the form of pairwise comparisons arises in many domains, including preference elicitation, sporting competitions, and peer grading among others. We consider parametric ordinal models for such pairwise comparison data involving a latent vector that represents the "qualities" of the ite…
MMbeddings reduces categorical embeddings by treating them as latent effects, significantly decreasing parameters and mitigating overfitting.
problem Large cardinalities in categorical embeddings lead to high parameter counts and overfitting.
method MMbeddings treats embeddings as latent random effects in a variational autoencoder framework, reducing parameter count and mitigating overfitting.
result MMbeddings consistently outperforms traditional embeddings across various tasks, demonstrating its potential in machine learning applications.
This work restricts hidden cardinality in causal models to infer causal relations.
problem Causal relations between variables with a common unobserved cause cannot be directly inferred.
method Derive inequality constraints from d-separation in causal models with known cardinalities of unobserved variables.
result Inference of causal relations is possible with additional assumptions about cardinalities.
Develops a new model for RLHF accounting for partially observed states and intermediate feedback.
problem Lack of models for partially observed states and intermediate feedback in RLHF.
method PORRL model with cardinal and dueling feedback methods.
result Demonstrates improved learning and alignment with new model-based and model-free methods.
We introduce the notion of a positive opetope and positive opetopic cardinals as certain finite combinatorial structures. The positive opetopic cardinals to positive-to-one polygraphs are like simple graphs to free omega-categories over omega-graphs, c.f. [MZ]. In particular, they allow us to give an explicit combinato…
We study and classify topologically invariant -ideals with a Borel base on the Hilbert cube and evaluate their cardinal characteristics. One of the results of this paper solves (positively) a known problem whether the minimal cardinalities of the families of Cantor sets covering the unit interval and the Hilbert cub…
New algorithm solves large cardinality-constrained clustering problems.
problem Optimizing clustering with cardinality constraints.
method Branch-and-cut technique with SDP relaxation and polyhedral cuts.
result Solves real-world instances 10 times larger than previous methods.
Size Matters: Cardinality-Constrained Clustering and Outlier Detection via Conic Optimizationmath.OC
Plain vanilla K-means clustering has proven to be successful in practice, yet it suffers from outlier sensitivity and may produce highly unbalanced clusters. To mitigate both shortcomings, we formulate a joint outlier detection and clustering problem, which assigns a prescribed number of datapoints to an auxiliary outl…
End-to-end portfolio optimization framework bypassing covariance matrix estimation.
problem Optimizing portfolios with large numbers of assets and constraints.
method Deep learning approach that directly optimizes asset distributions without forecasting.
result Framework outperforms classical methods and handles various constraints.
Improved classification model for high-cardinality categorical predictors.
problem Handling high-cardinality categorical predictors and non-linear data.
method Data-driven binning of spline functions and shrinkage estimators.
result Improved classification precision with interpretable predictors.
What makes you unique?stat.AP
Paper proposes a uniqueness Shapley measure to compare variable importance.
problem Comparing the importance of different variables in identifying subjects.
method Uses Shapley value to combine reductions in log cardinality due to revealing variables.
result Demonstrates speedup in calculating variable importance.
New approach to sparse optimal transport for matching tokens with experts.
problem Sparse matching of tokens with experts in neural networks.
method Sparsity-constrained optimal transport with cardinality constraints.
result Solves nonconvex cardinality constraints with gradient methods.
In recent years, multi object tracking (MOT) problem has drawn attention to it and has been studied in various research areas. However, some of the challenging problems including time dependent cardinality, unordered measurement set, and object labeling remain unclear. In this paper, we propose robust nonparametric met…
Let p be a puncture of a punctured sphere, and let Q be the set of all other punctures. We prove that the maximal cardinality of a set of arcs pairwise intersecting at most once, which start at p and end in Q, is |X|(|X| + 1). We deduce that the maximal cardinality of a set of arcs with arbitrary endpoints pairwise int…
The paper describes a cover of strata of k-differentials with a formula for fiber cardinality.
problem Understanding the ramification locus and cardinality of fibers in strata of k-differentials.
method Intersection calculations on multi-scale compactification and flat geometry.
result A formula for the cardinality of each fiber involving the k-factorial function.
HAMD optimizes cubic portfolios without quadratization, achieving better results.
problem Optimizing higher-order portfolio models with reduced distortion.
method Hybrid pipeline combining continuous Hamiltonian search, cardinality-preserving projection, and iterated local search.
result HAMD achieves significantly lower native cubic objective values than classical heuristics.
Paper tackles P vs NP problem in portfolio optimization with cardinality constraints and Black-Scholes derivatives.
problem Operationalizing the P vs NP problem in cardinality-constrained portfolio selection.
method Mixed-integer quadratic program with genetic algorithms, Monte Carlo sampling, and greedy screening.
result Cardinality constraint reshapes efficient frontier, highlighting trade-offs between stability and computational cost.