Graph Neural Networks (GNNs) are powerful to learn the representation of graph-structured data. Most of the GNNs use the message-passing scheme, where the embedding of a node is iteratively updated by aggregating the information of its neighbors. To achieve a better expressive capability of node influences, attention m…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “Cardinality Preserving Attention”
CardiGraphormer: Unveiling the Power of Self-Supervised Learning in Revolutionizing Drug Discoverycs.LG
CardiGraphormer uses SSL and GNNs to improve drug discovery.
problem Challenges in drug discovery due to combinatorial chemical space and limited approved drugs.
method Combines self-supervised learning, Graph Neural Networks, and Cardinality Preserving Attention.
result Enhanced predictive performance and interpretability in drug discovery.
A subgroup of a Kac-Moody group is called bounded if it is contained in the intersection of two finite type parabolic subgroups of opposite signs. In this paper, we study the isomorphisms between Kac-Moody groups over arbitrary fields of cardinality at least 4, which preserve the set of bounded subgroups. We show that …
Reweighted l1-algorithms have attracted a lot of attention in the field of applied mathematics. A unified framework of such algorithms has been recently proposed by Zhao and Li. In this paper we construct a few new examples of reweighted l1-methods. These functions are certain concave approximations of the l0-norm func…
Subject-specific Deep Neural Networks for Count Data with High-cardinality Categorical Featurescs.LG
Proposes a new DNN framework for count data with high-cardinality features.
problem Real-world data often have correlations and high-cardinality categorical features that traditional DNNs overlook.
method Introduces a hierarchical likelihood learning framework with gamma random effects for Poisson DNNs.
result Improves prediction performance by capturing nonlinear effects and subject-specific cluster effects.
New approach to sparse optimal transport for matching tokens with experts.
problem Sparse matching of tokens with experts in neural networks.
method Sparsity-constrained optimal transport with cardinality constraints.
result Solves nonconvex cardinality constraints with gradient methods.
HAMD optimizes cubic portfolios without quadratization, achieving better results.
problem Optimizing higher-order portfolio models with reduced distortion.
method Hybrid pipeline combining continuous Hamiltonian search, cardinality-preserving projection, and iterated local search.
result HAMD achieves significantly lower native cubic objective values than classical heuristics.
SAttention improves long sequence attention with smoothed skeleton sketching.
problem Quadratic complexity of vanilla Attention makes it unsuitable for long sequence tasks.
method SAttention uses smoothing and matrix sketching to balance information preservation and computation.
result SAttention significantly outperforms vanilla Attention and other Attention variants.
Novel approximation hierarchy for sparse quadratic programs.
problem Sparse Quadratic Programs with Cardinality Constraints.
method Exploits rank-dominating eigenvectors for min-max optimization over binary variables.
result Efficient screening of nonzero elements with scalable optimization algorithms.
Transformers preserve support and can approximate any continuous map.
problem Understanding the mathematical properties of transformers.
method Characterizing maps between measures that can be represented as transformers and proving their properties.
result Transformers preserve support and have uniformly continuous Fréchet derivatives.
Study improves top-k set prediction with low cardinality.
problem Improving top-k set prediction accuracy with low cardinality.
method Introduces new target loss function and surrogate losses.
result Demonstrates effectiveness of cardinality-aware algorithms.
Submodular functions are a broad class of set functions, which naturally arise in diverse areas. Many algorithms have been suggested for the maximization of these functions. Unfortunately, once the function deviates from submodularity, the known algorithms may perform arbitrarily poorly. Amending this issue, by obtaini…
In recent years, multi object tracking (MOT) problem has drawn attention to it and has been studied in various research areas. However, some of the challenging problems including time dependent cardinality, unordered measurement set, and object labeling remain unclear. In this paper, we propose robust nonparametric met…
A cardinality-constrained portfolio caps the number of stocks to be traded across and within groups or sectors. These limitations arise from real-world scenarios faced by fund managers, who are constrained by transaction costs and client preferences as they seek to maximize return and limit risk. We develop a new appro…
We study the cardinality of the set of manifolds homotopy equivalent to a given manifold M and compare it to the cardinality of the structure set of M.
SympFormer accelerates attention blocks using inertial dynamics on density spaces.
problem Improving the efficiency of self-attention blocks in Transformers.
method Introduced accelerated attention blocks derived from inertial Nesterov dynamics on density spaces.
result Accelerated attention blocks converge faster than classical blocks while preserving oracle calls.
Study invariant Lipschitz bandits, improving regret bounds.
problem Optimizing decisions under symmetry in online settings.
method Integrates side observations using group orbits into UniformMesh algorithm.
result Improved regret bound for invariant Lipschitz bandit class.
In this paper, we analyze the behavior of the multivariate symmetric uncertainty (MSU) measure through the use of statistical simulation techniques under various mixes of informative and non-informative randomly generated features. Experiments show how the number of attributes, their cardinalities, and the sample size …
Study on loops on non-orientable surfaces, determining cardinality and order.
problem Determining the cardinality and order of maximal complete 1-systems of loops on non-orientable surfaces.
method Proved the cardinality of maximal systems of arcs pairwise-intersecting at most once on a non-orientable surface is , and used this to determine the cardinality of maximal complete 1-systems of loops.
result Exact cardinality of maximal complete 1-systems of loops on punctured projective planes is determined.
In this paper we address cardinality estimation problem which is an important subproblem in query optimization. Query optimization is a part of every relational DBMS responsible for finding the best way of the execution for the given query. These ways are called plans. The execution time of different plans may differ b…
An extended Kleinian group whose orientation-preserving half is a Schottky group is called an extended Schottky group. These groups correspond to the real points in the Schottky space. Their geometric structures is well known and it permits to provide information on the locus of fixed points of symmetries of handlebodi…
The cardinality constraint is an intrinsic way to restrict the solution structure in many domains, for example, sparse learning, feature selection, and compressed sensing. To solve a cardinality constrained problem, the key challenge is to solve the projection onto the cardinality constraint set, which is NP-hard in ge…
Paper solves high-order portfolio optimization with cardinality constraint.
problem Solving non-convex cardinality constrained high-order portfolio optimization.
method Transformed cardinality constraint into penalty term, proposed pDCA, pDCAe, and SCA algorithms.
result Proposed algorithms achieve high utility and sparse solutions efficiently.
This study compares machine learning methods for high-cardinality categorical variables.
problem Machine learning struggles with high-cardinality categorical variables.
method Empirical comparison of tree-boosting, deep neural networks, and linear mixed effects models.
result Tree-boosting with random effects outperforms deep neural networks with random effects.
Learning rule consistency tied to non-existence of real-valued measurable cardinals.
problem Consistency of k-NN learning rule in metric spaces.
method Analyzing separable subspaces and density conditions.
result The k-NN classifier's consistency depends on the absence of real-valued measurable cardinals.
Quantum computing tackles non-convex portfolio optimization with cardinality constraints.
problem Non-convex portfolio optimization problems in asset management.
method Application of quantum annealing with non-linear cardinality constraints.
result Quantum portfolio optimization yields smaller, more profitable portfolios.
Cardinality potentials are a generally useful class of high order potential that affect probabilities based on how many of D binary variables are active. Maximum a posteriori (MAP) inference for cardinality potential models is well-understood, with efficient computations taking O(DlogD) time. Yet efficient marginalizat…
CardiCat generates synthetic data for high-cardinality tabular datasets.
problem Learning complexities of high-cardinality categorical features in tabular data.
method Substitutes one-hot encoding with regularized dual encoder-decoder embedding layers.
result Generates high-quality synthetic data with a smaller parameter space.
Homotopy cardinality counts augmentations of Legendrian knots.
problem Counting augmentations of Legendrian knots.
method Assigning ruling polynomials and proving homotopy cardinality results.
result Homotopy cardinality of representation categories is a multiple of ruling polynomials.
Structured high-cardinality data arises in many domains, and poses a major challenge for both modeling and inference. Graphical models are a popular approach to modeling structured data but they are unsuitable for high-cardinality variables. The count-min (CM) sketch is a popular approach to estimating probabilities in…
Asset pre-selection for a cardinality constrained index tracking portfolio with optional enhancementq-fin.PM
The paper optimizes asset selection for index trackers and enhanced trackers with varying cardinality constraints.
problem Optimizing asset selection for index trackers and enhanced trackers with cardinality constraints.
method Divided into two steps: asset pre-selection and asset weight estimation. Used eight pre-selection procedures with different combinations of selection methods and regression types.
result Out-of-sample tracking errors are roughly proportional to 1/sqrt(cardinality). OLS is more effective than LAD, BE marginally more effective than FS, and (n) marginally more effective than (c).
Introduces Spectral Attention for better long-range time series forecasting.
problem Challenges in capturing long-range dependencies in time series forecasting.
method Spectral Attention mechanism that preserves temporal correlations and long-range dependencies.
result Achieves state-of-the-art results on 11 real-world time series datasets.
We introduce a notion of cardinality for the augmentation category associated to a Legendrian knot or link in standard contact R^3. This `homotopy cardinality' is an invariant of the category and allows for a weighted count of augmentations, which we prove to be determined by the ruling polynomial of the link. We prese…
Great antipodal sets of complex Grassmannian manifolds as designs with the smallest cardinalitiesmath.CO
The aim of this paper is a characterization of great antipodal sets of complex Grassmannian manifolds as certain designs with the smallest cardinalities.
Paper uses deep learning for accurate, monotonic cardinality estimation.
problem Accurate and monotonic cardinality estimation for similarity selection.
method Feature extraction to Hamming space, followed by deep learning regression.
result Demonstrates improved query optimizer performance.
Graph embedding methods represent nodes in a continuous vector space, preserving information from the graph (e.g. by sampling random walks). There are many hyper-parameters to these methods (such as random walk length) which have to be manually tuned for every graph. In this paper, we replace random walk hyper-paramete…
Let and be connected orientable surfaces of genus , punctures, and empty boundary. Let also be an edge-preserving alternating map between their Hatcher-Thurston graphs. We prove that $g_{1} \leq g_{2…
AttS2S-VC: Sequence-to-Sequence Voice Conversion with Attention and Context Preservation Mechanismseess.AS
This paper describes a method based on a sequence-to-sequence learning (Seq2Seq) with attention and context preservation mechanism for voice conversion (VC) tasks. Seq2Seq has been outstanding at numerous tasks involving sequence modeling such as speech synthesis and recognition, machine translation, and image captioni…
Quandles can be regarded as generalizations of symmetric spaces. Among symmetric spaces, two-point homogeneous Riemannian manifolds would be the most fundamental ones. In this paper, we define two-point homogeneous quandles analogously, and classify those with prime cardinality.
Novel GLMMNet model tackles high-cardinality categorical features in actuarial applications.
problem Inadequate encoding methods for high-cardinality categorical features in actuarial data.
method Generalised Linear Mixed Model Neural Network (GLMMNet) integrating a generalised linear mixed model in a deep learning framework.
result GLMMNet often outperforms or performs comparably with entity embedded neural networks, providing transparency.
Study on top- classification with new loss functions and algorithms.
problem Improving multi-class classification accuracy and cardinality trade-off.
method Introducing cardinality-aware loss functions and deriving their consistency bounds.
result New cardinality-aware algorithms for top- classification.
A new unsupervised method removes CT metal artifacts using beta-CycleGAN and attention.
problem Metal artifact reduction in computed tomography (CT) images.
method Unsupervised learning using a beta-CycleGAN architecture with attention mechanism.
result Improved metal artifact removal that preserves image details.
Random shuffle method boosts HF dataset size 10-21 times.
problem Narrow dataset cardinality in HF clinical studies.
method Random shuffle method to enhance dataset size.
result Enhanced HF dataset size 10-21 times.
MMbeddings reduces categorical embeddings by treating them as latent effects, significantly decreasing parameters and mitigating overfitting.
problem Large cardinalities in categorical embeddings lead to high parameter counts and overfitting.
method MMbeddings treats embeddings as latent random effects in a variational autoencoder framework, reducing parameter count and mitigating overfitting.
result MMbeddings consistently outperforms traditional embeddings across various tasks, demonstrating its potential in machine learning applications.
We introduce a new local sparse attention layer that preserves two-dimensional geometry and locality. We show that by just replacing the dense attention layer of SAGAN with our construction, we obtain very significant FID, Inception score and pure visual improvements. FID score is improved from to on Im…
We present a study on reinforcement learning (RL) from human bandit feedback for sequence-to-sequence learning, exemplified by the task of bandit neural machine translation (NMT). We investigate the reliability of human bandit feedback, and analyze the influence of reliability on the learnability of a reward estimator,…
Realistic music generation has always remained as a challenging problem as it may lack structure or rationality. In this work, we propose a deep learning based music generation method in order to produce old style music particularly JAZZ with rehashed melodic structures utilizing a Bi-directional Long Short Term Memory…
This work restricts hidden cardinality in causal models to infer causal relations.
problem Causal relations between variables with a common unobserved cause cannot be directly inferred.
method Derive inequality constraints from d-separation in causal models with known cardinalities of unobserved variables.
result Inference of causal relations is possible with additional assumptions about cardinalities.