ptype infers data types robustly in real-world data.
problem Type inference fails with missing data and anomalies.
method Probabilistic robust type inference method.
result Outperforms existing methods.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “Data Types”
DCoM uses deep neural networks to detect semantic data types from raw column values.
problem Detecting semantic data types from dirty and unseen data.
method DCoM employs multi-input NLP-based deep neural networks trained on 686,765 data columns.
result DCoM outperforms existing methods significantly on 78 different semantic data types.
New model clusters mixed-type data with missing values, improving air quality analysis.
problem Clustering mixed-type data with missing values and regime persistence.
method Statistical jump model incorporating regime persistence and handling missing data.
result Superior performance in inferring persistent air quality regimes compared to traditional methods.
Proposes CDTD, a diffusion model for mixed-type tabular data.
problem Adapting diffusion models to mixed-type tabular data.
method Score matching and score interpolation for continuous features, adaptive noise schedules for categorical features.
result Consistently outperforms state-of-the-art models in mixed-type tabular data.
Correctly detecting the semantic type of data columns is crucial for data science tasks such as automated data cleaning, schema matching, and data discovery. Existing data preparation and analysis systems rely on dictionary lookups and regular expression matching to detect semantic types. However, these matching-based …
Generative model synthesizes complex data structures with composite and nested types.
problem Synthesizing complex data structures with composite and nested types.
method Generic framework using causal transformers for struct and list generation.
result Consistently outperforms state-of-the-art models on standard and complex hierarchical datasets.
Framework generates private synthetic data for unlabeled mixed-type data.
problem Generating private synthetic data for unlabeled mixed-type data.
method Combines autoencoders and GANs for differential privacy.
result Learned model generates synthetic data with similar statistical properties.
VAEM extends VAEs to handle mixed-type data heterogeneity.
problem Heterogeneous data with different types and marginal distributions.
method Two-stage training approach to handle mixed-type data.
result VAEM improves deep generative model performance on diverse tasks.
Joint and individual variation explained (JIVE) for integrated analysis of multiple data typesstat.ML
Research in several fields now requires the analysis of data sets in which multiple high-dimensional types of data are available for a common set of objects. In particular, The Cancer Genome Atlas (TCGA) includes data from several diverse genomic technologies on the same cancerous tumor samples. In this paper we introd…
Photometric Data-driven Classification of Type Ia Supernovae in the Open Supernova Catalogastro-ph.IM
Machine learning detects type Ia supernovae from photometric data.
problem Detecting type Ia supernovae accurately from photometric data.
method Machine learning approach using only real observation data.
result Good results on real data from the Open Supernovae Catalog.
A new method clusters mixed-type data efficiently.
problem Clustering mixed-type data with continuous and categorical variables.
method Extends Information Bottleneck principle to heterogeneous data using generalised product kernels.
result DIBmix outperforms four established methods in various scenarios.
Autoencoder-based integrative multi-omics data embedding that allows for confounder adjustmentsstat.AP
In the integrative analyses of omics data, it is often of interest to extract data representation from one data type that best reflect its relations with another data type. This task is traditionally fulfilled by linear methods such as canonical correlation analysis (CCA) and partial least squares (PLS). However, infor…
Fusing heterogeneous data setsq-bio.GN
In systems biology, it is common to measure biochemical entities at different levels of the same biological system. One of the central problems for the data fusion of such data sets is the heterogeneity of the data. This thesis discusses two types of heterogeneity. The first one is the type of data, such as metabolomic…
During the directional drilling, a bit may sometimes go to a nonproductive rock layer due to the gap about 20m between the bit and high-fidelity rock type sensors. The only way to detect the lithotype changes in time is the usage of Measurements While Drilling (MWD) data. However, there are no general mathematical mode…
The paper classifies U.S. crop types using hyperspectral satellite imagery.
problem Classifying crop types from hyperspectral satellite imagery.
method Gaussian Bayesian models and neural networks applied to NASA data.
result Bayesian methods outperform standard LDA and QDA.
New method for robust matrix completion with mixed data types.
problem Recovering a structured low rank matrix with mixed data types.
method Proposes a computationally feasible statistical approach with strong recovery guarantees for mixed data types.
result Strong recovery guarantees for low rank matrix completion with mixed data types.
Modern data acquisition based on high-throughput technology is often facing the problem of missing data. Algorithms commonly used in the analysis of such large-scale data often depend on a complete set. Missing value imputation offers a solution to this problem. However, the majority of available imputation methods are…
A novel graph spectral method for mixed categorical and numerical data.
problem Feature learning for mixed data types (numerical and categorical).
method Graph spectral decomposition of the graph Laplacian to model probabilistic dependence structure.
result Increased separability and clusterability of observations in the transformed feature space.
Mining Rank Datacs.LG
The problem of frequent pattern mining has been studied quite extensively for various types of data, including sets, sequences, and graphs. Somewhat surprisingly, another important type of data, namely rank data, has received very little attention in data mining so far. In this paper, we therefore addresses the problem…
Outlier detection amounts to finding data points that differ significantly from the norm. Classic outlier detection methods are largely designed for single data type such as continuous or discrete. However, real world data is increasingly heterogeneous, where a data point can have both discrete and continuous attribute…
Bayesian test assesses dependence between mixed data types.
problem Assessing dependence between text, image, and sound data.
method Bayesian kernelised correlation test using Dirichlet process model.
result Demonstrated effectiveness compared to other methods.
New model identifies cell-specific genes for cancer prognosis.
problem No statistical model to integrate multiscale cancer data.
method Bayesian generalized promotion time cure models (GPTCMs).
result Improves cancer prognosis by identifying cell-specific genes.
Aggregating multiple types of complex data in stock market prediction: A model-independent frameworkq-fin.CP
The increasing richness in volume, and especially types of data in the financial domain provides unprecedented opportunities to understand the stock market more comprehensively and makes the price prediction more accurate than before. However, they also bring challenges to classic statistic approaches since those model…
New method generates diverse EHR data types while maintaining privacy.
problem Privacy risks in sharing EHRs across large scales.
method Refined GAN model, feature constraints, and utility measures.
result New model preserves EHR statistics, correlations, and constraints.
There are many methods developed to approximate a cloud of vectors embedded in high-dimensional space by simpler objects: starting from principal points and linear manifolds to self-organizing maps, neural gas, elastic maps, various types of principal curves and principal trees, and so on. For each type of approximator…
DP synthetic data may inflate statistical test results, caution advised.
problem Inflated Type I errors in statistical tests on DP-synthetic data.
method Evaluation of Mann-Whitney U test, t-test, chi-squared test, and median test on DP-synthetic data generated from real-world and simulated datasets using various DP-synthetic data generation methods.
result A large portion of evaluation results showed inflated Type I errors, especially at low privacy levels.
This paper addresses the challenges in classifying textual data obtained from open online platforms, which are vulnerable to distortion. Most existing classification methods minimize the overall classification error and may yield an undesirably large type I error (relevant textual messages are classified as irrelevant)…
Given data over the joint distribution of two random variables and , we consider the problem of inferring the most likely causal direction between and . In particular, we consider the general case where both and may be univariate or multivariate, and of the same or mixed data types. We take an inf…
Energy trees handle complex data structures with multiple variable types.
problem Handling intricate data structures with various types of covariates.
method Energy trees, a regression and classification model, use energy statistics to accommodate structured covariates of different types.
result Energy trees maintain statistical foundations, interpretability, and robustness to overfitting.
In the Pioneer 100 (P100) Wellness Project (Price and others, 2017), multiple types of data are collected on a single set of healthy participants at multiple timepoints in order to characterize and optimize wellness. One way to do this is to identify clusters, or subgroups, among the participants, and then to tailor pe…
Paper proposes a new model using Capsule Networks to classify malware types.
problem Classifying malware types with varying behaviors.
method Ensemble Capsule Network model using bootstrap aggregating.
result Improved accuracy in classifying malware types.
New model clusters cells and individuals, revealing genetic influences on cell types.
problem Clustering nested data with group-level and observation-level variables.
method Nested Atoms Model (NAM), Bayesian nonparametric approach.
result Identifies clusters of genetically similar individuals with homogeneous cell-type profiles.
Proves existence and uniqueness of solutions for A_n tt*-Toda equations.
problem Existence and uniqueness of solutions for A_n tt*-Toda equations.
method Proof of existence and uniqueness for any n, new treatment of asymptotic data.
result Existence and uniqueness of global solutions for any n.
MMM model clusters mixed-type longitudinal data efficiently.
problem Challenges in clustering multivariate longitudinal mixed-type data.
method MMM model reorganizes data into a three-way structure, using a mixture of matrix-variate normal distributions.
result MMM model handles various data types (continuous, ordinal, binary, nominal, count) and temporal dependence.
A new method generates mixed-type features in tabular data with improved realism and accuracy.
problem Generating mixed-type features combining discrete and continuous data is challenging.
method A cascaded approach: first generates low-resolution categorical and coarse numerical features, then uses these in a high-resolution flow matching model.
result The model significantly improves detection scores, generating more realistic samples and capturing distributional details.
Let the circle act on a compact almost complex manifold . In this paper, we classify the fixed point data of the action if there are 4 fixed points and the dimension of the manifold is at most 6. First, if , then is a disjoint union of rotations on two 2-spheres. Second, if , we prove that th…
New method for mixed data types in graphical models.
problem Challenges in analyzing data with mixed variable types.
method Latent Gaussian copula models with leveraged polychoric and polyserial correlations.
result Flexible and scalable methodology for mixed data types.
PH-VAE models heavy-tailed data with flexible Phase-Type distributions.
problem Standard VAEs fail to capture heavy-tailed behavior in real-world data.
method PH-VAE uses Phase-Type distributions defined by continuous-time Markov chains to adaptively model tail behavior.
result PH-VAE significantly outperforms existing heavy-tail-aware VAEs in approximating diverse heavy-tailed distributions.
As entity type systems become richer and more fine-grained, we expect the number of types assigned to a given entity to increase. However, most fine-grained typing work has focused on datasets that exhibit a low degree of type multiplicity. In this paper, we consider the high-multiplicity regime inherent in data source…
The paper tackles data misappropriation in LLMs by embedding watermarks and testing for their presence.
problem Detecting data misappropriation in LLMs trained on copyrighted data.
method Embedding watermarks, formulating as hypothesis testing, developing statistical framework, constructing test statistics, determining optimal thresholds, controlling errors, establishing asymptotic optimality.
result The proposed statistical testing framework effectively detects data misappropriation in LLMs.
CausalMix generates synthetic data with causal controls for mixed-type tables.
problem Synthetic data for causal inference with mixed-type and multimodal tabular data.
method CausalMix combines Gaussian latent priors with data-type-specific decoders for control over overlap, confounding, and treatment effect heterogeneity.
result CausalMix achieves state-of-the-art distributional metrics and stable causal control.
Study shows different behaviors of noncompact hypersurfaces under mean curvature flow.
problem Analyzing stability of noncompact hypersurfaces with curvature blowup.
method Numerical overlap method to construct global solutions.
result Existence of near and far classes of initial data leading to distinct behaviors.
Blang simplifies Bayesian analysis for non-standard data types.
problem Bayesian inference for non-standard data structures.
method Bayesian declarative language, distribution continua, sequential Monte Carlo, non-reversible MCMC.
result Bayesian analysis on arbitrary data types is feasible and efficient.
Real Lagrangians in toric manifolds are classified by combinatorial data.
problem Classifying real Lagrangian submanifolds in toric symplectic manifolds.
method Established a real analog of the Delzant construction.
result Real Lagrangians in toric del Pezzo surfaces have all possible diffeomorphism types.
Study solves inverse problems for real principal type operators using unique data sets and ray transforms.
problem Determining coefficients in real principal type equations from boundary data.
method Unique data sets, bicharacteristic ray transforms, and propagation of singularities.
result Global uniqueness results for determining coefficients in nonlinear real principal type equations.
A method uses autoencoders to align multi-modal neuron data.
problem Inconsistent cell type definitions across different data modalities.
method Coupled training of autoencoders for cross-modal alignment.
result Representations learned by coupled autoencoders can identify single-modality sampled cell types.
Clustering is an essential technique for discovering patterns in data. The steady increase in amount and complexity of data over the years led to improvements and development of new clustering algorithms. However, algorithms that can cluster data with mixed variable types (continuous and categorical) remain limited, de…
Enhanced fuzzy system predicts chaotic time series with improved accuracy.
problem Forecasting chaotic time series with high uncertainty.
method Combines evolving fuzzy systems, participatory learning, KRLS, and type-2 fuzzy sets.
result Proposed model outperforms other methods in accuracy and complexity.