Research
On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,695 papers · 148 categories

Trend · papers per month

181361542722 · Jun 202019922001200920172026
48 results for dataset similarity

CLS measures dataset similarity through decision rule performance.

problem Measuring dataset similarity in machine learning, especially for transfer learning and domain adaptation.
method Cross-Learning Score (CLS) measures similarity through bidirectional generalization performance of decision rules, linking to cosine similarity under canonical linear models.
result CLS effectively measures dataset similarity and transferability, validated on synthetic and real-world datasets.

The paper studies batch decompositions of random datasets with probabilistic similarity constraints.

problem Understanding how to optimally split large datasets into batches for better model learning.
method Assumes independent data points from a space, defines similarity, and uses probabilistic and martingale methods to find bounds on batch sizes.
result Demonstrates an inherent tradeoff between relaxing similarity constraints and batch size, and provides bounds for maximum similarity subsets.

Topological parallax assesses AI models' geometric similarity to datasets for safety.

problem Ensuring AI models' robustness and safety in deep learning applications.
method Topological parallax compares a trained model to a reference dataset using Rips complexes and geodesic distortions.
result Topological parallax indicates whether a model shares similar multiscale geometric features with the dataset.

The performance of most the clustering methods hinges on the used pairwise affinity, which is usually denoted by a similarity matrix. However, the pairwise similarity is notoriously known for its vulnerability of noise contamination or the imbalance in samples or features, and thus hinders accurate clustering. To tackl…

2019-05-10abs ↗pdf ↗

Proposes a method to compare noisy high-dimensional datasets with low-dimensional manifolds.

problem Comparing distributions on manifolds in noisy high-dimensional datasets.
method Linking low-rank structure to manifold geometry, developing a scale-invariant distance measure.
result Superior robustness and statistical power compared to existing methods.

A new method selects the best feature selection technique for datasets.

problem Selecting the best feature selection method for unseen datasets.
method Data synthesis, meta features, fuzzy similarity, classification model training.
result Successfully recommended the best feature selection method for five out of eight datasets.

Proposes a method to compare neural networks using feature and gradient vectors.

problem Understanding the behavior of neural networks trained on different datasets.
method Defines a similarity index using feature and gradient vectors, and employs sketching techniques for efficient comparison.
result Demonstrates superior performance in computing similarity of neural networks trained on different datasets.

We introduce GSimCNN (Graph Similarity Computation via Convolutional Neural Networks) for predicting the similarity score between two graphs. As the core operation of graph similarity search, pairwise graph similarity computation is a challenging problem due to the NP-hard nature of computing many graph distance/simila…

2018-10-23abs ↗pdf ↗

Paper develops multivariate time series similarity and distance measures.

problem Compensating for misalignments in multivariate time series data.
method Adapted Independent and Dependent DTW strategies to seven elastic similarity and distance measures.
result Each measure achieves highest accuracy on at least one dataset, supporting their value.

Study reveals how data density affects similarity learning in noisy conditions.

problem Impact of noise on similarity learning in Siamese Neural Networks.
method Investigated Pair Label Noise and Single Label Noise on Siamese Neural Networks (SNNs).
result Data density is crucial for generalization in SNNs, leading to a phenomenon called DIBS.

Quantum-assisted VAE improves similarity search in high-dimensional datasets.

problem Finding fast and memory-efficient similarity search in high-dimensional data.
method Construct a space-efficient search index based on the latent space of a Quantum-assisted Variational Autoencoder (QVAE).
result Real-world speedups and memory-efficient scaling to half a billion data points.

Calculating similarities between objects defined by many heterogeneous data modalities is an important challenge in many multimedia applications. We use a multi-modal topic model as a basis for defining such a similarity between objects. We propose to compare the resulting similarities from different model realizations…

2014-05-27abs ↗pdf ↗

Hyperparameter optimization aims to find the optimal hyperparameter configuration of a machine learning model, which provides the best performance on a validation dataset. Manual search usually leads to get stuck in a local hyperparameter configuration, and heavily depends on human intuition and experience. A simple al…

2017-10-17abs ↗pdf ↗

We propose a novel learning framework to answer questions such as "if a user is purchasing a shirt, what other items will (s)he need with the shirt?" Our framework learns distributed representations for items from available textual data, with the learned representations representing items in a latent space expressing f…

2019-08-26abs ↗pdf ↗

This paper tackles noise in raw datasets to improve representation learning efficiency.

problem Noise in real-world datasets degrades representation learning quality.
method Proposes denoising Cosine-Similarity (dCS) loss to learn robust representations.
result Empirical results show the dCS loss outperforms baseline objective functions.

Improves document summarization by combining word embeddings and n-grams.

problem Exact word matching fails to measure semantic similarity between sentences.
method Uses deep embedding features and tf-idf features to improve sentence similarity measure; builds an improved sentence similarity graph; employs a submodular objective function; develops a Transformer-based compression model.
result Outperforms tf-idf based approach and achieves state-of-the-art performance on DUC04 dataset.

Paper proposes a supervised similarity framework for corporate bonds using RF proximities.

problem Challenges in measuring similarity for corporate bonds due to noisy data and lack of ground truth.
method Proposes a supervised similarity framework using Random Forest for corporate bonds, introducing a novel metric to evaluate similarities.
result Random Forest outperforms other methods in evaluating similarities for corporate bonds.

Method embeds numeric tabular datasets into a shared vector space for similarity and retrieval.

problem Lack of meaningful representation for numeric tabular datasets in large language models.
method Structured exploratory data analysis descriptors, sentence transformer embedding, CCA for cross-dataset alignment.
result Total P@1 score of 0.9 across 15 datasets, robust nearest-neighbor retrieval and cluster structure.

Defining similarity measures is a requirement for some machine learning methods. One such method is case-based reasoning (CBR) where the similarity measure is used to retrieve the stored case or set of cases most similar to the query case. Describing a similarity measure analytically is challenging, even for domain exp…

2020-01-15abs ↗pdf ↗

Adaptive methods learn from multiple datasets, leveraging similarities and robust to outliers.

problem Simultaneously analyze multiple datasets with possible similarities and differences.
method Adaptive multi-task learning methods that automatically utilize similarities and handle differences.
result Sharp statistical guarantees and robustness against outlier tasks demonstrated.

CLARITY compares dissimilar datasets, identifying structural and relationship inconsistencies.

problem Integrating qualitatively different datasets from various disciplines.
method Non-parametric approach decomposing similarities into structural and relationship components.
result Identifies and interprets inconsistencies between datasets.

Exploits class similarity for better machine learning models with confidence labels and projective loss functions.

problem Poor model performance due to confusing similar classes.
method Exploits class similarity with confidence labels and projective loss functions.
result Improved model performance on noisy labels.

We propose weighted inner product similarity\textit{weighted inner product similarity} (WIPS) for neural network-based graph embedding. In addition to the parameters of neural networks, we optimize the weights of the inner product by allowing positive and negative values. Despite its simplicity, WIPS can approximate arbitrary general similarities in…

2019-02-27abs ↗pdf ↗

Recommender systems have recently attracted many researchers in the deep learning community. The state-of-the-art deep neural network models used in recommender systems are typically multilayer perceptron and deep Autoencoder (DAE), among which DAE usually shows better performance due to its superior capability to reco…

2018-12-25abs ↗pdf ↗

Measuring the similarity of two files is an important task in malware analysis, with fuzzy hash functions being a popular approach. Traditional fuzzy hash functions are data agnostic: they do not learn from a particular dataset how to determine similarity; their behavior is fixed across all datasets. In this paper, we …

2018-12-17abs ↗pdf ↗

Bayesian methods detect significant IIA violations in similarity choice data.

problem Detecting IIA violations in similarity choice data complicates classical models.
method Proposed two statistical methods: classical goodness-of-fit test and Bayesian PPC.
result Significant IIA violations confirmed in both datasets, driven by context effects.

Study shows partially-typed NER datasets can match fully-typed ones in model performance.

problem Leveraging multiple partially-typed NER datasets for training models without fully-typed annotations.
method Systematic analysis and controlled experiments comparing partially-typed and fully-typed datasets.
result Models trained with partially-typed annotations can achieve similar performance to those trained with fully-typed annotations.