Hybrid clustering combines partitional and hierarchical clustering for computational effectiveness and versatility in cluster shape. In such clustering, a dissimilarity measure plays a crucial role in the hierarchical merging. The dissimilarity measure has great impact on the final clustering, and data-independent prop…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “density-based dissimilarity”
New research shows semantic data matching can degrade SSDL performance.
problem The limits of semantic data set matching in semi-supervised learning.
method Demonstrated through simulations and a new dissimilarity measure.
result Semantic data matching can degrade SSDL performance under non-IID data.
Unified framework for various probability distribution distances.
problem Handling diverse probability distribution distances in statistics.
method General framework covering density-based and distribution-function-based divergences.
result Unified approach to classical and modern statistical procedures.
We develop a new density-based clustering algorithm named CRAD which is based on a new neighbor searching function with a robust data depth as the dissimilarity measure. Our experiments prove that the new CRAD is highly competitive at detecting clusters with varying densities, compared with the existing algorithms such…
Skeleton clustering detects clusters in high-dimensional data without needing prototypes.
problem Detecting clusters in high-dimensional data with irregular shapes.
method Skeleton clustering combines prototype methods, density-based clustering, and hierarchical clustering using surrogate density measures.
result Skeleton clustering reliably detects clusters in multivariate and high-dimensional data.
A new measure DCSI quantifies separability for density-based clustering.
problem Quantifying meaningful clusters in data sets.
method Developed a new separability measure DCSI based on separation and connectedness.
result Correctly identifies touching or overlapping classes that do not correspond to meaningful density-based clusters.
Density-based clustering is the task of discovering high-density regions of entities (clusters) that are separated from each other by contiguous regions of low-density. DBSCAN is, arguably, the most popular density-based clustering algorithm. However, its cluster recovery capabilities depend on the combination of the t…
New framework quantifies uncertainty in flexible density-based clustering.
problem Uncertainty quantification in clustering with non-parametric density estimation.
method Martingale posterior distributions and density-based clustering.
result Efficient GPU-compatible inference on clustering structures with uncertainty.
SDCOR clusters massive datasets efficiently, detecting outliers with low memory usage.
problem Local outlier detection in large-scale datasets.
method Chunk-based density clustering with incremental updates.
result SDCOR achieves lower linear time complexity and better efficiency than traditional methods.
Clustering is an essential technique for discovering patterns in data. The steady increase in amount and complexity of data over the years led to improvements and development of new clustering algorithms. However, algorithms that can cluster data with mixed variable types (continuous and categorical) remain limited, de…
Stable density-based clustering via multiparameter persistence.
problem Density-based clustering stability to data perturbations.
method Degree-Rips construction, correspondence-interleaving distance, multiparameter stability analysis.
result Persistable pipeline yields stable, consistent density-based clustering.
Proposes DISCO, the first CVI for density-based clustering with noise.
problem Evaluation of noise assignments in density-based clustering.
method Density-connectivity and Silhouette Coefficient adaptation for noise evaluation.
result DISCO is the first CVI to explicitly assess noise assignments.
We improve density-based distances using normalizing flows and score matching.
problem Inaccurate density estimates and poor convergence in graph-based methods for high-dimensional spaces.
method Learn densities with normalizing flows and refine geodesics with a score model.
result Improved density-based distances that scale to high dimensions and improve numerical stability.
Density-based clustering techniques are used in a wide range of data mining applications. One of their most attractive features con- sists in not making use of prior knowledge of the number of clusters that a dataset contains along with their shape. In this paper we propose a new algorithm named Linear DBSCAN (Lin-DBSC…
FISHDBC is a flexible, incremental, scalable, and hierarchical density-based clustering algorithm. It is flexible because it empowers users to work on arbitrary data, skipping the feature extraction step that usually transforms raw data in numeric arrays letting users define an arbitrary distance function instead. It i…
This paper presents a simple but effective density-based outlier detection approach with the local kernel density estimation (KDE). A Relative Density-based Outlier Score (RDOS) is introduced to measure the local outlierness of objects, in which the density distribution at the location of an object is estimated with a …
We present an accelerated algorithm for hierarchical density based clustering. Our new algorithm improves upon HDBSCAN*, which itself provided a significant qualitative improvement over the popular DBSCAN algorithm. The accelerated HDBSCAN* algorithm provides comparable performance to DBSCAN, while supporting variable …
DMAE uses neural networks to cluster data with flexible dissimilarity functions.
problem Clustering data with complex dissimilarity functions.
method Integrates a dissimilarity mixture model into deep learning architectures.
result DMAE achieves competitive clustering accuracy compared to other methods.
Density-based spatial clustering of applications with noise (DBSCAN) is a data clustering algorithm which has the high-performance rate for dataset where clusters have the constant density of data points. One of the significant attributes of this algorithm is noise cancellation. However, DBSCAN demonstrates reduced per…
Random Forest proximity measures for multi-view classification.
problem Combining multiple heterogeneous data views for classification.
method Building dissimilarity representations for each view, fusing them dynamically.
result Dynamic View Selection improves multi-view classification performance.
Recently, deep clustering, which is able to perform feature learning that favors clustering tasks via deep neural networks, has achieved remarkable performance in image clustering applications. However, the existing deep clustering algorithms generally need the number of clusters in advance, which is usually unknown in…
New RDPC dissimilarity measure improves time series clustering.
problem Improving time series clustering methods for diverse data.
method Combining weighted Pearson correlation with largest element-wise differences.
result RDPC outperforms existing methods in complex datasets.
Expands causal clustering framework with hierarchical and density-based methods.
problem Identifying heterogeneous treatment effects in unknown subgroup structure.
method Integrates hierarchical and density-based clustering algorithms into causal k-means clustering.
result Plug-in estimators for causal clustering are simple and readily implementable.
DPSM clusters nodes in data and graph spaces via density propagation and subcluster merging.
problem Automatic clustering of nodes in data and graph spaces.
method Density-based node clustering with propagation process and spectral clustering on subclusters.
result DPSM effectively clusters nodes in both data and graph spaces.
Efficient clustering in high dimensions with Quick Shift and LSH.
problem Density-based clustering in high-dimensional data.
method Combines Quick Shift and LSH for efficient density estimation.
result Achieves almost linear time complexity for consistency.
A novel density-based approach QC detects outliers in data with high precision.
problem Detecting outliers in data with high precision and sensitivity.
method Quantum Clustering (QC) approach based on the density of data points.
result QC effectively finds hidden outliers and subtle outliers with parameter adjustment.
Parameter-free clustering method using cluster catch digraphs (CCDs).
problem Finding the correct number of clusters in data without specifying a parameter.
method Hybrid of density-based and graph-based clustering methods using Ripley's K function.
result Minimum dominating sets of RK-CCDs estimate and distinguish clusters from noise.
Pairwise similarities and dissimilarities between data points might be easier to obtain than fully labeled data in real-world classification problems, e.g., in privacy-aware situations. To handle such pairwise information, an empirical risk minimization approach has been proposed, giving an unbiased estimator of the cl…
Most density-based clustering methods largely rely on how well the underlying density is estimated. However, density estimation itself is also a challenging problem, especially the determination of the kernel bandwidth. A large bandwidth could lead to the over-smoothed density estimation in which the number of density …
This paper considers networks where relationships between nodes are represented by directed dissimilarities. The goal is to study methods that, based on the dissimilarity structure, output hierarchical clusters, i.e., a family of nested partitions indexed by a connectivity parameter. Our construction of hierarchical cl…
A recent proposal of data dependent similarity called Isolation Kernel/Similarity has enabled SVM to produce better classification accuracy. We identify shortcomings of using a tree method to implement Isolation Similarity; and propose a nearest neighbour method instead. We formally prove the characteristic of Isolatio…
DID measures similarity invariant to diffeomorphisms.
problem Measuring similarity invariance to diffeomorphisms.
method DID measures similarity as the solution to an optimization problem in a Reproducing Kernel Hilbert Space.
result DID is invariant to diffeomorphisms and can be efficiently approximated.
BOKE optimizes expensive functions with reduced computational costs.
problem High computational cost of Gaussian process-based Bayesian optimization.
method Kernel regression and density-based exploration integrated into confidence bounds.
result BOKE achieves global convergence and superior computational efficiency.
Improves Gower's similarity for mixed-type variables with automatic weighting.
problem Handling missing values and unbalanced variable contributions in Gower's similarity for mixed-type data.
method Automatic weighting scheme minimizing differences in correlation between contributing dissimilarities and weighted Gower's dissimilarity.
result Improved performance in classification and imputation of missing values.
Paper proposes a new efficient transport-based dissimilarity measure for time series classification.
problem Classifying time series with warping distortions.
method Defining a problem statement, proposing an Optimal Transport-based dissimilarity measure.
result The proposed method can solve the time series classification problem with reduced computational cost.
FPI methods compute barycenters of Gaussian sets for various dissimilarity measures.
problem Efficiently compute barycenters of Gaussian sets for multiple dissimilarity measures.
method Fixed-Point Iterations (FPI) for several dissimilarity measures.
result FPI provides a useful toolbox for fusion/reduction of Gaussian sets.
We present a simple, yet effective, approach to Semi-Supervised Learning. Our approach is based on estimating density-based distances (DBD) using a shortest path calculation on a graph. These Graph-DBD estimates can then be used in any distance-based supervised learning method, such as Nearest Neighbor methods and SVMs…
Paper uses DBSCAN variation to detect ship anomalies.
problem Detecting anomalous ship behavior.
method Variation of DBSCAN algorithm applied to AIS data.
result Alternative anomaly metric is more statistically informative.
Locality sensitive hashing (LSH) is a powerful tool for sublinear-time approximate nearest neighbor search, and a variety of hashing schemes have been proposed for different dissimilarity measures. However, hash codes significantly depend on the dissimilarity, which prohibits users from adjusting the dissimilarity at q…
Classification in the dissimilarity space has become a very active research area since it provides a possibility to learn from data given in the form of pairwise non-metric dissimilarities, which otherwise would be difficult to cope with. The selection of prototypes is a key step for the further creation of the space. …
New method clusters large datasets using geometric properties.
problem Clustering large datasets using DBSCAN* and HDBSCAN* is infeasible.
method Exploiting Euclidean space geometry, systematically construct clusters from subsets of data.
result Clusters of large datasets are possible with controlled subset sizes.
The one-class classification problem is a well-known research endeavor in pattern recognition. The problem is also known under different names, such as outlier and novelty/anomaly detection. The core of the problem consists in modeling and recognizing patterns belonging only to a so-called target class. All other patte…
We propose a novel method to determine the dissimilarity between subjects for functional data clustering. Spline smoothing or interpolation is common to deal with data of such type. Instead of estimating the best-representing curve for each subject as fixed during clustering, we measure the dissimilarity between subjec…
When it comes to clustering nonconvex shapes, two paradigms are used to find the most suitable clustering: minimum cut and maximum density. The most popular algorithms incorporating these paradigms are Spectral Clustering and DBSCAN. Both paradigms have their pros and cons. While minimum cut clusterings are sensitive t…
Optimizes basis for density-based atomic representations to enhance compactness and accuracy.
problem Improving the efficiency and accuracy of machine learning models for atomic properties.
method An unsupervised approach to determine the optimal basis set for atom density representations using splines.
result Optimal basis sets that encode structural information more compactly and accurately.
We introduce a new unsupervised learning problem: clustering wide-sense stationary ergodic stochastic processes. A covariance-based dissimilarity measure together with asymptotically consistent algorithms is designed for clustering offline and online datasets, respectively. We also suggest a formal criterion on the eff…
BERT improved for propaganda detection with imbalanced, dissimilar data.
problem BERT struggles with dissimilar imbalanced datasets in propaganda detection.
method Cost-sensitive BERT with dissimilarity measure for imbalanced, dissimilar datasets.
result Achieved second-highest score on sentence-level propaganda classification.
In this paper, we propose a Ward-like hierarchical clustering algorithm including spatial/geographical constraints. Two dissimilarity matrices and are inputted, along with a mixing parameter . The dissimilarities can be non-Euclidean and the weights of the observations can be non-uniform. The fi…