Develops a new cluster validity index to find multiple optimal cluster numbers.
problem Finding the optimal number of clusters in real-world data with varying densities, sizes, and shapes.
method A new correlation-based cluster validity index that yields multiple local peaks.
result The new index finds multiple optimal cluster numbers in various scenarios.
The paper challenges the validity of cluster validity measures in unsupervised learning.
problem The validity of cluster validity measures in selecting optimal clusterings.
method The authors investigate the use of cluster validity measures as objective functions in unsupervised learning and introduce a new variant of the Dunn index.
result Many cluster validity measures promote clusterings that do not match expert knowledge well.
New validity index for fuzzy-possibilistic c-means clustering.
problem Conflicting results in determining the optimal number of clusters due to noisy data points and outliers.
method Introducing a new validity index (FP index) for fuzzy-possibilistic c-means clustering.
result FP index works well in datasets with varying cluster shapes and densities.
New cluster validity index detects optimal number of clusters and secondary options.
problem Determining the optimal number of clusters in fuzzy clustering.
method Correlation-based fuzzy cluster validity index (WP index) using fuzzy c-means algorithm.
result WP index outperforms existing indexes in detecting optimal number of clusters and secondary options.
Study benchmarks 26 clustering validity measures.
problem Determining the best clustering solution from candidates.
method Enhanced revision of previous methodology with three sub-methodologies.
result Comprehensive evaluation of 26 internal validity indexes.
The paper extends cluster validity indices for incremental analysis.
problem Providing incremental alternatives for cluster validation.
method Extending iCVI family to include 6 incremental indices and examining their behavior under under- and over-partitioning.
result Over-partitioning is more challenging to detect than under-partitioning.
Introduces BCVI, a Bayesian cluster validity index for better cluster selection.
problem Selecting the optimal number of clusters in clustering algorithms.
method Develops BCVI using Dirichlet or generalized Dirichlet priors, evaluated against existing indices.
result BCVI offers clear advantages in scenarios where user expertise is valuable.
CDL index improves clustering validation for non-convex data.
problem Selecting clustering algorithms and hyperparameters without labeled data.
method CDL uses compactness, centers, and covariances to compute a probabilistic description length bound.
result CDL outperforms conventional CVIs on synthetic and image benchmarks.
CAF-HFCM automatically forms a cluster hierarchy and optimizes the number of clusters without trial-and-validation.
problem Challenges in determining the optimal number of clusters in fuzzy c-means.
method CAF-HFCM, an auto-fused hierarchical fuzzy c-means method.
result Automatic agglomeration and optimal number of clusters without validity indices.
Cluster analysis is used to explore structure in unlabeled data sets in a wide range of applications. An important part of cluster analysis is validating the quality of computationally obtained clusters. A large number of different internal indices have been developed for validation in the offline setting. However, thi…
New clustering algorithm uses reverse nearest neighbour for better density-based clustering.
problem Density-based clustering of separated high-density regions.
method Uses reverse nearest neighbour (RNN) queries to estimate densities and recover clusters.
result Outperforms DBSCAN and ISDBSCAN on synthetic and real-world data.
A new CVI called DSI evaluates clustering results without true labels.
problem No universal CVI for clustering without true labels.
method DSI based on data separability measure.
result DSI is an effective, unique, and competitive CVI.
Cluster analysis is widely used in the areas of machine learning and data mining. Fuzzy clustering is a particular method that considers that a data point can belong to more than one cluster. Fuzzy clustering helps obtain flexible clusters, as needed in such applications as text categorization. The performance of a clu…
iCVI-ARTMAP accelerates clustering with adaptive resonance theory and validity indices.
problem Improving clustering efficiency and accuracy using adaptive resonance theory.
method Integrates adaptive resonance theory (ARTMAP) with incremental cluster validity indices (iCVIs) for clustering.
result Significantly reduces clustering time and outperforms other methods on synthetic and real-world data.
Clustering is an extensive research area in data science. The aim of clustering is to discover groups and to identify interesting patterns in datasets. Crisp (hard) clustering considers that each data point belongs to one and only one cluster. However, it is inadequate as some data points may belong to several clusters…
HD-BWDM improves clustering validation in high-dimensional data.
problem Determining the right number of clusters in high-dimensional data.
method HD-BWDM integrates random projection, PCA, trimmed clustering, and medoid-based distances.
result HD-BWDM remains stable and interpretable under high-dimensional projections and contamination.
This paper evaluates and validates cluster results using external and internal evaluation methods.
problem Evaluating and validating the quality of clustering results.
method External evaluation using Homogeneity, Correctness, and V-measure scores; internal evaluation using Silhouette Index and Sum of Square Errors.
result Validation of the number of clusters using dendrogram and statistical frequency distribution.
In this paper we introduce three methods for re-scaling data sets aiming at improving the likelihood of clustering validity indexes to return the true number of spherical Gaussian clusters with additional noise features. Our method obtains feature re-scaling factors taking into account the structure of a given data set…
The paper analyzes indices based on counting object pairs for assessing partition agreement in unsupervised learning.
problem The difficulty in interpreting overall indices like Rand and adjusted Rand indices.
method Analysis of three families of indices based on counting object pairs, decomposing overall indices into cluster-level indices.
result Overall indices based on pair-counting approach are sensitive to cluster size imbalance and provide limited information on smaller clusters.
A new measure identifies clusters without assuming data distribution.
problem Identifying the correct number of clusters in data without distribution assumptions.
method Nonparametric interpoint distance-based approach.
result Superior to existing clustering measures, validated on synthetic and real data.
A new measure normalizes clustering accuracy to evaluate algorithms better.
problem Evaluation of clustering algorithms is challenging due to limitations of existing measures.
method Proposes a new, normalised clustering accuracy measure.
result The new measure identifies worst-case scenarios and is more interpretable.
It has been noticed that some external CVIs exhibit a preferential bias towards a larger or smaller number of clusters which is monotonic (directly or inversely) in the number of clusters in candidate partitions. This type of bias is caused by the functional form of the CVI model. For example, the popular Rand index (R…
A new measure DCSI quantifies separability for density-based clustering.
problem Quantifying meaningful clusters in data sets.
method Developed a new separability measure DCSI based on separation and connectedness.
result Correctly identifies touching or overlapping classes that do not correspond to meaningful density-based clusters.
New clustering algorithms optimize Average Silhouette Width for better cluster quality.
problem Optimizing clustering using Average Silhouette Width (ASW).
method Proposed two algorithms: OSil and FOSil, compared with existing methods.
result New methods prove useful and sensible in many cases, but with some weaknesses.
A new clustering evaluation index based on density estimation.
problem Improving internal clustering evaluation indices.
method The index is a mixture of Ambiguous and Similarity sub-indices, calculated using density estimation.
result The new index significantly outperforms other internal clustering evaluation indices.
Tree Index evaluates cluster quality by creating decision trees from data.
problem Evaluating the quality of cluster results from various techniques.
method Tree Index creates a decision tree from clustered data, combining entropy and depth of leaves.
result Tree Index discriminates between sensible and non-sensible clusters on brain dataset.
The paper uses TDA to select stocks for a sparse portfolio, improving performance across market scenarios.
problem Sparse portfolio selection in financial markets.
method Topological data analysis (TDA) for clustering stock price movements.
result The TDA-based clustering strategy significantly enhances sparse portfolio performance.
This paper proposes a new clustering method based on Stochastic Dominance for asset allocation.
problem Traditional clustering methods fail to capture risk dominance relationships among assets.
method Integrates Stochastic Dominance theory with machine learning algorithms to construct a Stochastic Dominance Coefficient Matrix and modify clustering algorithms.
result The proposed method effectively facilitates customized asset allocation for investors.
FunCLBM clusters time series data for autonomous driving validation.
problem Validation of autonomous driving systems using large amounts of data.
method FunCLBM model for co-clustering high-dimensional time series data.
result FunCLBM provides structured partition and clustering views of datasets.
CCP clusters correlated features and projects them to 1D for efficient dimensionality reduction.
problem Efficiency in handling large datasets with high intrinsic dimensions.
method CCP partitions features into correlated clusters and projects them to 1D based on sample correlations.
result CCP achieves efficient dimensionality reduction without matrix diagonalization.
Formula found for minimum ARI between clusterings of fixed sizes.
problem Understanding the lowest possible agreement between clusterings.
method Explicit formula derivation for minimum ARI.
result A specific pair of clusterings achieving the minimum ARI is provided.
We investigate the trading behavior of Finnish individual investors trading the stocks selected to compute the OMXH25 index in 2003 by tracking the individual daily investment decisions. We verify that the set of investors is a highly heterogeneous system under many aspects. We introduce a correlation based method that…
A new clustering framework optimizes customer search data for personalized travel recommendations.
problem Personalized travel recommendations based on customer search data.
method Multi-objective optimization-based clustering ensemble framework.
result Optimizes diversity in clustering ensemble search space and automatically determines the number of clusters.
EGMM improves clustering by better handling uncertainty with evidential framework.
problem Clustering uncertainty and complexity in data.
method Proposes EGMM, a new model-based clustering algorithm using belief functions and EM algorithm.
result EGMM generates more informative evidential partitions and outperforms other algorithms.
This paper tackles deep clustering evaluation challenges in high-dimensional data.
problem Evaluation of deep clustering methods is problematic due to the curse of dimensionality and variations in embedding spaces.
method Develops a theoretical framework to highlight the ineffectiveness of internal validation measures and proposes a systematic approach to applying clustering validity indices in deep learning.
result The proposed framework reduces misguidance from improper use of clustering validity indices in deep learning.
New model clusters mixed-type data with missing values, improving air quality analysis.
problem Clustering mixed-type data with missing values and regime persistence.
method Statistical jump model incorporating regime persistence and handling missing data.
result Superior performance in inferring persistent air quality regimes compared to traditional methods.
A neural-network model clusters subjects based on their lifetime distributions.
problem Clustering subjects into clusters based on their lifetime distributions.
method A neural-network based lifetime clustering model that maximizes divergence between empirical lifetime distributions of clusters.
result Significantly better lifetime clusters compared to competing approaches.
In this paper, a frequency coefficient based on the Sen-Shorrocks-Thon (SST) poverty index notion is proposed. The clustering SST index can be used as the method for determination of the connection between similar neighbor sub-clusters. Consequently, connections can reveal existence of natural homogeneous. Through esti…
In this paper we make two novel contributions to hierarchical clustering. First, we introduce an anomalous pattern initialisation method for hierarchical clustering algorithms, called A-Ward, capable of substantially reducing the time they take to converge. This method generates an initial partition with a sufficiently…
The clustering ensemble technique aims to combine multiple clusterings into a probably better and more robust clustering and has been receiving an increasing attention in recent years. There are mainly two aspects of limitations in the existing clustering ensemble approaches. Firstly, many approaches lack the ability t…
Proposes new random models for fuzzy clustering similarity measures.
problem Challenges in choosing a random model for fuzzy clustering similarity measures.
method Introduces three intuitive and explainable random models for fuzzy clusterings.
result Each random model has distinct behavior, emphasizing the importance of accurate model selection.
reval package selects best clustering solutions via stability-based validation.
problem Challenges in determining best clustering solutions due to lack of validation methods.
method Stability-based relative clustering validation methods.
result Determines best clustering solutions that generalize to unseen data.
BFPM relaxes fuzzy and possibilistic clustering restrictions, allowing full memberships in multiple clusters.
problem Issues with previous fuzzy and possibilistic clustering methods, especially overlapping clusters.
method BFPM allows larger membership values and multiple full memberships per object.
result BFPM overcomes conventional methods' limitations and improves data object tracking.
Meta-learning neural networks for better clustering representations.
problem Improving clustering performance with appropriate representations.
method Meta-learning method that trains neural networks for representations using VB inference with an infinite Gaussian mixture model.
result The method achieves higher clustering performance than existing methods.
Adjusted for chance measures are widely used to compare partitions/clusterings of the same data set. In particular, the Adjusted Rand Index (ARI) based on pair-counting, and the Adjusted Mutual Information (AMI) based on Shannon information theory are very popular in the clustering community. Nonetheless it is an open …
This paper compares two clustering evaluation metrics, revealing their differences and properties.
problem Understanding the differences between misclassification error distance and adjusted Rand index.
method Population origins, data analysis examples, detailed case studies, and simulation study.
result Reveals previous misconceptions about the two metrics and their distributions.
Paper introduces NICc for fast cluster-based validation of prediction models.
problem Validation of prediction models on clustered data.
method Derived NICc to approximate leave-one-cluster-out deviance for standard regression models.
result NICc provides more accurate model size and variable selection, especially with strong clustering.
CARVE validates clustering results using resampling and stability analysis.
problem Inconsistent and unreliable clustering results due to algorithm, preprocessing, and k sensitivity. method CARVE uses resampling-based validation and stability analysis to evaluate multiple clustering algorithms and hyperparameters.
result CARVE consistently recovers near-optimal clusterings and finer biological structure.