Paper explains distance-based classifiers using neural network structures.
problem Making distance-based classifiers explainable.
method Uncovering latent neural network structures in distance-based classifiers.
result Novel explanation approach outperforms baselines.
We consider classifiers for high-dimensional data under the strongly spiked eigenvalue (SSE) model. We first show that high-dimensional data often have the SSE model. We consider a distance-based classifier using eigenstructures for the SSE model. We apply the noise reduction methodology to estimation of the eigenvalue…
Time series classification is an increasing research topic due to the vast amount of time series data that are being created over a wide variety of fields. The particularity of the data makes it a challenging task and different approaches have been taken, including the distance based approach. 1-NN has been a widely us…
The reliable measurement of confidence in classifiers' predictions is very important for many applications and is, therefore, an important part of classifier design. Yet, although deep learning has received tremendous attention in recent years, not much progress has been made in quantifying the prediction confidence of…
Improves classification of microbiome data using mixture distributions.
problem Challenges in classifying sparse and heterogeneous microbiome count data.
method Distance-based classification using mixture distributions.
result The method outperforms existing distance-based classifiers and machine learning approaches.
DBLE improves confidence calibration of DNNs by learning distances in representation space.
problem Poor confidence calibration of deep neural networks (DNNs).
method DBLE trains a confidence model jointly with the classification model, using distances in the representation space.
result DBLE outperforms alternative single-model confidence calibration approaches and ensemble methods.
The study sets limits on how robust classifiers can be against adversarial attacks.
problem Understanding the limits of robustness in classification models against adversarial attacks.
method Utilized optimal transport theory to derive variational formulae and explicit lower-bounds on Bayes-optimal error.
result Explicit lower-bounds on the Bayes-optimal error for distance-based attacks, universal in geometry of class-conditional distributions.
A statistical model predicts generalization in few-shot learning.
problem Lack of validation sets in few-shot learning makes generalization estimation difficult.
method Introduced a Gaussian model of feature distribution and an unbiased estimator for class-conditional density distances.
result Our approach outperforms alternatives like leave-one-out cross-validation.
Mahalanobis distance detects anomalies well, but not for classification.
problem Detecting anomalies in neural classifier outputs.
method Analyzes Mahalanobis distance-based anomaly detection method, revealing its reliance on information not useful for classification.
result Combining Mahalanobis and ODIN methods improves anomaly detection performance and robustness.
Extends RF proximities to all supervised distance-based machine learning contexts.
problem Limited utility of RF proximities in various machine learning tasks.
method Introduces generalized Proximity Forest (PF) model and variant for regression.
result Demonstrates unique advantages over RF and k-nearest neighbors models.
ClusTR improves clustering-based models' robustness without adversarial training.
problem Improving clustering-based models' robustness.
method Proposes ClusTR, a clustering-based training framework for robust models without adversarial training.
result ClusTR outperforms adversarially-trained models by up to 4% under strong PGD attacks.
Despite numerous attempts to defend deep learning based image classifiers, they remain susceptible to the adversarial attacks. This paper proposes a technique to identify susceptible classes, those classes that are more easily subverted. To identify the susceptible classes we use distance-based measures and apply them …
Transforms distance-based outlier scores into interpretable probabilistic estimates.
problem Difficult interpretation of distance-based outlier scores.
method Generic transformation of scores into probabilistic estimates using distance probability distributions.
result Probabilistic transformation improves interpretability without impacting detection performance.
Improves uncertainty estimation and OOD detection in neural networks.
problem Accurate uncertainty estimation and OOD detection in neural networks.
method Investigates one-vs-all and distance-based logit representations for probabilities.
result One-vs-all formulations improve calibration without additional complexity.
Random forest can be adapted for open-set recognition with improved performance.
problem Handling unknown classes in real-world classification tasks.
method Incorporating distance metric learning and distance-based open-set recognition into random forest.
result The proposed method outperforms state-of-the-art open-set recognition methods.
A scalable version of MADD improves big-data classification speed.
problem High computational complexity of MADD in big data.
method Selecting a representative set and using Random Fourier Features.
result Achieves similar performance to MADD but at a fraction of the computing time.
Efficient method classifies locally stationary time series based on second-order characteristics.
problem Classifying locally stationary time series for various applications.
method Autoregressive approximation, ensemble aggregation, distance-based threshold.
result Zero misclassification error rate asymptotically for mildly differing second-order characteristics.
Neural networks learn distance-based representations, not just intensity.
problem Understanding how neural networks interpret and learn from internal activations.
method Manipulated ReLU and Absolute Value activations to observe sensitivity to distance and intensity perturbations.
result Neural networks are highly sensitive to small distance-based perturbations, challenging the intensity-based interpretation.
DSI measures dataset separability for neural networks.
problem Difficulty in separating different classes of data in neural networks.
method Created the Distance-based Separability Index (DSI) to quantify dataset separability.
result DSI effectively measures dataset separability and indicates similar distributions of different classes.
PAC-Bayesian bounds improve understanding of K-NN classifier performance.
problem Improving the understanding of K-NN classifier's generalization error.
method PAC-Bayesian analysis applied to K-NN classifier in kernel space.
result PAC-Bayesian bounds provide a function of the number of redundant training examples.
The paper improves uncertainty quantification for node classification using distance-based regularization.
problem Uncertainty in deep learning models, especially for node classification tasks.
method Graph posterior networks (GPNs) with UCE loss function, followed by a distance-based regularization.
result The proposed distance-based regularization outperforms state-of-the-art methods in OOD detection and misclassification detection.
Solves clustering contradictions by high-dimensional embedding with wide gaps.
problem Kleinberg's clustering axioms are contradictory.
method Embedding in high-dimensional space with wide gaps between clusters.
result Handles clustering contradictions by design.
BRDAD uses bagging and regularization to improve anomaly detection without labeled data.
problem Anomaly detection in unlabeled data with sensitivity to k-nearest neighbors. method Bagged regularized k-distances (BRDAD) for anomaly detection, converting to convex optimization. result BRDAD addresses sensitivity to hyperparameter choice and improves performance on large datasets.
Flexible classifier using Mahalanobis distances for non-elliptical distributions.
problem Classifying non-elliptical and multimodal distributions.
method Semiparametric classifier based on Mahalanobis distances and generalized additive models.
result The proposed classifiers outperform traditional methods in high-dimensional, low-sample-size scenarios.
The k-nearest neighbour (k-NN) classifier is one of the oldest and most important supervised learning algorithms for classifying datasets. Traditionally the Euclidean norm is used as the distance for the k-NN classifier. In this thesis we investigate the use of alternative distances for the k-NN classifier. We …
Unified approach to multiclass classification using Gabriel graphs.
problem Improving multiclass classification accuracy and efficiency.
method Integrates Gabriel graphs for binary and multiclass classification, proposing new activation functions and support edge neurons.
result Experimental results show superior performance compared to previous GG-based classifiers.
A new measure identifies clusters without assuming data distribution.
problem Identifying the correct number of clusters in data without distribution assumptions.
method Nonparametric interpoint distance-based approach.
result Superior to existing clustering measures, validated on synthetic and real data.
A method for ranking items using distance-based learning from positive and unlabeled data.
problem Learning to rank items without an analytic description of what constitutes a good ranking.
method Combining representations using an integer linear program for ranking items based on nominations.
result The method is effective in simulation and real data examples, especially when supervision is light.
Anomalies (unusual patterns) in time-series data give essential, and often actionable information in critical situations. Examples can be found in such fields as healthcare, intrusion detection, finance, security and flight safety. In this paper we propose new conformalized density- and distance-based anomaly detection…
Due to the popularity of the Internet and smart mobile devices, more and more financial transactions and activities have been digitalized. Compared to traditional financial fraud detection strategies using credit-related features, customers are generating a large amount of unstructured behavioral data every second. In …
For time series comparisons, it has often been observed that z-score normalized Euclidean distances far outperform the unnormalized variant. In this paper we show that a z-score normalized, squared Euclidean Distance is, in fact, equal to a distance based on Pearson Correlation. This has profound impact on many distanc…
Neural networks can learn distance metrics affecting model performance.
problem Understanding how neural networks learn and represent data.
method Experiments with six MNIST architectures, constrained to learn either distance or intensity representations.
result Distance-based learning affects model performance, validating the geometric framework.
GICDM corrects hubness in embedding spaces for better generative model evaluation.
problem Hubness phenomenon distorts distances in high-dimensional embedding spaces.
method Generative ICDM (GICDM) using multi-scale extension to correct neighborhood estimation.
result GICDM resolves hubness-induced failures and improves metric behavior.
Optimal posterior distributions improve SVM classifiers and parameter selection.
problem Improving SVM classifiers and selecting optimal regularization parameters.
method PAC-Bayesian approach with optimal posterior identification for stochastic classifiers.
result Optimal posteriors yield tight risk bounds and improved SVM performance.
A new Wasserstein K-means method for clustering probability distributions.
problem Clustering probability distributions using the Wasserstein metric.
method Distance-based K-means with SDP relaxation for Wasserstein barycenters. result Distance-based K-means outperforms centroid-based K-means for clustering probability distributions. In general, the clustering problem is NP-hard, and global optimality cannot be established for non-trivial instances. For high-dimensional data, distance-based methods for clustering or classification face an additional difficulty, the unreliability of distances in very high-dimensional spaces. We propose a distance-ba…
Learning expressive low-dimensional representations of ultrahigh-dimensional data, e.g., data with thousands/millions of features, has been a major way to enable learning methods to address the curse of dimensionality. However, existing unsupervised representation learning methods mainly focus on preserving the data re…
In light of the power problems of statistical tests and undisciplined use of alpha-based statistics to compare models, this paper proposes a unified set of distance-based performance metrics, derived as the square root of the sum of squared alphas and squared standard errors. The Bayesian investor views model performan…
New metric learning approach for tree data reduces computation cost.
problem Efficiently computing distances between ordered labeled trees.
method Introduced pq-grams and a differentiable weighted pq-gram distance, combined with LMNN for optimization.
result Significantly reduces computation time for tree classification problems.
Clustering is an essential data mining tool that aims to discover inherent cluster structure in data. As such, the study of clusterability, which evaluates whether data possesses such structure, is an integral part of cluster analysis. Yet, despite their central role in the theory and application of clustering, current…
Discriminative neural networks address class imbalance in coronary heart disease risk analysis.
problem Class imbalance in medical test data, especially in binary classification problems.
method Use of discriminative neural networks and contrastive loss with a Siamese network structure.
result The method effectively handles class imbalance, improving predictive models for coronary heart disease risk.
A new robust time series distance metric for k-NN classification.
problem Robustness against arbitrary data contamination in time series classification.
method Proposes a novel distance metric with worst-case O(nlogn) complexity. result Demonstrates competitive classification accuracy in k-NN time series classification.
Distance plays a fundamental role in measuring similarity between objects. Various visualization techniques and learning tasks in statistics and machine learning such as shape matching, classification, dimension reduction and clustering often rely on some distance or similarity measure. It is of tremendous importance t…
Distance-based tests, also called "energy statistics", are leading methods for two-sample and independence tests from the statistics community. Kernel-based tests, developed from "kernel mean embeddings", are leading methods for two-sample and independence tests from the machine learning community. A fixed-point transf…
New GP kernel handles mixed-categorical data, improving model accuracy.
problem Improving Gaussian process models for mixed-categorical data.
method Extends continuous exponential kernels to handle mixed-categorical variables.
result The proposed GP model gives higher likelihood and smaller residual error.
Study evaluates initialization strategies for infinite hidden Markov models.
problem Limited attention to initialization in infinite hidden Markov models.
method Systematically evaluated distance-based clustering, model-based, and uniform initializations.
result Distance-based clustering initializations consistently outperform other methods.
Outliers are ubiquitous in modern data sets. Distance-based techniques are a popular non-parametric approach to outlier detection as they require no prior assumptions on the data generating distribution and are simple to implement. Scaling these techniques to massive data sets without sacrificing accuracy is a challeng…
This paper explores ratio-based loss functions for machine learning.
problem Margin-based and distance-based loss functions for classification and regression.
method Investigation of ratio-based loss functions' properties.
result Proposed new ratio-based loss functions for regression.