New unsupervised feature selection method for imbalanced datasets.
problem Feature selection challenges in imbalanced multi-class datasets.
method Distance Rank Score using Spearman's Rank Correlation.
result Outperforms existing methods on clustering problems.
OMD monitors stock market dynamics through matrix trajectories and reveals crisis patterns.
problem Understanding and predicting stock market crises and sector rotations.
method Applying OMD to S\&P 500 returns over three crises, analyzing distance matrices and their spectra.
result Market dynamics show coherent changes during crises, with distinct sector leadership.
A new EnKF method for elliptic PDEs reduces dimensionality for accurate state estimation.
problem Elliptic PDEs in fluid flows make traditional EnKF regularization ineffective.
method Low-rank factorization of the Kalman gain based on the Jacobian spectrum.
result Inference can be performed in a low-dimensional subspace of the state space.
New RDPC dissimilarity measure improves time series clustering.
problem Improving time series clustering methods for diverse data.
method Combining weighted Pearson correlation with largest element-wise differences.
result RDPC outperforms existing methods in complex datasets.
The paper introduces tests for high-dimensional independence using maximum and average distance correlations.
problem Testing independence in high-dimensional data.
method Characterizes consistency properties, compares test statistics, examines null distributions, and presents a fast chi-square-based procedure.
result The proposed tests are non-parametric and applicable to various metrics.
OMD monitors stock market dynamics through matrix trajectories, revealing crisis patterns and sector rotations.
problem Understanding and predicting stock market dynamics during crises.
method Applying OMD to S&P 500 returns over three crises, analyzing distance matrices and their spectra.
result Market dynamics show coherent changes during crises, with sector-specific patterns and volatility clustering.
The study uses DCC for financial market analysis, revealing hidden correlations.
problem Identifying hidden nonlinear correlations in financial markets.
method Agglomerative hierarchical clustering with distance correlation coefficient.
result DCC reveals more information than Pearson correlation for financial data.
DeepFS uses deep neural networks to select significant features in ultra high-dimensional data.
problem Challenges in traditional feature selection methods for high-dimensional, low-sample-size data.
method Two-step nonparametric approach combining deep neural networks and feature screening.
result DeepFS effectively identifies significant features with high precision for ultra high-dimensional data.
New Hermite series estimator for Spearman rank correlation in non-stationary data.
problem Estimating time-varying Spearman rank correlation efficiently.
method Hermite series based sequential estimator for both stationary and non-stationary settings.
result Competitive performance compared to existing algorithms in simulations and real data.
High-dimensional, large-sample astrophysical databases of galaxy clusters, such as the Chandra Deep Field South COMBO-17 database, provide measurements on many variables for thousands of galaxies and a range of redshifts. Current understanding of galaxy formation and evolution rests sensitively on relationships between…
We generalize Mallows model to learn distance metrics from data.
problem Learning optimal distance metrics from noisy ranking data.
method Propose Lα distances and develop FPTAS for sampling and MLE. result Strong consistency of estimators for various α and β. Multi-label classification is a type of supervised learning where an instance may belong to multiple labels simultaneously. Predicting each label independently has been criticized for not exploiting any correlation between labels. In this paper we propose a novel approach, Nearest Labelset using Double Distances (NLDD)…
Paper proposes a chi-square test for distance correlation.
problem Testing distance correlation is computationally expensive.
method Proposes a chi-square test for distance correlation, non-parametric, fast, applicable to various metrics.
result Chi-square test exhibits similar power to permutation test and can be valid and universally consistent for testing independence.
This paper uses rank correlation methods to construct MSTs from financial returns, finding them more stable and robust.
problem Stability and robustness of MSTs constructed from financial correlation matrices.
method Pearson, Spearman, and Kendall's τ rank correlation methods applied to daily financial returns. result Rank MSTs are more stable and robust than MSTs constructed using Pearson correlation.
The paper describes correlations of spectra for higher rank Anosov representations.
problem Understanding correlations of spectra for Anosov representations of higher rank groups.
method Relates correlation problem to counting projections in truncated hypertubes.
result Extends previous work on rank one representations to higher rank.
Magnetoencephalography and electroencephalography (M/EEG) can reveal neuronal dynamics non-invasively in real-time and are therefore appreciated methods in medicine and neuroscience. Recent advances in modeling brain-behavior relationships have highlighted the effectiveness of Riemannian geometry for summarizing the sp…
Standardizes weighted ranking correlation coefficients to maintain zero expected value.
problem Measuring correlation between weighted rankings of items.
method Develops a standardization function g(·) that transforms coefficients to zero expected value under randomness.
result A general standardization function g(Γ) that preserves the domain [-1,1] and reduces to the identity for coefficients already satisfying zero-expected-value property.
Understanding and developing a correlation measure that can detect general dependencies is not only imperative to statistics and machine learning, but also crucial to general scientific discovery in the big data age. In this paper, we establish a new framework that generalizes distance correlation --- a correlation mea…
For time series comparisons, it has often been observed that z-score normalized Euclidean distances far outperform the unnormalized variant. In this paper we show that a z-score normalized, squared Euclidean Distance is, in fact, equal to a distance based on Pearson Correlation. This has profound impact on many distanc…
Paper extends multivariate rank tests for robust subspace detection.
problem Testing distributional similarity in multivariate data.
method Soft and subspace robust multivariate rank tests based on entropy regularized optimal transport.
result Trade-off between detection power and false alarm rate via projections.
A new method for Gaussian Processes handles mixed continuous and categorical inputs.
problem Modeling cross-correlations between continuous and categorical data.
method Low-Rank Correlation (LRC) method for Gaussian Processes with flexible rank approximation.
result LRC outperforms existing methods in estimating cross-correlations and predicting response surfaces.
This note improves correlation stress tests using geodesic distance.
problem Improving financial risk management through better covariance stress tests.
method Proposes a new geometrically invariant definition of correlation stress tests.
result Demonstrates a submanifold approach to stress testing covariance matrices.
Paper relaxes differential privacy for correlated features, improving privacy-utility trade-off.
problem Standard differential privacy ignores feature correlation, leading to suboptimal privacy-utility balance.
method Introduces CorrDP framework that accounts for feature correlation, using total variation distance for quantification.
result CorrDP algorithms outperform standard DP in synthetic and real-world datasets with insensitive features.
Bayesian method for dynamic correlation matrices improves accuracy and responsiveness.
problem Challenges in estimating time-varying correlation matrices, including slow adaptation, insufficient regularization, and diffuse uncertainty.
method Low-rank factor representation with dynamic shrinkage prior and multivariate factor stochastic volatility model.
result Improved accuracy and responsiveness compared to competing methods in various challenging scenarios.
The problem of filtering information from large correlation matrices is of great importance in many applications. We have recently proposed the use of the Kullback-Leibler distance to measure the performance of filtering algorithms in recovering the underlying correlation matrix when the variables are described by a mu…
Image ranking is to rank images based on some known ranked images. In this paper, we propose an improved linear ordinal distance metric learning approach based on the linear distance metric learning model. By decomposing the distance metric A as LTL, the problem can be cast as looking for a linear map between two …
We show that the Kullback-Leibler distance is a good measure of the statistical uncertainty of correlation matrices estimated by using a finite set of data. For correlation matrices of multivariate Gaussian variables we analytically determine the expected values of the Kullback-Leibler distance of a sample correlation …
Proposes a method to enhance multi-view learning by maximizing higher order correlations.
problem Losing intrinsic interconnections among multiple views in pairwise correlation maximization.
method Formulates multi-view data as a low rank approximation problem using higher order correlation tensor and solves it with the generating polynomial method.
result Consistently outperforms prior methods on real multi-view data.
Deep neural networks (DNNs) are notorious for their vulnerability to adversarial attacks, which are small perturbations added to their input images to mislead their prediction. Detection of adversarial examples is, therefore, a fundamental requirement for robust classification frameworks. In this work, we present a met…
Enhances labels from unlabeled data using sample correlations.
problem Lack of label distributions in real-world applications.
method Proposes LESC and gLESC methods to enhance label distributions.
result Improves performance of label enhancement through sample correlations.
Develops log-Euclidean Lie groups for SPD and correlation matrices.
problem Unifies various log-Euclidean constructions for SPD and correlation matrices.
method Theory and explicit isometries linking different log-Euclidean metrics.
result Explicit log-Euclidean metrics on SPD and correlation matrices.
The paper uses distance correlation for brain connectivity and a novel multi-task learning model for age prediction.
problem Estimating age-related gender differences in brain functional connectivity.
method Estimates functional connectivity using distance correlation and proposes a non-convex multi-task learning model.
result The proposed non-convex multi-task learning model outperforms other models in age prediction and gender-specific connectivity.
Reduces data leakage in distributed deep learning models.
problem Prevents reconstruction of sensitive raw data patterns during client communications.
method Reduces distance correlation between raw data and learned representations.
result Resilient to reconstruction attacks while maintaining model accuracy.
New methods rank players using covariates and comparisons, outperforming existing algorithms.
problem Ranking players based on incomplete and noisy pairwise comparisons.
method Three spectral ranking methods incorporating player covariates.
result Proposed methods outperform existing algorithms in simulations.
A self-supervised debiasing method using rank regularization mitigates spurious correlations in neural networks.
problem Spurious correlations cause biases in deep neural networks, affecting generalization.
method Spectral analysis of latent representations, rank regularization, self-supervised pretraining, debiasing of downstream tasks.
result The proposed framework significantly improves generalization performance and outperforms supervised debiasing approaches.
MAXENT method outperforms ML in sparse data with specific prior correlations.
problem Evaluating MAXENT method's validity limits and comparing it with ML.
method Bayesian decision theory, Dirichlet density, KL distance, regularized maximum likelihood.
result MAXENT can outperform ML in sparse data with specific prior correlations.
This paper addresses the problem of low-rank distance matrix completion. This problem amounts to recover the missing entries of a distance matrix when the dimension of the data embedding space is possibly unknown but small compared to the number of considered data points. The focus is on high-dimensional problems. We r…
Proposes a hierarchical clustering method for positive and negative dissimilarities.
problem Clustering dissimilarities, especially positive and negative.
method Hierarchical correlation clustering followed by tree preserving embedding.
result Performance on various datasets.
This paper ranks pre-trained DNNs using a novel SI measure.
problem Optimizing pre-trained DNN selection for transfer learning.
method Automated ranking via Separation Index (SI) on target datasets.
result Ranked pre-trained DNNs improve classification performance.
The density matrices are positively semi-definite Hermitian matrices of unit trace that describe the state of a quantum system. The goal of the paper is to develop minimax lower bounds on error rates of estimation of low rank density matrices in trace regression models used in quantum state tomography (in particular, i…
Private method measures nonlinear correlations between data hosted across two entities.
problem Measuring nonlinear correlations between sensitive data hosted across multiple parties while preserving privacy.
method Differentially private estimator of distance correlation.
result First private estimator of nonlinear correlations in a multi-party setup.
In this paper, we deal with the problem of inferring causal directions when the data is on discrete domain. By considering the distribution of the cause P(X) and the conditional distribution mapping cause to effect P(Y∣X) as independent random variables, we propose to infer the causal direction via comparing the di…
Novel beamforming method reduces errors in wireless networks.
problem Mitigating channel errors in wireless networks with relays.
method Low-rank and cross-correlation techniques for robust distributed beamforming.
result The proposed LRCC-RDB technique significantly improves SINR performance.
Completes the space of vector-valued one-forms on manifolds.
problem Metric incompleteness of the space of full-ranked one-forms.
method Distance equality and quotient structures.
result Concrete description of the metric completion of the space of full-ranked one-forms.
There has been an increasing interest in testing the equality of large Pearson's correlation matrices. However, in many applications it is more important to test the equality of large rank-based correlation matrices since they are more robust to outliers and nonlinearity. Unlike the Pearson's case, testing the equality…
BDC uses Distance Correlation for efficient Bayesian optimization of expensive functions.
problem Efficiently optimizing expensive black-box functions with Bayesian methods.
method Integrates Bayesian optimization with Distance Correlation for automatic exploration and exploitation.
result BDC performs similarly to popular BO methods on benchmark tests and real terrain optimization.
Develops a method for stress testing correlations of financial portfolios.
problem Stress testing correlations in financial asset portfolios.
method Parametric representation of correlations, Bayesian variable selection, joint distribution of stress scenarios.
result Inference of worst-case correlation scenarios using stress tests.
A method for ranking items using distance-based learning from positive and unlabeled data.
problem Learning to rank items without an analytic description of what constitutes a good ranking.
method Combining representations using an integer linear program for ranking items based on nominations.
result The method is effective in simulation and real data examples, especially when supervision is light.