New Hermite series estimator for Spearman rank correlation in non-stationary data.
problem Estimating time-varying Spearman rank correlation efficiently.
method Hermite series based sequential estimator for both stationary and non-stationary settings.
result Competitive performance compared to existing algorithms in simulations and real data.
We study the problem of rank aggregation: given a set of ranked lists, we want to form a consensus ranking. Furthermore, we consider the case of extreme lists: i.e., only the rank of the best or worst elements are known. We impute missing ranks by the average value and generalise Spearman's ρto extreme ranks. Our main …
Paper presents a technique using Spearman's Rank Correlation Coefficient for KE in TDs.
problem Extracting common characteristics and grouping similar TDs.
method Spearman's Rank Correlation Coefficient (SRCC) for KE.
result SRCC proves a comprehensive measure for high-quality KE.
New unsupervised feature selection method for imbalanced datasets.
problem Feature selection challenges in imbalanced multi-class datasets.
method Distance Rank Score using Spearman's Rank Correlation.
result Outperforms existing methods on clustering problems.
Standardizes weighted ranking correlation coefficients to maintain zero expected value.
problem Measuring correlation between weighted rankings of items.
method Develops a standardization function g(·) that transforms coefficients to zero expected value under randomness.
result A general standardization function g(Γ) that preserves the domain [-1,1] and reduces to the identity for coefficients already satisfying zero-expected-value property.
This paper uses rank correlation methods to construct MSTs from financial returns, finding them more stable and robust.
problem Stability and robustness of MSTs constructed from financial correlation matrices.
method Pearson, Spearman, and Kendall's τ rank correlation methods applied to daily financial returns. result Rank MSTs are more stable and robust than MSTs constructed using Pearson correlation.
Given a set V of n objects, an online ranking system outputs at each time step a full ranking of the set, observes a feedback of some form and suffers a loss. We study the setting in which the (adversarial) feedback is an element in V, and the loss is the position (0th, 1st, 2nd...) of the item in the outputted r…
We propose a semiparametric approach, named nonparanormal skeptic, for estimating high dimensional undirected graphical models. In terms of modeling, we consider the nonparanormal family proposed by Liu et al (2009). In terms of estimation, we exploit nonparametric rank-based correlation coefficient estimators includin…
Nonparametric correlations such as Spearman's rank correlation and Kendall's tau correlation are widely applied in scientific and engineering fields. This paper investigates the problem of computing nonparametric correlations on the fly for streaming data. Standard batch algorithms are generally too slow to handle real…
STRAPSim measures ETF portfolio similarity better than existing methods.
problem Measuring portfolio similarity for ETFs and portfolios.
method Semantic, two-level, residual-aware portfolio similarity computation.
result STRAPSim outperforms existing methods in predictive accuracy and ranking alignment.
In the IEEE Investment ranking challenge 2018, participants were asked to build a model which would identify the best performing stocks based on their returns over a forward six months window. Anonymized financial predictors and semi-annual returns were provided for a group of anonymized stocks from 1996 to 2017, which…
Several tasks in machine learning are evaluated using non-differentiable metrics such as mean average precision or Spearman correlation. However, their non-differentiability prevents from using them as objective functions in a learning framework. Surrogate and relaxation methods exist but tend to be specific to a given…
We introduce a new family of minmax rank aggregation problems under two distance measures, the Kendall τ and the Spearman footrule. As the problems are NP-hard, we proceed to describe a number of constant-approximation algorithms for solving them. We conclude with illustrative applications of the aggregation methods on…
Paper introduces differentiable sorting and ranking with O(nlogn) time complexity.
problem Non-differentiability of sorting and ranking operations in machine learning.
method Differentiable proxies constructed as projections onto the permutahedron and reduction to isotonic optimization.
result First differentiable sorting and ranking operators with O(nlogn) time and O(n) space complexity. The Chirikov standard map and the 2D Froeschlé map are investigated. A few thousand values of the Hurst exponent (HE) and the maximal Lyapunov exponent (mLE) are plotted in a mixed space of the nonlinear parameter versus the initial condition. Both characteristic exponents reveal remarkably similar structures in this s…
Study examines stock price correlations between Indonesian holding companies and their subsidiaries.
problem Understanding stock price relationships between holding companies and their subsidiaries.
method Spearman correlation analysis over 2013-2022, focusing on MNC Group and Emtek Group.
result Varying degrees of correlation between holding companies and their subsidiaries, with some showing inverse relationships.
The paper introduces a framework to select efficient datasets for preserving model rankings.
problem Efficient evaluation of machine learning models on small, representative datasets.
method Bootstrap aggregation, clustering, design criteria, random baselines, and greedy farthest-first (FAFI).
result Several selection strategies improve rank preservation compared to random subsets, especially in time series classification.
A new sparse benchmark metabench identifies key abilities from large benchmarks.
problem Redundancy and compression in existing benchmarks.
method Data from 5000+ LLMs to identify most informative items, distilling a sparse benchmark.
result Sparse benchmark metabench captures underlying abilities with high accuracy.
ChatGPT predicts stock market movements based on Bloomberg headlines, showing a positive correlation over short to medium terms.
problem Predicting stock market movements using news headlines.
method Used a two-stage prompt approach with a dataset of Bloomberg market summaries from 2010 to 2023.
result ChatGPT's sentiment scores correlate positively with future equity market returns over short to medium terms, with a negative correlation over longer horizons.
A simple method reduces bias in LLM auto-evaluators by controlling output length.
problem Bias in LLM auto-annotators, particularly length bias.
method Simple regression analysis to control for length difference and other mediators.
result Improved robustness and increased correlation with human preferences.
Paper proposes Coalitional BAE to improve explainability of unsupervised deep learning models.
problem Improving explainability of Autoencoder's predictions.
method Introduces Coalitional BAE, inspired by agent-based system theory, to reduce correlation in explanations.
result Improved quality of explanations using Coalitional BAE on publicly available datasets.
This paper proves a generalization bound for complex-valued neural networks scaling with spectral complexity.
problem Ensuring the performance of complex-valued neural networks on unseen data.
method Theoretical derivation using Maurey Sparsification Lemma and Dudley Entropy Integral, empirical validation on various datasets.
result The spectral complexity of weight matrices is a significant factor in the generalization ability of complex-valued neural networks.
This paper fills in local bounds for Spearman's footrule and Gini's gamma measures of association.
problem Local bounds for bivariate copulas with respect to Spearman's footrule and Gini's gamma measures.
method Computing quasi-copulas that are not copulas for certain values of the measures.
result Presented local bounds for Spearman's footrule and Gini's gamma measures.
Study examines Blomqvist's beta and four concordance measures on copulas.
problem Relating Blomqvist's beta to other concordance measures.
method Estimating concordance measures on copulas with fixed beta.
result Novel method for estimating concordance measures.
Novel fusion of autoencoders predicts sleepiness from speech.
problem Predicting sleepiness from speech recordings.
method Attention-based and recurrent sequence to sequence autoencoders for unsupervised representation learning.
result Fusion of autoencoders' representations achieves higher correlation with sleepiness scales.
In this paper, we propose a semiparametric approach, named nonparanormal skeptic, for efficiently and robustly estimating high dimensional undirected graphical models. To achieve modeling flexibility, we consider Gaussian Copula graphical models (or the nonparanormal) as proposed by Liu et al. (2009). To achieve estima…
Eliciting semantic similarity between concepts in the biomedical domain remains a challenging task. Recent approaches founded on embedding vectors have gained in popularity as they risen to efficiently capture semantic relationships The underlying idea is that two words that have close meaning gather similar contexts. …
Developers of text-to-speech synthesizers (TTS) often make use of human raters to assess the quality of synthesized speech. We demonstrate that we can model human raters' mean opinion scores (MOS) of synthesized speech using a deep recurrent neural network whose inputs consist solely of a raw waveform. Our best models …
Parallel deep learning architectures like fine-tuned BERT and MT-DNN, have quickly become the state of the art, bypassing previous deep and shallow learning methods by a large margin. More recently, pre-trained models from large related datasets have been able to perform well on many downstream tasks by just fine-tunin…
Efficiently predict LLM benchmarks using feature selection and regression.
problem Predicting full benchmark scores with minimal question subsets.
method Multiple regression with feature selection, using kernel ridge regression and mRMR.
result Improved prediction accuracy and ranking correlation across various benchmarks.
This paper proposes a new class of copulas which characterize the set of all twice continuously differentiable copulas. We show that our proposed new class of copulas is a new generalized copula family that include not only asymmetric copulas but also all smooth copula families available in the current literature. Spea…
Unified framework detects overfitting in crash classification models.
problem Evaluation metrics fail to detect overfitting in crash classification models.
method Random Matrix Theory and Heavy-Tailed Self-Regularization framework applied to various model types.
result Power-law exponent α reliably distinguishes well-regularized from overfit models.
An investigation is presented of how a comprehensive choice of five most important measures of concordance (namely Spearman's rho, Kendall's tau, Gini's gamma, Blomqvist's beta, and their weaker counterpart Spearman's footrule) relate to non-exchangeability, i.e., asymmetry on copulas. Besides these results, the method…
An automated metric to evaluate dialogue quality is vital for optimizing data driven dialogue management. The common approach of relying on explicit user feedback during a conversation is intrusive and sparse. Current models to estimate user satisfaction use limited feature sets and rely on annotation schemes with low …
We investigate the relative information content of six measures of dependence between two random variables X and Y for large or extreme events for several models of interest for financial time series. The six measures of dependence are respectively the linear correlation ρv+ and Spearman's rho ρs(v) conditio…
The paper describes correlations of spectra for higher rank Anosov representations.
problem Understanding correlations of spectra for Anosov representations of higher rank groups.
method Relates correlation problem to counting projections in truncated hypertubes.
result Extends previous work on rank one representations to higher rank.
Improved DeepONet variants using Transformer cross-conditioning enhance PDE solution efficiency.
problem Solving partial differential equations efficiently and accurately.
method Transformer-inspired DeepONet variants with bidirectional cross-conditioning.
result Improved efficiency and accuracy compared to modified DeepONet, with variant effectiveness tied to PDE characteristics.
A new method for Gaussian Processes handles mixed continuous and categorical inputs.
problem Modeling cross-correlations between continuous and categorical data.
method Low-Rank Correlation (LRC) method for Gaussian Processes with flexible rank approximation.
result LRC outperforms existing methods in estimating cross-correlations and predicting response surfaces.
Proposes a method to enhance multi-view learning by maximizing higher order correlations.
problem Losing intrinsic interconnections among multiple views in pairwise correlation maximization.
method Formulates multi-view data as a low rank approximation problem using higher order correlation tensor and solves it with the generating polynomial method.
result Consistently outperforms prior methods on real multi-view data.
Enhances labels from unlabeled data using sample correlations.
problem Lack of label distributions in real-world applications.
method Proposes LESC and gLESC methods to enhance label distributions.
result Improves performance of label enhancement through sample correlations.
New methods rank players using covariates and comparisons, outperforming existing algorithms.
problem Ranking players based on incomplete and noisy pairwise comparisons.
method Three spectral ranking methods incorporating player covariates.
result Proposed methods outperform existing algorithms in simulations.
A self-supervised debiasing method using rank regularization mitigates spurious correlations in neural networks.
problem Spurious correlations cause biases in deep neural networks, affecting generalization.
method Spectral analysis of latent representations, rank regularization, self-supervised pretraining, debiasing of downstream tasks.
result The proposed framework significantly improves generalization performance and outperforms supervised debiasing approaches.
Proposes clustering as a new evaluation method for clinical knowledge embedding.
problem Traditional Link Prediction evaluation protocol loses information and harms model accuracy.
method Proposes Clustering Evaluation Protocol as an alternative.
result Experimental results show the proposed protocol can potentially replace Link Prediction.
New ROC tools assess predictive abilities for any linearly ordered outcomes.
problem Fundamental restriction in ROC analysis for non-dichotomous outcomes.
method ROC movies and UROC curves for linearly ordered outcomes.
result CPA equals AUC for binary outcomes and relates to Spearman's coefficient for pairwise distinct outcomes.
LLM forecasting benchmarks suffer from information leakage, which confounds model performance.
problem LLM forecasting benchmarks suffer from information leakage.
method A retrieval-augmented LLM forecaster observes only decision-time information.
result The full pipeline obtains a median monthly Spearman rank IC of +0.154.
Forest tree species mapped with high accuracy using satellite data.
problem Classifying dominant tree species in Swedish forests.
method Extreme gradient boosting model with Bayesian optimization, combining Sentinel-1/2 satellite data and field observations.
result Overall accuracy of 85%, F1 score of 0.82, Matthews correlation coefficient of 0.81.
Optimal Word2Vec hyper-parameters improve NLP tasks.
problem Finding the best Word2Vec hyper-parameters for NLP tasks.
method Empirical evaluation of various hyper-parameter combinations on NLP tasks.
result The best hyper-parameters vary by task, and high analogy scores don't always correlate with performance.
Novel beamforming method reduces errors in wireless networks.
problem Mitigating channel errors in wireless networks with relays.
method Low-rank and cross-correlation techniques for robust distributed beamforming.
result The proposed LRCC-RDB technique significantly improves SINR performance.