arXiv research

New Hermite series estimator for Spearman rank correlation in non-stationary data.

We study the problem of rank aggregation: given a set of ranked lists, we want to form a consensus ranking. Furthermore, we consider the case of extreme lists: i.e., only the rank of the best or worst elements are known. We impute missing ranks by the average value and generalise Spearman's ρto extreme ranks. Our main …

New unsupervised feature selection method for imbalanced datasets.

Standardizes weighted ranking correlation coefficients to maintain zero expected value.

In the last decade, many diverse advances have occurred in the field of information extraction from data. Information extraction in its simplest form takes place in computing environments, where structured data can be extracted through a series of queries. The continuous expansion of quantities of data have therefore p…

This paper uses rank correlation methods to construct MSTs from financial returns, finding them more stable and robust.

Study examines stock price correlations between Indonesian holding companies and their subsidiaries.

Given a set $V$ of $n$ objects, an online ranking system outputs at each time step a full ranking of the set, observes a feedback of some form and suffers a loss. We study the setting in which the (adversarial) feedback is an element in $V$, and the loss is the position (0th, 1st, 2nd...) of the item in the outputted r…

A new sparse benchmark metabench identifies key abilities from large benchmarks.

ChatGPT predicts stock market movements based on Bloomberg headlines, showing a positive correlation over short to medium terms.

STRAPSim measures ETF portfolio similarity better than existing methods.

Nonparametric correlations such as Spearman's rank correlation and Kendall's tau correlation are widely applied in scientific and engineering fields. This paper investigates the problem of computing nonparametric correlations on the fly for streaming data. Standard batch algorithms are generally too slow to handle real…

A simple method reduces bias in LLM auto-evaluators by controlling output length.

This paper fills in local bounds for Spearman's footrule and Gini's gamma measures of association.

Novel fusion of autoencoders predicts sleepiness from speech.

Developers of text-to-speech synthesizers (TTS) often make use of human raters to assess the quality of synthesized speech. We demonstrate that we can model human raters' mean opinion scores (MOS) of synthesized speech using a deep recurrent neural network whose inputs consist solely of a raw waveform. Our best models …

We propose a semiparametric approach, named nonparanormal skeptic, for estimating high dimensional undirected graphical models. In terms of modeling, we consider the nonparanormal family proposed by Liu et al (2009). In terms of estimation, we exploit nonparametric rank-based correlation coefficient estimators includin…

This paper proposes a new class of copulas which characterize the set of all twice continuously differentiable copulas. We show that our proposed new class of copulas is a new generalized copula family that include not only asymmetric copulas but also all smooth copula families available in the current literature. Spea…

The Chirikov standard map and the 2D Froeschlé map are investigated. A few thousand values of the Hurst exponent (HE) and the maximal Lyapunov exponent (mLE) are plotted in a mixed space of the nonlinear parameter versus the initial condition. Both characteristic exponents reveal remarkably similar structures in this s…

An investigation is presented of how a comprehensive choice of five most important measures of concordance (namely Spearman's rho, Kendall's tau, Gini's gamma, Blomqvist's beta, and their weaker counterpart Spearman's footrule) relate to non-exchangeability, i.e., asymmetry on copulas. Besides these results, the method…

An automated metric to evaluate dialogue quality is vital for optimizing data driven dialogue management. The common approach of relying on explicit user feedback during a conversation is intrusive and sparse. Current models to estimate user satisfaction use limited feature sets and rely on annotation schemes with low …

We investigate the relative information content of six measures of dependence between two random variables $X$ and $Y$ for large or extreme events for several models of interest for financial time series. The six measures of dependence are respectively the linear correlation $ρ^+_v$ and Spearman's rho $ρ_s(v)$ conditio…

An investigation is presented of how a comprehensive choice of four most important measures of concordance (namely Spearman's rho, Kendall's tau, Spearman's footrule, and Gini's gamma) relate to the fifth one, i.e., the Blomqvist's beta. In order to work out these results we present a novel method of estimating the val…

Proposes clustering as a new evaluation method for clinical knowledge embedding.

This paper proves a generalization bound for complex-valued neural networks scaling with spectral complexity.

Forest tree species mapped with high accuracy using satellite data.

Optimal Word2Vec hyper-parameters improve NLP tasks.

In the IEEE Investment ranking challenge 2018, participants were asked to build a model which would identify the best performing stocks based on their returns over a forward six months window. Anonymized financial predictors and semi-annual returns were provided for a group of anonymized stocks from 1996 to 2017, which…

Paper proposes Coalitional BAE to improve explainability of unsupervised deep learning models.

Several tasks in machine learning are evaluated using non-differentiable metrics such as mean average precision or Spearman correlation. However, their non-differentiability prevents from using them as objective functions in a learning framework. Surrogate and relaxation methods exist but tend to be specific to a given…

We introduce a new family of minmax rank aggregation problems under two distance measures, the Kendall τ and the Spearman footrule. As the problems are NP-hard, we proceed to describe a number of constant-approximation algorithms for solving them. We conclude with illustrative applications of the aggregation methods on…

New method for robust financial portfolio analysis.

In this paper, we propose a semiparametric approach, named nonparanormal skeptic, for efficiently and robustly estimating high dimensional undirected graphical models. To achieve modeling flexibility, we consider Gaussian Copula graphical models (or the nonparanormal) as proposed by Liu et al. (2009). To achieve estima…

Eliciting semantic similarity between concepts in the biomedical domain remains a challenging task. Recent approaches founded on embedding vectors have gained in popularity as they risen to efficiently capture semantic relationships The underlying idea is that two words that have close meaning gather similar contexts. …

Study detects SLI in children from spontaneous narrative transcripts.

MONSTOR estimates influence in unseen networks with high accuracy.

Efficiently predict LLM benchmarks using feature selection and regression.

Paper introduces differentiable sorting and ranking with $O(n \log n)$ time complexity.

Unified framework detects overfitting in crash classification models.

Improved DeepONet variants using Transformer cross-conditioning enhance PDE solution efficiency.

New method evaluates AI stock prediction systems based on decision-making processes.

NLR models often perform worse than LR for outlying input data in environmental sciences.

The paper introduces a framework to select efficient datasets for preserving model rankings.

Paper introduces MVS to detect non-Markovian observations in reinforcement learning.

This paper presents performance analysis of hybrid model comprise of concordance and Genetic Programming (GP) to forecast financial market with some existing models. This scheme can be used for in depth analysis of stock market. Different measures of concordances such as Kendalls Tau, Ginis Mean Difference, Spearmans R…

A geometric account explains why 'The Dress' is ambiguous, predicting observable signatures in image processing.

We provide a set of copulas that can be interpreted as having the negative extreme dependence. This set of copulas is interesting because it coincides with countermonotonic copula for a bivariate case, and more importantly, is shown to be minimal in concordance ordering in the sense that no copula exists which is stric…

Parallel deep learning architectures like fine-tuned BERT and MT-DNN, have quickly become the state of the art, bypassing previous deep and shallow learning methods by a large margin. More recently, pre-trained models from large related datasets have been able to perform well on many downstream tasks by just fine-tunin…