Estimates Hurst exponent of log-volatility using KS statistic, addressing serial correlation in financial data.
problem Estimating Hurst exponent of log-volatility in financial time series with serial correlation.
method Proposes a random permutation procedure to remove serial correlation, using the Kolmogorov-Smirnov statistic for distribution-based estimation.
result Establishes the asymptotic variance of the estimator and reveals statistically significant hierarchy of roughness in volatility measures.
We present an extension of the Kolmogorov-Smirnov (KS) two-sample test, which can be more sensitive to differences in the tails. Our test statistic is an integral probability metric (IPM) defined over a higher-order total variation ball, recovering the original KS test as its simplest case. We give an exact representer…
A new method, InfoGuide, improves automatic clustering analysis.
problem Lack of automatic clustering analysis frameworks.
method Capturing traces of information gain between clustering retrievals.
result InfoGuide can enable more automatic clustering analysis.
Study evaluates two-sample tests for validating generative models in high dimensions.
problem Validating the performance and efficiency of non-parametric two-sample tests for high-dimensional generative models.
method Proposes and evaluates the sliced Wasserstein distance, mean of Kolmogorov-Smirnov statistics, and novel sliced Kolmogorov-Smirnov statistic.
result One-dimensional-based tests provide comparable sensitivity to other multivariate metrics but with lower computational cost.
Generative Adversarial Networks improve robust statistics for various distributions.
problem Estimating unknown parameters in adversarially corrupted samples.
method Designing GANs with specific loss functions for robust estimation.
result Extends robust estimation to broader families of distributions.
KSGAN uses KS distance for deep generative modeling.
problem Deep generative modeling challenges, especially for multivariate distributions.
method Formulates adversarial training as minimization of KS distance, using quantile function as critic.
result KSGAN trained distributions closely match target distributions.
Estimates financial market impacts of COVID-19 using time-varying kernel density.
problem Estimating the impact of COVID-19 on financial markets over time.
method Time-varying kernel density estimation with Kolmogorov-Smirnov statistic.
result Determines the chronology and regional disparities of financial market impacts.
The paper fits a seven-parameter GTS distribution to financial data.
problem Nonexistence of GTS probability density function makes MLE inadequate.
method Used fractional Fourier transform to circumvent MLE and provide good parameter estimation.
result The GTS distribution fits financial data significantly better than other models.
MAGDiff detects data shifts in neural networks without retraining.
problem Neural networks' sensitivity to data distribution shifts.
method Extracts MAGDiff representations from neural networks to detect shifts.
result MAGDiff representations improve data set shift detection.
SurvLIME-KS improves survival model explanations robustly.
problem Improving explanations of unreliable survival models.
method SurvLIME-KS combines Cox proportional hazards model and Kolmogorov-Smirnov bounds for robust optimization.
result SurvLIME-KS minimizes average distance and maximizes distance in approximating cumulative hazard functions.
The paper introduces a new method to detect rough volatility and market states using fractional derivatives.
problem Testing self-similarity in fractional processes from a single observed trajectory is difficult under long-range dependence.
method The paper introduces a regime-adaptive KS/GL--KS framework based on the discrete Grünwald--Letnikov (GL) fractional derivative.
result The method detects rough volatility and persistent, anti-persistent, or efficient market states in financial applications.
The paper introduces a spline-based method for calibrating neural networks.
problem Ensuring neural network outputs are reliable for safety-critical applications.
method Approximating the empirical cumulative distribution function using splines to map network outputs to calibrated probabilities.
result The spline-based recalibration consistently outperforms existing methods on calibration measures.
This paper reports empirical evidence that a neural networks model is applicable to the statistically reliable prediction of foreign exchange rates. Time series data and technical indicators such as moving average, are fed to neural nets to capture the underlying "rules" of the movement in currency exchange rates. The …
Accurate goodness-of-fit tests for the extreme tails of empirical distributions is a very important issue, relevant in many contexts, including geophysics, insurance, and finance. We have derived exact asymptotic results for a generalization of the large-sample Kolmogorov-Smirnov test, well suited to testing these extr…
We study a novel spline-like basis, which we name the "falling factorial basis", bearing many similarities to the classic truncated power basis. The advantage of the falling factorial basis is that it enables rapid, linear-time computations in basis matrix multiplication and basis matrix inversion. The falling factoria…
New test uses neural networks to compare distributions, outperforming traditional methods.
problem Comparing distributions in high dimensions and higher orders of smoothness.
method Integral probability metrics with Radon bounded variation functions and neural networks.
result The Radon-Kolmogorov-Smirnov (RKS) test outperforms traditional methods in distinguishing distributions.
We investigate the probability distributions of the recurrence intervals τ between consecutive 1-min returns above a positive threshold q>0 or below a negative threshold q<0 of two indices and 20 individual stocks in China's stock market. The distributions of recurrence intervals for positive and negative thresho…
Proposes a new TS algorithm for non-stationary bandits using KS tests.
problem Non-stationary multi-armed bandit problems.
method Active detection of change points using KS tests and adaptive Thompson Sampling.
result Sub-linear regret demonstrated for the two-armed bandit case.
We study the problem of estimating a nonparametric probability density under a large family of losses called Besov IPMs, which include, for example, Lp distances, total variation distance, and generalizations of both Wasserstein and Kolmogorov-Smirnov distances. For a wide variety of settings, we provide b…
In bankruptcy prediction, the proportion of events is very low, which is often oversampled to eliminate this bias. In this paper, we study the influence of the event rate on discrimination abilities of bankruptcy prediction models. First the statistical association and significance of public records and firmographics i…
We compute the analytic expression of the probability distributions F{AEX,+} and F{AEX,-} of the normalized positive and negative AEX (Netherlands) index daily returns r(t). Furthermore, we define the αre-scaled AEX daily index positive returns r(t)^αand negative returns (-r(t))^αthat we call, after normalization, the …
We propose a model for price formation in financial markets based on clearing of a standard call auction with random orders, and verify its validity for prediction of the daily closing price distribution statistically. The model considers random buy and sell orders, placed following demand- and supply-side valuation di…
We study the statistical properties of the recurrence intervals τ between successive trading volumes exceeding a certain threshold q. The recurrence interval analysis is carried out for the 20 liquid Chinese stocks covering a period from January 2000 to May 2009, and two Chinese indices from January 2003 to April 2…
We compute the analytic expression of the probability distributions F{FTSE100,+} and F{FTSE100,-} of the normalized positive and negative FTSE100 (UK) index daily returns r(t). Furthermore, we define the alpha re-scaled FTSE100 daily index positive returns r(t)^alpha and negative returns (-r(t))^alpha that we call, aft…
Nonparametric two sample or homogeneity testing is a decision theoretic problem that involves identifying differences between two random variables without making parametric assumptions about their underlying distributions. The literature is old and rich, with a wide variety of statistics having being intelligently desi…
The paper examines the stability of binary choice models using Gini index and scoring indicators.
problem Stability and discriminatory power of binary choice models.
method Derives the real Gini index and incorporates PSI and KS statistics into the model.
result The real Gini index should be less than the calculated Gini index when the population distribution changes.
In terms of the stock exchange returns, we compute the analytic expression of the probability distributions F{DAX,+} and F{DAX,-} of the normalized positive and negative DAX (Germany) index daily returns r(t). Furthermore, we define the alpha re-scaled DAX daily index positive returns r(t)^alpha and negative returns (-…
Proposes a framework to explain KS deterioration in credit risk models.
problem Inconsistent and ad hoc diagnosis of KS decline in credit risk models.
method Counterfactual diagnostic framework attributing KS decline to sampling variability, portfolio composition, covariate shift, and residual deterioration.
result The proposed approach provides more interpretable and governance-relevant explanations than threshold-based review alone.
The statistical properties of the return intervals τq between successive 1-min volatilities of 30 liquid Chinese stocks exceeding a certain threshold q are carefully studied. The Kolmogorov-Smirnov (KS) test shows that 12 stocks exhibit scaling behaviors in the distributions of τq for different thresholds q. …
This article proposes a method to quantify the structure of a bipartite graph using a network entropy per link. The network entropy of a bipartite graph with random links is calculated both numerically and theoretically. As an application of the proposed method to analyze collective behavior, the affairs in which parti…
Many problems in finance are related to first passage times. Among all of them, we chose three on which we contributed personally. Our first example relates Kolmogorov-Smirnov like goodness-of-fit tests, modified in such a way that tail events and core events contribute equally to the test (in the standard Kolmogorov-S…
Spectral denoising recovers meaningful network structure from noisy financial correlations.
problem Noise in empirical correlation matrices from financial returns obscures genuine interactions.
method Spectral decomposition to separate structured and random components.
result Structured networks derived from 10-16 eigenmodes exhibit stronger core-periphery organization and scale-free degree distributions.
Two methods improve Gaussian process predictive distributions' calibration.
problem Improving the reliability of Gaussian process predictive intervals.
method Introduces two methods: cps-gp and bcr-gp, both adapting conformal predictive systems to GP interpolation.
result Both methods provide finite-sample marginal calibration and smooth predictive distributions.
A new UU-test decides unimodality of datasets.
problem Deciding on the unimodality of a dataset for better data analysis.
method UU-test operates on the empirical cumulative density function (ecdf) to build a piecewise linear approximation that models the data as a Uniform Mixture Model.
result The UU-test provides a statistical model of the data in the form of a Uniform Mixture Model.
This paper studies the problem of Generalized Zero-shot Learning (G-ZSL), whose goal is to classify instances belonging to both seen and unseen classes at the test time. We propose a novel space decomposition method to solve G-ZSL. Some previous models with space decomposition operations only calibrate the confident pr…
Improved change point detection using matched filters for non-parametric tests.
problem False positives and localization ambiguity in non-parametric two-sample tests.
method Derived and applied matched filters for various two-sample tests.
result Matched filters reduce false positives and improve test precision.
SafeML monitors ML systems for safety and security risks.
problem Ensuring safety and explainability of ML systems in safety-critical domains.
method Statistical difference measures of ECDF to detect distributional shifts.
result Approach can detect invalid application contexts of ML components.
In this paper, we propose a hybrid bankcard response model, which integrates decision tree based chi-square automatic interaction detection (CHAID) into logistic regression. In the first stage of the hybrid model, CHAID analysis is used to detect the possibly potential variable interactions. Then in the second stage, t…
A new method combines federated learning and logistic regression for better credit scoring.
problem Improving credit scoring models while protecting data privacy.
method Projected gradient-based vertical federated learning (FL-LRBC) for logistic regression.
result Significant improvement in AUC and KS statistics due to data enrichment.
Study learns optimal auctions from corrupted or perturbed bidder valuation samples.
problem Learning revenue-optimal auctions from corrupted or perturbed samples.
method Proves upper bounds, proposes algorithms for learning near-optimal auctions.
result Proves tight upper bounds and proposes algorithms for near-optimal auctions.
We present a novel modulation level classification (MLC) method based on probability distribution distance functions. The proposed method uses modified Kuiper and Kolmogorov-Smirnov distances to achieve low computational complexity and outperforms the state of the art methods based on cumulants and goodness-of-fit test…
We study the problem of distinguishing between two distributions on a metric space; i.e., given metric measure spaces (X,d,μ1) and (X,d,μ2), we are interested in the problem of determining from finite data whether or not μ1 is μ2. The key is to use pairwise distances between observat…
Markov chain decoders improve generative models' ability to produce heavy-tailed data.
problem Generative models struggle with heavy-tailed distributions.
method Replaced Gaussian decoder with Markov chain-based Phase-Type distributions.
result Significantly reduced tail Kolmogorov-Smirnov distance and extreme quantile error.
The paper uses FRFT to fit GTS distribution to asset returns.
problem Modeling asset returns with GTS distribution.
method Fractional Fourier Transform (FRFT) for fitting.
result GTS distribution fits SPY ETF and Bitcoin BTC returns.
This study compares different types of normalizing flows for generating complex distributions.
problem Comparing different types of normalizing flows for generating complex distributions.
method Real-valued non-Volume preserving (RealNVP), masked autoregressive flow (MAF), coupling rational quadratic spline (C-RQS), and autoregressive rational quadratic spline (A-RQS) were compared using statistical tests.
result A-RQS algorithm outperforms others in terms of accuracy and training speed.
We investigate the probability distribution of the return intervals τ between successive 1-min volatilities of two Chinese indices exceeding a certain threshold q. The Kolmogorov-Smirnov (KS) tests show that the two indices exhibit multiscaling behavior in the distribution of τ, which follows a stretched exponent…
Principal component analysis (PCA) is very popular to perform dimension reduction. The selection of the number of significant components is essential but often based on some practical heuristics depending on the application. Only few works have proposed a probabilistic approach able to infer the number of significant c…
Energy markets and the associated energy futures markets play a crucial role in global economies. We investigate the statistical properties of the recurrence intervals of daily volatility time series of four NYMEX energy futures, which are defined as the waiting times τ between consecutive volatilities exceeding a gi…