Research
On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,657 papers · 148 categories

Trend · papers per month

3607191,0791,438 · Jun 202019922001200920172026
48 results for statistical data analysis

Paper proposes a statistical test for feature selection pipelines using selective inference.

problem Assessing the significance of feature selection pipelines in data analysis.
method Selective inference technique applied to feature selection pipelines composed of various algorithms.
result The proposed statistical test controls false positive feature selection probabilities.

Study proposes a statistical testing framework for evaluating clustering pipelines.

problem Quantifying the statistical reliability of clustering results from data analysis pipelines.
method Selective inference-based statistical testing framework for clustering pipelines.
result The proposed test controls the type I error rate and is effective in validating clustering results.

Unified platform for statistical and machine learning in bioinformatics.

problem Workflow inefficiencies in using multiple tools for data analysis.
method Automated hyperparameter optimization, feature importance analysis, statistical tests.
result Accelerates biological discovery workflows with methodological soundness.

The paper proposes methods to infer from privacy-protected data using simulation-based techniques.

problem Valid statistical inference from privacy-protected data is computationally challenging.
method Simulation-based inference methods, including sequential Monte Carlo and neural conditional density estimators.
result Valid statistical inferences can be made from privacy-protected data.

Making sense of a dataset in an automatic and unsupervised fashion is a challenging problem in statistics and AI. Classical approaches for {exploratory data analysis} are usually not flexible enough to deal with the uncertainty inherent to real-world data: they are often restricted to fixed latent interaction models an…

2018-07-24abs ↗pdf ↗

As regulators pay more attentions to losses rather than gains, we are able to derive a new class of risk statistics, named regulator-based risk statistics with scenario analysis in this paper. This new class of risk statistics can be considered as a kind of risk extension of risk statistics introduced by Kou et al. \ci…

2019-04-16abs ↗pdf ↗

The paper examines extreme value statistics of high-dimensional sample covariances, with applications in finance and image analysis.

problem Statistical validation of normal conditions in high-dimensional time series data.
method Generalizes the maximal deviation of sample autocovariances to high dimensions and applies Gumbel-type extreme value asymptotics.
result Gumbel-type extreme value asymptotics holds true for high-dimensional sample covariances.

The book chapter discusses tail risk analysis for financial data using extreme value statistics.

problem Serial dependence in financial time series complicates tail risk assessment.
method The approach involves unconditional and conditional quantile forecasting.
result Serial dependence impacts multivariate tail dependence.

In recent years, ideas from statistics and scientific computing have begun to interact in increasingly sophisticated and fruitful ways with ideas from computer science and the theory of algorithms to aid in the development of improved worst-case algorithms that are useful for large-scale scientific and Internet data an…

2010-10-08abs ↗pdf ↗

This describes a statistical technique called "tonsuring" for exploratory data analysis in finance. Instead of rejecting "outlier" data that conflicts with the model, this strips out "inlier" data to get a clearer picture of how the market changes for larger moves.

2011-10-20abs ↗pdf ↗

Adapts data analysis for growing data, improving generalization guarantees.

problem Challenges of overfitting and statistical validity in adaptive workflows with growing data.
method Generalizes adaptive analysis on dynamic data, incorporating time-varying empirical accuracy bounds and mechanisms.
result First generalization bounds for adaptive analysis on dynamic data, matching prior works' improvement over data splitting.

The paper gives picture of enrichment to economic and financial system analysis using agent-based models as a form of advanced study for financial economic data post-statistical-data analysis and micro-simulation analysis. Theoretical exploration is carried out by using comparisons of some usual financial economy syste…

2004-03-21abs ↗pdf ↗

Big Data bring new opportunities to modern society and challenges to data scientists. On one hand, Big Data hold great promises for discovering subtle population patterns and heterogeneities that are not possible with small-scale data. On the other hand, the massive sample size and high dimensionality of Big Data intro…

2013-08-07abs ↗pdf ↗

This paper introduces compositional data analysis for financial ratios, improving industry-level analysis.

problem Statistical issues with standard financial ratios at industry level.
method Compositional data analysis techniques for financial ratios.
result Improved analysis of financial ratios using compositional data methods.

Neuroimaging research has predominantly drawn conclusions based on classical statistics, including null-hypothesis testing, t-tests, and ANOVA. Throughout recent years, statistical learning methods enjoy increasing popularity, including cross-validation, pattern classification, and sparsity-inducing regression. These t…

2016-03-06abs ↗pdf ↗

Develops methods for causal inference in compositional data using instrumental variables.

problem Interpreting summary statistics like diversity indices as causal effects in compositional data.
method Statistical data transformations and regression techniques tailored for compositional data.
result Advantages and limitations of the proposed methods demonstrated on synthetic and real microbiome data.

The problem of complex data analysis is a central topic of modern statistical science and learning systems and is becoming of broader interest with the increasing prevalence of high-dimensional data. The challenge is to develop statistical models and autonomous algorithms that are able to acquire knowledge from raw dat…

2018-03-01abs ↗pdf ↗

Traditional statistical theory assumes that the analysis to be performed on a given data set is selected independently of the data themselves. This assumption breaks downs when data are re-used across analyses and the analysis to be performed at a given stage depends on the results of earlier stages. Such dependency ca…

2017-06-02abs ↗pdf ↗

Hypothesis testing is one of the most common types of data analysis and forms the backbone of scientific research in many disciplines. Analysis of variance (ANOVA) in particular is used to detect dependence between a categorical and a numerical variable. Here we show how one can carry out this hypothesis test under the…

2019-03-01abs ↗pdf ↗

Framework for efficient statistical estimation with privacy guarantees.

problem Statistical estimation problems with differential privacy constraints.
method High-dimensional Propose-Test-Release (HPTR) framework combining exponential mechanism, robust statistics, and resilience.
result Near-optimal utility guarantees and tight local sensitivity bounds for various statistical problems.

Real data often contain anomalous cases, also known as outliers. These may spoil the resulting analysis but they may also contain valuable information. In either case, the ability to detect such anomalies is essential. A useful tool for this purpose is robust statistics, which aims to detect the outliers by first fitti…

2017-07-31abs ↗pdf ↗

Big data sets must be carefully partitioned into statistically similar data subsets that can be used as representative samples for big data analysis tasks. In this paper, we propose the random sample partition (RSP) data model to represent a big data set as a set of non-overlapping data subsets, called RSP data blocks,…

2017-12-12abs ↗pdf ↗

The statistical analysis of data lying on a differentiable, locally Euclidean, manifold introduces a variety of challenges because the analogous measures to standard Euclidean statistics are local, that is only defined within a neighbourhood of each datapoint. This is because the curvature of the space means that the c…

2015-11-11abs ↗pdf ↗

Symbolic data analysis (SDA) is an emerging area of statistics concerned with understanding and modelling data that takes distributional form (i.e. symbols), such as random lists, intervals and histograms. It was developed under the premise that the statistical unit of interest is the symbol, and that inference is requ…

2018-09-11abs ↗pdf ↗

Neuroscience is undergoing faster changes than ever before. Over 100 years our field qualitatively described and invasively manipulated single or few organisms to gain anatomical, physiological, and pharmacological insights. In the last 10 years neuroscience spawned quantitative big-sample datasets on microanatomy, syn…

2016-08-05abs ↗pdf ↗

Machine learning and statistical modeling complement each other in healthcare analytics.

problem Choosing between machine learning and statistical modeling for analytics challenges.
method Choosing based on problem, data, and desired outcomes.
result Machine learning and statistical modeling are complementary, using similar principles but different tools.

For high dimensional data, some of the standard statistical techniques do not work well. So modification or further development of statistical methods are necessary. In this paper, we explore these modifications. We start with the important problem of estimating high dimensional covariance matrix. Then we explore some …

2018-08-08abs ↗pdf ↗

This paper proposes a statistical mechanics approach to the analysis of income distribution and inequality. A new distribution function, having its roots in the framework of k-generalized statistics, is derived that is particularly suitable to describe the whole spectrum of incomes, from the low-middle income region up…

2009-01-31abs ↗pdf ↗

Overview of high-dimensional time series regression methods.

problem Estimation and inference with high-dimensional time series data.
method Limit theory for high-dimensional dependent data, asymptotic theory for time series regression, statistical learning methods.
result Main limit theory results and asymptotic theory for high-dimensional time series regression.