arXiv research

APG samplers use neural suff stats to improve deep model inference.

Anomaly detection in database management systems (DBMSs) is difficult because of increasing number of statistics (stat) and event metrics in big data system. In this paper, I propose an automatic DBMS diagnosis system that detects anomaly periods with abnormal DB stat metrics and finds causal events in the periods. Rec…

In this article, we consider the sparse tensor singular value decomposition, which aims for dimension reduction on high-dimensional high-order data with certain sparsity structure. A method named Sparse Tensor Alternating Thresholding for Singular Value Decomposition (STAT-SVD) is proposed. The proposed procedure featu…

Ultrahigh-dimensional variable selection plays an increasingly important role in contemporary scientific discoveries and statistical research. Among others, Fan and Lv [J. R. Stat. Soc. Ser. B Stat. Methodol. 70 (2008) 849-911] propose an independent screening framework by ranking the marginal correlations. They showed…

We introduce extensions of stability selection, a method to stabilise variable selection methods introduced by Meinshausen and Bühlmann (J R Stat Soc 72:417-473, 2010). We propose to apply a base selection method repeatedly to random observation subsamples and covariate subsets under scrutiny, and to select covariates …

We present results about financial market observables, specifically returns and traded volumes. They are obtained within the current nonextensive statistical mechanical framework based on the entropy $S_{q}=k\frac{1-\sum\limits_{i=1}^{W} p_{i} ^{q}}{1-q} (q\in \Re)$ ($S_{1} \equiv S_{BG}=-k\sum\limits_{i=1}^{W}p_{i} \l…

The book explores essential stats and psychology for quantitative trading.

This article studies local and global inference for smoothing spline estimation in a unified asymptotic framework. We first introduce a new technical tool called functional Bahadur representation, which significantly generalizes the traditional Bahadur representation in parametric models, that is, Bahadur [Ann. Inst. S…

Neural Bayes simplifies computing complex stats for unsupervised learning.

Framework for real-time win probability and player ability in sports.

The rapid growth of Electronic Health Records (EHRs), as well as the accompanied opportunities in Data-Driven Healthcare (DDH), has been attracting widespread interests and attentions. Recent progress in the design and applications of deep learning methods has shown promising results and is forcing massive changes in h…

Correlation matrices play a key role in many multivariate methods (e.g., graphical model estimation and factor analysis). The current state-of-the-art in estimating large correlation matrices focuses on the use of Pearson's sample correlation matrix. Although Pearson's sample correlation matrix enjoys various good prop…

Nowadays with a growing number of online controlling systems in the organization and also a high demand of monitoring and stats facilities that uses data streams to log and control their subsystems, data stream mining becomes more and more vital. Hoeffding Trees (also called Very Fast Decision Trees a.k.a. VFDT) as a B…

We introduce and discuss a nonlinear kinetic equation of Boltzmann type which describes the influence of knowledge in the evolution of wealth in a system of agents which interact through the binary trades introduced in Cordier, Pareschi, Toscani, J. Stat. Phys. 2005. The trades, which include both saving propensity and…

Holistic GLMs add constraints for better model quality.

We prove a large deviations principle for the class of multidimensional affine stochastic volatility models considered in (Gourieroux, C. and Sufana, R., J. Bus. Econ. Stat., 28(3), 2010), where the volatility matrix is modelled by a Wishart process. This class extends the very popular Heston model to the multivariate …

New methods combine AI with traditional stats for better treatment effect estimation.

This paper simplifies computing higher-order $U$-statistics efficiently.

Motivated by a recent work of X. Chen and M. Zhu (Commun. Math. Stat., 1 (2013) 369-385), we establish a Trudinger-Moser inequality on compact Riemannian surface without boundary. The proof is based on blow-up analysis together with Carleson-Chang's result (Bull. Sci. Math. 110 (1986) 113-127). This inequality is diffe…

Paper uses stats to predict treatment choice based on illness probability.

We present analytical investigations of a multiplicative stochastic process that models a simple investor dynamics in a random environment. The dynamics of the investor's budget, $x(t)$, depends on the stochasticity of the return on investment, $r(t)$, for which different model assumptions are discussed. The fat-tail d…

We investigate relationship between annual electric power consumption per capita and gross domestic production (GDP) per capita for 131 countries. We found that the relationship can be fitted with a power-law function. We examine the relationship for 47 prefectures in Japan. Furthermore, we investigate values of annual…

Estimates LLC for deep linear networks up to 100M parameters.

Study investigates how machine learning models degrade over time, leading to patient safety issues.

Sharp thresholds and contiguity for community detection in contextual SBM.

Novel approach uses Gaussian processes to estimate conflict trends.

Spectral clustering is a technique that clusters elements using the top few eigenvectors of their (possibly normalized) similarity matrix. The quality of spectral clustering is closely tied to the convergence properties of these principal eigenvectors. This rate of convergence has been shown to be identical for both th…

In a recent paper in this journal [J. Stat. Mech. (2009) P02037] we proposed a new, physically motivated, distribution function for modeling individual incomes having its roots in the framework of the k-generalized statistical mechanics. The performance of the k-generalized distribution was checked against real data on…

Paper generalizes tensor-train approximation for complex random variables.

Localized sketching improves matrix multiplication and ridge regression complexity.

A framework assesses the quality of crowdsourced weather data.

We find that when measured in terms of dollar-turnover, and once $β$-neutralised and Low-Vol neutralised, the Size Effect is alive and well. With a long term t-stat of $5.1$, the "Cold-Minus-Hot" (CMH) anomaly is certainly not less significant than other well-known factors such as Value or Quality. As compared to marke…

agtboost speeds up gradient tree boosting with automatic complexity adjustment.

New method uniquely identifies causal structure from ordinal data.

We establish the existence of anomalous excess returns based on trend following strategies across four asset classes (commodities, currencies, stock indices, bonds) and over very long time scales. We use for our studies both futures time series, that exist since 1960, and spot time series that allow us to go back to 18…

New framework identifies strongly identifiable models from flexible generators.

MegazordNet combines stats and ML for better financial time series forecasting.

We develop an iterative subsampling approach to improve the computational efficiency of our previous work on solution path clustering (SPC). The SPC method achieves clustering by concave regularization on the pairwise distances between cluster centers. This clustering method has the important capability to recognize no…

The paper proves an energy identity for harmonic maps near singularities.

In this article, we study the regularity of minimizing and stationary $p$-harmonic maps between Riemannian manifolds. The aim is obtaining Minkowski-type volume estimates on the singular set $S(f)=\{x \ \ s.t. \ \ f \text{ is not continuous at } x\}$, as opposed to the weaker and non quantitative Hausdorff dimension bo…

Categorical regressor variables are usually handled by introducing a set of indicator variables, and imposing a linear constraint to ensure identifiability in the presence of an intercept, or equivalently, using one of various coding schemes. As proposed in Yuan and Lin [J. R. Statist. Soc. B, 68 (2006), 49-67], the gr…

We show that the emerging field of discrete differential geometry can be usefully brought to bear on crystallization problems. In particular, we give a simplified proof of the Heitmann-Radin crystallization theorem (R. C. Heitmann, C. Radin, J. Stat. Phys. 22, 281-287, 1980), which concerns a system of $N$ identical at…

Classifier detects LLM-generated text with guarantees.

Enhanced ECCD speeds up elastic net model training.

The paper analyzes covariate shift in nonparametric regression with Markovian data.

Gaussian process (GP) models have become a well-established frameworkfor the adaptive design of costly experiments, and notably of computerexperiments. GP-based sequential designs have been found practicallyefficient for various objectives, such as global optimization(estimating the global maximum or maximizer(s) of a …

Components of biological systems interact with each other in order to carry out vital cell functions. Such information can be used to improve estimation and inference, and to obtain better insights into the underlying cellular mechanisms. Discovering regulatory interactions among genes is therefore an important problem…

A new method ranks uncertainty vectors from multiple measures for robust prediction.