A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Bayesian framework proves thresholds for multi-graph alignment feasibility.
The paper establishes a nearly-sharp statistical threshold for efficient learning in Latent MDPs with separated components.
The interplay between computational efficiency and statistical accuracy in high-dimensional inference has drawn increasing attention in the literature. In this paper, we study computational and statistical boundaries for submatrix localization. Given one observation of (one or multiple non-overlapping) signal submatrix…
Noise makes learning linear thresholds hard, but algorithms can still learn near-optimal thresholds.
Study on detecting and recovering hidden dense cycles in random graphs.
Iterative thresholding algorithms seek to optimize a differentiable objective function over a sparsity or rank constraint by alternating between gradient steps that reduce the objective, and thresholding steps that enforce the constraint. This work examines the choice of the thresholding operator, and asks whether it i…
We present a novel distribution-free approach, the data-driven threshold machine (DTM), for a fundamental problem at the core of many learning tasks: choose a threshold for a given pre-specified level that bounds the tail probability of the maximum of a (possibly dependent but stationary) random sequence. We do not ass…
The stochastic block model is one of the oldest and most ubiquitous models for studying clustering and community detection. In an exciting sequence of developments, motivated by deep but non-rigorous ideas from statistical physics, Decelle et al. conjectured a sharp threshold for when community detection is possible in…
Iterative shrinkage/thresholding algorithm (ISTA) is a well-studied method for finding sparse solutions to ill-posed inverse problems. In this letter, we present a data-driven scheme for learning optimal thresholding functions for ISTA. The proposed scheme is obtained by relating iterations of ISTA to layers of a simpl…
We investigate the probability distributions of the recurrence intervals between consecutive 1-min returns above a positive threshold or below a negative threshold of two indices and 20 individual stocks in China's stock market. The distributions of recurrence intervals for positive and negative thresho…
Characterizes optimal reconstruction error in high-dimensional Gaussian mixtures.
Sharp thresholds and contiguity for community detection in contextual SBM.
The use of M-estimators in generalized linear regression models in high dimensional settings requires risk minimization with hard constraints. Of the known methods, the class of projected gradient descent (also known as iterative hard thresholding (IHT)) methods is known to offer the fastest and most scalable sol…
New algorithm learns sparse GLMs for binary outcomes efficiently.
In this paper, we propose an effective THresholding method based on ORder Statistic, called THORS, to convert an arbitrary scoring-type classifier, which can induce a continuous cumulative distribution function of the score, into a cost-sensitive one. The procedure, uses order statistic to find an optimal threshold for…
The paper sets thresholds for testing correlation in hypergraphs, distinguishing between independent and correlated states.
Under Markovian assumptions, we leverage a Central Limit Theorem (CLT) for the empirical measure in the test statistic of the composite hypothesis Hoeffding test so as to establish weak convergence results for the test statistic, and, thereby, derive a new estimator for the threshold needed by the test. We first show t…
QAOA matches classical tensor power iteration in spiked tensor model recovery.
In this paper, we investigate a multivariate multi-response (MVMR) linear regression problem, which contains multiple linear regression models with differently distributed design matrices, and different regression and output vectors. The goal is to recover the support union of all regression vectors using -reg…
Statistical-computational gap found in aligning multiple Gaussian graphs.
New robust method for high-dimensional data analysis in imaging studies.
Paper explores limits of high-order clustering with planted structures.
Proposes a method to choose thresholds for LLM evaluation metrics.
Traffic prediction plays a vital role in efficient planning and usage of network resources in wireless networks. While traffic prediction in wired networks is an established field, there is a lack of research on the analysis of traffic in cellular networks, especially in a content-blind manner at the user level. Here, …
A central problem of random matrix theory is to understand the eigenvalues of spiked random matrix models, in which a prominent eigenvector is planted into a random matrix. These distributions form natural statistical models for principal component analysis (PCA) problems throughout the sciences. Baik, Ben Arous and Pé…
New insights into binary perceptron reveal phase transitions and algorithmic thresholds.
A central problem of random matrix theory is to understand the eigenvalues of spiked random matrix models, introduced by Johnstone, in which a prominent eigenvector (or "spike") is planted into a random matrix. These distributions form natural statistical models for principal component analysis (PCA) problems throughou…
Develops a statistical test for IV, improving feature selection reliability.
Paper develops algorithms to maximize AUC in imbalanced classification.
Paper explores fair classification with bounded disparity using finite datasets.
Variable selection in linear models plays a pivotal role in modern statistics. Hard-thresholding methods such as regularization are theoretically ideal but computationally infeasible. In this paper, we propose a new approach, called the LAGS, short for "least absulute gradient selector", to this challenging yet i…
The assumption that the values of model parameters are known or correctly learned, i.e., the Nishimori condition, is one of the requirements for the detectability analysis of the stochastic block model in statistical inference. In practice, however, there is no example demonstrating that we can know the model parameter…
This review summarizes five Lasso optimization algorithms.
New algorithm STCV improves sparse model discovery from normalised data.
A simple thresholding technique improves graph selection in neural connectivity studies.
Existing nonconvex statistical optimization theory and methods crucially rely on the correct specification of the underlying "true" statistical models. To address this issue, we take a first step towards taming model misspecification by studying the high-dimensional sparse phase retrieval problem with misspecified link…
New algorithm resists contamination in high-dimensional regression with optimal performance.
L-ARC improves model fairness by localizing risk guarantees.
LLMs fail to match statistical ground truth despite stable run-to-run performance.
This work suggests modifications to a previously introduced class of heterogeneous agent models that allow for the inclusion of different types of agent motivations and behaviours in a unified way. The agents operate within a highly simplified environment where they are only able to be long or short one unit of the ass…
The analysis of Belief Propagation and other algorithms for the {\em reconstruction problem} plays a key role in the analysis of community detection in inference on graphs, phylogenetic reconstruction in bioinformatics, and the cavity method in statistical physics. We prove a conjecture of Evans, Kenyon, Peres, and Sch…
Complex performance measures, beyond the popular measure of accuracy, are increasingly being used in the context of binary classification. These complex performance measures are typically not even decomposable, that is, the loss evaluated on a batch of samples cannot typically be expressed as a sum or average of losses…
MTSSL optimizes threshold τ for better semi-supervised learning performance.
Study examines local extrema and crossing statistics in financial markets.
Study binary perceptrons' capacity using random duality theory.
We analyse the learning performance of Distributed Gradient Descent in the context of multi-agent decentralised non-parametric regression with the square loss function when i.i.d. samples are assigned to agents. We show that if agents hold sufficiently many samples with respect to the network size, then Distributed Gra…
Develops RES metrics for stable rare-event forecasting evaluation.
Sampling is a fundamental problem in computer science and statistics. However, for a given task and stream, it is often not possible to choose good sampling probabilities in advance. We derive a general framework for adaptively changing the sampling probabilities via a collection of thresholds.In general, adaptive samp…