arXiv research

Investigates probability of error in structured thresholding bandit problems.

We investigate the impact of available information on the estimation of the default probability within a generalized structural model for credit risk. The traditional structural model where default is triggered when the value of the firm's asset falls below a constant threshold is extended by relaxing the assumption of…

The fundamental group of the $2$-dimensional Linial-Meshulam random simplicial complex $Y_2(n,p)$ was first studied by Babson, Hoffman and Kahle. They proved that the threshold probability for simple connectivity of $Y_2(n,p)$ is about $p\approx n^{-1/2}$. In this paper, we show that this threshold probability is at mo…

Develops RES metrics for stable rare-event forecasting evaluation.

Sampling is a fundamental problem in computer science and statistics. However, for a given task and stream, it is often not possible to choose good sampling probabilities in advance. We derive a general framework for adaptively changing the sampling probabilities via a collection of thresholds.In general, adaptive samp…

We investigate the probability distributions of the recurrence intervals $τ$ between consecutive 1-min returns above a positive threshold $q>0$ or below a negative threshold $q<0$ of two indices and 20 individual stocks in China's stock market. The distributions of recurrence intervals for positive and negative thresho…

We define the information threshold in Bayesian decision-making.

New method calculates sensitivity of system failure probability.

We present a framework and analysis of consistent binary classification for complex and non-decomposable performance metrics such as the F-measure and the Jaccard measure. The proposed framework is general, as it applies to both batch and online learning, and to both linear and non-linear models. Our work follows recen…

Bayesian framework proves thresholds for multi-graph alignment feasibility.

Square percolation determines threshold for group divergence in random graphs.

Threshold found for hyperbolicity in random Coxeter groups.

The binary symmetric stochastic block model deals with a random graph of $n$ vertices partitioned into two equal-sized clusters, such that each pair of vertices is connected independently with probability $p$ within clusters and $q$ across clusters. In the asymptotic regime of $p=a \log n/n$ and $q=b \log n/n$ for fixe…

The interplay between computational efficiency and statistical accuracy in high-dimensional inference has drawn increasing attention in the literature. In this paper, we study computational and statistical boundaries for submatrix localization. Given one observation of (one or multiple non-overlapping) signal submatrix…

We provide high probability finite sample complexity guarantees for hidden non-parametric structure learning of tree-shaped graphical models, whose hidden and observable nodes are discrete random variables with either finite or countable alphabets. We study a fundamental quantity called the (noisy) information threshol…

The paper develops methods to predict the probability of achieving a user goal in a task, ensuring the system alerts when the probability falls below a threshold.

Majority bit estimation in noisy random recursive DAGs.

Spectral method detects communities in sparse hypergraphs, achieving detection threshold.

New algorithm achieves strong consistency in binary non-uniform hypergraph classification.

To estimate a sparse linear model from data with Gaussian noise, consilience from lasso and compressed sensing literatures is that thresholding estimators like lasso and the Dantzig selector have the ability in some situations to identify with high probability part of the significant covariates asymptotically, and are …

A new machine learning method calculates failure probability efficiently and accurately.

Class imbalance presents a major hurdle in the application of data mining methods. A common practice to deal with it is to create ensembles of classifiers that learn from resampled balanced data. For example, bagged decision trees combined with random undersampling (RUS) or the synthetic minority oversampling technique…

Being able to predict the occurrence of extreme returns is important in financial risk management. Using the distribution of recurrence intervals---the waiting time between consecutive extremes---we show that these extreme returns are predictable on the short term. Examining a range of different types of returns and th…

This project explores several Machine Learning methods to predict movie genres based on plot summaries. Naive Bayes, Word2Vec+XGBoost and Recurrent Neural Networks are used for text classification, while K-binary transformation, rank method and probabilistic classification with learned probability threshold are employe…

The paper provides high-probability bounds on false discovery proportions in conformal inference.

We study the pricing of credit derivatives with asymmetric information. The managers have complete information on the value process of the firm and on the default threshold, while the investors on the market have only partial observations, especially about the default threshold. Different information structures are dis…

In this paper, we study the sensitivity of the spectral clustering based community detection algorithm subject to a Erdos-Renyi type random noise model. We prove phase transitions in community detectability as a function of the external edge connection probability and the noisy edge presence probability under a general…

Polynomial-time algorithm solves random parity games with high probability.

We study confidence intervals based on hard-thresholding, soft-thresholding, and adaptive soft-thresholding in a linear regression model where the number of regressors $k$ may depend on and diverge with sample size $n$. In addition to the case of known error variance, we define and study versions of the estimators when…

Unified framework for pattern recovery in penalized and thresholded estimation.

Fewer degrees of freedom can train deep networks, showing a sharp phase transition.

Threshold tests have recently been proposed as a useful method for detecting bias in lending, hiring, and policing decisions. For example, in the case of credit extensions, these tests aim to estimate the bar for granting loans to white and minority applicants, with a higher inferred threshold for minorities indicative…

CTI produces efficient prediction intervals with guaranteed coverage.

We propose {graphical sure screening}, or GRASS, a very simple and computationally-efficient screening procedure for recovering the structure of a Gaussian graphical model in the high-dimensional setting. The GRASS estimate of the conditional dependence graph is obtained by thresholding the elements of the sample covar…

Finite-time queue peaks in stochastic networks have logarithmic scaling after geometric thresholds.

Azure (the cloud service provided by Microsoft) is composed of physical computing units which are called nodes. These nodes are controlled by a software component called Fabric Controller (FC), which can consider the nodes to be in one of many different states such as Ready, Unhealthy, Booting, etc. Some of these state…

Study proposes active learning method for estimating robust regions in uncertain function evaluations.

Analyzes biased random walks and corrupted intervals in adversarial settings.

Study compares two methods for predicting extreme atmospheric events.

Deepfake detection is formulated as a hypothesis testing problem to classify an image as genuine or GAN-generated. A robust statistics view of GANs is considered to bound the error probability for various GAN implementations in terms of their performance. The bounds are further simplified using a Euclidean approximatio…

Paper finds exact recovery threshold in general hypergraph model.

This paper sets thresholds for recovering vertex correspondences in partially correlated graphs.

Let $M$ be a compact, unit volume, Riemannian manifold with boundary. In this paper we study the homology of a random Čech-complex generated by a homogeneous Poisson process in $M$. Our main results are two asymptotic threshold formulas, an upper threshold above which the Čech complex recovers the $k$-th homology of $M…

Study tests if a probability measure is near a real algebraic variety.

In this research we study a finite horizon optimal purchasing problem for items with a mean reverting price process. Under this model a fixed amount of identical items are bought under a given deadline, with the objective of minimizing the cost of their purchasing price and associated holding cost. We prove that the op…

Paper proposes efficient AL algorithms for optimizing product performance under environmental variability.

Complex performance measures, beyond the popular measure of accuracy, are increasingly being used in the context of binary classification. These complex performance measures are typically not even decomposable, that is, the loss evaluated on a batch of samples cannot typically be expressed as a sum or average of losses…

Study robust estimation under varying corruption probabilities in data.