arXiv research

New algorithm controls type I error in NP classification under label noise.

We formulate statistical watermarking as hypothesis testing and establish near-optimal bounds.

Combines cost-sensitive and Neyman-Pearson paradigms for better binary classification.

Despite the great success of deep neural networks, the adversarial attack can cheat some well-trained classifiers by small permutations. In this paper, we propose another type of adversarial attack that can cheat classifiers by significant changes. For example, we can significantly change a face but well-trained neural…

The Neyman-Pearson (NP) paradigm in binary classification seeks classifiers that achieve a minimal type II error while enforcing the prioritized type I error controlled under some user-specified level $α$. This paradigm serves naturally in applications such as severe disease diagnosis and spam detection, where people h…

Motivated by problems of anomaly detection, this paper implements the Neyman-Pearson paradigm to deal with asymmetric errors in binary classification with a convex loss. Given a finite collection of classifiers, we combine them and obtain a new classifier that satisfies simultaneously the two following properties with …

SONAR improves outlier detection for streaming data with strong theoretical guarantees.

Most existing binary classification methods target on the optimization of the overall classification risk and may fail to serve some real-world applications such as cancer diagnosis, where users are more concerned with the risk of misclassifying one specific class than the other. Neyman-Pearson (NP) paradigm was introd…

FactTest assesses LLM factuality with Type I error control.

Optimal classification rules control error rates in multiclass mixture models.

DP synthetic data may inflate statistical test results, caution advised.

DP-SPRT improves privacy in sequential tests with near-optimal error rates.

Gaussian graphical model is a graphical representation of the dependence structure for a Gaussian random vector. It is recognized as a powerful tool in different applied fields such as bioinformatics, error-control codes, speech language, information retrieval and others. Gaussian graphical model selection is a statist…

In regression settings where explanatory variables have very low correlations and there are relatively few effects, each of large magnitude, we expect the Lasso to find the important variables with few errors, if any. This paper shows that in a regime of linear sparsity---meaning that the fraction of variables with a n…

We characterize the asymptotic performance of nonparametric one- and two-sample testing. The exponential decay rate or error exponent of the type-II error probability is used as the asymptotic performance metric, and an optimal test achieves the maximum rate subject to a constant level constraint on the type-I error pr…

Study detects signals in spiked Wigner models using log likelihood ratio.

Formula derived for sample complexity in binary hypothesis testing.

New method uses CDMs to improve CI testing without distributional assumptions.

A common issue for classification in scientific research and industry is the existence of imbalanced classes. When sample sizes of different classes are imbalanced in training data, naively implementing a classification method often leads to unsatisfactory prediction results on test data. Multiple resampling techniques…

WHOMP optimizes randomized controlled trials by minimizing subgroup bias.

The paper tackles data misappropriation in LLMs by embedding watermarks and testing for their presence.

Efficient tests achieve best error rates in high-dimensional hypothesis testing.

Improved generalization bounds for SGD in non-convex learning.

New metrics boost A/B-test power by up to 210%.

This paper addresses the challenges in classifying textual data obtained from open online platforms, which are vulnerable to distortion. Most existing classification methods minimize the overall classification error and may yield an undesirably large type I error (relevant textual messages are classified as irrelevant)…

Kernel tests assess equivalence between distributions without assuming specific moments.

Identifying statistical dependence between the features and the label is a fundamental problem in supervised learning. This paper presents a framework for estimating dependence between numerical features and a categorical label using generalized Gini distance, an energy distance in reproducing kernel Hilbert spaces (RK…

New methods improve inference after prediction without strong model assumptions.

Strict type-II blowup in harmonic map flow is proven to have Hölder continuous body map.

We construct new type II ancient compact solutions to the Yamabe flow. Our solutions are rotationally symmetric and converge, as $t \to -\infty$, to a tower of two spheres. Their curvature operator changes sign. We allow two time-dependent parameters in our ansatz. We use perturbation theory, via fixed point arguments,…

Paper proves uniqueness of Type II Yamabe metrics on manifolds.

We propose a nonparametric sequential test that aims to address two practical problems pertinent to online randomized experiments: (i) how to do a hypothesis test for complex metrics; (ii) how to prevent type $1$ error inflation under continuous monitoring. The proposed test does not require knowledge of the underlying…

Adapts Neyman-Pearson classification for both source and target distribution shifts.

In this paper, we study stability and instability problem for type-II partitioning problem. First, we make a complete classification of stable type-II stationary hypersurfaces in a ball in a space form as totally geodesic $n$-balls. Second, for general ambient spaces and convex domains, we give some topological restric…

We consider the weak detection problem in a rank-one spiked Wigner data matrix where the signal-to-noise ratio is small so that reliable detection is impossible. We propose a hypothesis test on the presence of the signal by utilizing the linear spectral statistics of the data matrix. The test is data-driven and does no…

Sharp 2-Wasserstein bounds for DDPMs derived from Föllmer process.

Paper introduces SCI to distinguish market signals from coordination.

The paper proves rigidity theorems for Type II singularities in Lagrangian flows.

New methods improve anomaly detection with reduced false positives.

Type II (ancient) solutions to the Ricci flow on surfaces are not yet classified. It is conjectured that the Rosenau solution and the cigar are the only solutions, modulo scaling. In this paper, we mainly study the backward limit and the circumference at spatial infinity of Type II ancient solutions on noncompact surfa…

In this paper, we propose a generalized scale mixture family of distributions, namely the Power Exponential Scale Mixture (PESM) family, to model the sparsity inducing priors currently in use for sparse signal recovery (SSR). We show that the successful and popular methods such as LASSO, Reweighted $\ell_1$ and Reweigh…

We study the statistical decision process of detecting the signal from a `signal+noise' type matrix model with an additive Wigner noise. We propose a hypothesis test based on the linear spectral statistics of the data matrix, which does not depend on the distribution of the signal or the noise. The test is optimal unde…

The paper develops a method to identify LLM-generated text without training.

New tests for binary classification regression functions without distribution assumptions.

Numerical simulations show stability of Type-II singularities in noncompact hypersurfaces.

New test for conditional independence using kernel embeddings.

We introduce an up-down coloring of a virtual-link diagram. The colorabilities give a lower bound of the minimum number of Reidemeister moves of type II which are needed between two 2-component virtual-link diagrams. By using the notion of a quandle cocycle invariant, we determine the necessity of Reidemeister moves of…

Kähler-Ricci flow shows type II singularity on Fano threefolds.