We formulate statistical watermarking as hypothesis testing and establish near-optimal bounds.
problem Statistical watermarking in the context of hypothesis testing.
method Formulated as a hypothesis testing problem, using coupling of output tokens and rejection regions.
result Established nearly matching upper and lower bounds on the number of i.i.d. tokens required for small Type I and Type II errors.
Paper presents robust boosting methods for label noise.
problem Boosting methods degrade in noisy environments.
method Robust Minimax Boosting (RMBoost) with theoretical guarantees.
result RMBoost provides strong classification accuracy and robustness.
New algorithm controls type I error in NP classification under label noise.
problem Label noise affects NP classification methods, reducing power.
method Proposes a label-noise-adjusted Neyman-Pearson algorithm.
result Improves power while controlling type I error under desired level.
Deep neural networks outperform other methods in estimating non-smooth functions with singularities.
problem Estimating functions with singularities on hypersurfaces.
method Developed a minimax rate analysis for DNNs, proving their almost optimal convergence rate.
result DNNs outperform other estimators in estimating non-smooth functions with singularities.
Develops a hypothesis testing framework for generalized Thurstone models.
problem Determining whether pairwise comparison data fits a generalized Thurstone model.
method Introduces separation distance and derives upper and lower bounds for testing.
result Critical threshold for testing depends on observation graph topology and scales as Θ((nk)−1/2) for complete graphs. New method for multiclass classification reduces error bounds.
problem High-dimensional multiclass classification with sparse coefficients.
method Sparse multinomial logistic regression with convex penalties.
result Plug-in classifiers achieve minimax generalization error bounds.
Quantile regression with ReLU networks achieves minimax rates for various function types.
problem Estimating quantiles from covariates with neural networks.
method Quantile regression with rectified linear unit (ReLU) neural networks.
result ReLU networks achieve minimax rates for broad collections of function types.
Convolutional neural networks (CNNs) have been shown to achieve optimal approximation and estimation error rates (in minimax sense) in several function classes. However, previous analyzed optimal CNNs are unrealistically wide and difficult to obtain via optimization due to sparse constraints in important function class…
Combines cost-sensitive and Neyman-Pearson paradigms for better binary classification.
problem Asymmetric binary classification problems with unequal error severities.
method Develops TUBE-CS algorithm to bridge cost-sensitive and Neyman-Pearson paradigms.
result High-probability control of population type I error.
Despite the great success of deep neural networks, the adversarial attack can cheat some well-trained classifiers by small permutations. In this paper, we propose another type of adversarial attack that can cheat classifiers by significant changes. For example, we can significantly change a face but well-trained neural…
Paper optimizes clustering for multi-layer networks and discrete mixtures.
problem Optimizing clustering in multi-layer networks and discrete mixtures.
method Two-stage method: tensor-based initialization and likelihood-based refinement.
result Achieves minimax optimal error rate for multi-layer networks and discrete mixtures.
Paper improves risk bounds for nonconvex-strongly-concave minimax problems.
problem Achieving sharper risk bounds for nonconvex-strongly-concave minimax problems.
method Using uniform localized convergence to derive high probability generalization error bounds.
result Derives n times faster excess primal risk bounds for popular algorithms.
The Neyman-Pearson (NP) paradigm in binary classification seeks classifiers that achieve a minimal type II error while enforcing the prioritized type I error controlled under some user-specified level α. This paradigm serves naturally in applications such as severe disease diagnosis and spam detection, where people h…
Motivated by problems of anomaly detection, this paper implements the Neyman-Pearson paradigm to deal with asymmetric errors in binary classification with a convex loss. Given a finite collection of classifiers, we combine them and obtain a new classifier that satisfies simultaneously the two following properties with …
SONAR improves outlier detection for streaming data with strong theoretical guarantees.
problem Outlier detection for non-stationary streaming data with high Type I/II errors.
method SONAR is an efficient SGD-based OCSVM solver with strong convex regularization and lifelong learning guarantees.
result SONAR outperforms traditional OCSVM in Type I/II error rates under non-stationary data.
We find the optimal error for a constrained regression model under a linear model.
problem Minimizing error while adhering to demographic parity constraints.
method Proposed a minimax optimal error analysis for a demographic parity-constrained regression problem within a linear model.
result The minimax optimal error is characterized by $Θ( rac{dM}{n})$.
Most existing binary classification methods target on the optimization of the overall classification risk and may fail to serve some real-world applications such as cancer diagnosis, where users are more concerned with the risk of misclassifying one specific class than the other. Neyman-Pearson (NP) paradigm was introd…
New methods estimate transport-growth pairs in unbalanced optimal transport.
problem Statistical guarantees for Monge-type estimation in unbalanced optimal transport remain limited.
method Developed two estimators for transport-growth pairs under different setups.
result Achieved minimax optimal rate for estimation of transport-growth pairs.
FactTest assesses LLM factuality with Type I error control.
problem Lack of rigorous factuality verification for LLMs.
method Formulates factuality testing as hypothesis testing, ensuring Type I and II error control.
result Improves model accuracy by over 40% in abstaining from unknown questions.
Adapts RKHS methods to estimate density ratios with optimal error.
problem Estimating density ratios from limited data.
method Minimizes regularized Bregman divergence in RKHS, with Lepskii type parameter choice.
result Adaptive minimax optimal error rate for quadratic loss.
Private two-sample tests under LDP achieve minimax rates for multinomial and continuous data.
problem Achieving statistical utility while maintaining privacy in two-sample testing.
method Private permutation tests for multinomial data and adaptive tests for continuous data.
result Minimax optimal tests for private two-sample testing under LDP.
Least Squares Estimators are suboptimal for 5D convex functions.
problem Suboptimality of Least Squares Estimators in estimating multidimensional convex functions.
method Analysis of natural subclasses of convex functions in random and fixed design settings.
result Risk of LSE is n−2/d while minimax risk is n−4/(d+4) for d≥5. Simple proof shows forecasts can be calibrated in a few periods.
problem Ensuring forecasts are calibrated over multiple periods.
method Uses minimax theorem to prove existence and guarantees calibration error.
result Calibration can be achieved in N3 periods with error at most 1/N. Optimal classification rules control error rates in multiclass mixture models.
problem Classifying observations in multiclass mixture models while controlling error rates.
method Finding optimal classification rules by searching an optimal region in the observation space, using Maximum A Posteriori (MAP) rule and heuristic computation.
result The FDR-like optimal rule can be significantly less conservative than thresholded MAP rules.
New fairness concept extends minimax fairness to lexicographic fairness.
problem Fairness in supervised learning, especially lexicographic fairness.
method Introduced approximate lexifairness, derived algorithms for finding solutions, and proved generalization bounds.
result Proved that approximate lexifairness on training data implies approximate lexifairness on true distribution.
DP synthetic data may inflate statistical test results, caution advised.
problem Inflated Type I errors in statistical tests on DP-synthetic data.
method Evaluation of Mann-Whitney U test, t-test, chi-squared test, and median test on DP-synthetic data generated from real-world and simulated datasets using various DP-synthetic data generation methods.
result A large portion of evaluation results showed inflated Type I errors, especially at low privacy levels.
Study minimax linear regression under quantile risk, improving existing bounds and providing new results.
problem Designing minimax procedures in linear regression under quantile risk.
method Analyzes realizable setting with Gaussian noise, extends to all p-th power error functions, develops new lower and upper bounds.
result Proves minimaxity of a variant of the min-max regression procedure for all p-th power error functions.
We consider semi-supervised regression when the predictor variables are drawn from an unknown manifold. A simple two step approach to this problem is to: (i) estimate the manifold geodesic distance between any pair of points using both the labeled and unlabeled instances; and (ii) apply a k nearest neighbor regressor b…
New adaptive test for NPIV models controls size and has superior power.
problem Testing inequality and equality restrictions in nonparametric IV models.
method Adaptive hypothesis test based on modified leave-one-out sample quadratic distance.
result Adaptive test attains the adaptive minimax rate of testing in L2. DP-SPRT improves privacy in sequential tests with near-optimal error rates.
problem Privacy constraints in sequential probability ratio tests.
method A wrapper for SPRT that uses a private mechanism to determine when to stop based on predefined intervals.
result DP-SPRT achieves near-optimal error rates and privacy guarantees.
Gaussian graphical model is a graphical representation of the dependence structure for a Gaussian random vector. It is recognized as a powerful tool in different applied fields such as bioinformatics, error-control codes, speech language, information retrieval and others. Gaussian graphical model selection is a statist…
Robust tests control type I error under data corruption.
problem Effective hypothesis testing under data corruption.
method General permutation tests using kernel MMD and HSIC metrics.
result Robust tests are minimax optimal and outperform private tests.
In regression settings where explanatory variables have very low correlations and there are relatively few effects, each of large magnitude, we expect the Lasso to find the important variables with few errors, if any. This paper shows that in a regime of linear sparsity---meaning that the fraction of variables with a n…
We characterize the asymptotic performance of nonparametric one- and two-sample testing. The exponential decay rate or error exponent of the type-II error probability is used as the asymptotic performance metric, and an optimal test achieves the maximum rate subject to a constant level constraint on the type-I error pr…
Study detects signals in spiked Wigner models using log likelihood ratio.
problem Detecting signals in rank-one spiked Wigner models with non-Gaussian noise.
method Proved asymptotic normality of log likelihood ratio and computed error thresholds.
result Optimal signal-to-noise ratio threshold for reliable detection.
Paper explores generalization of minimax learners, proposing a new metric.
problem Understanding how minimax learners perform on unseen data.
method Proposes a new metric, the primal gap, to study generalization of minimax learners.
result Derives generalization error bounds for the primal gap in nonconvex-concave settings.
New algorithms solve nonconvex-nonconcave minimax optimization problems.
problem Solving minimax optimization problems in machine learning.
method Two novel Newton-type algorithms for nonconvex-nonconcave minimax optimization.
result Proved local convergence at strict local minimax points.
Formula derived for sample complexity in binary hypothesis testing.
problem Determine the minimum number of samples to distinguish between two distributions.
method Developed a formula for sample complexity in both prior-free and Bayesian settings, using Jensen-Shannon and Hellinger divergences.
result Formula characterizes sample complexity for a wide range of error parameters, up to multiplicative constants.
The paper develops a minimax optimal method for high-dimensional regression using auxiliary data.
problem High-dimensional additive regression with heavy-tailed errors and transfer learning.
method Smooth backfitting estimator with local linear smoothing, followed by a two-stage estimation method.
result The method achieves the minimax optimal rate under certain conditions.
Wide residual networks generalize well with uniform convergence to RNTK as width increases.
problem Understanding the generalization ability of wide residual networks.
method Uniform convergence of residual network kernel to residual neural tangent kernel (RNTK).
result Generalization error converges to kernel regression error with respect to RNTK.
We develop tools for utilizing correspondence experiments to detect illegal discrimination by individual employers. Employers violate US employment law if their propensity to contact applicants depends on protected characteristics such as race or sex. We establish identification of higher moments of the causal effects …
New tensor model reduces GLM estimation error and sample complexity.
problem Estimating GLM coefficients with reduced sample complexity.
method Developed LSR tensor model and block coordinate descent algorithm.
result Minimax lower bound on estimation error, suggesting lower sample complexity.
Study analyzes convergence of parameter estimation in contaminated mixture of experts.
problem Challenges in learning from prompts in large-scale models.
method Convergence analysis, distinguishability condition, partial differential equations.
result Comprehensive convergence rates and minimax lower bounds for parameter estimation.
Given n samples from a population of individuals belonging to different types with unknown proportions, how do we estimate the probability of discovering a new type at the (n+1)-th draw? This is a classical problem in statistics, commonly referred to as the missing mass estimation problem. Recent results by Ohannes…
New method uses CDMs to improve CI testing without distributional assumptions.
problem Testing conditional independence when the conditional distribution is unknown.
method Uses conditional diffusion models (CDMs) to approximate X∣Z and a classifier-based CMI estimator. result Proposed method performs better than GAN-based CI tests and controls type I and II errors.
The paper optimizes k-NN for distributed learning with minimax optimal performance.
problem Minimizing error rates in classification, regression, and density estimation.
method Optimal aggregation of fixed-k nearest neighbors from multiple subsets of data.
result Achieves minimax optimal error rates up to a logarithmic factor.
A common issue for classification in scientific research and industry is the existence of imbalanced classes. When sample sizes of different classes are imbalanced in training data, naively implementing a classification method often leads to unsatisfactory prediction results on test data. Multiple resampling techniques…
WHOMP optimizes randomized controlled trials by minimizing subgroup bias.
problem Minimizing subgroup bias in randomized controlled trials.
method Wasserstein Homogeneity Partition (WHOMP) method.
result WHOMP optimally minimizes type I and type II errors in trials.