This paper shows how to combine optimal tests into log-optimal processes.
problem How to combine optimal sequential tests into log-optimal processes.
method Using a new class of WAIT e-processes, the paper aggregates asymptotically optimal sequential tests into asymptotically log-optimal processes.
result It is possible to aggregate asymptotically optimal sequential tests into asymptotically log-optimal e-processes.
Optimizes two-sample tests for non-Euclidean domains using spectral regularization.
problem Optimizing two-sample tests for non-Euclidean domains.
method Spectral regularization of MMD test to achieve minimax optimality.
result Proposes a spectral regularization method that improves test optimality.
New Bayesian optimization models for efficient material screening.
problem Efficiently screening materials with expensive and cheap tests.
method Flexible multi-test Bayesian optimization models with complex relationships.
result Demonstrated power on synthetic and real data.
GTBO uses group testing to optimize high-dimensional functions efficiently.
problem Challenges in optimizing high-dimensional, expensive functions due to the curse of dimensionality.
method GTBO combines testing and optimization phases to identify active variables and guide efficient optimization.
result GTBO outperforms state-of-the-art methods on high-dimensional optimization tasks.
Optimizes group testing for COVID-19 to reduce test numbers.
problem Minimizing tests for accurate infection detection.
method Bayesian approach with genetic algorithms and sub-modularity.
result Greedy-adaptive method provides theoretical guarantees.
Unified framework for structure learning via conditional independence testing.
problem Optimal structure learning and conditional independence testing.
method Established a fundamental connection and reduction between structure learning and conditional independence testing.
result Optimal rates for structure learning are determined by conditional independence testing rates.
Paper develops a minimax optimal test for goodness-of-fit using kernel Stein discrepancy.
problem Developing a robust goodness-of-fit test for general domains.
method Kernel Stein Discrepancy (KSD) with spectral regularization and adaptive testing.
result Proposed regularized test achieves minimax optimality up to a logarithmic factor.
We characterize the asymptotic performance of nonparametric one- and two-sample testing. The exponential decay rate or error exponent of the type-II error probability is used as the asymptotic performance metric, and an optimal test achieves the maximum rate subject to a constant level constraint on the type-I error pr…
Unified framework for optimal kernel tests across MMD, HSIC, and KSD.
problem Optimal testing in kernel-based hypothesis testing frameworks.
method Unified derivation of minimax rates, adaptive kernel selection methods.
result Unified power results across MMD, HSIC, and KSD.
A new test for comparing distributions is proposed, balancing optimality and efficiency.
problem Comparing distributions defined over non-Euclidean domains.
method Spectral-regularized two-sample test based on random Fourier features.
result The proposed test is minimax optimal under certain conditions.
Robust test for distributions under Hellinger distance, simpler than optimal tests.
problem Testing and estimating distributions robustly under Hellinger distance.
method Simple robust hypothesis test with optimal sample complexity, robust to Hellinger distance perturbations.
result Empirically demonstrated robustness and power of the test on canonical distributions.
Optimal sequential testing for Markovian data with lower and upper bounds.
problem Sequential hypothesis testing for Markovian data.
method Non-asymptotic lower bounds and optimal test design.
result Optimal test matches lower bound asymptotically.
A permutation-based SW test achieves minimax-optimal power for two-sample testing.
problem Nonparametric two-sample testing using the sliced Wasserstein distance.
method Proposes a permutation-based SW test and analyzes its performance.
result Achieves minimax separation rate n−1/2 over multinomial and bounded-support alternatives. Optimal tests developed for sequential experiments with asymptotic properties.
problem Performing hypothesis tests after sequential experiments without prior design.
method Analyze asymptotic properties of sequential experiments; develop tests for Gaussian process observations.
result Asymptotic power function of any test can be matched by a specific test in a limit experiment.
Optimal testing of discrete distributions with high probability, achieving sample complexity bounds.
problem Testing discrete distributions with high probability accuracy.
method Characterizing sample complexity as a function of parameters like δ, providing sample-optimal testers.
result Optimal algorithms for closeness and independence testing, achieving within constant factors of information-theoretic lower bounds.
Cheap permutation tests speed up distribution testing without sacrificing accuracy.
problem Efficiently testing distribution differences and independence.
method Group datapoints into bins and permute only these bins, using stored sufficient statistics.
result Cheap permutation tests maintain the accuracy and optimality of standard tests but are significantly faster.
Framework tests group fairness in machine learning models.
problem Detecting biases in machine learning classifiers.
method Optimal transport projections to audit group fairness.
result Statistical test for various fairness notions efficiently computed.
Tent adapts models during testing by minimizing entropy of predictions.
problem Adapting models to new data during testing with limited information.
method Test entropy minimization (tent) and online channel-wise affine transformations.
result Reduces generalization error on various datasets and benchmarks.
Adversarial training achieves optimal test error for shallow networks.
problem Achieving optimal adversarial test error for general data distributions.
method Applying new Rademacher complexity bounds and properties of optimal adversarial predictors.
result Adversarial training can achieve optimal adversarial test error for general data distributions.
We characterize the asymptotic performance of nonparametric goodness of fit testing. The exponential decay rate of the type-II error probability is used as the asymptotic performance metric, and a test is optimal if it achieves the maximum rate subject to a constant level constraint on the type-I error probability. We …
We study "active" decision making over sensor networks where the sensors' sequential probing actions are actively chosen by continuously learning from past observations. We consider two network settings: with and without central coordination. In the first case, the network nodes interact with each other through a centr…
Early stopping of iterative algorithms is an algorithmic regularization method to avoid over-fitting in estimation and classification. In this paper, we show that early stopping can also be applied to obtain the minimax optimal testing in a general non-parametric setup. Specifically, a Wald-type test statistic is obtai…
Optimizes test set size for accurate diagnosis using machine learning.
problem Determining the minimum test set size for accurate diagnosis.
method Proposes machine learning methods (LASSO and SVM) to predict optimal test set size.
result SVM achieves 90.4% accuracy with a reduced test set by 35.24%.
Near-optimal private tests for simple and MLR hypotheses developed under Gaussian differential privacy.
problem Developing private tests for simple and MLR hypotheses under Gaussian differential privacy.
method A private mean estimator with data-driven clamping bounds, constructing private test statistics.
result Private tests achieve the same asymptotic relative efficiency as non-private most powerful tests.
Robust tests control type I error under data corruption.
problem Effective hypothesis testing under data corruption.
method General permutation tests using kernel MMD and HSIC metrics.
result Robust tests are minimax optimal and outperform private tests.
GTBO uses group testing to optimize high-dimensional functions efficiently.
problem Optimizing expensive, high-dimensional functions with limited data.
method Group testing to identify active dimensions, then guide optimization.
result GTBO outperforms state-of-the-art methods on high-dimensional benchmarks.
Two-sample tests improve on existing methods for microtubule data.
problem Testing differences between two groups of filament data.
method Optimal lifts and manifold stability theorem applied to microtubule data.
result New tests outperform existing methods on simulated and real data.
Entropy regularized OT test assesses independence between samples.
problem Testing independence between two samples.
method Entropy regularized optimal transport.
result Non-asymptotic bounds for test statistic established.
Optimal AFs minimize RFR test error and sensitivity.
problem Finding optimal AFs for RFR to minimize test error and sensitivity.
method Closed-form solution for AFs minimizing test error and sensitivity under different functional parsimony.
result Optimal AFs can be linear, saturated linear, or Hermite polynomial expressions.
A simple algorithm for Gaussian mean testing with optimal sample complexity.
problem Distinguishing between standard Gaussian and other Gaussian distributions with unknown mean and covariance.
method An extremely simple algorithm with a one-page analysis.
result Optimal sample complexity of Θ(√d/ε^2) with sample linear time.
Study binary hypothesis testing with privacy and communication constraints.
problem Binary hypothesis testing under local differential privacy and communication constraints.
method Qualifies results as minimax or instance optimal, develops instance-optimal algorithms.
result Achieves minimum possible sample complexity under both privacy and communication constraints.
Real-world machine learning applications often have complex test metrics, and may have training and test data that are not identically distributed. Motivated by known connections between complex test metrics and cost-weighted learning, we propose addressing these issues by using a weighted loss function with a standard…
Optimizing news headlines is important for publishers and media sites. A compelling headline will increase readership, user engagement and social shares. At Yahoo Front Page, headline testing is carried out using a test-rollout strategy: we first allocate equal proportion of the traffic to each headline variation for a…
Hypothesis testing is one of the most common types of data analysis and forms the backbone of scientific research in many disciplines. Analysis of variance (ANOVA) in particular is used to detect dependence between a categorical and a numerical variable. Here we show how one can carry out this hypothesis test under the…
The paper develops a method to learn cost-optimal sequential testing policies from retrospective data.
problem Learning cost-optimal sequential decision policies from retrospective data with missing test results.
method Doubly robust Q-learning framework with path-specific inverse probability weights.
result The method reduces testing cost without compromising predictive accuracy.
New scaling laws optimize model size, training, and inference for better performance.
problem Trade-off between model size and inference cost in modern LLMs.
method Train-to-Test (T2) scaling laws that jointly optimize model size, training tokens, and inference samples. result Optimal pretraining decisions shift into overtraining regime, leading to stronger performance.
Algorithm predicts with optimal loss by abstaining from uncertain test examples.
problem Predicting with training data not matching test data.
method Transductive abstention algorithm using labeled and unlabeled test examples.
result Optimal prediction loss guarantees with additional term for abstaining cost.
Nonparametric tests via kernel embedding of distributions have witnessed a great deal of practical successes in recent years. However, statistical properties of these tests are largely unknown beyond consistency against a fixed alternative. To fill in this void, we study here the asymptotic properties of goodness-of-fi…
M-L2O adapts fast to new tasks by self-adapting during test-time.
problem L2O optimizers struggle with out-of-distribution tasks.
method Meta-training an L2O optimizer to adapt quickly to new tasks.
result M-L2O converges significantly faster than vanilla L2O with only 5 steps of adaptation.
A new method optimizes MMD test power by dynamically selecting kernels, overcoming traditional trade-offs.
problem Fixed kernels fail to distinguish certain distributions, leading to overfitting and variance collapse.
method Complexity-Penalized MMD (CP-MMD) criterion, derived from concentration inequality, optimizes kernel selection.
result CP-MMD maximizes true test power while ensuring unconditional Type-I validity, matching or exceeding state-of-the-art performance.
Optimized testing of discrete distributions using predicted data.
problem Testing discrete distributions with reduced sample complexity.
method Adaptable algorithms that use a predicted distribution to reduce sample size.
result Optimal sample complexity improvements with self-adjusting algorithms.
Paper extends multivariate rank tests for robust subspace detection.
problem Testing distributional similarity in multivariate data.
method Soft and subspace robust multivariate rank tests based on entropy regularized optimal transport.
result Trade-off between detection power and false alarm rate via projections.
New algorithm improves on static methods in Active Simple Hypothesis Testing.
problem Optimizing Active Simple Hypothesis Testing with active sampling.
method Game-theoretic formulation, differential games, PDEs, Blackwell Approachability.
result Proposes an efficient algorithm that outperforms static methods in ASHT.
Optimal tests for composite nulls achieve the KL inf lower bound.
problem Designing optimal tests for composite null hypotheses.
method Constructive schemes based on universal e-processes.
result Optimal tests match the KL inf lower bound as α → 0.
Sequential tests for nonparametric hypotheses using supermartingales.
problem Designing valid sequential tests for nonparametric null hypotheses.
method Using elicitable and identifiable functionals, nonnegative supermartingales, and Online Convex Optimization.
result Rigorous guarantees on asymptotic power for a wide range of alternative hypotheses.
Bayes-optimal learning of deep random networks with Gaussian weights is studied.
problem Learning a target function corresponding to a deep, extensive-width, non-linear neural network with random Gaussian weights.
method Closed-form expressions for Bayes-optimal test error, ridge regression, kernel and random features regression are computed.
result Optimally regularized ridge regression and kernel regression achieve Bayes-optimal performances, while logistic loss yields a near-optimal test error for classification.
Optimal distributed testing under communication constraints with shared randomness.
problem Signal detection in a distributed system with limited communication.
method Derivation of minimax testing errors, distributed testing algorithms, and theoretical lower bounds.
result Consistent nonparametric distributed testing is possible even with minimal communication.
This paper surveys some recent developments in fundamental limits and optimal algorithms for network analysis. We focus on minimax optimal rates in three fundamental problems of network analysis: graphon estimation, community detection, and hypothesis testing. For each problem, we review state-of-the-art results in the…