Robust test for distributions under Hellinger distance, simpler than optimal tests.
problem Testing and estimating distributions robustly under Hellinger distance.
method Simple robust hypothesis test with optimal sample complexity, robust to Hellinger distance perturbations.
result Empirically demonstrated robustness and power of the test on canonical distributions.
Simple methods combine statistical tests for out-of-distribution detection.
problem Detecting data points not following the training distribution.
method Combining classical parametric tests (Rao's score test) and a typicality test.
result Combining Fisher's method of test statistics improves out-of-distribution detection accuracy.
Optimal distributed testing under communication constraints with shared randomness.
problem Signal detection in a distributed system with limited communication.
method Derivation of minimax testing errors, distributed testing algorithms, and theoretical lower bounds.
result Consistent nonparametric distributed testing is possible even with minimal communication.
Study on continuous sequence classification with distribution uncertainty.
problem Classifying continuous sequences with varying distribution uncertainty.
method Proposes distribution-free tests for three test designs: fixed-length, sequential, and two-phase tests.
result Error probabilities decay exponentially fast for all test designs.
We propose a new setting for testing properties of distributions while receiving samples from several distributions, but few samples per distribution. Given samples from s distributions, p1,p2,…,ps, we design testers for the following problems: (1) Uniformity Testing: Testing whether all the pi's are …
Training on mixed distributions improves test performance even when components are unrelated.
problem Improving test performance with mismatched training and test distributions.
method Analyzing mixture distributions with different training and test proportions.
result Distribution shift can be beneficial, improving test performance even when components are unrelated.
Kernel tests assess equivalence between distributions without assuming specific moments.
problem Traditional goodness-of-fit tests fail to detect meaningful distributional differences.
method Proposes kernel-based tests using kernel Stein discrepancy and Maximum Mean Discrepancy.
result Tests assess the absence of meaningful distributional differences under controlled error rates.
Paper tests DPPs for diversity models, distinguishing them from other distributions.
problem Testing whether a given distribution is a Determinantal Point Process (DPP) or far from any DPP.
method Proposes the first algorithm for DPP testing and establishes a lower bound on sample complexity.
result Establishes a matching lower bound on the sample complexity of DPP testing.
Unified framework for global and local two-sample conditional distribution testing.
problem Testing equality of two conditional distributions.
method Distance and kernel methods, conditional U-statistics, local bootstrap.
result Developed reliable global and local tests.
The study examines property testing and estimation under non-identically distributed samples, finding necessary and sufficient sample complexities.
problem Property testing and estimation under non-identically distributed samples.
method Analysis of distributional property testing and estimation in settings with heterogeneous entities.
result Necessary and sufficient sample complexities for property testing and estimation under non-identically distributed samples.
New methods test discrete distributions faster with local privacy constraints.
problem Testing discrete distributions under local differential privacy constraints.
method Efficient randomized algorithms and test procedures, both non-interactive and interactive.
result Faster separation rates in interactive privacy mechanisms.
A method uses neural networks to approximate sampling distributions of test statistics.
problem Accurate modeling of p-value functions or cdfs for correct confidence set coverage.
method Uses neural networks to model the cdf of test statistics, approximating sampling distributions.
result Neural network approximations of sampling distributions are effective and simple.
AutoML simplifies two-sample tests for detecting distribution shifts.
problem Detecting distribution shifts between datasets.
method Uses mean discrepancy of a witness function with squared loss minimization.
result AutoML simplifies and improves two-sample testing performance.
Study robust hypothesis testing under Hellinger distance, proving lower bounds and providing tests.
problem Testing close variants of specified distributions robustly to Hellinger distance.
method Lower bound on slack factor, testing with Hellinger balls, symmetric chi-squared distance analysis.
result Lower bound on slack factor quantifies robustness under misspecification.
Estimates and tests treatment effects on entire outcome distributions.
problem Treatment effects on entire outcome distributions, not just averages.
method Proposes a novel estimand and doubly robust estimator, develops a test.
result First test with provably valid type 1 error guarantees in this setting.
Sequential tests for two-sample and independence testing using betting strategies.
problem Testing sequential data for two-sample and independence without kernel selection issues.
method Prediction-based betting strategies that adaptively determine distribution and joint distribution.
result Prediction-based tests outperform kernel-based approaches in high-dimensional or structured data settings.
Cheap permutation tests speed up distribution testing without sacrificing accuracy.
problem Efficiently testing distribution differences and independence.
method Group datapoints into bins and permute only these bins, using stored sufficient statistics.
result Cheap permutation tests maintain the accuracy and optimality of standard tests but are significantly faster.
Paper proposes a robust hypothesis testing method using Sinkhorn distance.
problem Hypothesis testing for small samples.
method Data-driven approach using Sinkhorn uncertainty sets.
result The method provides a more flexible detector compared to Wasserstein robust test.
New tests for distributional causal effects using improved kernel estimators.
problem Testing for higher-order moments and multidimensional outcomes affected by treatment.
method Improved kernel estimators based on doubly robust mean embeddings.
result New permutation-based tests for distributional causal effects with improved convergence rates.
Develops non-parametric tests for group symmetry in data.
problem Lack of statistical tests for group symmetry in data.
method Formulates and implements non-parametric tests for distributional symmetry under specified groups.
result Develops tests for conditional invariance/equivariance and applies them to real-world data.
We provide a distribution-free test that can be used to determine whether any two joint distributions p and q are statistically different by inspection of a large enough set of samples. Following recent efforts from Long et al. [1], we rely on joint kernel distribution embedding to extend the kernel two-sample test…
We propose a class of nonparametric two-sample tests with a cost linear in the sample size. Two tests are given, both based on an ensemble of distances between analytic functions representing each of the distributions. The first test uses smoothed empirical characteristic functions to represent the distributions, the s…
Paper tackles test-time adaptation for tabular data.
problem Performance degradation due to distribution shifts in testing.
method Proposes FTAT for robustly adapting tabular models during testing.
result FTAT outperforms state-of-the-art methods on benchmark datasets.
We consider the closeness testing problem for discrete distributions. The goal is to distinguish whether two samples are drawn from the same unspecified distribution, or whether their respective distributions are separated in L1-norm. In this paper, we focus on adapting the rate to the shape of the underlying distri…
Two semimetrics on probability distributions are proposed, given as the sum of differences of expectations of analytic functions evaluated at spatial or frequency locations (i.e, features). The features are chosen so as to maximize the distinguishability of the distributions, by optimizing a lower bound on test power f…
CTT compresses samples to test distributions near-linearly, outperforming existing methods.
problem Efficiently testing distributions with high power and near-linear runtime.
method Sample compression followed by permutation testing.
result CTT achieves near-linear runtime while maintaining high statistical power.
A new framework improves kernel Stein discrepancy tests for validating distributions.
problem Improving goodness-of-fit testing for non-normal distributions.
method Introducing Sf-KSD, a unifying framework for studying Stein operators in KSD-based tests.
result Sf-KSD guides the development of new tests and outperforms existing methods.
Optimal testing of discrete distributions with high probability, achieving sample complexity bounds.
problem Testing discrete distributions with high probability accuracy.
method Characterizing sample complexity as a function of parameters like δ, providing sample-optimal testers.
result Optimal algorithms for closeness and independence testing, achieving within constant factors of information-theoretic lower bounds.
New method relaxes TV distance for two-sample testing without distributional assumptions.
problem Challenges in certifying equality or providing tight bounds on TV distance for two distributions.
method Examined blurred total variation distance, a relaxation of TV distance.
result Provided theoretical guarantees for upper and lower bounds on blurred TV distance.
A new test statistic speeds up MMD while maintaining power.
problem Efficiently testing two distributions without permutations.
method Cross-MMD statistic based on sample-splitting and studentization.
result Cross-MMD has a limiting standard Gaussian distribution under the null.
Wide class of elliptically contoured distributions is a popular model of stock returns distribution. However the important question of adequacy of the model is open. There are some results which reject and approve such model. Such results are obtained by testing some properties of elliptical model for each pair of stoc…
New algorithms test independence with fewer samples by using predictive information.
problem Testing independence of distributions with limited samples.
method Augmented distribution testing framework that incorporates predictive information.
result Optimal sample complexity achieved, matching lower bounds.
A new method detects distribution shifts faster than existing CTMs.
problem Detecting distribution shifts in data streams with contamination issues.
method Uses a fixed reference dataset to compare each new sample, avoiding contamination.
result Detects distribution shifts faster and more reliably than standard CTMs.
A probabilistic framework for online test-time adaptation
problem Adapting models to new data under distributional shift
method State-space modelling architecture
result Characterizing parameter learning, time evolution, prior tuning, and prediction
This paper improves test-time adaptation for distribution shifts using confidence maximization and input transformation.
problem Improving deep networks' performance on data shifted from the training distribution.
method Proposes a novel loss function combining confidence maximization and batch-wise entropy maximization with an input transformation module.
result Significantly improves robustness of pretrained networks to corruptions on benchmarks like ImageNet-C.
New framework detects out-of-distribution samples efficiently.
problem Detecting samples from different distributions in deep neural networks.
method Statistical hypothesis testing framework combining evidence from entire network.
result Framework maintains Type I Error and achieves comparable results to state-of-the-art methods.
New test for conditional independence using kernel embeddings.
problem Testing conditional independence in high-dimensional settings.
method Analytic kernel embeddings, asymptotic distribution.
result New test outperforms existing methods in high-dimensional settings.
In this work, we consider the sample complexity required for testing the monotonicity of distributions over partial orders. A distribution p over a poset is monotone if, for any pair of domain elements x and y such that x⪯y, p(x)≤p(y). To understand the sample complexity of this problem, we intro…
Develops hypothesis tests for conditional distributions using learning-theoretic bounds.
problem Testing differences in conditional distributions and functionals.
method Transforming learning-theoretic bounds into hypothesis tests for conditional expectations.
result Establishes comprehensive foundation for conditional testing, including theoretical guarantees and practical implementations.
Improved KSD test for better detection of differences in distributions.
problem Low power of KSD test when distributions have same modes but different mixing proportions.
method Perturb the observed sample using Markov transition kernels to improve KSD test power.
result Perturbed KSD test can lead to substantially higher power than the original KSD test.
Proposes DR-ME test for interpretable distributional treatment effects.
problem Detects invisible differences in treatment effects on distributional outcomes.
method Semiparametrically efficient finite-location test using kernel witnesses and orthogonal features.
result DR-ME reveals causal-discrepancy coordinates and has noncentral chi-square local power.
Optimized testing of discrete distributions using predicted data.
problem Testing discrete distributions with reduced sample complexity.
method Adaptable algorithms that use a predicted distribution to reduce sample size.
result Optimal sample complexity improvements with self-adjusting algorithms.
In this work we present novel differentially private identity (goodness-of-fit) testers for natural and widely studied classes of multivariate product distributions: Gaussians in Rd with known covariance and product distributions over {±1}d. Our testers have improved sample complexity compared to …
New method uses CDMs to improve CI testing without distributional assumptions.
problem Testing conditional independence when the conditional distribution is unknown.
method Uses conditional diffusion models (CDMs) to approximate X∣Z and a classifier-based CMI estimator. result Proposed method performs better than GAN-based CI tests and controls type I and II errors.
Conditional independence testing is a fundamental problem underlying causal discovery and a particularly challenging task in the presence of nonlinear and high-dimensional dependencies. Here a fully non-parametric test for continuous data based on conditional mutual information combined with a local permutation scheme …
Robust hypothesis testing designs a test for worst-case distributions using kernel methods.
problem Design a robust test for hypothesis testing under uncertainty sets.
method Data-driven uncertainty sets constructed using kernel mean embeddings and maximum mean discrepancy (MMD). Bayesian and Neyman-Pearson settings investigated.
result Proposed robust kernel tests are exponentially consistent and asymptotically optimal.
Paper introduces ITD for detecting distributional changes in decentralized learning environments.
problem Detecting distributional changes in decentralized learning environments with data privacy and heterogeneity concerns.
method Introduces Integrated Transportation Distance (ITD) for two-sample testing in federated learning.
result ITD effectively aggregates information across distributed clients, detecting subtle distributional shifts.
In this paper, we propose Test-Time Training, a general approach for improving the performance of predictive models when training and test data come from different distributions. We turn a single unlabeled test sample into a self-supervised learning problem, on which we update the model parameters before making a predi…