The paper analyzes a neural network two-sample test using kernel analysis.
problem Determining if two datasets come from the same distribution.
method Time-analysis on a neural tangent kernel (NTK) two-sample test, extending to realistic neural network dynamics.
result Training times needed to detect deviations are well-separated in null and alternative hypothesis scenarios.
AutoML simplifies two-sample tests for detecting distribution shifts.
problem Detecting distribution shifts between datasets.
method Uses mean discrepancy of a witness function with squared loss minimization.
result AutoML simplifies and improves two-sample testing performance.
E-C2ST uses E-values for high-dimensional data two-sample tests.
problem Statistical testing for high-dimensional data.
method Combines split likelihood ratio tests and predictive independence tests, using E-values for anytime-valid sequential tests.
result E-C2ST achieves enhanced statistical power by partitioning datasets into multiple batches.
Study evaluates ML methods for two-sample testing with right-censored data.
problem Evaluating ML methods for two-sample testing with right-censored data.
method Developed and compared several ML-based methods with classical tests.
result Proposed methods outperform classical tests in terms of statistical power.
A nonparametric two-sample test using a parametric integral probability metric
problem Detecting distributional differences between two independent samples
method Propose a new two-sample test statistic based on a newly introduced integral probability metric (IPM)
result Establish theoretical guarantees for the associated two-sample testing procedure
Efficiently tests two distributions with few label queries.
problem Two-sample test with limited label information.
method Three-stage framework: classifier training, bimodal query, FR test.
result Significantly reduces Type II error compared to uniform querying.
Proposes counterfactual explanations for deep two-sample tests on high-dimensional data.
problem Limited interpretability of deep two-sample tests on high-dimensional data.
method Combines diffusion autoencoder and pretrained deep two-sample test model to generate counterfactuals.
result Counterfactual transformations increase p-values, indicating closer distribution similarity.
A graph-based method for two-sample testing across connected nodes.
problem Identifying nodes where two probability distributions differ significantly.
method Collaborative non-parametric two-sample testing (CTST) framework.
result CTST outperforms independent node tests by leveraging graph structure.
Semi-supervised method boosts two-sample testing with covariate data.
problem Two-sample testing with covariate information.
method Semi-supervised kernel test with asymptotic normality.
result Higher asymptotic power compared to existing methods.
Study on testing two populations with confounders.
problem Determining if two populations have the same distribution after accounting for confounding factors.
method Introduce two general frameworks for conditional two-sample testing.
result Demonstrated the power and validity of the proposed frameworks.
New framework compares credal sets for hypothesis testing with epistemic uncertainty.
problem Comparing distributions with partial ignorance and epistemic uncertainty.
method Credal two-sample testing framework for convex sets of probability measures.
result Direct integration of epistemic uncertainty in hypothesis testing.
Conformal C2ST turns weak classifiers into reliable two-sample tests.
problem Determining if two distributions are identical using weak classifiers.
method Developed conformal variants of the C2ST to convert any classifier scores into reliable p-values.
result Even weak classifiers can yield powerful and reliable two-sample tests.
Meta two-sample testing uses auxiliary data to quickly find powerful tests from limited samples.
problem Challenges in identifying powerful kernels for distinguishing complex distributions with limited data.
method Introduces meta two-sample testing (M2ST) to leverage abundant auxiliary data on related tasks.
result Proposed algorithms improve over baselines and identify powerful tests from scarce observations.
A test for comparing function samples using MMD.
problem Testing if two functional data samples come from the same distribution.
method Maximum Mean Discrepancy (MMD) for functional data, with theoretical scaling analysis.
result The proposed test is effective and robust to functional reconstructions.
We characterize the asymptotic performance of nonparametric one- and two-sample testing. The exponential decay rate or error exponent of the type-II error probability is used as the asymptotic performance metric, and an optimal test achieves the maximum rate subject to a constant level constraint on the type-I error pr…
The paper designs tests for comparing ranked preference data and finds significant differences.
problem Comparing pairwise comparison and ranking data in various applications.
method Developed two-sample tests for pairwise comparison and ranking data, proving upper and lower bounds.
result Upper and lower bounds show tightness of the proposed tests, and significant differences in preferences were found.
New test detects differences in heterogeneous datasets.
problem Detecting differences between two samples with unknown heterogeneity.
method Developed a nonparametric testing procedure that handles latent heterogeneity through a composite null.
result The test accurately detects differences in the presence of unknown heterogeneity.
Given two sets of independent samples from unknown distributions P and Q, a two-sample test decides whether to reject the null hypothesis that P=Q. Recent attention has focused on kernel two-sample tests as the test statistics are easy to compute, converge fast, and have low bias with their finite sample estimate…
We propose a two-sample testing procedure based on learned deep neural network representations. To this end, we define two test statistics that perform an asymptotic location test on data samples mapped onto a hidden layer. The tests are consistent and asymptotically control the type-1 error rate. Their test statistics…
Two-sample tests improve on existing methods for microtubule data.
problem Testing differences between two groups of filament data.
method Optimal lifts and manifold stability theorem applied to microtubule data.
result New tests outperform existing methods on simulated and real data.
Unified framework for global and local two-sample conditional distribution testing.
problem Testing equality of two conditional distributions.
method Distance and kernel methods, conditional U-statistics, local bootstrap.
result Developed reliable global and local tests.
Kernel test evaluates dynamical system data streams.
problem Evaluate if data streams from dynamical systems are from the same distribution.
method Proposes a novel kernel two-sample test for dynamical systems, addressing independence and autocorrelation challenges.
result Data-driven method with theoretical guarantees for anomaly detection.
We consider a two-sample hypothesis testing problem, where the distributions are defined on the space of undirected graphs, and one has access to only one observation from each model. A motivating example for this problem is comparing the friendship networks on Facebook and LinkedIn. The practical approach to such prob…
Unified data representation learning improves non-parametric two-sample testing.
problem Improving non-parametric two-sample testing accuracy.
method Proposes RL-TST framework combining IRs and DRs for better test power.
result RL-TST outperforms existing methods by leveraging both IRs and DRs.
Optimizes two-sample tests for non-Euclidean domains using spectral regularization.
problem Optimizing two-sample tests for non-Euclidean domains.
method Spectral regularization of MMD test to achieve minimax optimality.
result Proposes a spectral regularization method that improves test optimality.
Sequential tests for two-sample and independence testing using betting strategies.
problem Testing sequential data for two-sample and independence without kernel selection issues.
method Prediction-based betting strategies that adaptively determine distribution and joint distribution.
result Prediction-based tests outperform kernel-based approaches in high-dimensional or structured data settings.
Study on kernel tests for high-dimensional data, focusing on MMD and CLT.
problem Asymptotic behavior of kernel two-sample tests in high dimensions and large samples.
method Maximum mean discrepancy (MMD) with isotropic kernels, deriving asymptotic expansions and CLT.
result Interplay between moment discrepancy and dimension-and-sample orders in kernel tests.
We propose a class of kernel-based two-sample tests, which aim to determine whether two sets of samples are drawn from the same distribution. Our tests are constructed from kernels parameterized by deep neural nets, trained to maximize test power. These tests adapt to variations in distribution smoothness and shape ove…
The two-sample hypothesis testing problem is studied for the challenging scenario of high dimensional data sets with small sample sizes. We show that the two-sample hypothesis testing problem can be posed as a one-class set classification problem. In the set classification problem the goal is to classify a set of data …
A permutation-based SW test achieves minimax-optimal power for two-sample testing.
problem Nonparametric two-sample testing using the sliced Wasserstein distance.
method Proposes a permutation-based SW test and analyzes its performance.
result Achieves minimax separation rate n−1/2 over multinomial and bounded-support alternatives. Develops a two-sample test using projected Wasserstein distance to handle high-dimensional data.
problem Testing whether two high-dimensional samples come from the same distribution.
method Optimal projection to find a low-dimensional linear mapping that maximizes the Wasserstein distance between projected probability distributions.
result Characterizes the convergence rate of the projected Wasserstein distance and presents practical algorithms.
New method relaxes TV distance for two-sample testing without distributional assumptions.
problem Challenges in certifying equality or providing tight bounds on TV distance for two distributions.
method Examined blurred total variation distance, a relaxation of TV distance.
result Provided theoretical guarantees for upper and lower bounds on blurred TV distance.
Leveraging reference-only samples for two-sample testing under size asymmetry
problem Two-sample testing under size imbalance
method Adaptive aggregation of reference-dependent representations
result Strong performance with type I error control
A density ratio is defined by the ratio of two probability densities. We study the inference problem of density ratios and apply a semi-parametric density-ratio estimator to the two-sample homogeneity test. In the proposed test procedure, the f-divergence between two probability densities is estimated using a density-r…
A new MMD-based test combines kernels for two-sample testing without splitting data.
problem Efficiently testing if two datasets come from the same distribution without splitting data.
method Proposes a novel statistic based on Maximum Mean Discrepancy (MMD) that combines kernels, proving concentration bounds and showing data-dependent kernel selection.
result Exponential concentration bounds and improved test power compared to existing methods.
A new permutation method improves two-sample testing power.
problem Two-sample testing with improved power and validity.
method Structured block-restricted cross-swaps.
result Block-restricted permutations achieve higher power than full permutations.
We consider training a deep neural network to generate samples from an unknown distribution given i.i.d. data. We frame learning as an optimization minimizing a two-sample test statistic---informally speaking, a good generator network produces samples that cause a two-sample test to fail to reject the null hypothesis. …
Private two-sample tests under LDP achieve minimax rates for multinomial and continuous data.
problem Achieving statistical utility while maintaining privacy in two-sample testing.
method Private permutation tests for multinomial data and adaptive tests for continuous data.
result Minimax optimal tests for private two-sample testing under LDP.
The recent success of generative adversarial networks and variational learning suggests training a classifier network may work well in addressing the classical two-sample problem. Network-based tests have the computational advantage that the algorithm scales to large samples. This paper proposes a two-sample statistic …
In analyzing high-dimensional models, sparsity of the model parameter is a common but often undesirable assumption. In this paper, we study the following two-sample testing problem: given two samples generated by two high-dimensional linear models, we aim to test whether the regression coefficients of the two linear mo…
The goal of two-sample tests is to assess whether two samples, SP∼Pn and SQ∼Qm, are drawn from the same distribution. Perhaps intriguingly, one relatively unexplored method to build two-sample tests is the use of binary classifiers. In particular, construct a dataset by pairing the n examples in $S_…
A new test assesses text similarity between two groups of documents.
problem Comparing similarity between two groups of documents.
method Neural network-based language models estimate entropy, and a test statistic derived from an estimation-and-inference framework is used.
result The proposed test maintains the nominal Type one error rate while offering greater power compared to existing methods.
A new, fast kernel test for large data.
problem Efficient kernel two-sample tests for high-dimensional, large-scale data.
method A new kernel-based test that is computationally efficient and robust to high dimensions.
result The new test performs well across various alternatives and dimensions.
A new witness two-sample test improves data efficiency and power.
problem Nonparametric two-sample testing.
method Optimizes kernel and defines weights and basis points using training data.
result The new test is consistent, has well-controlled type-I error, and has comparable or higher power.
Proposes practical kernel tests for f-divergences with theoretical guarantees.
problem Two-sample testing and machine unlearning evaluation.
method Regularized f-divergence kernel tests, adaptive to hyperparameters. result Different f-divergences highlight localized differences. Improved MMD test for two-sample testing with random Fourier features.
problem Quadratic-time complexity of MMD test for large-scale analysis.
method Approximated MMD test using random Fourier features, investigating time-power trade-off.
result Sub-quadratic time complexity with same minimax separation rates as MMD test.
A family of maximum mean discrepancy (MMD) kernel two-sample tests is introduced. Members of the test family are called Block-tests or B-tests, since the test statistic is an average over MMDs computed on subsets of the samples. The choice of block size allows control over the tradeoff between test power and computatio…
Paper introduces ITD for detecting distributional changes in decentralized learning environments.
problem Detecting distributional changes in decentralized learning environments with data privacy and heterogeneity concerns.
method Introduces Integrated Transportation Distance (ITD) for two-sample testing in federated learning.
result ITD effectively aggregates information across distributed clients, detecting subtle distributional shifts.