The paper proves concentration inequalities for two-sample rank processes and applies them to ranking performance criteria.
problem Measuring the performance of ranking statistics between two populations.
method Proves concentration inequalities for two-sample rank processes indexed by VC classes of scoring functions.
result Generalization capacity of empirical maximizers of ranking performance criteria is investigated.
New gossip algorithms improve robustness of rank-based statistics in decentralized systems.
problem Ensuring robustness in decentralized AI and edge intelligence systems, especially in the presence of corrupted or adversarial data.
method Developed asynchronous gossip algorithms for computing rank-based statistics.
result First convergence rate bound for asynchronous gossip-based rank estimation.
Paper proposes a scoring function for detecting anomalies in large datasets.
problem Detecting outliers in large, feature-rich datasets.
method Binary classification problem with a two-sample linear rank statistic.
result Empirical results show the effectiveness of the proposed scoring function.
A new family of nonparametric statistics, the r-statistics, is introduced. It consists of counting the number of records of the cumulative sum of the sample. The single-sample r-statistic is almost as powerful as Student's t-statistic for Gaussian and uniformly distributed variables, and more powerful than the sign and…
The paper designs tests for comparing ranked preference data and finds significant differences.
problem Comparing pairwise comparison and ranking data in various applications.
method Developed two-sample tests for pairwise comparison and ranking data, proving upper and lower bounds.
result Upper and lower bounds show tightness of the proposed tests, and significant differences in preferences were found.
The paper improves spectral ranking methods for diverse comparison graphs.
problem Estimating preference scores from multiway comparisons with heterogeneous sizes.
method Develops a two-step spectral method for estimating preference scores and their uncertainties.
result The two-step spectral method achieves the same asymptotic efficiency as the Maximum Likelihood Estimator (MLE).
Weighted log-rank tests are arguably the most widely used tests by practitioners for the two-sample problem in the context of right-censored data. Many approaches have been considered to make weighted log-rank tests more robust against a broader family of alternatives, among them, considering linear combinations of wei…
The paper analyzes a neural network two-sample test using kernel analysis.
problem Determining if two datasets come from the same distribution.
method Time-analysis on a neural tangent kernel (NTK) two-sample test, extending to realistic neural network dynamics.
result Training times needed to detect deviations are well-separated in null and alternative hypothesis scenarios.
E-C2ST uses E-values for high-dimensional data two-sample tests.
problem Statistical testing for high-dimensional data.
method Combines split likelihood ratio tests and predictive independence tests, using E-values for anytime-valid sequential tests.
result E-C2ST achieves enhanced statistical power by partitioning datasets into multiple batches.
A nonparametric two-sample test using a parametric integral probability metric
problem Detecting distributional differences between two independent samples
method Propose a new two-sample test statistic based on a newly introduced integral probability metric (IPM)
result Establish theoretical guarantees for the associated two-sample testing procedure
AutoML simplifies two-sample tests for detecting distribution shifts.
problem Detecting distribution shifts between datasets.
method Uses mean discrepancy of a witness function with squared loss minimization.
result AutoML simplifies and improves two-sample testing performance.
There has been a recent surge of interest in studying permutation-based models for ranking from pairwise comparison data. Despite being structurally richer and more robust than parametric ranking models, permutation-based models are less well understood statistically and generally lack efficient learning algorithms. In…
We consider a two-sample hypothesis testing problem, where the distributions are defined on the space of undirected graphs, and one has access to only one observation from each model. A motivating example for this problem is comparing the friendship networks on Facebook and LinkedIn. The practical approach to such prob…
A graph-based method for two-sample testing across connected nodes.
problem Identifying nodes where two probability distributions differ significantly.
method Collaborative non-parametric two-sample testing (CTST) framework.
result CTST outperforms independent node tests by leveraging graph structure.
We consider training a deep neural network to generate samples from an unknown distribution given i.i.d. data. We frame learning as an optimization minimizing a two-sample test statistic---informally speaking, a good generator network produces samples that cause a two-sample test to fail to reject the null hypothesis. …
We propose a two-sample testing procedure based on learned deep neural network representations. To this end, we define two test statistics that perform an asymptotic location test on data samples mapped onto a hidden layer. The tests are consistent and asymptotically control the type-1 error rate. Their test statistics…
New test detects differences in heterogeneous datasets.
problem Detecting differences between two samples with unknown heterogeneity.
method Developed a nonparametric testing procedure that handles latent heterogeneity through a composite null.
result The test accurately detects differences in the presence of unknown heterogeneity.
A new method uses neural tangent kernel to efficiently compute MMD statistic.
problem Efficiently computing Maximum Mean Discrepancy (MMD) statistic with low memory and computational complexity.
method Identifies a connection between neural tangent kernel (NTK) and MMD to develop a computationally and memory-efficient approach.
result The proposed NTK-MMD statistic is validated through numerical experiments on synthetic and real-world datasets.
Study evaluates ML methods for two-sample testing with right-censored data.
problem Evaluating ML methods for two-sample testing with right-censored data.
method Developed and compared several ML-based methods with classical tests.
result Proposed methods outperform classical tests in terms of statistical power.
Proposes counterfactual explanations for deep two-sample tests on high-dimensional data.
problem Limited interpretability of deep two-sample tests on high-dimensional data.
method Combines diffusion autoencoder and pretrained deep two-sample test model to generate counterfactuals.
result Counterfactual transformations increase p-values, indicating closer distribution similarity.
Unified framework for global and local two-sample conditional distribution testing.
problem Testing equality of two conditional distributions.
method Distance and kernel methods, conditional U-statistics, local bootstrap.
result Developed reliable global and local tests.
A permutation-based SW test achieves minimax-optimal power for two-sample testing.
problem Nonparametric two-sample testing using the sliced Wasserstein distance.
method Proposes a permutation-based SW test and analyzes its performance.
result Achieves minimax separation rate n−1/2 over multinomial and bounded-support alternatives. A new permutation method improves two-sample testing power.
problem Two-sample testing with improved power and validity.
method Structured block-restricted cross-swaps.
result Block-restricted permutations achieve higher power than full permutations.
Semi-supervised method boosts two-sample testing with covariate data.
problem Two-sample testing with covariate information.
method Semi-supervised kernel test with asymptotic normality.
result Higher asymptotic power compared to existing methods.
The recent success of generative adversarial networks and variational learning suggests training a classifier network may work well in addressing the classical two-sample problem. Network-based tests have the computational advantage that the algorithm scales to large samples. This paper proposes a two-sample statistic …
Proposes practical kernel tests for f-divergences with theoretical guarantees.
problem Two-sample testing and machine unlearning evaluation.
method Regularized f-divergence kernel tests, adaptive to hyperparameters. result Different f-divergences highlight localized differences. A new test assesses text similarity between two groups of documents.
problem Comparing similarity between two groups of documents.
method Neural network-based language models estimate entropy, and a test statistic derived from an estimation-and-inference framework is used.
result The proposed test maintains the nominal Type one error rate while offering greater power compared to existing methods.
Efficiently tests two distributions using Nyström approximation of MMD.
problem Testing whether two sets of data are from the same distribution in large-scale scenarios.
method Nyström approximation of maximum mean discrepancy (MMD) for scalable testing.
result Finite-sample bound on power of the test for sufficiently separated distributions.
Proposes a method to select variables for kernel two-sample tests.
problem Determining whether two samples have the same distribution using informative variables.
method A framework based on kernel maximum mean discrepancy (MMD) for selecting a subset of variables.
result The sample size requirements for the three kernels depend on the number of selected variables, not the data dimension.
A new MMD-based test combines kernels for two-sample testing without splitting data.
problem Efficiently testing if two datasets come from the same distribution without splitting data.
method Proposes a novel statistic based on Maximum Mean Discrepancy (MMD) that combines kernels, proving concentration bounds and showing data-dependent kernel selection.
result Exponential concentration bounds and improved test power compared to existing methods.
Deep neural nets optimize kernel parameters for non-parametric two-sample tests.
problem Determining if two samples come from the same distribution.
method Deep kernels trained to maximize test power, adapting to distribution smoothness and shape.
result Deep kernels outperform simpler kernels in high dimensions and complex data.
A new test statistic speeds up MMD while maintaining power.
problem Efficiently testing two distributions without permutations.
method Cross-MMD statistic based on sample-splitting and studentization.
result Cross-MMD has a limiting standard Gaussian distribution under the null.
Study on kernel tests for high-dimensional data, focusing on MMD and CLT.
problem Asymptotic behavior of kernel two-sample tests in high dimensions and large samples.
method Maximum mean discrepancy (MMD) with isotropic kernels, deriving asymptotic expansions and CLT.
result Interplay between moment discrepancy and dimension-and-sample orders in kernel tests.
A family of maximum mean discrepancy (MMD) kernel two-sample tests is introduced. Members of the test family are called Block-tests or B-tests, since the test statistic is an average over MMDs computed on subsets of the samples. The choice of block size allows control over the tradeoff between test power and computatio…
New method controls FDR for sparse GLMs, identifying positive and negative relationships.
problem Sparse GLMs with high-dimensional data and varying sample size.
method Debiased-Lasso estimator and CLIME method for precision matrix estimation.
result Asymptotically controls directional FDR and FDV for sparse GLMs.
Develops a two-sample test using projected Wasserstein distance to handle high-dimensional data.
problem Testing whether two high-dimensional samples come from the same distribution.
method Optimal projection to find a low-dimensional linear mapping that maximizes the Wasserstein distance between projected probability distributions.
result Characterizes the convergence rate of the projected Wasserstein distance and presents practical algorithms.
Kernel two-sample testing is a useful statistical tool in determining whether data samples arise from different distributions without imposing any parametric assumptions on those distributions. However, raw data samples can expose sensitive information about individuals who participate in scientific studies, which make…
Improved MMD test for two-sample testing with random Fourier features.
problem Quadratic-time complexity of MMD test for large-scale analysis.
method Approximated MMD test using random Fourier features, investigating time-power trade-off.
result Sub-quadratic time complexity with same minimax separation rates as MMD test.
Nonparametric two sample or homogeneity testing is a decision theoretic problem that involves identifying differences between two random variables without making parametric assumptions about their underlying distributions. The literature is old and rich, with a wide variety of statistics having being intelligently desi…
Given two sets of independent samples from unknown distributions P and Q, a two-sample test decides whether to reject the null hypothesis that P=Q. Recent attention has focused on kernel two-sample tests as the test statistics are easy to compute, converge fast, and have low bias with their finite sample estimate…
Conformal C2ST turns weak classifiers into reliable two-sample tests.
problem Determining if two distributions are identical using weak classifiers.
method Developed conformal variants of the C2ST to convert any classifier scores into reliable p-values.
result Even weak classifiers can yield powerful and reliable two-sample tests.
sig-MMD tests compare path distributions using kernel methods.
problem Comparing path distributions in stochastic processes.
method Signature kernel for path space valued distributions.
result sig-MMD can lead to Type 2 errors in limited data settings.
This work studies the statistical performance of Sinkhorn iterations in estimating Schrödinger bridges.
problem Estimating Schrödinger bridges with limited samples.
method Intermediate Sinkhorn iterations applied to the time-dependent drifts of SDEs.
result Established a statistical bound on the squared total variation error of Sinkhorn bridge iterations.
Distance-based tests, also called "energy statistics", are leading methods for two-sample and independence tests from the statistics community. Kernel-based tests, developed from "kernel mean embeddings", are leading methods for two-sample and independence tests from the machine learning community. A fixed-point transf…
A tractable pseudo-metric for non-parametric distributions via SPD geometry.
problem Computing distances between non-parametric probability distributions is intractable.
method Two-stage framework: projection onto parametric family, embedding into SPD matrices.
result Closed-form pseudo-metric for two-sample hypothesis testing.
The goal of two-sample tests is to assess whether two samples, SP∼Pn and SQ∼Qm, are drawn from the same distribution. Perhaps intriguingly, one relatively unexplored method to build two-sample tests is the use of binary classifiers. In particular, construct a dataset by pairing the n examples in $S_…
Boosts kernel two-sample test power with multiple kernels.
problem Detecting differences between two distributions over metric spaces.
method Combining MMD estimates over multiple kernels using Mahalanobis distance.
result More powerful in detecting a wide range of alternatives in finite samples.
Paper introduces ITD for detecting distributional changes in decentralized learning environments.
problem Detecting distributional changes in decentralized learning environments with data privacy and heterogeneity concerns.
method Introduces Integrated Transportation Distance (ITD) for two-sample testing in federated learning.
result ITD effectively aggregates information across distributed clients, detecting subtle distributional shifts.