The paper designs tests for comparing ranked preference data and finds significant differences.
problem Comparing pairwise comparison and ranking data in various applications.
method Developed two-sample tests for pairwise comparison and ranking data, proving upper and lower bounds.
result Upper and lower bounds show tightness of the proposed tests, and significant differences in preferences were found.
Estimates density ratio for two-sample comparison using tree models.
problem Comparing two distributions given i.i.d. observations.
method Additive tree models with balancing loss for density ratio estimation.
result Bayesian inference provides uncertainty quantification for density ratio.
The paper improves spectral ranking methods for diverse comparison graphs.
problem Estimating preference scores from multiway comparisons with heterogeneous sizes.
method Develops a two-step spectral method for estimating preference scores and their uncertainties.
result The two-step spectral method achieves the same asymptotic efficiency as the Maximum Likelihood Estimator (MLE).
We consider training a deep neural network to generate samples from an unknown distribution given i.i.d. data. We frame learning as an optimization minimizing a two-sample test statistic---informally speaking, a good generator network produces samples that cause a two-sample test to fail to reject the null hypothesis. …
Develops a fast, accurate method for comparing networks.
problem Comparing networks with repeated observations and varying sizes/sparsity.
method A novel two-sample hypothesis testing method with theoretical guarantees.
result Outperforms existing tools in speed and accuracy, power-optimal.
A new MMD-based test combines kernels for two-sample testing without splitting data.
problem Efficiently testing if two datasets come from the same distribution without splitting data.
method Proposes a novel statistic based on Maximum Mean Discrepancy (MMD) that combines kernels, proving concentration bounds and showing data-dependent kernel selection.
result Exponential concentration bounds and improved test power compared to existing methods.
Optimizes two-sample tests for non-Euclidean domains using spectral regularization.
problem Optimizing two-sample tests for non-Euclidean domains.
method Spectral regularization of MMD test to achieve minimax optimality.
result Proposes a spectral regularization method that improves test optimality.
The paper introduces methods to identify key variables discriminating between two datasets.
problem Identifying variables that distinguish between two datasets.
method Introduces a mathematical notion of discriminating variables and proposes two methods for their selection.
result Proposed methods improve upon existing techniques in two-sample variable selection.
A new witness two-sample test improves data efficiency and power.
problem Nonparametric two-sample testing.
method Optimizes kernel and defines weights and basis points using training data.
result The new test is consistent, has well-controlled type-I error, and has comparable or higher power.
We consider the problem of comparing probability densities between two groups. A new probabilistic tensor product smoothing spline framework is developed to model the joint density of two variables. Under such a framework, the probability density comparison is equivalent to testing the presence/absence of interactions.…
A new test for comparing distributions is proposed, balancing optimality and efficiency.
problem Comparing distributions defined over non-Euclidean domains.
method Spectral-regularized two-sample test based on random Fourier features.
result The proposed test is minimax optimal under certain conditions.
Approach selects variables and time intervals for comparing high-dimensional time-series data.
problem Comparing high-dimensional time-series data for significant differences.
method Data is split into subintervals, and two-sample tests are performed on each to identify distinguishing variables.
result The approach effectively identifies variables and time intervals where data significantly differs.
A test for comparing networks using stochastic block models.
problem Determining if two network datasets come from the same model.
method Adopting stochastic block models, the study introduces an efficient algorithm to match estimated network parameters and develops a powerful test.
result The test is consistent and asymptotically follows a chi-squared distribution.
Study evaluates two-sample tests for validating generative models in high dimensions.
problem Validating the performance and efficiency of non-parametric two-sample tests for high-dimensional generative models.
method Proposes and evaluates the sliced Wasserstein distance, mean of Kolmogorov-Smirnov statistics, and novel sliced Kolmogorov-Smirnov statistic.
result One-dimensional-based tests provide comparable sensitivity to other multivariate metrics but with lower computational cost.
Study evaluates ML methods for two-sample testing with right-censored data.
problem Evaluating ML methods for two-sample testing with right-censored data.
method Developed and compared several ML-based methods with classical tests.
result Proposed methods outperform classical tests in terms of statistical power.
The paper analyzes a neural network two-sample test using kernel analysis.
problem Determining if two datasets come from the same distribution.
method Time-analysis on a neural tangent kernel (NTK) two-sample test, extending to realistic neural network dynamics.
result Training times needed to detect deviations are well-separated in null and alternative hypothesis scenarios.
We consider the hypothesis testing problem of detecting a shift between the means of two multivariate normal distributions in the high-dimensional setting, allowing for the data dimension p to exceed the sample size n. Specifically, we propose a new test statistic for the two-sample test of means that integrates a rand…
AutoML simplifies two-sample tests for detecting distribution shifts.
problem Detecting distribution shifts between datasets.
method Uses mean discrepancy of a witness function with squared loss minimization.
result AutoML simplifies and improves two-sample testing performance.
Divergence estimators based on direct approximation of density-ratios without going through separate approximation of numerator and denominator densities have been successfully applied to machine learning tasks that involve distribution comparison such as outlier detection, transfer learning, and two-sample homogeneity…
E-C2ST uses E-values for high-dimensional data two-sample tests.
problem Statistical testing for high-dimensional data.
method Combines split likelihood ratio tests and predictive independence tests, using E-values for anytime-valid sequential tests.
result E-C2ST achieves enhanced statistical power by partitioning datasets into multiple batches.
A nonparametric two-sample test using a parametric integral probability metric
problem Detecting distributional differences between two independent samples
method Propose a new two-sample test statistic based on a newly introduced integral probability metric (IPM)
result Establish theoretical guarantees for the associated two-sample testing procedure
Develops hypothesis tests for conditional distributions using learning-theoretic bounds.
problem Testing differences in conditional distributions and functionals.
method Transforming learning-theoretic bounds into hypothesis tests for conditional expectations.
result Establishes comprehensive foundation for conditional testing, including theoretical guarantees and practical implementations.
Efficiently tests two distributions with few label queries.
problem Two-sample test with limited label information.
method Three-stage framework: classifier training, bimodal query, FR test.
result Significantly reduces Type II error compared to uniform querying.
A graph-based method for two-sample testing across connected nodes.
problem Identifying nodes where two probability distributions differ significantly.
method Collaborative non-parametric two-sample testing (CTST) framework.
result CTST outperforms independent node tests by leveraging graph structure.
Proposes counterfactual explanations for deep two-sample tests on high-dimensional data.
problem Limited interpretability of deep two-sample tests on high-dimensional data.
method Combines diffusion autoencoder and pretrained deep two-sample test model to generate counterfactuals.
result Counterfactual transformations increase p-values, indicating closer distribution similarity.
Conformal C2ST turns weak classifiers into reliable two-sample tests.
problem Determining if two distributions are identical using weak classifiers.
method Developed conformal variants of the C2ST to convert any classifier scores into reliable p-values.
result Even weak classifiers can yield powerful and reliable two-sample tests.
Semi-supervised method boosts two-sample testing with covariate data.
problem Two-sample testing with covariate information.
method Semi-supervised kernel test with asymptotic normality.
result Higher asymptotic power compared to existing methods.
Paper proposes a new method to compare classifiers across multiple datasets.
problem Comparing classifiers over multiple datasets with multiple criteria.
method Adopting decision theory, the paper introduces generalized stochastic dominance for ranking classifiers.
result Generalized stochastic dominance can be used to rank classifiers and statistically tested.
Deep neural nets optimize kernel parameters for non-parametric two-sample tests.
problem Determining if two samples come from the same distribution.
method Deep kernels trained to maximize test power, adapting to distribution smoothness and shape.
result Deep kernels outperform simpler kernels in high dimensions and complex data.
Study on testing two populations with confounders.
problem Determining if two populations have the same distribution after accounting for confounding factors.
method Introduce two general frameworks for conditional two-sample testing.
result Demonstrated the power and validity of the proposed frameworks.
There has been a recent surge of interest in studying permutation-based models for ranking from pairwise comparison data. Despite being structurally richer and more robust than parametric ranking models, permutation-based models are less well understood statistically and generally lack efficient learning algorithms. In…
Two-sample tests improve on existing methods for microtubule data.
problem Testing differences between two groups of filament data.
method Optimal lifts and manifold stability theorem applied to microtubule data.
result New tests outperform existing methods on simulated and real data.
We characterize the asymptotic performance of nonparametric one- and two-sample testing. The exponential decay rate or error exponent of the type-II error probability is used as the asymptotic performance metric, and an optimal test achieves the maximum rate subject to a constant level constraint on the type-I error pr…
A test for comparing function samples using MMD.
problem Testing if two functional data samples come from the same distribution.
method Maximum Mean Discrepancy (MMD) for functional data, with theoretical scaling analysis.
result The proposed test is effective and robust to functional reconstructions.
New test detects differences in heterogeneous datasets.
problem Detecting differences between two samples with unknown heterogeneity.
method Developed a nonparametric testing procedure that handles latent heterogeneity through a composite null.
result The test accurately detects differences in the presence of unknown heterogeneity.
New framework compares credal sets for hypothesis testing with epistemic uncertainty.
problem Comparing distributions with partial ignorance and epistemic uncertainty.
method Credal two-sample testing framework for convex sets of probability measures.
result Direct integration of epistemic uncertainty in hypothesis testing.
Study on kernel tests for high-dimensional data, focusing on MMD and CLT.
problem Asymptotic behavior of kernel two-sample tests in high dimensions and large samples.
method Maximum mean discrepancy (MMD) with isotropic kernels, deriving asymptotic expansions and CLT.
result Interplay between moment discrepancy and dimension-and-sample orders in kernel tests.
New algorithm detects tensor dependence structure alterations efficiently.
problem Detecting alterations in tensor dependence structures.
method Tensor-normal distributions, decorrelation, centralization, SERA (Sparsity-Exploited Reranking Algorithm).
result The proposed SERA algorithm controls false discovery rates effectively.
We consider a two-sample hypothesis testing problem, where the distributions are defined on the space of undirected graphs, and one has access to only one observation from each model. A motivating example for this problem is comparing the friendship networks on Facebook and LinkedIn. The practical approach to such prob…
Given two sets of independent samples from unknown distributions P and Q, a two-sample test decides whether to reject the null hypothesis that P=Q. Recent attention has focused on kernel two-sample tests as the test statistics are easy to compute, converge fast, and have low bias with their finite sample estimate…
New method relaxes TV distance for two-sample testing without distributional assumptions.
problem Challenges in certifying equality or providing tight bounds on TV distance for two distributions.
method Examined blurred total variation distance, a relaxation of TV distance.
result Provided theoretical guarantees for upper and lower bounds on blurred TV distance.
Unified framework for global and local two-sample conditional distribution testing.
problem Testing equality of two conditional distributions.
method Distance and kernel methods, conditional U-statistics, local bootstrap.
result Developed reliable global and local tests.
Meta two-sample testing uses auxiliary data to quickly find powerful tests from limited samples.
problem Challenges in identifying powerful kernels for distinguishing complex distributions with limited data.
method Introduces meta two-sample testing (M2ST) to leverage abundant auxiliary data on related tasks.
result Proposed algorithms improve over baselines and identify powerful tests from scarce observations.
Kernel test evaluates dynamical system data streams.
problem Evaluate if data streams from dynamical systems are from the same distribution.
method Proposes a novel kernel two-sample test for dynamical systems, addressing independence and autocorrelation challenges.
result Data-driven method with theoretical guarantees for anomaly detection.
The paper proves concentration inequalities for two-sample rank processes and applies them to ranking performance criteria.
problem Measuring the performance of ranking statistics between two populations.
method Proves concentration inequalities for two-sample rank processes indexed by VC classes of scoring functions.
result Generalization capacity of empirical maximizers of ranking performance criteria is investigated.
Develops a two-sample test using projected Wasserstein distance to handle high-dimensional data.
problem Testing whether two high-dimensional samples come from the same distribution.
method Optimal projection to find a low-dimensional linear mapping that maximizes the Wasserstein distance between projected probability distributions.
result Characterizes the convergence rate of the projected Wasserstein distance and presents practical algorithms.
The two-sample hypothesis testing problem is studied for the challenging scenario of high dimensional data sets with small sample sizes. We show that the two-sample hypothesis testing problem can be posed as a one-class set classification problem. In the set classification problem the goal is to classify a set of data …
We propose a two-sample testing procedure based on learned deep neural network representations. To this end, we define two test statistics that perform an asymptotic location test on data samples mapped onto a hidden layer. The tests are consistent and asymptotically control the type-1 error rate. Their test statistics…