New research shows no trade-off between fairness and accuracy in machine learning.
problem The trade-off between fairness and accuracy in machine learning is a widely accepted belief.
method Using mismatched hypothesis testing and Chernoff information, the study demonstrates that optimal fairness and accuracy can be achieved simultaneously.
result There is no inherent trade-off between fairness and accuracy in ideal distributions, but it exists when measured with respect to biased datasets.
We test the hypothesis that interconnections across financial institutions can be explained by a diversification motive. This idea stems from the empirical evidence of the existence of long-term exposures that cannot be explained by a liquidity motive (maturity or currency mismatch). We model endogenous interconnection…
PACMAN provides bounds for classification tasks considering accuracy vs. negative log-loss mismatch.
problem Mismatch between accuracy and negative log-loss in classification tasks.
method Point-wise PAC approach over generalization gap, using likelihood ratio and concentration inequalities.
result PACMAN provides point-wise PAC bounds for the generalization problem.
Ensemble models improve prediction calibration for mismatched distributions.
problem Calibration issues in deep neural networks with mismatched train and test distributions.
method Simple data augmentation and mixing techniques for ensemble models.
result Improves calibration and accuracy on CIFAR10 and CIFAR100 benchmarks.
Training on mixed distributions improves test performance even when components are unrelated.
problem Improving test performance with mismatched training and test distributions.
method Analyzing mixture distributions with different training and test proportions.
result Distribution shift can be beneficial, improving test performance even when components are unrelated.
A framework for hypothesis testing on attributed graphs using sampling.
problem Statistical testing on graph data, especially large attributed graphs.
method Sampling-based framework with PHASE and PHASEopt for accurate and efficient hypothesis testing.
result PHASE and PHASEopt improve accuracy and efficiency of hypothesis testing in attributed graphs.
The paper sets thresholds for testing correlation in hypergraphs, distinguishing between independent and correlated states.
problem Testing correlation between two hypergraphs under different models.
method Derives sharp information-theoretic thresholds for distinguishing between null and alternative hypotheses.
result The testing threshold decreases as the hypergraph's uniformity (m) increases, making correlation testing easier for higher uniformity.
Study robust hypothesis testing under Hellinger distance, proving lower bounds and providing tests.
problem Testing close variants of specified distributions robustly to Hellinger distance.
method Lower bound on slack factor, testing with Hellinger balls, symmetric chi-squared distance analysis.
result Lower bound on slack factor quantifies robustness under misspecification.
Paper resolves open problems on sample complexity in binary hypothesis testing.
problem Open problems in distributed simple binary hypothesis testing under information constraints.
method One-shot lower bound on Bayes error, streamlined sample complexity formula, reverse data-processing inequality.
result Optimally tight sample complexity bounds for communication-constrained simple binary hypothesis testing.
Transforms any test into anytime-valid with sample savings.
problem Sequential data invalidates classical test guarantees.
method Predicts test outcomes to create anytime-valid stopping rules.
result Ensures Type-I error control and near-optimal power.
Paper proposes a new framework for hypothesis testing in imaging.
problem Challenges in hypothesis testing for imaging data.
method Combines self-supervised imaging, vision-language models, and non-parametric hypothesis testing.
result Demonstrates improved power and robust error control in image-based phenotyping.
Paper proposes a robust hypothesis testing method using Sinkhorn distance.
problem Hypothesis testing for small samples.
method Data-driven approach using Sinkhorn uncertainty sets.
result The method provides a more flexible detector compared to Wasserstein robust test.
Study hypothesis testing under quantized samples with communication constraints, achieving near-optimal sample complexity.
problem Optimizing hypothesis testing with quantized samples and communication constraints.
method Developed a polynomial-time algorithm achieving near-optimal sample complexity under communication constraints.
result Achieved near-optimal sample complexity under communication constraints, with a logarithmic factor increase over unconstrained setting.
New private algorithm for sequential hypothesis testing with privacy and error rate guarantees.
problem Privacy protection in sequential hypothesis testing for sensitive data.
method Renyi differential privacy, Wald's Sequential Probability Ratio Test (SPRT).
result Private algorithm with strong privacy guarantees and theoretical performance analysis.
New method optimizes language model performance for test-time strategies.
problem Mismatch between training objectives and test-time deployment of large language models.
method Tail-Extrapolated estimators to approximate best-of-N performance from limited training rollouts.
result Improved performance of best-of-N deployment across various models and datasets.
Robust test for distributions under Hellinger distance, simpler than optimal tests.
problem Testing and estimating distributions robustly under Hellinger distance.
method Simple robust hypothesis test with optimal sample complexity, robust to Hellinger distance perturbations.
result Empirically demonstrated robustness and power of the test on canonical distributions.
We investigated the impact of noisy linguistic features on the performance of a Japanese speech synthesis system based on neural network that uses WaveNet vocoder. We compared an ideal system that uses manually corrected linguistic features including phoneme and prosodic information in training and test sets against a …
Unified Bayesian framework improves clinical trial hypothesis testing.
problem Lack of transparency and inability to quantify evidence in traditional P-values.
method Interval null hypothesis framework combined with Bayes factor-based tests.
result Bayesian interval hypothesis testing ensures frequentist error control and interpretability.
Study on hypothesis testing games with adversarial classification, showing convergence rates.
problem Adversarial classification in hypothesis testing.
method Mixed strategy Nash equilibria analysis, concentration phenomena examination.
result Exponential rates of convergence of classification errors at equilibrium.
The problem of multiple hypothesis testing arises when there are more than one hypothesis to be tested simultaneously for statistical significance. This is a very common situation in many data mining applications. For instance, assessing simultaneously the significance of all frequent itemsets of a single dataset entai…
New framework compares credal sets for hypothesis testing with epistemic uncertainty.
problem Comparing distributions with partial ignorance and epistemic uncertainty.
method Credal two-sample testing framework for convex sets of probability measures.
result Direct integration of epistemic uncertainty in hypothesis testing.
Develops GLRT for defending against adversarial attacks in hypothesis testing.
problem Adversarial attacks on machine learning models causing misclassification.
method Generalized likelihood ratio test applied to composite hypothesis testing problem.
result GLRT approach yields competitive robustness-accuracy tradeoff under various attacks.
Framework for online hypothesis testing across various data types.
problem Testing various nonparametric hypotheses in data streams.
method Unified framework using operators on data distributions, leveraging ML models.
result Efficient, adaptive, and error-controlled sequential tests.
Proposes a new model for testing causal structural priors and synthesizing data.
problem Testing and synthesizing causal structural priors using nonparametric knowledge and neural networks.
method Causal Structural Hypothesis Testing (C-SHT) and Causal Structural Variational Hypothesis Testing (C-SVHT) using deep neural networks.
result Demonstrates out-of-distribution generalization error as a proxy for causal structural prior hypothesis testing.
Formula derived for sample complexity in binary hypothesis testing.
problem Determine the minimum number of samples to distinguish between two distributions.
method Developed a formula for sample complexity in both prior-free and Bayesian settings, using Jensen-Shannon and Hellinger divergences.
result Formula characterizes sample complexity for a wide range of error parameters, up to multiplicative constants.
Hypothesis testing is an important problem with applications in target localization, clinical trials etc. Many active hypothesis testing strategies operate in two phases: an exploration phase and a verification phase. In the exploration phase, selection of experiments is such that a moderate level of confidence on the …
The performance of automatic speech recognition (ASR) systems can be significantly compromised by previously unseen conditions, which is typically due to a mismatch between training and testing distributions. In this paper, we address robustness by studying domain invariant features, such that domain information become…
Develops a hypothesis testing framework for generalized Thurstone models.
problem Determining whether pairwise comparison data fits a generalized Thurstone model.
method Introduces separation distance and derives upper and lower bounds for testing.
result Critical threshold for testing depends on observation graph topology and scales as Θ((nk)−1/2) for complete graphs. Unseen data conditions can inflict serious performance degradation on systems relying on supervised machine learning algorithms. Because data can often be unseen, and because traditional machine learning algorithms are trained in a supervised manner, unsupervised adaptation techniques must be used to adapt the model to…
Hypothesis testing is an important cognitive process that supports human reasoning. In this paper, we introduce a computational hypothesis testing approach based on memory augmented neural networks. Our approach involves a hypothesis testing loop that reconsiders and progressively refines a previously formed hypothesis…
New framework for valid hypothesis testing in complex data settings.
problem Challenges in classical hypothesis testing frameworks.
method Add and subtract external noise to partition data, orthogonalize, and test hypotheses.
result Valid hypothesis tests can be conducted under minimal assumptions.
Markov regime switching models have been used in numerous empirical studies in economics and finance. However, the asymptotic distribution of the likelihood ratio test statistic for testing the number of regimes in Markov regime switching models has been an unresolved problem. This paper derives the asymptotic distribu…
New method improves spatial prediction validation accuracy.
problem Validation methods fail for spatial prediction tasks due to mismatch between validation and test locations.
method Proposes a new validation method that adapts existing covariate-shift ideas to spatial settings.
result Proves and demonstrates the new method's superiority in spatial prediction validation.
We propose a methodology for testing linear hypothesis in high-dimensional linear models. The proposed test does not impose any restriction on the size of the model, i.e. model sparsity or the loading vector representing the hypothesis. Providing asymptotically valid methods for testing general linear functions of the …
A method to learn robust policies for environments with model mismatches.
problem Training agents in high-stakes scenarios with mismatched training and real environments.
method Formalizes the perturbation as a zero-sum game to find Nash Equilibrium, which corresponds to the robust policy.
result Our algorithm can find a near-optimal robust policy with high probability using polynomial samples.
Paper proposes a method to handle linear regression with partially shuffled data.
problem Linear regression with mismatched predictors and responses.
method Pseudo-likelihood approach based on two-component mixture densities with EM optimization.
result The method can tolerate larger fractions of mismatches and estimate noise level.
Improved hypothesis testing and change-point detection using diffusion-based methods.
problem Limited power of score-based hypothesis tests and change-point detection.
method Extending score-based Fisher divergence to diffusion-divergence by multiplying score functions with a matrix-valued function or weight matrix.
result Theoretical quantification and demonstration of optimal performance of diffusion-based algorithms.
Paper revisits pre-validation method, improving hypothesis testing.
problem Improving hypothesis testing in pre-validated models with different feature dimensions.
method Extended problem formulation, analytical distribution, and bootstrap procedure.
result Proposed analytical distribution and bootstrap procedure for pre-validated predictors.
Tests if vertices in graphs have the same latent positions.
problem Testing equality of latent positions in random graphs.
method Empirical Mahalanobis distances from spectral embeddings.
result Test statistics follow chi-square distributions under null and local alternatives.
Unified framework for optimal kernel tests across MMD, HSIC, and KSD.
problem Optimal testing in kernel-based hypothesis testing frameworks.
method Unified derivation of minimax rates, adaptive kernel selection methods.
result Unified power results across MMD, HSIC, and KSD.
Private online FDR control for adaptive testing under differential privacy.
problem Controlling false discoveries in adaptive multiple hypothesis testing with privacy constraints.
method Private online algorithms based on non-private results, ensuring privacy and statistical performance.
result Strong guarantees for privacy and statistical performance in FDR and power.
New tests for binary classification regression functions without distribution assumptions.
problem Testing regression functions in binary classification without distributional assumptions.
method Conditional kernel mean embeddings and resampling-based framework.
result Distribution-free hypothesis tests with exact type I error control.
Learning reward functions can lead to poor policy performance despite low error.
problem Low error in learned reward functions does not guarantee low regret in policy performance.
method Mathematical analysis of reward learning and policy optimization.
result A low expected test error of the reward model guarantees low worst-case regret, but error-regret mismatch can occur with certain data distributions.
SVHN dataset's split affects generative models but not digit classification.
problem Distribution mismatch between SVHN training and test sets impacts generative models.
method Empirically showed distribution mismatch affects generative models; proposed mixing and re-splitting.
result Distribution mismatch in SVHN dataset significantly impacts probabilistic generative models.
Deep-learning method improves hypothesis testing for independence.
problem Improving hypothesis testing for independence using deep learning.
method Proposes deep-testing, a novel procedure that uses a deep neural network to distinguish between data generated under and outside a given statistical model.
result Deep-testing achieves the highest overall power against nineteen competing methods across various dependence structures.
The paper tests if LLMs' capabilities are executed by small subnetworks (circuits).
problem Understanding how LLMs execute their capabilities.
method Formalized criteria for circuits, developed hypothesis tests, applied to six circuits.
result Synthetic circuits align with idealized properties, while Transformer circuits vary in their alignment.
This study applies old and new generations of panel unit root tests to test the validity of long-run real interest rate parity (RIP) hypothesis for ten Central and Eastern European Countries (CEECs) with respect to the Euro area and an average of the CEECs' real interest rates, respectively. When the panel unit root te…
The two-sample hypothesis testing problem is studied for the challenging scenario of high dimensional data sets with small sample sizes. We show that the two-sample hypothesis testing problem can be posed as a one-class set classification problem. In the set classification problem the goal is to classify a set of data …