fcHMRF-LIS controls FDR in neuroimaging data, improving power and scalability.
problem Complex spatial dependencies and high variability in FDR control methods for neuroimaging data.
method fcHMRF-LIS integrates LIS-based testing with fcHMRF to model spatial structures efficiently.
result fcHMRF-LIS achieves accurate FDR control, lower FNR, and higher true positives compared to existing methods.
Background and Purpose: Biopsy is the main determinants of glioma clinical management, but require invasive sampling that fail to detect relevant features because of tumor heterogeneity. The purpose of this study was to evaluate the accuracy of a voxel-wise, multiparametric MRI radiomic method to predict features and d…
Permutation testing is a non-parametric method for obtaining the max null distribution used to compute corrected p-values that provide strong control of false positives. In neuroimaging, however, the computational burden of running such an algorithm can be significant. We find that by viewing the permutation testing …
Proposes a multi-resolution model for prostate cancer classification using mpMRI.
problem Improving voxel-wise classification of prostate cancer using multi-parametric MRI data.
method Multi-resolution Super Learner framework combining local base learners at multiple resolutions and spatial Gaussian kernel smoothing.
result Enhanced voxel-wise classification of prostate cancer status and clinical significance.
3D CNNs interpret brain MRI differences between men and women.
problem Interpreting 3D CNNs for voxel-wise brain MRI analysis.
method Three interpretation methods: Meaningful Perturbations, Grad CAM, and Guided Backpropagation.
result Voxel-wise 3D CNN interpretation of brain MRI data.
Diffusion MRI is the modality of choice to study alterations of white matter. In past years, various works have used diffusion MRI for automatic classification of AD. However, classification performance obtained with different approaches is difficult to compare and these studies are also difficult to reproduce. In the …
Proposes a strategy to train models with minimal labeled data.
problem Scarce and expensive labeled data for medical tasks.
method Recursive training strategy to use image-level annotations for pixel-level segmentation.
result Improved segmentation of intracranial hemorrhage in CT scans.
Deep learning (DL) has shown great potential in medical image enhancement problems, such as super-resolution or image synthesis. However, to date, little consideration has been given to uncertainty quantification over the output image. Here we introduce methods to characterise different components of uncertainty in suc…
Noninvasive MR-guided focused ultrasound (MRgFUS) treatments are promising alternatives to the surgical removal of malignant tumors. A significant challenge is assessing the viability of treated tissue during and immediately after MRgFUS procedures. Current clinical assessment uses the nonperfused volume (NPV) biomarke…
Adaptive sampling improves deep learning efficiency by focusing on informative samples.
problem Expensive annotations and loss of sample informativeness in deep learning.
method Adaptive latent space sampling based on hardness-aware strategy.
result The proposed framework outperforms random sampling on various datasets.
Event-based models (EBM) are a class of disease progression models that can be used to estimate temporal ordering of neuropathological changes from cross-sectional data. Current EBMs only handle scalar biomarkers, such as regional volumes, as inputs. However, regional aggregates are a crude summary of the underlying hi…
Non-invasive detection of cardiovascular disorders from radiology scans requires quantitative image analysis of the heart and its substructures. There are well-established measurements that radiologists use for diseases assessment such as ejection fraction, volume of four chambers, and myocardium mass. These measuremen…
A new method combines OCSVM with representation learning for UAD.
problem Detect anomalies without labeled data, especially in rare or unavailable cases.
method Custom loss formulation that aligns latent features with OCSVM decision boundary.
result Succeeds in detecting small, non-hyperintense lesions in MRI.
The paper uses distance correlation for brain connectivity and a novel multi-task learning model for age prediction.
problem Estimating age-related gender differences in brain functional connectivity.
method Estimates functional connectivity using distance correlation and proposes a non-convex multi-task learning model.
result The proposed non-convex multi-task learning model outperforms other models in age prediction and gender-specific connectivity.
Many methods for automated software test generation, including some that explicitly use machine learning (and some that use ML more broadly conceived) derive new tests from existing tests (often referred to as seeds). Often, the seed tests from which new tests are derived are manually constructed, or at least simpler t…
A family of maximum mean discrepancy (MMD) kernel two-sample tests is introduced. Members of the test family are called Block-tests or B-tests, since the test statistic is an average over MMDs computed on subsets of the samples. The choice of block size allows control over the tradeoff between test power and computatio…
USP test improves on Pearson's chi-squared and G-test for independence.
problem Deficiencies in Pearson's chi-squared and G-test for independence. method USP test based on U-statistic estimator of population dependence measure. result USP test controls size, handles small cell counts, and detects minimal violations of independence.
E-C2ST uses E-values for high-dimensional data two-sample tests.
problem Statistical testing for high-dimensional data.
method Combines split likelihood ratio tests and predictive independence tests, using E-values for anytime-valid sequential tests.
result E-C2ST achieves enhanced statistical power by partitioning datasets into multiple batches.
This paper provides a comprehensive survey of Machine Learning Testing (ML testing) research. It covers 144 papers on testing properties (e.g., correctness, robustness, and fairness), testing components (e.g., the data, learning program, and framework), testing workflow (e.g., test generation and test evaluation), and …
Cheap permutation tests speed up distribution testing without sacrificing accuracy.
problem Efficiently testing distribution differences and independence.
method Group datapoints into bins and permute only these bins, using stored sufficient statistics.
result Cheap permutation tests maintain the accuracy and optimality of standard tests but are significantly faster.
DRIFT uses RL to automate functional software testing efficiently.
problem Efficient and reliable automated software testing.
method DRIFT employs Q-learning with Graph Neural Networks on symbolic UI representations.
result DRIFT can robustly test software functionalities in a fully automated manner.
Optimizes two-sample tests for non-Euclidean domains using spectral regularization.
problem Optimizing two-sample tests for non-Euclidean domains.
method Spectral regularization of MMD test to achieve minimax optimality.
result Proposes a spectral regularization method that improves test optimality.
Robust test for distributions under Hellinger distance, simpler than optimal tests.
problem Testing and estimating distributions robustly under Hellinger distance.
method Simple robust hypothesis test with optimal sample complexity, robust to Hellinger distance perturbations.
result Empirically demonstrated robustness and power of the test on canonical distributions.
A new method for kernel tests without data splitting increases power.
problem Lack of power in kernel-based tests due to data splitting.
method Selective inference framework to learn hyperparameters and test on full sample.
result Empirically larger test power without data splitting, regardless of split proportion.
Framework for online hypothesis testing across various data types.
problem Testing various nonparametric hypotheses in data streams.
method Unified framework using operators on data distributions, leveraging ML models.
result Efficient, adaptive, and error-controlled sequential tests.
Model-X test detects conditional independence in streaming data.
problem Detecting conditional independence in data streams with arbitrary dependency.
method Sequential testing inspired by model-X and testing by betting.
result Significantly reduces type-I error rate and enhances data efficiency.
Simple methods combine statistical tests for out-of-distribution detection.
problem Detecting data points not following the training distribution.
method Combining classical parametric tests (Rao's score test) and a typicality test.
result Combining Fisher's method of test statistics improves out-of-distribution detection accuracy.
Unified score and distance-based GoF tests for model adequacy.
problem Difficulty in extending score-based GoF tests to nonparametric alternatives.
method Introducing semiparametric kernelized Stein discrepancy (SKSD) test.
result SKSD test is computationally efficient and universally consistent.
AutoML simplifies two-sample tests for detecting distribution shifts.
problem Detecting distribution shifts between datasets.
method Uses mean discrepancy of a witness function with squared loss minimization.
result AutoML simplifies and improves two-sample testing performance.
New methods for CI testing under model misspecification.
problem Challenges in CI testing with misspecified models.
method Proposes new approximations and upper bounds for testing errors of regression-based CI tests.
result Introduces the Rao-Blackwellized Predictor Test (RBPT) robust against misspecified inductive biases.
Discusses MultiFIT for multivariate dependence, comparing it to HSIC tests.
problem Comparing Multiscale Fisher's Independence Test (MultiFIT) to HSIC tests for multivariate dependence.
method Compares MultiFIT to HSIC tests, highlighting exact level control and performance limitations.
result Observes performance limitations of MultiFIT in terms of test power.
The repeated community-wide reuse of test sets in popular benchmark problems raises doubts about the credibility of reported test-error rates. Verifying whether a learned model is overfitted to a test set is challenging as independent test sets drawn from the same data distribution are usually unavailable, while other …
Develops hypothesis tests for conditional distributions using learning-theoretic bounds.
problem Testing differences in conditional distributions and functionals.
method Transforming learning-theoretic bounds into hypothesis tests for conditional expectations.
result Establishes comprehensive foundation for conditional testing, including theoretical guarantees and practical implementations.
Develops tests for conditional symmetry under group actions.
problem Testing conditional symmetry in distributions under group actions.
method Nonparametric randomization tests with kernel methods and asymptotic consistency.
result Tests achieve finite-sample Type I error control and power.
Paper proposes a new method for more accurate group testing of infected patients.
problem Identifying infected patients efficiently with reduced tests and corrected errors.
method Adaptive design of pools based on Bayesian posterior prediction using belief propagation algorithm.
result The proposed method results in more accurate identification of infected patients.
This paper introduces differentially private permutation tests for hypothesis testing.
problem Privacy concerns in sensitive data analysis.
method Differentially private permutation tests for kernel methods.
result Proposes dpMMD and dpHSIC for two-sample and independence testing, achieving optimal power.
Improved MMD test for two-sample testing with random Fourier features.
problem Quadratic-time complexity of MMD test for large-scale analysis.
method Approximated MMD test using random Fourier features, investigating time-power trade-off.
result Sub-quadratic time complexity with same minimax separation rates as MMD test.
Distance correlation has gained much recent attention in the data science community: the sample statistic is straightforward to compute and asymptotically equals zero if and only if independence, making it an ideal choice to discover any type of dependency structure given sufficient sample size. One major bottleneck is…
Paper proposes a robust test for high-dimensional models with large covariates and instruments.
problem Testing high-dimensional linear instrumental variable models with large covariates and instruments.
method Introduces a test based on the maximum norm of multiple parameters and a power-enhanced test.
result The proposed test is robust to heteroskedastic errors and has higher power than existing tests.
Polynomial-time test for detecting dense subgraphs in heterogeneous networks.
problem Detecting a planted community in heterogeneous networks.
method Proposes a polynomial-time test with a standard normal distribution null limiting distribution.
result The test is efficient and performs well in both simulations and real data.
The study analyzes group testing algorithms for identifying defective items with high confidence.
problem Identifying defective items from a population using group testing with high confidence.
method Formulated as a function learning problem using the PAC framework, analyzed three algorithms: column matching, combinatorial basis pursuit, and definite defectives.
result Derived bounds on the number of tests needed for approximate set identification, comparing with existing bounds and simulating performance.
We propose a novel adaptive test of goodness-of-fit, with computational cost linear in the number of samples. We learn the test features that best indicate the differences between observed samples and a reference model, by minimizing the false negative rate. These features are constructed via Stein's method, meaning th…
A permutation-based SW test achieves minimax-optimal power for two-sample testing.
problem Nonparametric two-sample testing using the sliced Wasserstein distance.
method Proposes a permutation-based SW test and analyzes its performance.
result Achieves minimax separation rate n−1/2 over multinomial and bounded-support alternatives. We conduct an extensive evaluation of price jump tests based on high-frequency financial data. After providing a concise review of multiple alternative tests, we document the size and power of all tests in a range of empirically relevant scenarios. Particular focus is given to the robustness of test performance to the …
This paper explores the information-theoretic limitations of graph property testing in zero-field Ising models. Instead of learning the entire graph structure, sometimes testing a basic graph property such as connectivity, cycle presence or maximum clique size is a more relevant and attainable objective. Since property…
CTT compresses samples to test distributions near-linearly, outperforming existing methods.
problem Efficiently testing distributions with high power and near-linear runtime.
method Sample compression followed by permutation testing.
result CTT achieves near-linear runtime while maintaining high statistical power.
Meta two-sample testing uses auxiliary data to quickly find powerful tests from limited samples.
problem Challenges in identifying powerful kernels for distinguishing complex distributions with limited data.
method Introduces meta two-sample testing (M2ST) to leverage abundant auxiliary data on related tasks.
result Proposed algorithms improve over baselines and identify powerful tests from scarce observations.
We consider the problem of comparing probability densities between two groups. A new probabilistic tensor product smoothing spline framework is developed to model the joint density of two variables. Under such a framework, the probability density comparison is equivalent to testing the presence/absence of interactions.…