Statistical tests that compare classification algorithms are univariate and use a single performance measure, e.g., misclassification error, measure, AUC, and so on. In multivariate tests, comparison is done using multiple measures simultaneously. For example, error is the sum of false positives and false negatives…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “Testing algorithms”
A simple algorithm for Gaussian mean testing with optimal sample complexity.
problem Distinguishing between standard Gaussian and other Gaussian distributions with unknown mean and covariance.
method An extremely simple algorithm with a one-page analysis.
result Optimal sample complexity of Θ(√d/ε^2) with sample linear time.
Simpler, faster algorithm for uniformity testing in the shuffle model.
problem Testing uniformity of data in the shuffle model with privacy constraints.
method Simplified analysis and use of privacy amplification via shuffling.
result An algorithm with the same guarantees but simpler and more streamlined.
The statistical comparison of multiple algorithms over multiple data sets is fundamental in machine learning. This is typically carried out by the Friedman test. When the Friedman test rejects the null hypothesis, multiple comparisons are carried out to establish which are the significant differences among algorithms. …
New algorithm reduces conditional independence tests needed for causal discovery.
problem Efficiently infer causal relations from observational data.
method Established an algorithm with complexity tests.
result Achieves exponent-optimality up to a logarithmic factor in terms of conditional independence tests.
The study analyzes group testing algorithms for identifying defective items with high confidence.
problem Identifying defective items from a population using group testing with high confidence.
method Formulated as a function learning problem using the PAC framework, analyzed three algorithms: column matching, combinatorial basis pursuit, and definite defectives.
result Derived bounds on the number of tests needed for approximate set identification, comparing with existing bounds and simulating performance.
The paper improves confidence intervals for test error using cross-validation.
problem Improving confidence intervals for test error in machine learning.
method Develops central limit theorems and consistent estimators for cross-validation.
result Provides asymptotically-exact confidence intervals and hypothesis tests.
New uniformity tester ensures consistent results across different samples.
problem Non-replicable behavior of uniformity testing algorithms.
method Develops a replicable uniformity tester with improved sample complexity.
result Achieves nearly linear dependence on replicability factor .
Algorithm learns from both labeled and arbitrary test examples, giving guarantees for bounded VC dimension classes.
problem Learning from arbitrary test examples, not just perturbations.
method Selective transductive learning algorithm that outputs abstaining predictions.
result Nontrivial guarantees for bounded VC dimension classes with arbitrary train and test distributions.
Approximate Kernel-based Conditional Independence Tests for Fast Non-Parametric Causal Discoverystat.ME
Constraint-based causal discovery (CCD) algorithms require fast and accurate conditional independence (CI) testing. The Kernel Conditional Independence Test (KCIT) is currently one of the most popular CI tests in the non-parametric setting, but many investigators cannot use KCIT with large datasets because the test sca…
Optimized testing of discrete distributions using predicted data.
problem Testing discrete distributions with reduced sample complexity.
method Adaptable algorithms that use a predicted distribution to reduce sample size.
result Optimal sample complexity improvements with self-adjusting algorithms.
Polynomial delay algorithm tests causal models with hidden variables.
problem Testing causal models with hidden variables in polynomial delay.
method c-component local Markov property (C-LMP) and polynomial delay algorithm.
result First algorithm for poly-delay testing of CIs in causal graphs with hidden variables.
New algorithms control FDX while achieving more power in online multiple testing.
problem Problems with previous online multiple testing methods, including high FDX and low power.
method Developed new dynamic algorithms that adjust testing levels based on accumulated wealth.
result SupLORD algorithm achieves higher power and FDR control in synthetic experiments.
New group testing method uses Belief Propagation for accurate screening.
problem Efficiently identifying infected samples in large groups with minimal tests.
method Belief Propagation algorithm for inference in group testing schemes.
result Significantly increased accuracy of infection identification with fewer tests.
New framework limits testing algorithmic stability under computational constraints.
problem Testing algorithmic stability is computationally hard.
method Unified framework for quantifying stability hardness.
result Exhaustive search is the only universally valid mechanism for certifying stability.
Statistical query algorithms and low-degree tests are nearly equivalent in high-dimensional hypothesis testing.
problem High-dimensional hypothesis testing and information-computation gaps.
method Analysis of statistical query framework and low-degree polynomials.
result Statistical query algorithms and low-degree polynomials are almost equivalent in power under mild conditions.
Algorithm predicts with optimal loss by abstaining from uncertain test examples.
problem Predicting with training data not matching test data.
method Transductive abstention algorithm using labeled and unlabeled test examples.
result Optimal prediction loss guarantees with additional term for abstaining cost.
This paper presents a comparison of six machine learning (ML) algorithms: GRU-SVM (Agarap, 2017), Linear Regression, Multilayer Perceptron (MLP), Nearest Neighbor (NN) search, Softmax Regression, and Support Vector Machine (SVM) on the Wisconsin Diagnostic Breast Cancer (WDBC) dataset (Wolberg, Street, & Mangasarian, 1…
AdaStop improves statistical testing for Deep RL algorithm comparisons.
problem Statistical reproducibility issues in Deep RL.
method AdaStop, a new statistical test based on multiple group sequential tests.
result AdaStop ensures theoretically sound comparisons of Deep RL algorithms.
We consider the problem of asynchronous online testing, aimed at providing control of the false discovery rate (FDR) during a continual stream of data collection and testing, where each test may be a sequential test that can start and stop at arbitrary times. This setting increasingly characterizes real-world applicati…
Test log-likelihood comparisons can be misleading.
problem Misinterpretation of test log-likelihood in model comparison.
method Simple examples of model comparison and forecast accuracy.
result Test log-likelihood does not always correlate with model accuracy.
A test detects unfairness in machine learning classifiers.
problem Detecting and mitigating algorithmic biases in machine learning.
method Optimal transport theory to quantify and mitigate bias.
result Proposes a statistical test for detecting unfair classifiers.
e-LOND algorithm controls FDR in online testing with arbitrary dependencies.
problem Online testing of hypotheses with unknown dependencies.
method e-LOND algorithm for FDR control under arbitrary dependence.
result e-LOND provides more power than existing methods through simulations.
FLOPART solves peak detection by creating accurate train and test set predictions.
problem Correctly detecting peaks in sequential data.
method Dynamic programming changepoint algorithm with zero train label errors.
result FLOPART provides highly accurate predictions on both train and test sets.
Near-optimal algorithms for private estimation and sequential testing of collision probabilitystat.ML
New algorithms estimate and test collision probability with near-optimal sample complexity.
problem Estimating and testing collision probability in discrete distributions.
method Developed algorithms for -local differential privacy and sequential testing.
result Achieved nearly optimal sample complexity for estimating and testing collision probability.
A new algorithm reduces CI tests for causal graph recovery.
problem Exponential CI tests limit causal discovery algorithms.
method CCPG (Causal Consistent Partition Graph) with polynomial CI tests.
result CCPG efficiently recovers causal graph with polynomial tests.
This paper explores two classes of model adaptation methods for Web search ranking: Model Interpolation and error-driven learning approaches based on a boosting algorithm. The results show that model interpolation, though simple, achieves the best results on all the open test sets where the test data is very different …
A Methodology for Customizing Clinical Tests for Esophageal Cancer based on Patient Preferencescs.LG
Tests for Esophageal cancer can be expensive, uncomfortable and can have side effects. For many patients, we can predict non-existence of disease with 100% certainty, just using demographics, lifestyle, and medical history information. Our objective is to devise a general methodology for customizing tests using user pr…
Optimizes group testing for COVID-19 to reduce test numbers.
problem Minimizing tests for accurate infection detection.
method Bayesian approach with genetic algorithms and sub-modularity.
result Greedy-adaptive method provides theoretical guarantees.
Unified framework for structure learning via conditional independence testing.
problem Optimal structure learning and conditional independence testing.
method Established a fundamental connection and reduction between structure learning and conditional independence testing.
result Optimal rates for structure learning are determined by conditional independence testing rates.
Testing MCMC codecs.SE
Markov Chain Monte Carlo (MCMC) algorithms are a workhorse of probabilistic modeling and inference, but are difficult to debug, and are prone to silent failure if implemented naively. We outline several strategies for testing the correctness of MCMC algorithms. Specifically, we advocate writing code in a modular way, w…
New Bayesian optimization models for efficient material screening.
problem Efficiently screening materials with expensive and cheap tests.
method Flexible multi-test Bayesian optimization models with complex relationships.
result Demonstrated power on synthetic and real data.
Paper proposes a new method for more accurate group testing of infected patients.
problem Identifying infected patients efficiently with reduced tests and corrected errors.
method Adaptive design of pools based on Bayesian posterior prediction using belief propagation algorithm.
result The proposed method results in more accurate identification of infected patients.
Paper proposes a statistical test for feature selection pipelines using selective inference.
problem Assessing the significance of feature selection pipelines in data analysis.
method Selective inference technique applied to feature selection pipelines composed of various algorithms.
result The proposed statistical test controls false positive feature selection probabilities.
In literature there are several studies on the performance of Bayesian network structure learning algorithms. The focus of these studies is almost always the heuristics the learning algorithms are based on, i.e. the maximisation algorithms (in score-based algorithms) or the techniques for learning the dependencies of e…
New algorithm improves on static methods in Active Simple Hypothesis Testing.
problem Optimizing Active Simple Hypothesis Testing with active sampling.
method Game-theoretic formulation, differential games, PDEs, Blackwell Approachability.
result Proposes an efficient algorithm that outperforms static methods in ASHT.
Study improves statistical power for detecting algorithmic bias in educational data.
problem Challenges in measuring algorithmic bias using ABROCA due to skewed distribution.
method Investigates ABROCA's distributional properties and proposes nonparametric randomization tests.
result ABROCA-based bias assessments are underpowered in typical EDM sample sizes.
New methods test discrete distributions faster with local privacy constraints.
problem Testing discrete distributions under local differential privacy constraints.
method Efficient randomized algorithms and test procedures, both non-interactive and interactive.
result Faster separation rates in interactive privacy mechanisms.
Proposes a new TS algorithm for non-stationary bandits using KS tests.
problem Non-stationary multi-armed bandit problems.
method Active detection of change points using KS tests and adaptive Thompson Sampling.
result Sub-linear regret demonstrated for the two-armed bandit case.
Private online FDR control for adaptive testing under differential privacy.
problem Controlling false discoveries in adaptive multiple hypothesis testing with privacy constraints.
method Private online algorithms based on non-private results, ensuring privacy and statistical performance.
result Strong guarantees for privacy and statistical performance in FDR and power.
Comparison Lift uses bandit algorithms to optimize online ad testing.
problem Optimizing online ad testing to maximize click-through rates.
method Bandit-based experimentation algorithm that adapts to test results.
result Ad click-through rates increased by 46% on average.
Study provides selective inference method for latent block models.
problem Challenges in constructing a test on a block structure selected by clustering algorithms.
method Developed a selective inference method for latent block models using squared residue minimization and simulated annealing.
result Proposed tests effectively handle selective bias in block structures compared to naive tests.
Early stopping of iterative algorithms is an algorithmic regularization method to avoid over-fitting in estimation and classification. In this paper, we show that early stopping can also be applied to obtain the minimax optimal testing in a general non-parametric setup. Specifically, a Wald-type test statistic is obtai…
New algorithm tests model calibration in nearly-linear time.
problem Testing model calibration from samples efficiently.
method Reformulated as minimum-cost flow, solved with dynamic programming.
result Optimal testing problem solved in nearly-linear time.
Efficient Training of Deep Classifiers for Wireless Source Identification using Test SNR Estimatescs.LG
Efficient deep learning for wireless source identification using test SNR estimates.
problem Training deep classifiers for wireless signal modulation and technology identification.
method Greedy training SNR Boosting and bootstrap aggregating (Bagging) based on training SNR values.
result Uniform improvement in accuracy across all SNR values with small subsets of training SNR values.
New testing method for robust actor-critic bandit algorithms.
problem Balancing data collection for app performance and user adherence.
method Modified actor-critic algorithm and novel testing procedure.
result Testing procedure is robust to critic misspecification.
OmniMatch algorithm perfectly matches graphs without edge correlation.
problem Graph matching in the absence of edge correlation.
method OmniMatch algorithm for seeded multiple graph matching.
result OmniMatch aligns unseeded vertices across multiple networks efficiently and perfectly.
Meta two-sample testing uses auxiliary data to quickly find powerful tests from limited samples.
problem Challenges in identifying powerful kernels for distinguishing complex distributions with limited data.
method Introduces meta two-sample testing (M2ST) to leverage abundant auxiliary data on related tasks.
result Proposed algorithms improve over baselines and identify powerful tests from scarce observations.