Many methods for automated software test generation, including some that explicitly use machine learning (and some that use ML more broadly conceived) derive new tests from existing tests (often referred to as seeds). Often, the seed tests from which new tests are derived are manually constructed, or at least simpler t…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “test case generation”
Research proposes a test case generation system for deep learning models using dataset properties.
problem Automated generation of extensive test cases for deep learning models is challenging.
method Measures dataset quality and proposes a test case generation system guided by dataset properties.
result Systematic test case generation for deep learning models is effective.
The increasing inclusion of Deep Learning (DL) models in safety-critical systems such as autonomous vehicles have led to the development of multiple model-based DL testing techniques. One common denominator of these testing techniques is the automated generation of test cases, e.g., new inputs transformed from the orig…
New methods test discrete distributions faster with local privacy constraints.
problem Testing discrete distributions under local differential privacy constraints.
method Efficient randomized algorithms and test procedures, both non-interactive and interactive.
result Faster separation rates in interactive privacy mechanisms.
This paper proposes a method to generate realistic test cases for image classifiers.
problem Ensuring neural networks for image classification are correct with adequate realistic test data.
method Captures patterns in a large input data space using a manifold, then generates fault-revealing test cases.
result Generates thousands of realistic yet fault-revealing test cases efficiently for well-trained models.
Automated testing improves deep learning model accuracy by 259.2%.
problem Ensuring robustness of deep learning models through automated testing.
method Jointly optimizing differential behaviors and neuron coverage; generating corner-cases; applying transformations and GANs.
result Deep learning model accuracy increased by 259.2% using automated generated corner cases.
Robust hypothesis testing designs a test for worst-case distributions using kernel methods.
problem Design a robust test for hypothesis testing under uncertainty sets.
method Data-driven uncertainty sets constructed using kernel mean embeddings and maximum mean discrepancy (MMD). Bayesian and Neyman-Pearson settings investigated.
result Proposed robust kernel tests are exponentially consistent and asymptotically optimal.
Study on continuous sequence classification with distribution uncertainty.
problem Classifying continuous sequences with varying distribution uncertainty.
method Proposes distribution-free tests for three test designs: fixed-length, sequential, and two-phase tests.
result Error probabilities decay exponentially fast for all test designs.
We propose a kernel-based nonparametric test of relative goodness of fit, where the goal is to compare two models, both of which may have unobserved latent variables, such that the marginal distribution of the observed variables is intractable. The proposed test generalizes the recently proposed kernel Stein discrepanc…
Develops GLRT for defending against adversarial attacks in hypothesis testing.
problem Adversarial attacks on machine learning models causing misclassification.
method Generalized likelihood ratio test applied to composite hypothesis testing problem.
result GLRT approach yields competitive robustness-accuracy tradeoff under various attacks.
Callisto: Entropy based test generation and data quality assessment for Machine Learning Systemscs.LG
CALLISTO generates tests and assesses ML data quality using prediction entropy.
problem Validating ML systems for accuracy and data quality.
method Entropy-based test generation and data quality assessment framework.
result CALLISTO detects up to 20x more errors than traditional methods.
A new method for testing deep learning models using CGF.
problem Traditional testing methods fail to cover corner cases in DNNs.
method Monte Carlo Tree Search for coverage-guided search.
result Generated inputs result in higher coverage than previous methods.
The paper tests hypotheses on two Lévy process-driven streams of observations.
problem Testing hypotheses on two Lévy process-driven streams of observations.
method Infinitesimal generators and super/sub-solutions are used to compute bounds and analyze the model.
result Bounds for infinitesimal generators are computed in terms of super/sub-solutions.
We investigate multiple testing and variable selection using the Least Angle Regression (LARS) algorithm in high dimensions under the assumption of Gaussian noise. LARS is known to produce a piecewise affine solution path with change points referred to as the knots of the LARS path. The key to our results is an express…
[Context:] Model-based testing is an instrument for automated generation of test cases. It requires identifying requirements in documents, understanding them syntactically and semantically, and then translating them into a test model. One light-weight language for these test models are Cause-Effect-Graphs (CEG) that ca…
New insights into CI tests reveal key factors for practical performance.
problem Understanding and improving CI tests in practical applications.
method Investigation of the Kernel-based Conditional Independence (KCI) test and analysis of its practical behavior.
result Errors in conditional mean embedding estimates and appropriate conditioning kernel selection are crucial for CI tests.
Two geometric tests for forward-flatness are shown to be dual.
problem Checking forward-flatness in discrete-time systems.
method Two geometric tests based on involutive distributions and integrable codistributions.
result The two tests are dual to each other.
Private CI tests for continuous Z with privacy constraints.
problem Testing conditional independence under differential privacy constraints.
method Developed two private CI testing procedures based on generalized covariance and conditional randomization tests.
result First private CI tests with rigorous theoretical guarantees for continuous Z.
How Reliable are Test Numbers for Revealing the COVID-19 Ground Truth and Applying Interventions?cs.SI
Agent-based model compares different COVID-19 testing policies and their effectiveness.
problem Understanding how different testing policies reveal the true number of infected cases.
method Developed an agent-based simulation framework in Python to model various testing policies and interventions.
result Contact Tracing consistently captures more positive cases than Random Symptomatic Testing, and LBT performs similarly.
We extend the problem of finding Hamiltonian-invariant volume forms on a Poisson manifold to the problem of construction of Hamiltonian-invariant generalized functions. For this we introduce the notion of generalized center of a Poisson algebra, which is the space of generalized Casimir functions. We study as the case …
Statistical tests for fairness in admissions data reveal hidden patterns.
problem Simpson's paradox in admissions data hides true gender bias.
method Introduces a new statistical test based on Pearl's instrumental-variable inequalities.
result Statistical tests for fairness coincide with causal notions for the Berkeley admissions case.
New algorithms for decision trees with noisy outcomes improve learning efficiency.
problem Learning with noisy outcomes in active learning.
method Approximation algorithms for optimal decision trees with persistent noise.
result Approximation algorithms provide nearly optimal performance guarantees.
Polynomial-time test for detecting dense subgraphs in heterogeneous networks.
problem Detecting a planted community in heterogeneous networks.
method Proposes a polynomial-time test with a standard normal distribution null limiting distribution.
result The test is efficient and performs well in both simulations and real data.
New algorithms test independence with fewer samples by using predictive information.
problem Testing independence of distributions with limited samples.
method Augmented distribution testing framework that incorporates predictive information.
result Optimal sample complexity achieved, matching lower bounds.
Proposes a new model for testing causal structural priors and synthesizing data.
problem Testing and synthesizing causal structural priors using nonparametric knowledge and neural networks.
method Causal Structural Hypothesis Testing (C-SHT) and Causal Structural Variational Hypothesis Testing (C-SVHT) using deep neural networks.
result Demonstrates out-of-distribution generalization error as a proxy for causal structural prior hypothesis testing.
Deep-learning method improves hypothesis testing for independence.
problem Improving hypothesis testing for independence using deep learning.
method Proposes deep-testing, a novel procedure that uses a deep neural network to distinguish between data generated under and outside a given statistical model.
result Deep-testing achieves the highest overall power against nineteen competing methods across various dependence structures.
Develops a hypothesis testing framework for generalized Thurstone models.
problem Determining whether pairwise comparison data fits a generalized Thurstone model.
method Introduces separation distance and derives upper and lower bounds for testing.
result Critical threshold for testing depends on observation graph topology and scales as for complete graphs.
We present an extension of the Kolmogorov-Smirnov (KS) two-sample test, which can be more sensitive to differences in the tails. Our test statistic is an integral probability metric (IPM) defined over a higher-order total variation ball, recovering the original KS test as its simplest case. We give an exact representer…
TARP tests accuracy of generative posterior estimators.
problem Assessing the accuracy of posterior estimators from generative models.
method TARP coverage testing method.
result TARP can detect inaccurate inferences in high-dimensional spaces.
New method reveals good classifiers are common in over-parameterized models.
problem Understanding how over-parameterized models generalize well.
method Developed a methodology to compute the full distribution of test errors.
result Test errors concentrate around a small typical value ε* rather than the worst-case model.
We present a novel family of nonparametric omnibus tests of the hypothesis that two unknown but estimable functions are equal in distribution when applied to the observed data structure. We developed these tests, which represent a generalization of the maximum mean discrepancy tests described in Gretton et al. [2006], …
Adaptive auditing improves AI robustness testing with anytime-valid guarantees.
problem Cost and time of annotation limit rigorous AI failure mode characterization.
method Introduces hypothesis testing framework for adaptive audits using SAVI.
result Proves anytime-valid type-I error control and robustness certification.
Optimal testing for densities under local differential privacy constraints.
problem Testing goodness-of-fit for densities under privacy constraints.
method Estimation of quadratic distance and minimax separation rates.
result First minimax optimal test under local differential privacy constraints.
Paper defends machine learning models from adversarial attacks using GLRT.
problem Adversarial attacks on machine learning models leading to misclassification.
method Generalized likelihood ratio test (GLRT) for robust classification.
result GLRT yields performance competitive with minimax approach under worst-case attacks, and better trade-off under weaker attacks.
We present a new test for studying asphericity and diagrammatic reducibility of group presentations. Our test can be applied to prove diagrammatic reducibility in cases where the classical weight test fails. We use this criterion to generalize results of J. Howie and S.M. Gersten on asphericity of LOTs and of Adian pre…
Develops hypothesis tests for conditional distributions using learning-theoretic bounds.
problem Testing differences in conditional distributions and functionals.
method Transforming learning-theoretic bounds into hypothesis tests for conditional expectations.
result Establishes comprehensive foundation for conditional testing, including theoretical guarantees and practical implementations.
Unified framework for structure learning via conditional independence testing.
problem Optimal structure learning and conditional independence testing.
method Established a fundamental connection and reduction between structure learning and conditional independence testing.
result Optimal rates for structure learning are determined by conditional independence testing rates.
New framework for valid hypothesis testing in complex data settings.
problem Challenges in classical hypothesis testing frameworks.
method Add and subtract external noise to partition data, orthogonalize, and test hypotheses.
result Valid hypothesis tests can be conducted under minimal assumptions.
Paper shows how to integrate quantization into neural compression models.
problem Integrating quantization into neural compression models.
method Integrates uniform noise channel at test time using universal quantization.
result Eliminates mismatch between training and test phases while maintaining differentiability.
Conditional Independence Testing with Heteroskedastic Data and Applications to Causal Discoverystat.ME
Improved CI test for heteroskedastic data enhances causal discovery.
problem CI testing assumptions fail in heteroskedastic data.
method Adapted partial correlation CI test for heteroskedastic noise.
result The adapted test outperforms standard CI test in heteroskedastic cases.
Since its inception, the modus operandi of multi-task learning (MTL) has been to minimize the task-wise mean of the empirical risks. We introduce a generalized loss-compositional paradigm for MTL that includes a spectrum of formulations as a subfamily. One endpoint of this spectrum is minimax MTL: a new MTL formulation…
Finite-Sample Properties of Model Specification Tests for Multivariate Dynamic Regression Modelsecon.EM
Proposes a new test for validating multivariate dynamic regression models.
problem Inadequate exogeneity conditions for conventional model specification tests in dynamic systems.
method Develops a generalized Durbin estimator for multiple-equation systems with dynamic dependencies, and constructs Wald tests.
result Bootstrap-based Wald tests improve finite-sample size control and validate the null hypothesis in multifactor models.
Deep neural nets optimize kernel parameters for non-parametric two-sample tests.
problem Determining if two samples come from the same distribution.
method Deep kernels trained to maximize test power, adapting to distribution smoothness and shape.
result Deep kernels outperform simpler kernels in high dimensions and complex data.
Develops a method for stress testing correlations of financial portfolios.
problem Stress testing correlations in financial asset portfolios.
method Parametric representation of correlations, Bayesian variable selection, joint distribution of stress scenarios.
result Inference of worst-case correlation scenarios using stress tests.
The reproducing kernel Hilbert space (RKHS) embedding of distributions offers a general and flexible framework for testing problems in arbitrary domains and has attracted considerable amount of attention in recent years. To gain insights into their operating characteristics, we study here the statistical performance of…
Testing that a Local Optimum of the Likelihood is Globally Optimum using Reparameterized Embeddingseess.SP
Many mathematical imaging problems are posed as non-convex optimization problems. When numerically tractable global optimization procedures are not available, one is often interested in testing ex post facto whether or not a locally convergent algorithm has found the globally optimal solution. When the problem is formu…
We propose a class of nonparametric two-sample tests with a cost linear in the sample size. Two tests are given, both based on an ensemble of distances between analytic functions representing each of the distributions. The first test uses smoothed empirical characteristic functions to represent the distributions, the s…
A minimalist approach generates synthetic tabular data with sparse PCA and XGBoost.
problem Generating robust synthetic tabular data for model testing.
method Minimalistic unsupervised SparsePCA encoder with XGBoost decoder.
result The method provides an alternative to raw and quantile perturbation for model robustness testing.