A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Training on mixed distributions improves test performance even when components are unrelated.
Paper proposes a new method for more accurate group testing of infected patients.
The study examines how extra compute during testing affects the performance of large language models.
Test-asset construction affects factor model performance.
A wild bootstrap method for nonparametric hypothesis tests based on kernel distribution embeddings is proposed. This bootstrap method is used to construct provably consistent tests that apply to random processes, for which the naive permutation-based bootstrap fails. It applies to a large group of kernel tests based on…
Risk monitoring detects when TTA models degrade at test time.
We characterize the asymptotic performance of nonparametric goodness of fit testing. The exponential decay rate of the type-II error probability is used as the asymptotic performance metric, and a test is optimal if it achieves the maximum rate subject to a constant level constraint on the type-I error probability. We …
Improves test set performance and reduces out-of-sample disappointment for unstable models.
This paper applies combinatorial testing to machine learning for robust model performance.
Simple policy search outperforms advanced learnable test-time augmentation techniques.
We propose a novel adaptive test of goodness-of-fit, with computational cost linear in the number of samples. We learn the test features that best indicate the differences between observed samples and a reference model, by minimizing the false negative rate. These features are constructed via Stein's method, meaning th…
Study evaluates RKHS choices for assessing graph models using KSD tests.
Statistical tests that compare classification algorithms are univariate and use a single performance measure, e.g., misclassification error, measure, AUC, and so on. In multivariate tests, comparison is done using multiple measures simultaneously. For example, error is the sum of false positives and false negatives…
Private online FDR control for adaptive testing under differential privacy.
In this paper we consider a Lagrange Multiplier-type test (LM) to detect change in the mean of time series with heteroskedasticity of unknown form. We derive the limiting distribution under the null, and prove the consistency of the test against the alternative of either an abrupt or smooth changes in the mean. We perf…
The relation between performance and stress is described by the Yerkes-Dodson Law but varies significantly between individuals. This paper describes a method for determining the individual optimal performance as a function of physiological signals. The method is based on attention and reasoning tests of increasing comp…
Efficiently tests two distributions with few label queries.
A permutation-based SW test achieves minimax-optimal power for two-sample testing.
Meta two-sample testing uses auxiliary data to quickly find powerful tests from limited samples.
A new framework improves kernel Stein discrepancy tests for validating distributions.
The statistical comparison of multiple algorithms over multiple data sets is fundamental in machine learning. This is typically carried out by the Friedman test. When the Friedman test rejects the null hypothesis, multiple comparisons are carried out to establish which are the significant differences among algorithms. …
Paper extends Chernoff sampling for active testing and parameter estimation, improving neural network and regression models.
AdaStop improves statistical testing for Deep RL algorithm comparisons.
Bayesian methods improve group testing for identifying infected patients.
This paper explores two classes of model adaptation methods for Web search ranking: Model Interpolation and error-driven learning approaches based on a boosting algorithm. The results show that model interpolation, though simple, achieves the best results on all the open test sets where the test data is very different …
Classification is a fundamental problem in machine learning and data mining. During the past decades, numerous classification methods have been presented based on different principles. However, most existing classifiers cast the classification problem as an optimization problem and do not address the issue of statistic…
New algorithm reduces conditional independence tests needed for causal discovery.
Statistical test evaluates if personalizing interventions is cost-effective.
We conduct an extensive evaluation of price jump tests based on high-frequency financial data. After providing a concise review of multiple alternative tests, we document the size and power of all tests in a range of empirically relevant scenarios. Particular focus is given to the robustness of test performance to the …
Concolic testing combines program execution and symbolic analysis to explore the execution paths of a software program. This paper presents the first concolic testing approach for Deep Neural Networks (DNNs). More specifically, we formalise coverage criteria for DNNs that have been studied in the literature, and then d…
Improved hypothesis testing and change-point detection using diffusion-based methods.
A study on optimizing data augmentation weights for improved test-time predictions.
A new CVaR test reduces group performance disparity detection complexity.
Discusses MultiFIT for multivariate dependence, comparing it to HSIC tests.
We investigated the impact of noisy linguistic features on the performance of a Japanese speech synthesis system based on neural network that uses WaveNet vocoder. We compared an ideal system that uses manually corrected linguistic features including phoneme and prosodic information in training and test sets against a …
In biospectroscopy, suitably annotated and statistically independent samples (e. g. patients, batches, etc.) for classifier training and testing are scarce and costly. Learning curves show the model performance as function of the training sample size and can help to determine the sample size needed to train good classi…
We introduce hyppo, a unified library for performing multivariate hypothesis testing, including independence, two-sample, and k-sample testing. While many multivariate independence tests have R packages available, the interfaces are inconsistent and most are not available in Python. hyppo includes many state of the art…
A new method reduces hyperparameter tuning evaluations by using sequential tests.
This paper evaluates test selection methods for deep neural networks, revealing their limitations.
New insights into SGD and generalization via shift-curvature and bias-curvature mechanisms.
This work constructs a hypothesis test for detecting whether an data-generating function belongs to a specific reproducing kernel Hilbert space , where the structure of is only partially known. Utilizing the theory of reproducing kernels, we reduce this hypothesis …
We perform a finite sample analysis of the detection levels for sparse principal components of a high-dimensional covariance matrix. Our minimax optimal test is based on a sparse eigenvalue statistic. Alas, computing this test is known to be NP-complete in general, and we describe a computationally efficient alternativ…
DRIFT uses RL to automate functional software testing efficiently.
We characterize the asymptotic performance of nonparametric one- and two-sample testing. The exponential decay rate or error exponent of the type-II error probability is used as the asymptotic performance metric, and an optimal test achieves the maximum rate subject to a constant level constraint on the type-I error pr…
Building machine translation (MT) test sets is a relatively expensive task. As MT becomes increasingly desired for more and more language pairs and more and more domains, it becomes necessary to build test sets for each case. In this paper, we investigate using Amazon's Mechanical Turk (MTurk) to make MT test sets chea…
The study examines how verifier imperfections impact test-time scaling techniques.
This technical report describes a practical field test on word-image classification in a very large collection of more than 300 diverse handwritten historical manuscripts, with 1.6 million unique labeled images and more than 11 million images used in testing. Results indicate that several deep-learning tests completely…
Given two networks with the same training loss on a dataset, when would they have drastically different test losses and errors? Better understanding of this question of generalization may improve practical applications of deep networks. In this paper we show that with cross-entropy loss it is surprisingly simple to ind…