Research
On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,695 papers · 148 categories

Trend · papers per month

76152227303 · Jun 202019922001200920172026
48 results for Runs test

Testing the implementation of deep learning systems and their training routines is crucial to maintain a reliable code base. Modern software development employs processes, such as Continuous Integration, in which changes to the software are frequently integrated and tested. However, testing the training routines requir…

2019-01-14abs ↗pdf ↗

A new protocol evaluates small machine learning improvements conservatively.

problem Uncertainty in small gains reported in machine learning papers.
method Paired bootstrap protocol with BCa confidence intervals and sign-flip permutation tests.
result Conservative evaluation reduces over-claiming of small improvements.

Statistical test verifies long-term rating system calibration with overlapping time windows.

problem Verifying supervisory requirements for overlapping time windows in rating systems.
method Analyzes long-run default rate distribution and correlation effects; presents conservative calibration test methods.
result Developed a test for individual and portfolio levels that can handle unknown variance.

A new test statistic counts tree co-occurrences to detect edge correlation between networks.

problem Detecting edge correlation between networks using latent vertex correspondence.
method The test statistic is based on counting co-occurrences of signed trees for a family of non-isomorphic trees.
result The test runs in n2+o(1)n^{2+o(1)} time and succeeds with high probability for large nn.

The study finds a long-term relationship between Dubai crude oil and US natural gas prices.

problem Examining the relationship between Dubai crude oil and US natural gas prices.
method Used unit root and cointegration tests, ARDL cointegration technique, and Toda-Yamamoto causality test.
result There is a long-run relationship with unidirectional causality from Dubai crude oil to US natural gas.

Markov Chain Monte Carlo (MCMC) algorithms are a workhorse of probabilistic modeling and inference, but are difficult to debug, and are prone to silent failure if implemented naively. We outline several strategies for testing the correctness of MCMC algorithms. Specifically, we advocate writing code in a modular way, w…

2014-12-16abs ↗pdf ↗

We propose an alternative framework to existing setups for controlling false alarms when multiple A/B tests are run over time. This setup arises in many practical applications, e.g. when pharmaceutical companies test new treatment options against control pills for different diseases, or when internet companies test the…

2017-06-16abs ↗pdf ↗

Bayesian optimization for long-term outcomes using fast and slow experiments.

problem Optimizing long-term system effects with short-term misleading results.
method Combining fast and slow experiments for Bayesian optimization.
result Sequential optimization over large action spaces in a short time.

We implement a Tensor Train layer in the TensorFlow Neural Machine Translation (NMT) model using the t3f library. We perform training runs on the IWSLT English-Vietnamese '15 and WMT German-English '16 datasets with learning rates {0.0004,0.0008,0.0012}\in \{0.0004,0.0008,0.0012\}, maximum ranks {2,4,8,16}\in \{2,4,8,16\} and a range of core dime…

2019-11-05abs ↗pdf ↗

The paper introduces a diagnostic method to detect grokking transitions in models before test accuracy improves.

problem Detecting the transition from training to generalization in machine learning models.
method Summarize task-dependent observables as empirical distributions, map them to Wasserstein/quantile coordinates, and analyze using Hankel dynamic mode decomposition.
result The diagnostic method achieves AUROC \(\approx\) 0.93 for grokking-vs-non-grokking discrimination at the run level.

Automated tests detect interactions in unstructured data.

problem Detecting interactions between latent variables in low-dimensional systems.
method Derive two interaction tests based on pairwise interventions and integrate them into an active learning pipeline.
result Tests can identify more known biological interactions than random search and standard active learning baselines.

In this paper we investigate the adaptive market efficiency of the agricultural commodity futures market, using a sample of eight futures contracts. Using a battery of nonlinear tests, we uncover the nonlinear serial dependence in the returns series. We run the Hinich portmanteau bicorrelation test to uncover the momen…

2014-12-27abs ↗pdf ↗

In this paper, we present our approach to solve a physics-based reinforcement learning challenge "Learning to Run" with objective to train physiologically-based human model to navigate a complex obstacle course as quickly as possible. The environment is computationally expensive, has a high-dimensional continuous actio…

2017-11-18abs ↗pdf ↗

Bayesian method improves adaptive testing item selection, ensuring full item exposure.

problem Adaptive testing selects items to estimate ability, but must also ensure diverse item exposure.
method Formulated as Bayesian model averaging, deriving optimal item sampling probabilities.
result Stochastic method achieves full item bank exposure without sacrificing accuracy.

SEFR is a fast, energy-efficient classifier for ultra-low power devices.

problem Running machine learning on battery-powered devices is challenging due to time and energy constraints.
method SEFR is an ultra-low power classifier with linear time complexity for training and testing.
result SEFR is 63 times faster and 70 times more energy efficient than state-of-the-art classifiers.

Data that is gathered adaptively --- via bandit algorithms, for example --- exhibits bias. This is true both when gathering simple numeric valued data --- the empirical means kept track of by stochastic bandit algorithms are biased downwards --- and when gathering more complicated data --- running hypothesis tests on c…

2018-06-06abs ↗pdf ↗

A new computationally efficient dependence measure, and an adaptive statistical test of independence, are proposed. The dependence measure is the difference between analytic embeddings of the joint distribution and the product of the marginals, evaluated at a finite set of locations (features). These features are chose…

2016-10-15abs ↗pdf ↗

Unified model for network risks, including bilateral and central clearing, with practical applications.

problem Managing risks in financial networks with multiple trading types.
method Developed a one-period XVA model with explicit formulas for various quantities.
result Illustrated practical uses for stress testing and portfolio optimization.

AutoPC optimizes hyperparameters for the PC algorithm to improve its performance.

problem The unsupervised nature of the PC algorithm makes it difficult to tune the Type I αα level.
method AutoPC optimizes αα directly for a chosen metric and ensures stability through a second run.
result AutoPC consistently outperforms state-of-the-art methods across multiple metrics.

As machine learning algorithms enter applications in industrial settings, there is increased interest in controlling their cpu-time during testing. The cpu-time consists of the running time of the algorithm and the extraction time of the features. The latter can vary drastically when the feature set is diverse. In this…

2012-06-27abs ↗pdf ↗

Active testing for large language models is made more efficient and accurate.

problem Efficient evaluation of large language models with limited labels.
method Cost-saving measures and in-context learning for constructing a surrogate model.
result Significantly more accurate evaluations of LLM performance compared to random data acquisition.

GTBO uses group testing to optimize high-dimensional functions efficiently.

problem Challenges in optimizing high-dimensional, expensive functions due to the curse of dimensionality.
method GTBO combines testing and optimization phases to identify active variables and guide efficient optimization.
result GTBO outperforms state-of-the-art methods on high-dimensional optimization tasks.

New test for SGD in binary classification reduces computation time.

problem Determining optimal stopping for SGD in binary classification.
method Proposes a new, simple, computationally inexpensive termination criterion for SGD.
result Termination criterion reduces expected misclassification probability.

New method tests independence with single nonstationary time series.

problem Testing independence in nonstationary nonlinear time series.
method Time-varying nonlinear regression, local long-run covariance estimation, strong Gaussian approximation.
result First framework for conditional independence testing with a single realization of a nonstationary nonlinear process.

We present an efficient algorithm for simultaneously training sparse generalized linear models across many related problems, which may arise from bootstrapping, cross-validation and nonparametric permutation testing. Our approach leverages the redundancies across problems to obtain significant computational improvement…

2013-07-31abs ↗pdf ↗