Research
On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,657 papers · 148 categories

Trend · papers per month

4182123164 · May 202619922001200920172026
48 results for Runtime Monitoring Language

Enhances reward specification in RL with a novel language-based approach.

problem Reward specification in RL can lead to unintended, potentially harmful behaviours.
method Developed a novel class of language-based Reward Machines using RML's built-in memory.
result Can specify non-regular, non-Markovian reward functions for complex tasks.

For using neural networks in safety critical domains, it is important to know if a decision made by a neural network is supported by prior similarities in training. We propose runtime neuron activation pattern monitoring - after the standard training process, one creates a monitor by feeding the training data to the ne…

2018-09-18abs ↗pdf ↗

GNMR controls runtime stability in low-precision language model training.

problem Efficient low-precision training faces numerical risks at specific operators.
method GNMR compares gradient norms to historical means, applying bounded recovery actions.
result GNMR preserves high-fidelity quality with sparse, budgeted recovery.

AI systems that explain their decisions can be monitored for harmful intentions.

problem Monitoring AI systems' decision-making processes for harmful intentions is imperfect and can miss some misbehavior.
method Monitoring the chain of thought (CoT) of AI systems that communicate in human language.
result CoT monitoring is a promising but fragile approach to AI safety.

The increasing use of machine-learning (ML) enabled systems in critical tasks fuels the quest for novel verification and validation techniques yet grounded in accepted system assurance principles. In traditional system development, model-based techniques have been widely adopted, where the central premise is that abstr…

2020-02-08abs ↗pdf ↗

We describe the concept of logical scaffolds, which can be used to improve the quality of software that relies on AI components. We explain how some of the existing ideas on runtime monitors for perception systems can be seen as a specific instance of logical scaffolds. Furthermore, we describe how logical scaffolds ma…

2019-09-12abs ↗pdf ↗

Novel method detects faults in helicopter transmissions using healthy data only.

problem Detecting faults in rare events in helicopter transmissions.
method Probabilistic anomaly detection using Bayesian measures.
result Method achieves competitive detection performance in real-world and benchmark applications.

Infrastructure monitors AI/ML radiology models across multiple sites.

problem Monitoring and improving AI/ML radiology models across multiple sites.
method Interactive radiology reporting, centralized cloud system, post-marketing surveillance.
result Efficient monitoring and iterative development of AI/ML models without radiologist burden.

Neural networks have demonstrated unmatched performance in a range of classification tasks. Despite numerous efforts of the research community, novelty detection remains one of the significant limitations of neural networks. The ability to identify previously unseen inputs as novel is crucial for our understanding of t…

2019-11-20abs ↗pdf ↗

Enhances anomaly detection in financial markets using AI agents.

problem Manual verification of financial market anomalies is time-consuming and error-prone.
method A multi-agent LLM framework for automated anomaly detection.
result Framework reduces human intervention and improves efficiency and accuracy.

Despite the eminent successes of deep neural networks, many architectures are often hard to transfer to irregularly-sampled and asynchronous time series that commonly occur in real-world datasets, especially in healthcare applications. This paper proposes a novel approach for classifying irregularly-sampled time series…

2019-09-26abs ↗pdf ↗

SpeakerStew verifies 46 languages with reduced training and inference costs.

problem Speaker verification for 46 languages with smart speaker interactions.
method Pooling multilingual data, triage between text-dependent and text-independent models.
result Training on multiple languages generalizes well and reduces computational requirements.

ACE models allow flexible conditioning and prediction of latent variables.

problem Lack of flexibility in conditioning and prediction of latent variables in probabilistic models.
method Introduces Amortized Conditioning Engine (ACE) that explicitly represents latent variables and allows runtime conditioning and prediction.
result ACE models outperform existing methods in diverse tasks like image completion, classification, Bayesian optimization, and simulation-based inference.

ZDP detects drift in large language models without labels, proving key theorems and metrics.

problem Detecting drift in large language models without task labels or output evaluations.
method Zero-Direction Probing (ZDP) framework based on null directions of transformer activations, proving theoretical guarantees.
result Proves the Variance--Leak Theorem, Fisher Null-Conservation, Rank--Leak bound, and logarithmic-regret guarantee.

Estimating the dependency of variables is a fundamental task in data analysis. Identifying the relevant attributes in databases leads to better data understanding and also improves the performance of learning algorithms, both in terms of runtime and quality. In data streams, dependency monitoring provides key insights …

2018-10-04abs ↗pdf ↗

Tribuo integrates ML models with provenance in Java.

problem Tracking and ensuring ML models behave appropriately as they grow in number and complexity.
method Integrates model training, inference, type-safety, runtime checking, and automatic provenance recording into a single Java library.
result Automatic recording of full processing pipeline and training details inside the model object.

StockTime predicts stock prices more accurately using LLMs and time series data.

problem Challenges in integrating time series data and natural language for stock price prediction.
method StockTime is a specialized LLM architecture that integrates textual and time series data to predict stock prices.
result StockTime outperforms recent LLMs in predicting stock prices with more accuracy.

Model predicts counterfactuals under domain shift and inaccessible variables.

problem Runtime domain corruption impairs counterfactual prediction.
method Subsumes counterfactual prediction under domain adaptation, uses adversarial domain adaptation to reduce distribution disparity.
result VEGAN outperforms baselines in individual-level treatment effect estimation.

PClean automates Bayesian data cleaning for specific datasets.

problem Bayesian inference for diverse and complex data cleaning.
method Domain-specific probabilistic programming language with custom models and inference.
result PClean programs outperform general-purpose PPLs in accuracy and runtime.

CoinTossX is a low-latency, open-source matching engine for financial trading.

problem Efficiently matching orders in financial markets with low latency and high throughput.
method Developed in Java, orders submitted via UDP SBE, low-latency message transport (Aeron Media Driver). Separates order generation and matching.
result Demonstrated low-latency, high-throughput performance in various deployment scenarios.

The paper proposes incentivizing human annotators with 'golden questions' to improve data quality.

problem Ensuring high-quality human annotations for training large language models.
method A principal-agent model is used to incentivize annotators with bonuses based on the maximum likelihood estimators (MLE) of their annotations. Hypothesis testing is applied to monitor the annotators' performance.
result The hypothesis testing rate for the principal-agent model is of Θ(1/nlogn)Θ(1/\sqrt{n \log n}), highlighting the importance of 'golden questions' for monitoring annotators.

The paper analyzes how quantization affects the Fisher Information Matrix's dominant eigenvalue.

problem The impact of quantization on the Fisher Information Matrix's dominant eigenvalue.
method The study examines spectral perturbation of the empirical Fisher Information Matrix under in-distribution input and quantized parameter perturbations.
result A bound on the eigenvalue under quantization noise, showing it strictly exceeds the unperturbed value at leading order.

Vecchia approximations provide the best accuracy-runtime trade-off for Gaussian process approximations.

problem High computational cost of Gaussian processes for large data sets.
method Systematic comparison of different Gaussian process approximations.
result Vecchia approximations consistently provide the best accuracy-runtime trade-off.

Stochastic momentum methods trade compute efficiency for serial runtime.

problem Stochastic momentum methods trade compute efficiency for serial runtime.
method Stochastic HB and ASGD for consistent linear regression with Gaussian covariates.
result HB preserves SGD-level CE over a larger batch-size window, allowing larger batches to reduce serial runtime until HB reaches its deterministic accelerated scale.

Bayesian method predicts runtime metrics for fog manufacturing.

problem Accurate prediction of runtime performance metrics in fog manufacturing.
method Bayesian sparse regression for multivariate mixed responses.
result Enhanced prediction and statistical inferences of runtime metrics.

Geometric stability predicts steerability and detects drift in language models.

problem Predicting steerability and detecting drift in language models.
method Supervised and unsupervised geometric stability measures.
result Supervised geometric stability predicts steerability with high accuracy and detects drift earlier.

Proposes a method to choose thresholds for LLM evaluation metrics.

problem Ensuring reliable large language models (LLMs) with correct threshold selection.
method Identify risks, stakeholders' risk tolerance, and use ground-truth data to determine thresholds.
result Demonstrates a concrete example with the Faithfulness metric and HaluBench dataset.

We propose Edward, a Turing-complete probabilistic programming language. Edward defines two compositional representations---random variables and inference. By treating inference as a first class citizen, on a par with modeling, we show that probabilistic programming can be as flexible and computationally efficient as t…

2017-01-13abs ↗pdf ↗