Research
On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,657 papers · 148 categories

Trend · papers per month

9182736 · Nov 201819922001200920172026
48 results for API monitoring

This paper investigates how machine learning APIs change over time and proposes an efficient method to monitor these changes.

problem Understanding and assessing changes in machine learning APIs over time.
method Systematic investigation of ML API shifts, proposing a principled adaptive sampling algorithm (MASA) for efficient estimation of confusion matrix shifts.
result MASA can accurately estimate confusion matrix shifts using up to 90% fewer samples compared to random sampling.

To save manual effort, developers often translate programs from one programming language to another, instead of implementing it from scratch. Translating application program interfaces (APIs) used in one language to functionally equivalent ones available in another language is an important aspect of program translation…

2019-06-10abs ↗pdf ↗

I present a web service for querying an embedding of entities in the Wikidata knowledge graph. The embedding is trained on the Wikidata dump using Gensim's Word2Vec implementation and a simple graph walk. A REST API is implemented. Together with the Wikidata API the web service exposes a multilingual resource for over …

2017-10-11abs ↗pdf ↗

CK simplifies ML model deployment and reproducibility with open APIs and DevOps.

problem Making ML models reproducible and deployable across different environments.
method Decompose complex systems into reusable sub-components with unified APIs and DevOps principles.
result Automatically co-design and optimize ML models for speed, accuracy, energy, and size.

Open-source Vizier optimizes complex systems for Google and beyond.

problem Optimizing large-scale systems with multiple objectives and constraints.
method Distributed, fault-tolerant, flexible API for blackbox optimization.
result OSS Vizier supports a wide range of optimization problems and is available as open-source.

InferPy is a Python package for probabilistic modeling with deep neural networks. It defines a user-friendly API that trades-off model complexity with ease of use, unlike other libraries whose focus is on dealing with very general probabilistic models at the cost of having a more complex API. In particular, this packag…

2019-08-29abs ↗pdf ↗

We consider the infinite-horizon discounted optimal control problem formalized by Markov Decision Processes. We focus on several approximate variations of the Policy Iteration algorithm: Approximate Policy Iteration, Conservative Policy Iteration (CPI), a natural adaptation of the Policy Search by Dynamic Programming a…

2014-05-12abs ↗pdf ↗

PettingZoo library accelerates multi-agent reinforcement learning research.

problem Challenges in multi-agent reinforcement learning, especially conceptual models of games.
method Developed PettingZoo library with AEC games model to address multi-agent reinforcement learning challenges.
result AEC games model addresses conceptual issues in multi-agent reinforcement learning environments.

Recently, machine learning (ML) has introduced advanced solutions to many domains. Since ML models provide business advantage to model owners, protecting intellectual property of ML models has emerged as an important consideration. Confidentiality of ML models can be protected by exposing them to clients only via predi…

2019-10-11abs ↗pdf ↗

Paper explores how non-neural simulators can enhance DP synthetic data generation.

problem Generating differentially private synthetic data without access to foundation models.
method Private Evolution (PE) framework using inference APIs and simulators.
result Sim-PE framework improves downstream classification accuracy and FID scores.

KL-constrained API shows optimization issues and improved with regularization.

problem Optimization issues in KL-constrained API algorithms.
method Comparison of KL divergence as a constraint vs. regularizer, empirical evaluation.
result KL-constrained API is not guaranteed to converge and incurs linear regret.

AI systems that explain their decisions can be monitored for harmful intentions.

problem Monitoring AI systems' decision-making processes for harmful intentions is imperfect and can miss some misbehavior.
method Monitoring the chain of thought (CoT) of AI systems that communicate in human language.
result CoT monitoring is a promising but fragile approach to AI safety.

OpenML is an online platform for open science collaboration in machine learning, used to share datasets and results of machine learning experiments. In this paper we introduce OpenML-Python, a client API for Python, opening up the OpenML platform for a wide range of Python-based tools. It provides easy access to all da…

2019-11-06abs ↗pdf ↗

Transformer models improve malware classification, especially for imbalanced datasets.

problem Imbalanced multiclass malware classification.
method Bagging-based random transformer forest (RTF) ensemble of pre-trained transformer models (BERT or CANINE).
result Ensemble of pre-trained transformer models achieves state-of-the-art F1-score of 0.6149 on a benchmark dataset.

Training machine learning (ML) models is expensive in terms of computational power, amounts of labeled data and human expertise. Thus, ML models constitute intellectual property (IP) and business value for their owners. Embedding digital watermarks during model training allows a model owner to later identify their mode…

2019-06-03abs ↗pdf ↗

This research tackles monitoring machine learning algorithms post-deployment, addressing performativity issues.

problem Monitoring machine learning algorithms after deployment, especially when they affect their own data-generating process.
method Uses causal inference techniques to navigate performativity and compares different monitoring criteria and data sources.
result Different monitoring systems have varying operating characteristics and implications for ML monitoring design.

PITMonitor monitors model calibration over time with formal error guarantees.

problem Fixed-sample tests applied to models over time can lead to false alarms.
method PITMonitor uses mixture e-processes to detect distributional shifts in probability integral transforms.
result PITMonitor achieves competitive detection rates on river's FriedmanDrift benchmark.

IDS algorithm optimizes sequential decisions in various monitoring settings.

problem Optimizing sequential decisions in complex monitoring scenarios.
method Information-directed sampling (IDS) algorithm for linear partial monitoring.
result IDS achieves nearly worst-case rate optimality in finite-action games.

A new method monitors unstructured 3D shapes without registration.

problem Error-prone registration and mesh reconstruction steps in PCD monitoring.
method Intrinsic geometric properties of shapes, using Laplacian and geodesic distances.
result Effective monitoring of defects without registration and mesh reconstruction.

Focuses on monitoring and explaining models in real-world applications.

problem Ensuring high quality machine learning services in production environments.
method Statistical techniques for model performance and data monitoring, explanations of predictions.
result Challenges and solutions for implementing monitoring and explanation in production models.

New monitoring method detects ML risk models' performance changes in medical interventions.

problem Monitoring ML risk models in healthcare is complicated by confounding medical interventions.
method Developed a new score-based CUSUM monitoring procedure with dynamic control limits.
result Valid inference is possible if conditional exchangeability or time-constant selection bias hold.