Runtime neuron activation monitoring warns of decisions not supported by training data.
problem Ensuring neural network decisions are backed by training data in safety-critical applications.
method Create a monitor by storing neuron activation patterns from training data. In operation, compare new inputs to monitor for similar patterns.
result Monitors can detect a significant portion of misclassifications not supported by training data with a low false-positive rate.
Logical scaffolds enhance AI software quality.
problem Improving AI component quality in software.
method Logical scaffolds as a method to improve AI components.
result Logical scaffolds can improve AI beyond perception systems.
Enhances reward specification in RL with a novel language-based approach.
problem Reward specification in RL can lead to unintended, potentially harmful behaviours.
method Developed a novel class of language-based Reward Machines using RML's built-in memory.
result Can specify non-regular, non-Markovian reward functions for complex tasks.
Deep learning speeds up real-time emission monitoring.
problem Real-time greenhouse gas emission monitoring under transient conditions.
method Bayesian inference with deep learning surrogate of CFD outputs.
result Near-real-time predictions with orders-of-magnitude faster runtimes.
A novel ML verification technique using manifold learning.
problem Ensuring trust in machine learning systems.
method Variational autoencoder for extracting a low-dimensional manifold from high-dimensional training data.
result The manifold provides diverse test data, fault-revealing test cases, and independent runtime trust assessment.
Novel method detects faults in helicopter transmissions using healthy data only.
problem Detecting faults in rare events in helicopter transmissions.
method Probabilistic anomaly detection using Bayesian measures.
result Method achieves competitive detection performance in real-world and benchmark applications.
Framework detects novel inputs in neural networks by monitoring hidden layers.
problem Novel inputs not classified by neural networks during training.
method Abstraction-based monitoring of hidden layers using 'boxes' to identify novel behaviors.
result Framework efficiently detects novel inputs with a balance between false warnings and true positives.
Tribuo integrates ML models with provenance in Java.
problem Tracking and ensuring ML models behave appropriately as they grow in number and complexity.
method Integrates model training, inference, type-safety, runtime checking, and automatic provenance recording into a single Java library.
result Automatic recording of full processing pipeline and training details inside the model object.
VTIRT speeds up IRT inference for dynamic learner proficiency.
problem Expensive and slow inference algorithms for dynamic IRT models.
method Variational Temporal IRT (VTIRT) for fast, accurate inference.
result Orders of magnitude speedup in inference runtime with accurate results.
SeFT classifies irregular time series with high scalability and data efficiency.
problem Classifying irregularly-sampled time series with unaligned measurements.
method SeFT (Set Functions for Time Series) based on differentiable set function learning, extremely parallelizable.
result SeFT performs competitively with healthcare time series datasets and significantly reduces runtime.
Model predicts counterfactuals under domain shift and inaccessible variables.
problem Runtime domain corruption impairs counterfactual prediction.
method Subsumes counterfactual prediction under domain adaptation, uses adversarial domain adaptation to reduce distribution disparity.
result VEGAN outperforms baselines in individual-level treatment effect estimation.
Proposes MCDE for estimating dependency in data streams.
problem Estimating dependency of variables in data streams is challenging.
method Monte Carlo Dependency Estimation (MCDE) framework.
result MWP dependency estimator shows superiority over state-of-the-art measures.
Serverless runtimes boost large-scale optimization efficiency.
problem Efficiently solving large-scale optimization problems.
method Master-worker setup with AWS Lambda, parallel optimization algorithm.
result Relative speedups up to 256 workers and efficiencies above 70% up to 64 workers.
Toolkit improves neural network safety for self-driving cars.
problem Ensuring safety of neural networks in autonomous driving systems.
method Structured approach using Goal Structuring Notation and recent scientific results.
result Improves quality of a level-3 autonomous driving component.
Vecchia approximations provide the best accuracy-runtime trade-off for Gaussian process approximations.
problem High computational cost of Gaussian processes for large data sets.
method Systematic comparison of different Gaussian process approximations.
result Vecchia approximations consistently provide the best accuracy-runtime trade-off.
Stochastic momentum methods trade compute efficiency for serial runtime.
problem Stochastic momentum methods trade compute efficiency for serial runtime.
method Stochastic HB and ASGD for consistent linear regression with Gaussian covariates.
result HB preserves SGD-level CE over a larger batch-size window, allowing larger batches to reduce serial runtime until HB reaches its deterministic accelerated scale.
AI systems that explain their decisions can be monitored for harmful intentions.
problem Monitoring AI systems' decision-making processes for harmful intentions is imperfect and can miss some misbehavior.
method Monitoring the chain of thought (CoT) of AI systems that communicate in human language.
result CoT monitoring is a promising but fragile approach to AI safety.
Change detection in multivariate time series has applications in many domains, including health care and network monitoring. A common approach to detect changes is to compare the divergence between the distributions of a reference window and a test window. When the number of dimensions is very large, however, the naive…
Bayesian method predicts runtime metrics for fog manufacturing.
problem Accurate prediction of runtime performance metrics in fog manufacturing.
method Bayesian sparse regression for multivariate mixed responses.
result Enhanced prediction and statistical inferences of runtime metrics.
The runtime for Kernel Partial Least Squares (KPLS) to compute the fit is quadratic in the number of examples. However, the necessity of obtaining sensitivity measures as degrees of freedom for model selection or confidence intervals for more detailed analysis requires cubic runtime, and thus constitutes a computationa…
Asynchronous SGD can speed up training with a trade-off of gradient staleness.
problem Asynchronous SGD suffers from gradient staleness, affecting convergence error.
method Theoretical analysis of the error-runtime trade-off considering random straggling delays.
result A method of gradually varying synchronicity in distributed SGD is proposed and demonstrated.
Algorithm decides if two hyperbolic 3-manifolds are homeomorphic.
problem Determining if two hyperbolic 3-manifolds are homeomorphic.
method Algorithm finds hyperbolic structures and compares geometric manifolds.
result Algorithm runs in bounded time for triangulations with at most t tetrahedra. This work tackles runtime complexity prediction for code, using machine learning and a new dataset.
problem Predicting runtime complexity of code is hard and mathematically impossible.
method Modelled as a machine learning task, using feature engineering and code embeddings, with a new dataset.
result Achieved state-of-the-art results in runtime complexity prediction.
Existing malware detectors on safety-critical devices have difficulties in runtime detection due to the performance overhead. In this paper, we introduce PROPEDEUTICA, a framework for efficient and effective real-time malware detection, leveraging the best of conventional machine learning (ML) and deep learning (DL) te…
Simple online monitor detects unsafe LLM outputs.
problem LLMs generate unsafe outputs despite training.
method Thresholding external verifier signal to decide alarms.
result Simple design competitive with advanced methods.
Paper introduces HISA for efficient FHE computations.
problem Efficiently evaluating encrypted neural networks.
method Developed HISA for FHE applications, including compiler and runtime.
result Generated code is faster than hand-optimized implementations.
This research tackles monitoring machine learning algorithms post-deployment, addressing performativity issues.
problem Monitoring machine learning algorithms after deployment, especially when they affect their own data-generating process.
method Uses causal inference techniques to navigate performativity and compares different monitoring criteria and data sources.
result Different monitoring systems have varying operating characteristics and implications for ML monitoring design.
For well over a quarter century, detection systems have been driven by models learned from input features collected from real or simulated environments. An artifact (e.g., network event, potential malware sample, suspicious email) is deemed malicious or non-malicious based on its similarity to the learned model at runt…
Paper improves ETF tail-risk monitoring reliability.
problem Unreliable ETF risk monitoring under degraded data.
method Combines quality checks, prediction, scoring, and adjustment.
result Improves tail-risk monitoring, especially during stressed periods.
In many recent applications, data is plentiful. By now, we have a rather clear understanding of how more data can be used to improve the accuracy of learning algorithms. Recently, there has been a growing interest in understanding how more data can be leveraged to reduce the required training runtime. In this paper, we…
Tensor analysis improves structural health monitoring of complex aerospace systems.
problem Highly redundant and correlated sensor data in structural health monitoring.
method Tensor-based learning for multi-way structural data analysis.
result Tensor-based approach successfully detects damage in aeroservoelastic models.
Adaptive activity monitoring framework for wearable sensors.
problem Efficiently monitor human activities with low power consumption.
method Switching Gaussian process model with block circulant embedding and FFT for inference.
result Optimized trade-off between sensor power consumption and prediction performance.
PITMonitor monitors model calibration over time with formal error guarantees.
problem Fixed-sample tests applied to models over time can lead to false alarms.
method PITMonitor uses mixture e-processes to detect distributional shifts in probability integral transforms.
result PITMonitor achieves competitive detection rates on river's FriedmanDrift benchmark.
The paper adds explanation to predictive process monitoring.
problem Equipping predictive business process monitoring with explanation capabilities.
method Used game theory of Shapley Values to obtain robust explanations.
result First time explanations given in predictive business process monitoring.
A new method monitors unstructured 3D shapes without registration.
problem Error-prone registration and mesh reconstruction steps in PCD monitoring.
method Intrinsic geometric properties of shapes, using Laplacian and geodesic distances.
result Effective monitoring of defects without registration and mesh reconstruction.
IDS algorithm optimizes sequential decisions in various monitoring settings.
problem Optimizing sequential decisions in complex monitoring scenarios.
method Information-directed sampling (IDS) algorithm for linear partial monitoring.
result IDS achieves nearly worst-case rate optimality in finite-action games.
Separating the short jobs from the long is a known technique to improve scheduling performance. In this paper we describe a method we developed for accurately predicting the runtimes classes of the jobs to enable this separation. Our method uses the fact that the runtimes can be represented as a mixture of overlapping …
Focuses on monitoring and explaining models in real-world applications.
problem Ensuring high quality machine learning services in production environments.
method Statistical techniques for model performance and data monitoring, explanations of predictions.
result Challenges and solutions for implementing monitoring and explanation in production models.
GNMR controls runtime stability in low-precision language model training.
problem Efficient low-precision training faces numerical risks at specific operators.
method GNMR compares gradient norms to historical means, applying bounded recovery actions.
result GNMR preserves high-fidelity quality with sparse, budgeted recovery.
Perhaps surprisingly, it is possible to predict how long an algorithm will take to run on a previously unseen input, using machine learning techniques to build a model of the algorithm's runtime as a function of problem-specific instance features. Such models have important applications to algorithm analysis, portfolio…
Surveying low-cost sensors for air quality monitoring and calibration.
problem Limited spatial resolution due to expensive environmental monitoring stations.
method Low-cost sensors with machine learning for calibration.
result Machine learning improves sensor accuracy over time.
Neural system optimizes glucose levels in diabetics.
problem Limited research on continuous glucose maintenance devices.
method Differential predictive control with neural policy and differentiable modeling.
result Improves glucose level optimization in real-time.
We present a numerical scheme to calculate fluctuation identities for exponential Lévy processes in the continuous monitoring case. This includes the Spitzer identities for touching a single upper or lower barrier, and the more difficult case of the two-barriers exit problem. These identities are given in the Fourier-L…
Optimal probing framework for scalable network monitoring.
problem Efficiently monitor growing cloud networks with limited budgets.
method A- and E-optimal experimental designs, Frank-Wolfe algorithm approximations.
result Significant reduction in probing budget with low estimation errors.
The paper proposes a method to create efficient remote monitoring models.
problem Large and complex machine learning models are unsuitable for remote monitoring on edge devices.
method Decompose the model into a simple local monitoring function and a complex correction term evaluated on the server.
result The proposed framework learns monitoring models with significantly reduced complexity that maintain safety.
Run2Survive uses survival analysis for algorithm selection, outperforming traditional methods.
problem Handling censored runtime data in algorithm selection.
method Decision-theoretic approach leveraging survival analysis for censored data.
result Run2Survive outperforms state-of-the-art AS approaches in experiments.
New monitoring method detects ML risk models' performance changes in medical interventions.
problem Monitoring ML risk models in healthcare is complicated by confounding medical interventions.
method Developed a new score-based CUSUM monitoring procedure with dynamic control limits.
result Valid inference is possible if conditional exchangeability or time-constant selection bias hold.
Validates neural networks inputs to protect against adversarial examples.
problem Ensuring neural networks robustness against adversarial attacks.
method Runtime local robustness verification based on normal distribution of robustness radii.
result Improves neural network accuracy and protects against adversarial examples.