Predictive process monitoring is concerned with the analysis of events produced during the execution of a business process in order to predict as early as possible the final outcome of an ongoing case. Traditionally, predictive process monitoring methods are optimized with respect to accuracy. However, in environments …
Paper introduces Influence Function to assess OOD generalization stability.
problem Assessing OOD generalization accuracy when target domains are unknown.
method Introduced Influence Function from robust statistics to monitor model stability.
result Accuracy on test domains and Influence Function variance can distinguish OOD algorithms and generalization quality.
We develop a monitoring procedure to detect changes in a large approximate factor model. Letting r be the number of common factors, we base our statistics on the fact that the (r+1)-th eigenvalue of the sample covariance matrix is bounded under the null of no change, whereas it becomes spiked under cha…
Study compares statistical properties and power of divergence measures for credit risk monitoring.
problem Detecting distributional shifts in credit risk models.
method Derives statistical properties and chi-square benchmark values for Jensen-Shannon Divergence and Kullback-Leibler Divergence, demonstrating their applicability in credit risk monitoring.
result Jensen-Shannon Divergence and Kullback-Leibler Divergence follow chi-square distributions and reveal practical trade-offs in minimizing false positives vs. detecting changes.
Adaptive-stepsize MCMC sampling inspired by Adam optimizer.
problem Improving numerical stability and convergence speed in MCMC sampling.
method Time-rescaled Langevin dynamics with an auxiliary relaxation equation and adaptive stepsize control.
result Automatic stepsize control improves accuracy and stability in numerical experiments.
A new method uses active learning to monitor industrial processes more accurately.
problem Classifying process states (IC, OC) with limited labeled data.
method Stream-based active learning for partially hidden Markov models.
result Improved dynamic recognition of process states, especially unseen classes.
Geometric stability predicts steerability and detects drift in language models.
problem Predicting steerability and detecting drift in language models.
method Supervised and unsupervised geometric stability measures.
result Supervised geometric stability predicts steerability with high accuracy and detects drift earlier.
Paper proposes tensor-based method for semiconductor manufacturing process control.
problem Challenges of traditional process control methods in high-dimensional image-based overlay errors.
method Builds a high-dimensional process model, proposes tensor-on-vector regression algorithms, designs EWMA controller for tensor data.
result The method reduces overlay errors using limited control recipes and is superior especially when disturbances are not stable.
AI systems that explain their decisions can be monitored for harmful intentions.
problem Monitoring AI systems' decision-making processes for harmful intentions is imperfect and can miss some misbehavior.
method Monitoring the chain of thought (CoT) of AI systems that communicate in human language.
result CoT monitoring is a promising but fragile approach to AI safety.
Symmetry-regularized Neural ODEs improve model stability and interpretability.
problem Improving the stability and physical interpretability of Neural ODEs.
method Integrating Lie symmetries and conservation laws into the loss function.
result Symmetry-regularized Neural ODEs enhance model stability and interpretability.
Simple online monitor detects unsafe LLM outputs.
problem LLMs generate unsafe outputs despite training.
method Thresholding external verifier signal to decide alarms.
result Simple design competitive with advanced methods.
VSCOUT detects anomalies in high-dimensional data using a hybrid VAE approach.
problem Challenges in classical SPC for high-dimensional, non-Gaussian data.
method Hybrid VAE architecture with ARD prior, ensemble filtering, and changepoint detection.
result VSCOUT achieves superior sensitivity to special-cause structure and controlled false alarms.
This research tackles monitoring machine learning algorithms post-deployment, addressing performativity issues.
problem Monitoring machine learning algorithms after deployment, especially when they affect their own data-generating process.
method Uses causal inference techniques to navigate performativity and compares different monitoring criteria and data sources.
result Different monitoring systems have varying operating characteristics and implications for ML monitoring design.
A new space-time model for interacting agents on the financial market is presented. It is a combination of the Curie-Weiss model and a space-time model introduced by Järpe 2005. Properties of the model are derived with focus on the critical temperature and magnetization. It turns out that the Hamiltonian is a sufficien…
Paper improves ETF tail-risk monitoring reliability.
problem Unreliable ETF risk monitoring under degraded data.
method Combines quality checks, prediction, scoring, and adjustment.
result Improves tail-risk monitoring, especially during stressed periods.
New risk models use chaotic attractors to predict extreme events.
problem Predicting Black Swan events in financial markets.
method Combining heavy-tailed priors with chaotic dynamics (Lorenz and Rossler systems).
result Models generate volatility clustering, fat tails, and extreme events.
PITMonitor monitors model calibration over time with formal error guarantees.
problem Fixed-sample tests applied to models over time can lead to false alarms.
method PITMonitor uses mixture e-processes to detect distributional shifts in probability integral transforms.
result PITMonitor achieves competitive detection rates on river's FriedmanDrift benchmark.
The paper adds explanation to predictive process monitoring.
problem Equipping predictive business process monitoring with explanation capabilities.
method Used game theory of Shapley Values to obtain robust explanations.
result First time explanations given in predictive business process monitoring.
IDS algorithm optimizes sequential decisions in various monitoring settings.
problem Optimizing sequential decisions in complex monitoring scenarios.
method Information-directed sampling (IDS) algorithm for linear partial monitoring.
result IDS achieves nearly worst-case rate optimality in finite-action games.
A new method monitors unstructured 3D shapes without registration.
problem Error-prone registration and mesh reconstruction steps in PCD monitoring.
method Intrinsic geometric properties of shapes, using Laplacian and geodesic distances.
result Effective monitoring of defects without registration and mesh reconstruction.
Focuses on monitoring and explaining models in real-world applications.
problem Ensuring high quality machine learning services in production environments.
method Statistical techniques for model performance and data monitoring, explanations of predictions.
result Challenges and solutions for implementing monitoring and explanation in production models.
Neural system optimizes glucose levels in diabetics.
problem Limited research on continuous glucose maintenance devices.
method Differential predictive control with neural policy and differentiable modeling.
result Improves glucose level optimization in real-time.
We present a numerical scheme to calculate fluctuation identities for exponential Lévy processes in the continuous monitoring case. This includes the Spitzer identities for touching a single upper or lower barrier, and the more difficult case of the two-barriers exit problem. These identities are given in the Fourier-L…
Optimal probing framework for scalable network monitoring.
problem Efficiently monitor growing cloud networks with limited budgets.
method A- and E-optimal experimental designs, Frank-Wolfe algorithm approximations.
result Significant reduction in probing budget with low estimation errors.
The paper proposes a method to create efficient remote monitoring models.
problem Large and complex machine learning models are unsuitable for remote monitoring on edge devices.
method Decompose the model into a simple local monitoring function and a complex correction term evaluated on the server.
result The proposed framework learns monitoring models with significantly reduced complexity that maintain safety.
We propose an algorithm for incremental learning of classifiers. The proposed method enables an ensemble of classifiers to learn incrementally by accommodating new training data. We use an effective mechanism to overcome the stability-plasticity dilemma. In incremental learning, the general convention is to use only th…
In deterministic optimization, line searches are a standard tool ensuring stability and efficiency. Where only stochastic gradients are available, no direct equivalent has so far been formulated, because uncertain gradients do not allow for a strict sequence of decisions collapsing the search space. We construct a prob…
In deterministic optimization, line searches are a standard tool ensuring stability and efficiency. Where only stochastic gradients are available, no direct equivalent has so far been formulated, because uncertain gradients do not allow for a strict sequence of decisions collapsing the search space. We construct a prob…
New monitoring method detects ML risk models' performance changes in medical interventions.
problem Monitoring ML risk models in healthcare is complicated by confounding medical interventions.
method Developed a new score-based CUSUM monitoring procedure with dynamic control limits.
result Valid inference is possible if conditional exchangeability or time-constant selection bias hold.
Emerging wearable sensors have enabled the unprecedented ability to continuously monitor human activities for healthcare purposes. However, with so many ambient sensors collecting different measurements, it becomes important not only to maintain good monitoring accuracy, but also low power consumption to ensure sustain…
Risk monitoring detects when TTA models degrade at test time.
problem Detecting when TTA models degrade at test time.
method Extended risk monitoring tools based on sequential testing with confidence sequences.
result Demonstrated effectiveness of TTA monitoring framework across various datasets and methods.
Paper optimizes a big data and ML risk monitoring system for financial markets.
problem Traditional risk monitoring methods are inadequate for modern financial markets due to data complexity and volume.
method Four-layer architecture integrating big data and advanced ML algorithms (LSTM, RF, GB).
result Significantly enhances efficiency and accuracy in risk management, especially in market crash risk detection.
For using neural networks in safety critical domains, it is important to know if a decision made by a neural network is supported by prior similarities in training. We propose runtime neuron activation pattern monitoring - after the standard training process, one creates a monitor by feeding the training data to the ne…
The significance of air pollution and the problems associated with it are fueling deployments of air quality monitoring stations worldwide. The most common approach for air quality monitoring is to rely on environmental monitoring stations, which unfortunately are very expensive both to acquire and to maintain. Hence e…
Structural health monitoring is a condition-based field of study utilised to monitor infrastructure, via sensing systems. It is therefore used in the field of aerospace engineering to assist in monitoring the health of aerospace structures. A difficulty however is that in structural health monitoring the data input is …
A monitoring procedure improves machine learning forecasts for digital platforms.
problem Maintaining accurate and stable forecasts for data streams at digital platforms.
method Developed a monitoring procedure to determine when to retrain machine learning algorithms.
result Monitor-based retraining produces accurate forecasts compared to benchmarks.
Study uses satellite data to predict tailings dam collapse risk.
problem Detecting early signs of tailings dam instability.
method Spectral analysis of satellite InSAR displacement time series data.
result Algorithm detects risk milestones up to 5 months before dam collapse.
Novel framework monitors cardiac image segmentation models in real-time.
problem Ensuring continuous high model performance and segmentation results in clinics.
method Formulated as anomaly detection, the framework derives surrogate quality measures for segmentation.
result Demonstrated accurate, fast, and scalable quality control monitoring.
New Thompson sampling algorithm for stochastic partial monitoring achieves logarithmic regret.
problem Limited feedback in sequential learning problems.
method Developed a novel Thompson-sampling-based algorithm to sample from the posterior distribution exactly.
result Achieved logarithmic regret bound of O(log T) for a linearized variant of the problem.
DFKI Cabin Simulator tests visual monitoring functions in vehicles.
problem Validating novel human-vehicle interfaces and driver assistance systems.
method Driving simulator with in-cabin mock-up and camera system.
result Validation of in-cabin monitoring functions for advanced driver assistance and automated driving.
Paper monitors DNN accuracy to enhance trustworthiness.
problem Varying DNN accuracy in practice and lack of ground truth labels.
method Post-hoc accuracy monitor model using Monte-Carlo dropout ensemble.
result Accuracy monitor provides close-to-true accuracy estimation.
Proposes real-time risk monitoring for machine learning systems under unknown shifts.
problem Dynamic distribution shifts challenge real-world machine learning systems' risk assurances.
method Sequential hypothesis testing with 'testing by betting' to detect risk violations.
result Effective real-time risk monitoring under various unknown shifts.
A registration-free framework monitors shape and color in 4D point clouds.
problem Monitoring shape and color changes in complex parts without registration.
method Laplace-Beltrami operator spectral properties for geometric and color feature capture; combined monitoring scheme for shape and color anomalies.
result Effective detection of shape deformations and color anomalies without registration or mesh reconstruction.
Conformal prediction fails under severe feature turnover in COVID-19 supply chain tasks.
problem Dealing with distribution shift in conformal prediction models.
method Using COVID-19 as a natural experiment across 8 supply chain tasks, analyzing SHAP explanations.
result Coverage drops vary widely (0% to 86.7%) and correlate with single-feature dependence.
IDS optimizes regret in stochastic partial monitoring with linear rewards.
problem Optimizing decision-making in uncertain environments with linear rewards.
method Information Directed Sampling (IDS) for stochastic partial monitoring.
result Achieves optimal regret rates in all observable game regimes.
Amazon SageMaker Model Monitor detects drift in deployed ML models.
problem Ensuring high performance of ML models in production environments.
method Automatically detects data, concept, bias, and feature attribution drift in real-time.
result Maintains high quality models by providing alerts and corrective actions.
IUPM monitors machine learning models under gradual shifts using optimal transport and active labeling.
problem Gradual distribution shifts lead to unnoticed accuracy declines in machine learning models.
method Incremental Uncertainty-aware Performance Monitoring (IUPM) using optimal transport and active labeling.
result IUPM outperforms existing baselines in gradual shift scenarios and guides label acquisition more effectively.
Develops a tool to identify abnormal blood smear results based on CBC tests.
problem Manual review of blood smears by technologists is time-consuming and inconsistent.
method Cost-sensitive Lasso-penalized additive logistic regression combined with stability selection.
result The tool correctly identifies true cutoff values for abnormal smear results.