BAMS uses Bayesian sampling to discover AV failures more efficiently and accurately.
problem Discovering potential failure cases in autonomous vehicles efficiently and accurately.
method Bayesian adaptive multifidelity sampling (BAMS) prioritizes exploration of low performance regions.
result BAMS discovers 10 times more issues than traditional methods with narrower rate estimates.
ProEval efficiently estimates AI performance and discovers failures using pre-trained Gaussian Processes.
problem Resource-intensive evaluation of generative AI models.
method ProEval uses pre-trained Gaussian Processes and Bayesian quadrature to estimate performance and discover failures.
result ProEval requires significantly fewer samples to achieve accurate performance estimates and reveals more diverse failure cases.
New approach uses dynamic programming to efficiently discover failures in autonomous vehicle simulations.
problem Efficiently discovering rare failure events in autonomous vehicle simulations.
method Approximate dynamic programming and scene decomposition to estimate failure distribution.
result Increased number of failures discovered compared to baseline approaches.
Causal analysis reveals regional discrepancies in TOPCAT trial results.
problem Inconclusive results in TOPCAT trial for heart failure treatment.
method Causal discovery methods with domain knowledge integration.
result Significant causal effects shown for some subgroups globally.
The paper investigates causal relationships in heart failure prediction using machine learning.
problem Understanding the causal relationships between clinical variables and heart failure.
method Proposes a new computational framework for causal structure discovery (CSD) of mixed-type clinical variables for binary disease outcomes.
result Feature importance from nonlinear classifiers strongly correlates with causal strength of variables, but not differentiating cause and effect.
CB-SLICE identifies concept-based error slices in deep learning models.
problem Systematic errors in deep learning models on specific groups.
method Concept Bottleneck Models (CBMs) and concept representations.
result CB-SLICE outperforms state-of-the-art methods in error slice identification.
AFTNet uses a network-constrained Weibull model for biomarker discovery.
problem Discovering biomarkers from survival data with correlated predictors.
method Survival analysis method based on Weibull AFT model, incorporating network constraints and penalized likelihood for variable selection.
result Theoretical consistency and efficient algorithm for AFTNet estimator validated on synthetic and real data.
DG separates successes and failures by gating updates with advantage and surprisal.
problem Negative learning from surprising data in distributed reinforcement learning.
method DG gates each update with the product of advantage and surprisal, suppressing failures and preserving successes.
result DG outperforms other methods in various challenging reinforcement learning tasks.
Drug discovery projects entail cycles of design, synthesis, and testing that yield a series of chemically related small molecules whose properties, such as binding affinity to a given target protein, are progressively tailored to a particular drug discovery goal. The use of deep learning technologies could augment the …
New framework learns complex AI attitudes from heterogeneous data.
problem Heterogeneous ordinal structure in AI attitudes, poorly captured by existing methods.
method Monotone Gaussian score embedding, BNP complexity discovery, confirmatory fixed-K estimation.
result Reduced holdout MSE by 25.8% over single-graph baseline.
ECCIT improves conditional independence tests by calibrating for miscalibration.
problem Inaccurate frequentist guarantees in CITs, especially in small samples and misspecified models.
method Empirically Calibrated Conditional Independence Tests (ECCIT) that optimize and correct for miscalibration.
result ECCIT achieves valid FDR with higher power than existing calibration strategies.
Conformal prediction fails to cover minority classes in imbalanced datasets, but a class-conditional fix improves coverage.
problem Conformal prediction fails to cover minority classes in imbalanced datasets, leading to poor performance on rare labels.
method Class-conditional conformal prediction to improve coverage of minority classes.
result Class-conditional conformal prediction restores minority coverage to target with a modest increase in prediction-set size.
TSML tackles anomaly detection and pattern discovery in industrial time series data.
problem Extracting and exploiting information from large industrial data to reduce downtimes and manufacturing errors.
method TSML uses a pipeline of lightweight filters to process industrial time series data in parallel.
result TSML effectively detects anomalies and discovers patterns in industrial time series data.
New protocol identifies impossible edge orientations in causal graphs.
problem Causal-discovery algorithms cannot distinguish edge directions without assumptions.
method Discrete impossibility certificates and oracle queries.
result Upper bound of 1+K expert interactions for DAG recovery. Alternative to likelihood-based LSNM model selection, residual independence testing is more robust to noise misspecification.
problem Cause-effect inference in location-scale noise models with misspecified noise distributions.
method Residual independence testing as an alternative to likelihood-based model selection.
result Residual independence testing is more robust to noise misspecification.
C-PP-COAD detects anomalies with limited real data, reducing dependency on real calibration data.
problem Limited real calibration data for online anomaly detection.
method Context-aware prediction-powered conformal online anomaly detection (C-PP-COAD).
result Significantly reduces dependency on real calibration data without compromising FDR control.
CSTS benchmarks time series clustering by evaluating correlation structures.
problem Lack of validated ground truth for objectively assessing clustering quality.
method Synthetic benchmark CSTS for evaluating correlation structures in multivariate time series data.
result CSTS enables precise diagnosis of methodological limitations in correlation-based time series clustering.
Automatic methods for generating state-of-the-art neural network architectures without human experts have generated significant attention recently. This is because of the potential to remove human experts from the design loop which can reduce costs and decrease time to model deployment. Neural architecture search (NAS)…
FactorMiner discovers financial alpha factors with low redundancy.
problem Finding novel financial alpha factors in a vast search space.
method Modular Skill Architecture and Experience Memory to distill and guide exploration.
result FactorMiner constructs a diverse library of high-quality factors with competitive performance.
Economics tool predicts failure times in reliability systems.
problem Predicting optimal failure times in weighted k-out-of-n reliability systems with heterogeneous component failure.
method Using rational expectations to analyze and predict failure times in reliability systems with heterogeneous component failure.
result Different measures are optimal for predicting system failure depending on component failure distributions.
XGES improves GES by favoring early edge deletion, outperforming GES in finite data settings.
problem Learning directed acyclic graphs from finite data.
method Extremely Greedy Equivalent Search (XGES) improves GES by favoring early edge deletion.
result XGES consistently outperforms GES in recovering the correct graphs, and is 10 times faster.
Unified model predicts multi-mode failure with multi-sensor data.
problem Independent failure mode and RUL prediction ignores inherent relationship.
method Hierarchical Bayesian framework with Cox model, Gaussian process, and multinomial distributions.
result Robust uncertainty quantification and accurate prediction of multi-mode failure.
PAGER detects failures in deep regression models using a new framework.
problem Detecting failures in deep regression models.
method PAGER uses a combination of epistemic uncertainty and manifold non-conformity scores.
result PAGER accurately characterizes and detects failures in deep regressors.
Risk Advisor predicts and mitigates ML deployment failures.
problem Predicting and mitigating test-time failure risks of ML systems.
method Post-hoc meta-learner for estimating failure risks and uncertainties.
result Reliably predicts deployment-time failure risks across various ML models.
CalNF models rare failures with limited data, improving safety in autonomous systems.
problem Challenges in modeling and debugging rare safety-critical failures due to limited data.
method CalNF, a self-regularized framework for posterior learning from limited data.
result Achieves state-of-the-art performance on data-limited failure modeling and inverse problems.
Prognostics and Health Management (PHM) is an emerging engineering discipline which is concerned with the analysis and prediction of equipment health and performance. One of the key challenges in PHM is to accurately predict impending failures in the equipment. In recent years, solutions for failure prediction have evo…
This work improves safety validation of autonomous vehicles by finding interpretable failures.
problem Finding interpretable failures of autonomous systems in simulation.
method Signal temporal logic expressions optimized for high likelihood and human interpretability.
result Our methodology finds more interpretable failures with higher likelihood compared to baseline approaches.
Framework predicts remaining useful life of DSH subsystems under unknown failure modes.
problem Predicting remaining useful life of DSH subsystems with unknown failure modes.
method Unsupervised framework using mixture of Gaussian regressions and Expectation-Maximization algorithm.
result Improved prediction accuracy and interpretability of RUL.
The paper calculates the likelihood of a financial market failure involving multiple major banks.
problem Estimating the probability of a market failure involving multiple globally important banks.
method Multivariate Cox process across G-SIBs, deriving various theorems on market failure probabilities.
result The probability of a market failure increases with the number of G-SIBs and is inevitable if there are too many.
New EiV models correct bias in operator learning with noisy data.
problem Bias in operator learning due to noisy independent variables.
method Developed EiV models for MOR-Physics and DeepONet.
result EiV models reduce bias in noisy operator learning.
Paper proposes a method to predict disk failures using multi-layer domain adaptive learning.
problem Traditional machine learning models struggle to predict disk failures due to limited data.
method Multi-layer domain adaptive learning with source and target domains.
result The proposed method improves failure prediction accuracy on disk data with few failure samples.
We aim to predict and explain service failures in supply-chain networks, more precisely among last-mile pickup and delivery services to customers. We analyze a dataset of 500,000 services using (1) supervised classification with Random Forests, and (2) Association Rules. Our classifier reaches an average sensitivity of…
GE finds failures in autonomous systems without domain heuristics.
problem Finding failures in autonomous systems without domain-specific heuristics.
method Adaptive stress testing using go-explore (GE) algorithm.
result GE finds failures in scenarios other RL techniques cannot solve.
Method distinguishes between failures and domain shifts in industrial data streams.
problem Confusing domain shifts with failures in industrial data.
method Modified Page-Hinkley changepoint detector and supervised domain-adaptation-based anomaly detection.
result Allows differentiation between failures and domain shifts.
System predicts respiratory failure up to 8 hours early.
problem Early detection of respiratory failure in ICU patients.
method Machine learning on ICU patient monitoring data.
result System outperforms traditional clinical decision-making.
New method avoids failures in physics-constrained systems using active learning.
problem Handling fatal failures in systems governed by physics constraints.
method Develops a novel active learning method that considers implicit physics constraints.
result Achieves zero-failure in composite fuselage assembly process without explicit failure regions.
This paper addresses the problem of evaluating learning systems in safety critical domains such as autonomous driving, where failures can have catastrophic consequences. We focus on two problems: searching for scenarios when learned agents fail and assessing their probability of failure. The standard method for agent e…
DeepSIP predicts network failures' impact using CNN from syslog and traffic data.
problem Predicting service impact from network failures.
method Temporal multimodal CNN for predicting time to recovery and traffic loss.
result DeepSIP reduced prediction error by approximately 50%.
In the last two years, more than 200 papers have been written on how machine learning (ML) systems can fail because of adversarial attacks on the algorithms and data; this number balloons if we were to incorporate papers covering non-adversarial failure modes. The spate of papers has made it difficult for ML practition…
Two BO methods improve reliability optimization for rare failures.
problem Maximizing reliability of designs subject to random perturbations.
method Bayesian optimization with Thompson sampling and knowledge gradient.
result Proposed methods outperform existing techniques in extreme failure probability scenarios.
New method finds failures in high-fidelity simulators with fewer steps.
problem Finding failures in high-fidelity simulators is expensive and impractical.
method Adaptive stress testing with backward algorithm adaptation from low-fidelity to high-fidelity.
result Significantly fewer high-fidelity simulation steps needed to find failures.
Characterization of a patient clinical phenotype is central to biomedical informatics. ICD codes, assigned to inpatient encounters by coders, is important for population health and cohort discovery when clinical information is limited. While ICD codes are assigned to patients by professionals trained and certified in c…
Entropy-based GP adaptive design improves failure probability estimation.
problem Limited accuracy in failure probability estimation due to model evaluation costs.
method Entropy-based Gaussian process (GP) adaptive design combined with multifidelity importance sampling (MFIS).
result More accurate failure probability estimates and higher confidence.
Brain uses synaptic failure to sample from posterior distributions.
problem Bayesian inference in the brain's probabilistic computations.
method Adapting synaptic failure to sample posterior predictive distributions.
result Synaptic failure enables sampling of complete posterior predictive distributions.
Study constructs balanced datasets for seismic failure prediction.
problem Imbalanced datasets limit machine learning performance in seismic failure prediction.
method Framework with three steps: GMF identification, probability density estimation, and sample transformation.
result Framework improves machine learning performance in seismic failure mode prediction.
Proposes a deep neural network for early disk drive failure prediction.
problem Early prediction of disk drive failure using multivariate time series sensor data.
method Enriched features derived from sensor data through transformations, combined with ensemble learning and deep neural network architecture.
result Significantly improved classification accuracy in predicting disk drive failure.
Improves FI-PINNs by combining re-sampling and subset simulation for better failure probability estimation.
problem Estimating failure probability in physics-informed neural networks (PINNs).
method Adaptive sampling with re-sampling and subset simulation, using cosine-annealing for uniform to adaptive transition.
result Significant improvement in estimating failure probability and generating new training points in the failure region.
A new algorithm learns from failures to optimize under constraints efficiently.
problem Optimizing under unknown constraints with limited failure tolerance.
method Excursion search controls risk as a function of a failure budget.
result The algorithm achieves lower regret and uses failures budget more efficiently.