Paper defines and quantifies safety risks in deep neural networks.
problem Safety concerns in deep neural networks applied to critical sectors.
method Defines safety property, computes maximum safe radius, identifies new risk class, develops algorithm.
result Method achieves competitive performance in safety quantification.
By building on a recently introduced genetic-inspired attribute-based conceptual framework for safety risk analysis, we propose a novel methodology to compute construction univariate and bivariate construction safety risk at a situational level. Our fully data-driven approach provides construction practitioners and aca…
Logarithmic regret strategies for safe multi-armed bandits with safety risk constraints.
problem Maximizing reward while avoiding unsafe arms under safety risk constraints.
method Doubly optimistic strategies with pseudo-regret formulation.
result Logarithmic regret bounds for safe multi-armed bandits.
Machine learning algorithms are increasingly influencing our decisions and interacting with us in all parts of our daily lives. Therefore, just like for power plants, highways, and myriad other engineered sociotechnical systems, we must consider the safety of systems involving machine learning. In this paper, we first …
CARL safely adapts RL agents for safety-critical tasks.
problem Safety hazards in RL for safety-critical tasks.
method CARL combines model-based RL and cautious adaptation.
result CARL achieves higher rewards with fewer failures in safety-critical tasks.
New risk metric for AI systems reduces safety risks with minimal data.
problem Risk assessment in multi-agent AI systems.
method Free Energy Principle applied to risk metrics, introducing Cumulative Risk Exposure.
result Gatekeepers improve system safety in autonomous vehicle fleets.
A framework identifies worst-case decision points in safety-critical scenarios, improving risk assessment by 10 hours.
problem Identifying worst-case outcomes in safety-critical decision-making under uncertainty.
method Explicitly estimating distributions of expected return to identify dead-ends, tuning based on risk tolerance.
result Significantly improves risk assessment, providing indications 10 hours earlier and increasing detection by 20%.
Simple online monitor detects unsafe LLM outputs.
problem LLMs generate unsafe outputs despite training.
method Thresholding external verifier signal to decide alarms.
result Simple design competitive with advanced methods.
Machine learning algorithms increasingly influence our decisions and interact with us in all parts of our daily lives. Therefore, just as we consider the safety of power plants, highways, and a variety of other engineered socio-technical systems, we must also take into account the safety of systems involving machine le…
We enhance conformal prediction for risk-averse decisions with action-conditional guarantees.
problem Uncertainty quantification and safety guarantees for machine learning decisions.
method Action-conditional conformal prediction, pinball-loss minimization.
result Action-conditional prediction sets optimize risk-averse decision-making.
Failures are challenging for learning to control physical systems since they risk damage, time-consuming resets, and often provide little gradient information. Adding safety constraints to exploration typically requires a lot of prior knowledge and domain expertise. We present a safety measure which implicitly captures…
SafeML monitors ML systems for safety and security risks.
problem Ensuring safety and explainability of ML systems in safety-critical domains.
method Statistical difference measures of ECDF to detect distributional shifts.
result Approach can detect invalid application contexts of ML components.
New method improves safety analytics by addressing imbalanced data issues.
problem Imbalanced safety datasets lead to inaccurate predictions and management problems.
method Extended accident triangle theory and three oversampling methods.
result Robust improvements in machine learning algorithms for safety analytics.
Urban traffic systems worldwide are suffering from severe traffic safety problems. Traffic safety is affected by many complex factors, and heavily related to all drivers' behaviors involved in traffic system. Drivers with aggressive driving behaviors increase the risk of traffic accidents. In order to manage the safety…
The paper assesses how equity tail risk impacts US Treasury bond returns.
problem The effects of equity tail risk on the US government bond market.
method Estimating equity tail risk using option-implied stock market volatility and assessing its predictive power in reduced-form regressions and a term structure model.
result Equity tail risk significantly predicts one-month excess returns on Treasuries.
Recently, along with the emergence of food scandals, food supply chains have to face with ever-increasing pressure from compliance with food quality and safety regulations and standards. This paper aims to explore critical factors of compliance risk in food supply chain with an illustrated case in Vietnamese seafood in…
Risk-averse model uncertainty framework for safe reinforcement learning.
problem Safe decision making in uncertain environments.
method Risk-averse perspective towards model uncertainty using coherent distortion risk measures; equivalent to distributionally robust safe reinforcement learning problems; efficient, model-free implementation.
result Demonstrates robust performance and safety across perturbed test environments.
A novel approach for safe offline RL using latent safety constraints.
problem Balancing safety constraints and reward maximization in offline RL.
method Conditional Variational Autoencoders for latent safety modeling, Constrained Reward-Return Maximization.
result Our approach maintains safety compliance while optimizing rewards, outperforming existing methods.
Tail-Safe hedging uses reinforcement learning with a safety layer to manage financial risks.
problem Managing financial risks in derivatives trading with robustness and explainability.
method Combines distributional reinforcement learning with a CBF-QP safety layer to enforce financial constraints.
result Improves risk management without degrading central performance and avoids hard constraint violations.
Safe linear bandits over unknown polytopes avoid safety violations and suboptimal actions.
problem Online approach to linear programming with unknown constraints and risks.
method Doubly-optimistic strategy (DOSS) for safe linear bandits over polytopes.
result DOSS achieves tight bounds on efficacy regret and safety violations.
This project presents the results of a partnership between the Data Science for Social Good fellowship, Jakarta Smart City and Pulse Lab Jakarta to create a video analysis pipeline for the purpose of improving traffic safety in Jakarta. The pipeline transforms raw traffic video footage into databases that are ready to …
Paper proposes government indemnification for AI risks to solve judgment-proof problem.
problem Uninsurable risks from AI, especially existential risks, create a judgment-proof problem.
method A government-provided, mandatory indemnification program using risk-priced fees and Bayesian Truth Serum.
result The approach better leverages private information and signals risk mitigation efforts.
Theoretical limits on verifying self-improving systems without risking unbounded utility.
problem Formalizing and proving the limits of safety verification for self-improving systems.
method Developed dual conditions and used Holder's inequality, NP counting method, and Lipschitz bounds to establish impossibility and ceiling results.
result A classifier-based safety gate cannot simultaneously permit unbounded beneficial self-modification and bounded cumulative risk.
The paper shows why AI safety doesn't generalize across tasks.
problem AI safety fails to generalize across unseen tasks.
method Theoretical analysis of linear-quadratic control with H∞-robustness, empirical demonstrations in simulated quadcopter navigation and CRM. result The mapping from task specification to an optimal controller has a higher Lipschitz constant with safety requirements than without, indicating inherent complexity of safety.
High-dimensional models pose both safety benefits and risks.
problem Emergent problems in safety alignment due to high-dimensional representations.
method Detailed visualizations and lower-dimensional subspace projections.
result Dimensional reduction preserves safety alignment while avoiding linear jailbreaking.
Unified approach detects traffic conflicts across various interactions.
problem Inconsistent detection of traffic conflicts across different interactions.
method Unified probabilistic approach decomposes conflicts into statistical learning tasks.
result Effective collision warnings across diverse datasets and environments.
The paper explores new risk models for autonomous driving.
problem Risk management and actuarial modeling for autonomous vehicles.
method Examines technical difficulties and proposes a novel risk model.
result The new model better reflects real-world driving safety.
Bike usage in Smart Cities becomes paramount for sustainable urban development. Cycling provides tremendous opportunities for a more healthy lifestyle, lower energy consumption and carbon emissions as well as reduction of traffic jams. While the number of cyclists increase along with the expansion of bike sharing initi…
A new RL model ensures safe learning in uncertain environments.
problem Safe reinforcement learning in uncertain, partially observable environments.
method Lyapunov-based uncertainty quantification and Transformers for memory.
result Significant improvement in safety and optimality in grid-world tasks.
Deep-PrAE improves rare-event simulation for black-box systems.
problem Evaluating rare safety-critical events in learning-based systems.
method Combines deep neural networks with IS to create statistically guaranteed estimations.
result Deep-PrAE provides accurate bounds on safety-critical event probabilities.
Although aviation accidents are rare, safety incidents occur more frequently and require a careful analysis to detect and mitigate risks in a timely manner. Analyzing safety incidents using operational data and producing event-based explanations is invaluable to airline companies as well as to governing organizations s…
A new framework for SPX and VIX hedging that combines AI and market dynamics.
problem Jointly hedging SPX and VIX exposures under transaction costs and regime shifts.
method Integrates an SSVI-based implied-volatility surface and a Cboe-compliant VIX computation with a control layer that enforces safety as constraints.
result Reduces expected shortfall while suppressing nuisance turnover in a reproducible synthetic environment.
Safe RL-based vibration control using LQR guidance.
problem Training risks in RL-based vibration control.
method Hybrid control framework combining LQR and RL.
result LQR controller outperforms uncontrolled scenario.
New method improves reliability of object detection models.
problem Lack of reliability in neural networks for safety-critical applications.
method Sequential Conformal Risk Control (SeqCRC) for object detection.
result Valid statistical guarantees for any dataset size without prior knowledge.
Generates multimodal safety-critical scenarios for robustness evaluation of decision-making algorithms.
problem Lack of comprehensive evaluation of neural network robustness under real-world scenarios.
method Proposes a flow-based multimodal scenario generator using weighted likelihood maximization and gradient-based sampling.
result Demonstrates improved testing efficiency and multimodal modeling capability compared to traditional methods.
Improves pre-trial risk assessments by making them safer without changing existing rules.
problem Improving pre-trial risk assessments while maintaining deterministic rules.
method Developed a maximin robust optimization approach to find a safer policy.
result Can safely improve certain components of the risk assessment instrument.
Enhances crowd safety through AI and data-driven models.
problem Improving crowd safety during events.
method Innovative data collection, AI, and machine learning.
result Accurate multi-day forecasts for event planning.
A new framework ensures model safety by retaining old model capabilities while improving new tasks.
problem Catastrophic forgetting in continual learning systems.
method Retention-centric framework with data-dependent constraints and an efficient constrained optimization algorithm.
result The method ensures that new models retain important capabilities of old models while improving new tasks.
Develops optimal uncertainty quantification for risk-averse decision makers.
problem Quantifying prediction uncertainty for risk-sensitive domains.
method Decision-theoretic foundations connecting uncertainty quantification with risk-averse decision-making.
result Risk-Averse Calibration (RAC) algorithm provides optimal prediction sets for risk-averse decision makers.
This paper tackles sandbagging in AI safety evaluations.
problem AI agents may hide dangerous capabilities to avoid being deactivated.
method Developed a simple model of strategic deception in sequential decision-making tasks.
result Demonstrated that optimal rational agents exhibit sandbagging behavior.
Predict real-time crash risks during hurricane evacuations using connected vehicle data.
problem Mitigate crash risks during hurricane evacuations by predicting high-risk locations.
method Used connected vehicle data to predict crash risks in real-time, considering weather and traffic features.
result Gaussian Process Boosting and Extreme Gradient Boosting models performed best, with recall of 0.91.
Paper explores anchoring for vision models, improving generalization and safety.
problem Anchoring can lead to undesirable shortcuts, limiting generalization.
method Introduced a new anchored training protocol with a regularizer to mitigate undesirable shortcuts.
result Significant performance gains in generalization and safety metrics.
We study the risk assessment of uncertain cash flows in terms of dynamic convex risk measures for processes as introduced in Cheridito, Delbaen, and Kupper (2006). These risk measures take into account not only the amounts but also the timing of a cash flow. We discuss their robust representation in terms of suitably p…
CRC improves multivariate forecasting accuracy without risking performance degradation.
problem Systematic errors and lack of guarantees in multivariate forecasters.
method CRC uses a causality-inspired encoder and hybrid corrector with a safety mechanism.
result CRC consistently improves accuracy and ensures high non-degradation rates.
New framework uses OR to ensure AI systems make safe decisions.
problem Ensuring generative AI systems make safe decisions as they gain autonomy.
method Developed a conceptual framework combining flow-based models and adversarial robustness.
result Increased autonomy requires new OR approaches for feasibility, robustness, and stress testing.
Paper presents LSTMMDN for hourly bike flow estimation in Copenhagen.
problem Sparse or unavailable hourly bike flow data for safety analysis.
method Hybrid LSTM MDN model for hourly bike flow estimation.
result 66-77% more accurate bike flow estimates compared to calibration factors.
Safe exploration method for RL under disturbance ensures safety with probabilistic guarantees.
problem Safe reinforcement learning in real environments with disturbance.
method Uses partial prior knowledge and conservative inputs to ensure state constraint satisfaction.
result Guaranteed safety with pre-specified probability in the presence of stochastic disturbance.
CalNF models rare failures with limited data, improving safety in autonomous systems.
problem Challenges in modeling and debugging rare safety-critical failures due to limited data.
method CalNF, a self-regularized framework for posterior learning from limited data.
result Achieves state-of-the-art performance on data-limited failure modeling and inverse problems.