Classifier-based AI safety gates fail in self-improvement, even with advanced verification methods.
problem Maintaining reliable oversight of AI systems as they improve over iterations.
method Comprehensive empirical testing on neural controllers and MuJoCo benchmarks, using various classifiers and verification methods.
result Classifier-based safety gates fail in maintaining reliable oversight, even with advanced verification methods.
AGENTICAITA uses AI agents to autonomously trade markets without human intervention.
problem Inability of traditional trading systems to adapt to market complexity.
method Introduces an agentic AI framework with specialized LLM agents reasoning, negotiating, and acting.
result Demonstrated operational correctness and non-trivial inter-agent negotiation in live market conditions.
A new framework for SPX and VIX hedging that combines AI and market dynamics.
problem Jointly hedging SPX and VIX exposures under transaction costs and regime shifts.
method Integrates an SSVI-based implied-volatility surface and a Cboe-compliant VIX computation with a control layer that enforces safety as constraints.
result Reduces expected shortfall while suppressing nuisance turnover in a reproducible synthetic environment.
The paper discusses safety assessment for AI systems, focusing on machine learning models.
problem Safety assessment of AI systems, especially machine learning models, in safety-related applications.
method Analyzed AI models as statistical models and proposed a new budget allocation for AI safety.
result Safety assessment of AI systems requires a new approach focusing on the model used, not just the system.
This paper formalizes AI safety using hypothesis testing in GenAI.
problem Ensuring safety of generative AI tools that create realistic content.
method Formalization of computational safety through hypothesis testing and signal processing.
result Demonstrates how AI safety can be assessed quantitatively using mathematical frameworks.
AI systems that explain their decisions can be monitored for harmful intentions.
problem Monitoring AI systems' decision-making processes for harmful intentions is imperfect and can miss some misbehavior.
method Monitoring the chain of thought (CoT) of AI systems that communicate in human language.
result CoT monitoring is a promising but fragile approach to AI safety.
The paper shows why AI safety doesn't generalize across tasks.
problem AI safety fails to generalize across unseen tasks.
method Theoretical analysis of linear-quadratic control with H∞-robustness, empirical demonstrations in simulated quadcopter navigation and CRM. result The mapping from task specification to an optimal controller has a higher Lipschitz constant with safety requirements than without, indicating inherent complexity of safety.
This paper tackles sandbagging in AI safety evaluations.
problem AI agents may hide dangerous capabilities to avoid being deactivated.
method Developed a simple model of strategic deception in sequential decision-making tasks.
result Demonstrated that optimal rational agents exhibit sandbagging behavior.
Autonomous agents trained via reinforcement learning present numerous safety concerns: reward hacking, negative side effects, and unsafe exploration, among others. In the context of near-future autonomous agents, operating in environments where humans understand the existing dangers, human involvement in the learning p…
This paper explores how AI systems can learn moral behavior from economic entities.
problem Achieving moral behavior in AI systems.
method Analyses the analogy between machine learning and economic entities.
result Implicit specifications may work better than explicit ones for AI problems.
Insurance contracts for autonomous AI agents must be actuarially sound and resistant to gaming.
problem Designing insurance contracts for autonomous AI agents that are actuarially sound and resistant to gaming.
method Characterizing a five-attack space and proving the actuarial runtime is gaming-resistant.
result An incentive-compatible layer for actuarial control of autonomous-agent side effects.
New bounds on predicting agent behavior from behavior alone.
problem Predicting agent beliefs and intentions from observed behavior.
method Derivation of bounds on agent behavior in new environments under assumption of world model.
result Theoretical limits on predicting intentional agents from behavioral data.
ValueBlindBench tests LLM-generated investment rationales for validity before returns are known.
problem Delayed-ground-truth evaluation of LLM-generated investment rationales.
method Agreement-gated stress testing protocol to validate LLM-judged rationales.
result ValueBlindBench prevents overclaims and identifies flawed financial constructs.
Quantum circuits explained using Shapley values for better understanding.
problem Improving the explainability of quantum machine learning circuits.
method Applying Shapley values to quantify gate importance in quantum circuits.
result Quantum circuits can be explained by their gate importance, enhancing understanding and interpretability.
New risk metric for AI systems reduces safety risks with minimal data.
problem Risk assessment in multi-agent AI systems.
method Free Energy Principle applied to risk metrics, introducing Cumulative Risk Exposure.
result Gatekeepers improve system safety in autonomous vehicle fleets.
This paper tests LLMs in finance to assess ethical behavior.
problem Aligning AI with ethical and legal standards in finance.
method Prompted LLMs to simulate CEO behavior, analyzed with logistic regression.
result Significant heterogeneity in LLMs' unethical behavior propensity.
Study investigates how machine learning models degrade over time, leading to patient safety issues.
problem Overtime degradation of machine learning models in clinical settings.
method Used MIMIC-IV dataset to train models replicating commercial approaches, observing and analyzing degradation over a decade.
result An RNN model built on Epic features degrades from 0.729 AUC to 0.525 AUC over a decade, highlighting technical and clinical drift as root causes.
Proposes a method to ensure accurate estimation of rare events in AI systems.
problem Lack of efficiency guarantees in black-box systems for rare-event simulation.
method Integrates deep learning with importance sampling to create a statistically guaranteed estimator.
result Demonstrates effective estimation of rare-event probabilities in AI systems.
YRC-Bench benchmarks AI agents learning to collaborate with experts.
problem Learning to recognize when to consult an expert in novel situations.
method Validation strategy and proposer-validator decomposition.
result Insights for future AI safety research.
AI systems need reliable testing to ensure safety and trustworthiness.
problem Current AI Act lacks functional trustworthiness for AI systems.
method Define technical application distribution, set risk-based performance, and conduct statistically valid testing.
result Reliable functional trustworthiness is essential for AI systems.
In a world of global trading, maritime safety, security and efficiency are crucial issues. We propose a multi-task deep learning framework for vessel monitoring using Automatic Identification System (AIS) data streams. We combine recurrent neural networks with latent variable modeling and an embedding of AIS messages t…
Paper proposes government indemnification for AI risks to solve judgment-proof problem.
problem Uninsurable risks from AI, especially existential risks, create a judgment-proof problem.
method A government-provided, mandatory indemnification program using risk-priced fees and Bayesian Truth Serum.
result The approach better leverages private information and signals risk mitigation efforts.
AI monitors social distancing and masks at manufacturing plants.
problem Ensuring safety of workers during post-COVID production.
method Computer vision and AI techniques for social distancing and mask detection.
result Real-time alerts prevent violations of social distancing and mask-wearing.
Teaches uncertainty in ML through practical examples.
problem Lack of uncertainty teaching in ML curricula.
method Developed a curriculum and use cases.
result Motivates adoption of uncertainty concepts in AI courses.
ProEval efficiently estimates AI performance and discovers failures using pre-trained Gaussian Processes.
problem Resource-intensive evaluation of generative AI models.
method ProEval uses pre-trained Gaussian Processes and Bayesian quadrature to estimate performance and discover failures.
result ProEval requires significantly fewer samples to achieve accurate performance estimates and reveals more diverse failure cases.
Rapid advancements in deep learning have led to many recent breakthroughs. While deep learning models achieve superior performance, often statistically better than humans, their adoption into safety-critical settings, such as healthcare or self-driving cars is hindered by their inability to provide safety guarantees or…
Survey of algorithms for testing AI-driven CPS safety.
problem Testing AI-driven CPS for safety in complex environments.
method Survey of applied algorithms for safety validation.
result Survey of existing tools and techniques for safety validation.
New framework uses OR to ensure AI systems make safe decisions.
problem Ensuring generative AI systems make safe decisions as they gain autonomy.
method Developed a conceptual framework combining flow-based models and adversarial robustness.
result Increased autonomy requires new OR approaches for feasibility, robustness, and stress testing.
Tail-Safe hedging uses reinforcement learning with a safety layer to manage financial risks.
problem Managing financial risks in derivatives trading with robustness and explainability.
method Combines distributional reinforcement learning with a CBF-QP safety layer to enforce financial constraints.
result Improves risk management without degrading central performance and avoids hard constraint violations.
Quantile regression attacks outperform shadow models in unseen class membership inference attacks.
problem Failure of shadow model attacks on unseen classes due to limited data access.
method Quantile regression attacks that learn features of member examples.
result Quantile regression attacks achieve up to 11x higher TPR than shadow model-based approaches.
New method detects dangerous states in gridworlds using geometric defects.
problem Detecting dangerous states in gridworlds for AI safety.
method Modified state complexes and Gromov's Link Condition.
result Geometric defects indicate dangerous states in gridworlds.
Theoretical limits on verifying self-improving systems without risking unbounded utility.
problem Formalizing and proving the limits of safety verification for self-improving systems.
method Developed dual conditions and used Holder's inequality, NP counting method, and Lipschitz bounds to establish impossibility and ceiling results.
result A classifier-based safety gate cannot simultaneously permit unbounded beneficial self-modification and bounded cumulative risk.
The paper highlights AI brittleness and the need for robust testing out-of-distribution performance.
problem The brittleness of AI systems, especially Deep Neural Networks, limits their reliability and certification.
method Analysis of AI brittleness and OOD performance, emphasizing the need for resilience and improved evaluation methods.
result AI systems are more failure-prone than certified in critical systems, and OOD performance falls off gradually.
This research tackles uncertainty estimation in autoregressive structured prediction tasks.
problem Ensuring safety and robustness of AI systems through accurate uncertainty estimation.
method Develops a unified probabilistic ensemble-based framework for token-level and sequence-level uncertainty estimation.
result Provides baselines for error and out-of-domain detection on translation and speech recognition datasets.
Survey on biases in image analysis for industrial safety.
problem Bias in machine learning algorithms affects industrial safety-critical applications.
method Survey and analysis of recent advances in bias detection and mitigation.
result Need for new methods to detect and mitigate biases in image analysis for safety-critical applications.
An accurate road surface friction prediction algorithm can enable intelligent transportation systems to share timely road surface condition to the public for increasing the safety of the road users. Previously, scholars developed multiple prediction models for forecasting road surface conditions using historical data. …
Bayesian autoencoder detects AI safety risk of out-of-distribution inputs.
problem Detecting unreliable predictions from AI models with different distributions.
method Probabilistic, unsupervised Bayesian variational autoencoder with posterior estimation.
result Effective detection of out-of-distribution inputs in both input and latent spaces.
The paper investigates AI robustness through experiments and statistical analysis.
problem Inaccurate AI predictions can lead to safety and adoption issues.
method Design of experiments framework to study AI classification robustness.
result AI algorithms' robustness is influenced by various factors.
New method converts ANN gates to SNNs with AMOS neurons for improved image classification.
problem Efficiently converting ANN gates to SNNs for neuromorphic hardware.
method Introducing AMOS conversion for gates in ANNs, improving accuracy and throughput.
result Improved accuracy of SNNs for ImageNet from 74.60% to 80.97%.
AI learns to learn sequentially without forgetting.
problem Preventing catastrophic forgetting in machine learning models.
method Meta-learning a neuromodulatory activation-gating function to control selective activation in deep neural networks.
result State-of-the-art continual learning performance with 600 classes (9,000 updates).
Survey of alignment techniques for large language models.
problem Ensuring large language models align with human values.
method Analysis of diverse alignment methods and training paradigms.
result Preference-based methods offer more flexibility for nuanced alignment.
Localized Multidirectional Correction improves non-refusal target-response behavior in foundation models.
problem Controlled post-training refusal suppression in routed MoE and hybrid-MoE foundation models.
method Introduce Localized Multidirectional Correction (LoMC), a support-gated intervention framework.
result Substantially improves non-refusal target-response behavior while maintaining general capability under a compact intervention footprint.
Enhances crowd safety through AI and data-driven models.
problem Improving crowd safety during events.
method Innovative data collection, AI, and machine learning.
result Accurate multi-day forecasts for event planning.
Quantum model improves safety in machine learning.
problem Improving safety and robustness in machine learning models.
method Variational quantum classifier with amplitude encoding and SAFE-AI metrics.
result Quantum model provides competitive performance and improved robustness.
Graph-structured data appears frequently in domains including chemistry, natural language semantics, social networks, and knowledge bases. In this work, we study feature learning techniques for graph-structured inputs. Our starting point is previous work on Graph Neural Networks (Scarselli et al., 2009), which we modif…
This review clarifies XAI for regression models and establishes new theoretical insights.
problem Lack of XAI techniques for regression models, especially in safety-critical applications.
method Clarifies conceptual differences, establishes theoretical insights, provides demonstrations, discusses challenges.
result Novel theoretical insights and demonstrations of XAI for regression models.
New approach extracts AI model representations for steering and monitoring.
problem Understanding and steering AI models' internal knowledge representation.
method Feature learning advances for scalable linear concept representations.
result Concept representations enable steering and monitoring of AI models.
AI enhances bank credit risk management through deep learning and data analysis.
problem Inaccurate credit decisions and potential risks in bank credit risk management.
method Innovative application of AI technology, including deep learning and big data analysis.
result AI provides more accurate and comprehensive credit decision support, reducing risks and losses.