arXiv research

This paper detects Markov violations in RL with noise, improving policy development.

Paper introduces MVS to detect non-Markovian observations in reinforcement learning.

New algorithm reduces sample complexity for safe reinforcement learning.

New algorithm reduces regret in CMDPs without cancellation of errors.

New algorithm for reinforcement learning in uncertain environments with unknown thresholds.

New bounds for learning near-optimal policies in CMDPs with constraints.

As datasets capturing human choices grow in richness and scale -- particularly in online domains -- there is an increasing need for choice models that escape traditional choice-theoretic axioms such as regularity, stochastic transitivity, and Luce's choice axiom. In this work we introduce the Pairwise Choice Markov Cha…

New PDMP samplers improve BNN inference with accelerated computation.

New RL algorithm achieves sublinear regret and constraint violation without simulators.

New algorithm reduces regret and constraint violation in adversarial CMDP learning.

The paper uses conformal prediction for forecasting conflict sequences in Markov processes.

A key task in Bayesian statistics is sampling from distributions that are only specified up to a partition function (i.e., constant of proportionality). However, without any assumptions, sampling (even approximately) can be #P-hard, and few works have provided "beyond worst-case" guarantees for such settings. For log-c…

The paper analyzes CMDPs, balancing exploration and exploitation to avoid constraint violations.

In order to satisfy safety conditions, an agent may be constrained from acting freely. A safe controller can be designed a priori if an environment is well understood, but not when learning is employed. In particular, reinforcement learned (RL) controllers require exploration, which can be hazardous in safety critical …

New algorithm learns linear dynamical systems from measurements.

Paper develops a method to estimate value of a policy in confounded MDPs.

Paper revisits Deep Variational Information Bottleneck and proposes a new optimization approach.

Proposes M-CHMM for robust modeling of multivariate healthcare time series.

Bayesian methods detect significant IIA violations in similarity choice data.

Proposes a weaker faithfulness assumption for causal discovery.

The assumption of positivity in causal inference (also known as common support and co-variate overlap) is necessary to obtain valid causal estimates. Therefore, confirming it holds in a given dataset is an important first step of any causal analysis. Most common methods to date are insufficient for discovering non-posi…

Paper tackles HMM learning with unknown missing observation locations.

The correspondence between Riemann-Finsler geometries and effective field theories with spin-independent Lorentz violation is explored. We obtain the general quadratic action for effective scalar field theories in any spacetime dimension with Lorentz-violating operators of arbitrary mass dimension. Classical relativist…

This contribution to the CPT'13 meeting briefly introduces Lorentz and CPT violation and outlines two recent developments in the field.

In many real-world reinforcement learning (RL) problems, besides optimizing the main objective function, an agent must concurrently avoid violating a number of constraints. In particular, besides optimizing performance it is crucial to guarantee the safety of an agent during training as well as deployment (e.g. a robot…

In classical reinforcement learning, when exploring an environment, agents accept arbitrary short term loss for long term gain. This is infeasible for safety critical applications, such as robotics, where even a single unsafe action may cause system failure. In this paper, we address the problem of safely exploring fin…

Bipartite Riemann-Finsler geometries with complementary Finsler structures are constructed. Calculable examples are presented based on a bilinear-form coefficient for explicit Lorentz violation.

Survey of RL methods for control systems with time delays.

Detecting faults and SLA violations in a timely manner is critical for telecom providers, in order to avoid loss in business, revenue and reputation. At the same time predicting SLA violations for user services in telecom environments is difficult, due to time-varying user demands and infrastructure load conditions. In…

We present a new approach to assessing the robustness of neural networks based on estimating the proportion of inputs for which a property is violated. Specifically, we estimate the probability of the event that the property is violated under an input model. Our approach critically varies from the formal verification f…

PDMP samplers improve Bayesian PDE coefficient inference.

CVTMLE improves statistical inference in settings of positivity or Donsker class violations.

The study proves strong cosmic censorship violation for spherically symmetric dust clouds.

We show that any open subset of a contact manifold of dimension greater than three contains a certain non-convex hypersurface violating the Thurston-Bennequin inequality.

This paper considers online convex optimization over a complicated constraint set, which typically consists of multiple functional constraints and a set constraint. The conventional online projection algorithm (Zinkevich, 2003) can be difficult to implement due to the potentially high computation complexity of the proj…

NucleusDiff models atomic nuclei interactions to prevent separation violations in drug design.

New algorithm reduces regret and constraint violation in online convex optimization with complex constraints.

Optimistic algorithm reduces regret and constraint violations in online convex optimization with adversarial constraints.

Algorithm minimizes loss and constraint violations in online convex optimization with smooth penalties.

CausalCompass evaluates TSCD robustness under violations of modeling assumptions.

Paper proposes a model-free algorithm for CMDPs with long-term constraints, achieving optimal regret bounds.

In real-world scenarios, the observation data for reinforcement learning with continuous control is commonly noisy and part of it may be dynamically missing over time, which violates the assumption of many current methods developed for this. We addressed the issue within the framework of partially observable Markov Dec…

Paper tackles constrained bandit problems with a new learning framework.

A new RL model ensures safe learning in uncertain environments.

Causality violations are typically seen as unrealistic and undesirable features of a physical model. The following points out three reasons why causality violations, which Bonnor and Steadman identified even in solutions to the Einstein equation referring to ordinary laboratory situations, are not necessarily undesirab…

Measures policy-violating content prevalence with ML-assisted sampling and LLM labeling.

Motivated by the widespread use of temporal-difference (TD-) and Q-learning algorithms in reinforcement learning, this paper studies a class of biased stochastic approximation (SA) procedures under a mild "ergodic-like" assumption on the underlying stochastic noise sequence. Building upon a carefully designed multistep…

We study primal-dual type stochastic optimization algorithms with non-uniform sampling. Our main theoretical contribution in this paper is to present a convergence analysis of Stochastic Primal Dual Coordinate (SPDC) Method with arbitrary sampling. Based on this theoretical framework, we propose Optimality Violation-ba…