arXiv research

Paper solves POMDPs in continuous time and discrete spaces.

Study a continuous-time PA problem with private effort and consumption decisions.

The paper tackles finding optimal treatment sequences in continuous state spaces.

We consider the optimization of an uncertain objective over continuous and multi-dimensional decision spaces in problems in which we are only provided with observational data. We propose a novel algorithmic framework that is tractable, asymptotically consistent, and superior to comparable methods on example problems. O…

Stochastic domains often involve risk-averse decision makers. While recent work has focused on how to model risk in Markov decision processes using risk measures, it has not addressed the problem of solving large risk-averse formulations. In this paper, we propose and analyze a new method for solving large risk-averse …

Study shows risk-averse investors have consistent ranking of risky assets.

SupRB learns rules for continuous decision problems from examples.

New techniques optimize decision trees for interpretable machine learning.

Many real-world control problems involve both discrete decision variables - such as the choice of control modes, gear switching or digital outputs - as well as continuous decision variables - such as velocity setpoints, control gains or analogue outputs. However, when defining the corresponding optimal control or reinf…

New algorithms boost SAT solver performance by optimizing restart strategies.

Study of bandit problem with Poisson decision times and Lévy processes.

Continuous-time algorithms improve online learning performance.

Partially observable Markov decision processes (POMDPs) with continuous state and observation spaces have powerful flexibility for representing real-world decision and control problems but are notoriously difficult to solve. Recent online sampling-based algorithms that use observation likelihood weighting have shown un…

Investment decision triggered by a convex curve in a two-factor uncertainty model.

We study the Combinatorial Pure Exploration problem with Continuous and Separable reward functions (CPE-CS) in the stochastic multi-armed bandit setting. In a CPE-CS instance, we are given several stochastic arms with unknown distributions, as well as a collection of possible decisions. Each decision has a reward accor…

Paper proposes a risk-aware decision-making framework for real-world sequential decisions.

The topic of learning to solve optimization problems has received interest from both the operations research and machine learning communities. In this work, we combine techniques from both fields to address the problem of learning to generate decisions to instances of continuous optimization problems where the feasible…

New RL framework models continuous-time dynamics using neural ODEs.

We learn sensor trees from training data to minimize sensor acquisition costs during test time. Our system adaptively selects sensors at each stage if necessary to make a confident classification. We pose the problem as empirical risk minimization over the choice of trees and node decision rules. We decompose the probl…

We formalise the widespread idea of interpreting neural network decisions as an explicit optimisation problem in a rate-distortion framework. A set of input features is deemed relevant for a classification decision if the expected classifier score remains nearly constant when randomising the remaining features. We disc…

New method optimizes decision-making in uncertain environments.

Bayesian decision theory outlines a rigorous framework for making optimal decisions based on maximizing expected utility over a model posterior. However, practitioners often do not have access to the full posterior and resort to approximate inference strategies. In such cases, taking the eventual decision-making task i…

Improved language models learn complex distributions using Fourier series.

Deep neural networks achieve optimal learning rates for high-dimensional classification.

New framework optimizes decisions under uncertainty considering causal and continuous data.

We develop robust Markov Decision Processes with risk measures for uncertain environments.

SMART combines decision trees and MARS for better regression modeling.

Paper tackles non-uniform coverage planning for robots.

Creating impact in real-world settings requires artificial intelligence techniques to span the full pipeline from data, to predictive models, to decisions. These components are typically approached separately: a machine learning model is first trained via a measure of predictive accuracy, and then its predictions are u…

New method learns binary decision trees efficiently.

Continuous-time optimal stopping solved with deep reinforcement learning

Proposes a new decision rule for continuous treatments.

Study resource allocation strategies in sequential decisions with unknown rewards.

Study minimax rates for binary classifier estimation with margin conditions.

Study on inventory management under uncertainty using smooth ambiguity preference.

New method extends low-rank MDPs to continuous action spaces.

Proposes a method for generating prediction intervals in dose-response models using conformal prediction.

Human decision-making underlies all economic behavior. For the past four decades, human decision-making under uncertainty has continued to be explained by theoretical models based on prospect theory, a framework that was awarded the Nobel Prize in Economic Sciences. However, theoretical models of this kind have develop…

We seek to learn an effective policy for a Markov Decision Process (MDP) with continuous states via Q-Learning. Given a set of basis functions over state action pairs we search for a corresponding set of linear weights that minimizes the mean Bellman residual. Our algorithm uses a Kalman filter model to estimate those …

New method tackles anomaly detection in video surveillance using continual learning.

The (contextual) multi-armed bandit problem (MAB) provides a formalization of sequential decision-making which has many applications. However, validly evaluating MAB policies is challenging; we either resort to simulations which inherently include debatable assumptions, or we resort to expensive field trials. Recently …

We study the problem of attacking a machine learning model in the hard-label black-box setting, where no model information is revealed except that the attacker can make queries to probe the corresponding hard-label decisions. This is a very challenging problem since the direct extension of state-of-the-art white-box at…

We consider Markov Decision Problems defined over continuous state and action spaces, where an autonomous agent seeks to learn a map from its states to actions so as to maximize its long-term discounted accumulation of rewards. We address this problem by considering Bellman's optimality equation defined over action-val…

Logarithmic regret for continuous-time reinforcement learning.

Paper proposes a new method to select memory data for online class-incremental learning.

The paper analyzes RL in high-frequency market making with theoretical and practical implications.

Develops deep jump learning for continuous treatment OPE.

OMLE combines optimism and MLE for efficient sequential decision making.