Advances in mobile computing technologies have made it possible to monitor and apply data-driven interventions across complex systems in real time. Markov decision processes (MDPs) are the primary model for sequential decision problems with a large or indefinite time horizon. Choosing a representation of the underlying…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “decision process”
Develops methods for finding counterfactual explanations in sequential decision making.
problem Finding counterfactual explanations for sequential decision making processes.
method Formal characterization of sequential actions and states using Markov decision processes and Gumbel-Max structural causal model. Introduces a polynomial time algorithm based on dynamic programming.
result Algorithm finds optimal counterfactual explanations for sequential decision making.
Post-processing predictors reduces calibration errors for decision-making.
problem Predictors with low calibration error for machine learning may have high error for decision-making.
method Post-processing with ε distance to calibration adds noise to make predictions differentially private.
result Post-processing achieves O(√ε) ECE and CDL, asymptotically optimal.
New algorithm solves uncertain Markov decision processes using Wasserstein uncertainty.
problem Solving Markov decision processes with uncertain transition probabilities.
method Distributionally robust -learning algorithm for Wasserstein uncertainty.
result Convergence of the algorithm proved and demonstrated with real data.
Scalable and Decentralized Algorithms for Anomaly Detection via Learning-Based Controlled Sensingcs.LG
New algorithms detect anomalies in processes with minimal delay.
problem Sequentially selecting and observing processes to identify anomalies.
method Developed centralized and decentralized anomaly detection algorithms using reinforcement learning.
result Minimizes delay in decision making while identifying anomalies accurately.
AI learns to design chemical processes efficiently.
problem Designing efficient chemical processes.
method Hierarchical reinforcement learning and graph neural networks.
result Quick learning in various decision spaces.
We solve a broad class of sequential decision-making problems with partially observed states.
problem Sequential decision-making under uncertainty with partially observed states.
method Modeling as a partially observed Markov decision process (POMDP) and separating state and modulation process.
result The approach allows for specialized approximate solution procedures.
Study of bandit problem with Poisson decision times and Lévy processes.
problem Continuous-time multi-armed bandit problem with Poisson decision times.
method Gittins index policy applied to spectrally one-sided Lévy processes.
result Gittins index converges to classical Lévy bandit index.
The Chain-of-Decision approach improves forecasting of financial professionals' trading decisions.
problem Challenges in forecasting professionals' behaviors, especially in trading decisions.
method Integrates an opinion-generator-in-the-loop to provide subjective analysis based on news items.
result Promising improvements in the proposed tasks' performance.
New learning process for neural classifiers simplifies decision weights.
problem Training decision layer weights in neural classifiers.
method Choosing appropriate loss functions and solving constrained optimization problems.
result A new learning process for pre-decision weights that is simple and effective.
Paper presents an algorithm for optimal regret in communicating Markov decision processes.
problem Achieving optimal regret in Markov decision processes with a communicating assumption.
method The algorithm explicitly tracks the constant K(M) to learn optimally, balancing exploration, co-exploration, and exploitation.
result The algorithm achieves asymptotically optimal regret for communicating Markov decision processes.
New method improves decision-making accuracy without complex calculations.
problem Improving decision-making accuracy in machine learning.
method Introducing a new measure called calibration decision loss () for structured families of post-processing functions.
result Proves upper and lower bounds for natural classes of post-processing functions.
A new model predicts spatio-temporal data using adaptive decision trees and point processes.
problem Predicting spatio-temporal data with real-life applications.
method Hawkes process, adaptive decision tree, joint optimization algorithm.
result Significant improvement in predictions compared to standard methods.
Study minimax-optimal rates for offline decision-making with function approximation.
problem Statistical complexity of offline decision-making with function approximation.
method Near minimax-optimal rates for stochastic contextual bandits and Markov decision processes, using pseudo-dimension and behavior policy.
result Established performance limits and new characterization of behavior policy.
Research in psychology and neuroscience has successfully modeled decision making as a process of noisy evidence accumulation to a decision bound. While there are several variants and implementations of this idea, the majority of these models make use of a noisy accumulation between two absorbing boundaries. A common as…
Gaussian processes adapted for non-Euclidean spaces enhance decision-making.
problem Applying Gaussian processes in non-Euclidean spaces.
method Developed pathwise conditioning and Gaussian process models over non-Euclidean spaces.
result Efficient Gaussian process models for non-Euclidean spaces.
We address the problem of inverse reinforcement learning in Markov decision processes where the agent is risk-sensitive. In particular, we model risk-sensitivity in a reinforcement learning framework by making use of models of human decision-making having their origins in behavioral psychology, behavioral economics, an…
Develops an actor-critic algorithm for risk-sensitive Markov decision processes.
problem Risk-sensitive cost criterion in Markov decision processes.
method Actor-critic algorithm with function approximation.
result Asymptotic convergence of the actor-critic algorithm.
Study optimality in safety-constrained Markov decision processes using asynchronous value iteration and modified Q-learning.
problem Optimality in safety-constrained Markov decision processes with multichain structure.
method Formulated as a zero-sum game, constructed asynchronous value iteration scheme and modified Q-learning algorithm.
result Resolved Bellman's principle of optimality for multichain Markov decision processes and provided learning algorithms.
CPR models complex decision processes by breaking them into context-specific policies, improving interpretability and accuracy.
problem Interpreting dynamic human decision-making processes in medical contexts.
method Develops Contextualized Policy Recovery (CPR) framework for multi-task learning, modeling each context-specific policy as a linear map.
result Achieves state-of-the-art performance in predicting medical decisions, closing the gap between interpretable and black-box methods.
Risk-controlled post-processing optimizes decision policies under risk constraints.
problem Optimizing decision policies with risk constraints for better outcomes.
method Developed a post-processing algorithm that selects a threshold based on fitted fallback policy and score, leveraging tools from algorithmic stability and stochastic processes.
result The post-processed policy achieves precise expected risk control under exchangeability and meets or nearly meets risk budgets while preserving more agreement with the baseline.
Overview of risk-sensitive Markov decision processes with Optimized Certainty Equivalent.
problem Optimizing decision-making under risk in Markov processes.
method Analyzes risk-sensitive criteria using Optimized Certainty Equivalent, including entropic risk and Conditional Value-at-Risk.
result Conditions for the existence of optimal policies and solution procedures are provided.
GPs' decisions can vary significantly with different kernels, even if kernels are qualitatively similar.
problem Robustness of GP decisions to kernel choice.
method Solved a constrained optimization problem over a finite-dimensional space to identify changes in GP decisions.
result Decisions made with a GP can be non-robust to kernel choice, even with qualitatively similar kernels.
A popular methodology for building binary decision-making classifiers in the presence of imperfect information is to first construct a non-binary "scoring" classifier that is calibrated over all protected groups, and then to post-process this score to obtain a binary decision. We study the feasibility of achieving vari…
A scalable online method for Gaussian processes that improves decision-making in various applications.
problem Scalability issues with Gaussian processes for online decision-making.
method Online variational conditioning (OVC) for SVGPs.
result OVC enables efficient online learning and decision-making with SVGPs.
Bayesian method infers local rules for collective animal movement.
problem Learn local rules governing long-term group behaviors.
method Bayesian Inverse Reinforcement Learning with Linearly-Solvable Markov Decision Process.
result Recover true costs and find value of collective movement.
Sampling Decisionscs.LG
Introduces DF framework for sampling decisions from target distributions.
problem Sampling from target distributions with additional guidance.
method DF framework based on MDP and Path Integral Diffusion.
result DF enhances guided sampling across various applications.
In Random Forests, proximity distances are a metric representation of data into decision space. By observing how changes in input map to the movement of instances in this space we are able to determine the independent contribution of each feature to the decision-making process. For binary feature vectors, this process …
A new MDP with Bandits approach for sequential decision making in linear-flow scenarios.
problem Sequential decision making with limited feedback in a linear-flow context.
method Formulated as an MDP with Bandits, using Thompson sampling for action selection and exact dynamic programming for allocation.
result The proposed MDP with Bandits algorithm outperforms other methods in sequential decision making.
Unified framework for human-like decision making in various sequential tasks.
problem Real-life decision-making involves diverse strategies leading to similar outcomes.
method Two-stream reward processing mechanism for flexible and unified models.
result Framework unified MAB, CB, and RL with comparable performance.
Dynamic probabilistic forecasts guide optimal decisions in uncertain processes.
problem Optimal decision making in processes influenced by uncertain random factors.
method Stochastic models for probabilistic forecast evolution, calibrated from ensemble forecasts.
result Optimal decision strategies determined using dynamic probabilistic forecasts.
Algorithm finds safe zones in policy Markov Decision Processes to limit trajectory escape.
problem Finding safe zones in policy Markov Decision Processes to limit trajectory escape.
method Bi-criteria approximation learning algorithm with polynomial sample complexity.
result Achieves almost 2 approximation for both escape probability and safe zone size.
This work extends reinforcement learning to handle non-cumulative objectives.
problem Optimizing functions of rewards rather than their sum in decision processes.
method Mapping NCMDPs to standard MDPs for reinforcement learning.
result Reinforcement learning techniques can be applied to NCMDPs.
Modeling decision-making dynamics in MDD patients using RL-HMM.
problem Characterize reward learning strategies in MDD patients.
method Proposed RL-HMM framework to analyze reward-based decision-making.
result MDD patients show reduced engagement in RL compared to healthy controls.
In this paper, we use reinforcement learning to find effective decoding strategies for binary linear codes. We start by reviewing several iterative decoding algorithms that involve a decision-making process at each step, including bit-flipping (BF) decoding, residual belief propagation, and anchor decoding. We then ill…
The Markov assumption (MA) is fundamental to the empirical validity of reinforcement learning. In this paper, we propose a novel Forward-Backward Learning procedure to test MA in sequential decision making. The proposed test does not assume any parametric form on the joint distribution of the observed data and plays an…
Fair Pipelinescs.CY
This work facilitates ensuring fairness of machine learning in the real world by decoupling fairness considerations in compound decisions. In particular, this work studies how fairness propagates through a compound decision-making processes, which we call a pipeline. Prior work in algorithmic fairness only focuses on f…
A new framework designs experiments for better decision-making.
problem Suboptimal experimental designs for downstream decision-making.
method Amortized decision-aware Bayesian Experimental Design (BED) with Transformer Neural Decision Process (TNDP).
result TNDP effectively designs experiments and facilitates accurate decision-making.
We develop robust Markov Decision Processes with risk measures for uncertain environments.
problem Uncertainty in Markov Decision Processes and its impact on risk measures.
method Formulation as a Stackelberg game, robust cost and value iterations, existence of optimal policies.
result Existence of deterministic optimal policies for robust optimization and risk measures.
The paper tackles batch policy learning in Markov Decision Processes, focusing on average reward maximization.
problem Maximizing long-term average reward in Markov Decision Processes with batch learning.
method Doubly robust estimator for average reward, optimization algorithm for optimal policy, finite-sample regret guarantee.
result The proposed method achieves semiparametric efficiency and provides a finite-sample regret guarantee.
The paper tackles robust policy learning in MDPs using statistical methods.
problem Offline data-driven sequential decision making in MDPs.
method Evaluates policies using average rewards centered at policy-induced stationary distributions. Developed a statistically efficient method for estimating robust optimal policies.
result Established a rate-optimal regret bound up to a logarithmic factor.
We introduce Threatened Markov Decision Processes (TMDPs) as an extension of the classical Markov Decision Process framework for Reinforcement Learning (RL). TMDPs allow suporting a decision maker against potential opponents in a RL context. We also propose a level-k thinking scheme resulting in a novel learning approa…
New algorithms boost SAT solver performance by optimizing restart strategies.
problem Optimizing decision-making under time constraints with restarts.
method Developed online learning algorithms for a bandit problem with controlled restarts.
result Achieved and regret bounds.
The paper calculates the value of information in high-dimensional decision making.
problem Determining the value of acquiring new information in high-dimensional decision problems.
method Using tools from sub-Gaussian processes and generic chaining for asymptotic analysis.
result Asymptotic results on the expected value of information as dimensionality increases.
Deep neural networks (DNNs) may outperform human brains in complex tasks, but the lack of transparency in their decision-making processes makes us question whether we could fully trust DNNs with high stakes problems. As DNNs' operations rely on a massive number of both parallel and sequential linear/nonlinear computati…
Study optimal periodic dividend strategies for risky businesses with transaction costs.
problem Optimal periodic dividend strategies for spectrally positive Lévy risk processes with fixed transaction costs.
method Investigates periodic strategies for a Poisson arrival process of decision times.
result A periodic strategy is optimal with lump sum dividends net of transaction costs.
A distinctive property of human and animal intelligence is the ability to form abstractions by neglecting irrelevant information which allows to separate structure from noise. From an information theoretic point of view abstractions are desirable because they allow for very efficient information processing. In artifici…
Optimizes harvesting in biopharmaceutical fermentation with limited data.
problem High variability in fermentation outputs due to model risk with small data.
method Stochastic model, Bayesian approach, Markov decision process.
result Improves fermentation output and reduces variability.