This paper develops a framework for efficient decision-making under time pressure.
problem Efficient decision-making under time pressure and subjective tradeoffs.
method Unified framework for evidence-based decision-making under time pressure.
result Ability to model and understand decision-making behavior under time constraints.
Novel IRL method identifies suboptimal medical decisions in ICU data.
problem Identifying suboptimal medical decisions in clinical settings.
method Incorporates Inverse Reinforcement Learning with a pruning step to identify and remove suboptimal actions.
result Pruning step effectively identifies clinical priorities and values from suboptimal data.
We model human decision-making behaviors in a risk-taking task using inverse reinforcement learning (IRL) for the purposes of understanding real human decision making under risk. To the best of our knowledge, this is the first work applying IRL to reveal the implicit reward function in human risk-taking decision making…
Given a set of human's decisions that are observed, inverse optimization has been developed and utilized to infer the underlying decision making problem. The majority of existing studies assumes that the decision making problem is with a single objective function, and attributes data divergence to noises, errors or bou…
The paper addresses statistical inference for online decision-making in a contextual bandit setting.
problem Understanding the performance of reward models in online decision-making with contextual information.
method The paper uses the contextual bandit framework with a linear reward model and the ε-greedy policy to address the exploration-exploitation dilemma. It employs the martingale central limit theorem and inverse propensity score weighting to establish asymptotic normality of parameter estimators. result The online ordinary least squares estimator and the online weighted least squares estimator are asymptotically normal, providing insights into the performance of the reward model.
Neural network approximates Bayesian decision-making parameters.
problem Analytical intractability of Bayesian decision-making in naturalistic tasks.
method Neural amortization of Bayesian actor model.
result Efficient gradient-based inference of Bayesian actor model parameters.
Bayesian method estimates dynamics from near-optimal trajectories.
problem Estimating dynamics from near-optimal expert trajectories in reinforcement learning.
method Constraint-based Bayesian approach integrating expert near-optimality.
result Significant improvements in decision-making and transfer success.
Paper shows how noisy data can improve robust decision-making.
problem The challenge of noisy data in decision-making.
method Distributionally robust optimization (DRO) with a novel ambiguity set construction.
result Noisy data can lead to more robust and equitable decisions.
New method disentangles perceptual uncertainty and behavioral costs in partially observable systems.
problem Tackles inverse optimal control for non-linear partially observable systems.
method Probabilistic approach using maximum causal entropy formulations and local linearization.
result Disentangles perceptual factors and behavioral costs in sequential decision-making.
GEAR uses auxiliary data to estimate optimal decisions in studies with limited primary outcomes.
problem Estimating optimal decisions when primary outcomes are not available in experimental samples.
method GEAR uses augmented inverse propensity weighting to estimate optimal decisions based on auxiliary data.
result GEAR estimators and value estimators have established asymptotic properties and are validated in simulations and a real application.
The paper proposes a method to explain expert decisions by modeling preferences with 'what if' outcomes.
problem Interpreting and auditing decision-making policies in institutions.
method Integrating counterfactual reasoning into batch inverse reinforcement learning.
result The method effectively recovers accurate and interpretable descriptions of expert behavior.
Characterizing statistical properties of solutions of inverse problems is essential for decision making. Bayesian inversion offers a tractable framework for this purpose, but current approaches are computationally unfeasible for most realistic imaging applications in the clinic. We introduce two novel deep learning bas…
New method learns decisions from collective preferences without individual covariates.
problem Making decisions online without individual covariates.
method Collaborative filtering, matrix completion bandit, ε-greedy policy, online gradient descent, inverse propensity weighting.
result Method outperforms benchmarks and reveals new discoveries.
Inverse reinforcement learning (IRL) infers a reward function from demonstrations, allowing for policy improvement and generalization. However, despite much recent interest in IRL, little work has been done to understand the minimum set of demonstrations needed to teach a specific sequential decision-making task. We fo…
We address the problem of inverse reinforcement learning in Markov decision processes where the agent is risk-sensitive. In particular, we model risk-sensitivity in a reinforcement learning framework by making use of models of human decision-making having their origins in behavioral psychology, behavioral economics, an…
Paper addresses regret minimization and inference in high-dimensional online decision-making.
problem Regret minimization and statistical inference in high-dimensional online decision-making.
method Integrates ε-greedy bandit algorithm with hard thresholding for sparse bandit parameters and debiasing method for inference.
result Achieves either O(T1/2) regret or O(T1/2)-consistent inference, with trade-off between exploration and exploitation. New method uses CNN for seismic inversion uncertainty quantification.
problem Uncertainty quantification in seismic inversion for noisy data.
method Plug-and-Play Stein Variational Gradient Descent (PnP-SVGD) with CNN denoiser.
result High-resolution, trustworthy posterior samples for subsurface structures.
The paper addresses human-like decision-making in multi-agent systems using bounded risk-sensitive Markov Games.
problem Modeling human-like decision-making in multi-agent systems with risk-seeking and loss-aversion behaviors.
method Forward policy design and inverse reward learning with iterative reasoning and cumulative prospect theory.
result The proposed algorithms demonstrate both risk-averse and risk-seeking behaviors in multi-agent systems.
This work combines machine learning with physical models to solve inverse problems efficiently.
problem Solving inverse problems in the presence of missing physics and recovering parameters.
method Variational autoencoding with a physically structured decoder network and stochastic local approximations.
result The method accelerates inference for Bayesian inverse problems and acts as a regularizer encoding prior physical information.
New framework calibrates decision robustness using inverse conformal risk control.
problem Inadequate robustness levels in decision-making due to ad hoc choices.
method Constructs valid estimators to trace miscoverage-regret Pareto frontier.
result Provides distribution-free, finite-sample guarantees on robustness levels.
An online decision-making algorithm using stochastic gradient descent for big data.
problem Efficiently updating decision rules in online decision making with big data.
method Stochastic gradient descent for online updates, asymptotic normality of estimators.
result Asymptotic normality of parameter and value estimators, enabling statistical inference.
AC-State discovers minimal latent state for control.
problem Discover minimal latent state from sensory information.
method Multi-step inverse model with information bottleneck.
result Guaranteed discovery of control-endogenous latent states.
The paper proposes a method to learn and leverage contextual preference distributions for better decision-making.
problem Heterogeneous and context-dependent human preferences in decision-making problems.
method A sequential learning-and-optimization pipeline using a bounded-variance score function gradient estimator to train a predictive model mapping contextual features to preference distributions.
result The approach reduces average post-decision surprise by up to 25 times compared to risk-averse baselines in a ridesharing environment.
Optimizes insurance pricing by accounting for policyholders' price sensitivity.
problem Traditional insurance pricing does not consider policyholders' price sensitivity.
method Formulates insurance pricing as a decision-making problem and uses off-policy evaluation and stochastic control.
result Neural networks outperform existing techniques for policy optimization.
Efficient algorithm for learning from indirect feedback in complex decision-making scenarios.
problem Learning from indirect feedback in realistic scenarios with personalized mechanisms.
method IGW algorithm for policy optimization, extending reward-estimator construction from single-step to multi-step.
result Achieves sublinear regret guarantee for contextual episodic MDPs with personalized feedback.
This study develops a dynamic inverse optimization framework to recover hidden, time-varying preferences from observed allocation trajectories.
problem The gap between classical optimization theory and real-world practice, especially in the presence of drift and shocks.
method Dynamic inverse optimization framework using a drift-aware estimator grounded in convex analysis and online learning theory.
result Sharp static and dynamic regret bounds for the framework, demonstrating its responsiveness to gradual drift and sudden shocks.
Active IRL selects optimal human demonstrations for learning AI preferences.
problem Costly human demonstrations in IRL for autonomous systems.
method Information-theoretic acquisition function for selecting informative human demonstrations.
result Efficiently reduces human effort in learning AI preferences.
Pessimistic estimator improves multi-objective policy optimization.
problem Optimizing multi-objective policies from existing data.
method Pessimistic estimator based on inverse propensity scores (IPS).
result Pessimistic estimator outperforms naive IPS estimator in theory and experiments.
Paper proposes a robust method for inferring parameters in multiobjective optimization.
problem Uncertainty in hypothetical decision-making problem, data quality, and parameter space.
method Wasserstein distributionally robust approach for inverse multiobjective optimization.
result WRO-IMOP minimizes worst-case expected loss over a Wasserstein ball of distributions.
New methods improve off-policy evaluation for survival outcomes with censoring.
problem Systematic underestimation of policy performance due to censoring bias in survival outcomes.
method Proposes IPCW-IPS and IPCW-DR to handle censoring bias in survival outcomes.
result The proposed methods are unbiased and achieve double robustness.
Unified framework for estimating reward functions in competitive games.
problem Estimating unknown reward functions in competitive games.
method Unified framework with entropy regularization for reward function recovery.
result Strong theoretical guarantees and practical effectiveness demonstrated.
New method improves causal effect estimation by addressing imbalance in training data.
problem Imbalance between treatment and control groups in training data.
method Combines distributionally robust optimization and weight regularization.
result Consistent improvements over existing methods in experiments.
Paper proposes a new framework for hypothesis testing in imaging.
problem Challenges in hypothesis testing for imaging data.
method Combines self-supervised imaging, vision-language models, and non-parametric hypothesis testing.
result Demonstrates improved power and robust error control in image-based phenotyping.
New method reduces regret in budgeted learning problems.
problem Decision-making with limited reward queries.
method Confidence-Budget Matching (CBM) principle.
result CBM-based algorithms perform well in adversarial settings.
Paper uses inverse optimization to measure risk preference from investment portfolios.
problem Measuring subjective risk preference in investment portfolios.
method Inverse optimization on mean-variance framework.
result Quantified risk preference parameters validated with existing measures.
Method learns evolving policies in healthcare contexts.
problem Understanding non-stationary behavior in evolving decision-making processes.
method Inverse Contextual Bandits (ICB) approach for learning interpretable representations of non-stationary behavior.
result Demonstrated applicability and accuracy of ICB method in liver transplantation policies.
INVERT connects neural representations to human-understandable concepts.
problem Lack of understanding and statistical significance in existing explainability methods.
method Inverse Recognition (INVERT) approach that connects learned representations to human-understandable concepts.
result INVERT provides interpretable metrics and statistical significance for representation alignment.
New method identifies subgroups in censored data.
problem Identifying meaningful patterns in heterogeneous populations.
method Combining inverse probability weighting, M-estimation, and concave pairwise fusion penalization.
result Robust approach for censored data under heterogeneous AFT models.
Proposes a real-time anomaly detection system using IRL.
problem Real-time anomaly detection in sequential data.
method Uses inverse reinforcement learning to infer reward function and evaluate anomalies.
result Effective in identifying anomalies in real-world data.
New RL approach handles non-exponential discounting for sequential decisions.
problem Modeling human discounting in sequential decision-making tasks.
method Generalized model-based reinforcement learning with arbitrary discount functions, using Hamilton-Jacobi-Bellman equation and collocation method.
result Validated approach on simulated problems, showing applicability to human discounting.
Unified LP framework for offline reward learning from human demonstrations and feedback.
problem Reward learning from human demonstrations and feedback with robustness and sample efficiency.
method A novel linear programming framework for offline reward learning.
result Unified LP framework achieves better performance compared to MLE.
A new method generates counterfactual treatment outcomes for time-varying treatments.
problem Estimating counterfactual outcomes for time-varying treatments with high-dimensional outcomes.
method Conditional generative framework with inverse probability re-weighting.
result Our method outperforms state-of-the-art baselines in generating high-quality counterfactual samples.
This study improves uncertainty estimation in FWI using SVGD with annealing and PCA.
problem Uncertainty quantification in FWI solutions.
method Annealed Stein Variational Gradient Descent (SVGD) combined with multi-scale strategy and PCA.
result Improved uncertainty estimation in FWI solutions.
Causal Interaction Trees identify treatment subgroup effects in observational data.
problem Identifying subgroups with enhanced treatment effects in observational studies.
method Extending Classification and Regression Trees with subgroup-specific treatment effect estimators.
result The proposed algorithms enhance treatment effect heterogeneity in subgroups.
Paper tackles robust decision-making from multiple sites with shared structure.
problem Learning robust sequential decisions from heterogeneous multi-site datasets.
method Group-Robust MDPs with d-rectangular uncertainty sets, feature-wise worst-case aggregation, and cluster-level pooling.
result Proves suboptimality bound for robust planning policy under robust partial coverage assumption.
This work bridges federated learning and contextual bandits, enhancing FL's utility.
problem Limited use of federated learning in contextual bandits despite its potential.
method Proposes FedIGW, a novel federated contextual bandits design that leverages regression-based algorithms and integrates various FL components.
result FedIGW better harnesses FL innovations and provides flexible, modular, and seamless integration of FL elements.
This review explores causal decision-making to improve decision quality.
problem Effective decision-making requires understanding causal relationships.
method Causal structure learning, causal effect learning, and causal policy learning.
result Challenges in causal decision-making are identified and recent advances are discussed.
Framework for robust decision making in changing environments with privacy constraints.
problem Interactive decision making in changing environments with constraints.
method Hybrid Decision Making with Structured Observations (hybrid DMSO) framework, local differentially private decision making, query-based learning, robust and smooth decision making.
result Strong connections and bounds derived for DEC, SQ dimension, local minimax complexity, learnability, and joint differential privacy.