Subjective expected utility theory assumes that decision-makers possess unlimited computational resources to reason about their choices; however, virtually all decisions in everyday life are made under resource constraints - i.e. decision-makers are bounded in their rationality. Here we experimentally tested the predic…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “human decision subjects”
Study on model accuracy when human subjects adapt to predictions.
problem Model accuracy on human-adaptive data.
method Formalized transferability study with upper and lower bounds.
result Performance gap between source and induced target distributions.
Conformal prediction sets improve human decision making by quantifying model uncertainty.
problem Humans signal uncertainty and offer alternatives when unsure, but machine learning models often lack this feature.
method Conducted a randomized controlled trial with human subjects given conformal prediction sets.
result Human accuracy improves when given conformal prediction sets compared to fixed-size prediction sets.
AI system narrows human decision options for better outcomes.
problem Improving human decision-making in sequential tasks.
method Developed a decision support system using a pre-trained AI agent to limit human action choices.
result Participants outperformed AI and solo play in a wildfire mitigation game.
Human irrationality can improve AI design, study shows.
problem Improving AI by learning from human decision-making biases.
method Developed a novel POMDP model to simulate human decision-making in contextual choice tasks.
result Reinforcement learners can exploit human irrationalities to make better decisions.
With an eye towards human-centered automation, we contribute to the development of a systematic means to infer features of human decision-making from behavioral data. Motivated by the common use of softmax selection in models of human decision-making, we study the maximum likelihood parameter estimation problem for sof…
New decision support systems use prediction sets to help experts update their predictions, improving performance.
problem Challenges in explaining and updating predictions from decision support systems.
method Developed a methodology leveraging nested structure of prediction sets and counterfactual monotonicity to improve performance.
result Limiting experts' agency leads to better performance in decision support systems based on prediction sets.
Testing Ambiguity and Machina Preferences Within a Quantum-theoretic Framework for Decision-makingecon.GN
The Machina thought experiments pose to major non-expected utility models challenges that are similar to those posed by the Ellsberg thought experiments to subjective expected utility theory (SEUT). We test human choices in the `Ellsberg three-color example', confirming typical ambiguity aversion patterns, and the `Mac…
This article outlines a method for automatically generating models of dynamic decision-making that both have strong predictive power and are interpretable in human terms. This is useful for designing empirically grounded agent-based simulations and for gaining direct insight into observed dynamic processes. We use an e…
AI enhances financial services but humans are irreplaceable for empathy, presence, and ethics.
problem AI's limitations in financial services, especially with small datasets and human judgment.
method EPOCH framework highlighting five irreplaceable human capabilities: Empathy, Presence, Opinion, Creativity, and Hope.
result Humans are essential for trust, innovation, and consumer experience in financial services.
Partner-aware algorithms improve AI collaboration in multi-agent settings.
problem Improving AI cooperation in teams with shared rewards.
method Proposed Partner-Aware strategy extending Upper Confidence Bound for decentralized MAB.
result Achieves logarithmic regret in collaborative decision-making.
Paper introduces -Perfect to estimate model-human correlation in subjective datasets.
problem Inherent noise in subjective ratings limits model-human correlation quantification.
method Defines -Perfect as highest achievable correlation between perfect predictor and human ratings. Estimates based on heteroscedastic noise scenarios.
result Demonstrates -Perfect can distinguish model limitations from data quality issues.
Develops a measure for subjective explainability of ML predictions.
problem Ensuring transparency and trust in automated decision-making.
method Information-theoretic concepts applied to conditional entropy of predictions given user feedback.
result EERM principle balances subjective explainability and risk.
There is a consensus that human and non-human subjects experience temporal distortions in many stages of their perceptual and decision-making systems. Similarly, intertemporal choice research has shown that decision-makers undervalue future outcomes relative to immediate ones. Here we combine techniques from informatio…
Examines parallels between human subjects and texts for causal inference.
problem Ambiguity and fallacies in causal inference using textual data.
method Two strategies: shifting from traits to perceptions and from concepts to parts.
result Highlights the importance of clarifying fundamental concepts.
Was it fair that Harry was hired but not Barry? Was it fair that Pam was fired instead of Sam? How can one ensure fairness when an intelligent algorithm takes these decisions instead of a human? How can one ensure that the decisions were taken based on merit and not on protected attributes like race or sex? These are t…
Recent years have seen a boom in interest in machine learning systems that can provide a human-understandable rationale for their predictions or decisions. However, exactly what kinds of explanation are truly human-interpretable remains poorly understood. This work advances our understanding of what makes explanations …
Reward shaping speeds up human learning through IRL.
problem Slow learning in humans, especially for challenging tasks.
method Extended IRL algorithm with kernel methods, conducted experiments with online game players.
result Players learn desired policies more quickly with reward shaping.
Machine learning uses crowdworkers; determining their status as human subjects is tricky.
problem Determining the appropriate status of ML crowdworkers as human subjects.
method Investigation of natural language processing studies to expose challenges and propose solutions.
result Potential loophole in the U.S. Common Rule for ML research oversight.
Novel approach to learning models based on subjective timescales for better exploration and decision-making.
problem Learning models over multi-step timescales in environments with intermediate states.
method Developed a subjective-timescale model (STM) based on episodic memories, enabling systematic variation of temporal extent of predictions.
result STM produces more informative action-conditioned roll-outs, leading to better decision-making and exploration.
IDT learns human preferences from uncertain decisions, even when humans are suboptimal.
problem Learning human preferences from uncertain and suboptimal decisions.
method Inverse decision theory (IDT) framework, statistical analysis of IDT, characterizing sample complexity.
result Learning preferences is easier when decisions are more uncertain, even if humans are suboptimal.
K-fold Cross Validation is commonly used to evaluate classifiers and tune their hyperparameters. However, it assumes that data points are Independent and Identically Distributed (i.i.d.) so that samples used in the training and test sets can be selected randomly and uniformly. In Human Activity Recognition datasets, we…
Digital personas improve survey results for stable attributes but fail for subjective responses.
problem When can digital personas reliably approximate human survey findings?
method Using LISS panel, constructed personas from background variables and survey histories, tested against held-out post-cutoff answers.
result Digital personas improve alignment with human response distributions for stable attributes but fail for subjective responses.
Accurately predicting the outcome of sporting events has been a goal for many groups who seek to maximize profit. What makes this challenging is that the outcome of an event can be influenced by many factors that dynamically change across time. Oddsmakers attempt to estimate these factors by using both algorithmic and …
Embodied cognition states that semantics is encoded in the brain as firing patterns of neural circuits, which are learned according to the statistical structure of human multimodal experience. However, each human brain is idiosyncratically biased, according to its subjective experience history, making this biological s…
Modeling and Interpreting Real-world Human Risk Decision Making with Inverse Reinforcement Learningcs.LG
We model human decision-making behaviors in a risk-taking task using inverse reinforcement learning (IRL) for the purposes of understanding real human decision making under risk. To the best of our knowledge, this is the first work applying IRL to reveal the implicit reward function in human risk-taking decision making…
Bayesian framework evaluates predictors of subjective visual tasks.
problem Evaluating uncertainty in machine learning predictors for tasks with subjective annotations.
method Bayesian framework to estimate epistemic uncertainty from human labels.
result Framework successfully applied to four image classification tasks.
Framework adds human knowledge to AI decisions to improve outcomes.
problem Conflict between AI recommendations and human insights.
method Develops a framework for integrating human knowledge as a guardrail for AI decisions.
result Human knowledge can improve AI decisions, especially in specific pitfalls.
When machine predictors can achieve higher performance than the human decision-makers they support, improving the performance of human decision-makers is often conflated with improving machine accuracy. Here we propose a framework to directly support human decision-making, in which the role of machines is to reframe pr…
Conformal prediction helps quantify uncertainty but its use by humans is unclear.
problem Uncertainty quantification in predictions for human decision making.
method Decision theoretic framework for evaluating predictive uncertainty.
result Conformal prediction sets and human decision making goals are in tension.
Human decision-making underlies all economic behavior. For the past four decades, human decision-making under uncertainty has continued to be explained by theoretical models based on prospect theory, a framework that was awarded the Nobel Prize in Economic Sciences. However, theoretical models of this kind have develop…
Designs algorithms to assist humans without affecting their decisions.
problem Algorithms often fail to improve human decisions.
method Formalizes algorithm design using potential outcomes framework and monotonicity assumption.
result Derives minimax optimal recommendation algorithms for limited data.
DECE visualizes machine learning decisions with counterfactual explanations.
problem Making machine learning models transparent and explainable.
method Interactive visualization system supporting counterfactual explanations at instance- and subgroup-levels.
result DECE enables users to explore and understand machine learning model decisions.
Optimal allocation of human effort to correct AI assessments in decision-making.
problem How to allocate costly human effort to correct noisy or biased AI-generated assessments.
method Decision-theoretic framework treating AI assessments as signals and human judgments as costly information. Developed estimation procedures under nonparametric and linear models.
result Our approach substantially outperforms LLM-only predictions and achieves performance comparable to full human review while using only 20-30% of the human information.
New research shows machine-assisted decisions can still be unfair even when the algorithm is fair.
problem Ensuring fairness in decisions made with machine-assisted human input.
method Formal model and lab experiment to analyze how machine predictions affect human decisions.
result Excluding information about protected groups from machine predictions can increase disparities.
New framework evaluates model explanations based on decision task improvement.
problem Evaluation of model explanations often misses practical value.
method Decision-theoretic framework quantifying three key values.
result Provides benchmarks and interprets human-AI decision support.
Application of intelligent systems especially in smart homes and health-related topics has been drawing more attention in the last decades. Training Human Activity Recognition (HAR) models -- as a major module -- requires a fair amount of labeled data. Despite training with large datasets, most of the existing models w…
Deep neural networks (DNN) trained in a supervised way suffer from two known problems. First, the minima of the objective function used in learning correspond to data points (also known as rubbish examples or fooling images) that lack semantic similarity with the training data. Second, a clean input can be changed by a…
This work advances collaborative decision making by combining human and AI strengths in uncertainty quantification.
problem Current AI lacks robust decision-making capabilities under uncertainty, especially in high-stakes contexts.
method Introduces Human AI Collaborative Uncertainty Quantification (HACUQ) framework, formalizing AI-human collaboration and developing calibration algorithms.
result Optimal collaborative prediction sets follow a two-threshold structure, and online adaptation algorithms can adapt to evolving human behavior.
Explainable AI improves human decision accuracy but does not enhance it significantly.
problem Improving human decision-making through explainable AI.
method Comparing human decision accuracy with and without AI predictions, including or excluding explanations.
result Providing AI predictions improves human decision accuracy, but explanations do not significantly enhance it.
Proposes a new method to explain model predictions for consumer recourse.
problem Current explanation methods fail to provide meaningful recourse to decision subjects.
method Develops feature responsiveness scores to highlight actionable features.
result Standard practices can undermine decision subjects by highlighting unresponsive features.
New approach to algorithmic fairness for human-AI collaboration considers compliance with human decisions.
problem Current fairness approaches assume perfect human compliance, but real-world compliance is often poor.
method Defines compliance-robustly fair algorithms and proposes an optimization strategy to improve fairness.
result Algorithmic recommendations can improve fairness even if humans do not fully comply with fair algorithms.
Reinforcement Learning AI commonly uses reward/penalty signals that are objective and explicit in an environment -- e.g. game score, completion time, etc. -- in order to learn the optimal strategy for task performance. However, Human-AI interaction for such AI agents should include additional reinforcement that is impl…
We proposed a probabilistic approach to joint modeling of participants' reliability and humans' regularity in crowdsourced affective studies. Reliability measures how likely a subject will respond to a question seriously; and regularity measures how often a human will agree with other seriously-entered responses coming…
A model for human-machine decision-making with private info and opacity.
problem Optimizing decisions in a human-machine system with private info and opacity.
method Formulated as a two-player learning problem, proved lower and upper bounds on optimality.
result Simple coordination strategy is nearly minimax optimal, efficient learning possible under certain assumptions.
Framework uses human judgment to distinguish algorithmically indistinguishable cases.
problem Clarifying human-AI collaboration in prediction and decision tasks.
method Integrates human judgment to distinguish algorithmically indistinguishable cases.
result Improves performance of any feasible algorithmic predictor.
Framework for responsible LLM deployment with human involvement and decentralized technologies.
problem Challenges in deploying LLMs for high-stakes decisions, including data security and accountability.
method Interactive human involvement through multiple iterations, decentralized technologies, and automated auditing.
result Enhanced security and accountability in LLM deployment for financial decisions.
Pairwise ranking aligns subjective clinical evaluations with objective indicators.
problem Aligning subjective clinical evaluations with objective indicators for improved diagnosis.
method Pairwise ranking methods to align subjective evaluations with objective indicators.
result The resulting score improves classification accuracy and provides a nuanced severity assessment.