RLHF fails when humans only partially observe, leading to inflated or overjustified feedback.
problem Failure of reinforcement learning from human feedback in partially observable environments.
method Formal definition of failure cases, modeling human as Boltzmann rational, analyzing information provided by feedback.
result RLHF can deceptively inflate or overjustify feedback when humans have partial observations.
IDT learns human preferences from uncertain decisions, even when humans are suboptimal.
problem Learning human preferences from uncertain and suboptimal decisions.
method Inverse decision theory (IDT) framework, statistical analysis of IDT, characterizing sample complexity.
result Learning preferences is easier when decisions are more uncertain, even if humans are suboptimal.
New framework predicts diverse, contextually plausible 3D human motions.
problem Predicting multiple plausible future 3D poses given observed poses.
method Developed a new variational framework that conditions latent variable on past observation to encourage relevant information.
result Our approach generates motions of higher quality and preserves contextual information.
New method infers human sensorimotor costs from behavior.
problem Inferring human sensorimotor costs from observed behavior.
method Inverse optimal control with signal-dependent noise.
result Recovering costs and benefits in sensorimotor behavior.
A robot assists a human in a bandit task to learn and improve performance.
problem Learning preferences in humans when they are also learning.
method Introduces assistive multi-armed bandit, where a robot helps a human maximize cumulative reward.
result Human performance can be better when effectively communicating observed rewards to the robot, not just by learning optimally.
Human irrationality can improve AI design, study shows.
problem Improving AI by learning from human decision-making biases.
method Developed a novel POMDP model to simulate human decision-making in contextual choice tasks.
result Reinforcement learners can exploit human irrationalities to make better decisions.
Robot learns from human actions to perform complex tasks.
problem Learning complex skills from interaction data with embodiment differences.
method Formulated graphical model, treated action as observed variable, used domain-dependent prior.
result Robotic planning agent can learn tool use from human observations.
Machine learning aids in shark detection at Muizenberg Beach.
problem Improving shark spotting efficiency using automated methods.
method Defined desirable properties, selected mathematical techniques, and partially implemented model.
result Extract useful information from shark images despite geometric transformations.
A novel memory mechanism for reinforcement learning agents that stores past events in human-readable language.
problem Lack of interpretability in reinforcement learning agent's memory mechanisms.
method Uses CLIP to associate visual inputs with language tokens, then feeds these tokens to a pretrained language model.
result Significantly faster convergence on challenging continuous recognition tasks.
Bayesian Neural Networks show similar developmental stages as human learning.
problem Comparing human and machine learning stages.
method Operationalized developmental stages as data-set size, analyzed on three tasks.
result Bayesian Neural Networks exhibit similar developmental patterns to human learning.
This paper proposes a method to use LLMs as auxiliary evaluators in place of human judges.
problem The need for cost-effective and scalable evaluation of AI systems.
method Formulates a two-stage sampling design with LLM evaluations and human ratings, using a doubly robust estimator.
result Proposes a method to determine optimal sample sizes for human and LLM ratings.
An algorithm detects anomalies based on human perception principles.
problem Anomaly detection in data.
method Inspired by Gestalt psychology and Helmholtz principle, the algorithm models anomalies as unexpected elements in random distributions.
result The algorithm efficiently detects anomalies with minimal user intervention and promising results on multivariate data.
Human-AI teaming suffers from calibration issues.
problem Human-AI teaming
method Assume calibrated models and humans
result Existing methods for combination do not preserve human's calibration.
Study causal effects on humans in mixed human-AI systems with unobserved unit types.
problem Estimating causal effects on humans in systems with unobserved unit types and interaction networks.
method Assumed human-AI prior, causal message passing (CMP) framework, subpopulation analysis.
result Consistently recover human-specific causal effects using subpopulations with varying expected human composition and treatment exposure.
Many real-world engineering problems rely on human preferences to guide their design and optimization. We present PrefOpt, an open source package to simplify sequential optimization tasks that incorporate human preference feedback. Our approach extends an existing latent variable model for binary preferences to allow f…
Autonomous systems can substantially enhance a human's efficiency and effectiveness in complex environments. Machines, however, are often unable to observe the preferences of the humans that they serve. Despite the fact that the human's and machine's objectives are aligned, asymmetric information, along with heterogene…
Robot learns to imitate human interactions through deep learning.
problem Teaching robots to coordinate actions with human partners.
method Deep learning framework for motion embedding, prediction, and trajectory generation.
result Importance of predictive and adaptive components for successful imitation.
AI assistants often give convincing but incorrect responses to match user beliefs.
problem Sycophancy in AI assistants that use human feedback.
method Examined five AI assistants across four tasks, analyzed human preference data, and compared model outputs against preference models.
result Sycophancy is a general behavior of AI assistants, driven in part by human preference judgments.
Diffusion models mimic human actions in sequential tasks.
problem Cloning human behavior in dynamic environments is challenging.
method Adapting diffusion models to handle stochastic, multimodal, and correlated actions.
result Diffusion models closely replicate human behavior in robotic and gaming tasks.
Model creates human-like text descriptions for time series data.
problem Creating textual summaries for complex time series data that mimic human behavior.
method Utility estimation model based on Bayesian network to rank patterns in time series data.
result Output is a natural language description of time series that matches human summary.
Machine learning models are vulnerable to adversarial examples: small changes to images can cause computer vision models to make mistakes such as identifying a school bus as an ostrich. However, it is still an open question whether humans are prone to similar mistakes. Here, we address this question by leveraging recen…
NDIGO learns world from noisy observations.
problem Learning from noisy and partial observations.
method NDIGO, a self-supervised discovery model.
result NDIGO outperforms state-of-the-art methods in noisy conditions.
VFDS selects dynamic features for efficient HAR tasks, optimizing performance-cost trade-offs.
problem Optimizing feature selection for varying costs and dynamic contexts in machine learning tasks.
method Bayesian learning framework with variational dynamic selection policy.
result VFDS selects different features under changing contexts, saving sensory costs while maintaining HAR accuracy.
TraLFM models human mobility patterns from traffic trajectories.
problem Understanding human mobility patterns from traffic data.
method Latent factor modeling of sequential, personal, and temporal factors.
result TraLFM significantly outperforms state-of-the-art methods in latent factor analysis and next location prediction.
Study models human investors' sub-rational behavior in financial markets.
problem Lack of a comprehensive model for human sub-rationality in financial markets.
method Flexible reinforcement learning model incorporating five human sub-rational aspects.
result Model accurately reproduces human behavior and reveals insights into market dynamics.
CausalRM models rewards from user feedback, overcoming noise and bias.
problem Aligning language models with user preferences from noisy, biased feedback.
method Causal-theoretic reward modeling framework addressing noise and bias in observational feedback.
result CausalRM learns accurate reward signals from noisy and biased observational feedback.
Deformation estimation of elastic object assuming an internal organ is important for the computer navigation of surgery. The aim of this study is to estimate the deformation of an entire three-dimensional elastic object using displacement information of very few observation points. A learning approach with a neural net…
Paper detects changes in human behavior using smartphones.
problem Detecting changes in high-dimensional, heterogeneous, and periodic data.
method Hierarchical model with latent variables and non-stationary periodic covariance functions.
result The method accurately detects changes in human behavior using smartphones.
Humans outperform DNNs on image degradations, especially when signals weaken.
problem Comparing human and DNN robustness in object recognition under various image manipulations.
method Comparison of robustness between humans and DNNs (ResNet-152, VGG-19, GoogLeNet) on object recognition under 12 types of image degradations.
result Humans consistently outperform DNNs on object recognition, especially under weaker signals.
A framework helps reinforcement learning agents understand and decompose tasks from human demonstrations.
problem Sparse-reward tasks where demonstrations are used as sources of causal knowledge.
method Develops causal models through observation and reasons from this knowledge to decompose tasks.
result A basic implementation of Reasoning from Demonstration (RfD) is effective in sparse-reward tasks.
Deep learning mimics successful traders in financial markets.
problem Tackling profitable trading behavior in financial markets.
method Using deep learning neural networks to replicate adaptive trading behavior.
result Deep learning can outperform human traders and even improve performance.
The problem of human activity recognition is central for understanding and predicting the human behavior, in particular in a prospective of assistive services to humans, such as health monitoring, well being, security, etc. There is therefore a growing need to build accurate models which can take into account the varia…
Deep COACH learns complex tasks from human feedback in Minecraft.
problem Learning complex behaviors from human feedback efficiently.
method Deep reinforcement learning with policy updates based on human critiques.
result Demonstrated effectiveness in Minecraft with reduced sample complexity.
The paper offers a checklist for comparing human and machine visual perception.
problem Comparing human and machine visual perception.
method Designing, conducting, and interpreting experiments to investigate mechanisms.
result Feedback mechanisms may not be necessary for visual reasoning tasks.
Inferring intent from observed behavior has been studied extensively within the frameworks of Bayesian inverse planning and inverse reinforcement learning. These methods infer a goal or reward function that best explains the actions of the observed agent, typically a human demonstrator. Another agent can use this infer…
The study explains how humans make sub-optimal decisions using learned approximations.
problem Human decision-making often deviates from optimal strategies.
method Combining meta-learning models with resource-bounded objectives.
result Empirical evidence shows patterns in human decision-making that align with the model.
TraderTalk uses LLMs to simulate human trading interactions in financial markets.
problem Simulating realistic human trading interactions in financial markets.
method Hybrid ABM with LLM-generated behaviors for detailed conversations.
result Successfully replicates trade-to-order volume ratios in financial markets.
VHGM-MAE generates synthetic humans from healthcare data.
problem Handling high-dimensional, sparse healthcare data with missing values.
method Masked autoencoder (MAE) tailored for healthcare data, addressing heterogeneity, missingness, and high-dimensionality.
result VHGM-MAE outperforms existing methods in missing value imputation and synthetic data generation.
Robots learn conservatively from human corrections, avoiding unintended changes to their objectives.
problem Robots learn from human corrections but may not align with the intended objectives due to misspecified hypothesis spaces.
method Robots reason in real-time about the relevance of human corrections to their hypothesis space, learning more conservatively.
result Robots can avoid unintended learning from human corrections, improving alignment with intended objectives.
Study shows semi-supervised learning improves human activity recognition with minimal user input.
problem Improving human activity recognition models using incremental learning.
method Three approaches: non-supervised, semi-supervised, and supervised learning were compared.
result Semi-supervised learning achieves similar accuracy to supervised learning with minimal user input.
Survey of mobility studies using mobile phone data.
problem Understanding human mobility patterns.
method Data Science techniques applied to mobile phone datasets.
result Applications in urban planning, data traffic prediction, etc.
Method learns from video demonstrations with human feedback.
problem Teaching autonomous agents using video demonstrations and human feedback.
method Constructs a mapping between standard and visual representations using a neural network.
result Effective in teaching a hopper agent to perform a backflip with minimal human feedback.
Interpole learns transparent decision-making policies from data.
problem Understanding human decision-making in opaque environments.
method Interpole combines belief-update and belief-action mapping estimation.
result Interpole provides interpretable models of decision-making behavior.
Improved real-time visualizations of conversation turns using dynamic attention weights.
problem Uniform attention weights in sequential analysis tasks prevent meaningful visualization.
method Developed a method to track changes in turn importance over time.
result More informative real-time visuals confirmed by human reviewers.
Humans take advantage of real world symmetries for various tasks, yet capturing their superb symmetry perception mechanism with a computational model remains elusive. Motivated by a new study demonstrating the extremely high inter-person accuracy of human perceived symmetries in the wild, we have constructed the first …
A two-stream reinforcement learning model improves decision-making across human and neuropsychiatric studies.
problem Improving reinforcement learning models to better simulate human decision-making and neuropsychiatric conditions.
method Proposes a two-stream reinforcement learning model that processes positive and negative rewards and incorporates reward-processing biases.
result The two-stream model outperforms standard Q-learning and SARSA methods on various tasks and datasets.
Unified LP framework for offline reward learning from human demonstrations and feedback.
problem Reward learning from human demonstrations and feedback with robustness and sample efficiency.
method A novel linear programming framework for offline reward learning.
result Unified LP framework achieves better performance compared to MLE.
Algorithm learns actions from past states in complex tasks.
problem Learning policies from human feedback is expensive.
method Combining learned feature encoder with inverse models to simulate past actions.
result Algorithm can infer specific skills from single state.