Study human-machine interaction with private info using offline RL.
problem Confounding bias and distributional mismatch in offline RL for human-guided interaction.
method Developed a novel identification result and OPE method to address confounding bias, and used pessimism to tackle distributional mismatch.
result Policy pair converges to optimal one at satisfactory rate under mild assumptions.
Study on predictable forward processes in trading without frequent evaluations.
problem Trading performance evaluation times not matching trading times.
method Solving a linear functional equation to construct predictable forward processes.
result Predictable forward processes are inherently myopic and optimal strategies do not use future information.
New AI assistant for power grid operators simplifies complex decision-making.
problem Complexity and uncertainty in power grid operations.
method Unified human-machine interface and AI integration.
result Development of a new assistant framework for power grid operators.
Speech emotion recognition is a challenging task and an important step towards more natural human-machine interaction. We show that pre-trained language models can be fine-tuned for text emotion recognition, achieving an accuracy of 69.5% on Task 4A of SemEval 2017, improving upon the previous state of the art by over …
We study the problem of designing AI agents that can robustly cooperate with people in human-machine partnerships. Our work is inspired by real-life scenarios in which an AI agent, e.g., a virtual assistant, has to cooperate with new users after its deployment. We model this problem via a parametric MDP framework where…
Cooperative communication plays a central role in theories of human cognition, language, development, culture, and human-robot interaction. Prior models of cooperative communication are algorithmic in nature and do not shed light on why cooperation may yield effective belief transmission and what limitations may arise …
In human-in-the-loop machine learning, the user provides information beyond that in the training data. Many algorithms and user interfaces have been designed to optimize and facilitate this human--machine interaction; however, fewer studies have addressed the potential defects the designs can cause. Effective interacti…
Affective computing has become a very important research area in human-machine interaction. However, affects are subjective, subtle, and uncertain. So, it is very difficult to obtain a large number of labeled training samples, compared with the number of possible features we could extract. Thus, dimensionality reductio…
MCAL reduces labeling costs by 6x for auto-labeling data sets.
problem Expensive human annotation for ground-truth data sets.
method Iterative approach that trains a classifier to auto-label part of the data set, determining which samples to label using humans and which to label using the classifier at each step.
result 6x lower overall cost compared to human labeling the entire data set, always cheaper than competing strategies.
Interfacing a kinetic action of a person to an action of a machine system is an important research topic in many application areas. One of the key factors for intimate human-machine interaction is the ability of the control algorithm to detect and classify different user commands with shortest possible latency, thus ma…
PHASE dataset simulates complex social interactions in physical environments.
problem Lack of datasets for evaluating physically grounded perception of complex social interactions.
method Created PHASE dataset of 2D animations with procedural generation and physics engine.
result SIMPLE model outperforms neural networks in recognizing complex social interactions.
The emerging paradigm of Human-Machine Inference Networks (HuMaINs) combines complementary cognitive strengths of humans and machines in an intelligent manner to tackle various inference tasks and achieves higher performance than either humans or machines by themselves. While inference performance optimization techniqu…
A model for human-machine decision-making with private info and opacity.
problem Optimizing decisions in a human-machine system with private info and opacity.
method Formulated as a two-player learning problem, proved lower and upper bounds on optimality.
result Simple coordination strategy is nearly minimax optimal, efficient learning possible under certain assumptions.
Active learning aims to reduce labeling efforts by selectively asking humans to annotate the most important data points from an unlabeled pool and is an example of human-machine interaction. Though active learning has been extensively researched for classification and ranking problems, it is relatively understudied for…
Framework ensures alignment between humans and machines in LLMs.
problem Human-machine misalignment in LLMs scoring mechanisms.
method Lightweight calibration framework for blackbox models.
result Provably guarantees alignment between humans and machines.
Artificial intelligence (AI) is intrinsically data-driven. It calls for the application of statistical concepts through human-machine collaboration during generation of data, development of algorithms, and evaluation of results. This paper discusses how such human-machine collaboration can be approached through the sta…
Paper tackles dynamic behavior of variable topology mechanisms, presenting new transition conditions.
problem Dynamic behavior of mechanisms with changing kinematic topology.
method Presented new transition conditions for variable topology mechanisms using projected motion equations and Voronets equations.
result Results show the dynamic behavior of joint locking in 3R and 6DOF mechanisms.
Autonomous systems can substantially enhance a human's efficiency and effectiveness in complex environments. Machines, however, are often unable to observe the preferences of the humans that they serve. Despite the fact that the human's and machine's objectives are aligned, asymmetric information, along with heterogene…
When might human input help (or not) when assessing risk in fairness domains? Dressel and Farid (2018) asked Mechanical Turk workers to evaluate a subset of defendants in the ProPublica COMPAS data for risk of recidivism, and concluded that COMPAS predictions were no more accurate or fair than predictions made by human…
Human-machine interaction is increasingly dependent on speech communication. Machine Learning models are usually applied to interpret human speech commands. However, these models can be fooled by adversarial examples, which are inputs intentionally perturbed to produce a wrong prediction without being noticed. While mu…
Emotion recognition from speech is one of the key steps towards emotional intelligence in advanced human-machine interaction. Identifying emotions in human speech requires learning features that are robust and discriminative across diverse domains that differ in terms of language, spontaneity of speech, recording condi…
As machine learning systems move from computer-science laboratories into the open world, their accountability becomes a high priority problem. Accountability requires deep understanding of system behavior and its failures. Current evaluation methods such as single-score error metrics and confusion matrices provide aggr…
We present a systematic analysis on the performance of a phonetic recogniser when the window of input features is not symmetric with respect to the current frame. The recogniser is based on Context Dependent Deep Neural Networks (CD-DNNs) and Hidden Markov Models (HMMs). The objective is to reduce the latency of the sy…
Object tracking is an ubiquitous problem that appears in many applications such as remote sensing, audio processing, computer vision, human-machine interfaces, human-robot interaction, etc. Although thoroughly investigated in computer vision, tracking a time-varying number of persons remains a challenging open problem.…
Society's drive toward ever faster socio-technical systems, means that there is an urgent need to understand the threat from 'black swan' extreme events that might emerge. On 6 May 2010, it took just five minutes for a spontaneous mix of human and machine interactions in the global trading cyberspace to generate an unp…
Cooperation is often implicitly assumed when learning from other agents. Cooperation implies that the agent selecting the data, and the agent learning from the data, have the same goal, that the learner infer the intended hypothesis. Recent models in human and machine learning have demonstrated the possibility of coope…
An assistant learns to mediate decisions between humans and experts, balancing risk and learning.
problem Learning to mediate decisions between imperfect humans and expert knowledge.
method Formalizes online decision mediation, proposes a policy to balance immediate loss and future generalization.
result Consistent gains over benchmarks in decision-making performance.
The importance of automated Facial Emotion Recognition (FER) grows the more common human-machine interactions become, which will only continue to increase dramatically with time. A common method to describe human sentiment or feeling is the categorical model the `7 basic emotions', consisting of `Angry', `Disgust', `Fe…
Automatic understanding of human affect using visual signals is of great importance in everyday human-machine interactions. Appraising human emotional states, behaviors and reactions displayed in real-world settings, can be accomplished using latent continuous dimensions (e.g., the circumplex model of affect). Valence …
Investigation of human brain states through electroencephalograph (EEG) signals is a crucial step in human-machine communications. However, classifying and analyzing EEG signals are challenging due to their noisy, nonlinear and nonstationary nature. Current methodologies for analyzing these signals often fall short bec…
Compared to humans, machine learning models generally require significantly more training examples and fail to extrapolate from experience to solve previously unseen challenges. To help close this performance gap, we augment single-task neural networks with a meta-recognition model which learns a succinct model code vi…
Dual-stage sEMG classification improves gesture recognition accuracy.
problem Improving accuracy in hand gesture recognition from sEMG signals.
method Dual-stage classification approach: first stage groups similar activities, second stage classifies within groups.
result Dual-stage classification yields significantly higher accuracy than single-stage approach.
This paper evaluates deferring systems using causal inference.
problem Evaluating the impact of deferring systems on model accuracy is challenging.
method The paper uses a causal inference framework to evaluate deferring systems, distinguishing between scenarios with and without access to human predictions.
result The approach allows identifying causal effects of deferring strategies on predictive accuracy.
Combines human and model predictions for improved accuracy.
problem Improving classification accuracy when both human and model predictions are imperfect.
method Uses confusion matrices and calibration to combine probabilistic model outputs with human class-level predictions.
result Human-model combinations consistently outperform either alone, with accuracy gains even with limited human input.
Deep neural networks improve sEMG-based hand gesture classification.
problem Accurate classification of hand gestures from sEMG signals.
method Master-slave architecture with DNNs and synthetic feature data.
result Up to 9% improvement in accuracy with synthetic data.
Due to concerns about human error in crowdsourcing, it is standard practice to collect labels for the same data point from multiple internet workers. We here show that the resulting budget can be used more effectively with a flexible worker assignment strategy that asks fewer workers to analyze easy-to-label data and m…
Bayesian CNN estimates uncertainty in COVID-19 detection.
problem Uncertainty in deep learning predictions for medical diagnosis.
method Drop-weight based Bayesian Convolutional Neural Networks (BCNN).
result Uncertainty correlates with prediction accuracy.
Despite showing state-of-the-art performance, deep learning for speech recognition remains challenging to deploy in on-device edge scenarios such as mobile and other consumer devices. Recently, there have been greater efforts in the design of small, low-footprint deep neural networks (DNNs) that are more appropriate fo…
CA connects visual concepts to neural network representations.
problem Global association of human interpretable concepts with neural network internals.
method Established a taxonomy for CA methods and reviewed state-of-the-art research.
result Unified understanding of CA methods and datasets.
Surrogate-based analysis of interactions via local effect smooths
problem Detecting and characterizing feature interactions in machine learning models
method Surrogate-based analysis using generalized additive models
result Empirical validation of effectiveness for pairwise interactions
Factorization Machine (FM) is a widely used supervised learning approach by effectively modeling of feature interactions. Despite the successful application of FM and its many deep learning variants, treating every feature interaction fairly may degrade the performance. For example, the interactions of a useless featur…
ShapleyBO explains BO's decisions, enhancing human-AI collaboration in robotics.
problem Lack of explainability in BO's black box nature, especially in human-in-the-loop applications.
method Uses game-theoretic Shapley values to quantify and interpret BO's parameter contributions.
result ShapleyBO identifies exploration and exploitation contributions, disentangling aleatoric and epistemic uncertainty.
A new method detects interactions in neural networks using topological analysis.
problem Detecting interactions between input features in neural networks.
method Topological analysis of neural network connectivity to quantify interaction strength.
result The PID algorithm outperforms state-of-the-art baselines in interaction detection tasks.
The paper offers a checklist for comparing human and machine visual perception.
problem Comparing human and machine visual perception.
method Designing, conducting, and interpreting experiments to investigate mechanisms.
result Feedback mechanisms may not be necessary for visual reasoning tasks.
iKF method uncovers complex variable interactions for scientific discovery.
problem Limited interpretability of existing models in decision-making applications.
method Iterative Kings' Forests (iKF) method to uncover multi-order interactions.
result iKF provides strong interpretive power for explainable modeling.
Decoding language representations directly from the brain can enable new Brain-Computer Interfaces (BCI) for high bandwidth human-human and human-machine communication. Clinically, such technologies can restore communication in people with neurological conditions affecting their ability to speak. In this study, we prop…
A graph neural network detects beneficial feature interactions for recommender systems.
problem Feature interactions are crucial but not all are beneficial for recommendation accuracy.
method Graph neural network with L0 activation regularization for edge prediction.
result The model outperforms baselines and automatically identifies beneficial feature interactions.
We describe and extract time-ordered multibody interactions from complex systems.
problem Complex systems with temporal and multibody dependencies.
method Decompose multivariate Markov chains into time-ordered multibody interactions. Algorithm to extract interactions from data. Measure complexity of interaction ensembles.
result Robust and efficient algorithm to infer time-ordered multibody interactions from data.