RNNs improve predictive precompute for faster user interface response times.
problem Improving user interface response times in mobile and web applications.
method Application of recurrent neural networks (RNNs) for predictive precompute.
result RNN models improve prediction accuracy and reduce computational cost.
Neural pedagogical agent updates user models in real-time for mobile education apps.
problem Real-time user modeling for dynamic mobile education platforms.
method Bidirectional recurrent neural networks with attention mechanism over embedded question-response pairs.
result Model outperforms existing approaches in predicting user response correctness.
Automated dialogue quality evaluation using user satisfaction estimates across multiple domains.
problem Lack of automated and domain-independent dialogue quality evaluation metrics.
method Created a new Response Quality annotation scheme, introduced five domain-independent feature sets, and experimented with six machine learning models.
result Gradient Boosting Regression model achieved best prediction performance, with a 16% relative improvement in binary satisfaction class prediction accuracy.
User response prediction is a crucial component for personalized information retrieval and filtering scenarios, such as recommender system and web search. The data in user response prediction is mostly in a multi-field categorical format and transformed into sparse representations via one-hot encoding. Due to the spars…
Machine learning detects survey validity from user behavior.
problem Detecting valid responses in web surveys.
method Uses mouse activity and machine learning models (LSTM, HMM).
result Predicts survey validity without analyzing specific answers.
New methods protect privacy while providing accurate prediction sets.
problem Privacy-preserving conformal prediction for untrusted aggregators.
method Two LDP approaches: k-ary randomized response and binary search response.
result Finite-sample coverage guarantees and robust coverage under randomization.
Bayesian optimization agent learns user preferences from pairwise comparisons.
problem Learning user preferences from unknown and infinite choices.
method Sequential Bayesian optimization with pairwise comparisons.
result Optimal agent strategy minimizes remaining system uncertainty.
New metric estimates user satisfaction for dialogue quality evaluation.
problem Sparse and intrusive explicit user feedback for dialogue quality evaluation.
method Created a new Response Quality annotation scheme and developed a turn-level User Satisfaction metric using five domain-independent feature sets and six machine learning models.
result Gradient boosting regression achieved best correlation of ~0.79 between predicted and annotated user satisfaction labels.
Study optimizes classifiers for credit card mail campaigns and default prediction.
problem Optimizing classifiers for credit card mail campaigns and default prediction.
method Three distinct models: response, risk, and response-risk. Optimized various performance metrics.
result Random Forest classifier achieves highest accuracy (83.2%) in multi-class response-risk model.
Adaptive querying learns user psychometrics with AI personas.
problem Learning user psychometrics within query budgets.
method Persona-induced latent variable model with AI personas and large language model response distributions.
result Persona-based posteriors deliver accurate probabilistic predictions.
Model learns metrics and preferences from user comparisons.
problem Simultaneous metric and preference learning from user comparisons.
method Jointly learns a metric and latent ideal points for each user.
result Model captures individual preferences and learns metrics efficiently.
User response prediction makes a crucial contribution to the rapid development of online advertising system and recommendation system. The importance of learning feature interactions has been emphasized by many works. Many deep models are proposed to automatically learn high-order feature interactions. Since most featu…
Previous work on recommender systems mainly focus on fitting the ratings provided by users. However, the response patterns, i.e., some items are rated while others not, are generally ignored. We argue that failing to observe such response patterns can lead to biased parameter estimation and sub-optimal model performanc…
This paper compares two loss functions for learning from aggregated responses and introduces an interpolating estimator.
problem Learning from aggregated responses in privacy-sensitive settings.
method Investigates bag-level and instance-level loss functions, and introduces an interpolating estimator.
result Instance-level loss can be seen as a regularized form of bag-level loss, leading to improved estimators.
E-scores assess LLM outputs for correctness, addressing p-hacking issues.
problem Limited principled mechanisms to assess generative model correctness.
method Use e-values to complement LLM outputs with e-scores, providing flexibility in tolerance levels.
result Achieves guarantees of correctness assessment and upper bounds size distortion.
Using machine learning in high-stakes applications often requires predictions to be accompanied by explanations comprehensible to the domain user, who has ultimate responsibility for decisions and outcomes. Recently, a new framework for providing explanations, called TED, has been proposed to provide meaningful explana…
A new dataset tracks user interactions and click responses in online marketplaces.
problem Lack of exposure data in recommender systems datasets.
method Proposes a novel dataset including slates and click responses, allowing more accurate likelihood models.
result Models using exposure data show more natural likelihood, reducing bias towards previously exposed items.
RecSim creates customizable simulation environments for RSs.
problem Creating realistic simulation environments for RSs.
method Configurable platform for authoring simulation environments.
result Enables pushing the limits of RL and RS techniques.
The majority of conversations a dialogue agent sees over its lifetime occur after it has already been trained and deployed, leaving a vast store of potential training signal untapped. In this work, we propose the self-feeding chatbot, a dialogue agent with the ability to extract new training examples from the conversat…
The paper develops methods to predict the probability of achieving a user goal in a task, ensuring the system alerts when the probability falls below a threshold.
problem Ensuring an autonomous system achieves the user's goal with calibrated probability estimates.
method Invertible conformal prediction using Probability-space Conformalized Quantile Regression (PCQR) to produce well-calibrated conditional prediction intervals.
result The method produces well-calibrated probabilities that the cumulative reward will fall within a user-specified target interval, with finite-sample guarantees.
Large-scale industrial recommender systems are usually confronted with computational problems due to the enormous corpus size. To retrieve and recommend the most relevant items to users under response time limits, resorting to an efficient index structure is an effective and practical solution. The previous work Tree-b…
Recent studies have shown that information disclosed on social network sites (such as Facebook) can be used to predict personal characteristics with surprisingly high accuracy. In this paper we examine a method to give online users transparency into why certain inferences are made about them by statistical models, and …
AI assistants often give convincing but incorrect responses to match user beliefs.
problem Sycophancy in AI assistants that use human feedback.
method Examined five AI assistants across four tasks, analyzed human preference data, and compared model outputs against preference models.
result Sycophancy is a general behavior of AI assistants, driven in part by human preference judgments.
Look-ahead reasoning helps predict strategic user behavior on learning platforms.
problem Optimization criteria on learning platforms do not reflect users' priorities.
method Formalized level-k thinking and contrasted collective and selfish behavior.
result Coordination benefits users but does not offer higher-level reasoning advantages in the long run.
Novel Bayesian model improves EEG-based BCI character selection.
problem Accurately identifying target-related responses in EEG-based BCIs.
method Probit-link Split-and-merge Gaussian Process (P-SMGP) prior for feature selection.
result Reduces computational complexity and provides interpretable statistical interpretations.
The measurement and analysis of Electrodermal Activity (EDA) offers applications in diverse areas ranging from market research, to seizure detection, to human stress analysis. Unfortunately, the analysis of EDA signals is made difficult by the superposition of numerous components which can obscure the signal informatio…
Posterior conformal prediction improves prediction interval validity for subgroups.
problem Marginal and conditional prediction interval validity for subgroups.
method Modeling conditional nonconformity score distribution as a mixture of cluster distributions.
result PCP produces tighter prediction intervals, especially for well-represented clusters.
Solves TOD systems' query annotation problem without explicit annotations.
problem Training TOD systems without explicit KB query annotation.
method Reinforcement learning (RL) and pipelined approach for query prediction and system training.
result Improved RL agent with modifications for TOD tasks.
IntelligentPooling improves treatment decisions in mHealth.
problem Optimizing treatment decisions in mobile health with limited data and non-stationary responses.
method Generalized Thompson-Sampling bandit algorithms to IntelligentPooling, addressing differential response, limited data, and non-stationary responses.
result IntelligentPooling achieves 26% lower regret compared to state-of-the-art methods.
BART is extended to handle various response variables.
problem Modeling nonlinear regression functions for diverse response types.
method Generalized Bayesian Additive Regression Trees (GBART) for exponential family distributions.
result The posterior concentrates at a minimax rate for certain response distributions.
Recent years have seen a boom in interest in machine learning systems that can provide a human-understandable rationale for their predictions or decisions. However, exactly what kinds of explanation are truly human-interpretable remains poorly understood. This work advances our understanding of what makes explanations …
Active learning suffers from biased non-response, which this paper addresses.
problem Active learning's effectiveness is compromised by biased non-response in real-world contexts.
method Proposes a cost-based correction to the sampling strategy, UCB-EU, to mitigate the impact of biased non-response.
result UCB-EU successfully reduces the harm from labelling non-response in many settings.
Modern machine learning methods are critical to the development of large-scale personalized learning systems that cater directly to the needs of individual learners. The recently developed SPARse Factor Analysis (SPARFA) framework provides a new statistical model and algorithms for machine learning-based learning analy…
A new Input-Output model, called the Multi-Entity Input-Output (MEIO) model, is introduced to estimate the responsibility of entities of an ecosystem on the footprint of each other. It assumed that the ecosystem is comprised of end users, service providers, and utilities. The proposed MEIO modeling approach can be seen…
Response time improves alignment with diverse human preferences.
problem Standard aggregation of feedback ignores heterogeneity and anonymity.
method Augmenting feedback with response time data and modeling decisions with DDM.
result Estimator of heterogeneous preferences converges to true average preference.
CAFL breaks feedback loops in recommender systems using causal inference.
problem Feedback loops in recommender systems compromise recommendation quality and homogenize user behavior.
method Causal Adjustment for Feedback Loops (CAFL) algorithm that breaks feedback loops using causal inference.
result CAFL improves recommendation quality compared to prior correction methods.
More and more AI services are provided through APIs on cloud where predictive models are hidden behind APIs. To build trust with users and reduce potential application risk, it is important to interpret how such predictive models hidden behind APIs make their decisions. The biggest challenge of interpreting such predic…
A new algorithm estimates item parameters in item response theory models.
problem Estimating item parameters in item response theory models.
method Computation of the stationary distribution of a Markov chain defined on an item-item graph.
result Our algorithm is consistent and enjoys favorable optimality properties.
E2Tree explains random forest models in regression tasks.
problem Lack of transparency in random forest models.
method E2Tree extends random forest to regression by explaining model predictions through graphical representation and dissimilarity measures.
result E2Tree provides a transparent explanation of random forest models in regression tasks.
Common statistical prediction models often require and assume stationarity in the data. However, in many practical applications, changes in the relationship of the response and predictor variables are regularly observed over time, resulting in the deterioration of the predictive performance of these models. This paper …
Neural Assistant integrates knowledge reasoning and dialogue generation in a single model.
problem Challenges in task-oriented dialog systems, including multi-turn language understanding and generation, knowledge retrieval and reasoning, and action prediction.
method Develops a single neural network model that jointly predicts text responses and actions from conversation history and external knowledge.
result The model learns to reason on external knowledge with weak supervision, improving factual accuracy and language generation performance.
FPP creates interpretable 2D embeddings for high-dimensional data.
problem Discovering interpretable relationships in high-dimensional data.
method Function preserving projections (FPP) for scalable linear embeddings.
result FPP reveals non-linear patterns of user-selected response functions.
iSplit LBI predicts individualized partial rankings from ties, outperforming state-of-the-art methods.
problem Predicting partial rankings from pairwise comparisons with ties, considering individual preferences.
method Variable splitting-based algorithm (iSplit LBI) that generates a sequence of estimations with a regularization path, decomposing parameters into abnormal signals, personalized signals, and random noise.
result iSplit LBI significantly outperforms state-of-the-art alternatives in predicting individualized partial rankings.
Debias recommender systems by accounting for hidden confounders using network information.
problem Debiased recommender systems to reduce bias caused by hidden confounders.
method Leverage network information to disentangle user conformity and item popularity, modeling exposure and ratings while controlling hidden confounders.
result The proposed method effectively debiases recommender systems, improving recommendation accuracy.
Suppose that we wish to estimate a user's preference vector w from paired comparisons of the form "does user w prefer item p or item q?," where both the user and items are embedded in a low-dimensional Euclidean space with distances that reflect user and item similarities. Such observations arise in numerous se…
SCR diversifies recommendations by applying style transfer to user profiles.
problem Diversify personalized recommendations without losing relevance.
method Style injection using Conditional Variational Autoencoder (CVAE) architecture.
result 12% improvement in NDCG@20 and 22% improvement in AUC across all classes.
survex explains machine learning survival models, improving model transparency.
problem Lack of tools to explain machine learning survival models.
method Introduces survex R package using explainable AI techniques.
result Improves model reliability and detects biases in survival models.
This study improves user segmentation for online news recommendation systems.
problem Challenges in building modern recommender systems due to dynamic environments and data sparsity.
method Trend-responsive unsupervised user segmentation using multi-armed bandits.
result Significant improvements in online A/B tests compared to global-optimization algorithms.