SADCBO optimizes contextual variables by balancing relevance and cost.
problem Optimizing contextual variables with varying costs and unknown relevance.
method Adaptive selection of relevant contextual variables using sensitivity analysis and early stopping.
result Consistent improvement in optimization across various examples.
Elliptical Attention improves transformer performance by focusing on contextually relevant features.
problem Transformer models suffer from representation collapse and are vulnerable to contaminated samples.
method Uses Mahalanobis distance to define hyper-ellipsoidal neighborhoods for attention weights.
result Elliptical Attention reduces representation collapse and enhances model robustness.
DMNL bandits optimize assortment choices balancing relevance and diversity.
problem Balancing relevance-driven choice with within-assortment diversity.
method Augments MNL choice probabilities with a submodular diversity function, proposing a white-box UCB-based algorithm.
result Achieves at least a (1−e+11)-approximate regret bound of $ ilde{O}\left(d \sqrt{T/K}
ight)$. In many cases, feature selection is often more complicated than identifying a single subset of input variables that would together explain the output. There may be interactions that depend on contextual information, i.e., variables that reveal to be relevant only in some specific circumstances. In this setting, the con…
Novel algorithm reduces feature inclusion in online decision-making.
problem Optimizing decision-making for personalized user experiences with fairness.
method Online Batched Sequential Inclusion (OBSI) algorithm for sequential feature inclusion.
result OBSI outperforms other algorithms in terms of regret, relevance of features, and compute.
New algorithm estimates treatment effects for more efficient contextual bandits.
problem Contextual bandits struggle with action-independent reward redundancies.
method Reduces contextual bandits to heterogeneous treatment effect estimation.
result Heterogeneous treatment effect estimation leads to more efficient model estimation.
Recommender systems, medical diagnosis, network security, etc., require on-going learning and decision-making in real time. These -- and many others -- represent perfect examples of the opportunities and difficulties presented by Big Data: the available information often arrives from a variety of sources and has divers…
Uncertainty in economics still poses some fundamental problems illustrated, e.g., by the Allais and Ellsberg paradoxes. To overcome these difficulties, economists have introduced an interesting distinction between 'risk' and 'ambiguity' depending on the existence of a (classical Kolmogorovian) probabilistic structure m…
Contextual bandits are online learners that, given an input, select an arm and receive a reward for that arm. They use the reward as a learning signal and aim to maximize the total reward over the inputs. Contextual bandits are commonly used to solve recommendation or ranking problems. This paper considers a learning s…
The paper tackles carousel personalization in music streaming apps using contextual bandits.
problem Selecting relevant items to display in carousels for personalized content recommendation.
method Modeling carousel personalization as a contextual multi-armed bandit problem with multiple plays, cascade-based updates and delayed batch feedback.
result Empirically shows the effectiveness of the framework in capturing characteristics of real-world carousels.
Study on how attention in prompt-tuning affects large language models.
problem Limited theoretical understanding of prompt-tuning and attention in LLMs.
method Exploration of prompt-tuning for one-layer attention architectures, contextual mixture-models, and self-contained prompt-attention model.
result Softmax-prompt-attention is more expressive than self-attention and linear-prompt-attention under contextual data model.
Contextual policy search (CPS) is a class of multi-task reinforcement learning algorithms that is particularly useful for robotic applications. A recent state-of-the-art method is Contextual Covariance Matrix Adaptation Evolution Strategies (C-CMA-ES). It is based on the standard black-box optimization algorithm CMA-ES…
REN addresses uncertainty in user feedbacks for better recommendation systems.
problem Recurrent neural networks focus solely on item relevance, neglecting diverse item exploration.
method Proposes REN, a new type of recurrent neural network that balances relevance and exploration while accounting for representation uncertainty.
result REN achieves satisfactory long-term rewards on synthetic and real-world recommendation datasets, outperforming state-of-the-art models.
This paper studies systematic exploration for reinforcement learning with rich observations and function approximation. We introduce a new model called contextual decision processes, that unifies and generalizes most prior settings. Our first contribution is a complexity measure, the Bellman rank, that we show enables …
BerlinUCB learns from episodic rewards in nonstationary contexts.
problem Online learning with episodic rewards in nonstationary environments.
method BerlinUCB integrates clustering for self-supervision.
result BerlinUCB outperforms standard contextual bandits in various scenarios.
Study off-policy evaluation and learning in dynamic pricing with context.
problem Dynamic personalized pricing and operations management problems with high-dimensional user types.
method Formalize causal structure, leverage single time-step evaluation, estimate marginal MDP.
result Improved out-of-sample policy performance in dynamic and capacitated pricing.
The paper tackles policy learning in dynamic environments using causal methods.
problem Existing reinforcement learning algorithms assume static mechanisms, but real-world systems often have changing mechanisms.
method The paper introduces multi-environment contextual bandits and policy invariance to handle environmental shifts.
result An optimal invariant policy is guaranteed to generalize across environments under suitable assumptions.
We consider the stochastic contextual bandit problem with additional regularization. The motivation comes from problems where the policy of the agent must be close to some baseline policy which is known to perform well on the task. To tackle this problem we use a nonparametric model and propose an algorithm splitting t…
LLM-Lasso uses LLMs to improve feature selection in Lasso regression.
problem Improving feature selection in Lasso regression with domain-specific knowledge.
method Combines LLMs with Lasso regularization to generate feature weights.
result Outperforms standard Lasso and feature selection baselines in biomedical studies.
Direct contextual policy search methods learn to improve policy parameters and simultaneously generalize these parameters to different context or task variables. However, learning from high-dimensional context variables, such as camera images, is still a prominent problem in many real-world tasks. A naive application o…
Contextualizing financial news improves stock price predictions.
problem Predicting stock prices from financial news requires understanding historical context.
method Proposed a method using a large language model for main articles and a small model for historical context.
result Historical context significantly improves model performance across methods and time horizons.
A new approach learns to represent context for nonstationary bandits.
problem Nonstationary contextual bandits where patterns change over time.
method Combines recurrent neural networks with contextual linear bandit algorithm.
result Consistently outperforms handcrafted historical contexts and other methods.
CB-RL solves complex decision-making problems with contextual information and exogenous events.
problem Optimal policy in strategic decision-making problems that depend on environmental configuration and exogenous events.
method Contextual Bilevel Reinforcement Learning (CB-RL) with a stochastic Hyper Policy Gradient Descent (HPGD) algorithm.
result Demonstrated convergence and performance of the HPGD algorithm for reward shaping and tax design.
SLiCE learns contextual node embeddings for link prediction in heterogeneous networks.
problem Link prediction requires specific contextual information not captured by static node embeddings.
method Self-supervised pre-training with localized attention mechanisms.
result SLiCE significantly outperforms existing methods on link prediction tasks.
Study wSAA for contextual decisions, improving uncertainty quantification under computational constraints.
problem Uncertainty quantification limitations in wSAA for contextual stochastic optimization.
method Establish central limit theorems and asymptotic-normality-based confidence intervals for optimal costs.
result Over-optimizing can mitigate misspecification and preserve asymptotic normality, albeit at a slower convergence rate.
Many sequential decision-making tasks require choosing at each decision step the right action out of the vast set of possibilities by extracting actionable intelligence from high-dimensional data streams. Most of the times, the high-dimensionality of actions and data makes learning of the optimal actions by traditional…
A contextual care protocol is used by a medical practitioner for patient healthcare, given the context or situation that the specified patient is in. This paper proposes a method to build an automated self-adapting protocol which can help make relevant, early decisions for effective healthcare delivery. The hybrid mode…
This paper improves context-aware recommender systems by selecting and incorporating relevant low-dimensional contextual information.
problem Generating accurate recommendations is not enough; contextual information can cause issues like battery drain and privacy.
method Developed a feature-selection algorithm based on genetic algorithms to reduce context dimensions while maintaining explainability.
result The approach improves accuracy and transparency in recommendations, outperforming state-of-the-art models.
New algorithm poLinUCB improves online learning in content recommendation platforms.
problem Improving efficiency in content recommendation platforms with post-serving context.
method Novel contextual bandit problem with post-serving contexts and a new algorithm, poLinUCB.
result Achieves tight regret under standard assumptions and significant benefit of utilizing post-serving contexts.
We propose the Generalized Policy Elimination (GPE) algorithm, an oracle-efficient contextual bandit (CB) algorithm inspired by the Policy Elimination algorithm of \cite{dudik2011}. We prove the first regret optimality guarantee theorem for an oracle-efficient CB algorithm competing against a nonparametric class with i…
Relevance ranking and result diversification are two core areas in modern recommender systems. Relevance ranking aims at building a ranked list sorted in decreasing order of item relevance, while result diversification focuses on generating a ranked list of items that covers a broad range of topics. In this paper, we s…
We introduce a new deep learning architecture for predicting price movements from limit order books. This architecture uses a causal convolutional network for feature extraction in combination with masked self-attention to update features based on relevant contextual information. This architecture is shown to significa…
Study improves vehicle motion prediction by incorporating traffic density.
problem Lack of contextual knowledge in predicting driving behavior.
method Categorize and define external conditions, evaluate motion prediction approach.
result Traffic density significantly improves motion prediction accuracy.
Presentation bias is one of the key challenges when learning from implicit feedback in search engines, as it confounds the relevance signal with uninformative signals due to position in the ranking, saliency, and other presentation factors. While it was recently shown how counterfactual learning-to-rank (LTR) approache…
New algorithms tackle latent bandit problems with lower regret.
problem Learning optimal actions in a latent state setting.
method General algorithms based on UCBs and Thompson sampling, aware of model uncertainty.
result Our methods have lower regret than classic bandit policies when latent states are fewer than actions.
New method tracks significant shifts in nonparametric bandits.
problem Tracking significant changes in nonparametric contextual bandits.
method Proposed a notion of 'experienced significant shifts' to adapt to minimax rate without knowledge of change parameters.
result Experienced significant shifts count fewer changes than traditional metrics, leading to an adaptive algorithm.
We tackle the task of diverse 3D human motion prediction, that is, forecasting multiple plausible future 3D poses given a sequence of observed 3D poses. In this context, a popular approach consists of using a Conditional Variational Autoencoder (CVAE). However, existing approaches that do so either fail to capture the …
New loss functions optimize pricing policies using transaction data, ensuring expected revenue guarantees.
problem Optimizing pricing policies with transaction data where valuation data is not directly observed.
method Introducing convex loss functions for contextual pricing, focusing on log-concave valuation distributions.
result Proved expected revenue bounds for generalized hinge and quantile pricing loss functions.
Recommender systems today have become an essential component of any commercial website. Collaborative filtering approaches, and Matrix Factorization (MF) techniques in particular, are widely used in recommender systems. However, the natural data sparsity problem limits their performance where users generally interact w…
Context-awareness in smart mobile applications is a growing area of study, because of it's intelligence in the applications. In order to build context-aware intelligent applications, mining contextual behavioral rules of individual smartphone users utilizing their phone log data is the key. However, to mine these rules…
A new algorithm for selecting top-k arms in extreme contextual bandits with improved efficiency.
problem Selecting top-k arms from a large set with contextual information and limited rewards.
method Proposes an algorithm for both non-extreme and extreme settings, using Inverse Gap Weighting and arm hierarchy models.
result Achieves improved regret guarantees for extreme settings with significant computational and statistical efficiency.
Modern approaches to stock pricing in quantitative finance are typically founded on the 'Black-Scholes model' and the underlying 'random walk hypothesis'. Empirical data indicate that this hypothesis works well in stable situations but, in abrupt transitions such as during an economical crisis, the random walk model fa…
IITK wins FinSim 2020 task on financial hypernym detection.
problem Classifying financial terms into hypernym concepts in an external ontology.
method Used context-dependent and context-independent word embeddings (Word2vec and BERT) for classification.
result Ranked 1st based on mean rank and accuracy metrics.
DDPMs can reproduce medical image context, showing interpolation between samples.
problem Understanding DDPMs' ability to learn spatial context in medical imaging.
method Used stochastic context models (SCMs) to produce training data and assess DDPMs' performance.
result DDPMs can generate contextually correct images, interpolating between samples.
Efficient algorithm for mobile health provides timely physical activity suggestions.
problem Inefficient reinforcement learning for mobile health settings.
method Contextual bandit algorithm with linear mixed effects model and hyper-parameter updating.
result Improves speed and accuracy by up to 99% and 56%.
Contextualized ML learns context-dependent effects using deep learning.
problem Learning heterogeneous and context-dependent effects in data.
method Applying deep learning to the meta-relationship between contextual information and context-specific parametric models.
result Unified framework for cluster analysis and cohort modeling.
IDS improves reinforcement learning with contextual information.
problem Optimizing IDS for contextual reinforcement learning.
method Investigated contextual bandit problems and proposed a computationally-efficient IDS.
result Contextual IDS outperforms conditional IDS by considering future contexts.
Models extract relevant EHR snippets to aid radiologists in diagnosis.
problem Difficulty in identifying relevant patient record information for diagnosis.
method Distantly supervised transformer-based neural model for extractive summarization.
result Models yield better extractive summaries than unsupervised approaches.