Research
On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,695 papers · 148 categories

Trend · papers per month

162323485646 · Jun 202019922001200920172026
48 results for Probabilistic Reward Machines

Paper introduces PRMs to learn non-Markovian stochastic rewards for reinforcement learning.

problem Lack of structured representation for non-Markovian stochastic rewards in reinforcement learning.
method Introduces probabilistic reward machines (PRMs) and presents an algorithm to learn them from decision processes.
result Algorithm proves correct and convergent for learning PRMs from decision processes.

Efficient RL in PRMs with improved regret bound.

problem Reinforcement learning in probabilistic reward machines with non-Markovian rewards.
method Design of an algorithm with a new regret bound of O~(HOAT+H2O2A3/2+HT)\widetilde{O}(\sqrt{HOAT} + H^2O^2A^{3/2} + H\sqrt{T}).
result Improved regret bound over existing methods, matching lower bound up to a logarithmic factor.

New algorithm minimizes regret in multi-agent bandit problem with probabilistic communication.

problem Minimizing group regret in multi-agent multi-armed bandit problem with probabilistic communication.
method Proposes a new UCB-based algorithm for decentralized multi-agent multi-armed bandit problem on dd-regular graphs with probabilistic communication.
result The proposed algorithm outperforms state-of-the-art algorithms in minimizing group regret.

RePULSe improves language model alignment by reducing undesired outputs without sacrificing overall performance.

problem Aligning language models with human preferences while minimizing undesired outputs.
method Integrates probabilistic inference into RL training to reduce undesired outputs.
result RePULSe achieves a better balance between expected reward and undesired output probability.

This paper tackles distribution shift in model-based offline RL, proposing a shifts-aware reward method.

problem Distribution shift challenges model-based offline RL by distorting value estimation and policy optimization.
method The paper disentangles the problem into model bias and policy shift, proposing a shifts-aware reward through probabilistic inference.
result The proposed shifts-aware reward method effectively mitigates distribution shift and improves policy optimization.

Enhances reward specification in RL with a novel language-based approach.

problem Reward specification in RL can lead to unintended, potentially harmful behaviours.
method Developed a novel class of language-based Reward Machines using RML's built-in memory.
result Can specify non-regular, non-Markovian reward functions for complex tasks.

The paper examines how updates to probabilistic models influence behavior based on evidence.

problem Understanding how updates to probabilistic models influence behavior based on evidence.
method Study of KL-regularized soft updates as Bayesian posterior updates within a single probabilistic model.
result Posterior updates determine relative incentives but not absolute rewards, which are ambiguous up to context-specific baselines.

New algorithm uses machine learning to predict rewards for decision-making problems.

problem Sequential decision-making under uncertainty with scarce online data.
method Machine Learning-Assisted Upper Confidence Bound (MLA-UCB) algorithm.
result Proves to improve cumulative regret even with biased surrogate rewards.

Generative flow networks use RL to learn probabilistic models efficiently.

problem Training generative models with RL for compositional discrete objects.
method Reformulate GFlowNet training as entropy-regularized RL with specific reward and regularizer.
result Entropy-regularized RL can be competitive with established GFlowNet training methods.

We consider a problem of stochastic online learning with general probabilistic graph feedback, where each directed edge in the feedback graph has probability pijp_{ij}. Two cases are covered. (a) The one-step case, where after playing arm ii the learner observes a sample reward feedback of arm jj with independent prob…

2019-03-04abs ↗pdf ↗

New scoring rules improve probabilistic classification model evaluation.

problem Traditional scoring rules misalign with the preference for correct classifications.
method Introduces Penalized Brier Score (PBS) and Penalized Logarithmic Loss (PLL) to modify proper scoring rules.
result PBS and PLL better identify optimal checkpoints and early stopping points, leading to superior F1 scores.

A new method for reinforcement learning that adapts to different domains using auxiliary classifiers.

problem Training reinforcement learning agents to perform well in different domains with varying dynamics.
method Learning auxiliary classifiers to distinguish source-domain from target-domain transitions and modifying the reward function accordingly.
result The approach improves transfer performance in reinforcement learning tasks with varying dynamics.

The paper proposes a model reward scheme for collaborative ML based on Shapley value and information gain.

problem Designing fair incentives for collaborative machine learning.
method The paper proposes a reward scheme based on Shapley value and information gain, with properties like fairness and stability.
result The proposed reward scheme satisfies fairness and trade-offs between desirable properties via an adjustable parameter.

RLMH improves adaptive MCMC by optimizing contrastive divergence reward.

problem Tuning MCMC samplers is challenging and time-consuming.
method Formulated Metropolis-Hastings as a Markov decision process and used RL to adaptively tune it.
result A novel reward function based on contrastive divergence outperforms existing ones.

Discusses new probabilistic morphisms and geometric methods in machine and statistical learning.

problem Addressing challenges in statistical, machine, and manifold learning.
method Introduces category of probabilistic morphisms and geometric methods.
result New insights and applications in various learning fields.

Unified approach to path planning using probabilistic inference on factor graphs.

problem Path planning problems using probabilistic inference.
method Unified framework using probabilistic factor graphs and message composition rules.
result Unified approach includes various algorithms like Sum-product, Max-product, Dynamic programming, and mixed criteria.

PGBM creates probabilistic predictions efficiently.

problem Creating probabilistic predictions for large-scale data.
method Approximates leaf weights as random variables, learns moments via stochastic tree ensemble update equations.
result PGBM offers significant speedup and accuracy improvements over existing methods.

Extends PCVM for multi-class classification with improved accuracy.

problem Lack of probabilistic outputs and contradictory predictions in multi-class classification.
method Proposes mPCVM with two learning algorithms: top-down and bottom-up.
result Superior performance, especially with many classes, validated on synthetic and benchmark data.

Inverse optimal control, also known as inverse reinforcement learning, is the problem of recovering an unknown reward function in a Markov decision process from expert demonstrations of the optimal policy. We introduce a probabilistic inverse optimal control algorithm that scales gracefully with task dimensionality, an…

2012-06-18abs ↗pdf ↗

Paper explores limits and possibilities of aligning LLMs with human preferences.

problem Aligning LLMs with diverse human preferences to ensure fairness and informed outcomes.
method Analysis of probabilistic representation of human preferences and preservation of diverse preferences.
result LLMs can't fully align with human preferences using reward-based approaches due to Condorcet cycles, but mixed strategies are statistically possible.

Probabilistic models can handle causal inference without special tools.

problem Confusion over necessary tools for causal inference.
method Demonstrated through concrete examples that causal questions can be answered using standard probabilistic models.
result Causal questions can be addressed using standard probabilistic modelling and inference.

New algorithm optimizes for long-term user satisfaction in delayed reward settings.

problem Optimizing for long-term user satisfaction in delayed reward settings.
method Developed a predictive model of delayed rewards and a bandit algorithm that combines rewards and surrogate outcomes.
result Our algorithm significantly outperforms methods that optimize for short-term proxies or rely solely on delayed rewards.

Paper quantifies uncertainty in probabilistic models using Gaussian Processes.

problem Assessing reliability of probabilistic machine learning predictions.
method Systematic framework for estimating epistemic and aleatoric uncertainty, using Gaussian Processes and Monte Carlo sampling.
result Effective approach for quantifying prediction confidence in probabilistic models.

Study introduces TeMoP model for better stock market predictions.

problem Decreasing prediction errors and robustness across datasets in machine learning models.
method Probabilistic multiple lag order model based on trend encoding.
result TeMoP model outperforms machine learning models in accuracy and stability across different stock indexes.

New algorithm robust to probabilistic unbounded adversarial attacks in bandit problems.

problem Powerful adversaries that can catastrophically perturb the revealed reward in bandit problems.
method Proposes med-E-UCB and med-εε-greedy algorithms based on sample median for robustness.
result Achieves O(logT)\mathcal{O}(\log T) pseudo-regret under arbitrary and unbounded reward perturbation.

Probabilistic programming is a powerful abstraction for statistical machine learning. Applying static analysis methods to probabilistic programs could serve to optimize the learning process, automatically verify properties of models, and improve the programming interface for users. This field of static analysis for pro…

2019-09-10abs ↗pdf ↗

Robots need models of human behavior for both inferring human goals and preferences, and predicting what people will do. A common model is the Boltzmann noisily-rational decision model, which assumes people approximately optimize a reward function and choose trajectories in proportion to their exponentiated reward. Whi…

2020-01-13abs ↗pdf ↗

Training-free method improves large language model sequence quality via reward-guided sampling.

problem Optimizing large language model sequence quality over token likelihood.
method Reward-augmented target distribution combined with Sequential Monte Carlo sampling.
result Significant gains in sequence generation and mathematical reasoning tasks.

Transfer learning improves sparse, interpretable probabilistic classification.

problem Sparse and interpretable models in transfer learning.
method Two transfer learning extensions integrated into sparse and interpretable probabilistic classification vector machine.
result Transfer learning extensions improve sparsity and performance.

Enhances GFlowNets with distributional approach for risk-sensitive policies.

problem Limited applicability of current GFlowNet framework in handling stochastic reward functions.
method Adopting a distributional paradigm, parameterizing each edge flow through quantile functions, and introducing a risk-sensitive learning algorithm.
result Significant improvement on benchmarks due to enhanced training algorithm, even in deterministic reward settings.

Pymc-learn\textit{Pymc-learn} is a Python package providing a variety of state-of-the-art probabilistic models for supervised and unsupervised machine learning. It is inspired by scikit-learn\textit{scikit-learn} and focuses on bringing probabilistic machine learning to non-specialists. It uses a general-purpose high-level language that…

2018-10-31abs ↗pdf ↗

The paper presents a probabilistic framework for SPD matrices in machine learning.

problem Machine learning on SPD matrices is fragmented; this paper aims to unify it.
method Unified probabilistic framework using Gaussian distributions and Bayes classifiers.
result Different SPD machine learning tools can be reinterpreted and extended using Gaussian distributions.

This thesis proposes a derivatives hedging framework using deep learning and reinforcement learning.

problem Traditional hedging models fail in complex, uncertain markets due to assumptions like continuous trading and zero transaction costs.
method Integrates deep learning and reinforcement learning, using a spatiotemporal attention-based Transformer for probabilistic forecasting and hedging.
result The proposed method significantly outperforms traditional approaches in U.S. and Chinese financial markets.

Study on reward poisoning attacks on CMAB, revealing attackability depends on adversary's knowledge.

problem Reward poisoning attacks on Combinatorial Multi-Armed Bandits (CMAB).
method Provided a sufficient and necessary condition for attackability, devised an attack algorithm.
result Attackability of CMAB depends on adversary's knowledge of the bandit instance.