Paper clusters event sequences using a reinforcement learning approach with policy mixture model.
problem Clustering event sequences with varying temporal patterns.
method Reinforcement learning with a policy mixture model, decomposing sequences into states and actions.
result Effective clustering of event sequences into underlying policies, outperforming existing methods.
Proposes a Riemannian optimization for policy improvement in MDPs.
problem Optimizing policy functions in Markov decision processes (MDPs).
method Riemannian proximal optimization algorithm with Gaussian mixture model (GMM).
result Guaranteed convergence and efficacy demonstrated in preliminary experiments.
Combines parametric and nonparametric models for better off-policy evaluation.
problem Improving off-policy evaluation in reinforcement learning.
method Mixture-of-experts approach combining parametric and nonparametric models.
result Mixture-based approach outperforms individual models and state-of-the-art estimators.
A new framework for robust policy learning in MDPs with linear mixture dynamics.
problem Off-dynamics challenge in real-world decision-making problems.
method Linear mixture DRMDP framework, meta algorithm for robust policy learning.
result The new framework provides a more refined representation of uncertainties.
New method learns MDP policies without projections, achieving near optimal results.
problem Learning policies for large MDPs with high sample complexity.
method Projection-free stochastic primal-dual method for approximate linear programming.
result PAC sample complexity analysis and improved efficiency compared to existing methods.
Paper proposes interpretable RL policies from a mixture of experts.
problem Making RL policies transparent and understandable in real-world applications.
method Policy iteration scheme with interpretable experts and prototypical states.
result Proposed algorithm learns policies comparable to neural networks but more interpretable.
Improved SAC with AWMP for better control tasks.
problem Discontinuous and non-smooth optimal policies in reinforcement learning.
method Advantage Weighted Mixture Policy (AWMP) for SAC, learning state-specific weights.
result SAC with AWMP outperforms SAC in four control tasks.
Variational Proximal Policy Optimization improves reinforcement learning from human feedback.
problem Policy mode collapse and brittle exploration loops in reinforcement learning.
method Particle-based variational inference framework with Mixture-of-Experts architecture.
result Significant improvements in complex reasoning benchmarks.
New framework shifts bandit algorithms from expected reward to preference metrics, optimizing mixtures of arms.
problem Traditional bandit algorithms focus on expected rewards, ignoring variability and risk.
method Introduces preference metrics (PMs) and designs algorithms to optimize mixtures of arms.
result Optimal policy selects mixtures of arms based on specific weights, not a single best arm.
This paper applies CPI to deep RL, improving stability and performance.
problem Improving stability and performance in deep reinforcement learning.
method Combines Conservative Policy Iteration with deep neural networks and adaptive mixture rates.
result Demonstrates improved stability and performance in deep RL algorithms.
The paper tackles robust policy learning from multiple data sources.
problem Learning a policy that generalizes across diverse settings from multiple heterogeneous data sources.
method Proposes a minimax regret optimization objective and a policy learning algorithm combining doubly robust offline policy evaluation and no-regret learning.
result Achieves minimal worst-case mixture regret up to a moderated vanishing rate of the total data across all sources.
A new method for steering large agent populations efficiently.
problem Controlling the configuration of a swarm of identical, interacting cooperative agents.
method Mean-Field Schrodinger Bridges with Gaussian Mixture Models.
result A highly efficient parameterization to approximate optimal solutions of the MFSB problem in closed form.
Monotonic policy improvement and off-policy learning are two main desirable properties for reinforcement learning algorithms. In this paper, by lower bounding the performance difference of two policies, we show that the monotonic policy improvement is guaranteed from on- and off-policy mixture samples. An optimization …
In this paper, we propose a novel maximum causal Tsallis entropy (MCTE) framework for imitation learning which can efficiently learn a sparse multi-modal policy distribution from demonstrations. We provide the full mathematical analysis of the proposed framework. First, the optimal solution of an MCTE problem is shown …
New algorithm reduces regret in linear mixture SSPs without cost bounds.
problem Learning optimal paths in stochastic environments with cost constraints.
method Extended value iteration with variance-aware confidence set.
result Achieves nearly minimax optimal regret bound of O ( d B ∗ K ) O(dB_*\sqrt{K}) O ( d B ∗ K ) . New algorithm reduces dynamic regret for MDPs with unknown transition and adversarial rewards.
problem Episodic linear mixture MDPs with unknown transition and adversarial rewards.
method Combines occupancy-measure-based global optimization and policy-based variance-aware value-targeted regression.
result Achieves near-optimal dynamic regret of O ~ ( d H 3 K + H K ( H + P ˉ K ) ) \widetilde{\mathcal{O}}(d \sqrt{H^3 K} + \sqrt{HK(H + \bar{P}_K)}) O ( d H 3 K + H K ( H + P ˉ K ) ) . Paper introduces risk-sensitive bandits with optimal arm mixtures.
problem Designing algorithms for risk-sensitive multi-armed bandits.
method Formalizes risk-sensitive bandits framework, identifies optimal arm mixtures, designs regret-efficient algorithms.
result Regret-efficient algorithms track optimal arm mixtures or solitary arms.
New method estimates state-action stationary distribution for better off-policy policy evaluation.
problem Accurately estimating state-action stationary distribution for off-policy policy evaluation.
method Estimated Mixture Policy (EMP) for state and state-action stationary distribution corrections.
result Empirical validation shows improved accuracy over state-of-the-art methods.
New algorithm learns policies without explicit rewards for MDPs.
problem Learning policies for MDPs without explicit rewards.
method UCRL-RFE algorithm with linear function approximation.
result UCRL-RFE achieves optimal policy with minimal exploration samples.
HO2 learns options from data efficiently, improving robot manipulation tasks.
problem Learning options from raw pixel inputs in 3D robot manipulation tasks.
method HO2 infers likely option choices and trains all policy components off-policy.
result HO2 outperforms existing methods on 3D robot manipulation tasks.
A green simulation-assisted reinforcement learning method for biomanufacturing.
problem Complexity, high variability, lead time, and limited historical data in biopharmaceutical manufacturing.
method Quantifies model risk, uses posterior distribution, and selectively reuses simulation data.
result Demonstrates promising performance in online learning and decision making.
New approach for off-policy learning in contextual bandits with performance guarantees.
problem Improving performance of logging policies in contextual bandits.
method PAC-Bayesian analysis of policy mixtures, providing tighter generalization bounds and tractable optimization algorithms.
result Proved tighter generalization bounds and demonstrated effectiveness in practical scenarios.
The paper proposes a structure learning model for efficient reinforcement learning.
problem Efficient sample-based decision making in complex environments.
method Temporal Gaussian Mixture Model with structure learning.
result The model discovers the number of states and transition probabilities.
PFPN uses particle filtering to improve character control in physics-based simulations.
problem Premature commitment to suboptimal actions in high-dimensional continuous control problems for articulated characters.
method Proposes a particle-based action policy using particle filtering to dynamically explore and discretize the action space.
result Demonstrates better imitation performance and robustness to external perturbations compared to Gaussian policies.
Swift Hydra uses RL and generative AI to improve anomaly detection.
problem Generalization to unseen anomalies in critical systems.
method Generative AI and reinforcement learning (RL) for synthesizing diverse anomaly samples.
result Swift Hydra outperforms state-of-the-art models on ADBench benchmark.
New distributions allow greedy arm selection in sparse bandit problems.
problem Sparse contextual bandit problem with sparse parameters and feature distributions.
method Introduced new distribution classes and demonstrated that mixtures of these distributions are also greedy-applicable.
result Greedy algorithm applicable to a wider range of arm feature distributions, including those with origin-asymmetric support.
New algorithm learns optimal path in reinforcement learning with linear approximations.
problem Optimal path learning in reinforcement learning with linear approximations.
method Proposes novel algorithm with Hoeffding-type and Bernstein-type confidence sets.
result Achieves near-optimal regret guarantee for linear mixture SSP.
New WildMix dataset and Spectro-Temporal Transformer model for better monoaural audio source separation.
problem Challenging monoaural audio source separation.
method Introducing WildMix dataset and Spectro-Temporal Transformer model (STT) with Spectro-Temporal Encoder (STE).
result STT outperforms previous baselines on the WildMix dataset.
Algorithm learns from offline data to improve performance in target environment.
problem Learning from offline data in a target environment with unknown shifts.
method Adaptive algorithm that uses offline data to improve performance when informative.
result Algorithm provably improves performance over purely online learning when offline data are informative.
New method prevents forgetting in learning new tasks.
problem Poor ability of models to solve new problems without forgetting.
method Task-agnostic hierarchical information-theoretic optimality principle with Mixture-of-Variational-Experts layer.
result Demonstrated competitive performance in continual supervised and reinforcement learning.
Paper shows equivalence between two alignment methods and introduces a new algorithm.
problem Ensuring human alignment of large language models for useful, safe, and pleasant user experience.
method Introduces IPO-MD algorithm, showing equivalence between IPO and Nash-MD methods.
result Equivalence between IPO and Nash-MD methods proven when considering online version of IPO.
Bayesian networks learn sub-population differences from data.
problem Inference from a single network structure can be misleading when data populations are heterogeneous.
method A mixture of Bayesian networks where component probabilities depend on individual characteristics.
result Identifies both network structures and demographic predictors of sub-population membership.
Trust-region method improves Gaussian mixture models for complex distributions.
problem Learning accurate approximations of complex, multimodal distributions.
method Information-geometric trust regions for principled exploration, lower bound optimization, online component adaptation.
result Improved GMM approximations with better quality and efficiency.
Paper optimizes learning MDPs with adversarial rewards using linear models.
problem Learning MDPs with adversarial rewards and changing reward functions.
method Proposes POWERS, an optimistic policy optimization algorithm.
result Achieves nearly optimal regret bound of O ( d H T ) O(dH\sqrt{T}) O ( d H T ) . New algorithm tackles adversarial RL without horizon constraints.
problem Adversarial reinforcement learning with unknown transition kernel.
method Uses weighted least square estimator and occupancy measure for policy search.
result Achieves near-optimal regret bound of O ( K ) O(\sqrt{K}) O ( K ) . This paper studies the potential of the return distribution for exploration in deterministic reinforcement learning (RL) environments. We study network losses and propagation mechanisms for Gaussian, Categorical and Gaussian mixture distributions. Combined with exploration policies that leverage this return distributio…
vsOED optimizes experiment design with reinforcement learning for Bayesian models.
problem Optimizing the sequence of experiments in Bayesian models for efficient data collection.
method Reinforcement learning with variational posterior approximations to optimize design policy.
result vsOED achieves superior sample efficiency compared to existing methods.
The article develops a model for skewness risk in risk parity portfolios.
problem Managing skewness risk in asset allocation models.
method Modeling asset returns with skewness and jumps, deriving analytical formulas for risk contributions.
result Skewness-based risk parity portfolios outperform volatility-based portfolios in managing jump risks.
SPAQL improves RL by adaptively partitioning state-action space and learning a time-invariant policy.
problem Efficient model-free reinforcement learning with scalable algorithms.
method Adaptive Q-learning with UCB and Boltzmann exploration, automatically tuning temperature.
result SPAQL converges faster and uses fewer resources than AQL, showing higher sample efficiency.
TRM improves long-horizon LLM RL by masking divergent sequences.
problem Long-horizon reinforcement learning with LLMs suffers from off-policy mismatch and approximation errors.
method Derives and applies trust region bounds to control divergence, proposing Trust Region Masking.
result First non-vacuous monotonic improvement guarantees for long-horizon LLM-RL.
TRM improves long-horizon reinforcement learning for LLMs by masking divergent sequences.
problem Long-horizon reinforcement learning for LLMs suffers from off-policy mismatch and approximation errors.
method Derives and applies trust region bounds to control divergence, proposing Trust Region Masking.
result First non-vacuous monotonic improvement guarantees for long-horizon LLM-RL.
Mixreg improves RL generalization by mixing diverse training environments.
problem Overfitting in RL agents trained in limited environments.
method Trains on a mixture of diverse observations and imposes linearity constraints.
result Mixreg outperforms baselines on unseen testing environments.
CQL (ReDS) learns from varied driving behaviors, improving offline RL performance.
problem Learning from datasets with non-uniform variability in behavior policies.
method Reweighting the data distribution to allow per-state flexibility in following the behavior policy.
result CQL (ReDS) improves performance in various offline RL tasks.
Paper tackles overestimation bias in continuous control, improving performance by 25%.
problem Overestimation bias in off-policy learning.
method Truncated Quantile Critics (TQC) combines distributional representation, truncation, and ensembling of critics.
result TQC outperforms state-of-the-art methods by 25% on the Humanoid environment.
Enhances mixture models with classifier-defined weights.
problem Density evaluation and sampling in mixture models.
method Introduces Classifier Weighted Mixtures (CWM) with functional weights.
result Improves expressivity in variational estimation without increasing complexity.
The paper addresses treatment recommendation problems by optimizing distributional characteristics.
problem Optimizing treatment recommendations based on distributional targets.
method Characterizes the problem's difficulty and proposes near-optimal policies.
result Characterizes the difficulty of the problem and proposes near-regret optimal policies.
Optimal mixtures of generative models outperform individual models on image datasets.
problem Selecting the best single model from a group of trained generative models.
method Formulated a quadratic optimization problem and proposed the Mixture-UCB algorithm for efficient selection.
result Mixture of generative models achieves better evaluation scores than individual models on benchmark datasets.
New bounds on sample size for identifying mixture models with grouped samples.
problem Identifying mixture models with minimal sample size.
method Generalized identifiability bounds for mixture models with grouped samples.
result Identifiability with ( 2 m − 1 ) / ( k − 1 ) (2m-1)/(k-1) ( 2 m − 1 ) / ( k − 1 ) samples per group, with no improvement possible.