Enhances RL by controlling policy stochasticity through trajectory entropy constraints.
problem Non-stationary Q-value estimation and short-sighted entropy tuning in maximum entropy RL.
method Proposes TECRL framework with separate Q-functions for reward and entropy, enforcing a trajectory entropy constraint.
result DSAC-E algorithm achieves higher returns and better stability on OpenAI Gym benchmarks.
A new reinforcement learning method reduces action complexity for robust control.
problem Deep reinforcement learning's susceptibility to spurious correlations.
method Minimizing trajectory entropy to encourage simple, predictable actions.
result Trajectory Entropy Reinforcement Learning achieves superior performance and robustness.
Topological entropy measures the number of distinguishable orbits in a dynamical system, thereby quantifying the complexity of chaotic dynamics. One approach to computing topological entropy in a two-dimensional space is to analyze the collective motion of an ensemble of system trajectories taking into account how traj…
The paper introduces a new intrinsic reward method for exploration in reinforcement learning.
problem Improving exploration in reinforcement learning agents.
method Intrinsic rewards proportional to the entropy of future state-action features.
result The new objective leads to improved visitation of features within individual trajectories.
A neural network method estimates entropy production from system trajectories.
problem Estimating entropy production from system trajectories without detailed dynamics.
method Developed a neural estimator (NEEP) for entropy production (EP).
result NEEP rigorously proves to provide stochastic EP by optimizing an objective function.
CEM-GD combines CEM and gradient descent for efficient model-based RL.
problem Efficient planning in continuous control settings with large prediction horizons.
method Combines CEM for exploration and gradient descent for exploitation.
result Achieves better performance with fewer samples and less computation time.
We consider the problem of learning from demonstrated trajectories with inverse reinforcement learning (IRL). Motivated by a limitation of the classical maximum entropy model in capturing the structure of the network of states, we propose an IRL model based on a generalized version of the causal entropy maximization pr…
New method learns cell trajectories and network interactions from single-cell data.
problem Network inference in systems biology from steady-state data.
method Min-entropy estimation for stochastic dynamics, leveraging both temporal and perturbational data.
result Jointly learns cellular trajectories and network interactions.
Improved exploration methods for reinforcement learning with reduced sample complexity.
problem Challenges in reinforcement learning exploration in unknown environments.
method Proposed game-theoretic and trajectory entropy algorithms with improved sample complexity.
result Established statistical advantage of entropy-regularized MDPs for exploration and reduced sample complexity.
Final version. To appear in Discrete and Continuous Dynamical Systems - A.
This work extends ME-RL using diffusion models to sample optimal policies.
problem Sampling from the optimal policy trajectory distribution in ME-RL.
method Introducing Diffusion-Augmented Markov Decision Processes (DA-MDPs) to minimize reverse KL divergence.
result DA-MDPs enable seamless integration into various ME-RL methods and outperform baselines.
New method infers population dynamics from snapshots using path space optimization.
problem Recover dynamics of a population from its temporal marginals.
method Grid-free algorithm using Schrödinger bridges coupled via noisy gradient descent in mean-field limit.
result Global convergence to min-entropy estimator with end-to-end theoretical guarantees.
In Multi-Goal Reinforcement Learning, an agent learns to achieve multiple goals with a goal-conditioned policy. During learning, the agent first collects the trajectories into a replay buffer, and later these trajectories are selected randomly for replay. However, the achieved goals in the replay buffer are often biase…
The study analyzes the performance of a nonparametric estimator for dynamical systems.
problem Analyzing the performance of a nonparametric estimator for dynamical systems.
method Nonparametric least squares estimator (LSE) and information-theoretic methods.
result Rate-optimal error bounds for nonparametric hypotheses classes.
Paper proposes a policy-search algorithm to learn entropy-maximizing exploration policies in reward-free environments.
problem Reward-free learning in high-dimensional, continuous-control domains.
method Maximum Entropy POLicy optimization (MEPOL) algorithm that maximizes a non-parametric state entropy estimate.
result MEPOL learns a maximum-entropy exploration policy that facilitates learning various reward-based tasks.
Develops a method to infer cell trajectories from RNA sequencing data.
problem Inferring cell trajectories from single cell RNA-sequencing data.
method Entropy-regularized optimal transport for global optimization.
result Proves and implements a method to recover ground truth trajectories from limited samples.
Improved motion prediction for self-driving cars using trajectory sets and auxiliary losses.
problem Accurately predicting future vehicle motion for self-driving cars.
method Classification over trajectory sets with an auxiliary loss for off-road predictions and spatial-temporal relationships.
result Significant improvement in motion prediction performance on small datasets using map information.
Generative diffusion models are analyzed for their information dynamics.
problem Lack of a unified theoretical understanding of generative diffusion models.
method Integrated perspective connecting information-theoretic, dynamical, and thermodynamic aspects.
result Generative bandwidth is directly governed by the divergence of the score function's vector field.
New method learns stochastic thermodynamics from system currents.
problem Understanding entropy production in complex dynamical systems.
method Constructs learning framework using currents and machine learning loss functions.
result Derives loss functions for thermodynamic functions directly from dynamics.
Paper introduces DTAE to optimize RL algorithms, balancing exploration and exploitation.
problem Balancing exploration and exploitation in reinforcement learning.
method Soft policy optimization with entropy and dual-track advantage estimator (DTAE).
result DTAE accelerates RL algorithm convergence and improves performance.
Improved CEM for fast real-time planning in high-dimensional control tasks.
problem Sampling inefficiency of CEM in real-time planning.
method Novel additions to CEM including temporally-correlated actions and memory.
result 2.7-22x less samples and 1.2-10x performance increase.
The celebrated KAM Theory says that if one makes a small perturbation of a non-degenerate completely integrable system, we still see a huge measure of invariant tori with quasi-periodic dynamics in the perturbed system. These invariant tori are known as KAM tori. What happens outside KAM tori draws a lot of attention. …
Study Transformer layers under cross-entropy training using mean field control.
problem Understanding the behavior of Transformer layers in cross-entropy training.
method Continuous-depth mean field control analysis, treating depth as time and layer parameters as controls.
result Derivation of a Pontryagin condition for the limiting population problem, involving the softmax residual.
HCLM framework uses entropy regularization for open learning systems.
problem Real-world AI challenges and limitations of deep learning.
method Dynamical and information-theoretic framework with entropy regularization.
result Geometric entropy surrogates, especially log-determinant covariance entropy, induce stronger and more stable information forces.
We show that a small perturbation of the boundary distance function of a simple Finsler metric on the n-disc is also the boundary distance function of some Finsler metric. (Simple metric form an open class containing all flat metrics.) The lens map is map that sends the exit vector to the entry vector as a geodesic c…
QNA uses quantum-inspired density operators to diagnose market dependence and structural risk.
problem Lack of unified operator representation for market dependence and structural risk diagnostics.
method Quantum Network of Assets (QNA) framework using density operators.
result QNA entropy remains strongly related to covariance spectral entropy but becomes distinct with multi-feature rolling trajectories.
Proposes an active RBI framework using Rényi information measures for more informed decision-making.
problem Optimal latent variable estimates in real-time settings with streaming noisy observations.
method Unified inference and query selection steps through Rényi entropy and α-divergence; new objective called Momentum for exploration.
result Analytically demonstrates superior performance compared to conventional methods like mutual information.
New algorithm infers trajectories from partial observations using optimal transport.
problem Inferring trajectories from partial observations of coupled systems.
method Extends MFL algorithm to latent SDEs using observable state space models and partial observations.
result Experiments show significant outperformance over latent-free baseline.
Study shows fast rates for inverse reinforcement learning with linear rewards.
problem Entropy-regularized min-max inverse reinforcement learning in finite-horizon MDPs.
method Structural and statistical analysis of Min-Max-IRL with pseudo-self-concordance.
result Both trajectory-level KL divergence and parameter error decay at O(n−1). Improved machine learning method estimates entropy production robustly.
problem Estimating entropy production from trajectory data.
method Variational representation of α-divergence loss functions. result Optimal α=−0.5 yields best performance. EntroPath learns manifold geometry from diffusion paths.
problem Learning geodesic geometry from data graphs with spurious shortcuts.
method Maximum Entropy Path Ensemble Embedding (MERW) with k-step diffusion paths.
result EntroPath converges to squared geodesic distance in the short-time limit.
This paper optimizes trading strategies to minimize risk and maximize profit while accounting for market uncertainty.
problem Optimizing trading strategies to minimize risk and maximize profit while accounting for market uncertainty.
method Relative entropy-regularized robust optimal control problem, modeled as a stochastic differential game.
result Analytical expressions for optimal strategy and trajectory are derived under specific assumptions.
In recent studies on model-based reinforcement learning (MBRL), incorporating uncertainty in forward dynamics is a state-of-the-art strategy to enhance learning performance, making MBRLs competitive to cutting-edge model free methods, especially in simulated robotics tasks. Probabilistic ensembles with trajectory sampl…
A new method reduces compounding errors in model-based reinforcement learning.
problem Compounding errors in long horizon predictions from model-based reinforcement learning.
method Maximum Entropy Model Rollouts (MEMR) with non-uniform sampling and prioritized experience replay.
result Significantly reduces computation requirements compared to other model-based methods.
Modeling driver trajectories using inverse reinforcement learning and random utility.
problem Modeling rational driver behavior in road networks from sparse sensor data.
method Apply random utility theory to model unknown reward function, introduce extended state, and use Markov decision process.
result Maximum entropy inverse reinforcement learning is a special case of the proposed approach.
New method predicts Alzheimer's risk with individual uncertainty estimates.
problem Predicting conversion from mild cognitive impairment to Alzheimer's disease.
method Persistent homology of clinical trajectories combined with stacking ensemble.
result Pipeline achieves high accuracy and individual-level uncertainty quantification.
ESPRESSO segments time-series data for better human activity recognition.
problem Segmenting high-dimensional time-series data for applications like HAR.
method ESPRESSO combines entropy and shape analysis for multi-dimensional time-series segmentation.
result ESPRESSO outperforms four state-of-the-art methods across seven datasets.
Patient subtyping based on temporal observations can lead to significantly nuanced subtyping that acknowledges the dynamic characteristics of diseases. Existing methods for subtyping trajectories treat the evolution of clinical observations as a homogeneous process or employ data available at regular intervals. In real…
In this paper, we propose an implicit gradient descent algorithm for the classic k-means problem. The implicit gradient step or backward Euler is solved via stochastic fixed-point iteration, in which we randomly sample a mini-batch gradient in every iteration. It is the average of the fixed-point trajectory that is c…
Deep learning speeds up protein mapping entropy calculation.
problem Efficiently calculating the mapping entropy of protein structures.
method Deep graph networks for accelerating mapping entropy computation.
result Deep graph networks achieve a speedup factor of up to 10^5.
Recently, researchers proposed various low-precision gradient compression, for efficient communication in large-scale distributed optimization. Based on these work, we try to reduce the communication complexity from a new direction. We pursue an ideal bijective mapping between two spaces of gradient distribution, so th…
In this paper, we introduce an actor-critic algorithm called Deep Value Model Predictive Control (DMPC), which combines model-based trajectory optimization with value function estimation. The DMPC actor is a Model Predictive Control (MPC) optimizer with an objective function defined in terms of a value function estimat…
This work extends entropic optimal transport to non-product reference couplings, focusing on Gaussian cases.
problem Finding a diffuse coupling between two measures with non-product reference couplings.
method Reduction of the entropic optimal transport problem to a matrix optimization problem.
result Complete description of the solution for non-product reference couplings, including primal and dual variables.
Develops statistical framework for resolving reward function ambiguity in inverse reinforcement learning.
problem Non-uniqueness of reward functions in inverse reinforcement learning.
method Entropy regularization combined with least-squares reconstruction of the reward from the soft Bellman residual.
result Least-squares reward function is unique and consistent with the expert policy.
Introduces alternators for modeling sequences, outperforming baselines.
problem Modeling complex sequential data with stability and efficiency.
method Two neural networks (OTN and FTN) alternate between outputting samples in observation and feature spaces, learned via cross-entropy criterion.
result Alternators outperform strong baselines in various domains (Lorenz equations, Neuroscience, Climate Science).
We study the sparse entropy-regularized reinforcement learning (ERL) problem in which the entropy term is a special form of the Tsallis entropy. The optimal policy of this formulation is sparse, i.e.,~at each state, it has non-zero probability for only a small number of actions. This addresses the main drawback of the …
This work proposes a new feature for transportation mode classification using GPS trajectories.
problem Classifying transportation modes from GPS trajectories to optimize urban mobility.
method The Ordinal Pattern Transition Graph and its self-transition probability are used for classification.
result The proposed feature outperforms existing methods in transportation mode classification.
Graph Neural Networks model 3D granular flow simulations.
problem Accurate modeling of complex 3D granular flow processes.
method Graph Neural Networks approach to simulate 3D granular flow using LIGGGHTS.
result Machine learning trajectories match physical granular flow processes.