Contrastive learning estimates transition kernels for continuous-time stochastic processes.
problem Estimating transition kernels for continuous-time stochastic processes without labeled data.
method Contrastive learning applied to strong-mixing continuous-time stochastic processes.
result Contrastive learning can estimate transition kernels for small-to-mid-range intervals in the diffusion case.
Develops a new framework for conditional independence.
problem Generalizing previous notions of conditional independence.
method Introduces transition probability spaces and transitional random variables.
result Satisfies all desired relevance relations except symmetry.
There are various parametric models for analyzing pairwise comparison data, including the Bradley-Terry-Luce (BTL) and Thurstone models, but their reliance on strong parametric assumptions is limiting. In this work, we study a flexible model for pairwise comparisons, under which the probabilities of outcomes are requir…
New model for pairwise comparisons without stochastic transitivity.
problem Suboptimal performance of models assuming stochastic transitivity in real-world scenarios.
method Proposes a general family of statistical models using a skew-symmetric matrix.
result Achieves minimax-rate optimality and adapts to data sparsity.
Study shows strong min-max principle for phase transitions.
problem Understanding nodal sets near minimal hypersurfaces.
method Analogous to White's principle, applies to Allen-Cahn energy.
result Strong min-max principle for phase transitions.
Study improves weak error estimates for rough volatility models.
problem Efficient numerical schemes for non-Markovian stochastic processes with rough volatility.
method Analyzes weak rates for a class of stochastic processes with rough stochastic volatility.
result Weak rate is of order min{3H+0.5, 1} for a large class of test functions.
New estimators show consistent estimation is possible with randomized item assignment.
problem Estimating underlying comparison probabilities from noisy pairwise comparisons.
method Study permutation-based models under strong stochastic transitivity, randomized item assignment.
result Rates of estimators are optimal for a large class of graphs.
New algorithms for batched dueling bandits with improved regret bounds.
problem Batched dueling bandits with noisy pairwise comparisons.
method Developed algorithms for two settings: Condorcet winner and strong stochastic transitivity.
result Regret bounds match sequential bounds using only a logarithmic number of batches.
Adjoint sampler targets infinite-dimensional function spaces for efficient sampling.
problem Limited theory and algorithms for sampling infinite-dimensional function spaces.
method Adjoint Sampler for infinite-dimensional function spaces based on stochastic maximum principle.
result FAS achieves superior performance in synthetic and real systems.
New RL algorithm optimizes policies with bandit feedback, matching previous bounds.
problem Optimizing policies with unknown transitions and bandit feedback.
method Optimistic Trust Region Policy Optimization (TRPO) algorithm.
result Sub-linear regret bounds for both stochastic and adversarial rewards.
New issue found in value-based reinforcement learning for stochastic environments.
problem Value-based reinforcement learning struggles with stochastic state transitions.
method Demonstrated using a multiobjective Markov Decision Process (MOMDP).
result Approaches may converge to Pareto-dominated solutions instead of optimal ones.
The existence of stationary Markov perfect equilibria in stochastic games is shown under a general condition called "(decomposable) coarser transition kernels". This result covers various earlier existence results on correlated equilibria, noisy stochastic games, stochastic games with finite actions and state-independe…
A contraction analysis improves model-based RL's error recovery.
problem Theoretical understanding of model-based reinforcement learning.
method Contraction analysis applied to both stochastic and deterministic state transitions.
result Error reduction in cumulative reward using branched rollouts.
The paper models market crashes as phase transitions, finding dynamic transitions offer better predictions.
problem Understanding and predicting extreme financial events like market crashes.
method Employing phase transition theory, focusing on endogenous crashes, and comparing DPT, CPT, and SPT.
result Dynamic phase transitions provide more accurate predictions of market crashes compared to critical and stochastic models.
In this paper we study how to learn stochastic, multimodal transition dynamics in reinforcement learning (RL) tasks. We focus on evaluating transition function estimation, while we defer planning over this model to future work. Stochasticity is a fundamental property of many task environments. However, discriminative f…
Study models forest transitions with deep learning for parameter estimation.
problem Complex dynamics of forest, agricultural, and abandoned lands.
method Developed a stochastic differential equation model and used deep learning for parameter estimation.
result Deep learning approach estimates model parameters from time-series data.
Kernel-based reinforcement learning (KBRL) stands out among reinforcement learning algorithms for its strong theoretical guarantees. By casting the learning problem as a local kernel approximation, KBRL provides a way of computing a decision policy which is statistically consistent and converges to a unique solution. U…
Forré introduces a new conditional independence notion for mixed variables.
problem Unified framework for random and non-stochastic variables.
method Unified framework of transitional conditional independence and causal calculus for iDMGs.
result Unified framework connects conditional independencies to graphical separation criteria.
A novel approach models rating transitions using Lie groups and Deep Learning.
problem Modeling rating transitions with geometric properties and stochastic processes.
method Introducing Itô-SDEs on Lie groups, using TimeGAN for calibration, and examining rating matrix properties.
result The geometric approach using Lie groups and Deep Learning generates a good fit for rating transitions.
New neural processes use stacked Markov operators to improve flexibility.
problem Improving flexibility in neural processes.
method Stacking neural parameterized Markov transition operators in function space.
result MNPs outperform baseline models on various tasks.
Proposes a method for approximating transition densities of SDEs driven by gamma processes.
problem Calculating transition densities for SDEs driven by gamma processes.
method Taylor-type approximation and conditional expectation of multiple stochastic integrals.
result Efficiency of the proposed method demonstrated through numerical tests.
Neural models learn continuous-time Markov chain transition rates from data.
problem Learning transition rates for complex stochastic systems.
method Neural networks to model nonlinear transition rates from observed data.
result Neural models outperform traditional methods in accuracy.
This paper optimizes the number of comparisons needed to find the best k items from pairwise comparisons.
problem Finding the best k items from pairwise comparisons with limited comparisons.
method Developed algorithms for finding probably approximately correct and exact best k items under stochastic conditions.
result Upper and lower bounds on the number of comparisons for finding the best k items, with matching upper bounds for PAC best k items.
Develops CLTs for Markov chain transition probabilities and policies.
problem Estimating transition probabilities and policies in controlled Markov chains.
method Non-parametric estimator for transition matrices; CLTs for value, Q-, and advantage functions; goodness-of-fit tests.
result Asymptotic normality of estimators under specific logging policies.
SPEDER extracts state-action abstraction from dynamics for reinforcement learning.
problem Curse of dimensionality and limited applicability of spectral methods.
method Spectral Decomposition Representation (SPEDER) that extracts state-action abstraction from dynamics without policy dependence.
result Theoretical analysis establishes sample efficiency in online and offline settings.
Strong parallels can be drawn between the theory of minimal hypersurfaces and the theory of phase transitions. Borrowing ideas from the former we extend recent results on the regularity of stable phase transition interfaces to the finite Morse index case. As an application we present a PDE-based proof of the celebrated…
New algorithm optimally evaluates policies with linear approximations.
problem Policy evaluation with linear function approximation.
method Accelerated, variance-reduced fast temporal difference algorithm (VRFTD).
result VRFTD matches both deterministic and stochastic lower bounds.
Algorithm samples constrained stochastic differential equations.
problem Sampling stochastic differential equations with complex constraints.
method Pathspace Metropolis-adjusted manifold sampling.
result Demonstrated effectiveness in various constrained conditions.
The interaction between transitivity and sparsity, two common features in empirical networks, implies that there are local regions of large sparse networks that are dense. We call this the blessing of transitivity and it has consequences for both modeling and inference. Extant research suggests that statistical inferen…
Method estimates noise transition matrix from noisy labels without relying on unreliable class-posterior estimation.
problem Estimating noise transition matrix from noisy data.
method Total variation regularization to encourage distinguishable predicted probabilities.
result Consistent estimator of the noise transition matrix under mild assumptions.
New algorithm achieves data-dependent regret bounds in MDPs with unknown transitions.
problem Achieving best-of-both-worlds guarantees with data-dependent regret bounds in MDPs with unknown transitions.
method Optimistic follow-the-regularized-leader algorithm with new optimistic Q-function estimators and transition bonus.
result First-order, second-order, and path-length bounds with polylog(T) regret in the stochastic regime.
The thesis explores prediction games with different opponents and moves, revealing intrinsic barriers and efficient algorithms.
problem Understanding and designing efficient algorithms for prediction games with various opponents and move orders.
method Geometric insights into three types of prediction games: general learning task, prediction with expert advice, and online convex optimization.
result Revealed intrinsic barriers and developed computationally efficient learning algorithms with strong theoretical guarantees.
Study shows Elo models fail to accurately measure transitive strength in competitive games.
problem Elo models fail to correctly identify the transitive component in real-world competitive games.
method Investigated the challenge of identifying the transitive component in games, proposed an extension of the Elo score.
result Disc ranking system assigns two scores: skill and consistency.
Paper introduces TtT, market-implied transition time, from greenium term structure.
problem Estimating market-implied transition time to a low-carbon economy.
method Develops inference theory for TtT, introduces two stochastic models.
result Combines two-layer analysis for consistent estimation of diffusion parameters.
Method uses neural networks for high-dimensional committor function calculations.
problem Computing committor functions for high-dimensional stochastic processes.
method Parameterizes committor function with neural networks and optimizes weights using stochastic algorithms.
result Achieves moderate accuracy for high-dimensional problems.
Improved POMDP regret to sqrt(T) with known observation model.
problem Average-reward POMDPs with unknown transition model but known observation model.
method Optimistic algorithm using deterministic policies and novel estimation techniques.
result First approach with regret guarantee of sqrt(T) against optimal policy.
Elo rating outperforms complex models in skill estimation despite model misspecification.
problem Elo rating's reliability in skill estimation is questioned due to model misspecification.
method Interpreted Elo as online gradient descent and conducted synthetic experiments on non-BT models.
result Elo's superior performance in win rate prediction compared to complex models is explained through its sparsity and ranking effectiveness.
Enhances machine learning for complex systems by embedding transition manifolds.
problem Identifying low-dimensional dynamics in high-dimensional multiscale systems.
method Kernel embeddings of transition manifolds in reproducing kernel Hilbert spaces.
result Robust and more efficient algorithm for identifying reaction coordinates.
Curiosity-Critic improves world model training by focusing on cumulative prediction error.
problem Training world models with intrinsic rewards that consider cumulative prediction error.
method Curiosity-Critic uses a surrogate reward based on the difference between current and asymptotic prediction errors, estimated online by a co-trained critic.
result Curiosity-Critic outperforms other methods in training speed and final world model accuracy.
New algorithm learns mixtures of subcubes efficiently, with applications to decision trees.
problem Learning mixtures of subcubes over binary space.
method Higher-order multilinear moments for nO(logk)-time learning. result Polynomial dependence on 1/ε for decision trees with stochastic transitions. Generative Stochastic Networks (GSNs) have been recently introduced as an alternative to traditional probabilistic modeling: instead of parametrizing the data distribution directly, one parametrizes a transition operator for a Markov chain whose stationary distribution is an estimator of the data generating distributio…
Paper analyzes LPSA algorithm for constrained optimization, revealing phase transitions and bias-variance trade-offs.
problem Optimization problems with linear constraints.
method Loopless projection stochastic approximation (LPSA) with jump diffusion approximation.
result LPSA trajectories converge to SDEs, revealing asymptotic behaviors and phase transitions.
Proposes a method to learn a transition operator for generating samples.
problem Learning a transition operator for generating samples efficiently and biologically plausibly.
method Directly learns a stochastic transition operator via variational methods, encouraging it to 'walk back' quickly to data points.
result The learned transition operator generates high-quality samples and matches the data distribution well beyond the length of individual training trajectories.
GDM models time series with smoother transitions and interpretable states.
problem Capturing smooth, variable-speed transitions and stochastic mixtures of states.
method Introduces a continuous relaxation of discrete states and a Gumbel noise model.
result Models real-world datasets more faithfully with smoother dynamics and interpretable states.
Generative diffusion models exhibit phase transitions in statistical mechanics, impacting their performance.
problem Understanding the performance and capabilities of generative diffusion models.
method Reformulating generative diffusion models using statistical mechanics, focusing on phase transitions and symmetry breaking.
result Generative diffusion models undergo second-order phase transitions with mean-field universality, critical instability, and mean-field critical exponents.
Estimates bisimulation metrics from sample streams, not full transition models.
problem Estimating Markov chain metrics from limited sample data.
method Stochastic optimization using linear programming and primal-dual method.
result Validated through empirical evaluations, providing sample complexity guarantees.
This paper proposes a stochastic model using the concept of Markov chains for the inter-state transitions of the millisecond order quasi-stable phase synchronized patterns or synchrostates, found in multi-channel Electroencephalogram (EEG) signals. First and second order transition probability matrices are estimated fo…
New analysis of Muon and SignSGD on matrix-valued least squares problems.
problem Understanding the behavior of Muon and SignSGD on matrix-valued least squares problems.
method Derive explicit deterministic dynamics to study learning behavior of Muon and SignSGD.
result Muon and SignSGD exhibit different optimal learning rates and convergence characteristics based on batch size and data covariance.