arXiv research

Method learns dynamics of slow variables from stochastic data.

Paper introduces slow kill for efficient large-scale variable screening.

Paper shows how SFA fits into FBM framework for time series separation.

We provide a rigorous numerical computation method to validate periodic, homoclinic and heteroclinic orbits as the continuation of singular limit orbits for the fast-slow system $x' = f(x,y,ε), y' = εg(x,y,ε)$ with one-dimensional slow variable $y$. Our validation procedure is based on topological tools called isolatin…

In this paper we investigate the life-span of classical solutions to the hyperbolic geometric flow in two space variables with slow decay initial data. By establishing some new estimates on the solutions of linear wave equations in two space variables, we give a lower bound of the life-span of classical solutions to th…

Hierarchical pretraining with slow-fast ODEs

The success of enhanced sampling molecular simulations that accelerate along collective variables (CVs) is predicated on the availability of variables coincident with the slow collective motions governing the long-time conformational dynamics of a system. It is challenging to intuit these slow CVs for all but the simpl…

Inspired by the success of deep learning techniques in the physical and chemical sciences, we apply a modification of an autoencoder type deep neural network to the task of dimension reduction of molecular dynamics data. We can show that our time-lagged autoencoder reliably finds low-dimensional embeddings for high-dim…

Anytime MiniBatch speeds up online distributed optimization by handling slow nodes.

Markov state models (MSMs) and Master equation models are popular approaches to approximate molecular kinetics, equilibria, metastable states, and reaction coordinates in terms of a state space discretization usually obtained by clustering. Recently, a powerful generalization of MSMs has been introduced, the variationa…

This paper studies the problem of learning the conditional distribution of a high-dimensional output given an input, where the output and input may belong to two different domains, e.g., the output is a photo image and the input is a sketch image. We solve this problem by cooperative training of a fast thinking initial…

We extend Neural Processes (NPs) to sequential data through Recurrent NPs or RNPs, a family of conditional state space models. RNPs model the state space with Neural Processes. Given time series observed on fast real-world time scales but containing slow long-term variabilities, RNPs may derive appropriate slow latent …

Coordinate ascent variational inference is an important algorithm for inference in probabilistic models, but it is slow because it updates only a single variable at a time. Block coordinate methods perform inference faster by updating blocks of variables in parallel. However, the speed and stability of these algorithms…

We give examples of data-generating models under which Breiman's random forest may be extremely slow to converge to the optimal predictor or even fail to be consistent. The evidence provided for these properties is based on mostly intuitive arguments, similar to those used earlier with simpler examples, and on numerica…

Exogenous state variables and rewards can slow down reinforcement learning by injecting uncontrolled variation into the reward signal. We formalize exogenous state variables and rewards and identify conditions under which an MDP with exogenous state can be decomposed into an exogenous Markov Reward Process involving on…

Generative framework learns effective, lower-dimensional models from high-dimensional data.

A framework learns multiscale dynamics from single trajectories using normalizing flows.

Variable selection for models including interactions between explanatory variables often needs to obey certain hierarchical constraints. The weak or strong structural hierarchy requires that the existence of an interaction term implies at least one or both associated main effects to be present in the model. Lately, thi…

LSS learns molecular trajectories from MD data.

This work interprets SFA through variational inference, relaxing linearity constraints.

The year 2017 saw the rise and fall of the crypto-currency market, followed by high variability in the price of all crypto-currencies. In this work, we study the abrupt transition in crypto-currency residuals, which is associated with the critical transition (the phenomenon of critical slowing down) or the stochastic t…

Policy gradient methods ignore the potential value of adjusting environment variables: unobservable state features that are randomly determined by the environment in a physical setting, but are controllable in a simulator. This can lead to slow learning, or convergence to suboptimal policies, if the environment variabl…

During this last decades, several attempts to construct slow invariant manifold of the Lorenz-Krishnamurthy five-mode model of slow-fast interactions in the atmosphere have been made by various authors. Unfortunately, as in the case of many two-time scales singularly perturbed dynamical systems the various asymptotic p…

We develop a 2D travel time tomography method which regularizes the inversion by modeling groups of slowness pixels from discrete slowness maps, called patches, as sparse linear combinations of atoms from a dictionary. We propose to use dictionary learning during the inversion to adapt dictionaries to specific slowness…

The continuous dynamical system approach to deep learning is explored in order to devise alternative frameworks for training algorithms. Training is recast as a control problem and this allows us to formulate necessary optimality conditions in continuous time using the Pontryagin's maximum principle (PMP). A modificati…

Due to the phenomenon of "posterior collapse," current latent variable generative models pose a challenging design choice that either weakens the capacity of the decoder or requires augmenting the objective so it does not only maximize the likelihood of the data. In this paper, we propose an alternative that utilizes t…

Study homogenizes equations on parallelizable manifolds using tensor localization and periodicity.

We propose Power Slow Feature Analysis, a gradient-based method to extract temporally slow features from a high-dimensional input stream that varies on a faster time-scale, as a variant of Slow Feature Analysis (SFA) that allows end-to-end training of arbitrary differentiable architectures and thereby significantly ext…

Time-lagged autoencoders (TAEs) have been proposed as a deep learning regression-based approach to the discovery of slow modes in dynamical systems. However, a rigorous analysis of nonlinear TAEs remains lacking. In this work, we discuss the capabilities and limitations of TAEs through both theoretical and numerical an…

Some model reduction techniques for multiple time-scale dynamical systems make use of the identification of low dimensional slow invariant attracting manifolds (SIAM) in order to reduce the dimensionality of the phase space by restriction to the slow flow. The focus of this work is on a proposition and discussion of a …

Derives a biologically plausible neural network for Slow Feature Analysis.

Langevin autoencoders improve deep latent variable models with efficient posterior sampling.

Advances in asynchronous optimization methods for machine learning.

Nonparametric mixture models based on the Dirichlet process are an elegant alternative to finite models when the number of underlying components is unknown, but inference in such models can be slow. Existing attempts to parallelize inference in such models have relied on introducing approximations, which can lead to in…

Estimates and infers multi-stage stationary treatment policies with variable selection.

Study the averaging principle for non-autonomous slow-fast systems and apply it to financial local stochastic volatility models.

Quasiclassical generalized Weierstrass representation for highly corrugated surfaces with slow modulation in the three-dimensional space is proposed. Integrable deformations of such surfaces are described by the dispersionless Veselov-Novikov hierarchy.

New method learns collective variables using autoencoders for molecular simulations.

The paper studies invariant complex manifolds in holomorphic slow-fast systems.

A single slow-growing tree matches Random Forest's performance.

Deep neural networks learn by averaging fast variables, revealing a Gaussian process.

Reducing volatility proxy improves apparent market correlation dynamics.

PAN+SR tackles scalable symbolic regression for large $p$ datasets.

Researchers reconstruct stiffness tensors from limited data in anisotropic elasticity.

This work learns effective dynamics from short-term data of stochastic systems.

We aim to create the highest possible quality of treatment-control matches for categorical data in the potential outcomes framework. Matching methods are heavily used in the social sciences due to their interpretability, but most matching methods do not pass basic sanity checks: they fail when irrelevant variables are …

Slow feature analysis (SFA) is a method for extracting slowly varying driving forces from quickly varying nonstationary time series. We show here that it is possible for SFA to detect a component which is even slower than the driving force itself (e.g. the envelope of a modulated sine wave). It is shown that it depends…

The paper tackles reinforcement learning with exogenous variables and rewards.