arXiv research

Deep learning methods improve overlapping speaker separation across languages and noise.

In human perception and cognition, a fundamental operation that brains perform is interpretation: constructing coherent neural states from noisy, incomplete, and intrinsically ambiguous evidence. The problem of interpretation is well matched to an early and often overlooked architecture, the attractor network---a recur…

A central challenge faced by memory systems is the robust retrieval of a stored pattern in the presence of interference due to other stored patterns and noise. A theoretically well-founded solution to robust retrieval is given by attractor dynamics, which iteratively clean up patterns during recall. However, incorporat…

Study analyzes Echo State Network parameters for Rossler attractor dynamics.

Identifying computational mechanisms for memorization and retrieval of data is a long-standing problem at the intersection of machine learning and neuroscience. Our main finding is that standard overparameterized deep neural networks trained using standard optimization methods implement such a mechanism for real-valued…

Introduction: Machine learning provides fundamental tools both for scientific research and for the development of technologies with significant impact on society. It provides methods that facilitate the discovery of regularities in data and that give predictions without explicit knowledge of the rules governing a syste…

Although deep learning has shown great success in recent years, researchers have discovered a critical flaw where small, imperceptible changes in the input to the system can drastically change the output classification. These attacks are exploitable in nearly all of the existing deep learning classification frameworks.…

We study the problem of learning associative memory -- a system which is able to retrieve a remembered pattern based on its distorted or incomplete version. Attractor networks provide a sound model of associative memory: patterns are stored as attractors of the network dynamics and associative retrieval is performed by…

Learning three data points can generate all types of periodic orbits in a neural network.

Paper analyzes coexisting hidden and self-excited attractors in an economic system.

New method reconstructs hidden dynamics from low-dimensional time series.

Characterizes knotted toroidal sets as attractors in 3D.

This paper studies closed 3-manifolds which are the attractors of a system of finitely many affine contractions that tile $\mathbb{R}^3$. Such attractors are called self-affine tiles. Effective characterization and recognition theorems for these 3-manifolds as well as theoretical generalizations of these results to hig…

The paper is focused on the existence problem of attractors for foliations. Since the existence of an attractor is a transversal property of the foliation, it is natural to consider foliations admitting transversal geometric structures. As transversal structures are chosen Cartan geometries due to their universality. T…

Isolating individual instruments in a musical mixture has a myriad of potential applications, and seems imminently achievable given the levels of performance reached by recent deep learning methods. While most musical source separation techniques learn an independent model for each instrument, we propose using a common…

Machine learning classifiers are often trained to recognize a set of pre-defined classes. However, in many applications, it is often desirable to have the flexibility of learning additional concepts, with limited data and without re-training on the full training set. This paper addresses this problem, incremental few-s…

This paper classifies expanding attractors and non-transitive Anosov flows on specific knot and manifold spaces.

Neural networks can model chaos efficiently by becoming geometrically chaotic.

Unified Bayesian framework predicts cryptocurrency market dynamics and volatility.

We introduce a mathematical model on the dynamics of demand and supply incorporating collectability and saturation factors. Our analysis shows that when the fluctuation of the determinants of demand and supply is strong enough, there is chaos in the demand-supply dynamics. Our numerical simulation shows that such a cha…

Researchers analyze how RNNs solve intent detection tasks using dynamical systems theory.

The paper studies dimensions of attractors for modified Leray-alpha equation on various surfaces.

Study bounds topological entropy of toroidal attractors.

We prove a theorem on structural stability of smooth attractor-repellor endomorphisms of compact manifolds, with singularities. By attractor-repellor, we mean that the non-wandering set of the dynamics $f$ is the disjoint union of a repulsive compact subset with a hyperbolic attractor on which $f$ acts bijectively. The…

If there exists a diffeomorphism $f$ on a closed, orientable $n$-manifold $M$ such that the non-wandering set $Ω(f)$ consists of finitely many orientable $(\pm)$ attractors derived from expanding maps, then $M$ must be a rational homology sphere; moreover all those attractors are of topological dimension $n-2$. Expandi…

It has been argued in the past that high-dimensional neural networks do not exhibit local minima capable of trapping an optimisation algorithm. However, the relationship between loss surface modality and the neural architecture parameters, such as the number of hidden neurons per layer and the number of hidden layers, …

The article contains a construction of a self-similar dendryte which cannot be the attractor of any self-similar zipper.

ERDMD discovers sparse, nonuniformly timed DMD models from chaotic attractors.

A new method for analyzing high-dimensional time-series data using deep neural networks.

In this paper we focus on compacta $K \subseteq \mathbb{R}^3$ which possess a neighbourhood basis that consists of nested solid tori $T_i$. We call these sets toroidal. In \cite{hecyo1} we defined the genus of a toroidal set as a generalization of the classical notion of genus from knot theory. Here we introduce the se…

Sigmoid autoencoders can implement associative memory with certain conditions.

ESNs with transfer learning predict long-term chaotic patterns in spatiotemporal dynamical systems.

New risk models use chaotic attractors to predict extreme events.

The exploding and vanishing gradient problem has been the major conceptual principle behind most architecture and training improvements in recurrent neural networks (RNNs) during the last decade. In this paper, we argue that this principle, while powerful, might need some refinement to explain recent developments. We r…

New theory explains how chaotic training improves neural network generalization.

In this paper we consider the realization of DE attractors by self-diffeomorphisms of manifolds. For any expanding self-map $φ:M\to M$ of a connected, closed $p$-dimensional manifold $M$, one can always realize a $(p,q)$-type attractor derived from $φ$ by a compactly-supported self-diffeomorphsm of $\RR^{p+q}$, as long…

As a first step to understand how complicated attractors for dynamical systems can be, one may consider the following realizability problem: given a continuum $K \subseteq \mathbb{R}^3$, decide when $K$ can be realized as an attractor for a homeomorphism of $\mathbb{R}^3$. In this paper we introduce toroidal sets as th…

Recurrent neural networks (RNNs) are a widely used tool for modeling sequential data, yet they are often treated as inscrutable black boxes. Given a trained recurrent network, we would like to reverse engineer it--to obtain a quantitative, interpretable description of how it solves a particular task. Even for simple ta…

Motivated by the study in Morse theory and Smale's work in dynamics, the following questions are studied and answered: (1) When does a 3-manifold admit an automorphism having a knotted Smale solenoid as an attractor? (2) When does a 3-manifold admit an automorphism whose non-wandering set consists of Smale solenoids? T…

In this paper we study the Lorenz equations using the perspective of the Conley index theory. More specifically, we examine the evolution of the strange set that these equations posses throughout the different values of the parameter. We also analyze some natural Morse decompositions of the global attractor of the syst…

The loss surface of deep neural networks has recently attracted interest in the optimization and machine learning communities as a prime example of high-dimensional non-convex problem. Some insights were recently gained using spin glass models and mean-field approximations, but at the expense of strongly simplifying th…

In this paper we study the cohomological Conley index of arbitrary isolated invariant continua for continuous maps $f \colon U \subseteq \mathbb{R}^d \to \mathbb{R}^d$ by analyzing the topological structure of their unstable manifold. We provide a simple dynamical interpretation for the first cohomological Conley index…

New study shows min-max algorithms can converge to non-stationary points.

We analyze the dynamical properties of a tetrahedron transformation on the space of non-degenerate tetrahedra which can be identified with the non-compact globally symmetric $8$-dimensional space $\mbox{Sl}(3,\mathbb{R}) / \mbox{So}(3,\mathbb{R})$. We establish the existence of a local attractor which coincides with th…

The OGY method is one of control methods for a chaotic system. In the method, we have to calculate a stabilizing periodic orbit embedded in its chaotic attractor. Thus, we cannot use this method in the case where a precise mathematical model of the chaotic system cannot be identified. In this case, the delayed feedback…

An iterated function system $Φ$ consisting of contractive similarity mappings has a unique attractor $F \subseteq \mathbb{R}^d$ which is invariant under the action of the system, as was shown by Hutchinson [Hut]. This paper shows how the action of the function system naturally produces a tiling $\mathcal{T}$ of the con…

The paper studies bifurcations in discrete dynamical systems on manifolds.

Let $M$ be a manifold or (more generally) a locally compact, metrizable ANR. If $K$ is an attractor for a flow in $M$, with basin of attraction $\mathcal{A}(K)$, it is well known that the inclusion $i : K \subseteq \mathcal{A}(K)$ is always a shape equivalence. In this paper we investigate to what extent this generaliz…