New method interprets recurrent neural networks via excitable network attractors.
problem Understanding the inner workings of recurrent neural networks.
method Excitable network attractors for mechanistic interpretation.
result Recurrent neural networks' behavior can be interpreted using excitable network attractors.
Paper analyzes coexisting hidden and self-excited attractors in an economic system.
problem Existence of coexisting hidden and self-excited attractors in economic systems.
method Integer and fractional order analysis of an economic system.
result Integer-order system exhibits multiple combinations of coexisting hidden and self-excited attractors.
Neurons and networks in the cerebral cortex must operate reliably despite multiple sources of noise. To evaluate the impact of both input and output noise, we determine the robustness of single-neuron stimulus selective responses, as well as the robustness of attractor states of networks of neurons performing memory ta…
The FitzHugh-Nagumo equation provides a simple mathematical model of cardiac tissue as an excitable medium hosting spiral wave vortices. Here we present extensive numerical simulations studying long-term dynamics of knotted vortex string solutions for all torus knots up to crossing number 11. We demonstrate that FitzHu…
Convolutional attractor nets improve image completion and super-resolution.
problem Construct coherent neural states from noisy data.
method Revisit and extend attractor networks with convolutional bipartite architecture.
result Demonstrates potential for image completion and super-resolution.
Study analyzes Echo State Network parameters for Rossler attractor dynamics.
problem Understanding the influence of network type on Echo State Network performance.
method Experimental analysis of Echo State Network parameters using Rossler attractor.
result Exploration of how network type affects Echo State Network performance.
Generative memory model avoids vanishing gradients to robustly retrieve patterns.
problem Robust retrieval of stored patterns in the presence of interference and noise.
method Training a generative distributed memory without explicitly simulating attractor dynamics, using a likelihood-based Lyapunov function.
result The model converges to correct patterns upon iterative retrieval and achieves competitive performance as a memory model and a generative model.
Learning three data points can generate all types of periodic orbits in a neural network.
problem Can learning three data points generate all types of periodic orbits in a neural network?
method Investigated a continuous one-dimensional map with period three in a random neural network in its thermodynamic limit.
result Almost all learned periods are unstable, and each network has its own characteristic attractors.
Paper refines RNN training by analyzing smoothness and attractors.
problem Exploding and vanishing gradients in RNNs.
method Refined concept of exploding gradients using cost function smoothness.
result RNNs need to learn attractors to fully use their power.
Deep neural networks can store and recall data efficiently.
problem Identifying computational mechanisms for memorization and retrieval of data.
method Training overparameterized autoencoders and sequence encoders using standard optimization methods.
result Overparameterized autoencoders and sequence encoders store and recall data efficiently as attractors.
Reverse engineered RNNs reveal line attractor dynamics for sentiment classification.
problem Understanding how recurrent neural networks solve sequential tasks like sentiment classification.
method Dynamical systems analysis to reverse engineer trained RNNs, identifying fixed points and linearized dynamics.
result Trained RNNs converge to low-dimensional line attractor dynamics, providing interpretable solutions.
Characterizes knotted toroidal sets as attractors in 3D.
problem Characterizing knotted toroidal sets as attractors in R3. method Global and local dynamical systems analysis, homeomorphisms and flows.
result Sufficient conditions for incompressible surfaces as attractors.
GRUs exhibit diverse dynamical behaviors but cannot mimic continuous attractors.
problem Understanding and predicting the dynamics of GRUs for neural data.
method Continuous time dynamical systems analysis of GRU networks.
result GRUs can represent stable limit cycles, multi-stable dynamics, and homoclinic bifurcations but not continuous attractors.
Persistency of excitation ensures correct parameter estimation in neural networks.
problem Ensuring correct parameter estimation in neural networks during training.
method Analyzed gradient descent dynamics in a two-layer neural network and proposed a new algorithm.
result Conditions for persistent excitation of network weights are difficult to satisfy in multi-layer networks.
The paper is focused on the existence problem of attractors for foliations. Since the existence of an attractor is a transversal property of the foliation, it is natural to consider foliations admitting transversal geometric structures. As transversal structures are chosen Cartan geometries due to their universality. T…
Develops machine learning models for excited states of CH2NH2+.
problem Accurately predicting excited-state properties and couplings for CH2NH2+.
method Combines neural networks and kernel ridge regression, encoding electronic states in inputs.
result Improved accuracy in predicting excited-state properties and couplings.
This paper classifies expanding attractors and non-transitive Anosov flows on specific knot and manifold spaces.
problem Classifying expanding attractors and non-transitive Anosov flows on specific knot and manifold spaces.
method Using the derived Anosov (DA) expanding attractor and the Franks-Williams manifold, the paper proves the uniqueness of these structures.
result The DA expanding attractor and the Franks-Williams non-transitive Anosov flow are the unique structures supported by N0 and M0 respectively. Neural networks can model chaos efficiently by becoming geometrically chaotic.
problem Lack of theoretical understanding of how neural networks learn chaos.
method Employed a geometric perspective to show neural networks can model chaotic dynamics.
result Neural networks can reconstruct strange attractors and accurately predict local divergence rates.
Paper tackles incremental few-shot learning with novel classes.
problem Learning new classes with limited data and without re-training.
method Attention Attractor Network (AAN) for incremental few-shot learning.
result AAN helps recognize new classes without forgetting old ones.
Deep learning methods improve overlapping speaker separation across languages and noise.
problem Overlapping speaker separation in realistic scenarios.
method Deep clustering and deep attractor networks.
result Deep learning methods are effective for a broad range of languages and can handle untrained languages with common features.
Multiplex Network Hawkes model for systemic risk measurement
problem Investigate how contagion in financial networks is affected by different transmission channels
method Multiplex Network Hawkes model
result Sparse contagion pathways, with systemic-risk transmission concentrated in outward flows from a small number of influential institutions
Deep QMC method accurately computes electronic excited states.
problem Accurate calculation of electronic excited states in large systems.
method Extends variational QMC with deep neural networks for excited states.
result Consistently achieves high accuracy for low-lying excited states.
We introduce a mathematical model on the dynamics of demand and supply incorporating collectability and saturation factors. Our analysis shows that when the fluctuation of the determinants of demand and supply is strong enough, there is chaos in the demand-supply dynamics. Our numerical simulation shows that such a cha…
The paper characterizes toroidal sets as attractors for flows and homeomorphisms of R^3.
problem Characterizing toroidal sets as attractors for flows and homeomorphisms in R^3.
method Defined self-geometric index and genus to analyze toroidal sets and their attractor properties.
result Characterized toroidal sets that can be realized as attractors for flows but not for homeomorphisms.
The paper studies dimensions of attractors for modified Leray-alpha equation on various surfaces.
problem Investigate attractor dimensions of the modified Leray-alpha equation.
method Existence and uniqueness of weak solutions, global attractor existence, estimates for vorticity scalar equations, Kolmogorov flows.
result Established upper and lower bounds for Hausdorff and fractal dimensions of global attractors on S2 and T2. 3D ConvNets improved with Project & Excite for medical imaging segmentation.
problem Improving segmentation performance in 3D medical imaging.
method Proposed Project & Excite (PE) modules for 3D F-CNNs, extending 2D recalibration methods.
result Project & Excite modules boost segmentation performance up to 0.3 in Dice Score.
Study bounds topological entropy of toroidal attractors.
problem Bounding entropy of toroidal attractors.
method Analyzing topological properties of toroidal sets to bound entropy.
result Entropy of toroidal attractors is bounded from below.
We prove a theorem on structural stability of smooth attractor-repellor endomorphisms of compact manifolds, with singularities. By attractor-repellor, we mean that the non-wandering set of the dynamics f is the disjoint union of a repulsive compact subset with a hyperbolic attractor on which f acts bijectively. The…
The paper shows knots and solenoids that cannot be attractors of self-homeomorphisms of R^3.
problem Deciding when a continuum can be an attractor for a homeomorphism of R^3.
method Introducing toroidal sets and defining their genus, then using these tools to find counterexamples.
result Knots and solenoids exist that cannot be attractors of self-homeomorphisms of R^3.
If there exists a diffeomorphism f on a closed, orientable n-manifold M such that the non-wandering set Ω(f) consists of finitely many orientable (±) attractors derived from expanding maps, then M must be a rational homology sphere; moreover all those attractors are of topological dimension n−2. Expandi…
MEG models for dynamic networks estimate dependencies and shared latent space relationships.
problem Modeling dynamic networks with shared latent space relationships and dependencies.
method MEG combines mutually exciting point processes and latent space models to estimate node-specific parameters and unobserved edges.
result MEG models can estimate intensities for unobserved edges, useful for anomaly detection in real-world applications.
Meta-learning approach for fast and compressive energy-based memory models.
problem Learning associative memory with fast and compressive models for complex data.
method Meta-learning approach to energy-based memory models (EBMM) using arbitrary neural architectures.
result Demonstrated associative retrieval outperforming existing systems in reconstruction error and compression rate.
This study examines how neural network architecture parameters affect loss surface modality.
problem Understanding the relationship between neural architecture parameters and loss surface modality.
method Fitness landscape analysis of neural network loss surfaces under various architecture settings.
result An increase in problem dimensionality, hidden layer width, and architecture depth affects the modality of loss surfaces.
The article contains a construction of a self-similar dendryte which cannot be the attractor of any self-similar zipper.
New method improves neural network robustness to adversarial attacks.
problem Improving adversarial robustness of neural networks.
method Inspired by adaptive control theory, the approach uses persistency of excitation to constrain gradient descent updates.
result Networks trained with the PoE-motivated learning rate schedule are significantly more robust to adversarial attacks.
ERDMD discovers sparse, nonuniformly timed DMD models from chaotic attractors.
problem Discovering high-fidelity, nonuniformly timed DMD models from chaotic data.
method Entropic regression for nonlinear information flow detection, combined with multi-step DMD.
result ERDMD produces highly efficient and robust models with minimal complexity.
Consider observing a collection of discrete events within a network that reflect how network nodes influence one another. Such data are common in spike trains recorded from biological neural networks, interactions within a social network, and a variety of other settings. Data of this form may be modeled as self-excitin…
In this paper we consider a mean-field model of interacting diffusions for the monetary reserves in which the reserves are subjected to a self- and cross-exciting shock. This is motivated by the financial acceleration and fire sales observed in the market. We derive a mean-field limit using a weak convergence analysis …
This study improves text-to-speech synthesis using GANs for glottal excitation.
problem Slow inference and computational cost of WaveNet and difficulty in parallel training of GANs.
method Adopted GANs for parallel waveform generation in speech signal and glottal excitation.
result GAN-based glottal excitation model achieves quality and voice similarity on par with WaveNet.
A new method separates instruments in music using shared embeddings.
problem Separating individual instruments in a musical mixture.
method Common embedding space for all instruments, auxiliary Gaussian mixture model.
result Outperforms mask-inference baseline on MUSDB-18 dataset.
New method reconstructs hidden dynamics from low-dimensional time series.
problem Reconstructing hidden dynamics from limited experimental data.
method Autoencoder trained with a novel latent-space loss function.
result Reconstructs strange attractors better than existing techniques.
Study the evolution of the Lorenz strange set using Conley index theory.
problem Understanding the evolution of the Lorenz strange set through parameter changes.
method Application of Conley index theory to analyze the global attractor and its Morse decompositions.
result Identification and analysis of bifurcations and the role of the strange set in these transformations.
ESNs with transfer learning predict long-term chaotic patterns in spatiotemporal dynamical systems.
problem Predicting long-term statistical patterns of spatiotemporally chaotic dynamical systems.
method Echo state networks (ESNs) with transfer learning.
result ESNs with transfer learning accurately predict long-term statistical properties of spatiotemporally chaotic PDEs.
New risk models use chaotic attractors to predict extreme events.
problem Predicting Black Swan events in financial markets.
method Combining heavy-tailed priors with chaotic dynamics (Lorenz and Rossler systems).
result Models generate volatility clustering, fat tails, and extreme events.
This paper studies closed 3-manifolds which are the attractors of a system of finitely many affine contractions that tile R3. Such attractors are called self-affine tiles. Effective characterization and recognition theorems for these 3-manifolds as well as theoretical generalizations of these results to hig…
New theory explains how chaotic training improves neural network generalization.
problem Understanding how chaotic training improves neural network generalization.
method Representing stochastic optimizers as random dynamical systems and introducing a new dimension concept.
result Generalization in chaotic training depends on the complete Hessian spectrum and partial determinants.
In this paper we consider the realization of DE attractors by self-diffeomorphisms of manifolds. For any expanding self-map φ:M→M of a connected, closed p-dimensional manifold M, one can always realize a (p,q)-type attractor derived from φ by a compactly-supported self-diffeomorphsm of $\RR^{p+q}$, as long…
A new channel locality block improves CNN performance.
problem Improving the performance of convolutional neural networks.
method Proposed a variant of Squeeze-and-Excitation block using convolutional layers to learn nearby channel correlation.
result Our C-Local block achieved higher accuracy than the standard SE block on the cifar-10 dataset.