Reservoir Memory Machines solve benchmark tasks faster than Neural Turing Machines.
problem Training Neural Turing Machines is hard and limits their applicability.
method Proposes Reservoir Memory Machines, combining neural network flexibility with Turing machine capabilities, but with faster training via alignment and linear regression.
result Reservoir Memory Machines solve benchmark tasks as well as Neural Turing Machines but are much faster to train.
Study characterizes memory capacity of quantum reservoirs using transmon qubits.
problem Understanding the memory capacity of quantum reservoirs built with transmon qubits.
method Characterized memory capacity of quantum reservoirs using transmon qubits from IBM, focusing on NMSE and topology complexity.
result Found a peak in memory capacity for configurations with n-1 self-loops, suggesting optimal design for forecasting tasks.
Next-gen reservoir computers fail to predict complex processes, highlighting need for better architectures.
problem Predicting complex, non-Markovian processes with recurrent neural networks.
method Lower bound from Fano's inequality and analysis of large probabilistic state machines.
result Next-generation reservoir computers have an error probability at least 60% higher than optimal for highly non-Markovian processes.
New method combines long-memory reservoirs for accurate dengue forecasting from short data.
problem Accurate dengue forecasting from short, noisy, non-stationary, and nonlinear data.
method Fractional ESN and Wavelet ESN frameworks integrating long-term memory.
result fESN and wESN outperform baselines in multiple dengue datasets and forecasting horizons.
Optical ESNs enable flexible, efficient machine learning with reduced energy.
problem Implementing universal computational capabilities in machine learning.
method Optical implementation of ESNs leveraging stimulated Brillouin scattering.
result Efficient, scalable, and memory-capable optical reservoir computing.
This paper provides a mathematical framework for time-delay reservoir computing.
problem Lack of rigorous mathematical foundations for reservoir computing properties.
method Control-theoretic framework, formal definitions of separation and fading memory, explicit lower bound derivation.
result Established formal definitions and connections to stability notions for time-delay systems.
Quantum systems with scrambling improve temporal information processing, but scaling requires exponential overhead.
problem Scalability and memory retention of quantum reservoirs in temporal information processing.
method Examined a quantum reservoir processing framework with scrambling reservoirs modeled by high-order unitary designs, analyzed in noiseless and noisy settings.
result Memory retention improves exponentially with reservoir size but worsens with reservoir iterations, requiring exponential shot overhead for scaling.
Quantum reservoir computing improves volatility forecasting.
problem Forecasting realized volatility in finance.
method Quantum reservoir computing with Ising Hamiltonian and feature selection.
result Quantum reservoir computing outperforms benchmarks in volatility forecasting.
New explanation of reservoir computing using random projections.
problem Understanding the randomness in reservoir computing.
method Constructing strongly universal reservoir systems as random projections of state-space systems.
result Approximation of any fading memory filters class by training a linear readout for each filter.
The paper analyzes the performance of delay-based reservoir computing using eigenvalue analysis.
problem Quantifying the performance of delay-based reservoir computing.
method Eigenvalue analysis of the dynamical system to predict reservoir computing performance.
result The performance of a reservoir computing system can be predicted by analyzing the small signal response and eigenvalue spectrum.
This paper investigates the role of sparsity in Reservoir Computing networks.
problem Designing efficient Recurrent Neural Networks (RNNs) with hidden recurrent layers.
method Empirical investigation of sparsity in input-reservoir connections and recurrent connections.
result Sparsity, particularly in input-reservoir connections, enhances the network's temporal memory and dimensionality.
This work tackles online memory selection in continual learning using information theory.
problem Online selection of a representative replay memory from data streams.
method Information-theoretic criteria (surprise, learnability) and Bayesian model for efficient computation.
result InfoRS improves robustness against data imbalance compared to reservoir sampling.
New definitions of ESP for quantum reservoir computing handle non-stationary systems.
problem Traditional ESP does not apply to non-stationary systems.
method Introduce two new categories of ESP: non-stationary ESP and subset/subspace ESP.
result Demonstrates correspondence between non-stationary ESP and QRC with NARMA tasks.
Model combines long-term and short-term memory using conceptors.
problem Transfer between long-term and short-term memory.
method Recurrent neural network with gated reservoir for short-term memory and conceptors for long-term memory.
result Standard operations on conceptors allow combining long-term memories and describing their effect on short-term memory.
The combination of machine learning and quantum computing has emerged as a promising approach for addressing previously untenable problems. Reservoir computing is an efficient learning paradigm that utilizes nonlinear dynamical systems for temporal information processing, i.e., processing of input sequences to produce …
BNNs enhance reservoir computing by acting as generalization filters.
problem Understanding how BNNs integrate with reservoir computing.
method Optogenetics and calcium imaging to record BNNs, reservoir computing framework.
result BNNs improve reservoir computing performance through generalization.
New methods improve Reservoir Computing for chaotic time series prediction.
problem Chaotic time series prediction in Reservoir Computing.
method Established Recurrent Kernel limit, introduced Structured Reservoir Computing.
result Structured Reservoir Computing is faster and more memory-efficient.
EuSN uses Euler discretization for stable, non-dissipative reservoir computing.
problem Designing stable and efficient reservoir computing models.
method Forward Euler discretization and antisymmetric recurrent matrices.
result EuSN outperforms standard RC models in long-term memory tasks and time-series classification.
The extension of deep learning towards temporal data processing is gaining an increasing research interest. In this paper we investigate the properties of state dynamics developed in successive levels of deep recurrent neural networks (RNNs) in terms of short-term memory abilities. Our results reveal interesting insigh…
Unified framework for blending ML and mechanistic models in dynamical systems.
problem Learning dynamical systems from noisy, partially observed data.
method A unifying framework that combines mechanistic and machine learning approaches.
result Proves that hybrid models can learn memory-dependent model error.
Reservoir computing is a neural network approach for processing time-dependent signals that has seen rapid development in recent years. Physical implementations of the technique using optical reservoirs have demonstrated remarkable accuracy and processing speed at benchmark tasks. However, these approaches require an e…
Stochastic reservoir computing is shown to be a universal approximator.
problem Theoretical justification for using stochastic reservoirs in machine learning.
method Investigated stochastic reservoir computing using probabilities of reservoir states as readout.
result Stochastic reservoir computers are universal approximating classes.
Parameterized state space models in the form of recurrent networks are often used in machine learning to learn from data streams exhibiting temporal dependencies. To break the black box nature of such models it is important to understand the dynamical features of the input driving time series that are formed in the sta…
Episodic memory is a psychology term which refers to the ability to recall specific events from the past. We suggest one advantage of this particular type of memory is the ability to easily assign credit to a specific state when remembered information is found to be useful. Inspired by this idea, and the increasing pop…
Reservoir Computing (RC) refers to a Recurrent Neural Networks (RNNs) framework, frequently used for sequence learning and time series prediction. The RC system consists of a random fixed-weight RNN (the input-hidden reservoir layer) and a classifier (the hidden-output readout layer). Here we focus on the sequence lear…
Quantum reservoir computing needs coherence influx for effective information processing.
problem Understanding and optimizing quantum reservoir computing.
method Theoretical and numerical analysis of quantum systems, focusing on coherence influx and spectral radius of Pauli transfer matrix.
result Coherence influx is essential for realizing nonstationary echo state property in quantum reservoir computing.
Efficient echo state network with explicit memory performs well on benchmark tasks.
problem Training differentiable neural computers is difficult and time-consuming.
method Echo state network with an explicit memory.
result Echo state network can recognize all regular languages, including those contractive networks cannot.
Quantum ELMs use a quantum reservoir to learn from data, with limits on expressivity and scalability.
problem Understanding the limits of quantum ELMs for machine learning tasks.
method Decomposed QELM predictions into Fourier series to analyze expressivity and scalability.
result Expressivity of QELMs is limited by the number of Fourier frequencies and observables, and scalability is hindered by hardware noise and entanglement.
Improved machine learning for reservoir optimization problems.
problem Optimizing control in high-dimensional storage problems.
method Modified dynamic programming algorithm with neural networks for Bellman values and conditional cuts.
result Neural networks outperform classical feedforward networks in estimating Bellman values.
IMeL turns RL into SL by interpolating improved experiences.
problem Improving reinforcement learning efficiency and scalability.
method IMeL uses a reservoir of experiences and a NN regressor for interpolation.
result IMeL achieves preliminary results and proposes itself as a baseline.
Machine learning predicts dam-break flood wave behavior accurately.
problem Predicting long-term wave behavior in dam-break floods.
method Solved Saint-Venant equations using Lax-Wendroff scheme, trained RC-ESN with flow depth data.
result RC-ESN model predicts 286 time-steps ahead with RMSE < 0.01, outperforming LSTM.
Unified treatment of RC in stochastic and deterministic settings.
problem Understanding and generalizing reservoir computing in both deterministic and stochastic contexts.
method Investigation of state-space systems, analysis of fading memory and solution stability, introduction of stochastic echo states.
result Generality of fading memory and solution stability in state-space systems, even without the echo state property.
In this paper, the echo state network (ESN) memory capacity, which represents the amount of input data an ESN can store, is analyzed for a new type of deep ESNs. In particular, two deep ESN architectures are studied. First, a parallel deep ESN is proposed in which multiple reservoirs are connected in parallel allowing …
Reduced reservoir size for faster edge computing.
problem Efficiently reducing computational resources for reservoir computing.
method Concatenating past or drifting states of the reservoir to the output layer.
result Reduced reservoir size up to one tenth without significant error increase.
Ring-reservoir networks simplify graph embeddings efficiently.
problem Efficient graph embeddings using deep neural networks.
method Progressive simplification of Reservoir Computing models to ring topology.
result Ring-reservoir networks show consistent advantages in predictive performance.
"Sparse" neural networks, in which relatively few neurons or connections are active, are common in both machine learning and neuroscience. Whereas in machine learning, "sparsity" is related to a penalty term that leads to some connecting weights becoming small or zero, in biological brains, sparsity is often created wh…
Stabilizes online learning by using weighted reservoir sampling.
problem Real-world deployment sensitivity to outliers causes low accuracy in final solutions.
method Weighted reservoir sampling to stabilize ensemble model without additional data passes.
result Risk of ensemble classifier is bounded with respect to the underlying online learning method's regret.
New reservoir computing approach handles infinite-dimensional systems.
problem Approximating and generalizing complex input/output systems.
method Randomly generated echo state networks with neural networks.
result Proves universal approximation properties for new class of systems.
Predicting unobserved bifurcations in time series with unsupervised parameter extraction.
problem Predicting system behavior with unknown parameters from time series data.
method Reservoir computing framework for unsupervised extraction of slowly varying system parameters.
result Model predicts unknown bifurcations not present in training data.
Study forecasts food security trends using real-time data.
problem Food insecurity prediction for sub-national regions.
method Quantitative methodology combining various machine learning models.
result Reservoir Computing model performs best in food security prediction.
We explore the hyperparameter space of reservoir computers used for forecasting of the chaotic Lorenz '63 attractor with Bayesian optimization. We use a new measure of reservoir performance, designed to emphasize learning the global climate of the forecasted system rather than short-term prediction. We find that optimi…
Reservoir computing's success depends on mapping different input time series to separable states.
problem Quantifying the ability of random linear reservoirs to map different input time series.
method Mathematical framework using spectral properties of the connectivity matrix.
result Separation capacity is fully characterized by the spectral properties of the connectivity matrix.
With recent progress in algorithms and the availability of massive amounts of computation power, application of machine learning techniques is becoming a hot topic in the oil and gas industry. One of the most promising aspects to apply machine learning to the upstream field is the rock facies classification in reservoi…
A reservoir computer is a complex dynamical system, often created by coupling nonlinear nodes in a network. The nodes are all driven by a common driving signal. In this work, three dimension estimation methods, false nearest neighbor, covariance and Kaplan-Yorke dimensions, are used to estimate the dimension of the res…
New insights on stability in reservoir computing for better performance.
problem Stability in reservoir computing networks.
method Using the recurrent kernel limit for large reservoir sizes.
result Quantitative characterization of stability and chaos frontier.
We infer both microscopic and macroscopic behaviors of a three-dimensional chaotic fluid flow using reservoir computing. In our procedure of the inference, we assume no prior knowledge of a physical process of a fluid flow except that its behavior is complex but deterministic. We present two ways of inference of the co…
We present a framework that enables estimation of low-dimensional sub-resolution reservoir properties directly from seismic data, without requiring the solution of a high dimensional seismic inverse problem. Our workflow is based on the Bayesian evidential learning approach and exploits learning the direct relation bet…
Deep learning speeds up pressure prediction in carbon storage reservoirs.
problem Accurately forecasting reservoir pressure in geologic carbon storage projects with sparse well data.
method Combining InSAR surface displacement data with deep learning and data assimilation techniques.
result Workflow can predict reservoir pressure with high efficiency and uncertainty quantification.