New empirical index reveals wider validity of Echo State Property in input-driven reservoirs.
problem Lack of proper input consideration in Echo State Property conditions.
method Introduced an empirical Echo State Property index to analyze stability of reservoirs with input signals.
result The actual domain of Echo State Property validity is wider than literature conditions suggest.
State-space systems generate probabilistic dependencies between inputs and outputs.
problem Understanding probabilistic dependencies in state-space systems.
method Introducing a probabilistic framework and proving sufficient conditions for output existence and uniqueness.
result State-space systems can generate probabilistic dependencies, even without functional relations.
Unified treatment of RC in stochastic and deterministic settings.
problem Understanding and generalizing reservoir computing in both deterministic and stochastic contexts.
method Investigation of state-space systems, analysis of fading memory and solution stability, introduction of stochastic echo states.
result Generality of fading memory and solution stability in state-space systems, even without the echo state property.
This research unifies concepts of fading memory in RNNs.
problem Unclear relationships between fading memory concepts in RNNs.
method Unified language and new proofs for fading memory concepts.
result Clarified relationships between fading memory concepts.
New definitions of ESP for quantum reservoir computing handle non-stationary systems.
problem Traditional ESP does not apply to non-stationary systems.
method Introduce two new categories of ESP: non-stationary ESP and subset/subspace ESP.
result Demonstrates correspondence between non-stationary ESP and QRC with NARMA tasks.
The paper proposes count echo state networks for forecasting graduate student enrollments.
problem Forecasting graduate student enrollments from historical data.
method Developed hierarchical count echo state networks and compared them to Poisson autoregressions and negative binomial models.
result Hierarchical negative binomial based echo state network is the superior model.
Echo state networks with random weights can approximate any continuous system.
problem Approximating continuous dynamical systems using echo state networks.
method Randomly generated internal weights and a sampling procedure for activation functions.
result Echo state networks with random weights can approximate any continuous casual time-invariant operators with high probability.
Neuro-inspired recurrent neural network algorithms, such as echo state networks, are computationally lightweight and thereby map well onto untethered devices. The baseline echo state network algorithms are shown to be efficient in solving small-scale spatio-temporal problems. However, they underperform for complex task…
Study analyzes Echo State Network parameters for Rossler attractor dynamics.
problem Understanding the influence of network type on Echo State Network performance.
method Experimental analysis of Echo State Network parameters using Rossler attractor.
result Exploration of how network type affects Echo State Network performance.
The study of deep recurrent neural networks (RNNs) and, in particular, of deep Reservoir Computing (RC) is gaining an increasing research attention in the neural networks community. The recently introduced Deep Echo State Network (DeepESN) model opened the way to an extremely efficient approach for designing deep neura…
Efficient echo state network with explicit memory performs well on benchmark tasks.
problem Training differentiable neural computers is difficult and time-consuming.
method Echo state network with an explicit memory.
result Echo state network can recognize all regular languages, including those contractive networks cannot.
Study on deep and wide echo state networks for forecasting complex time series.
problem Performance analysis of deep reservoir computing models.
method Investigates the impact of partitioning neurons and parallel pathways on forecasting accuracy.
result Wide and deep networks outperform shallow models in forecasting multiscale spatiotemporal data.
ESNs with transfer learning predict long-term chaotic patterns in spatiotemporal dynamical systems.
problem Predicting long-term statistical patterns of spatiotemporally chaotic dynamical systems.
method Echo state networks (ESNs) with transfer learning.
result ESNs with transfer learning accurately predict long-term statistical properties of spatiotemporally chaotic PDEs.
ESNs trained with Tikhonov least squares approximate ergodic dynamical systems in L2(μ) norm.
problem Approximating ergodic dynamical systems using ESNs.
method Tikhonov least squares regression on ESNs trained on observations from an ergodic dynamical system.
result ESNs trained with Tikhonov least squares approximate the target function in the L2(μ) norm.
Study on state dynamics in Deep Echo State Networks, revealing the importance of inter-reservoir connections.
problem Understanding state dynamics in multi-layered RNNs.
method Tools from information theory and numerical analysis.
result Inter-reservoir connections enrich representations in higher layers of DeepESNs.
This work introduces a tensor-based method to perform supervised classification on spatiotemporal data processed in an echo state network. Typically when performing supervised classification tasks on data processed in an echo state network, the entire collection of hidden layer node states from the training dataset is …
Ensemble of MFESNs improves macroeconomic forecasting.
problem Improving accuracy in macroeconomic forecasting.
method Hedge and Follow-the-Leader schemes applied to MFESNs.
result Ensemble models outperform individual models in forecasting.
Echo state networks are powerful recurrent neural networks. However, they are often unstable and shaky, making the process of finding an good ESN for a specific dataset quite hard. Obtaining a superb accuracy by using the Echo State Network is a challenging task. We create, develop and implement a family of predictably…
Many neural networks use the tanh activation function, however when given a probability distribution as input, the problem of computing the output distribution in neural networks with tanh activation has not yet been addressed. One important example is the initialization of the echo state network in reservoir computing…
CESAR improves wind speed and power forecasting for high-resolution simulations.
problem Accurate high-resolution wind forecasting for efficient power grid management.
method A spatio-temporal neural network model using deep convolutional autoencoder and echo state network.
result CESAR provides up to 17% improvement in wind speed and power forecasting compared to best alternatives.
New technique halves scan time for multi-echo MR images.
problem Slow acquisition of multi-echo magnetic resonance images.
method Structured deep dictionary learning for adaptive reconstruction.
result Scan time reduced by half compared to state-of-the-art.
Efficient cross-validation method for Echo State Networks reduces validation time complexity.
problem Hyper-parameter tuning and validation for Echo State Networks (ESNs).
method Proposes several k-fold cross-validation schemes for ESNs with an efficient algorithm. result Cross-validation of ESNs can be done for the same time complexity as a single split validation.
Study shows structured reservoirs improve deep ESN performance.
problem Improving performance of deep reservoir computing networks.
method Investigated structured reservoir topologies in deep ESNs.
result Structured reservoirs significantly enhance predictive performance.
ESN models predict intraday stock returns efficiently.
problem Intraday stock return prediction using machine learning.
method Echo State Network (ESN) models with random parameters.
result ESN models achieve strong forecasting performance efficiently.
New reservoir computing approach handles infinite-dimensional systems.
problem Approximating and generalizing complex input/output systems.
method Randomly generated echo state networks with neural networks.
result Proves universal approximation properties for new class of systems.
There is a large amount of interest in understanding users of social media in order to predict their behavior in this space. Despite this interest, user predictability in social media is not well-understood. To examine this question, we consider a network of fifteen thousand users on Twitter over a seven week period. W…
Optical ESNs enable flexible, efficient machine learning with reduced energy.
problem Implementing universal computational capabilities in machine learning.
method Optical implementation of ESNs leveraging stimulated Brillouin scattering.
result Efficient, scalable, and memory-capable optical reservoir computing.
Physics-informed ESNs improve chaotic system prediction accuracy.
problem Predicting chaotic systems while adhering to physical laws.
method Integrates physics constraints into ESN training through an additional loss function.
result Physics-informed ESNs predict chaotic systems with a 2 Lyapunov time improvement.
New approach speeds up optimization with repeated gradient steps on same batch.
problem Performance bottlenecks in massive parallel pipelines with large batch sizes.
method Data echoing, taking repeated gradient steps on the same batch.
result Data echoing affords speedups on curvature-dominated part of convergence rate.
A new method suppresses echo more effectively with lower latency.
problem Echo cannot be fully removed by linear filters due to nonlinear relationships.
method Uses a modified Conv-TasNet with multiple streams of input signals.
result Efficacy validated in both single-talk and double-talk situations.
New analysis reveals optimal regularization for ESNs, avoiding double descent.
problem Characterizing and optimizing Echo State Networks (ESNs) for precise bias-variance.
method Random matrix theory applied to ESNs in a teacher-student setting.
result ESNs achieve lower MSE with limited training samples and teacher memory.
Spatially aware ESN detects anomalies in chaotic time series.
problem Automated anomaly detection in chaotic time series, especially turbulent ocean simulations.
method Extended Echo State Network with spatially aware input maps and loss function.
result Spatial ESN reduces anomaly detection to thresholding of prediction error.
The memory capacity of linear echo state networks is accurately calculated using new numerical methods.
problem Numerical evaluations of memory capacity in recurrent neural networks often contradict theoretical bounds.
method Developed robust numerical approaches exploiting MC neutrality with respect to the input mask matrix.
result Memory curves fully agree with theory when using the proposed methods.
Echo state network (ESN) is viewed as a temporal non-orthogonal expansion with pseudo-random parameters. Such expansions naturally give rise to regressors of various relevance to a teacher output. We illustrate that often only a certain amount of the generated echo-regressors effectively explain the variance of the tea…
Reservoir Memory Machines solve benchmark tasks faster than Neural Turing Machines.
problem Training Neural Turing Machines is hard and limits their applicability.
method Proposes Reservoir Memory Machines, combining neural network flexibility with Turing machine capabilities, but with faster training via alignment and linear regression.
result Reservoir Memory Machines solve benchmark tasks as well as Neural Turing Machines but are much faster to train.
Quantum reservoir computing needs coherence influx for effective information processing.
problem Understanding and optimizing quantum reservoir computing.
method Theoretical and numerical analysis of quantum systems, focusing on coherence influx and spectral radius of Pauli transfer matrix.
result Coherence influx is essential for realizing nonstationary echo state property in quantum reservoir computing.
Discrete return (DR) Laser Detection and Ranging (Ladar) systems provide a series of echoes that reflect from objects in a scene. These can be first, last or multi-echo returns. In contrast, Full-Waveform (FW)-Ladar systems measure the intensity of light reflected from objects continuously over a period of time. In a c…
GANs improve echo frame generation with labeled patches.
problem Automated echocardiography with limited labels.
method Conditional GAN with patch-based discriminator.
result GAN-enhanced echo frames match given segmentation masks.
Study converts echocardiography views using adversarial models.
problem Limited information sharing among echocardiography views.
method Adversarial generative models trained with LSGAN, PatchGAN, and cGAN.
result Generated views correlate with real views in LV area and length.
Feedback improves ESN performance by 30-60% across various tasks.
problem Reducing computational complexity in ESNs for complex sequential data processing.
method State feedback to modify the internal reservoir state of ESNs.
result Feedback significantly improves ESN performance, reducing error by 30-60%.
We define polynomial tangle invariants ∇Ts via Kauffman states and Alexander codes and investigate some of their properties. In particular, we prove symmetry relations for ∇Ts of 4-ended tangles and deduce that the multivariable Alexander polynomial is invariant under Conway mutation. The invariants $…
Echo noise improves compression in autoencoders with exact rate-distortion.
problem Limitations of Gaussian noise in lossy compression and VAEs.
method Introduces Echo noise, a data-driven noise channel with exact mutual information.
result Echo noise leads to improved bounds on log-likelihood and dominates VAEs.
Jointly reduces echo, reverberation, and noise using neural networks.
problem Simultaneous reduction of acoustic echo, reverberation, and noise.
method Multichannel Gaussian modeling and neural network for joint optimization of filters.
result Outperforms individual and joint non-spectral models.
Proposes Textual Echo Cancellation to improve speech recognition.
problem Improving speech recognition performance and user experience for smart devices.
method A novel sequence-to-sequence model with multi-source attention that processes both the microphone mixture signal and source text of TTS playback.
result Demonstrates enhanced speech recognition performance and reduced latency.
Optimizes ESN hyperparameters for time series datasets using Bayesian optimization.
problem Determining optimal ESN hyperparameters for efficient model training.
method Bayesian optimization to find hyperparameters that generalize across similar time series.
result Reduces the number of ESN models needed for a dataset while maintaining good performance.
DESS MRI and kernel learning enable precise myelin water quantification.
problem Quantifying myelin water content in the brain.
method Optimized DESS scans combined with kernel learning for precise estimation of myelin water fraction.
result DESS PERK ff estimates are quantitatively similar to conventional MESE MWF estimates.
Reinforcement learning crypto agent achieves high returns on Bitcoin derivatives.
problem Maximizing returns on volatile cryptocurrency markets.
method Online transfer learning with an echo state network and recurrent reinforcement learning.
result Achieves a total return of 350%, net of transaction costs, over five years.
ESN model helps understand climate event impacts.
problem Understanding complex climate event impacts.
method Feature importance methods for ESNs on spatio-temporal climate data.
result Characterized relationships between Mount Pinatubo eruption variables.