Echo noise improves compression in autoencoders with exact rate-distortion.
problem Limitations of Gaussian noise in lossy compression and VAEs.
method Introduces Echo noise, a data-driven noise channel with exact mutual information.
result Echo noise leads to improved bounds on log-likelihood and dominates VAEs.
Jointly reduces echo, reverberation, and noise using neural networks.
problem Simultaneous reduction of acoustic echo, reverberation, and noise.
method Multichannel Gaussian modeling and neural network for joint optimization of filters.
result Outperforms individual and joint non-spectral models.
A new method suppresses echo more effectively with lower latency.
problem Echo cannot be fully removed by linear filters due to nonlinear relationships.
method Uses a modified Conv-TasNet with multiple streams of input signals.
result Efficacy validated in both single-talk and double-talk situations.
The paper proposes count echo state networks for forecasting graduate student enrollments.
problem Forecasting graduate student enrollments from historical data.
method Developed hierarchical count echo state networks and compared them to Poisson autoregressions and negative binomial models.
result Hierarchical negative binomial based echo state network is the superior model.
Echo state networks with random weights can approximate any continuous system.
problem Approximating continuous dynamical systems using echo state networks.
method Randomly generated internal weights and a sampling procedure for activation functions.
result Echo state networks with random weights can approximate any continuous casual time-invariant operators with high probability.
GANs improve echo frame generation with labeled patches.
problem Automated echocardiography with limited labels.
method Conditional GAN with patch-based discriminator.
result GAN-enhanced echo frames match given segmentation masks.
Study converts echocardiography views using adversarial models.
problem Limited information sharing among echocardiography views.
method Adversarial generative models trained with LSGAN, PatchGAN, and cGAN.
result Generated views correlate with real views in LV area and length.
New technique halves scan time for multi-echo MR images.
problem Slow acquisition of multi-echo magnetic resonance images.
method Structured deep dictionary learning for adaptive reconstruction.
result Scan time reduced by half compared to state-of-the-art.
Neuro-inspired recurrent neural network algorithms, such as echo state networks, are computationally lightweight and thereby map well onto untethered devices. The baseline echo state network algorithms are shown to be efficient in solving small-scale spatio-temporal problems. However, they underperform for complex task…
Accurate on-device keyword spotting (KWS) with low false accept and false reject rate is crucial to customer experience for far-field voice control of conversational agents. It is particularly challenging to maintain low false reject rate in real world conditions where there is (a) ambient noise from external sources s…
Study analyzes Echo State Network parameters for Rossler attractor dynamics.
problem Understanding the influence of network type on Echo State Network performance.
method Experimental analysis of Echo State Network parameters using Rossler attractor.
result Exploration of how network type affects Echo State Network performance.
Efficient echo state network with explicit memory performs well on benchmark tasks.
problem Training differentiable neural computers is difficult and time-consuming.
method Echo state network with an explicit memory.
result Echo state network can recognize all regular languages, including those contractive networks cannot.
This paper presents a stochastic logic time delay reservoir design. The reservoir is analyzed using a number of metrics, such as kernel quality, generalization rank, performance on simple benchmarks, and is also compared to a deterministic design. A novel re-seeding method is introduced to reduce the adverse effects of…
Proposes Textual Echo Cancellation to improve speech recognition.
problem Improving speech recognition performance and user experience for smart devices.
method A novel sequence-to-sequence model with multi-source attention that processes both the microphone mixture signal and source text of TTS playback.
result Demonstrates enhanced speech recognition performance and reduced latency.
Study on deep and wide echo state networks for forecasting complex time series.
problem Performance analysis of deep reservoir computing models.
method Investigates the impact of partitioning neurons and parallel pathways on forecasting accuracy.
result Wide and deep networks outperform shallow models in forecasting multiscale spatiotemporal data.
Tuna-AI uses ML to predict tuna biomass from oceanography and echo-sounder data.
problem Estimating tuna biomass from limited data sources.
method Developed a Machine Learning model using 3-day echo-sounder data and oceanographic data.
result Tuna-AI predicts tuna biomass with high accuracy using a 3-day window of data.
New approach speeds up optimization with repeated gradient steps on same batch.
problem Performance bottlenecks in massive parallel pipelines with large batch sizes.
method Data echoing, taking repeated gradient steps on the same batch.
result Data echoing affords speedups on curvature-dominated part of convergence rate.
Classifies spherical objects filled with water or air using acoustic echoes and Form Function.
problem Automatic recognition of spherical objects based on acoustic echoes.
method Form Function analysis of wideband sonar data, fed into a Multilayer Perceptron (MLP) classifier.
result Performance of the MLP classifier compared to SVM, highlighting the effectiveness of Form Function.
The paper analyzes feedback loops in recommender systems causing echo chambers and filter bubbles.
problem Feedback loops in recommender systems leading to echo chambers and filter bubbles.
method Theoretical analysis of user dynamics and recommender system behavior.
result Solutions to slow down system degeneracy and understanding echo chambers and filter bubbles.
Study on state dynamics in Deep Echo State Networks, revealing the importance of inter-reservoir connections.
problem Understanding state dynamics in multi-layered RNNs.
method Tools from information theory and numerical analysis.
result Inter-reservoir connections enrich representations in higher layers of DeepESNs.
This research unifies concepts of fading memory in RNNs.
problem Unclear relationships between fading memory concepts in RNNs.
method Unified language and new proofs for fading memory concepts.
result Clarified relationships between fading memory concepts.
Ensemble of MFESNs improves macroeconomic forecasting.
problem Improving accuracy in macroeconomic forecasting.
method Hedge and Follow-the-Leader schemes applied to MFESNs.
result Ensemble models outperform individual models in forecasting.
Echo state networks are powerful recurrent neural networks. However, they are often unstable and shaky, making the process of finding an good ESN for a specific dataset quite hard. Obtaining a superb accuracy by using the Echo State Network is a challenging task. We create, develop and implement a family of predictably…
Unified treatment of RC in stochastic and deterministic settings.
problem Understanding and generalizing reservoir computing in both deterministic and stochastic contexts.
method Investigation of state-space systems, analysis of fading memory and solution stability, introduction of stochastic echo states.
result Generality of fading memory and solution stability in state-space systems, even without the echo state property.
This work introduces a tensor-based method to perform supervised classification on spatiotemporal data processed in an echo state network. Typically when performing supervised classification tasks on data processed in an echo state network, the entire collection of hidden layer node states from the training dataset is …
CESAR improves wind speed and power forecasting for high-resolution simulations.
problem Accurate high-resolution wind forecasting for efficient power grid management.
method A spatio-temporal neural network model using deep convolutional autoencoder and echo state network.
result CESAR provides up to 17% improvement in wind speed and power forecasting compared to best alternatives.
Many neural networks use the tanh activation function, however when given a probability distribution as input, the problem of computing the output distribution in neural networks with tanh activation has not yet been addressed. One important example is the initialization of the echo state network in reservoir computing…
Efficient cross-validation method for Echo State Networks reduces validation time complexity.
problem Hyper-parameter tuning and validation for Echo State Networks (ESNs).
method Proposes several k-fold cross-validation schemes for ESNs with an efficient algorithm. result Cross-validation of ESNs can be done for the same time complexity as a single split validation.
Study shows structured reservoirs improve deep ESN performance.
problem Improving performance of deep reservoir computing networks.
method Investigated structured reservoir topologies in deep ESNs.
result Structured reservoirs significantly enhance predictive performance.
Discrete return (DR) Laser Detection and Ranging (Ladar) systems provide a series of echoes that reflect from objects in a scene. These can be first, last or multi-echo returns. In contrast, Full-Waveform (FW)-Ladar systems measure the intensity of light reflected from objects continuously over a period of time. In a c…
In this article, we derive a new stepsize adaptation for the normalized least mean square algorithm (NLMS) by describing the task of linear acoustic echo cancellation from a Bayesian network perspective. Similar to the well-known Kalman filter equations, we model the acoustic wave propagation from the loudspeaker to th…
ESN models predict intraday stock returns efficiently.
problem Intraday stock return prediction using machine learning.
method Echo State Network (ESN) models with random parameters.
result ESN models achieve strong forecasting performance efficiently.
Optical ESNs enable flexible, efficient machine learning with reduced energy.
problem Implementing universal computational capabilities in machine learning.
method Optical implementation of ESNs leveraging stimulated Brillouin scattering.
result Efficient, scalable, and memory-capable optical reservoir computing.
Physics-informed ESNs improve chaotic system prediction accuracy.
problem Predicting chaotic systems while adhering to physical laws.
method Integrates physics constraints into ESN training through an additional loss function.
result Physics-informed ESNs predict chaotic systems with a 2 Lyapunov time improvement.
Paper proposes a method to estimate intra-observer variability in echocardiography quality assessment.
problem Intra-observer variability in echocardiography quality assessment impacts deep neural network reliability.
method Modeling intra-observer variability as aleatoric uncertainty in a regression problem.
result The proposed method reduces error from 0.11 to 0.09, improving test accuracy by 5.7%.
State-space systems generate probabilistic dependencies between inputs and outputs.
problem Understanding probabilistic dependencies in state-space systems.
method Introducing a probabilistic framework and proving sufficient conditions for output existence and uniqueness.
result State-space systems can generate probabilistic dependencies, even without functional relations.
ESNs with transfer learning predict long-term chaotic patterns in spatiotemporal dynamical systems.
problem Predicting long-term statistical patterns of spatiotemporally chaotic dynamical systems.
method Echo state networks (ESNs) with transfer learning.
result ESNs with transfer learning accurately predict long-term statistical properties of spatiotemporally chaotic PDEs.
New analysis reveals optimal regularization for ESNs, avoiding double descent.
problem Characterizing and optimizing Echo State Networks (ESNs) for precise bias-variance.
method Random matrix theory applied to ESNs in a teacher-student setting.
result ESNs achieve lower MSE with limited training samples and teacher memory.
Study uses echo-sounder buoys to analyze tuna schools' association with dFADs globally.
problem Understanding temporal trends of tuna schools' association to drifting objects.
method Applied Machine Learning to examine binary and regression outputs of tuna schools' colonization and disaggregation times.
result Median colonization and disaggregation times varied by ocean, with Pacific having longest soak and colonization times.
ESNs trained with Tikhonov least squares approximate ergodic dynamical systems in L2(μ) norm.
problem Approximating ergodic dynamical systems using ESNs.
method Tikhonov least squares regression on ESNs trained on observations from an ergodic dynamical system.
result ESNs trained with Tikhonov least squares approximate the target function in the L2(μ) norm.
Echo state network (ESN) is viewed as a temporal non-orthogonal expansion with pseudo-random parameters. Such expansions naturally give rise to regressors of various relevance to a teacher output. We illustrate that often only a certain amount of the generated echo-regressors effectively explain the variance of the tea…
Spatially aware ESN detects anomalies in chaotic time series.
problem Automated anomaly detection in chaotic time series, especially turbulent ocean simulations.
method Extended Echo State Network with spatially aware input maps and loss function.
result Spatial ESN reduces anomaly detection to thresholding of prediction error.
The memory capacity of linear echo state networks is accurately calculated using new numerical methods.
problem Numerical evaluations of memory capacity in recurrent neural networks often contradict theoretical bounds.
method Developed robust numerical approaches exploiting MC neutrality with respect to the input mask matrix.
result Memory curves fully agree with theory when using the proposed methods.
The study of deep recurrent neural networks (RNNs) and, in particular, of deep Reservoir Computing (RC) is gaining an increasing research attention in the neural networks community. The recently introduced Deep Echo State Network (DeepESN) model opened the way to an extremely efficient approach for designing deep neura…
New definitions of ESP for quantum reservoir computing handle non-stationary systems.
problem Traditional ESP does not apply to non-stationary systems.
method Introduce two new categories of ESP: non-stationary ESP and subset/subspace ESP.
result Demonstrates correspondence between non-stationary ESP and QRC with NARMA tasks.
New framework models echo chamber learning, proving tight bounds on algorithm performance.
problem Echo chambers in machine learning where systems learn from self-annotated data.
method Online Learning in the Replay Setting, Extended Threshold dimension, closure-based learner.
result Proves tight bounds on algorithm performance against replay adversaries.
Stochastic reservoir computing is shown to be a universal approximator.
problem Theoretical justification for using stochastic reservoirs in machine learning.
method Investigated stochastic reservoir computing using probabilities of reservoir states as readout.
result Stochastic reservoir computers are universal approximating classes.
Reservoir Memory Machines solve benchmark tasks faster than Neural Turing Machines.
problem Training Neural Turing Machines is hard and limits their applicability.
method Proposes Reservoir Memory Machines, combining neural network flexibility with Turing machine capabilities, but with faster training via alignment and linear regression.
result Reservoir Memory Machines solve benchmark tasks as well as Neural Turing Machines but are much faster to train.