A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
New methods improve Reservoir Computing for chaotic time series prediction.
In this paper, we first give a direct proof for two recurrence relations of the heat kernels for hyperbolic spaces in \cite{DM}. Then, by similar computation, we give two similar recurrence relations of the heat kernels for spheres. Finally, as an application, we compute the diagonal of heat kernels for odd dimensional…
Paper introduces RNTK for recurrent neural networks, improving performance across various datasets.
RNNs are reinterpreted as kernel methods using neural ODEs.
Substring kernels are classical tools for representing biological sequences or text. However, when large amounts of annotated data are available, models that allow end-to-end training such as neural networks are often preferred. Links between recurrent neural networks (RNNs) and substring kernels have recently been dra…
New kernels boost RNN performance on non-time-series data.
Many applications in speech, robotics, finance, and biology deal with sequential data, where ordering matters and recurrent structures are common. However, this structure cannot be easily captured by standard kernel functions. To model such structure, we propose expressive closed-form kernel functions for Gaussian proc…
This work optimizes reservoir computing models by linking recurrence and non-linear dynamics.
New analysis explains pathology of deep Gaussian processes.
We investigate time-dependent data analysis from the perspective of recurrent kernel machines, from which models with hidden units and gated memory cells arise naturally. By considering dynamic gating of the memory cell, a model closely related to the long short-term memory (LSTM) recurrent neural network is derived. E…
The study uses response theory to understand RNNs processing input signals.
New insights on stability in reservoir computing for better performance.
Develops a deep non-stationary kernel for non-stationary spatio-temporal point processes.
Proposes a new model for time-to-event prediction with uncertainty quantification.
Method learns latent dynamics of complex systems from noisy data.
Identifies conditions for multiple invariant probabilities in Markov kernels.
This paper improves speech synthesis using a DGP with SRU for naturalness.
We propose and evaluate new techniques for compressing and speeding up dense matrix multiplications as found in the fully connected and recurrent layers of neural networks for embedded large vocabulary continuous speech recognition (LVCSR). For compression, we introduce and study a trace norm regularization technique f…
Linear RNNs exhibit a bias towards shorter memory due to initialization variance.
Tensor programs prove neural network limits for any architecture.
Recurrent convolution (RC) shares the same convolutional kernels and unrolls them multiple steps, which is originally proposed to model time-space signals. We argue that RC can be viewed as a model compression strategy for deep convolutional neural networks. RC reduces the redundancy across layers. However, the perform…
Gated recurrent neural networks have achieved remarkable results in the analysis of sequential data. Inside these networks, gates are used to control the flow of information, allowing to model even very long-term dependencies in the data. In this paper, we investigate whether the original gate equation (a linear projec…
In this paper, motivated by finding sharp Li-Yau type gradient estimate for positive solution of heat equations on complete Riemannian manifolds with negative Ricci curvature lower bound, we first introduce the notion of Li-Yau multiplier set and show that it can be computed by heat kernel of the manifold. Then, an opt…
Performing machine learning on structured data is complicated by the fact that such data does not have vectorial form. Therefore, multiple approaches have emerged to construct vectorial representations of structured data, from kernel and distance approaches to recurrent, recursive, and convolutional neural networks. Re…
Deep learning (DL) is one of the most prominent branches of machine learning. Due to the immense computational cost of DL workloads, industry and academia have developed DL libraries with highly-specialized kernels for each workload/architecture, leading to numerous, complex code-bases that strive for performance, yet …
The Fisher information metric is an important foundation of information geometry, wherein it allows us to approximate the local geometry of a probability distribution. Recurrent neural networks such as the Sequence-to-Sequence (Seq2Seq) networks that have lately been used to yield state-of-the-art performance on speech…
Unified theory for deep and recurrent networks using Gaussian processes.
KATA improves associative recall by optimizing feature maps derived from nonnegative attention weights.
We study the near diagonal asymptotic expansion of the generalized Bergman kernel of the renormalized Bochner-Laplacian on high tensor powers of a positive line bundle over a compact symplectic manifold. We show how to compute the coefficients of the expansion by recurrence and give a closed formula for the first two o…
Action recognition greatly benefits motion understanding in video analysis. Recurrent networks such as long short-term memory (LSTM) networks are a popular choice for motion-aware sequence learning tasks. Recently, a convolutional extension of LSTM was proposed, in which input-to-hidden and hidden-to-hidden transitions…
Beta Basis Function Neural Network (BBFNN) is a special kind of kernel basis neural networks. It is a feedforward network typified by the use of beta function as a hidden activation function. Beta is a flexible transfer function representing richer forms than the common existing functions. As in every network, the arch…
This paper improves RNN generalization without normalization conditions.
Bayesian time series forecasting improves by dynamically adapting to recent information.
Conventional principal component analysis (PCA) finds a principal vector that maximizes the sum of second powers of principal components. We consider a generalized PCA that aims at maximizing the sum of an arbitrary convex function of principal components. We present a gradient ascent algorithm to solve the problem. Fo…
Parameterized state space models in the form of recurrent networks are often used in machine learning to learn from data streams exhibiting temporal dependencies. To break the black box nature of such models it is important to understand the dynamical features of the input driving time series that are formed in the sta…
The effectiveness of deep neural architectures has been widely supported in terms of both experimental and foundational principles. There is also clear evidence that the activation function (e.g. the rectifier and the LSTM units) plays a crucial role in the complexity of learning. Based on this remark, this paper discu…
Accurately predicting industrial aging processes makes it possible to schedule maintenance events further in advance, ensuring a cost-efficient and reliable operation of the plant. So far, these degradation processes were usually described by mechanistic or simple empirical prediction models. In this paper, we evaluate…
Mathematical methods characterize RNNs' asymptotics as hidden units and data grow.
The pullback approach to global Finsler geometry is adopted. Three classes of recurrence in Finsler geometry are introduced and investigated: simple recurrence, Ricci recurrence and concircular recurrence. Each of these classes consists of four types of recurrence. The interrelationships between the different types of …
Study on biharmonic hypersurfaces with specific recurrent operators in Euclidean space.
The aim of the present paper is to investigate new types of recurrence in Finsler geometry, namely, hyper-generalized recurrence and generalized conharmonic recurrence. The properties of such recurrences and their relations to other Finsler recurrences are studied.
To generalize the notion of recurrent manifold, there are various recurrent like conditions in the literature. In this paper we present a recurrent like structure, namely, \textit{super generalized recurrent manifold}, which generalizes both the hyper generalized recurrent manifold and weakly generalized recurrent mani…
This paper describes a method for learning low-dimensional approximations of nonlinear dynamical systems, based on neural-network approximations of the underlying Koopman operator. Extended Dynamic Mode Decomposition (EDMD) provides a useful data-driven approximation of the Koopman operator for analyzing dynamical syst…
Two special Finsler spaces have been introduced and investigated, namely -recurrent Finsler space and consircularly recurrent Finsler space. The defining properties of these spaces are formulated in terms of the first curvature tensor of Cartan connection. The following three results constitute the main object of …
The object of the present paper is to obtain the characterization of a warped product semi-Riemannian manifold with a special type of recurrent like structure, called super generalized recurrent. As consequence of this result we also find out the necessary and sufficient conditions for a warped product manifold to sati…
The present paper deals with the proper existence of a generalized class of recurrent manifolds, namely, hyper-generalized recurrent manifolds. We have established the proper existence of various generalized notions of recurrent manifolds. For this purpose we have presented a metric and computed its curvature propertie…
We show that in dimension n>3 the class of simple conformally recurrent space-times coincides with the class of conformally recurrent pp-waves.
Paper improves Monte Carlo sampling with new theoretical insights and methods.