arXiv research

A new KAN variant uses sinusoidal activations to approximate functions.

Paper revises power theory using classical mechanics concepts.

Periodic activation functions improve neural network reliability and interpretability.

Algorithm finds frequencies, amplitudes, and phases of sinusoids in noisy data.

New algorithm trains deep neural networks without global optimization.

Neural networks struggle with periodic functions, a new activation fixes this.

Paper proposes robust LAD estimators for 2D sinusoidal model, proving consistency and normality.

WaveQ uses sinusoidal regularization to optimize deep quantization for neural networks, improving both efficiency and accuracy.

Sine activation functions enable two-layer neural networks to learn modular addition more efficiently.

Random sinusoidal features are a popular approach for speeding up kernel-based inference in large datasets. Prior to the inference stage, the approach suggests performing dimensionality reduction by first multiplying each data vector by a random Gaussian matrix, and then computing an element-wise sinusoid. Theoretical …

Autoencoder estimates parameters of noisy, multi-component damped signals.

Frequency-specific patterns of neural activity are traditionally interpreted as sustained rhythmic oscillations, and related to cognitive mechanisms such as attention, high level visual processing or motor control. While alpha waves (8-12 Hz) are known to closely resemble short sinusoids, and thus are revealed by Fouri…

A new method uses sinusoidal functions to represent timestamps as dense vectors for improving irregularly sampled time series learning.

Learning to infer Bayesian posterior from a few-shot dataset is an important step towards robust meta-learning due to the model uncertainty inherent in the problem. In this paper, we propose a novel Bayesian model-agnostic meta-learning method. The proposed method combines scalable gradient-based meta-learning with non…

Deep quantization of neural networks (below eight bits) offers significant promise in reducing their compute and storage cost. Albeit alluring, without special techniques for training and optimization, deep quantization results in significant accuracy loss. To further mitigate this loss, we propose a novel sinusoidal r…

Estimation of functions of $d$ variables is considered using ridge combinations of the form $\textstyle\sum_{k=1}^m c_{1,k} φ(\textstyle\sum_{j=1}^d c_{0,j,k}x_j-b_k)$ where the activation function $φ$ is a function with bounded value and derivative. These include single-hidden layer neural networks, polynomials, …

It is well-known that the robustness of artificial neural networks (ANNs) is important for their wide ranges of applications. In this paper, we focus on the robustness of the classification ability of a spiking neural network which receives perturbed inputs. Actually, the perturbation is allowed to be arbitrary styles.…

Paper studies Transformer learning theory for Euclidean and Riemannian domains.

The study proposes a new interest rate model that captures long-term periodicity in U.S. Treasury yields.

The fundamental frequency (F0) represents pitch in speech that determines prosodic characteristics of speech and is needed in various tasks for speech analysis and synthesis. Despite decades of research on this topic, F0 estimation at low signal-to-noise ratios (SNRs) in unexpected noise conditions remains difficult. T…

New model for pricing volatility derivatives considering rough volatility and jumps.

A new method, based on the original theory of conservation of sum of kinetic and potential energy defined for prices is proposed and applied on Dow Jones Industrials Average (DJIA). The general trends averaged over months or years gave a roughly conserved total energy, with three different potential energies, i.e. posi…

Numerical simulations show stability of Type-II singularities in noncompact hypersurfaces.

New method for Transformer models to encode position information without sequential bias.

BEKAN uses RBFs and evolutionary methods to solve PDEs with boundary conditions.

We show that finite-width deep ReLU neural networks yield rate-distortion optimal approximation (Bölcskei et al., 2018) of polynomials, windowed sinusoidal functions, one-dimensional oscillatory textures, and the Weierstrass function, a fractal function which is continuous but nowhere differentiable. Together with thei…

A neural network for online NP classification with reduced complexity.

In the recent years Generative Adversarial Networks (GANs) have demonstrated significant progress in generating authentic looking data. In this work we introduce our simple method to exploit the advancements in well established image-based GANs to synthesise single channel time series data. We implement Wasserstein GAN…

Frequency estimation is a fundamental problem in signal processing, with applications in radar imaging, underwater acoustics, seismic imaging, and spectroscopy. The goal is to estimate the frequency of each component in a multisinusoidal signal from a finite number of noisy samples. A recent machine-learning approach u…

In this paper, we introduce a novel method to interpret recurrent neural networks (RNNs), particularly long short-term memory networks (LSTMs) at the cellular level. We propose a systematic pipeline for interpreting individual hidden state dynamics within the network using response characterization methods. The ranked …

Gradient-EM Bayesian meta-learning accelerates adaptation with reduced computation and improved robustness.

Exploiting the fact that most arrival processes exhibit cyclic behaviour, we propose a simple procedure for estimating the intensity of a nonhomogeneous Poisson process. The estimator is the super-resolution analogue to Shao 2010 and Shao & Lii 2011, which is a sum of $p$ sinusoids where $p$ and the frequency, amplitud…

Paper uses DMD to embed time in spatiotemporal forecasting.

Time-aware deep learning methods improve spatial downscaling of atmospheric pollutants.

Paper introduces a new method for Transformers with linear complexity.

EmDT generates synthetic fraud data to improve detection accuracy.

Symmetry-regularized Neural ODEs improve model stability and interpretability.

The gauge theory of arbitrage was introduced by Ilinski in [arXiv:hep-th/9710148] and applied to fast money flows in [arXiv:cond-mat/9902044]. The theory of fast money flow dynamics attempts to model the evolution of currency exchange rates and stock prices on short, e.g.\ intra-day, time scales. It has been used to ex…

This paper develops fundamental limits of deep neural network learning by characterizing what is possible if no constraints are imposed on the learning algorithm and on the amount of training data. Concretely, we consider Kolmogorov-optimal approximation through deep neural networks with the guiding theme being a relat…

The paper studies the problem of recovering a spectrally sparse object from a small number of time domain samples. Specifically, the object of interest with ambient dimension $n$ is assumed to be a mixture of $r$ complex multi-dimensional sinusoids, while the underlying frequencies can assume any value in the unit disk…

Principal component analysis (PCA) is a popular method for projecting data onto uncorrelated components in lower dimension, although the optimal number of components is not specified. Likewise, multiple signal classification (MUSIC) algorithm is a popular PCA-based method for estimating directions of arrival (DOAs) of …

This paper explores robust recovery of a superposition of $R$ distinct complex exponential functions from a few random Gaussian projections. We assume that the signal of interest is of $2N-1$ dimensional and $R<<2N-1$. This framework covers a large class of signals arising from real applications in biology, automation,…

Understanding the relationships between different properties of data, such as whether a connectome or genome has information about disease status, is becoming increasingly important in modern biological datasets. While existing approaches can test whether two properties are related, they often require unfeasibly large …

This paper presents a general solution for a recent model by Keen for endogenous money creation. The solution provides an analytic framework that explains all significant dynamical features of Keen's model and their parametric dependence, including an exact result for both the period and subsidence rate of the Great Mo…

Motivated by the classical Euler elastic curves, David A. Singer posed in 1999 the problem of determining a plane curve whose curvature is given in terms of its position. We propound the same question in Lorentz-Minkowski plane, focusing on spacelike and timelike curves. In this article, we study those curves in $\math…

This paper won 1st place in forecasting and investment challenges, improving on meta-learning and parametric models.

Many activation functions have been proposed in the past, but selecting an adequate one requires trial and error. We propose a new methodology of designing activation functions within a neural network at each layer. We call this technique an "activation ensemble" because it allows the use of multiple activation functio…

This paper studies activation sparsity in large language models, finding key trends and implications.