arXiv research

A new KAN variant uses sinusoidal activations to approximate functions.

WaveQ uses sinusoidal regularization to optimize deep quantization for neural networks, improving both efficiency and accuracy.

A new method uses sinusoidal functions to represent timestamps as dense vectors for improving irregularly sampled time series learning.

Algorithm finds frequencies, amplitudes, and phases of sinusoids in noisy data.

Paper proposes robust LAD estimators for 2D sinusoidal model, proving consistency and normality.

Random sinusoidal features are a popular approach for speeding up kernel-based inference in large datasets. Prior to the inference stage, the approach suggests performing dimensionality reduction by first multiplying each data vector by a random Gaussian matrix, and then computing an element-wise sinusoid. Theoretical …

Deep quantization of neural networks (below eight bits) offers significant promise in reducing their compute and storage cost. Albeit alluring, without special techniques for training and optimization, deep quantization results in significant accuracy loss. To further mitigate this loss, we propose a novel sinusoidal r…

Autoencoder estimates parameters of noisy, multi-component damped signals.

Paper revises power theory using classical mechanics concepts.

We show that finite-width deep ReLU neural networks yield rate-distortion optimal approximation (Bölcskei et al., 2018) of polynomials, windowed sinusoidal functions, one-dimensional oscillatory textures, and the Weierstrass function, a fractal function which is continuous but nowhere differentiable. Together with thei…

New algorithm trains deep neural networks without global optimization.

It is well-known that the robustness of artificial neural networks (ANNs) is important for their wide ranges of applications. In this paper, we focus on the robustness of the classification ability of a spiking neural network which receives perturbed inputs. Actually, the perturbation is allowed to be arbitrary styles.…

Periodic activation functions improve neural network reliability and interpretability.

Neural networks struggle with periodic functions, a new activation fixes this.

The study proposes a new interest rate model that captures long-term periodicity in U.S. Treasury yields.

The fundamental frequency (F0) represents pitch in speech that determines prosodic characteristics of speech and is needed in various tasks for speech analysis and synthesis. Despite decades of research on this topic, F0 estimation at low signal-to-noise ratios (SNRs) in unexpected noise conditions remains difficult. T…

Frequency-specific patterns of neural activity are traditionally interpreted as sustained rhythmic oscillations, and related to cognitive mechanisms such as attention, high level visual processing or motor control. While alpha waves (8-12 Hz) are known to closely resemble short sinusoids, and thus are revealed by Fouri…

This paper develops fundamental limits of deep neural network learning by characterizing what is possible if no constraints are imposed on the learning algorithm and on the amount of training data. Concretely, we consider Kolmogorov-optimal approximation through deep neural networks with the guiding theme being a relat…

Sine activation functions enable two-layer neural networks to learn modular addition more efficiently.

Estimation of functions of $d$ variables is considered using ridge combinations of the form $\textstyle\sum_{k=1}^m c_{1,k} φ(\textstyle\sum_{j=1}^d c_{0,j,k}x_j-b_k)$ where the activation function $φ$ is a function with bounded value and derivative. These include single-hidden layer neural networks, polynomials, …

A new method, based on the original theory of conservation of sum of kinetic and potential energy defined for prices is proposed and applied on Dow Jones Industrials Average (DJIA). The general trends averaged over months or years gave a roughly conserved total energy, with three different potential energies, i.e. posi…

Exploiting the fact that most arrival processes exhibit cyclic behaviour, we propose a simple procedure for estimating the intensity of a nonhomogeneous Poisson process. The estimator is the super-resolution analogue to Shao 2010 and Shao & Lii 2011, which is a sum of $p$ sinusoids where $p$ and the frequency, amplitud…

Paper uses DMD to embed time in spatiotemporal forecasting.

Symmetry-regularized Neural ODEs improve model stability and interpretability.

RI-DeepONet learns neural operators from arbitrary sensor data.

Numerical simulations show stability of Type-II singularities in noncompact hypersurfaces.

New method for Transformer models to encode position information without sequential bias.

Time-aware deep learning methods improve spatial downscaling of atmospheric pollutants.

Let $f^{\star}$ be a function on $\mathbb{R}^d$ with an assumption of a spectral norm $v_{f^{\star}}$. For various noise settings, we show that $\mathbb{E}\|\hat{f} - f^{\star} \|^2 \leq \left(v^4_{f^{\star}}\frac{\log d}{n}\right)^{1/3}$, where $n$ is the sample size and $\hat{f}$ is either a penalized lea…

Study evaluates different mathematical models for three case studies using statistical fitting.

BEKAN uses RBFs and evolutionary methods to solve PDEs with boundary conditions.

In the recent years Generative Adversarial Networks (GANs) have demonstrated significant progress in generating authentic looking data. In this work we introduce our simple method to exploit the advancements in well established image-based GANs to synthesise single channel time series data. We implement Wasserstein GAN…

Frequency estimation is a fundamental problem in signal processing, with applications in radar imaging, underwater acoustics, seismic imaging, and spectroscopy. The goal is to estimate the frequency of each component in a multisinusoidal signal from a finite number of noisy samples. A recent machine-learning approach u…

In this paper, we introduce a novel method to interpret recurrent neural networks (RNNs), particularly long short-term memory networks (LSTMs) at the cellular level. We propose a systematic pipeline for interpreting individual hidden state dynamics within the network using response characterization methods. The ranked …

Learning to infer Bayesian posterior from a few-shot dataset is an important step towards robust meta-learning due to the model uncertainty inherent in the problem. In this paper, we propose a novel Bayesian model-agnostic meta-learning method. The proposed method combines scalable gradient-based meta-learning with non…

Paper studies Transformer learning theory for Euclidean and Riemannian domains.

This paper explores robust recovery of a superposition of $R$ distinct complex exponential functions from a few random Gaussian projections. We assume that the signal of interest is of $2N-1$ dimensional and $R<<2N-1$. This framework covers a large class of signals arising from real applications in biology, automation,…

Gradient-EM Bayesian meta-learning accelerates adaptation with reduced computation and improved robustness.

Understanding the relationships between different properties of data, such as whether a connectome or genome has information about disease status, is becoming increasingly important in modern biological datasets. While existing approaches can test whether two properties are related, they often require unfeasibly large …

Motivated by the classical Euler elastic curves, David A. Singer posed in 1999 the problem of determining a plane curve whose curvature is given in terms of its position. We propound the same question in Lorentz-Minkowski plane, focusing on spacelike and timelike curves. In this article, we study those curves in $\math…

New model for pricing volatility derivatives considering rough volatility and jumps.

Paper proposes a new method for training small models on regression problems.

This paper won 1st place in forecasting and investment challenges, improving on meta-learning and parametric models.

Alternative wavelet analysis method for financial signals.

Paper introduces a new method for Transformers with linear complexity.

Kernel method is a very powerful tool in machine learning. The trick of kernel has been effectively and extensively applied in many areas of machine learning, such as support vector machine (SVM) and kernel principal component analysis (kernel PCA). Kernel trick is to define a kernel function which relies on the inner-…

The maximum mean discrepancy (MMD) is a recently proposed test statistic for two-sample test. Its quadratic time complexity, however, greatly hampers its availability to large-scale applications. To accelerate the MMD calculation, in this study we propose an efficient method called FastMMD. The core idea of FastMMD is …

EmDT generates synthetic fraud data to improve detection accuracy.