A new KAN variant uses sinusoidal activations to approximate functions.
problem Approximating multivariable functions using neural networks.
method Replacing inner and outer functions in Kolmogorov-Arnold representation with weighted sinusoidal functions.
result The new KAN variant outperforms fixed-frequency Fourier transform and achieves comparable performance to MLPs.
WaveQ uses sinusoidal regularization to optimize deep quantization for neural networks, improving both efficiency and accuracy.
problem Deep quantization reduces bitwidth but can lead to significant accuracy loss due to inter-layer dependencies.
method WaveQ employs sinusoidal regularization to learn multiple quantization parameters during gradient-based training, balancing compute efficiency and accuracy.
result WaveQ achieves accuracy preservation and efficiency gains across various deep networks, outperforming state-of-the-art techniques.
SinReQ adds sinusoidal regularization to improve quantized neural networks.
problem Accuracy loss in quantized deep neural networks.
method SinReQ adds a periodic term to the objective function of quantized training algorithms.
result SinReQ closes the accuracy gap by 32.4% and 27.5% compared to DoReFa and WRPN respectively.
A new method uses sinusoidal functions to represent timestamps as dense vectors for improving irregularly sampled time series learning.
problem Challenges in supervised learning with irregularly sampled time series due to irregular time intervals.
method Proposes a novel method to represent timestamps as dense vectors using sinusoidal functions, called Time Embeddings.
result Improves LSTM-based and classical machine learning models, especially with very irregular data.
Algorithm finds frequencies, amplitudes, and phases of sinusoids in noisy data.
problem Finding frequencies, amplitudes, and phases of sinusoids in noisy data.
method Maximum likelihood approach to estimate tone parameters from contaminated observations. Successively estimates frequencies and jointly optimizes amplitudes and phases.
result Near-linear computational complexity (O(N)) for estimating M number of sinusoidal sources. Paper proposes robust LAD estimators for 2D sinusoidal model, proving consistency and normality.
problem Estimation of parameters in 2D sinusoidal models with outliers or heavy-tailed noise.
method Least absolute deviation (LAD) estimators for robust parameter estimation.
result Strong consistency and asymptotic normality of LAD estimators for 2D sinusoidal model parameters.
Deep ReLU networks can approximate various signal types with exponential error decay.
problem Approximating different signal structures with deep neural networks.
method Demonstrated approximation of polynomials, sinusoidal functions, oscillatory textures, and fractals.
result Finite-width deep ReLU networks require fewer connections than wide finite-depth networks for smooth function approximation.
Random sinusoidal features are a popular approach for speeding up kernel-based inference in large datasets. Prior to the inference stage, the approach suggests performing dimensionality reduction by first multiplying each data vector by a random Gaussian matrix, and then computing an element-wise sinusoid. Theoretical …
Autoencoder estimates parameters of noisy, multi-component damped signals.
problem Parameter estimation of damped sinusoidal signals under rapid decay and noise.
method Autoencoder-based approach using latent space for frequency, phase, decay, and amplitude estimation.
result High accuracy in parameter estimation, robustness to subdominant components and phase differences.
Paper revises power theory using classical mechanics concepts.
problem Clarifying instantaneous power definitions for circuit elements.
method Defines power using classical mechanics concepts like velocity and momentum.
result General and compact expression for inductance, capacitance, and resistance powers.
New algorithm trains deep neural networks without global optimization.
problem Training deep neural networks efficiently and without global optimization.
method Uses random complex exponential activation functions and Markov Chain Monte Carlo sampling.
result Consistently attains theoretical approximation rate for residual networks.
It is well-known that the robustness of artificial neural networks (ANNs) is important for their wide ranges of applications. In this paper, we focus on the robustness of the classification ability of a spiking neural network which receives perturbed inputs. Actually, the perturbation is allowed to be arbitrary styles.…
Periodic activation functions improve neural network reliability and interpretability.
problem Neural networks reinforce hidden biases, making them unreliable and hard to interpret.
method Introduce periodic activation functions in Bayesian neural networks to establish a connection with stationary Gaussian process priors.
result Periodic activation functions, including sinusoidal, triangular, and ReLU, improve model performance and sensitivity to perturbations.
A new neural network improves frequency estimation from noisy signals.
problem Estimating frequencies of sinusoidal components in noisy signals.
method A novel neural network architecture combined with a module to detect the number of frequencies.
result Significantly more accurate frequency estimation at medium-to-high noise levels.
Generative Adversarial Networks create time series data from images.
problem Generating realistic time series data from images.
method Wasserstein GANs with gradient penalty for stability, synthesizing sinusoidal, PPG, and ECG data.
result Successfully generated time series data using image-based GANs.
Method interprets LSTMs at the cell level for better understanding of their dynamics.
problem Understanding the dynamics of LSTMs at the cell level.
method A systematic pipeline for interpreting individual hidden state dynamics using response characterization methods.
result Identifies neurons with insightful dynamics and quantifies their impact on network performance.
Neural networks struggle with periodic functions, a new activation fixes this.
problem Neural networks fail to learn simple periodic functions.
method Proposed a new activation function, x+sin2(x), to learn periodic functions. result The new activation function successfully learns and predicts periodic functions.
The study proposes a new interest rate model that captures long-term periodicity in U.S. Treasury yields.
problem The conventional Hull-White model fails to adequately capture long-term economic cycles in interest rates.
method The study introduces a sinusoidal Hull-White model with a time-varying mean reversion speed.
result The proposed model improves bond pricing and interest rate derivative valuation, especially for longer maturities.
Frequency-specific patterns of neural activity are traditionally interpreted as sustained rhythmic oscillations, and related to cognitive mechanisms such as attention, high level visual processing or motor control. While alpha waves (8-12 Hz) are known to closely resemble short sinusoids, and thus are revealed by Fouri…
Improved F0 estimation in noisy speech with neural networks.
problem Difficult F0 estimation at low SNRs in unexpected noise.
method Waveform-to-sinusoid regression using RNN trained on supervised data.
result Significant improvement in FPE and GPE rates compared to existing methods.
Sine activation functions enable two-layer neural networks to learn modular addition more efficiently.
problem Learning modular addition with two-layer neural networks.
method Introduced and analyzed sine activation functions, providing theoretical and empirical evidence.
result Sine activation functions allow for constant-width network realizations of modular addition, whereas ReLU networks require linear width scaling.
Estimation of functions of d variables is considered using ridge combinations of the form ∑k=1mc1,kφ(∑j=1dc0,j,kxj−bk) where the activation function φ is a function with bounded value and derivative. These include single-hidden layer neural networks, polynomials, …
A new method, based on the original theory of conservation of sum of kinetic and potential energy defined for prices is proposed and applied on Dow Jones Industrials Average (DJIA). The general trends averaged over months or years gave a roughly conserved total energy, with three different potential energies, i.e. posi…
Exploiting the fact that most arrival processes exhibit cyclic behaviour, we propose a simple procedure for estimating the intensity of a nonhomogeneous Poisson process. The estimator is the super-resolution analogue to Shao 2010 and Shao & Lii 2011, which is a sum of p sinusoids where p and the frequency, amplitud…
The paper explores the limits of deep neural networks in approximating various function classes.
problem Characterizing the limits of deep neural networks in function approximation.
method Develops a theory relating function complexity and network complexity, using Kolmogorov complexity.
result Deep networks are optimal approximants for various function classes and provide exponential approximation accuracy.
Paper uses DMD to embed time in spatiotemporal forecasting.
problem Forecasting long-range seasonal dependencies in spatiotemporal data.
method Dynamic Mode Decomposition (DMD) for time representation.
result DMD-based embedding improves long-horizon forecasting accuracy.
Symmetry-regularized Neural ODEs improve model stability and interpretability.
problem Improving the stability and physical interpretability of Neural ODEs.
method Integrating Lie symmetries and conservation laws into the loss function.
result Symmetry-regularized Neural ODEs enhance model stability and interpretability.
RI-DeepONet learns neural operators from arbitrary sensor data.
problem Discretization of input functions limits practical applications of DeepONet.
method Introduces RI-DeepONet and two dictionary learning algorithms for INRs.
result RINO handles arbitrary sensor data robustly and applies to various problems.
Numerical simulations show stability of Type-II singularities in noncompact hypersurfaces.
problem Stability of Type-II singularities in noncompact hypersurfaces with rotationally-symmetric perturbations.
method Adaptation of the overlap method to include angular dependence.
result MCF of noncompact hypersurfaces with angular dependence behaves similarly to rotationally-symmetric perturbations, developing Type-II or Type-I singularities.
New method for Transformer models to encode position information without sequential bias.
problem Lack of flexible and learnable position encoding for Transformer models.
method Continuous dynamical model to learn position encoding.
result Consistent improvements over baselines in various NLP tasks.
Time-aware deep learning methods improve spatial downscaling of atmospheric pollutants.
problem Transform coarse satellite data of atmospheric pollutants into high-resolution fields.
method Super-resolution deep residual networks and UNet architectures are extended with a temporal module encoding observation time.
result Temporal modules significantly improve downscaling performance and convergence speed.
Let f⋆ be a function on Rd with an assumption of a spectral norm vf⋆. For various noise settings, we show that E∥f^−f⋆∥2≤(vf⋆4nlogd)1/3, where n is the sample size and f^ is either a penalized lea…
Study evaluates different mathematical models for three case studies using statistical fitting.
problem Estimating outcomes in population dynamics, temperature variations, and market equilibrium.
method Applied various statistical equations (e.g., fractional exponential, sinusoidal) to three case studies.
result Optimal models differ by case study (fractional exponential for population dynamics, sinusoidal for temperature and market equilibrium).
BEKAN uses RBFs and evolutionary methods to solve PDEs with boundary conditions.
problem Enforcing boundary conditions in neural networks for PDE solutions.
method Boundary condition-guaranteed evolutionary Kolmogorov-Arnold Network (BEKAN) with radial basis functions (RBFs). Incorporates Dirichlet, periodic, and Neumann conditions.
result BEKAN outperforms MLP and B-splines KAN in solving PDEs with boundary conditions.
Paper studies Transformer learning theory for Euclidean and Riemannian domains.
problem Understanding and optimizing Transformer networks for regression tasks.
method Constructive approximation framework using softmax partition of unity and attention mechanism.
result Transformer can achieve uniform ε-approximation error with minimal parameters.
This paper explores robust recovery of a superposition of R distinct complex exponential functions from a few random Gaussian projections. We assume that the signal of interest is of 2N−1 dimensional and R<<2N−1. This framework covers a large class of signals arising from real applications in biology, automation,…
Study of curves in Lorentz-Minkowski plane with curvature dependent on position.
problem Determine curves in Lorentz-Minkowski plane whose curvature depends on their position.
method Use geometric angular and linear momentum to find curves through quadratures.
result Find several new families of Lorentzian spirals and curves.
Gradient-EM Bayesian meta-learning accelerates adaptation with reduced computation and improved robustness.
problem Efficient and robust adaptation to new tasks with uncertainty assessment.
method Extends Bayesian meta-learning with gradient-EM algorithm, decoupling inner-update from meta-update.
result Improves accuracy with less computation cost and enhanced robustness to uncertainty.
Understanding the relationships between different properties of data, such as whether a connectome or genome has information about disease status, is becoming increasingly important in modern biological datasets. While existing approaches can test whether two properties are related, they often require unfeasibly large …
New model for pricing volatility derivatives considering rough volatility and jumps.
problem Modeling instantaneous volatility with rough volatility and jumps.
method Generalized fractional Ornstein-Uhlenbeck process with Lévy subordinator and sinusoidal-composite Lévy process.
result Pricing-hedging formulae for power-type derivatives on average forward variance are derived.
Paper proposes a new method for training small models on regression problems.
problem Training small models for regression problems with noisy labels.
method Developed a new loss function and a multi-task network approach.
result Improved model accuracy on various datasets, consistent across different levels of annotation errors.
This paper won 1st place in forecasting and investment challenges, improving on meta-learning and parametric models.
problem Forecasting and investment challenges in time-series data.
method Hypernetworks and adversarial portfolios to design time-series models.
result Outperformed state-of-the-art meta-learning methods and conventional parametric models.
Bayesian meta-learning method learns uncertainty from few data.
problem Learning uncertainty from limited data in meta-learning.
method Combines gradient-based meta-learning with nonparametric variational inference.
result Robust meta-update mechanism prevents overfitting.
Alternative wavelet analysis method for financial signals.
problem Analyzing oscillations in financial signals with noise.
method Modeling financial signals as isolated events producing ripples of various frequencies.
result Element analysis distinguishes between noise and logically matched generators.
Paper introduces a new method for Transformers with linear complexity.
problem No efficient relative positional encoding for linear Transformer models.
method Stochastic Positional Encoding (SPE) that replaces classical RPE.
result SPE behaves like RPE and performs well on benchmarks.
Kernel method is a very powerful tool in machine learning. The trick of kernel has been effectively and extensively applied in many areas of machine learning, such as support vector machine (SVM) and kernel principal component analysis (kernel PCA). Kernel trick is to define a kernel function which relies on the inner-…
The maximum mean discrepancy (MMD) is a recently proposed test statistic for two-sample test. Its quadratic time complexity, however, greatly hampers its availability to large-scale applications. To accelerate the MMD calculation, in this study we propose an efficient method called FastMMD. The core idea of FastMMD is …
EmDT generates synthetic fraud data to improve detection accuracy.
problem Imbalanced datasets in fraud detection lead to poor performance on rare fraudulent transactions.
method EmDT uses UMAP clustering to identify fraudulent patterns and a Transformer denoising network to generate synthetic data.
result EmDT significantly improves classification performance compared to existing methods.