New method recovers sparse vectors from random sinusoidal features.
problem Recovering sparse vectors from random sinusoidal features.
method Proposes a numerically stable algorithm for sparse vector reconstruction.
result Sparse vectors can be reliably recovered from random sinusoidal features.
New algorithm trains deep neural networks without global optimization.
problem Training deep neural networks efficiently and without global optimization.
method Uses random complex exponential activation functions and Markov Chain Monte Carlo sampling.
result Consistently attains theoretical approximation rate for residual networks.
A new KAN variant uses sinusoidal activations to approximate functions.
problem Approximating multivariable functions using neural networks.
method Replacing inner and outer functions in Kolmogorov-Arnold representation with weighted sinusoidal functions.
result The new KAN variant outperforms fixed-frequency Fourier transform and achieves comparable performance to MLPs.
Algorithm finds frequencies, amplitudes, and phases of sinusoids in noisy data.
problem Finding frequencies, amplitudes, and phases of sinusoids in noisy data.
method Maximum likelihood approach to estimate tone parameters from contaminated observations. Successively estimates frequencies and jointly optimizes amplitudes and phases.
result Near-linear computational complexity (O(N)) for estimating M M M number of sinusoidal sources. Paper proposes robust LAD estimators for 2D sinusoidal model, proving consistency and normality.
problem Estimation of parameters in 2D sinusoidal models with outliers or heavy-tailed noise.
method Least absolute deviation (LAD) estimators for robust parameter estimation.
result Strong consistency and asymptotic normality of LAD estimators for 2D sinusoidal model parameters.
WaveQ uses sinusoidal regularization to optimize deep quantization for neural networks, improving both efficiency and accuracy.
problem Deep quantization reduces bitwidth but can lead to significant accuracy loss due to inter-layer dependencies.
method WaveQ employs sinusoidal regularization to learn multiple quantization parameters during gradient-based training, balancing compute efficiency and accuracy.
result WaveQ achieves accuracy preservation and efficiency gains across various deep networks, outperforming state-of-the-art techniques.
SinReQ adds sinusoidal regularization to improve quantized neural networks.
problem Accuracy loss in quantized deep neural networks.
method SinReQ adds a periodic term to the objective function of quantized training algorithms.
result SinReQ closes the accuracy gap by 32.4% and 27.5% compared to DoReFa and WRPN respectively.
Autoencoder estimates parameters of noisy, multi-component damped signals.
problem Parameter estimation of damped sinusoidal signals under rapid decay and noise.
method Autoencoder-based approach using latent space for frequency, phase, decay, and amplitude estimation.
result High accuracy in parameter estimation, robustness to subdominant components and phase differences.
A new method uses sinusoidal functions to represent timestamps as dense vectors for improving irregularly sampled time series learning.
problem Challenges in supervised learning with irregularly sampled time series due to irregular time intervals.
method Proposes a novel method to represent timestamps as dense vectors using sinusoidal functions, called Time Embeddings.
result Improves LSTM-based and classical machine learning models, especially with very irregular data.
A neural network for online NP classification with reduced complexity.
problem Online nonlinear Neyman-Pearson classification.
method Single hidden layer feedforward neural network (SLFN) initialized with random Fourier features (RFFs). Uses stochastic gradient descent for sequential learning.
result Expedited online adaptation and powerful nonlinear Neyman-Pearson modeling.
Paper revises power theory using classical mechanics concepts.
problem Clarifying instantaneous power definitions for circuit elements.
method Defines power using classical mechanics concepts like velocity and momentum.
result General and compact expression for inductance, capacitance, and resistance powers.
Deep ReLU networks can approximate various signal types with exponential error decay.
problem Approximating different signal structures with deep neural networks.
method Demonstrated approximation of polynomials, sinusoidal functions, oscillatory textures, and fractals.
result Finite-width deep ReLU networks require fewer connections than wide finite-depth networks for smooth function approximation.
A new neural network improves frequency estimation from noisy signals.
problem Estimating frequencies of sinusoidal components in noisy signals.
method A novel neural network architecture combined with a module to detect the number of frequencies.
result Significantly more accurate frequency estimation at medium-to-high noise levels.
Generative Adversarial Networks create time series data from images.
problem Generating realistic time series data from images.
method Wasserstein GANs with gradient penalty for stability, synthesizing sinusoidal, PPG, and ECG data.
result Successfully generated time series data using image-based GANs.
Method interprets LSTMs at the cell level for better understanding of their dynamics.
problem Understanding the dynamics of LSTMs at the cell level.
method A systematic pipeline for interpreting individual hidden state dynamics using response characterization methods.
result Identifies neurons with insightful dynamics and quantifies their impact on network performance.
EmDT generates synthetic fraud data to improve detection accuracy.
problem Imbalanced datasets in fraud detection lead to poor performance on rare fraudulent transactions.
method EmDT uses UMAP clustering to identify fraudulent patterns and a Transformer denoising network to generate synthetic data.
result EmDT significantly improves classification performance compared to existing methods.
Spiking neural networks maintain robust classification even with perturbed inputs.
problem Maintaining robustness of spiking neural networks under perturbed inputs.
method Extensive experiments on the XOR problem and benchmark datasets using SpikeProp algorithm.
result Classification ability of spiking neural networks is not significantly reduced by sinusoidal and Gaussian perturbations.
A new method estimates nonhomogeneous Poisson process intensities with super-resolution.
problem Estimating cyclic arrival rates of nonhomogeneous Poisson processes.
method Super-resolution estimation using sinusoidal waves with unknown parameters.
result Finite sample guarantees for super-resolution estimation under suitable conditions.
The study proposes a new interest rate model that captures long-term periodicity in U.S. Treasury yields.
problem The conventional Hull-White model fails to adequately capture long-term economic cycles in interest rates.
method The study introduces a sinusoidal Hull-White model with a time-varying mean reversion speed.
result The proposed model improves bond pricing and interest rate derivative valuation, especially for longer maturities.
Paper uses DMD to embed time in spatiotemporal forecasting.
problem Forecasting long-range seasonal dependencies in spatiotemporal data.
method Dynamic Mode Decomposition (DMD) for time representation.
result DMD-based embedding improves long-horizon forecasting accuracy.
This paper explores robust recovery of a superposition of R R R distinct complex exponential functions from a few random Gaussian projections. We assume that the signal of interest is of 2 N − 1 2N-1 2 N − 1 dimensional and R < < 2 N − 1 R<<2N-1 R << 2 N − 1 . This framework covers a large class of signals arising from real applications in biology, automation,…
Kernel method is a very powerful tool in machine learning. The trick of kernel has been effectively and extensively applied in many areas of machine learning, such as support vector machine (SVM) and kernel principal component analysis (kernel PCA). Kernel trick is to define a kernel function which relies on the inner-…
Orthogonal random features approximate a Bessel kernel, offering sharper bounds than random Fourier features.
problem Approximating Gaussian kernel efficiently for large datasets.
method Use of Haar orthogonal matrices to construct orthogonal random features and analyze their bias and variance.
result Orthogonal random features approximate a Bessel kernel, not the Gaussian kernel, with sharper bounds.
Time-aware deep learning methods improve spatial downscaling of atmospheric pollutants.
problem Transform coarse satellite data of atmospheric pollutants into high-resolution fields.
method Super-resolution deep residual networks and UNet architectures are extended with a temporal module encoding observation time.
result Temporal modules significantly improve downscaling performance and convergence speed.
Improved F0 estimation in noisy speech with neural networks.
problem Difficult F0 estimation at low SNRs in unexpected noise.
method Waveform-to-sinusoid regression using RNN trained on supervised data.
result Significant improvement in FPE and GPE rates compared to existing methods.
New method learns complex brain signal patterns from EEG/MEG data.
problem Complex waveforms in brain signals not captured by linear filters.
method Multivariate convolutional sparse coding (CSC) algorithm.
result Reveals non-sinusoidal mu-shaped patterns in brain signals.
A new method, based on the original theory of conservation of sum of kinetic and potential energy defined for prices is proposed and applied on Dow Jones Industrials Average (DJIA). The general trends averaged over months or years gave a roughly conserved total energy, with three different potential energies, i.e. posi…
Random ReLU features are shown to be a universally consistent learning algorithm but struggle with complex functions.
problem Approximating complex functions with random ReLU features.
method Study of random ReLU features through their RKHS and composition of functions.
result Random ReLU features can efficiently approximate complex functions but not as well as multi-layer ReLU networks.
EERF improves supervised learning by using data-dependent random features.
problem Improving generalization in supervised learning with fewer random features.
method EERF algorithm based on a data-dependent score function.
result EERF requires fewer random features to achieve a certain generalization error.
Proposes semi-random features for nonlinear function approximation.
problem Nonlinear function approximation in machine learning.
method Semi-random features as a middle ground between deep learning and kernel methods.
result Proves universal approximation and generalization for deep semi-random features.
Sparse random features improve accuracy in data-scarce settings.
problem Limited accuracy of random feature methods in data-scarce applications.
method Sparse random feature expansion using compressive sensing.
result Improved generalization bounds for sparse random features.
We analyze random feature maps for high-dimensional data using spectral methods.
problem Understanding the spectrum of random feature maps for high-dimensional data.
method We use concentration phenomena from random matrix theory to analyze the Gram matrix of random feature maps for Gaussian mixture models.
result Our results provide insights into the interplay between nonlinearity and data statistics.
Efficient random binning features improve kernel methods for large datasets.
problem Kernel methods' quadratic complexity limits their scalability to large datasets.
method Proposes and analyzes Random Binning (RB) features, showing faster convergence and parallelizability.
result RB features achieve faster convergence rates and parallelizability advantages compared to other random features.
Random feature maps improve forecasting with cheaper computation.
problem Improving forecasting accuracy with random feature maps.
method Developed a hit-and-run algorithm to select optimal internal weights.
result Optimal internal weights lead to superior forecasting skill.
RFSVM with random features achieves faster learning rates.
problem Improving the learning rate of SVM with random features.
method Support Vector Machine with N ≪ m N\ll m N ≪ m random features, optimized feature map, and reweighted feature selection. result RFSVM achieves faster learning rates than O ( 1 / m ) O(1/\sqrt{m}) O ( 1/ m ) under low noise assumptions. Kernel ridgeless regression with random features shows good generalization without explicit regularization.
problem Generalization of kernel ridgeless regression without explicit regularization.
method Investigation of ridgeless regression with random features and stochastic gradient descent, exploring the effect of random features error and spectral density optimization.
result Random features error exhibits the double-descent curve, leading to improved generalization.
RFRBoost uses random features to boost deep residual neural networks, improving performance and computational efficiency.
problem Improving performance of deep residual neural networks (RFNNs) while preserving convex optimization benefits.
method Random Feature Representation Boosting (RFRBoost) using boosting theory and random features at each layer.
result RFRBoost significantly outperforms RFNNs and end-to-end trained MLP ResNets in small- to medium-scale tabular datasets.
A faster graph kernel using optical random features.
problem High computation cost of graphlet kernel due to isomorphism test.
method Kernel random features, optical random features, mean kernel metric.
result The proposed method is orders of magnitude faster with similar or better accuracy.
New method improves feature importance assessment in random forests.
problem Improving feature importance measures for random forests.
method Hypothesis testing via self-normalized feature-residual correlation test (FACT).
result The method provides theoretically justified feature importance tests with controlled type I error and appealing power.
Gradient descent amplifies random features in neural networks to useful ones.
problem Generalization in neural networks trained on corrupted data.
method Characterization of feature-learning process in two-layer ReLU networks trained by gradient descent.
result Gradient descent amplifies random features to useful ones, achieving near optimal generalization error.
A new method for approximating softmax and Gaussian kernels with reduced error.
problem Approximating softmax and Gaussian kernels with low error.
method Simplex Random Features (SimRFs) and SimRFs+.
result SimRFs provide the smallest MSE among weight-independent geometrically-coupled PRF mechanisms.
Hermite polynomials improve private data generation by reducing feature count.
problem Infinite-dimensional features in kernel mean embedding are impractical for private data generation.
method Replace random features with Hermite polynomial features, leveraging their ordered nature.
result Hermite polynomial features yield a more accurate approximation of kernel mean embedding with fewer features.
Random feature matrices' singular values concentrate near their full expectation in high dimensions.
problem Characterizing the spectra of random feature matrices for regression problems.
method Analyzing two settings of input variables (random or well-separated) with conditions on dimension, complexity ratio, and sampling variance.
result The singular values of random feature matrices concentrate near their full expectation and near one with high probability.
HARFE approximates sparse additive functions using random features and ridge regression.
problem Approximating high-dimensional sparse additive functions.
method Hard-ridge random feature expansion with sparse ridge regression and hard-thresholding pursuit.
result HARFE method converges with a given error bound and achieves lower error than other algorithms.
Paper proposes a new method for training small models on regression problems.
problem Training small models for regression problems with noisy labels.
method Developed a new loss function and a multi-task network approach.
result Improved model accuracy on various datasets, consistent across different levels of annotation errors.
Study shows exponential convergence in classification errors using random features and SGD.
problem Scalability issues in kernel methods for large datasets.
method Binary classification problem with random features and stochastic gradient descent.
result Exponential convergence rate of expected classification error achieved.
Enhances random forest performance with exogenous randomness.
problem Improving random forest performance through exogenous randomness.
method Developed non-asymptotic MSE expansions for individual trees and forests, identified two types of randomness, and conducted simulations.
result Exogenous randomness, particularly feature subsampling, reduces both bias and variance of random forests.
Random Fourier features classification achieves fast learning rates with fewer features.
problem Improving classification efficiency with fewer features.
method Utilizing Lipschitz continuous loss functions and regularity conditions, the study reduces the number of features required for classification.
result Random Fourier features classification can achieve O ( 1 / n ) O(1/\sqrt{n}) O ( 1/ n ) learning rate with only Ω ( n log n ) Ω(\sqrt{n} \log n) Ω ( n log n ) features.