Improved electrical load forecasting model using Fourier-enhanced RNN.
problem Electrical load time series downscaling with high accuracy and low error.
method Combines recurrent neural network with Fourier seasonal embeddings and self-attention.
result Significantly reduces RMSE across different time horizons compared to existing methods.
DeepPhaseCut uses neural networks to improve Fourier phase retrieval.
problem Fourier phase retrieval from magnitude data.
method Unsupervised feed-forward neural network with cycleGAN training.
result Outperforms existing methods in Fourier phase retrieval.
FNN approximates functions and solves PDEs with periodic BCs.
problem Approximating and solving periodic functions and PDEs.
method Fourier neural network architecture with activation and loss functions.
result FNN can solve PDEs with periodic BCs and is interpretable.
RNNs solve modular addition tasks using low rank and sparse Fourier structures.
problem Solving modular addition tasks with recurrent neural networks.
method Identified low rank structures and sparse Fourier representations in RNN weights.
result RNNs robust to removing individual frequencies but degrade with more ablation.
Equivariant neural networks use symmetry to interpret complex data.
problem Interpreting and understanding the behavior of equivariant neural networks.
method Decompose layers into simple representations and analyze nonlinear activation functions.
result Equivariant neural networks can be interpreted using a filtration generalizing Fourier series.
Combines Fourier methods and RNNs for efficient time series prediction.
problem Efficiently processing and predicting time series data with memory and computational constraints.
method Uses short-time Fourier transform and weight reductions through low pass filtering in a Spectral RNN.
result Predicts time series data from chaotic systems and real-world data.
Post-averaging improves neural network robustness against adversarial attacks.
problem Adversarial attacks on neural networks.
method Post-averaging technique to smooth high frequency components.
result Post-averaging method successfully defends over 95% of adversarial samples without significant performance degradation.
New bounds on ReLU networks for low-regular functions.
problem Bounding approximation error for ReLU networks on low-regular functions.
method Complexity analysis of Fourier features residual networks to ReLU networks.
result Approximation error bound proportional to target function norm and inversely proportional to network width and depth.
Study extends neural network approximation to time-varying PDEs using Fourier-Lebesgue spaces.
problem Limitation to static PDEs and different time-domain regularity.
method Extend spectral Barron spaces to anisotropic weighted Fourier-Lebesgue spaces, measure approximation error in Bochner-Sobolev norm.
result Established bound on approximation rate for functions in anisotropic weighted Fourier-Lebesgue spaces.
Random SNNs are stable and simple, with low-frequency Fourier spectra.
problem Stability and robustness of spiking neural networks.
method Boolean function analysis and Fourier spectrum concentration.
result Random LIF-SNNs are stable and biased towards simple functions.
FourNet approximates financial transition densities using Fourier transforms.
problem Approximating transition densities in finance with high accuracy.
method FourNet is a novel FFNN with Gaussian activation, learning from characteristic functions.
result FourNet can approximate transition densities arbitrarily well with finite neurons.
Simpler neural network for spherical images using Clebsch-Gordan transforms.
problem Learning spherical images rotation invariantly.
method Clebsch-Gordan transform for nonlinearity, avoiding repeated Fourier transforms.
result Improved performance compared to previous methods.
GWNN uses graph wavelets for efficient graph CNNs.
problem Spectral graph CNNs' high computational cost and lack of interpretability.
method Graph wavelet transform for efficient graph convolution.
result GWNN significantly outperforms spectral graph CNNs.
Improved sampling strategy reduces Fourier measurements for neural network signals.
problem Efficiently sampling signals from neural networks with random Fourier matrices.
method Model-adapted sampling strategy with improved sample complexity.
result Reduced sample complexity from O(kdnα∞²) to O(kdα²₂) measurements.
Method uses neural networks to fit nonlinear operators from data.
problem Finding nonlinear integro-differential operators from data.
method Parametrizes spatial operator with neural networks and Fourier transforms.
result Can recover spatial operators in fractional heat and Kuramoto-Sivashinsky equations.
Unified method for deriving ridgelet transforms for various neural network architectures.
problem Deriving closed-form expressions for ridgelet transforms in modern neural network architectures.
method Unified Fourier slice method to derive ridgelet transforms for diverse neural network types.
result Systematic method to derive ridgelet transforms for various neural network architectures.
A neural network method estimates densities from characteristic functions.
problem Estimating fixed-horizon probability densities from empirical characteristic functions.
method Data-driven Fourier-mixture neural-network method trained in Fourier space.
result Competitive performance and clear gains on heavy-tailed targets.
Deep neural networks often fit low-frequency functions, contrary to conventional numerical schemes.
problem Understanding the implicit bias of deep neural networks in fitting training data.
method Fourier analysis perspective applied to DNNs training process.
result Deep neural networks tend to fit training data by low-frequency functions, contrary to conventional numerical schemes.
New PINN architectures learn high-frequency features using Fourier features.
problem PINNs struggle with high-frequency or multi-scale features.
method Employ spatio-temporal and multi-scale random Fourier features.
result Effective PINN models for multi-scale PDEs.
Deep neural network estimates support of sparse signals for improved phase retrieval.
problem Sparse phase retrieval from Fourier magnitudes with support estimation.
method Trained deep neural network (DNN) provides extended support estimate E larger than the support T. result DNN-based support estimation improves signal reconstruction performance with lower complexity.
Study explores how neural networks and Transformers learn modular arithmetic with multiple inputs.
problem Understanding how neural networks and Transformers learn modular arithmetic with multiple inputs.
method Analytical characterization of features learned by neural networks and Transformers, focusing on margin maximization and Fourier spectra.
result Neural networks and Transformers require a minimum neuron count of \( m \geq 2^{2k-2} \cdot (p-1) \) to solve modular addition problems with \( k \) inputs and modulus \( p \).
FNSDA adapts to new dynamics via Fourier space adaptation.
problem Generalizing to unseen dynamical systems with limited data.
method Automatic partitioning of known environments in Fourier modes and adaptation of specific modes for new environments.
result FNSDA achieves superior or competitive generalization performance with reduced parameter cost.
New algorithm trains deep neural networks without global optimization.
problem Training deep neural networks efficiently and without global optimization.
method Uses random complex exponential activation functions and Markov Chain Monte Carlo sampling.
result Consistently attains theoretical approximation rate for residual networks.
ARFF reduces spectral bias in SGD-trained neural networks.
problem Spectral bias in two-layer neural networks.
method Comparison of SGD and ARFF on spectral bias and robustness.
result ARFF yields a closer to zero spectral bias compared to SGD.
GNIs induce a regulariser that penalizes high-frequency components in neural network activations.
problem Understanding the regularizing effect of Gaussian noise injections on neural network activations.
method Deriving the explicit regularizer by marginalizing out injected noise and analyzing its effect in the Fourier domain.
result GNIs induce a regularizer that produces calibrated classifiers with large margins.
A new CNN-based algorithm improves Fourier ptychography for faster, more robust image reconstruction.
problem Slow and inefficient Fourier ptychography reconstruction under system aberrations.
method A CNN-based iterative phase retrieval algorithm trained on GPUs.
result Significantly faster and more robust image reconstruction under system aberrations.
Neural networks compress and sample WDN contamination dynamics efficiently.
problem Infrastructure monitoring of complex, networked systems like water distribution networks is expensive and challenging.
method Developed Graph Fourier Transform (GFT) operators and neural networks (NN) for efficient data collection and inference.
result High accuracy reconstruction of contamination dynamics using only 5-10% of the sample set.
This paper analyzes SHAP values using Fourier expansions for model interpretability.
problem Understanding and interpreting SHAP values in complex models.
method Developed a spectral framework using Fourier expansions for SHAP values in various model regimes.
result SHAP values are Lipschitz continuous in the deterministic regime and converge to Gaussian process values in the probabilistic regime.
HFNO enhances interpretability of turbulent flows through parallel wavenumber bin processing.
problem Opaque inner workings of Fourier Neural Operators (FNOs) hinder physical interpretability.
method Introduces HFNO, a novel FNO-based architecture that processes wavenumber bins in parallel, enhancing interpretability.
result HFNO decomposes turbulent flows across various scales, enabling increased interpretability and multiscale modeling.
Fourier Neural Operators accurately predict dynamics of high-dimensional ionic models.
problem Approximating stiff, multiscale ionic models using neural networks.
method Fourier Neural Operators for learning dynamics of high-dimensional ionic models.
result Fourier Neural Operators can accurately predict dynamics of high-dimensional ionic models.
STFNets learn signals from time-frequency perspective, outperforming traditional models.
problem Learning signals from IoT data with better features in the frequency domain.
method Integrates Short-Time Fourier Transform into neural networks for direct frequency domain feature learning.
result Significantly outperforms state-of-the-art models in various experiments.
Neural networks learn simpler features first, then more complex ones; Fourier analysis reveals this pattern.
problem Understanding the learning dynamics of neural networks, especially with natural image data.
method Fourier analysis of translation-invariant and power-law spectra to study feature learning.
result Simple neural networks first rely on amplitude information, then phase information, and power-law spectra can accelerate learning phase information.
Neural networks learn spectral representations for group composition.
problem Understanding structured emergence in neural network training.
method Lifting gradient flow to Fourier domain, proving convergence to irreducible representations.
result Neurons converge to single irreducible representations, cross-layer coefficients align.
Novel neural network layer improves long-range interactions in point clouds.
problem Efficiently treating long-range interactions in point clouds.
method Long-range convolutional (LRC)-layer that incorporates Fourier transforms.
result Global all-to-all convolution operation performed in nearly-linear time.
New neural network improves MRI reconstruction for non-Cartesian data.
problem Improving MRI reconstruction for non-Cartesian data acquisitions.
method Density-compensated unrolled neural networks.
result Density-compensated unrolled neural networks outperform baselines.
This study shows how optimizer choice affects adversarial robustness in neural networks.
problem Understanding and improving adversarial robustness in neural networks.
method Revisiting known results linking robust classifiers and minimum norm solutions, combining them with recent optimizer bias findings.
result Achieving both perfect standard accuracy and robustness with certain optimizers under specific conditions.
Butterfly-Net improves CNN performance with structured connections and initialization.
problem Improving the performance of convolutional neural networks (CNNs).
method Butterfly-Net introduces structured and sparse cross-channel connections, and Butterfly initialization strategy.
result Butterfly-Net approximates Fourier representations with exponentially decaying error as depth increases.
Graph CNN method improves classification of irregular spatial data like building patterns.
problem Challenges in analyzing irregular spatial data with machine learning.
method Graph Fourier transform and convolution theorem to convert irregular spatial data into a learnable format.
result Significantly improved classification of building patterns compared to other methods.
Proposes FRU to stabilize gradients and improve long-term dependencies in RNNs.
problem Challenges in training RNNs for tasks with long-term dependencies.
method Introduces Fourier Recurrent Units (FRU) that stabilize gradients and improve expressivity.
result FRU stabilizes gradients and has stronger expressive power, leading to better performance.
Deep neural networks correct Mie scattering in FTIR spectra of biological samples.
problem Mie scattering obscures biochemically relevant spectral information in FTIR spectra of biological samples.
method Deep neural networks to approximate the preprocessing function that removes Mie scattering.
result The model is faster and more generalizable across different tissue types.
Study reveals how neural network biases align with adversarial attack frequencies.
problem Correlation between neural network biases and adversarial attacks.
method Fourier transform analysis of network implicit bias and adversarial perturbations.
result Network bias and adversarial attack frequencies are highly correlated.
Paper extends scattering network for fast, scalable feature extraction.
problem Efficiently extracting features from structured inputs.
method Derives a Fourier domain model for scattering network with higher order nonlinearities.
result Achieves linear time complexity for fast computations.
Generative model improves audio synthesis from TF features.
problem Challenges in generating high-quality audio from TF features.
method Used a GAN to generate invertible TF features from short-time Fourier transforms.
result Generated TF features from GAN outperformed direct waveform generation.
Study approximates operator learning for PDEs using Fourier multipliers.
problem Approximating operator behavior for PDE simulations.
method Approximation of operator symbols in Fourier domain using semi-norms.
result Identifies conditions for achieving predefined approximation error.
New model predicts time series quantiles for nonstationary data.
problem Nonparametric probabilistic forecasting of nonstationary univariate time series.
method Composite Quantile Fourier Neural Network (QFNN) for extrapolation-based nonlinear quantile regression.
result Effective in providing high quality and accurate probabilistic predictions.
Random Fourier features improve tabular deep learning convergence.
problem Tabular deep learning convergence issues.
method Random Fourier projections as a pre-processing step, projecting inputs into a fixed feature space.
result Random Fourier pre-processing accelerates tabular deep learning convergence.
UNOT solves optimal transport problems efficiently using neural networks.
problem Computational expense in solving optimal transport problems.
method UNOT (Universal Neural Optimal Transport) uses Fourier Neural Operators to predict OT distances and plans accurately and efficiently.
result UNOT achieves up to 7.4x speedup over the Sinkhorn algorithm while maintaining accuracy.
Deep neural networks are biased towards low frequencies, affecting global behavior.
problem Understanding the limitations of neural networks in capturing high-frequency patterns.
method Using Fourier analysis, the study examines the spectral bias of neural networks and their expressivity.
result Deep ReLU networks are biased towards low frequency functions, making it difficult to capture local fluctuations.