PhaseDNN speeds up learning of high-dimensional functions across wide frequencies.
problem Training high-dimensional functions at wide frequencies is slow and inefficient.
method Parallel DNNs with frequency-specific training and phase shifts.
result PhaseDNN achieves uniform learning across wide frequencies.
Study shows how neural network learning rates vary with function frequency.
problem Understanding how neural networks learn functions of different frequencies.
method Approximated neural network dynamics with a linear system, analyzed eigenfunctions and eigenvalues.
result A shallow neural network without bias cannot learn low frequency functions with odd frequencies.
Deeper neural networks learn lower frequency functions faster, according to a new principle.
problem Understanding why deeper learning is faster.
method Fourier analysis and filtering method to separate and analyze the frequency distribution of neural network outputs.
result Deeper hidden layers in neural networks bias towards lower frequency functions during training.
Deep neural networks often fit low-frequency functions, contrary to conventional numerical schemes.
problem Understanding the implicit bias of deep neural networks in fitting training data.
method Fourier analysis perspective applied to DNNs training process.
result Deep neural networks tend to fit training data by low-frequency functions, contrary to conventional numerical schemes.
Deep networks often capture low frequency functions, improving generalization.
problem Understanding deep learning's generalization ability.
method Showed F-Principle holds for various loss functions and applied it to differential equations.
result Deep networks capture low frequency functions, leading to better generalization.
Proves monotonicity of parabolic frequency on all manifolds without curvature assumptions.
problem Monotonicity of parabolic frequency on manifolds.
method Analyzes parabolic frequency function on manifolds, proving monotonicity without curvature assumptions.
result Monotonicity of parabolic frequency on all manifolds, no curvature assumption needed.
Study on parabolic frequency monotonicity on compact manifolds.
problem Analyzing solutions of the heat equation on Riemannian manifolds.
method Generalization of R. Hamilton's matrix Harnack inequality for small time.
result Parabolic frequency functional is almost increasing on compact manifolds with nonnegative sectional curvature.
Study growth rates of harmonic functions on curved surfaces.
problem Understanding the growth rates of harmonic functions on curved surfaces.
method Gradient estimate and frequency analysis on complete surfaces and manifolds with non-negative curvature.
result Existence and properties of nonconstant polynomial growth harmonic functions on manifolds with maximal volume growth.
Proposes MscaleDNN for solving high-dimensional PDEs efficiently.
problem Solving high-dimensional PDEs efficiently.
method Radial scaling in frequency domain and compact support activation functions.
result Increased power in multi-scale resolution and high frequency capturing.
Generative networks improve fluid simulation quality by focusing on high frequencies.
problem Low-frequency details missing in fluid simulation reconstructions.
method Frequency-aware loss function for generative networks.
result Improved perceptual quality of fluid simulation results in mid-frequency bands.
A new model of V1 using orientation, frequency, and phase.
problem Understanding the complex behavior of V1 simple cells.
method Developed a sub-Riemannian model based on Gabor functions.
result The model enhances images using orientation, frequency, and phase.
Deep neural networks are biased towards low frequencies, affecting global behavior.
problem Understanding the limitations of neural networks in capturing high-frequency patterns.
method Using Fourier analysis, the study examines the spectral bias of neural networks and their expressivity.
result Deep ReLU networks are biased towards low frequency functions, making it difficult to capture local fluctuations.
A new search-control strategy improves Dyna's efficiency.
problem Improving sample efficiency in model-based reinforcement learning.
method Proposes a novel search-control strategy by sampling high frequency regions of the value function.
result Empirically shows that high frequency regions require more samples to approximate, suggesting a better search-control strategy.
Study uses multi-kernel Hawkes models to analyze high-frequency price dynamics.
problem Understanding responsive speeds of market participants in high-frequency trading.
method Multi-kernel Hawkes models with conditional Hessian analysis for optimization.
result Existence of multi-kernels (UHF, VHF, HF) in high-frequency price dynamics.
Informer model with GMADL loss outperforms benchmarks in high frequency Bitcoin trading.
problem Developing automated trading strategies for high frequency Bitcoin data.
method Informer architecture with RMSE, GMADL, and Quantile loss functions.
result Informer model with GMADL loss function outperforms benchmarks in trading outcomes.
Estimates chirp signal frequencies using probabilistic models.
problem Estimating instantaneous frequencies of chirp signals when true forms are unknown.
method Non-linear Gaussian processes and stochastic filters/smothers for posterior estimation.
result The method outperforms state-of-the-art methods on synthetic and real-world datasets.
Spectral Convolution Networks speed up computation by applying convolution and activation in the frequency domain.
problem Performance increase in convolution networks comes with repeated transform computations.
method Implement convolution and activation in the frequency domain using Fourier or Laplace transformations.
result Reduced number of transforms and overall complexity by computing both convolution and activation in the frequency domain.
Study non-parametric frequency-domain system identification from finite samples.
problem Frequency-domain system identification from limited data.
method Empirical Transfer Function Estimate (ETFE) under sub-Gaussian colored noise and stability assumptions.
result ETFE estimates are concentrated around true values with a finite-sample rate of Ntot−1/3 for all frequencies in the H∞ norm. New CFNN architecture approximates functions with machine accuracy.
problem Function approximation with high precision.
method Chebyshev Feature Neural Network (CFNN) with learnable frequencies.
result Achieves machine accuracy in function approximation.
A new WNN framework selects wavelet bases for efficient learning.
problem Challenges in constructing accurate wavelet bases and high computational costs in WNN.
method Introduces a constructive WNN that selects initial bases and trains functions by introducing new bases for predefined accuracy while reducing computational costs.
result Significantly improves computational efficiency through a frequency estimator and wavelet-basis increase mechanism.
A new method integrates Fourier basis expansion and mapping for improved time series forecasting.
problem Inconsistent starting cycles and series length issues in Fourier-based methods.
method Fourier Basis Mapping (FBM) method that integrates time-frequency features through Fourier basis expansion and mapping.
result FBM addresses inconsistencies and preserves temporal characteristics, achieving SOTA performance.
Frequency bias affects neural network training on non-uniform data.
problem Understanding how frequency bias impacts neural networks trained on non-uniformly distributed data.
method Used the Neural Tangent Kernel (NTK) model to explore the effect of variable density on training dynamics.
result Convergence time for learning a pure harmonic function depends on the local density at a point.
The paper rigorously investigates the Frequency Principle in deep neural networks.
problem Understanding the training dynamics of deep neural networks.
method Theoretical investigation of Frequency Principle at three stages of training.
result Theorem providing quantitative understanding of Frequency Principle for general DNNs.
Paper proposes a new loss function to improve image reconstruction quality.
problem Blurred images when using pixel loss for convolutional autoencoders.
method Introduces spatial frequency loss (SFL) to mitigate blurring.
result Reduced blurs in reconstructed images using SFL.
SSMs have a built-in bias towards low-frequency components, which can be adjusted.
problem Frequency bias in SSMs affects their performance on long-range sequences.
method Proposed two mechanisms to tune frequency bias: scaling initialization or applying a Sobolev-norm-based filter.
result Tuning frequency bias improves SSMs' performance on long-range sequence learning tasks.
Study shows neural networks learn low frequencies first, proposing solutions.
problem Frequency bias in neural network learning process.
method Developed a PDE to unravel frequency dynamics, used Fourier Features model.
result Appropriate weight initialization can eliminate or control frequency bias.
Study sharp asymptotic estimates for conical ends using frequency functions.
problem Proving sharp asymptotic estimates for almost eigenfunctions of drift Laplacians on conical ends.
method Used a weighted variant of Almgren's frequency functions.
result Obtained a purely elliptic proof of uniqueness of self-shrinkers and self-expanders of the mean curvature flow.
FMMNN combines sine activations with multi-component, multi-layer structure for high-frequency function approximation.
problem Effective representation and learning of high-frequency features in neural networks.
method Introduces FMMNN with sine-type activations and multi-component, multi-layer structure.
result FMMNN achieves strong accuracy and favorable convergence on oscillatory function-approximation benchmarks.
Study describes frequencies of geodesics on hyperbolic surfaces as genus grows.
problem Large genus asymptotic behaviors of geodesic frequencies on hyperbolic surfaces.
method Proof of conjecture involving separating and nonseparating geodesics.
result Explicit function $f( rac{n}{g})$ for frequency ratio given.
Researchers reveal how neural networks implicitly favor low-frequency functions.
problem Why deep neural networks generalize well despite having more parameters than samples.
method Proposed a linear F-Principle dynamics to predict and explain the learning of two-layer ReLU NNs.
result Explicitly penalizing higher frequencies in the optimization process improves generalization.
Optical DNNet boosts accuracy with multiple frequency-channels.
problem Improving the accuracy of optical neural networks.
method Developed a novel optical diffractive deep neural network with multiple frequency-channels.
result Multiple frequency-channels significantly increase network accuracy.
New TVBO algorithm optimizes time-varying functions with varying sampling frequencies.
problem Optimizing time-varying, expensive, noisy functions with constant frequency assumption.
method Formulated practical recommendations and derived upper regret bound for varying sampling frequencies.
result BOLT algorithm outperforms state-of-the-art TVBO algorithms in experiments.
Studies in recent years have demonstrated that neural organization and structure impact an individual's ability to perform a given task. Specifically, individuals with greater neural efficiency have been shown to outperform those with less organized functional structure. In this work, we compare the predictive ability …
A new approach to reinforcement learning improves policy performance by adjusting control frequency.
problem Improving reinforcement learning performance by optimizing control frequency.
method Introducing action persistence and a novel algorithm, PFQI, to learn optimal value function at a given persistence.
result PFQI effectively learns optimal value function with action persistence, improving reinforcement learning performance.
Trains a neural network to predict high-frequency trading outcomes.
problem Predicting the fill probability function for high-frequency trading.
method High-quality high-frequency data and neural network training with a weighted loss function.
result Strong state dependence properties of the fill probability function.
DNNs initially capture low-frequency components before high-frequency ones, a phenomenon called F-Principle.
problem Understanding why DNNs generalize well despite overfitting.
method Empirical study on real and synthetic datasets, focusing on frequency components captured by DNNs.
result DNNs capture dominant low-frequency components first, then high-frequency ones, a phenomenon called F-Principle.
The paper clusters stocks using high-frequency NSE data, identifying IT and banking sectors.
problem Describing joint behavior of stocks beyond regression and correlation.
method Applied Kernel Principal Component Analysis (KPCA) and Functional Principal Component Analysis (FPCA) to high-frequency data.
result Identified two prominent clusters: IT and banking sectors, with smaller clusters from automobile and energy sectors.
A compact Riemannian manifold may be immersed into Euclidean space by using high frequency Laplace eigenfunctions. We study the geometry of the manifold viewed as a metric space endowed with the distance function from the ambient Euclidean space. As an application we give a new proof of a result of Burq-Lebeau and othe…
CNNs use a bottleneck structure to focus on a few frequencies, affecting function representation.
problem Understanding how CNNs focus on specific frequencies in their feature learning.
method Defined Convolution Bottleneck (CBN) structure, measured CBN rank, and analyzed parameter norms.
result Parameter norm scales with depth and CBN rank, and networks with optimal parameters exhibit this structure.
We consider globally hyperbolic flat spacetimes in 2+1 and 3+1 dimensions, in which a uniform light signal is emitted on the r-level surface of the cosmological time for r→0. We show that the frequency of this signal, as perceived by a fixed observer, is a well-defined, bounded function which is generally not co…
SPGD improves adversarial training efficiency and accuracy.
problem Improving adversarial training efficiency and accuracy with fewer steps.
method Adversarial-sample generation from a frequency domain perspective, extending PGD to the frequency domain.
result SPGD achieves greater adversarial accuracy compared to PGD with fewer attack steps.
Quantum models can approximate any function if data encoding allows for a rich enough frequency spectrum.
problem Theoretical properties of quantum machine learning models, particularly their expressive power.
method Investigated how data encoding affects the expressive power of parametrized quantum circuits.
result Quantum models can access increasingly rich frequency spectra by repeating data encoding gates, potentially making them universal function approximators.
New harmonic functions show nodal sets can be topologically complex despite frequency and regularity constraints.
problem Understanding the topology of nodal sets of harmonic functions with bounded frequency and regularity.
method Constructing harmonic functions on the unit ball with specific properties.
result The Betti numbers of the nodal set can be arbitrarily large, contradicting previous topological bounds.
Proposes AWS method for precise speech enhancement using DNN.
problem T-F resolution problem in fixed-resolution short-time frequency transforms.
method Incorporates trainable adaptive window switching into speech enhancement procedure.
result Achieved higher signal-to-distortion ratio than conventional methods.
Log-concavity proven for multinomial likelihoods under specific constraints.
problem Log-concavity of multinomial likelihoods under interval censoring constraints.
method Proved log-concavity by showing M-convex subsets of the discrete simplex.
result Likelihood function is completely log-concave.
HFformer outperforms LSTM in high-frequency trading with multiple signals.
problem Improving high-frequency trading performance using deep learning models.
method Introducing HFformer, a hybrid Transformer model for time series forecasting.
result HFformer achieves higher cumulative PnL than LSTM in backtesting.
VAEs analyzed using harmonic analysis, showing how variance controls frequency content and robustness.
problem Understanding and optimizing VAEs for robustness and frequency control.
method Viewing VAE latent space as Gaussian space, deriving results on variance and frequency content, and demonstrating soft Lipschitz constraints.
result Increasing encoder variance reduces high frequency content and improves adversarial robustness.
Much work has been done on feature selection. Existing methods are based on document frequency, such as Chi-Square Statistic, Information Gain etc. However, these methods have two shortcomings: one is that they are not reliable for low-frequency terms, and the other is that they only count whether one term occurs in a …