Paper develops a new model for predicting volatility surface.
problem Predicting volatility in financial markets is challenging due to its non-observable nature and complex dynamics.
method Physics-informed convolutional transformer architecture.
result The new model outperforms other deep-learning architectures in predicting volatility surface.
Study combines CNNs and LSTMs for ECG classification, improving performance with attention mechanisms.
problem Improving ECG classification accuracy for weakly supervised tasks.
method Investigates ConvLSTM networks and various attention mechanisms to capture both morphological and temporal features.
result Attention mechanisms improve classification performance and provide interpretable saliency maps.
Paper develops a new model for forecasting ocean currents.
problem Forecasting the Loop Current and its eddies for the Gulf of Mexico.
method Physics-informed Tensor-train ConvLSTM, incorporating prior physical knowledge.
result PITT-ConvLSTM outperforms state-of-the-art methods in volumetric velocity forecasting.
Accurate taxi demand-supply forecasting is a challenging application of ITS (Intelligent Transportation Systems), due to the complex spatial and temporal patterns. We investigate the impact of different spatial partitioning techniques on the prediction performance of an LSTM (Long Short-Term Memory) network, in the con…
As a crucial component in intelligent transportation systems, traffic flow prediction has recently attracted widespread research interest in the field of artificial intelligence (AI) with the increasing availability of massive traffic mobility data. Its key challenge lies in how to integrate diverse factors (such as te…
DISTANA predicts and denoises spatial wave dynamics.
problem Identifying causality in spatially distributed, non-linear dynamical processes.
method Generative, recurrent graph convolution neural network architecture (DISTANA).
result DISTANA outperforms alternative approaches in denoising and predicting complex spatial wave propagation.
New model predicts citywide air pollution using deep learning.
problem Predicting air quality for entire cities is challenging due to spatiotemporal factors.
method Convolutional Long Short-Term Memory (ConvLSTM) model for spatiotemporal data.
result ConvLSTM model outperforms other methods in predicting citywide air pollution.
Paper proposes a framework for precise daily default risk prediction of Chinese credit bonds.
problem Inadequate and inaccurate bond information disclosure creates risk of default for investors.
method Framework includes summarizing factors impacting defaults, constructing a risk index system, and using ConvLSTM neural network for prediction.
result The model provides more responsive and accurate daily default risk predictions than authoritative ratings.
Paper proposes a novel approach to improve spatiotemporal precipitation forecasts.
problem Improving accuracy of spatiotemporal precipitation forecasts for flood damage mitigation.
method Introduces a rain-code fusion approach using ConvLSTM and multi-frame fusion for spatiotemporal precipitation code-to-code forecasting.
result Demonstrates enhanced accuracy in precipitation forecasts beyond 3 timesteps using the rain-code fusion.
Proposes ACLAE-DT for unsupervised anomaly detection in multivariate time series.
problem Challenges in building anomaly detection frameworks for multivariate time series data.
method Attention-based ConvLSTM Autoencoder with Dynamic Thresholding.
result Demonstrates superior performance over state-of-the-art methods.
Predicts financial asset dependencies using spatiotemporal patterns.
problem Complex dependency structures in financial assets for risk mitigation.
method Proposes Asset Dependency Matrix (ADM) and Asset Dependency Neural Network (ADNN) with ConvLSTM for spatiotemporal asset dependency prediction.
result ADNN outperforms baselines in predicting asset dependencies and their applications.
Spatiotemporal sequence prediction is an important problem in deep learning. We study next-frame(s) video prediction using a deep-learning-based predictive coding framework that uses convolutional, long short-term memory (convLSTM) modules. We introduce a novel reduced-gate convolutional LSTM(rgcLSTM) architecture that…
Traffic flow prediction is crucial for urban traffic management and public safety. Its key challenges lie in how to adaptively integrate the various factors that affect the flow changes. In this paper, we propose a unified neural network module to address this problem, called Attentive Crowd Flow Machine~(ACFM), which …
Taxi demand prediction has recently attracted increasing research interest due to its huge potential application in large-scale intelligent transportation systems. However, most of the previous methods only considered the taxi demand prediction in origin regions, but neglected the modeling of the specific situation of …
Nowadays, multivariate time series data are increasingly collected in various real world systems, e.g., power plants, wearable devices, etc. Anomaly detection and diagnosis in multivariate time series refer to identifying abnormal status in certain time steps and pinpointing the root causes. Building such a system, how…
Graphs of neural networks are represented to preserve symmetry, improving performance across various tasks.
problem Lack of equivariance in neural network representations of other neural networks.
method Represent neural networks as computational graphs and use graph neural networks to preserve permutation symmetry.
result Single model encodes diverse neural architectures, outperforming state-of-the-art methods.
Neural networks can approximate functions uniformly across various measures.
problem Universal approximation of functions across different probability measures.
method Proving neural networks are dense in Orlicz spaces, extending classical theorems.
result Neural networks uniformly approximate functions for weakly compact families of measures.
Optimal rates for shallow ReLU networks in nonparametric regression.
problem Approximating smooth and non-smooth functions with shallow ReLU networks.
method Analysis of shallow ReLUk neural networks, using variation norms and deep learning theory. result Optimal approximation rates for shallow ReLU networks in nonparametric regression.
Novel framework explains generalization in deep neural networks.
problem Understanding and improving generalization in deep neural networks.
method Topological Quantum Neural Networks as the semi-classical limit of Deep Neural Networks.
result Demonstrates that the perceptron, viewed as the semi-classical limit, achieves similar results to standard neural networks without training.
Investigates how neural network graph structure impacts predictive performance.
problem Lack of understanding between neural network graph structure and predictive performance.
method Developed relational graph representation to analyze neural networks, identifying a 'sweet spot' for improved performance.
result Identified a 'sweet spot' in relational graph structure that significantly improves neural network predictive performance.
Equivariant neural networks use symmetry to interpret complex data.
problem Interpreting and understanding the behavior of equivariant neural networks.
method Decompose layers into simple representations and analyze nonlinear activation functions.
result Equivariant neural networks can be interpreted using a filtration generalizing Fourier series.
Two new criteria help understand the advantage of deep neural networks.
problem Understanding the advantage of deepening neural networks.
method Proposed two new criteria to evaluate the expressivity of functions computable by deep neural networks.
result Increasing layers is more effective than increasing units in improving the expressivity of deep neural networks.
Paper benchmarks quantum neural networks against classical ones for binary classification tasks.
problem Comparing quantum neural networks with classical ones for binary classification.
method Evaluated with two toy examples, focusing on model complexity and training data size.
result EQNN and QNN outperform ENN and DNN for smaller parameter sets and training data samples.
Secret neural networks hidden within trained models.
problem Excess capacity in neural networks allows embedding secret models.
method Novel framework for hiding secret neural networks within carrier networks.
result Detection of hidden networks is computationally infeasible.
This paper shows overparameterized deep neural networks are convex and learn useful features.
problem Analyzing fully trained overparameterized deep neural networks.
method Generalized neural feature repopulation technique.
result Overparameterized deep neural networks are inherently convex and learn useful features.
The paper proves consistency of neural networks with regularization.
problem Overfitting in neural networks with large scale data.
method Theoretical framework of neural networks with regularization, sieves method, and minimal neural networks theory.
result The estimated neural network converges to the true underlying function as sample size increases.
New proof shows neural networks can represent all multivariate functions.
problem Representing all multivariate functions with neural networks.
method Proved that three-layer neural networks can represent both continuous and discontinuous functions.
result Three-layer neural networks can represent all multivariate functions, including discontinuous ones.
New learning rules for wide neural networks without backpropagation.
problem Training wide neural networks efficiently and without backpropagation.
method Input-weight alignment driven by gradient descent in the NTK regime.
result Biologically-motivated learning rules equivalent to backpropagation in wide networks.
DeepWeightFlow generates diverse neural network weights efficiently.
problem Generating complete neural network weights efficiently and accurately.
method Flow Matching in weight space with Git Re-Basin and TransFusion.
result DeepWeightFlow generates high-accuracy neural networks without fine-tuning.
Neural networks improve nonparametric regression with measurement errors.
problem Nonparametric regression with measurement errors.
method Proposes a neural network design using FNN, normalizing flow, and inference network.
result Neural network approach is more flexible and superior or comparable to classical methods.
Graph Metanetworks process diverse neural architectures efficiently.
problem Processing diverse neural architectures efficiently.
method Builds metanetworks using graph neural networks to process graphs representing input neural networks.
result Proves GMNs are expressive and equivariant to parameter permutation symmetries.
A powerful network teaches a weak one, improving its performance.
problem Improving the performance of a weak neural network using a more powerful one.
method During training, a weak network learns features from a strong network to minimize feature distance.
result A weak neural network can increase its performance by learning from a more powerful network.
Neural networks can model chaos efficiently by becoming geometrically chaotic.
problem Lack of theoretical understanding of how neural networks learn chaos.
method Employed a geometric perspective to show neural networks can model chaotic dynamics.
result Neural networks can reconstruct strange attractors and accurately predict local divergence rates.
Affine spiking neural networks learn efficiently and generalize well.
problem Learning with spiking neural networks, especially with positive weights.
method Affine encoders and decoders, continuous parameter dependence, gradient-based training.
result Affine spiking neural networks can approximate shallow ReLU networks and generalize well.
Unified platform SOCRATES for neural network analysis.
problem Analyzing neural networks for bugs and fairness.
method Standardized format, assertion language, and multiple analysis algorithms.
result Unified platform for neural network analysis.
Explains equivariant neural networks for machine learning.
problem Understanding equivariance in neural networks.
method Simple mathematical treatment of neural network concepts.
result Clarifies the mathematical basis of equivariant neural networks.
Research on neural networks has gained significant momentum over the past few years. Because training is a resource-intensive process and training data cannot always be made available to everyone, there has been a trend to reuse pre-trained neural networks. As such, neural networks themselves have become research data.…
DCNs mimic neuronal networks for improved neural classification.
problem Lack of topological similarity between DNNs and biological neural networks.
method Developed DCNs with topologies inspired by real-world neuronal networks.
result High classification accuracy achieved by DCNs.
Neural Tangents simplifies infinite-width neural networks for research.
problem Training and studying infinite-width neural networks.
method High-level API for specifying complex architectures, analytical or gradient-based training, and automatic distribution.
result Analytical training of infinite-width networks and automatic parallelization.
Bayesian Neural Networks improve uncertainty estimation in deep learning.
problem Lack of robustness and sensitivity to out-of-distribution samples in DNNs.
method Empirical evaluation of Bayesian Neural Networks against point estimate DNNs.
result Bayesian Neural Networks provide better uncertainty quantification and performance.
Neural networks are mathematically represented via quiver representations.
problem Understanding how neural networks process data and create representations.
method Representing neural networks as quiver representations with activation functions.
result Neural networks' computations can be studied algebraically and geometrically.
Adapting robust statistics to neural networks, researchers found neural networks can be more robust with certain loss functions.
problem The robustness of neural networks in complex learning tasks.
method Adapting the regression breakdown point from robust statistics to neural networks and comparing different configurations and contamination settings.
result Neural networks can benefit from robust loss functions, as demonstrated in extensive simulations.
Paper introduces a noise-robust classification method using hypergraph neural networks.
problem Noisy label learning problem in image datasets.
method PCA for dimensionality reduction, then applies graph-based semi-supervised learning methods including hypergraph neural network.
result Our proposed hypergraph neural network achieves the best performance when noise level increases.
Paper introduces new neural network models and theories.
problem Understanding neural networks beyond over-parameterized regime.
method Develops two exact models and a novel representor theory.
result Provides insights into neural network training and kernel evolution.
New neural networks combine additive regression with traditional architectures.
problem Performance limitations and high parameter requirements of traditional neural networks.
method Introduce hybrid deep additive neural networks with simpler activation and basis functions.
result Hybrid neural networks achieve better performance with fewer parameters.
This paper analyzes the limits of neural networks under adversarial attacks.
problem Understanding how neural networks perform under adversarial noise.
method Defined the adversarial VC-dimension and combined it with the VC-dimension of halfspace classifiers.
result Concluded the adversarial VC-dimension of neural networks with sign activation functions.
QRPNNs use quaternion-valued recurrent correlation neural networks to solve cross-talk issues.
problem Cross-talk problem in QRCNNs.
method Combining non-local projection learning with QRCNNs.
result QRPNNs exhibit greater storage capacity and noise tolerance.
New neural networks learn distribution functions using quantiles and moments.
problem Approximating functions of distributions in probability spaces.
method Quantile and moment neural networks, mixing quantile and moment features.
result Moment neural network outperforms others for bivariate distributions.