QRPNNs use quaternion-valued recurrent correlation neural networks to solve cross-talk issues.
problem Cross-talk problem in QRCNNs.
method Combining non-local projection learning with QRCNNs.
result QRPNNs exhibit greater storage capacity and noise tolerance.
Complex-valued neural networks are not a new concept, however, the use of real-valued models has often been favoured over complex-valued models due to difficulties in training and performance. When comparing real-valued versus complex-valued neural networks, existing literature often ignores the number of parameters, r…
Complex-valued neural networks avoid spurious local minima.
problem Finding spurious local minima in neural networks.
method Proved no spurious local minima for shallow complex neural networks with quadratic activations.
result Complex-valued weights eliminate spurious local minima in neural networks.
SVD improves neural network optimization.
problem Optimizing neural networks.
method Using SVD as an initial guess for neural network parameters.
result Better optimization results.
New method uses extreme value theory to estimate neural network errors.
problem Quantifying the error of neural networks, especially for large values.
method Applying extreme value theory to approximate the distribution of error.
result Developed a new estimator for the shape parameter of the Pareto distribution.
Bayesian sparsification improves complex-valued neural networks by 50-100x with minimal performance loss.
problem Efficiently compressing complex-valued neural networks for embedded systems.
method Extending Sparse Variational Dropout to complex-valued networks and conducting a numerical study.
result Achieved state-of-the-art performance on MusicNet with 50-100x compression.
Develops vector-valued RKBS for neural networks and operators.
problem Understanding function spaces of Rd-valued neural networks and neural operators. method Defines and constructs vector-valued RKBS (vv-RKBS) without restrictive assumptions.
result Establishes Representer Theorem for neural architectures.
Study on network-valued processes with asynchronous updates, proving consistency in community and changepoint estimation.
problem Understanding the behavior of network-valued stochastic processes with asynchronous updates.
method Analysis of concentration properties of aggregated adjacency and Laplacian matrices for lazy network-valued stochastic processes.
result Demonstrates consistency of estimators in community and changepoint estimation problems.
Extends RCNNs to handle hypercomplex-valued data.
problem Processing high-dimensional data.
method Develops mathematical background and conditions for hypercomplex-valued RCNNs.
result Hypercomplex-valued RCNNs can always settle at an equilibrium.
CVNN outperforms RVNN on non-circular data.
problem Classifying complex-valued data with statistical dependence.
method Comparison of CVNN and RVNN on non-circular data.
result CVNN outperforms RVNN in accuracy and generalization.
Study examines growth dynamics and trade-off between value and cost in evolving networks.
problem Understanding growth and trade-off in real-world networks.
method Investigates preferential attachment in temporal networks with modified BA model and differential equations.
result Illustrates future equilibrium of evolving networks and trade-off between value and cost.
Paper introduces vector-valued variation spaces for multi-output neural networks.
problem Understanding and optimizing multi-output neural networks.
method Development of vector-valued variation spaces and representer theorem.
result Novel bounds for layer widths in deep networks and a convex optimization method for compression.
We introduce a novel Deep Reinforcement Learning (DRL) algorithm called Deep Quality-Value (DQV) Learning. DQV uses temporal-difference learning to train a Value neural network and uses this network for training a second Quality-value network that learns to estimate state-action values. We first test DQV's update rules…
The logcosh loss function helps neural networks learn set-valued functions better.
problem Learning set-valued functions with neural networks.
method Using artificial neural networks with logcosh loss.
result Neural networks with logcosh loss can classify samples based on set-valued functions.
Researchers develop neural networks for approximating functions in Banach spaces.
problem Approximating Banach space valued continuous functions.
method Quasi-interpolation Banach space valued neural network operators using algebraic sigmoid functions.
result Jackson type inequalities for function approximation.
Graph neural network executes value iteration for flexible environments.
problem Value iteration in flexible environments with direct supervision.
method Graph Neural Network (GNN) executing value iteration algorithm.
result GNN accurately models value iteration across diverse environments.
Enhances neural networks with prior function values to improve accuracy.
problem Improving neural network accuracy in regions without training data.
method Develops a probabilistic approach to augment BNNs with prior function values.
result Predictions rely more on prior information in uncertain regions.
This paper proves a generalization bound for complex-valued neural networks scaling with spectral complexity.
problem Ensuring the performance of complex-valued neural networks on unseen data.
method Theoretical derivation using Maurey Sparsification Lemma and Dudley Entropy Integral, empirical validation on various datasets.
result The spectral complexity of weight matrices is a significant factor in the generalization ability of complex-valued neural networks.
UVU simplifies value uncertainty quantification in RL.
problem Estimating epistemic uncertainty in value functions for reinforcement learning.
method UVU uses squared prediction errors between an online learner and a fixed, randomly initialized target network, incorporating policy-conditional value uncertainty.
result UVU achieves equal performance to large ensembles on challenging offline RL settings, with computational savings.
Paper presents an efficient exploration method for reinforcement learning.
problem Efficient exploration in reinforcement learning with uncertainty quantification.
method Parameterized Indexed Value Function (PIV) using index sampling.
result Proves the regret bound for learning PIV in a tabular setting and proposes PINs for computational learning.
New graph learning model can approximate any function and handle edge values.
problem Graph learning models' limitations in approximating functions and handling edge values.
method Proposes a Graph Neural Network that can approximate any function and handle arbitrary edge values.
result Proves the model is strictly more expressive than existing models.
This research explores complex-valued neural networks and their implementation.
problem The challenges of implementing complex-valued neural networks and their potential for non-complex data.
method Detailed theory and implementation of CVNN, including Wirtinger calculus, complex backpropagation, and modules like complex layers and activation functions. Python implementation using cvnn toolbox.
result Demonstrates the potential of CVNN for non-complex data through simulations.
A new framework explains mixed models by propagating Shapley values.
problem Making complex models like neural networks and stacked models explainable for healthcare applications.
method DeepSHAP framework for layer-wise propagation of Shapley values.
result DeepSHAP enables attributions for mixed models and theoretically justifies attributions with respect to a background distribution.
New method selects edges in stock networks using multiple threshold values.
problem Balancing prominent correlations and network connectivity in stock networks.
method Uses multiple distributions in a maximum likelihood estimator for selecting threshold values.
result Proposed method develops networks with appropriate connectivities.
New theorem for deep neural networks improves classification margins.
problem Improving classification margins in deep neural networks.
method Local class-purity theorem and margin p-values for training and testing samples.
result Enhanced understanding and computation of classification margins.
FS-GCLSTM predicts stock returns by leveraging value-chain relationships.
problem Traditional time series models fail to capture complex interdependencies in modern markets.
method FS-GCLSTM integrates value-chain networks and graph convolutions to predict stock returns.
result FS-GCLSTM consistently delivers superior portfolio performance compared to traditional models.
Study shows XRP price correlates with transaction network metrics.
problem Understanding the relationship between cryptoasset price and network metrics.
method Analysis of correlation tensor spectra, random matrix theory comparison, singular values investigation.
result Distinct correlation between XRP price and singular values during bubble and non-bubble periods.
nUDEs use neural networks to model biology without negative values.
problem Unrealistic negative values in hybrid models of biology.
method Developed non-negative UDEs (nUDEs) with regularization techniques.
result nUDEs provide realistic solutions for biological models.
With globalization, countries are more connected than before by trading flows, which currently amount to at least 36 trillion dollars. Interestingly, approximately 30-60 percent of global exports consist of intermediate products. Therefore, the trade flow network of a particular product with high added values can be re…
Under-parameterization hinders deep RL's efficiency.
problem Implicit under-parameterization impairs data-efficiency in deep RL.
method Characterized and mitigated the rank collapse of value network features.
result Controlling rank collapse improves deep RL performance.
Proposes a method to constrain singular values of convolutional kernels in neural networks.
problem Avoiding exploding/vanishing gradient problems and improving generalizability in neural networks.
method Introduces a penalty function to constrain singular values of convolutional kernels around 1, and derives an algorithm for optimization.
result Demonstrates the effectiveness of the method through numerical examples.
A new Shapley value approach for neural networks interpretable and stable.
problem Neural networks' interpretability and training stability issues.
method Shapley value approximation for ReLU activation, globally continuous Shapley gradient, Shapley Activation function.
result SA consistently outperforms ReLU in training convergence, accuracy, and stability.
PPG separates policy and value function training phases for better reinforcement learning efficiency.
problem Challenges in traditional reinforcement learning methods for policy and value function optimization.
method Integrates Phasic Policy Gradient framework that splits policy and value function training into distinct phases.
result Significantly improves sample efficiency on Procgen Benchmark compared to PPO.
Deep learning has become an area of interest in most scientific areas, including physical sciences. Modern networks apply real-valued transformations on the data. Particularly, convolutions in convolutional neural networks discard phase information entirely. Many deterministic signals, such as seismic data or electrica…
It is well known that the initialization of weights in deep neural networks can have a dramatic impact on learning speed. For example, ensuring the mean squared singular value of a network's input-output Jacobian is O(1) is essential for avoiding the exponential vanishing or explosion of gradients. The stronger condi…
Sobolev training helps neural nets fit function values and derivatives.
problem Training neural nets to match function values and derivatives accurately.
method Using Sobolev loss with gradient flow for overparameterized networks.
result Gradient flow from random initialization can fit any function and its derivatives.
We present a novel optimization strategy for training neural networks which we call "BitNet". The parameters of neural networks are usually unconstrained and have a dynamic range dispersed over all real values. Our key idea is to limit the expressive power of the network by dynamically controlling the range and set of …
FVI method calculates bicausal OT with neural networks, outperforming other methods.
problem Computing bicausal optimal transport with adapted coupling structures.
method FVI method using multilayer neural networks to approximate value functions.
result FVI method outperforms linear programming and Sinkhorn methods in scalability.
Paper tackles regularization and sparsification for quaternion neural networks.
problem Regularizing and sparsifying quaternion neural networks for compactness and real-time applications.
method Developed targeted regularization strategies for quaternion neural networks, extending l1 and structured regularization.
result Tailored strategies significantly reduce the number of connections and neurons, resulting in smaller, more compact networks.
Complex-valued neural networks can approximate any continuous function.
problem Generalizing the universal approximation theorem to complex-valued networks.
method Characterizing activation functions for complex networks to approximate any continuous function.
result Different activation functions are required for deep vs shallow complex networks to achieve universal approximation.
A communication-efficient method controls FDR in network settings.
problem Controlling FDR in networks with limited communication.
method Sample-and-Forward: a flexible procedure for multihop networks.
result Nodes can control FDR without sharing p-values, achieving power and FDR control.
Paper uses RNN to predict SaaS user lifetime value.
problem Predicting user lifetime value in SaaS applications.
method Recurrent Neural Network with multi-cell architecture, accounting for cohort, age-in-system, and contemporaneous information.
result Significantly improved prediction accuracy compared to existing models.
Survey of complex-valued neural networks for improved performance.
problem Lack of complex-valued neural networks in machine learning frameworks.
method Literature review of CVNNs.
result Advantages of CVNNs over real-valued neural networks.
Proposes a network-based strategy to manage financial market risks.
problem Managing extreme events in volatile financial markets.
method Extreme value theory, network model, maximum independent set, value at risk, expected shortfall.
result Developed portfolio strategies improve risk diversification.
We proposed the expected energy-based restricted Boltzmann machine (EE-RBM) as a discriminative RBM method for classification. Two characteristics of the EE-RBM are that the output is unbounded and that the target value of correct classification is set to a value much greater than one. In this study, by adopting featur…
The problem of explaining the behavior of deep neural networks has recently gained a lot of attention. While several attribution methods have been proposed, most come without strong theoretical foundations, which raises questions about their reliability. On the other hand, the literature on cooperative game theory sugg…
Proposes QNN to protect input privacy in neural networks.
problem Protecting input privacy in neural networks.
method Quaternion-valued neural network (QNN) to hide input information.
result QNN effectively protects input privacy without significant accuracy loss.
Residual networks analyzed using linearization for stability under perturbations.
problem Understanding the behavior of residual networks under small input perturbations.
method Linearization of residual units and network stages, using singular value decomposition for stability analysis.
result Most singular values of residual units are 1, but scaling and weights significantly affect them.