Investigates how neural network graph structure impacts predictive performance.
problem Lack of understanding between neural network graph structure and predictive performance.
method Developed relational graph representation to analyze neural networks, identifying a 'sweet spot' for improved performance.
result Identified a 'sweet spot' in relational graph structure that significantly improves neural network predictive performance.
Quantum neural network and tensor network models outperform classical models in Japanese stock market predictions.
problem Improving stock return predictions using quantum and quantum-inspired machine learning.
method Evaluation of quantum neural network and tensor network models against classical models like linear and neural networks.
result Tensor network model outperforms classical models in Japanese stock market, including linear and neural network models.
A powerful network teaches a weak one, improving its performance.
problem Improving the performance of a weak neural network using a more powerful one.
method During training, a weak network learns features from a strong network to minimize feature distance.
result A weak neural network can increase its performance by learning from a more powerful network.
Paper benchmarks quantum neural networks against classical ones for binary classification tasks.
problem Comparing quantum neural networks with classical ones for binary classification.
method Evaluated with two toy examples, focusing on model complexity and training data size.
result EQNN and QNN outperform ENN and DNN for smaller parameter sets and training data samples.
Graphs of neural networks are represented to preserve symmetry, improving performance across various tasks.
problem Lack of equivariance in neural network representations of other neural networks.
method Represent neural networks as computational graphs and use graph neural networks to preserve permutation symmetry.
result Single model encodes diverse neural architectures, outperforming state-of-the-art methods.
This work investigates the framework and performance issues of the composite neural network, which is composed of a collection of pre-trained and non-instantiated neural network models connected as a rooted directed acyclic graph for solving complicated applications. A pre-trained neural network model is generally well…
New neural networks combine additive regression with traditional architectures.
problem Performance limitations and high parameter requirements of traditional neural networks.
method Introduce hybrid deep additive neural networks with simpler activation and basis functions.
result Hybrid neural networks achieve better performance with fewer parameters.
New method finds sparse networks without labels, improving performance.
problem Sparse connectivity in neural networks to reduce memory and energy demands.
method Neural Tangent Transfer method to find sparse networks without labels.
result Sparse networks achieve higher classification performance and faster convergence.
This work theoretically investigates the performance of a composite neural network. A composite neural network is a rooted directed acyclic graph combining a set of pre-trained and non-instantiated neural network models, where a pre-trained neural network model is well-crafted for a specific task and targeted to approx…
We propose a new scalable method to optimize the architecture of an artificial neural network. The proposed algorithm, called Greedy Search for Neural Network Architecture, aims to determine a neural network with minimal number of layers that is at least as performant as neural networks of the same structure identified…
Study evaluates neural networks based on random graph structures and finds key performance indicators.
problem Understanding and optimizing neural network architectures using graph theory.
method Evaluation of neural networks with random graph structures, focusing on structural and numerical properties.
result A new numerical graph characteristic selects a set of quasi-1-dimensional graphs that perform well.
This study explores training Bayesian neural networks at scale using high-performance computing.
problem Challenges in training Bayesian neural networks at scale due to computational overhead.
method High-performance computing with distributed training, network pruning.
result Pruning up to 80% of the network can reduce inference time by 7.0% without significant accuracy loss.
Improved neural network convergence with causal Bayesian modeling in retail performance.
problem Improving neural network convergence in retail performance models.
method Causal Bayesian neural network implementation, removal of weakest SEM path, Flipout layers, Vadam optimizer.
result Neural network convergence improved with removal of the weakest SEM path.
Not all neural network architectures are created equal, some perform much better than others for certain tasks. But how important are the weight parameters of a neural network compared to its architecture? In this work, we question to what extent neural network architectures alone, without learning any weight parameter…
Predicting neural network accuracy from weights without testing.
problem Predicting neural network performance based on weights alone.
method Used simple statistics of weights to rank neural networks' performance.
result Simple predictors can rank networks' performance with high accuracy (R2 score > 0.98).
Neural networks simplify SDR in regression tasks.
problem Sufficient dimension reduction in regression problems.
method Applying neural networks with rank regularization to estimate the central mean subspace.
result Neural networks effectively perform SDR, consistent with theoretical estimations.
Neural networks outperform kernel methods in classifying high-dimensional Gaussian mixtures.
problem Classifying high-dimensional Gaussian mixtures using kernel methods and neural networks.
method Theoretical analysis and derivation of learning dynamics for 2LNN and comparison with kernel methods.
result 2LNN can achieve near-optimal performance on high-dimensional Gaussian mixture classification tasks, surpassing kernel methods.
Study bounds graph neural networks' over-parameterized error.
problem Understanding graph neural networks' performance in over-parameterized regimes.
method Developed mean-field regime bounds for graph convolutional and message passing neural networks.
result Established upper bounds with a convergence rate of O(1/n) for generalization error. A method to analyze neural network performance by measuring layer saturation.
problem Understanding which layers contribute to network performance.
method Layer saturation method: restricts layer output to eigenspace of variance matrix.
result Layer saturation indicates which layers contribute to network performance.
Efficient echo state network with explicit memory performs well on benchmark tasks.
problem Training differentiable neural computers is difficult and time-consuming.
method Echo state network with an explicit memory.
result Echo state network can recognize all regular languages, including those contractive networks cannot.
New method reduces neural image compression run-time by 50%.
problem Computational efficiency of neural image compression models.
method Automatic network optimization to reduce decoder complexity.
result Decreased decoder run-time by over 50%.
Paper optimizes neural network initialization using SMT solvers.
problem Improving neural network performance through better initialization.
method Reduces initialization to SMT problem solving.
result Proposed method achieves better performance than random initialization.
Neural networks can learn distance metrics affecting model performance.
problem Understanding how neural networks learn and represent data.
method Experiments with six MNIST architectures, constrained to learn either distance or intensity representations.
result Distance-based learning affects model performance, validating the geometric framework.
New method compresses neural networks up to 14x with minimal performance loss.
problem Compressing neural networks for real-time applications.
method Post-training rank-selection method called Rank-Tuning.
result High compression rates with minimal performance degradation.
Fine-tuning neural networks to guarantee performance on specific examples can also introduce incorrect inputs.
problem Ensuring reliable performance of neural networks on specific examples.
method Using SMT solvers to fine-tune ReLU neural networks to guarantee outcomes on a finite set of particular examples.
result Fine-tuning can introduce incorrect inputs that trigger unexpected performance.
RicciNets prunes neural networks by removing edges of low importance based on Ricci curvature, reducing FLOPs by 35%.
problem Pruning neural networks to reduce computational load and improve efficiency.
method RicciNets uses Ricci curvature to prune edges of low importance in a randomly wired neural network, reducing FLOPs.
result Reduction of almost 35% in FLOPs with no performance degradation.
Deep learning using multi-layer neural networks (NNs) architecture manifests superb power in modern machine learning systems. The trained Deep Neural Networks (DNNs) are typically large. The question we would like to address is whether it is possible to simplify the NN during training process to achieve a reasonable pe…
This paper introduces a new method for neural networks that doesn't need a global coordinate system.
problem The lack of a global coordinate system in neural networks limits their performance and explainability.
method Proposes a learnable topological layer that works in a general metric space (Hilbert space) without requiring a Euclidean space.
result The proposed method eliminates the need for a costly parametrization stage and achieves optimal network performance.
Improves neural network performance by enriching training dataset.
problem Achieving worst-case performance guarantees in neural networks.
method Adapting training dataset during training to reduce worst-case violations.
result Improved worst-case performance guarantees in neural networks.
Proposes TAGI for efficient Gaussian inference in Bayesian neural networks.
problem Efficient inference in Bayesian neural networks with complex architectures.
method Analytical method for tractable approximate Gaussian inference (TAGI).
result Matches performance of gradient-based methods with O(n) computational complexity. Enhances neural network regression performance by modeling weight and variance uncertainty.
problem Improving predictive performance of neural networks for regression tasks.
method Extended Blundell's framework to include variance uncertainty, using a full posterior distribution over variance parameters.
result Explicitly modeling variance uncertainty improves generalization of Bayesian neural networks.
Neural networks learn discrete tasks on continuous data via emergent geometry.
problem Understanding how neural networks perform discrete computations on continuous data.
method Analysis of Riemannian pullback metric across neural network layers.
result Neural networks learn to discretize continuous inputs and perform logical operations on these discretized variables.
We present a supervised neural network model for polyphonic piano music transcription. The architecture of the proposed model is analogous to speech recognition systems and comprises an acoustic model and a music language model. The acoustic model is a neural network used for estimating the probabilities of pitches in …
Stochastic Bayesian Neural Network improves scalability and performance.
problem Challenges in calculating posterior distribution in Bayesian Neural Networks.
method Maximizes Evidence Lower Bound using Stochastic Evidence Lower Bound objective function.
result Demonstrates improved performance and scalability over previous algorithms.
This review compares GAMs and neural networks on real-world tabular data.
problem Comparing the performance and characteristics of GAMs and neural networks in tabular data applications.
method Systematic review following PRISMA guidelines, extracting and analysing key attributes from 143 papers and 430 datasets.
result No consistent evidence of superiority for either GAMs or neural networks, with performance trade-offs depending on dataset characteristics.
Paper proposes ECOC for deep neural network ensembles to improve performance.
problem Designing an ensemble of deep networks is time-consuming and often not beneficial.
method ECOC framework applied to deep networks, with design strategies to balance accuracy and complexity.
result Proposed combinatory technique achieves highest classification performance.
Unified scaling laws reveal how model size and training time impact neural network performance.
problem Understanding how much performance improvement can be expected from scaling model size or data volume.
method Established scale-time equivalence and combined it with a linear model analysis of double descent.
result Unified theoretical scaling laws explain previously unexplained phenomena and offer a more accessible path to training large models.
Although artificial neural networks have occasionally been used for Quantitative Structure-Activity/Property Relationship (QSAR/QSPR) studies in the past, the literature has of late been dominated by other machine learning techniques such as random forests. However, a variety of new neural net techniques along with suc…
The paper proposes a method for better uncertainty estimation in neural networks.
problem Estimating predictive uncertainty in neural networks is crucial but challenging.
method The paper proposes a function-space variational inference method to infer a posterior distribution over functions.
result The proposed method leads to state-of-the-art uncertainty estimation and predictive performance.
In this paper we propose an investing strategy based on neural network models combined with ideas from game-theoretic probability of Shafer and Vovk. Our proposed strategy uses parameter values of a neural network with the best performance until the previous round (trading day) for deciding the investment in the curren…
Gradient amplification boosts deep learning model performance without increasing training time.
problem Vanishing gradients in deep neural networks.
method Gradient amplification approach to prevent vanishing gradients and training strategy to enable/disable across epochs.
result Improves performance of deep learning models with reduced training time.
In this paper we present a statistical analysis about the characteristics that we intend to influence in the performance of the neural networks in terms of assertiveness in the prediction of Brazilian stock returns. We created a population of architectures for analysis and extracted the sample that had the best asserti…
Capsule networks improve on traditional neural networks by using vector activations.
problem Comparing capsule networks to traditional neural networks to validate their benefits.
method Deep visualization analysis, feature encoding across vector components, and instantiation parameter encoding.
result Capsule features encode information differently and provide benefits in computer vision applications.
The study examines if ReLU activation function is optimal for modularity in neural networks.
problem Finding the best activation function for modularity in neural networks.
method Comparing ReLU with other activation functions for modularity and performance.
result ReLU may not be the best choice for modularity, suggesting other functions could be more suitable.
The EMNLP 2018 workshop BlackboxNLP was dedicated to resources and techniques specifically developed for analyzing and understanding the inner-workings and representations acquired by neural models of language. Approaches included: systematic manipulation of input to neural networks and investigating the impact on thei…
Deep neural networks have revolutionized many fields such as computer vision and natural language processing. Inspired by this recent success, deep learning started to show promising results for Time Series Classification (TSC). However, neural networks are still behind the state-of-the-art TSC algorithms, that are cur…
FIRE PBT improves neural network training by focusing on long-term performance.
problem Greedy decision mechanisms in PBT lead to poor long-term performance.
method FIRE PBT uses a fitness metric to encourage long-term performance over short-term improvements.
result FIRE PBT outperforms PBT on ImageNet and matches hand-tuned learning rates.
This work uses decision trees to encode relevant features and their interactions into neural networks, improving model performance.
problem Overfitting in neural networks with many irrelevant variables.
method Defines a mapping to encode decision tree extracted relationships into a neural network.
result The approach outperforms fully connected neural networks and tree-based methods.