Proposes a method to improve neural architectures reproducibly.
problem Lack of reproducibility in Neural Architecture Transformer (NAT).
method Differentiable Neural Architecture Transformation (DNAT).
result DNAT outperforms NAT and is applicable to various models and datasets.
Neural network MCMC sampler maximizes proposal entropy for efficient sampling.
problem Inefficient sampling from complex probability distributions.
method Proposes a neural network MCMC sampler that maximizes proposal entropy.
result Significantly higher efficiency in various sampling tasks.
Proposes deep graph persistence to address neural persistence issues in deep learning.
problem Variance of weights and lack of spatial structure in deep neural networks impact neural persistence.
method Extends neural persistence to the whole network, considering interactions between layers.
result Deep graph persistence alleviates variance-related issues and captures persistent paths through the network.
Proposes a neural network for contextual regression.
problem Improving model efficiency and interpretability in regression with contextual features.
method Simple contextual neural network (SCtxtNN) that separates context identification from context-specific regression.
result SCtxtNN achieves lower excess mean squared error and more stable performance than feed-forward neural networks.
We propose a simple method that combines neural networks and Gaussian processes. The proposed method can estimate the uncertainty of outputs and flexibly adjust target functions where training data exist, which are advantages of Gaussian processes. The proposed method can also achieve high generalization performance fo…
Convolutional Neural Processes improve data efficiency in neural processes.
problem Improving data efficiency in neural processes for small datasets.
method Convolutional Neural Processes (ConvNPs) improve data efficiency by leveraging translation equivariance and convolutional neural networks.
result ConvNPs enhance the performance of neural processes in small-data problems.
Proposes neural dynamic mode decomposition for end-to-end modeling of nonlinear dynamics.
problem Understanding and modeling nonlinear dynamical systems.
method Trains neural networks to minimize forecast error based on spectral decomposition in the lifted space.
result Demonstrates effectiveness in eigenvalue estimation and forecast performance.
Proposes a new method for uncertainty estimation in neural networks.
problem Estimating uncertainty in neural networks.
method Samples outputs from Gaussian distributions parametrized by mean and variance sub-layers.
result Achieves better uncertainty quality than other methods.
GrADE uses graph neural networks and Neural ODE for solving time-dependent nonlinear PDEs efficiently.
problem Solving time-dependent nonlinear PDEs is computationally challenging and time-consuming.
method GrADE combines graph neural networks for spatial modeling and Neural ODE for temporal modeling, using attention mechanisms.
result GrADE efficiently solves PDEs, demonstrating scalability and better accuracy compared to existing methods.
This paper studies neural network operators and their convergence properties.
problem Understanding the approximation and convergence of neural network operators.
method Proves density results, convergence estimates, and Voronovskaya-type theorems.
result Establishes quantitative convergence estimates and derives Voronovskaya-type theorems.
Neural Ordinary Differential Equation (Neural ODE) has been proposed as a continuous approximation to the ResNet architecture. Some commonly used regularization mechanisms in discrete neural networks (e.g. dropout, Gaussian noise) are missing in current Neural ODE networks. In this paper, we propose a new continuous ne…
In this paper we propose an investing strategy based on neural network models combined with ideas from game-theoretic probability of Shafer and Vovk. Our proposed strategy uses parameter values of a neural network with the best performance until the previous round (trading day) for deciding the investment in the curren…
New BNN architectures reduce computational cost for uncertainty quantification.
problem High computational cost in Bayesian neural networks.
method Partial trace-class Bayesian neural networks (PaTraC BNNs).
result Comparable uncertainty quantification with fewer parameters.
Memory-augmented neural networks (MANNs) have been shown to outperform other recurrent neural network architectures on a series of artificial sequence learning tasks, yet they have had limited application to real-world tasks. We evaluate direct application of Neural Turing Machines (NTM) and Differentiable Neural Compu…
This study shows why training Neural ODEs is hard and proposes a new method.
problem Training Neural ODEs is challenging, especially in practice.
method Proposed a new stabilization method and provided an analytical convergence analysis.
result Insights and techniques for researchers starting work on Neural ODEs.
We propose two neural network based mixture models in this article. The proposed mixture models are explicit in nature. The explicit models have analytical forms with the advantages of computing likelihood and efficiency of generating samples. Computation of likelihood is an important aspect of our models. Expectation-…
In this article, we propose a novel technique for classification of the Murmurs in heart sound. We introduce a novel deep neural network architecture using parallel combination of the Recurrent Neural Network (RNN) based Bidirectional Long Short-Term Memory (BiLSTM) & Convolutional Neural Network (CNN) to learn visual …
Recently low displacement rank (LDR) matrices, or so-called structured matrices, have been proposed to compress large-scale neural networks. Empirical results have shown that neural networks with weight matrices of LDR matrices, referred as LDR neural networks, can achieve significant reduction in space and computation…
Proposes a method to optimize neural network initialization using marginal likelihood maximization.
problem Optimizing hyperparameters for neural network initialization.
method Leverages the connection between neural networks and Gaussian processes to infer optimal hyperparameters.
result Marginal likelihood maximization provides near-optimal prediction performance on MNIST classification tasks.
PHP connects to ReLU neural networks for scalable Bayesian inference.
problem Scalability and Bayesian inference in two-layer ReLU neural networks.
method PHP with Gaussian prior, decomposition propositions, annealed sequential Monte Carlo.
result PHP provides an alternative scalable representation for two-layer ReLU neural networks.
TSSM splits neural networks for parallel training with minimal accuracy loss.
problem Accuracy degradation in parallel training of deep neural networks.
method TSSM reformulates alternating minimization to achieve parallelism with minimal accuracy loss.
result TSSM achieves significant speedup without accuracy loss on multiple datasets.
Neural network approach simplifies multiscale problem homogenization.
problem Homogenizing multiscale problems with varying microscale structures.
method Derivative-free neural network with Brownian walkers.
result Neural network method is computationally efficient and robust.
Proposes BATer for improved adversarial example detection.
problem Detecting adversarial examples in neural networks.
method Introduces a Bayesian adversarial example detector (BATer) using random components in a Bayesian neural network.
result BATer outperforms state-of-the-art detectors in adversarial example detection.
While many approaches to make neural networks more fathomable have been proposed, they are restricted to interrogating the network with input data. Measures for characterizing and monitoring structural properties, however, have not been developed. In this work, we propose neural persistence, a complexity measure for ne…
Embeds sparsity in deep neural networks, allowing exact zero parameters.
problem Learning sparse structures in deep networks.
method Embeds sparsity into neural network structure, allowing exact zero parameters during training.
result Can learn both structured and unstructured sparsity.
Proposes a new method to estimate Bayesian neural network depth.
problem Estimating the depth of Bayesian neural networks.
method Uses a discrete truncated normal distribution to learn depth mean and variance, inferring posterior distributions by minimizing variational free energy.
result Improves test accuracy and reduces posterior depth variance on the spiral dataset.
Proposes a method to create prediction intervals for neural networks using cross-validation.
problem Lack of prediction intervals for neural networks.
method k-fold cross-validation to construct conformal prediction intervals.
result Proposed method produces narrower intervals with similar coverage compared to SC method.
A new interpolation method speeds up neural ODE training.
problem Efficiently approximating gradients in neural ODEs.
method Interpolation-based technique to approximate gradients.
result Our method trains neural ODEs faster than the reverse dynamic method.
New DDMs use neural networks for solving equations on manifold shapes.
problem Solving equations on complex, high-dimensional shapes.
method Physics-informed neural networks combined with domain decomposition methods.
result Validated methods work well on various shapes in high dimensions.
Model change points in time-series data with neural SDEs and variational autoencoders.
problem Modeling change points in time-series data with neural stochastic differential equations.
method Proposes a novel model formulation and training procedure based on the variational autoencoder framework, alternating between updating neural SDE parameters and change points.
result Demonstrates the expressive power of the proposed model in modeling both classical parametric SDEs and real datasets with distribution shifts.
Proposes a new model for complex multivariate event data.
problem Modeling complex multivariate event data with spatio-temporal dynamics.
method Integrates spatial information into latent state evolution through learned temporal and spatial decay dynamics.
result Successfully recovers sensible temporal and spatial intensity structure in multivariate spatio-temporal point patterns.
SurvBeNIM explains survival models using neural networks.
problem Explaining predictions of survival models.
method Extends Beran estimator with neural importance functions.
result Improved explanation of survival models compared to existing methods.
Proposes a new framework for predicting stock market movements using sparse neural architectures.
problem Challenging problem of predicting stock market movements using technical indicators.
method Multi-criteria optimization approach to evolve sparse neural architectures.
result Evolved parsimonious networks with better generalization capabilities.
Stable neural flows ensure robustness and efficiency in deep learning.
problem Ensuring robustness and stability in deep learning models.
method Introducing a stable variant of neural ODEs with a neural network parametrizing an energy functional, solving as an optimal control problem with adjoint sensitivity analysis.
result The proposed model provides robustness against input perturbations and low computational burden.
Robust deep neural networks estimate multi-dimensional functional data robustly.
problem Estimating location function from multi-dimensional functional data robustly.
method Deep neural networks with ReLU activation, robust to outliers and model misspecification.
result Uniform convergence rates for robust deep neural network estimators.
Novel approach embeds loss tunnels in neural networks, revealing insights into their structure.
problem Understanding the structure of neural network loss surfaces, especially low-loss tunnels.
method Directly embedding loss tunnels into the loss landscape of neural networks.
result Improved insights into the length and structure of loss tunnels, and better subspace inference in Bayesian neural networks.
Paper tackles division difficulty, proposing new methods to improve accuracy.
problem Division is the most challenging arithmetic operation for both humans and computers.
method Proposes two novel approaches: Neural Reciprocal Unit (NRU) and Neural Multiplicative Reciprocal Unit (NMRU), and improves an existing division module.
result Improves division accuracy from 70.2% to 91.6%.
The paper proposes a neural network architecture inspired by Langevin Monte Carlo for sampling from target distributions.
problem Sampling from complex target distributions efficiently.
method A neural network architecture inspired by Langevin Monte Carlo is proposed to map samples from a simple reference distribution to samples from the target.
result The proposed neural network architecture achieves approximation rates in the Wasserstein-2 distance for smooth, log-concave target distributions.
Proposes a neural network loss function for better uncertainty estimation.
problem Challenges in estimating predictive uncertainty of neural networks.
method Bayesian Validation Metric (BVM) framework with ensemble learning.
result Competitive and robust uncertainty estimation on in-distribution and out-of-distribution data.
Neural GDEs improve graph prediction by blending discrete structures and differential equations.
problem Dynamic graph prediction challenges in irregularly sampled data.
method Continuous-depth graph neural networks (GNNs) with Neural GDEs.
result Neural GDEs enhance performance across various applications.
We propose a new scalable method to optimize the architecture of an artificial neural network. The proposed algorithm, called Greedy Search for Neural Network Architecture, aims to determine a neural network with minimal number of layers that is at least as performant as neural networks of the same structure identified…
Neural Thompson Sampling uses deep neural networks for contextual bandit problems.
problem Solving contextual multi-armed bandit problems.
method Adapts deep neural networks for exploration and exploitation in a novel posterior distribution.
result Guaranteed cumulative regret of O(T1/2) for bounded reward functions. Neural networks have shown great potential in many applications like speech recognition, drug discovery, image classification, and object detection. Neural network models are inspired by biological neural networks, but they are optimized to perform machine learning tasks on digital computers. The proposed work explores…
Proposes Neural Complexity (NC) for predicting and explaining generalization in deep neural networks.
problem Challenges in specifying a suitable complexity measure for deep neural networks to predict and explain generalization.
method A meta-learning framework that learns a scalar complexity measure through interactions with many heterogeneous tasks.
result Trained NC model can be added to standard training loss to regularize any task learner.
Meta-learning neural networks for better clustering representations.
problem Improving clustering performance with appropriate representations.
method Meta-learning method that trains neural networks for representations using VB inference with an infinite Gaussian mixture model.
result The method achieves higher clustering performance than existing methods.
Meta-learning neural networks to solve diverse PDEs efficiently.
problem Efficiently solving new PDE problems with minimal training.
method Neural network meta-learning of PDE problem representations.
result Meta-learned neural networks predict PDE solutions with high accuracy.
A new faster neural network training method using backprojection.
problem Training feedforward neural networks more efficiently.
method Projection and reconstruction at each layer to force projected data and reconstructed labels to be similar.
result The proposed method is faster than backpropagation and gives insights into networks.
Proposes a new method for continual learning in neural networks.
problem Challenges in applying sequential Bayesian inference to neural networks.
method Sequential function-space variational inference.
result Neural networks trained with the proposed method achieve better predictive accuracy.