The paper identifies potential adversarial samples near decision boundaries of neural networks.
problem Vulnerability of deep neural networks to small perturbations of inputs.
method Developed a method to explore near decision boundaries of trained classifiers to identify potential adversarial samples.
result Potential adversarial samples represent only 61% of the test data but cover more than 82% of adversarial samples produced by iFGSM and 92% of those by DeepFool on CIFAR10.
Transfer learning boosts chemically accurate neural network potentials for organic molecules.
problem Developing accurate interatomic potentials from ab-initio data.
method Discriminative fine-tuning of pre-trained neural networks.
result Fine-tuning with energy labels alone can achieve accurate atomic forces.
New method uses geometric moments for accurate machine learning potentials.
problem Creating high-dimensional potential energy surfaces efficiently.
method Feed-forward neural networks with invariant local molecular descriptors based on geometric moments.
result Accuracy comparable to established models, high efficiency.
This paper introduces a new potential function using Tsallis entropy for neural network optimization.
problem The challenge of obtaining exponential convergence in neural network optimization.
method Utilizes a linearized potential function based on Csiszár type of Tsallis entropy.
result Derives an exponential convergence result in neural network optimization.
Deep quantum neural networks applied to finance for efficient risk management.
problem Efficiently solving numerical problems in finance, especially risk management.
method Application of deep quantum neural networks to finance, focusing on implied volatilities, option prices, and Greeks.
result Deep quantum neural networks can compute Greeks analytically and efficiently solve financial numerical problems.
Improved neural network models predict molecular and material properties efficiently.
problem Training neural networks for accurate interatomic potentials is computationally expensive.
method Gaussian moment-based neural networks with improved architecture and active learning.
result The new models achieve high accuracy and reduced training times.
Mirror flow in shallow neural networks shows similar implicit bias to gradient flow, with key differences in curvature penalties.
problem Analyzing implicit bias in shallow neural networks with mirror flow.
method Characterization through variational problems and scaled potentials.
result Mirror flow with scaled potentials induces a rich class of biases not captured by RKHS norms.
We consider the problem of learning an interpretable potential energy function from a Hamiltonian system's trajectories. We address this problem for classical, separable Hamiltonian systems. Our approach first constructs a neural network model of the potential and then applies an equation discovery technique to extract…
hyperSBINN improves drug cardiosafety assessment by efficiently modeling cardiac action potentials.
problem Complexity and limited data in modeling cardiac effects of drugs.
method Combining meta-learning with SBINNs to solve parameterized cardiac action potential models.
result hyperSBINN outperforms traditional solvers in speed and accuracy for predicting APD90 values.
Review of efficient neural networks for TinyML on resource-constrained devices.
problem Resource constraints on ultra-low power MCUs for deep learning models.
method Model compression, quantization, low-rank factorization, model pruning, hardware acceleration, algorithm-architecture co-design.
result Optimized neural network architectures for minimal resource utilization on MCUs.
This work surveys attacks and defenses on edge neural networks.
problem Security challenges of edge neural networks due to their compute and memory intensity, data-independence, and privacy risks.
method Taxonomy of attacks and defenses on edge-deployed neural networks.
result New security considerations and approaches are needed for edge DNNs.
We introduce neural Markov logic networks (NMLNs), a statistical relational learning system that borrows ideas from Markov logic. Like Markov logic networks (MLNs), NMLNs are an exponential-family model for modelling distributions over possible worlds, but unlike MLNs, they do not rely on explicitly specified first-ord…
Proposes a differentiable LSE-ICNN for modeling multi-well potentials.
problem Modeling multi-well potentials in various scientific domains.
method Log-sum-exponential (LSE) mixture of input convex neural network (ICNN) modes.
result Smooth surrogate that retains convexity within basins and allows gradient-based learning.
In this paper, we apply neural networks into digital marketing world for the purpose of better targeting the potential customers. To do so, we model the customer online behaviours using dedicated neural network architectures. Starting from user searched keywords in a search engine to the landing page and different foll…
ChatGPT enhances GNN for stock movement prediction.
problem Predicting stock movements using textual data.
method Integrates ChatGPT's graph inference into GNN for stock movement forecasting.
result Model outperforms state-of-the-art benchmarks in stock movement forecasting.
Committee neural network models improve accuracy and enable active learning for interatomic potentials.
problem Improving accuracy and generalization error in interatomic potentials.
method Adapting committee models to neural networks, using multiple models with shared descriptors, and applying active learning to select configurations.
result Committee disagreement provides a measure of generalization error and guides active learning to minimize it.
New dataset abla2DFT for drug-like molecules benchmarks neural network potentials.
problem Lack of large, diverse datasets for training neural network potentials in quantum chemistry.
method Developed a new dataset abla2DFT containing energies, forces, and molecular properties for drug-like molecules. result First dataset with relaxation trajectories for drug-like molecules.
The need for advanced materials has led to the development of complex, multi-component alloys or solid-solution alloys. These materials have shown exceptional properties like strength, toughness, ductility, electrical and electronic properties. Current development of such material systems are hindered by expensive expe…
New fusion blocks improve equivariant neural networks for molecular dynamics.
problem Designing equivariant neural networks for tasks with global symmetries.
method Using fusion diagrams from tensor networks to design novel equivariant components.
result Improved performance with fewer parameters on chemical problems.
Neural networks can represent complex piecewise functions efficiently.
problem Representing continuous piecewise affine functions with neural networks.
method Two hidden layers with ReLU activation, O(p) neurons for p pieces. result CPA functions can be represented by a neural network with linear size.
Bayesian neural networks integrate uncertainty into neural networks for improved performance.
problem Overconfidence, lack of interpretability, and adversarial attacks in neural networks.
method Integrates Bayesian inference into neural networks to address limitations.
result BNNs improve model performance and provide uncertainty estimates.
Neural networks learn vector fields constrained by linear operators.
problem Learning vector fields from physical systems with linear operator constraints.
method Model the target function as a linear transformation of a potential field, which is a neural network.
result Predictions of the target function satisfy the linear operator constraints.
Tensor networks reveal limitations for efficient text description but suggest potential for images.
problem Efficiently describing large text and image data sets using tensor networks.
method Investigation of mutual information scaling, introduction of mutual information estimators, and use of autoregressive and convolutional neural networks.
result Text data cannot be efficiently described by 1D tensor networks, while images may be better described by 2D tensor networks.
Neural networks have shown great potential in many applications like speech recognition, drug discovery, image classification, and object detection. Neural network models are inspired by biological neural networks, but they are optimized to perform machine learning tasks on digital computers. The proposed work explores…
Neural networks mimic algorithms to solve complex problems.
problem Current machine learning methods struggle with generalisation and efficiency.
method Representing algorithms in a continuous space and adapting them to real-world problems.
result Neural networks can execute classical algorithms more efficiently.
The paper connects neural networks to Mahalanobis distance for interpretability.
problem Lack of interpretability in neural networks.
method Establishes a connection between neural network linear layers and Mahalanobis distance.
result Provides a foundation for more interpretable neural network models.
Stochastic neural networks with infinite width become deterministic, reducing training variance.
problem Understanding how stochasticity in neural networks affects learning and regularization.
method Theoretical analysis of stochastic neural networks with infinite width.
result As the width of an optimized stochastic neural network increases, its predictive variance on the training set decreases to zero.
New method selects neural network architectures without needing data.
problem Choosing efficient deep neural network architectures.
method Developed the deep frame potential to quantify network capacity.
result Deep frame potential correlates with generalization error.
TeaNet uses GCNs to model complex atomic interactions inspired by electronic relaxation.
problem Creating a universal interatomic potential for all elements.
method Tensor-embedded atom network (TeaNet) using graph convolutional neural networks (GCNs).
result TeaNet achieves good performance (19 meV/atom) for structures and reactions involving elements from H to Ar.
Noether's framework reveals symmetry-breaking in neural networks.
problem Understanding the role of symmetry breaking in neural networks.
method Developed a theoretical framework using Lagrangian mechanics.
result Identified 'kinetic symmetry breaking' and its effect on learning dynamics.
We propose a new algorithm to learn a one-hidden-layer convolutional neural network where both the convolutional weights and the outputs weights are parameters to be learned. Our algorithm works for a general class of (potentially overlapping) patches, including commonly used structures for computer vision tasks. Our a…
Deep neural networks solve optimal risk sharing problems.
problem Optimally sharing financial positions among agents with different risk measures.
method Neural network-based framework to compute inf-convolution and optimal allocations.
result Convergence of neural network approximations to theoretical values.
Graph neural networks help assess how global changes affect plant-pollinator networks.
problem Interpreting GNN results to understand how global changes impact plant-pollinator networks.
method Simulation study and application on Spipoll dataset to assess effects of global changes on pollination networks.
result GNNs can detect interactive effects between covariates and plant genera on pollination network connectivity.
This paper studies the potential of the return distribution for exploration in deterministic reinforcement learning (RL) environments. We study network losses and propagation mechanisms for Gaussian, Categorical and Gaussian mixture distributions. Combined with exploration policies that leverage this return distributio…
This paper compares methods for handling mixed-attribute data in GFMM neural networks.
problem Handling datasets with mixed features in GFMM neural networks.
method Three main methods: encoding, combining with other classifiers, and specific learning algorithms.
result Encoding methods and combining with decision trees improve GFMM models' performance.
Qualitative analysis of MC dropout for NN model uncertainty.
problem Measuring uncertainty in neural network models.
method Mathematical formulation of Monte Carlo dropout and its benefits/costs in NN models.
result Potential benefits and associated costs of using MC dropout in NN models.
NNs accurately predict energy eigenvalues and other physical phenomena in 1D quantum mechanics.
problem Understanding how neural networks interpret physics.
method Training NNs to predict energy eigenvalues from potentials and testing their ability to generalize.
result NNs can predict physical phenomena not learned during training, indicating a new way of understanding physics.
Graph Neural Networks improve financial fraud detection.
problem Complex financial transactions pose challenges in fraud detection.
method Unified framework of GNN methodologies applied to financial fraud detection.
result GNNs excel at capturing complex relational patterns in financial networks.
Convex dual network improves neural network reconstruction for medical imaging.
problem Non-convex nature of neural networks hinders their use in sensitive applications.
method Introduces a convex duality framework for a two-layer fully-convolutional ReLU denoising network.
result Training neural networks with weight decay regularization induces path sparsity and piecewise linear filtering.
Deep neural networks solve parameter estimation for FitzHugh-Nagumo ODEs.
problem Estimating parameters of a nonlinear dynamical system from noisy time series data.
method Dense and convolutional neural networks for inverse problem solving.
result Deep neural networks accurately estimate FitzHugh-Nagumo model parameters from noisy data.
Neural machine learning methods, such as deep neural networks (DNN), have achieved remarkable success in a number of complex data processing tasks. These methods have arguably had their strongest impact on tasks such as image and audio processing - data processing domains in which humans have long held clear advantages…
DNPUs improve neural network performance with high-capacity nanoelectronic nodes.
problem Limited performance of single DNPUs in solving complex classification problems.
method Developed DNPUs as high-capacity neurons and implemented multi-DNPU networks.
result Feed-forward DNPU networks improve single DNPU performance from 77% to 94% test accuracy.
Wider networks improve natural accuracy but worsen perturbation stability, affecting overall robustness.
problem Understanding the tradeoff between natural accuracy and perturbation stability in wider neural networks for adversarial robustness.
method Careful examination of the relationship between network width, robust regularization parameter λ, and perturbation stability using neural tangent kernels.
result Wider networks can achieve better natural accuracy but worse perturbation stability, leading to potentially worse overall model robustness.
This paper describes recent development and test implementation of a continuous time recurrent neural network that has been configured to predict rates of change in securities. It presents outcomes in the context of popular technical analysis indicators and highlights the potential impact of continuous predictive capab…
Graph neural networks improve equipment health monitoring from multisensor data.
problem Leveraging complex machinery structure for condition-based maintenance.
method Captured machinery structure as a graph and used graph neural networks (GNNs) to model time-series data.
result GNN-based RUL estimation model outperforms RNNs and CNNs on turbofan engine benchmark.
We establish a margin based data dependent generalization error bound for a general family of deep neural networks in terms of the depth and width, as well as the Jacobian of the networks. Through introducing a new characterization of the Lipschitz properties of neural network family, we achieve significantly tighter g…
SNN architecture shows gradient descent converges to regularized solution in matrix sensing problems.
problem Understanding implicit regularization in neural networks for matrix sensing.
method Developed Spectral Neural Networks (SNN) for matrix learning problems, rigorously demonstrating implicit regularization.
result Gradient descent converges to the solution of a regularized learning problem in matrix sensing problems.
Despite existing work on ensuring generalization of neural networks in terms of scale sensitive complexity measures, such as norms, margin and sharpness, these complexity measures do not offer an explanation of why neural networks generalize better with over-parametrization. In this work we suggest a novel complexity m…