A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
We improve neural network explainability by bypassing batch normalization.
This work proves the asymptotic freeness of layerwise Jacobians in MLPs with Haar orthogonal matrices.
Use simplified layerwise linear models to understand neural dynamics.
In this paper, we tackle the problem of explanations in a deep-learning based model for recommendations by leveraging the technique of layer-wise relevance propagation. We use a Deep Convolutional Neural Network to extract relevant features from the input images before identifying similarity between the images in featu…
While the authors of Batch Normalization (BN) identify and address an important problem involved in training deep networks-- Internal Covariate Shift-- the current solution has certain drawbacks. Specifically, BN depends on batch statistics for layerwise input normalization during training which makes the estimates of …
A new method for learning Bayesian neural networks using layerwise inference.
Proves deep networks can learn hierarchical structures efficiently.
While the authors of Batch Normalization (BN) identify and address an important problem involved in training deep networks-- \textit{Internal Covariate Shift}-- the current solution has certain drawbacks. For instance, BN depends on batch statistics for layerwise input normalization during training which makes the esti…
Paper introduces MCRP for estimating feature relevance uncertainty in neural networks.
The paper proposes a scalable framework for uncertainty quantification and propagation in surrogate-based Bayesian inference.
Efficient algorithm for CMDPs reduces to offline density estimation.
New method improves neural network training by scaling perturbations layerwise.
InteractionNet models noncovalent protein-ligand interactions with GNNs and explains predictions.
MambaLRP enhances Mamba models' explainability and performance.
Neural networks have become increasingly prevalent within the geosciences, although a common limitation of their usage has been a lack of methods to interpret what the networks learn and how they make decisions. As such, neural networks have often been used within the geosciences to most accurately identify a desired o…
Recently, a technique called Layer-wise Relevance Propagation (LRP) was shown to deliver insightful explanations in the form of input space relevances for understanding feed-forward neural network classification decisions. In the present work, we extend the usage of LRP to recurrent neural networks. We propose a specif…
The paper studies how noise synchronizes tokens in deep transformer models.
LNPE enhances local connections in embeddings using extended neighbor propagation.
New method explains GNN predictions using walks.
AlphaPruning optimizes LLM pruning using HT-SR theory for better performance.
Training large deep neural networks on massive datasets is computationally very challenging. There has been recent surge in interest in using large batch stochastic optimization methods to tackle this issue. The most prominent algorithm in this line of research is LARS, which by employing layerwise adaptive learning ra…
Proposes efficient training method for deep thin networks.
Pixel-wise relevance method shows how CNNs classify faces, varying across datasets and tasks.
XAI identifies key time steps for early crop classification.
Deep Gaussian Processes (DGP) are hierarchical generalizations of Gaussian Processes (GP) that have proven to work effectively on a multiple supervised regression tasks. They combine the well calibrated uncertainty estimates of GPs with the great flexibility of multilayer models. In DGPs, given the inputs, the outputs …
Efficiently learns deep factor graphs using Gaussian belief propagation.
While neural networks have acted as a strong unifying force in the design of modern AI systems, the neural network architectures themselves remain highly heterogeneous due to the variety of tasks to be solved. In this chapter, we explore how to adapt the Layer-wise Relevance Propagation (LRP) technique used for explain…
In this work we propose a heteroscedastic generalization to RVM, a fast Bayesian framework for regression, based on some recent similar works. We use variational approximation and expectation propagation to tackle the problem. The work is still under progress and we are examining the results and comparing with the prev…
Estimating machine learning performance 'in the wild' is both an important and unsolved problem. In this paper, we seek to examine, understand, and predict the pointwise competence of classification models. Our contributions are twofold: First, we establish a statistically rigorous definition of competence that general…
We propose a novel approach for nonlinear regression using a two-layer neural network (NN) model structure with sparsity-favoring hierarchical priors on the network weights. We present an expectation propagation (EP) approach for approximate integration over the posterior distribution of the weights, the hierarchical s…
NES optimizes discrete structured VAEs effectively without gradient propagation.
We introduce a program aimed to studying problems arising from the theory of complex networks with differential geometric means. We study the propagation of influences on manifolds assuming that at each point only a finite number of propagation velocities are allowed. This leads to the computation of the volume of the …
The muti-layer information bottleneck (IB) problem, where information is propagated (or successively refined) from layer to layer, is considered. Based on information forwarded by the preceding layer, each stage of the network is required to preserve a certain level of relevance with regards to a specific hidden variab…
Current techniques for explainable AI have been applied with some success to image processing. The recent rise of research in video processing has called for similar work n deconstructing and explaining spatio-temporal models. While many techniques are designed for 2D convolutional models, others are inherently applica…
Deep learning is also known as hierarchical learning, where the learner _learns_ to represent a complicated target function by decomposing it into a sequence of simpler functions to reduce sample and time complexity. This paper formally analyzes how multi-layer neural networks can perform such hierarchical learning _ef…
Training avoids edge of stability by aligning Jacobian matrices.
SAEs struggle with curved activation manifolds, revealing layer-dependent scaling laws.
New optimizers control network width scaling, improving stability and transfer across different model sizes.
Dynamic trees are mixtures of tree structured belief networks. They solve some of the problems of fixed tree networks at the cost of making exact inference intractable. For this reason approximate methods such as sampling or mean field approaches have been used. However, mean field approximations assume a factorized di…
A Global Navigation Satellite System (GNSS) uses a constellation of satellites around the earth for accurate navigation, timing, and positioning. Natural phenomena like space weather introduce irregularities in the Earth's ionosphere, disrupting the propagation of the radio signals that GNSS relies upon. Such disruptio…
Attribution methods aim to explain a neural network's prediction by highlighting the most relevant image areas. A popular approach is to backpropagate (BP) a custom relevance score using modified rules, rather than the gradient. We analyze an extensive set of modified BP methods: Deep Taylor Decomposition, Layer-wise R…
Injectivity of ReLU networks is characterized for generative models and inverse problems.
Paper compares hard and soft EM for BN learning from incomplete data.
Layer-wise relevance propagation (LRP) is a recently proposed technique for explaining predictions of complex non-linear classifiers in terms of input variables. In this paper, we apply LRP for the first time to natural language processing (NLP). More precisely, we use it to explain the predictions of a convolutional n…
Develops methods to measure and set function-space learning rates in neural networks.
IBP-R improves verified adversarial robustness with simple, effective interval bound propagation.
Learning long-term dependencies in extended temporal sequences requires credit assignment to events far back in the past. The most common method for training recurrent neural networks, back-propagation through time (BPTT), requires credit information to be propagated backwards through every single step of the forward c…
The study finds that supply chain information from LLM embeddings improves stock returns predictions.