A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
This work makes vision networks more interpretable by identifying key neurons.
It has been widely assumed that a neural network cannot be recovered from its outputs, as the network depends on its parameters in a highly nonlinear way. Here, we prove that in fact it is often possible to identify the architecture, weights, and biases of an unknown deep ReLU network by observing only its output. Ever…
New method finds lattice polygons that can be dissected into triangles with integer areas.
Solved Dudeney's 100-year-old puzzle about triangle to square dissection.
An involutive diffeomorphism of a connected smooth manifold is called dissecting if the complement of its fixed point set is not connected. Dissecting involutions on a complete Riemannian manifold are closely related to constructive quantum field theory through the work of Dimock and Jaffe/Ritter on the constru…
In this note, given a regular Courant algebroid, we compute its group of automorphisms relative to a dissection. We also propose an infinitesimal version and recover examples of the literature.
Study predicts risk of true-lumen narrowing after ATAAD surgery using CT data.
Modeling aortic wall inhomogeneities to predict dissection risks.
The self intersection of an immersion i : S^2 \to R^3 dissects S^2 into pieces which are planar surfaces (unless i is an embedding). In this work we determine what collections of planar surfaces may be obtained in this way. In particular, for every n we construct an immersion i : S^2 \to R^3 with 2n triple points, for …
INVERT connects neural representations to human-understandable concepts.
New model reveals balance crucial for robust neural coding.
The vast majority of market impact studies assess each product individually, and the interactions between the different order flows are disregarded. This strong approximation may lead to an underestimation of trading costs and possible contagion effects. Transactions in fact mediate a significant part of the correlatio…
Interpreting the behaviors of Deep Neural Networks (usually considered as a black box) is critical especially when they are now being widely adopted over diverse aspects of human life. Taking the advancements from Explainable Artificial Intelligent, this paper proposes a novel technique called Auto DeepVis to dissect c…
Pruning is a standard technique for removing unnecessary structure from a neural network to reduce its storage footprint, computational demands, or energy consumption. Pruning can reduce the parameter-counts of many state-of-the-art neural networks by an order of magnitude without compromising accuracy, meaning these n…
Describes explaining neurons in deep representations using compositional logical concepts.
SeReNe prunes neurons with low sensitivity to reduce network size.
Under-parameterized networks can either copy or average teacher weights, leading to universal optimal solutions.
We develop Neuron Shapley as a new framework to quantify the contribution of individual neurons to the prediction and performance of a deep network. By accounting for interactions across neurons, Neuron Shapley is more effective in identifying important filters compared to common approaches based on activation patterns…
Modeling hidden neurons in SNNs using mesoscopic approximations.
Topological methods improve neuron analysis and tracer injection summary.
Inspired by complexity and diversity of biological neurons, our group proposed quadratic neurons by replacing the inner product in current artificial neurons with a quadratic operation on input data, thereby enhancing the capability of an individual neuron. Along this direction, we are motivated to evaluate the power o…
When we are faced with challenging image classification tasks, we often explain our reasoning by dissecting the image, and pointing out prototypical aspects of one class or another. The mounting evidence for each of the classes helps us make our final decision. In this work, we introduce a deep network architecture -- …
We propose a new generic type of stochastic neurons, called -neurons, that considers activation functions based on Jackson's -derivatives with stochastic parameters . Our generalization of neural network architectures with -neurons is shown to be both scalable and very easy to implement. We demonstrate expe…
It will be shown that according to theorems of K. Menger, every neuron grid if identified with a curve is able to preserve the adopted qualitative structure of a data space. Furthermore, if this identification is made, the neuron grid structure can always be mapped to a subset of a universal neuron grid which is constr…
Neural networks learn to mimic brain neurons with two-input activation functions, improving performance and robustness.
Unseen or out-of-domain data can seriously degrade the performance of a neural network model, indicating the model's failure to generalize to unseen data. Neural net pruning can not only help to reduce a model's size but can improve the model's generalization capacity as well. Pruning approaches look for low-salient ne…
CHANI learns classification tasks with local transformations inspired by biology.
Researchers develop methods to learn neuron dynamics from colored noise.
Despite our extensive knowledge of biophysical properties of neurons, there is no commonly accepted algorithmic theory of neuronal function. Here we explore the hypothesis that single-layer neuronal networks perform online symmetric nonnegative matrix factorization (SNMF) of the similarity matrix of the streamed data. …
"Sparse" neural networks, in which relatively few neurons or connections are active, are common in both machine learning and neuroscience. Whereas in machine learning, "sparsity" is related to a penalty term that leads to some connecting weights becoming small or zero, in biological brains, sparsity is often created wh…
Continuous deep learning architectures have recently re-emerged as Neural Ordinary Differential Equations (Neural ODEs). This infinite-depth approach theoretically bridges the gap between deep learning and dynamical systems, offering a novel perspective. However, deciphering the inner working of these models is still a…
Stable unactivated neurons reduce expressiveness in ReLU networks.
Graph Neural Nets (GNNs) have received increasing attentions, partially due to their superior performance in many node and graph classification tasks. However, there is a lack of understanding on what they are learning and how sophisticated the learned graph functions are. In this work, we propose a dissection of GNNs …
Deep neural networks (DNNs) are known for extracting useful information from large amounts of data. However, the representations learned in DNNs are typically hard to interpret, especially in dense layers. One crucial issue of the classical DNN model such as multilayer perceptron (MLP) is that neurons in the same layer…
Single-spike neurons can approximate as well as multi-spike neurons.
The challenge of assigning importance to individual neurons in a network is of interest when interpreting deep learning models. In recent work, Dhamdhere et al. proposed Total Conductance, a "natural refinement of Integrated Gradients" for attributing importance to internal neurons. Unfortunately, the authors found tha…
Optimal neuron activation functions improve neural network performance.
The artificial neural network is a popular framework in machine learning. To empower individual neurons, we recently suggested that the current type of neurons could be upgraded to 2nd order counterparts, in which the linear operation between inputs to a neuron and the associated weights is replaced with a nonlinear qu…
Novel 'strong neuron' improves deep learning efficiency and robustness.
The practical success of widely used machine learning (ML) and deep learning (DL) algorithms in Artificial Intelligence (AI) community owes to availability of large datasets for training and huge computational resources. Despite the enormous practical success of AI, these algorithms are only loosely inspired from the b…
With the recent success of deep neural networks in computer vision, it is important to understand the internal working of these networks. What does a given neuron represent? The concepts captured by a neuron may be hard to understand or express in simple terms. The approach we propose in this paper is to characterize t…
Single ReLU neuron's gradient dynamics reveal support vectors as key to generalization.
We view a neural network as a distributed system of which neurons can fail independently, and we evaluate its robustness in the absence of any (recovery) learning phase. We give tight bounds on the number of neurons that can fail without harming the result of a computation. To determine our bounds, we leverage the fact…
This paper presents a new artificial neuron model capable of learning its receptive field in the topological domain of inputs. The model provides adaptive and differentiable local connectivity (plasticity) applicable to any domain. It requires no other tool than the backpropagation algorithm to learn its parameters whi…
Novel method decorrelates neurons for better deep learning model generalization.
A neuron is a basic physiological and computational unit of the brain. While much is known about the physiological properties of a neuron, its computational role is poorly understood. Here we propose to view a neuron as a signal processing device that represents the incoming streaming data matrix as a sparse vector of …
Sometimes it is not enough for a DNN to produce an outcome. For example, in applications such as healthcare, users need to understand the rationale of the decisions. Therefore, it is imperative to develop algorithms to learn models with good interpretability (Doshi-Velez 2017). An important factor that leads to the lac…
New training method for ReLU networks achieves optimal weight size for memorization.