A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
SPTN uses invertible transformations to improve sum-product networks.
Transformers can simulate MLE for Bayesian network sequences.
Researchers improve transformer networks' optimization and understanding.
Unified method for deriving ridgelet transforms for various neural network architectures.
B-cos transforms improve neural network interpretability by aligning weights.
Data augmentations are important ingredients in the recipe for training robust neural networks, especially in computer vision. A fundamental question is whether neural network features encode data augmentation transformations. To answer this question, we introduce a systematic approach to investigate which layers of ne…
Quantum Ridgelet Transform speeds up neural network learning.
This paper proposes network recasting as a general method for network architecture transformation. The primary goal of this method is to accelerate the inference process through the transformation, but there can be many other practical applications. The method is based on block-wise recasting; it recasts each source bl…
CaTs use DAGs with transformers to enforce causal constraints, improving neural network robustness.
Scattering transforms are non-trainable deep convolutional architectures that exploit the multi-scale resolution of a wavelet filter bank to obtain an appropriate representation of data. More importantly, they are proven invariant to translations, and stable to perturbations that are close to translations. This stabili…
The study analyzes the convergence rate of a large transformer model.
Proposes a neural network method to correct residual distortions in coordinate transformations.
Enhances non-life insurance pricing models using transformer models.
Data augmentation (DA) is fundamental against overfitting in large convolutional neural networks, especially with a limited training dataset. In images, DA is usually based on heuristic transformations, like geometric or color transformations. Instead of using predefined transformations, our work learns data augmentati…
Transforms between neural networks using manifold-learning techniques.
Learning image transformations is essential to the idea of mental simulation as a method of cognitive inference. We take a connectionist modeling approach, using planar neural networks to learn fundamental imagery transformations, like translation, rotation, and scaling, from perceptual experiences in the form of image…
Top-down information plays a central role in human perception, but plays relatively little role in many current state-of-the-art deep networks, such as Convolutional Neural Networks (CNNs). This work seeks to explore a path by which top-down information can have a direct impact within current deep networks. We explore …
The wavelet transform has seen success when incorporated into neural network architectures, such as in wavelet scattering networks. More recently, it has been shown that the dual-tree complex wavelet transform can provide better representations than the standard transform. With this in mind, we extend our previous meth…
We prove that the global minimum of the backpropagation (BP) training problem of neural networks with an arbitrary nonlinear activation is given by the ridgelet transform. A series of computational experiments show that there exists an interesting similarity between the scatter plot of hidden parameters in a shallow ne…
Probabilistic STNs improve image classification and robustness.
Transformers can outperform feedforward and recurrent networks due to dynamic sparsity.
Fusion of transformer networks using optimal transport for improved performance.
The key idea of current deep learning methods for dense prediction is to apply a model on a regular patch centered on each pixel to make pixel-wise predictions. These methods are limited in the sense that the patches are determined by network architecture instead of learned from data. In this work, we propose the dense…
GIT-Net uses neural networks to approximate PDE operators efficiently.
Transformer networks approximate Hölder and Sobolev functions with fixed-depth networks.
The paper explores properties of the Radon transform in relation to neural networks and ridges.
Previous research has shown that computation of convolution in the frequency domain provides a significant speedup versus traditional convolution network implementations. However, this performance increase comes at the expense of repeatedly computing the transform and its inverse in order to apply other network operati…
This thesis explores GNNs, categorizing them into local and global approaches.
An audiovisual speaker conversion method is presented for simultaneously transforming the facial expressions and voice of a source speaker into those of a target speaker. Transforming the facial and acoustic features together makes it possible for the converted voice and facial expressions to be highly correlated and f…
We present graph wavelet neural network (GWNN), a novel graph convolutional neural network (CNN), leveraging graph wavelet transform to address the shortcomings of previous spectral graph CNN methods that depend on graph Fourier transform. Different from graph Fourier transform, graph wavelet transform can be obtained …
The paper provides convergence guarantees for ODE-based generative models using transformers.
Variational inference provides approximations to the computationally intractable posterior distribution in Bayesian networks. A prominent medical application of noisy-or Bayesian network is to infer potential diseases given observed symptoms. Previous studies focus on approximating a handful of complicated pathological…
Study presents a method to induce a generalized neural network from joint group invariant functions.
This research studies affine invariance in continuous-domain convolutional neural networks.
Transformer-M learns molecular data in 2D or 3D formats.
Single-layer Transformer can approximate any sequence mapping.
Study explores how neural networks and Transformers learn modular arithmetic with multiple inputs.
Transformers excel at sparse token selection, surpassing FCNs in both worst and average cases.
The scattering transform is a multilayered wavelet-based deep learning architecture that acts as a model of convolutional neural networks. Recently, several works have introduced generalizations of the scattering transform for non-Euclidean settings such as graphs. Our work builds upon these constructions by introducin…
We propose unitary group convolutions (UGConvs), a building block for CNNs which compose a group convolution with unitary transforms in feature space to learn a richer set of representations than group convolution alone. UGConvs generalize two disparate ideas in CNN architecture, channel shuffling (i.e. ShuffleNet) and…
How can we recognise social roles of people, given a completely unlabelled social network? We present a transfer learning approach to network role classification based on feature transformations from each network's local feature distribution to a global feature space. Experiments are carried out on real-world datasets.…
New mesh network preserves symmetries in deep learning.
Transformers approximate mean-field dynamics of indistinguishable particles.
Transformers struggle to approximate smooth functions, relying on piecewise constant approximations.
How can prior knowledge on the transformation invariances of a domain be incorporated into the architecture of a neural network? We propose Equivariant Transformers (ETs), a family of differentiable image-to-image mappings that improve the robustness of models towards pre-defined continuous transformation groups. Throu…
We introduce a general method of performing Residual Network inference and learning in the JPEG transform domain that allows the network to consume compressed images as input. Our formulation leverages the linearity of the JPEG transform to redefine convolution and batch normalization with a tune-able numerical approxi…
Derives an empirical capacity model for self-attention neural networks.
Equivariance is a nice property to have as it produces much more parameter efficient neural architectures and preserves the structure of the input through the feature mapping. Even though some combinations of transformations might never appear (e.g. an upright face with a horizontal nose), current equivariant architect…