arXiv research

Study on the geometric Dyson Brownian motion of non-square matrix products.

We analyze the eigenvalue distribution of a neural network's kernel under specific scaling.

Paper analyzes the free energy of CNNs with skip connections in Bayesian learning.

Ensemble learning is a method of combining multiple trained models to improve model accuracy. We propose the usage of such methods, specifically ensemble average, inside Convolutional Neural Network (CNN) architectures by replacing the single convolutional layers with Inner Average Ensembles (IEA) of multiple convoluti…

New method predicts neural network performance using free probability theory.

A framework uses free probability to analyze Transformer models.

In this letter, we generalize the convolutional NMF by taking the $β$-divergence as the contrast function and present the correct multiplicative updates for its factors in closed form. The new updates unify the $β$-NMF and the convolutional NMF. We state why almost all of the existing updates are inexact and approximat…

Classifies compact multiplicity free quasi-Hamiltonian manifolds.

Let $(M,ω)$ be a connected symplectic manifold on which a connected Lie group $G$ acts properly and in a Hamiltonian fashion with moment map $μ:M \lra \mf g^*$. Our purpose is investigate multiplicity-free actions, giving criteria to decide a multiplicity freenes of the action. As an application we give the complete cl…

Study on Bayesian deep linear networks with multiple outputs and convolutional layers.

Yes, they do. This paper provides the first empirical demonstration that deep convolutional models really need to be both deep and convolutional, even when trained with methods such as distillation that allow small or shallow models of high accuracy to be trained. Although previous research showed that shallow feed-for…

A (quasi-)Hamiltonian manifold is called multiplicity free if all of its symplectic reductions are 0-dimensional. In this paper, we classify multiplicity free Hamiltonian actions for (twisted) loop groups or, equivalently, multiplicity free (twisted) quasi-Hamiltonian manifolds for simply connected compact Lie groups. …

Analog arrays are a promising upcoming hardware technology with the potential to drastically speed up deep learning. Their main advantage is that they compute matrix-vector products in constant time, irrespective of the size of the matrix. However, early convolution layers in ConvNets map very unfavorably onto analog a…

Paper connects algebraic and analytic methods for braid group representations.

We present an alternative layer to convolution layers in convolutional neural networks (CNNs). Our approach reduces the complexity of convolutions by replacing it with binary decisions. Those binary decisions are used as indexes to conditional distributions where each weight represents a leaf in a decision tree. This m…

Winograd convolution is widely used in deep neural networks (DNNs). Existing work for DNNs considers only the subset Winograd algorithms that are equivalent to Toom-Cook convolution. We investigate a wider range of Winograd algorithms for DNNs and show that these additional algorithms can significantly improve floating…

In this paper, we introduce a Deep Convolutional Analysis Dictionary Model (DeepCAM) by learning convolutional dictionaries instead of unstructured dictionaries as in the case of deep analysis dictionary model introduced in the companion paper. Convolutional dictionaries are more suitable for processing high-dimensiona…

In this work we detail the application of a fast convolution algorithm computing high dimensional integrals to the context of multiplicative noise stochastic processes. The algorithm provides a numerical solution to the problem of characterizing conditional probability density functions at arbitrary time, and we applie…

We compute the sheaf of automorphisms of a multiplicity free Hamiltonian manifold over its momentum polytope and show that its higher cohomology groups vanish. Together with a theorem of Losev, arXiv:math/0612561, this implies a conjecture of Delzant: a compact multiplicity free Hamiltonian manifold is uniquely determi…

ViLT is a faster vision-and-language model without convolution or region supervision.

Two new inverse-free ELM algorithms for incremental and decremental learning are proposed.

Deep convolutional networks often append additive constant ("bias") terms to their convolution operations, enabling a richer repertoire of functional mappings. Biases are also used to facilitate training, by subtracting mean response over batches of training images (a component of "batch normalization"). Recent state-o…

In this paper, we propose a deep multimodal fusion network to fuse multiple modalities (face, iris, and fingerprint) for person identification. The proposed deep multimodal fusion algorithm consists of multiple streams of modality-specific Convolutional Neural Networks (CNNs), which are jointly optimized at multiple fe…

Study shows deterministic equivalent for neural network kernel convergence.

CNN predicts stock fluctuations using company news headlines.

We introduce the notion of a conjugation-free geometric presentation for a fundamental group of a line arrangement's complement, and we show that the fundamental groups of the following family of arrangements have a conjugation-free geometric presentation: A real arrangement L, whose graph of multiple points is a union…

The introduction of convolutional layers greatly advanced the performance of neural networks on image tasks due to innately capturing a way of encoding and learning translation-invariant operations, matching one of the underlying symmetries of the image domain. In comparison, there are a number of problems in which the…

We show that Tolman's example (of a six dimensional Hamiltonian $T^2$-space with isolated fixed points and no compatible Kähler structure) can be constructed from the flag variety $U(3)/U(1)^3$ by $U(2)$-equivariant symplectic surgery. This implies that Tolman's space has a ``transversal multiplicity-free'' action of $…

Spatio-temporal prediction plays an important role in many application areas especially in traffic domain. However, due to complicated spatio-temporal dependency and high non-linear dynamics in road networks, traffic prediction task is still challenging. Existing works either exhibit heavy training cost or fail to accu…

Functor connects Lie groupoid algebras to bornological structures.

Post-hoc explanations improve CNNs by replacing final linear layer with k-means classifier.

We classify compact, connected Hamiltonian and quasi-Hamiltonian manifolds of cohomogeneity one (which is the same as being multiplicity free of rank one). Here the group acting is a compact connected Lie group (simply connected in the quasi-Hamiltonian case). This work is a concretization of the more general classific…

In this paper, we develop a method for unsupervised clustering of two-way (matrix) data by combining two recent innovations from different fields: the Sparse Subspace Clustering (SSC) algorithm [10], which groups points coming from a union of subspaces into their respective subspaces, and the t-product [18], which was …

The study proves the existence of free boundary minimal disks in convex regions.

The complex analytic methods have found a wide range of applications in the study of multiplicity-free representations. This article discusses, in particular, its applications to the question of restricting highest weight modules with respect to reductive symmetric pairs. We present a number of multiplicity-free branch…

Study risk sharing with Lambda VaR under diverse beliefs.

CDEFs reduce model complexity and uncover time correlations.

In our study, we demonstrate the synergy effect between convolutional neural networks and the multiplicity of SMILES. The model we propose, the so-called Convolutional Neural Fingerprint (CNF) model, reaches the accuracy of traditional descriptors such as Dragon (Mauri et al. [22]), RDKit (Landrum [18]), CDK2 (Willigha…

Convolutional Neural Network (CNN) has demonstrated impressive ability to represent hyperspectral images and to achieve promising results in hyperspectral image classification. However, traditional CNN models can only operate convolution on regular square image regions with fixed size and weights, so they cannot univer…

Recurrent convolution (RC) shares the same convolutional kernels and unrolls them multiple steps, which is originally proposed to model time-space signals. We argue that RC can be viewed as a model compression strategy for deep convolutional neural networks. RC reduces the redundancy across layers. However, the perform…

Health professionals can use natural language processing (NLP) technologies when reviewing electronic health records (EHR). Machine learning free-text classifiers can help them identify problems and make critical decisions. We aim to develop deep learning neural network algorithms that identify EHR progress notes perta…

Deep learning has been widely applied and brought breakthroughs in speech recognition, computer vision, and many other domains. The involved deep neural network architectures and computational issues have been well studied in machine learning. But there lacks a theoretical foundation for understanding the approximation…

Proves multiplicity one for boundary minimal hypersurfaces in compact manifolds.

Superior performance and ease of implementation have fostered the adoption of Convolutional Neural Networks (CNNs) for a wide array of inference and reconstruction tasks. CNNs implement three basic blocks: convolution, pooling and pointwise nonlinearity. Since the two first operations are well-defined only on regular-s…

Study multiplicity-free covering of graded manifolds, proving equivalence of categories.

Learning representation on graph plays a crucial role in numerous tasks of pattern recognition. Different from grid-shaped images/videos, on which local convolution kernels can be lattices, however, graphs are fully coordinate-free on vertices and edges. In this work, we propose a Gaussian-induced convolution (GIC) fra…

Deep neural network compression techniques such as pruning and weight tensor decomposition usually require fine-tuning to recover the prediction accuracy when the compression ratio is high. However, conventional fine-tuning suffers from the requirement of a large training set and the time-consuming training procedure. …

Explicit Taylor series for the volume of tubes in Lie groups