arXiv research

VC dimensions of group CNNs are infinite for certain kernels and groups.

New method improves grouped convolutions on edge devices.

Study convolution of invariant valuations on Lie groups.

Although group convolutional networks are able to learn powerful representations based on symmetry patterns, they lack explicit means to learn meaningful relationships among them (e.g., relative positions and poses). In this paper, we present attentive group equivariant convolutions, a generalization of the group convo…

We introduce Group equivariant Convolutional Neural Networks (G-CNNs), a natural generalization of convolutional neural networks that reduces sample complexity by exploiting symmetries. G-CNNs use G-convolutions, a new type of layer that enjoys a substantially higher degree of weight sharing than regular convolution la…

This research studies affine invariance in continuous-domain convolutional neural networks.

Study learns convolution operators on compact Abelian groups using regularization.

We present a novel approach which is able to explore the configuration of grouped convolutions within neural networks. Group-size Series (GroSS) decomposition is a mathematical formulation of tensor factorisation into a series of approximations of increasing rank terms. GroSS allows for dynamic and differentiable selec…

Generalizes CNNs for Lie group equivariance across various data types.

The effectiveness of Convolutional Neural Networks stems in large part from their ability to exploit the translation invariance that is inherent in many learning problems. Recently, it was shown that CNNs can exploit other invariances, such as rotation invariance, by using group convolutions instead of planar convoluti…

Introduces new algebraic structures for relational groupoids and proves a reduction theorem.

Random convolutional networks can be fooled with adversarial examples.

Convolutional neural networks have been extremely successful in the image recognition domain because they ensure equivariance to translations. There have been many recent attempts to generalize this framework to other domains, including graphs and data lying on manifolds. In this paper we give a rigorous, theoretical t…

Paper connects algebraic and analytic methods for braid group representations.

An impossibility result shows limitations in learning symmetries and equivariant functions.

We introduce the new notion of convolution of a (smooth or generalized) valuation on a group $G$ and a valuation on a manifold $M$ acted upon by the group. In the case of a transitive group action, we prove that the spaces of smooth and generalized valuations on $M$ are modules over the algebra of compactly supported g…

Explicit encoding of group actions in deep features makes it possible for convolutional neural networks (CNNs) to handle global deformations of images, which is critical to success in many vision tasks. This paper proposes to decompose the convolutional filters over joint steerable bases across the space and the group …

We propose unitary group convolutions (UGConvs), a building block for CNNs which compose a group convolution with unitary transforms in feature space to learn a richer set of representations than group convolution alone. UGConvs generalize two disparate ideas in CNN architecture, channel shuffling (i.e. ShuffleNet) and…

L-CNNs maintain gauge symmetry on non-Abelian lattice theories.

We present a PDE-based framework that generalizes Group equivariant Convolutional Neural Networks (G-CNNs). In this framework, a network layer is seen as a set of PDE-solvers where geometrically meaningful PDE-coefficients become the layer's trainable weights. Formulating our PDEs on homogeneous spaces allows these net…

Group convolutional neural networks (G-CNNs) can be used to improve classical CNNs by equipping them with the geometric structure of groups. Central in the success of G-CNNs is the lifting of feature maps to higher dimensional disentangled representations, in which data characteristics are effectively learned, geometri…

We propose a semantic segmentation model that exploits rotation and reflection symmetries. We demonstrate significant gains in sample efficiency due to increased weight sharing, as well as improvements in robustness to symmetry transformations. The group equivariant CNN framework is extended for segmentation by introdu…

Currents on Lie groups form a Hopf algebra structure.

Study on VC dimension of GCNNs with input resolution effects.

Group-equivariant subsampling layers improve CNNs' equivariance.

Encoding the scale information explicitly into the representation learned by a convolutional neural network (CNN) is beneficial for many computer vision tasks especially when dealing with multiscale inputs. We study, in this paper, a scaling-translation-equivariant (ST-equivariant) CNN with joint convolutions across th…

GCNNs on homogeneous spaces use vector bundles and Hilbert spaces.

This work introduces a method for almost equivariance in neural networks using Lie algebra convolutions.

The success of deep convolutional architectures is often attributed in part to their ability to learn multiscale and invariant representations of natural signals. However, a precise study of these properties and how they affect learning guarantees is still missing. In this paper, we consider deep convolutional represen…

We study channel number reduction in combination with weight binarization (1-bit weight precision) to trim a convolutional neural network for a keyword spotting (classification) task. We adopt a group-wise splitting method based on the group Lasso penalty to achieve over 50% channel sparsity while maintaining the netwo…

Researchers enhance EfficientNet models for practical efficiency on Graphcore IPU.

Explicit Taylor series for the volume of tubes in Lie groups

E2GC optimizes energy efficiency in DNNs by balancing computational and data movement costs.

LieTransformer extends self-attention to Lie groups for improved deep learning tasks.

Study risk sharing among agents with varying risk preferences.

Simplifies convolutions using tensor networks and einsum for efficient second-order methods.

ChebLieNet uses Lie groups to create invariant spectral graph networks.

We study Kontsevich's deformation quantization for the dual of a finite-dimensional real Lie algebra (or superalgebra) g. In this case the Kontsevich star-product defines a new convolution on S(g), regarded as the space of distributions supported at 0 in g. For p in S(g), we show that the convolution operator f->f*p is…

Unified Long-Moody and Katz methods for constructing local systems.

New multigrid approach reduces CNN parameters by focusing on structured convolutions.

This paper completes the construction of arbitrary order conformally invariant differential operators in higher spin spaces. Jan Slovák has classified all conformally invariant differential operators on locally conformally flat manifolds. We complete his results in higher spin theory by giving explicit expressions for …

The abstract theorem extends a Lie group result to Lie groupoids.

New optimization algorithms on orthogonal group for machine learning.

Introspects convolutional speech recognition models using Gradient-adjusted Neuron Activation Profiles.

Group equivariant neural networks simplify complex tasks with group representation theory.

Coordinate-independent convolutions on manifolds avoid reference frame ambiguity.

Convolutional Neural Networks (CNNs) require a large amount of annotated data to learn from, which is often difficult to obtain in the medical domain. In this paper we show that the sample complexity of CNNs can be significantly improved by using 3D roto-translation group convolutions (G-Convs) instead of the more conv…

Machine learning techniques have become increasingly popular in the field of resting state fMRI (functional magnetic resonance imaging) network based classification. However, the application of convolutional networks has been proposed only very recently and has remained largely unexplored. In this paper we describe a c…