VC dimensions of group CNNs are infinite for certain kernels and groups.
problem Estimating the generalization capacity of group convolutional neural networks.
method Identifying precise VC dimension estimates for simple sets of group CNNs.
result Two-parameter families of convolutional neural networks have an infinite VC dimension for infinite groups and certain kernels.
Enhances group convolutional networks with attention to learn meaningful relationships.
problem Lack of explicit means to learn meaningful relationships among symmetry patterns.
method Introduces attentive group equivariant convolutions, applying attention during convolution.
result Consistently outperforms conventional group convolutional networks on benchmark datasets.
New method improves grouped convolutions on edge devices.
problem Efficiently implementing grouped convolutions on edge devices.
method Grouped Spatial Pack Convolutions (GSPC) in TVM.
result GSPC outperforms existing implementations by 3.4x, 8x, and 4x on average.
Study convolution of invariant valuations on Lie groups.
problem Understanding convolution of valuations on Lie groups.
method Explicit formula for left-invariant valuations, showing existence of smooth bi-invariant valuations, defining convolution on arbitrary Lie groups.
result Unified convolution operations on Lie groups.
We introduce Group equivariant Convolutional Neural Networks (G-CNNs), a natural generalization of convolutional neural networks that reduces sample complexity by exploiting symmetries. G-CNNs use G-convolutions, a new type of layer that enjoys a substantially higher degree of weight sharing than regular convolution la…
This research studies affine invariance in continuous-domain convolutional neural networks.
problem Recognizing patterns and features under affine transformations in continuous domains.
method Introduces a new criterion for assessing affine invariance, embeds images into the affine Lie group, and analyzes convolution over this group.
result Extends the scope of geometrical transformations that deep-learning pipelines can handle.
GroSS enables efficient search for grouped convolutional architectures.
problem Training grouped convolutional architectures efficiently and effectively.
method GroSS: Group-Size Series Decomposition for Grouped Architecture Search.
result Simultaneous training of differing numbers of groups within a single layer and all possible combinations between layers.
HexaConv improves CNN performance by using hexagonal filters and group convolutions.
problem Improving CNN performance by exploiting more symmetries.
method Implementing planar and group convolutions over hexagonal lattices.
result HexaConv outperforms conventional CNNs on aerial scene classification.
Generalizes equivariance and convolution to compact groups for neural networks.
problem Ensuring equivariance in neural networks for various domain actions.
method Representation theory and noncommutative harmonic analysis.
result Convolution is necessary and sufficient for equivariance to compact group actions.
Study learns convolution operators on compact Abelian groups using regularization.
problem Learning convolution operators on compact Abelian groups.
method Regularization-based approach with ridge regression estimator.
result Characterizes the accuracy of the estimator in terms of finite sample bounds.
Generalizes CNNs for Lie group equivariance across various data types.
problem Equivariance to transformations like rotations for non-image data.
method Constructs equivariant convolutional layers for Lie groups.
result Models conserve linear and angular momentum in Hamiltonian systems.
UGConvs improve CNN accuracy with unitary transforms.
problem Improving CNN accuracy with richer representations.
method UGConvs combine group convolutions with unitary transforms.
result HadaNets achieve similar accuracy to circulant networks with lower complexity.
Introduces new algebraic structures for relational groupoids and proves a reduction theorem.
problem Developing algebraic tools for relational groupoids.
method Introduces relational groupoids and convolution algebras, provides examples, and proves a reduction theorem.
result Establishes a reduction theorem recovering the usual convolution of Lie groupoids.
Random convolutional networks can be fooled with adversarial examples.
problem Existence of adversarial examples for random convolutional networks.
method Utilizing isoperimetric inequalities on the special orthogonal group so(d). result Adversarial examples exist for various random convolutional networks.
Paper connects algebraic and analytic methods for braid group representations.
problem Constructing representations of braid groups using algebraic and analytic approaches.
method Katz-Long-Moody construction and multiplicative middle convolution for KZ-type equations.
result Multiplicative middle convolution preserves unitarity and provides an algorithm to determine the signature of a Hermitian matrix.
An impossibility result shows limitations in learning symmetries and equivariant functions.
problem Learning symmetries and equivariant functions simultaneously is impossible under certain conditions.
method Careful study of approximation for groups and semigroups, analysis of neural networks.
result Linearly equivariant networks can be used to learn equivariant functions, but group-convolutional networks have limitations.
We introduce the new notion of convolution of a (smooth or generalized) valuation on a group G and a valuation on a manifold M acted upon by the group. In the case of a transitive group action, we prove that the spaces of smooth and generalized valuations on M are modules over the algebra of compactly supported g…
RotDCF decomposes CNN filters for rotation-equivariant deep networks.
problem Handling global deformations in images for vision tasks.
method Decomposes convolutional filters over joint steerable bases for rotation-equivariance.
result Significantly reduces model size and computational complexity while preserving performance.
B-Spline CNNs on Lie Groups expand G-CNNs to arbitrary groups.
problem Leveraging geometric structure for improved feature learning.
method Lifting feature maps to B-spline expansions on Lie algebra.
result G-CNNs on Lie groups outperform classical 2D CNNs.
PDE-based G-CNNs add geometric symmetries to CNNs without augmentation.
problem Designing CNNs with built-in symmetries like rotation.
method Formulate CNN layers as PDE solvers on homogeneous spaces.
result PDE-G-CNNs achieve better performance with fewer parameters.
Channel pruning and weight binarization improve keyword spotting accuracy.
problem Improving accuracy of keyword spotting in neural networks.
method Group-wise splitting method using group Lasso penalty for channel sparsity, combined with 1-bit weight precision.
result Achieved over 50% channel sparsity with minimal accuracy loss.
Paper learns to rotate filters for group convolutions.
problem Difficult to rotate 3x3 filters on pixel grids.
method Learn filter basis and rotation-invariant coefficients; switch basis for rotation.
result Produces feature maps insensitive to input rotations.
L-CNNs maintain gauge symmetry on non-Abelian lattice theories.
problem Applying convolutional neural networks to non-Abelian lattice gauge theories while preserving gauge symmetry.
method Developed a geometric formulation of L-CNNs that are equivariant under global symmetries and gauge transformations.
result Convolutional operations in L-CNNs are a specific case of gauge-equivariant neural networks on SU(N) principal bundles. Currents on Lie groups form a Hopf algebra structure.
problem Understanding algebraic structure of currents on Lie groups.
method Defined Hopf algebra structure on currents using convolution and wedge product.
result Explicit formulas for Hopf algebra operations on currents are derived.
Study on VC dimension of GCNNs with input resolution effects.
problem Understanding the generalization capabilities of GCNNs.
method Derived upper and lower bounds for VC dimension, analyzed factors affecting it.
result Extended previous results on VC dimension of GCNNs, providing insights into input resolution dependence.
Group-equivariant subsampling layers improve CNNs' equivariance.
problem Non-translation equivariance in subsampling operations.
method Translation and group-equivariant subsampling/upsampling layers.
result Group-equivariant autoencoders learn equivariant representations.
GCNNs on homogeneous spaces use vector bundles and Hilbert spaces.
problem Learning data on homogeneous spaces with global symmetry.
method Analysis of G-equivariant convolutional layers on homogeneous G/K spaces, using vector bundles and reproducing kernel Hilbert spaces. result A precise criterion for expressing G-equivariant layers as convolutional layers, leading to stronger results for some groups. Improved sample efficiency in semantic segmentation with rotation equivariant CNNs.
problem Efficiently segmenting images with rotation and reflection symmetries.
method Introduced rotation-equivariant CNNs with new equivariant convolutions and transposed convolutions.
result Significant gains in sample efficiency and robustness to symmetry transformations.
Generalizes CNNs on homogeneous spaces like Euclidean and spherical surfaces.
problem Classifying and understanding equivariant CNNs on homogeneous spaces.
method Develops a theory for equivariant maps between field spaces of given types.
result Equivariant kernels correspond to the most general kind of equivariant linear maps.
This work introduces a method for almost equivariance in neural networks using Lie algebra convolutions.
problem Real-world data often does not conform to strict group equivariances, leading to underperformance in models.
method Definition and practical implementation of almost equivariance through Lie algebra convolutions.
result Demonstrated the validity of the approach through benchmarking against fully equivariant settings.
The success of deep convolutional architectures is often attributed in part to their ability to learn multiscale and invariant representations of natural signals. However, a precise study of these properties and how they affect learning guarantees is still missing. In this paper, we consider deep convolutional represen…
ScDCFNet improves multiscale image classification with reduced model size.
problem Improving performance in multiscale image classification.
method Decomposed convolutional filters for ST-equivariant CNNs.
result ScDCFNet achieves significantly improved performance in multiscale image classification.
Graph Convolutional Neural Networks identify molecular functional groups.
problem Automatic discovery of molecular functional groups to reduce lab experiments.
method Graph Convolutional Neural Networks (GCNNs) trained on relational graphs of molecules.
result Grad-CAM method identified the most specific and relevant molecular substructures.
Researchers enhance EfficientNet models for practical efficiency on Graphcore IPU.
problem Improving practical efficiency of EfficientNet models on high-performance accelerators.
method Group convolutions, proxy-normalized activations, and reduced training resolution.
result Improves practical efficiency for both training and inference on Graphcore IPU.
Paper introduces method to make neural networks symmetrical.
problem Creating symmetrical neural networks for data with inherent symmetries.
method Introduces a method for modifying neural networks to enforce equivariance.
result Group convolutional neural networks are a special case of the introduced framework.
Explicit Taylor series for the volume of tubes in Lie groups
problem Computing the volume of tubes in riemannian manifolds
method Using bi-invariant metrics
result Explicit Taylor series for the volume of a tube in a Lie group
E2GC optimizes energy efficiency in DNNs by balancing computational and data movement costs.
problem Imbalance between computational complexity and data reuse in GConv leads to suboptimal energy efficiency.
method Developed an optimum group size model and proposed E2GC module with constant group size.
result E2GC modules improve energy efficiency by 10.8% and 4.73% on P100 and P4000 GPUs, respectively.
LieTransformer extends self-attention to Lie groups for improved deep learning tasks.
problem Improving deep learning performance through group equivariant self-attention.
method LieSelfAttention layers that are equivariant to arbitrary Lie groups and their discrete subgroups.
result Competitive experimental results on various tasks.
Study risk sharing among agents with varying risk preferences.
problem Risk sharing among agents with heterogeneous risk measures.
method Derive explicit solutions for inf-convolution and counter-monotonic inf-convolution under varying risk seeking.
result Explicit solutions for inf-convolution and counter-monotonic inf-convolution can be represented by a generalization of distortion risk measures.
Simplifies convolutions using tensor networks and einsum for efficient second-order methods.
problem Complexity in analyzing and applying convolutions in deep learning.
method Viewing convolutions as tensor networks, drawing diagrams, and using einsum for efficient computation.
result Accelerates a KFAC variant up to 4.5x with reduced memory overhead.
ChebLieNet uses Lie groups to create invariant spectral graph networks.
problem Handling anisotropic data in graph neural networks.
method Develops anisotropic convolutional layers on Lie groups with Riemannian metrics.
result Demonstrates the effectiveness of balancing equivariance and invariance.
We study Kontsevich's deformation quantization for the dual of a finite-dimensional real Lie algebra (or superalgebra) g. In this case the Kontsevich star-product defines a new convolution on S(g), regarded as the space of distributions supported at 0 in g. For p in S(g), we show that the convolution operator f->f*p is…
Unified Long-Moody and Katz methods for constructing local systems.
problem Constructing representations of braid groups and local systems.
method Katz-Long-Moody functor unifying Long-Moody and Katz methods.
result Extends Katz algorithm to various topological spaces.
New multigrid approach reduces CNN parameters by focusing on structured convolutions.
problem Redundancy in standard CNNs leads to high parameter count.
method Replace standard convolutions with structured multilevel convolutions.
result Linearly proportional number of parameters to network width, no loss in accuracy.
This paper completes the construction of arbitrary order conformally invariant differential operators in higher spin spaces. Jan Slovák has classified all conformally invariant differential operators on locally conformally flat manifolds. We complete his results in higher spin theory by giving explicit expressions for …
The abstract theorem extends a Lie group result to Lie groupoids.
problem Expressing functions on Lie groupoids as convolutions of two functions.
method Using a lemma from Dixmier-Malliavin, Lie algebroids, and exponential map.
result Every smooth, compactly-supported function on a Lie groupoid can be expressed as a finite sum of convolutions of two such functions.
New optimization algorithms on orthogonal group for machine learning.
problem Efficient optimization on the orthogonal group for machine learning tasks.
method Stochastic geometric algorithms on Lie groups.
result Strong performance on diverse machine learning tasks.
Introspects convolutional speech recognition models using Gradient-adjusted Neuron Activation Profiles.
problem Lack of interpretability in deep learning ASR models.
method Gradient-adjusted Neuron Activation Profiles (GradNAPs) for feature and representation visualization.
result Gains insight into how data is processed in convolutional ASR models.