New translation equivariant neural processes improve spatio-temporal data modeling.
problem Improving posterior prediction maps for spatio-temporal data.
method Introduced translation equivariant transformers within neural processes.
result TE-TNPs outperform non-equivariant TNPs and other baselines.
Group-equivariant subsampling layers improve CNNs' equivariance.
problem Non-translation equivariance in subsampling operations.
method Translation and group-equivariant subsampling/upsampling layers.
result Group-equivariant autoencoders learn equivariant representations.
We introduce the Convolutional Conditional Neural Process (ConvCNP), a new member of the Neural Process family that models translation equivariance in the data. Translation equivariance is an important inductive bias for many learning problems including time series modelling, spatial data, and images. The model embeds …
New methods improve translation-equivariant neural processes for modeling unknown functions.
problem Modeling unknown latent functions from irregularly sampled measurements.
method Volterra series and set Fourier convolutions to address translation-equivariance and efficiency.
result Improved translation-equivariant neural processes with analytical transparency and linear scalability.
ConvNP improves SP prediction with translation equivariance and coherent samples.
problem Predicting stationary stochastic processes with coherent samples.
method Convolutional Neural Processes (ConvNP) with a new maximum-likelihood objective.
result ConvNP outperforms standard NPs and demonstrates strong generalization on various tasks.
Paper introduces Laplace-HDC for better binary hyperdimensional computing.
problem Improving binary hyperdimensional computing for spatial information.
method Develops Laplace-HDC using the Laplace kernel and Haar convolutional features.
result Laplace-HDC outperforms previous methods in encoding spatial information.
Generalizes CNNs for Lie group equivariance across various data types.
problem Equivariance to transformations like rotations for non-image data.
method Constructs equivariant convolutional layers for Lie groups.
result Models conserve linear and angular momentum in Hamiltonian systems.
Encoding the scale information explicitly into the representation learned by a convolutional neural network (CNN) is beneficial for many computer vision tasks especially when dealing with multiscale inputs. We study, in this paper, a scaling-translation-equivariant (ST-equivariant) CNN with joint convolutions across th…
GNP models predictive correlations and outperforms NPs.
problem Training and understanding of Neural Processes.
method Proposed a new model, Gaussian Neural Process (GNP), which incorporates translation equivariance and provides universal approximation guarantees.
result Demonstrates encouraging performance and provides universal approximation guarantees.
Bayesian convolutional deep sets improve ambiguity in stationary process modeling.
problem Ambiguity in translation equivariant functional representations due to insufficient data points.
method Introduce Bayesian convolutional deep sets with task-dependent stationary prior.
result Improves representation quality compared to kernel smoother and non-parametric models.
Wavelet Networks learn from raw time-series data, outperforming conventional CNNs.
problem Learning from raw time-series data efficiently and effectively.
method Constructing scale-translation equivariant neural networks based on wavelet symmetries.
result Wavelet Networks outperform conventional CNNs on raw waveforms and spectrograms.
SE(3)-Transformers maintain equivariance for 3D data under rotations and translations.
problem Ensuring stable and predictable performance in 3D data under transformations.
method Introducing a self-attention module that is equivariant under continuous 3D roto-translations.
result The SE(3)-Transformer outperforms non-equivariant and non-attention models on real-world datasets.
New neural network processes 3D volumes with improved equivariance.
problem Improving neural network performance on 3D volumes with symmetries.
method Equivariant neural network using moving frames approach.
result Trained model outperforms benchmarks in medical volume classification.
A new metric mav offers a practical alternative to costly Riemannian distance.
problem Efficiently compute Riemannian distance on SE(3) invariant metrics.
method Propose mav distance, defined as Riemannian length of a curve.
result Mav distance offers a trainable invariant for geometric deep learning.
This work relaxes GNN symmetries to approximate automorphisms, improving model performance.
problem Improving graph neural network performance on asymmetric graphs.
method Formalizing approximate symmetries via graph coarsening, introducing a bias-variance formula.
result Best generalization performance achieved by choosing a larger symmetry group than automorphisms but smaller than permutations.
We present a PDE-based framework that generalizes Group equivariant Convolutional Neural Networks (G-CNNs). In this framework, a network layer is seen as a set of PDE-solvers where geometrically meaningful PDE-coefficients become the layer's trainable weights. Formulating our PDEs on homogeneous spaces allows these net…
Convolutional Neural Processes improve data efficiency in neural processes.
problem Improving data efficiency in neural processes for small datasets.
method Convolutional Neural Processes (ConvNPs) improve data efficiency by leveraging translation equivariance and convolutional neural networks.
result ConvNPs enhance the performance of neural processes in small-data problems.
New method constructs translationally equivariant hyperbolic affine spheres.
problem Constructing translationally equivariant hyperbolic affine spheres.
method Noncompact Iwasawa factorization via DPW method and Weierstrass elliptic functions.
result Every translationally equivariant hyperbolic affine sphere is equiaffinely equivalent to one with a circle, hyperbola, or parabola slice curve.
Proposes learning invariances in neural networks using a weight-space approach.
problem Learning invariances from data in neural networks remains an open problem.
method Minimizes a lower bound on the marginal likelihood in weight space.
result Results in higher performing models with naturally learned invariances.
The effectiveness of Convolutional Neural Networks (CNNs) has been substantially attributed to their built-in property of translation equivariance. However, CNNs do not have embedded mechanisms to handle other types of transformations. In this work, we pay attention to scale changes, which regularly appear in various t…
Generative adversarial networks fix aliasing issues by making signals continuous.
problem Alias-free generation in GANs to prevent unwanted information leakage.
method Interpreting all signals as continuous, deriving small architectural changes.
result Generative models match FID of StyleGAN2 but have better internal representations.
Researchers analyze neural process architectures and their representational capacities.
problem Understanding what functions can be represented by different neural process architectures.
method Analyzing four types of neural process architectures: CNPs, ANPs, TNPs, and their latent variants.
result Prove these architectures form a strict hierarchy and characterize their representational capabilities.
EquivCNP learns group symmetries for conditional data.
problem Learning conditional models with data symmetries.
method Group equivariant decomposition and Lie group convolutional layers.
result EquivCNP achieves comparable performance and zero-shot generalization.
Transformation Equivariant Representations (TERs) aim to capture the intrinsic visual structures that equivary to various transformations by expanding the notion of {\em translation} equivariance underlying the success of Convolutional Neural Networks (CNNs). For this purpose, we present both deterministic AutoEncoding…
Translating or rotating an input image should not affect the results of many computer vision tasks. Convolutional neural networks (CNNs) are already translation equivariant: input image translations produce proportionate feature map translations. This is not the case for rotations. Global rotation equivariance is typic…
Equivariant neural networks improve performance and generalization in complex scalar field theory tasks.
problem Improving performance and generalization in neural networks for complex scalar field theory tasks.
method Incorporating translational equivariance into neural network architectures.
result Equivariant neural networks significantly outperform non-equivariant networks in various tasks, including those beyond the training set and across different lattice sizes.
Develops approximately equivariant neural processes for better data modeling.
problem Real-world data often breaks exact equivariance; how to model this?
method General approach to creating approximately equivariant architectures, applicable to any model and symmetry group.
result Approximately equivariant neural processes outperform non-equivariant and strictly equivariant models in regression tasks.
GCNNs gain rotation invariance with more training augmentation, making SVD-Universal more effective.
problem Improving robustness of GCNNs to adversarial attacks.
method SVD-Universal technique applied to GCNNs trained with larger rotations.
result SVD-Universal becomes more effective as GCNNs gain rotation invariance.
Measures equivariance in vision models using Lie derivative.
problem Understanding the role of equivariance in recent vision models.
method Introducing Lie derivative to measure equivariance with strong mathematical foundations and minimal hyperparameters.
result Many violations of equivariance can be linked to spatial aliasing in network layers, and larger models tend to display more equivariance.
This paper develops optimal transport methods on the roto-translation group SE2.
problem Optimal transport on the roto-translation group SE2 for image analysis.
method Develops a computational framework for optimal transportation over Lie groups, focusing on SE2. Uses Sinkhorn-like algorithm with efficient distance approximations.
result Advances in image barycentric interpolation, orientation field interpolation, and Wasserstein flows on SE2.
There is a previously identified equivalence between wide fully connected neural networks (FCNs) and Gaussian processes (GPs). This equivalence enables, for instance, test set predictions that would have resulted from a fully Bayesian, infinitely wide trained FCN to be computed without ever instantiating the FCN, but b…
A new geometric perceptron model improves 3D shape classification.
problem Challenges in geometric tasks involving point clouds using machine learning.
method Introduces multilayer geometric perceptron (MLGP) with geometric neurons.
result MLGP outperforms vanilla MLP in 3D shape classification and noise resistance.