Bayesian network learns data invariances without augmentation.
problem Learning invariances in neural networks without manual design.
method Bayesian approach infers weight-sharing schemes from data.
result Model outperforms non-invariant networks on specific tasks.
New research shows invariant networks can approximate any continuous function.
problem Can invariant networks approximate any continuous invariant function?
method Considered a general case where G acts on Rn by permuting coordinates. Proved two main results: 1) G-invariant networks are universal with high-order tensors, 2) higher-order tensors are necessary for universality with some groups. result Invariant networks can approximate any continuous invariant function under certain conditions.
Efficient neural network invariant to symmetry subgroups.
problem Designing neural networks invariant to symmetry subgroups for computational efficiency.
method A new G-invariant transformation module and multi-layer perceptron. result The proposed architecture is computationally and memory efficient, and universal.
Learn invariances in neural networks by optimizing over augmentation parameters.
problem Lack of knowledge about present invariances and their extent in data.
method Parameterize a distribution over augmentations and optimize network parameters and augmentation parameters simultaneously.
result Recover correct set and extent of invariances on various tasks from training data alone.
New method for invariant neural networks using probabilistic symmetries.
problem Improving neural network performance in data-scarce, non-i.i.d., or unsupervised settings.
method Characterizing neural network structures invariant under compact group actions using probabilistic symmetry.
result Established a link between functional and probabilistic symmetry, yielding generative representations of invariant distributions.
The study analyzes neural network predictions of knot invariants and finds that braid representations work best.
problem Understanding and predicting knot invariants using neural networks.
method Investigated different knot representations and invariants, proposed a cosine similarity score.
result Braid representations are best for predicting knot invariants, and some invariants are easier to learn than others.
Study improves neural network generalization for invariant and equivariant data.
problem Developing a generalization theory for invariant and equivariant neural networks.
method Introducing quotient feature spaces to measure the effect of group actions on properties and proving a generalization error bound.
result The volume of quotient feature spaces can describe the generalization error and invariance/equivariance significantly improve the bound.
Deep neural networks can approximate invariant/equivariant functions with fewer parameters.
problem Approximating functions that respect group symmetries with neural networks.
method Constructing deep neural networks with G-actions and G-equivariant/invariant affine transformations. result Deep neural networks can approximate G-invariant/equivariant functions with exponentially fewer parameters. Proposes learning invariances in neural networks using a weight-space approach.
problem Learning invariances from data in neural networks remains an open problem.
method Minimizes a lower bound on the marginal likelihood in weight space.
result Results in higher performing models with naturally learned invariances.
This research studies affine invariance in continuous-domain convolutional neural networks.
problem Recognizing patterns and features under affine transformations in continuous domains.
method Introduces a new criterion for assessing affine invariance, embeds images into the affine Lie group, and analyzes convolution over this group.
result Extends the scope of geometrical transformations that deep-learning pipelines can handle.
Frame Averaging makes neural networks invariant or equivariant to new symmetries.
problem Designing neural networks that respect symmetries while being expressive and efficient.
method Introduces Frame Averaging (FA) as a systematic framework to adapt architectures to become invariant or equivariant to new symmetries.
result Frame Averaging guarantees exact invariance or equivariance while being simpler to compute than full group averaging.
New network learns non-parametric invariances from data.
problem Modeling non-parametric invariances in data.
method Introduces PRC-NPTN networks with permanent random connectomes.
result Improves generalization and outperforms existing methods.
Paper proposes Unification Networks to learn invariants from examples.
problem Learning to recognize common underlying principles across examples.
method End-to-end differentiable neural network approach with soft unification.
result Learning invariants improves performance on various datasets.
This paper tackles non-vacuous generalization bounds in ReLU networks by resolving rescaling invariances.
problem Non-vacuous generalization guarantees for ReLU networks with rescaling invariances.
method Proposes a lifted representation to resolve rescaling invariances and studies KL-based rescaling-invariant PAC-Bayes bounds.
result KL-based rescaling-invariant PAC-Bayes bounds provide tighter guarantees and resolve discrepancies in network complexity.
A universal collection of 4 invariants improves neural network accuracy for molecular dynamics.
problem Improving accuracy of neural networks in molecular dynamics.
method Developed a universal collection of 4 smooth scalar invariants on M(3) x M(3) and evaluated their effectiveness in a PONITA neural network architecture.
result Using a universal collection of invariants significantly improves neural network accuracy.
New approach for deep neural networks to learn invariance through adversarial forgetting.
problem Learning invariance for deep neural networks in the presence of nuisance and bias factors.
method Adversarial forgetting mechanism to induce amnesia to unwanted data factors.
result State-of-the-art performance in learning invariance across various datasets and tasks.
Invariant polynomials improve machine learning performance.
problem Improving machine learning algorithms using invariant polynomials.
method Developed and implemented Lorentz- and permutation-invariant polynomial generators in neural networks.
result Reduction in loss on training and validation data with Hironaka decompositions.
Group-invariant neural networks improve approximation accuracy for symmetric functions.
problem Improving approximation accuracy for symmetric functions using neural networks.
method Investigates the generalization error of group-invariant neural networks within the Barron framework.
result Group invariance introduces a factor δ that can significantly improve approximation accuracy when it is small.
Characterizes invariant and equivariant linear layers for graphs.
problem Maximal collection of invariant and equivariant linear layers for graphs.
method Characterization of all permutation invariant and equivariant linear layers for graphs.
result Dimension of linear layers for edge-value graph data is 2 and for k-tuples of nodes, it is the k-th and 2k-th Bell numbers.
Interactive platform for knot invariant computation and identification.
problem Efficient computation and identification of knot invariants.
method Unified web platform using Feynman ribbon diagrams and tensor networks.
result First platform combining construction, evaluation, computation, and identification.
Deep neural networks approximate functions in shift-invariant spaces with controlled error.
problem Approximating functions in shift-invariant spaces with neural networks.
method Using deep ReLU neural networks, estimating approximation error bounds based on network width and depth.
result Deep neural networks achieve optimal approximation rates for Sobolev spaces up to a logarithmic factor.
Permutation invariant network learns Wasserstein metrics.
problem Understanding the space of probability measures and comparing distributions.
method Permutation invariant network mapping samples to a low-dimensional space.
result Network can generalize to compute distances between unseen densities and learn moments.
Framework adds invariance to pretrained networks without fine-tuning.
problem Adding invariance to pretrained networks without altering original behavior.
method Post-training augmentation invariance framework with Markov-Wasserstein minimization and Wasserstein correlation maximization losses.
result Adapter networks improve classification accuracy on rotated and noisy images.
Theoretical comparison of three invariance approaches in deep linear networks.
problem Understanding invariance in deep linear networks.
method Data augmentation, regularization, and hard-wiring approaches.
result Regularization introduces additional critical points, but they remain saddles except for the global optimum.
New method relaxes spatial invariance in locally connected layers, improving accuracy.
problem Improving classification accuracy with locally connected layers.
method Designing a low-rank locally connected layer with varying spatially varying combining weights.
result Relaxing spatial invariance improves classification accuracy over convolution and locally connected layers.
New flatness measure for deep networks invariant to scaling.
problem Lack of invariant flatness measures for deep networks under parameter rescaling.
method Introduced a quotient manifold structure and Hessian-based invariant flatness measure.
result Confirms that Large-Batch SGD minima are sharper than Small-Batch SGD minima.
We add prior knowledge to deep networks to make them invariant to transformations.
problem Creating deep networks invariant to transformations like rotation.
method A novel layer based on invariant integration to enforce feature space invariances.
result State-of-the-art performance on the Rotated-MNIST dataset.
Deep networks are vulnerable to adversarial attacks due to excessive invariance.
problem Adversarial vulnerability of deep neural networks.
method Decomposed adversarial errors into sensitivity and invariance. Proposed an extended cross-entropy loss to encourage consideration of all task-dependent features.
result Deep networks are vulnerable to adversarial attacks due to excessive invariance, not just sensitivity.
Three training regimes found for scale-invariant neural networks on the sphere.
problem Training scale-invariant neural networks on the sphere with varying effective learning rate.
method Investigated three regimes of training: convergence, chaotic equilibrium, and divergence.
result Discovered three distinct training regimes with unique characteristics.
This research quantifies neural networks using magnitude, a topological invariant.
problem Understanding the generalization capabilities of neural networks.
method Using a novel topological invariant called magnitude to study neural network representations.
result Magnitude dimension is theoretically connected to generalisation error and can predict it.
Unified framework for invariance to nuisance and bias factors in neural networks.
problem Inducing independence to nuisance and bias factors in neural networks without labeled data.
method Unified invariance framework using competitive training between prediction and reconstruction tasks, coupled with disentanglement and adversarial learning.
result Outperforms previous works at inducing invariance to nuisance factors and achieves state-of-the-art performance at learning independence to biasing factors.
The paper extends hypothesis testing to non-diagonalizable matrices, improving network statistics inference.
problem Testing on non-diagonalizable matrices for network statistics.
method Generalizes Wald and t-tests to non-symmetric matrices, controlling convergence rates.
result Improved inference on network statistics from directed networks.
Recently, researchers have started applying convolutional neural networks (CNNs) with one-dimensional convolutions to clinical tasks involving time-series data. This is due, in part, to their computational efficiency, relative to recurrent neural networks and their ability to efficiently exploit certain temporal invari…
DeepHoyer introduces differentiable, scale-invariant sparsity measures for neural networks.
problem Efficiently sparsifying neural networks with scale-invariant sparsity measures.
method Developed DeepHoyer, a set of differentiable, scale-invariant sparsity-inducing regularizers based on the Hoyer measure.
result DeepHoyer produces sparser neural networks than previous methods, maintaining similar accuracy.
New algorithm learns invariant representations for robust neural networks.
problem Learning robust neural network representations that are invariant to certain factors.
method Causal perspective and distribution matching approach.
result Empirically, the algorithm achieves state-of-the-art performance on domain generalization.
Training deep neural networks is known to require a large number of training samples. However, in many applications only few training samples are available. In this work, we tackle the issue of training neural networks for classification task when few training samples are available. We attempt to solve this issue by pr…
New approach connects 3D Chern-Simons theory to spectral networks.
problem Understanding Chern-Simons invariants in 3D manifolds.
method Constructing equivalences between bundles and spectral networks.
result New formulas for Chern-Simons invariants of 3D manifolds.
Deep neural networks predict knot invariants across dimensions with high accuracy.
problem Predicting knot invariants in different dimensions using machine learning.
method Two-layer feed-forward neural networks trained on various knot invariants.
result Neural networks achieve high accuracy in predicting knot invariants like s and g. This paper classifies G-invariant shallow neural networks.
problem Designing optimal G-invariant neural architectures for G-invariant target functions. method Proving theorems about the classification and morphisms of G-invariant single-hidden-layer neural networks. result Classification of G-invariant shallow neural networks and characterization of morphisms. Machine learning classifies braids and discovers new invariants.
problem Classifying and discovering invariants of braids and flat braids.
method Supervised learning with neural networks to classify braids as trivial or non-trivial.
result Found new convenient invariants of braids, including a complete invariant of flat braids.
This paper proves new universality theorems for invariant and equivariant GNNs.
problem Designing GNNs that are invariant or equivariant under node permutations.
method Introduced a new class of invariant and equivariant GNNs with a single hidden layer.
result Universal invariant and equivariant GNNs can be constructed with a single set of parameters.
CLN2INV learns precise loop invariants from program traces.
problem Automated verification of real-world programs with complex loops.
method Continuous Logic Network (CLN) for learning precise loop invariants from program execution traces.
result CLN2INV significantly outperforms existing approaches on the Code2Inv dataset.
Paper proposes CNN with SIFT for rotation invariant feature extraction.
problem Max-pooling layer discards rotational information, leading to rotation invariance issues.
method Uses SIFT descriptor to capture orientation and spatial relationships.
result Improves feature extraction on MNIST and fashionMNIST datasets.
Automates machine learning of correlations between knot invariants.
problem Discovering and validating new relationships between knot invariants.
method Trained a neural network on 200,000 sets of knot invariants to predict an output invariant.
result Found novel correlations not explained by known results in knot theory.
Investigates spectral properties of neural networks, showing invariance under certain conditions.
problem Understanding the spectral evolution and invariance in linear-width neural networks.
method Empirical and theoretical analysis of spectra of weight matrices in high-dimensional settings.
result Spectra of weight matrices are invariant under certain training conditions, with implications for feature learning.
New neural architectures invariant to sign flips and basis symmetries for graph representation learning.
problem Learning invariant graph representations from eigenvectors.
method SignNet and BasisNet neural architectures that are invariant to sign flips and basis symmetries.
result Proven to be universal, approximating any continuous function of eigenvectors with desired invariances.
L-CNNs preserve gauge symmetry in lattice simulations.
problem Breaking gauge symmetry in neural network models.
method Lattice gauge equivariant convolutional neural networks (L-CNNs).
result L-CNNs represent gauge invariant functions on the lattice.
New framework improves neural network robustness without sacrificing accuracy.
problem Adversarial inputs compromise neural network robustness.
method Extract and model invariances of objects to enhance classification robustness.
result Invariances improve both robustness and accuracy in classification tasks.