Symmetry in loss functions constrains model parameters, leading to specific learning outcomes.
problem Understanding and leveraging symmetries in neural networks to improve learning outcomes.
method Analyzing the impact of loss function symmetries on model parameters and learning behavior.
result Mirror-reflection symmetries in loss functions lead to constraints on model parameters, influencing learning outcomes.
This paper aims to incorporate passive symmetries in machine learning for better generalization.
problem Machine learning's reliance on arbitrary choices leads to passive symmetries that can limit generalization.
method Translation among physics, mathematics, and machine learning to understand and implement passive symmetries.
result Respecting passive symmetries can improve machine learning's ability to generalize.
Latent MoS learns multiple symmetries for efficient dynamic learning.
problem Efficiently learning dynamics from limited system measurements.
method Latent Mixture of Symmetries (Latent MoS) with hierarchical architecture.
result Latent MoS outperforms baselines in interpolation and extrapolation tasks.
Unified framework for enforcing, discovering, and promoting symmetry in machine learning.
problem Symmetry in machine learning models and data.
method Unified mathematical framework using Lie derivatives, convex regularization, and nuclear norm relaxation.
result Unified approach to symmetry in machine learning tasks.
Machine learning explores symmetries in field theory and algebra.
problem Understanding symmetries in field theory and algebra.
method Using neural networks to analyze conformal field theory and Lie algebra representation theory.
result Recent advances in machine learning have uncovered new symmetries.
Method improves deep learning models for datasets with mixed approximate symmetries.
problem Improving deep learning models for datasets with mixed approximate symmetries.
method Regularizer-based approach to build models for datasets with mixed approximate symmetries.
result Our method achieves better accuracy than prior approaches while discovering the approximate symmetry levels correctly.
New symmetries improve meta-reinforcement learning's generalization.
problem Black-box meta reinforcement learning struggles with generalization.
method Introduce symmetries to a black-box meta RL system.
result Incorporating symmetries improves generalization to new environments.
The paper tackles learning symmetries in data without expert knowledge.
problem Learning symmetries in data from raw data without prior knowledge.
method Develops methods to select eigenvectors for orthogonal symmetries and compares their effectiveness.
result The problem of learning symmetries is as hard as the graph automorphism problem in the worst case, but can be simplified with certain restrictions.
Automatically learns flexible symmetry constraints in neural networks using gradients.
problem Fixed hard constraints on neural network functions that cannot be adapted.
method Improves parameterisations of soft equivariance and optimizes marginal likelihood using differentiable Laplace approximations.
result Achieves equivalent or improved performance on image classification tasks compared to baselines with hard-coded symmetry.
Symmetry in inverse problems leads to multiple solutions, but breaking symmetry helps deep learning.
problem Symmetry in physical systems causes multiple solutions in inverse problems, hindering deep learning.
method Careful symmetry breaking on training data helps solve inverse problems and improve deep learning performance.
result Symmetry breaking on training data significantly improves deep learning performance in inverse problems.
In this paper we explore methods to exploit symmetries for ensuring sample efficiency in reinforcement learning (RL), this problem deserves ever increasing attention with the recent advances in the use of deep networks for complex RL tasks which require large amount of training data. We introduce a novel method to dete…
Noether's framework reveals symmetry-breaking in neural networks.
problem Understanding the role of symmetry breaking in neural networks.
method Developed a theoretical framework using Lagrangian mechanics.
result Identified 'kinetic symmetry breaking' and its effect on learning dynamics.
Symmetry, a central concept in understanding the laws of nature, has been used for centuries in physics, mathematics, and chemistry, to help make mathematical models tractable. Yet, despite its power, symmetry has not been used extensively in machine learning, until rather recently. In this article we show a general wa…
New framework discovers non-affine continuous symmetries in neural networks.
problem Lack of efficient methods for detecting non-affine continuous symmetries in neural networks.
method Computational framework for discovering infinitesimal generators of multi-parameter group actions.
result Framework can discover non-affine continuous symmetries in neural networks.
GE-autoencoder identifies spontaneous symmetry breaking in systems.
problem Locating phase boundaries and identifying spontaneously broken symmetries in systems.
method Group-equivariant autoencoder using group theory to constrain parameters and learn invariant order parameters.
result GE-autoencoder accurately determines spontaneous symmetry breaking and estimates critical temperatures more efficiently.
New method detects symmetries beyond affine transformations.
problem Current methods limit symmetry detection to affine transformations.
method Framework for discovering continuous symmetry beyond affine transformations.
result Method is competitive for large sample sizes and superior for small sample sizes.
Unconstrained models learn physical symmetries effectively with simple data augmentation.
problem Ensuring physical symmetries in machine learning models.
method Rigorous metrics to measure symmetry content, data augmentation strategy, architectural analysis.
result Unconstrained models can learn approximate equivariant behavior with simple data augmentation.
Finding a generally accepted formal definition of a disentangled representation in the context of an agent behaving in an environment is an important challenge towards the construction of data-efficient autonomous agents. Higgins et al. recently proposed Symmetry-Based Disentangled Representation Learning, a definition…
Humans take advantage of real world symmetries for various tasks, yet capturing their superb symmetry perception mechanism with a computational model remains elusive. Motivated by a new study demonstrating the extremely high inter-person accuracy of human perceived symmetries in the wild, we have constructed the first …
This work extends PAC-Bayesian learning guarantees to non-compact symmetries and non-invariant data.
problem Lack of theoretical guarantees explaining the benefits of symmetries in machine learning models.
method Adapting and tightening PAC-Bayes bounds for non-compact symmetries and non-invariant data distributions.
result Theoretical evidence that symmetric models are preferable for symmetric data, beyond compact groups and invariant distributions.
Symmetry unifies AI learning dynamics, complexity, and representation.
problem Fragmented theories of AI learning mechanisms.
method Synthesis of parameter symmetry in AI models.
result Parameter symmetry breaking and restoration unify AI learning hierarchies.
New method breaks symmetry in neural networks, improving sample efficiency.
problem Symmetry in neural networks limits their ability to learn unique features.
method Introduces 'relaxed equivariance' to overcome symmetry limitations.
result Equivariant multilayer perceptrons (E-MLPs) can now break symmetry at the sample level.
Analyzes symmetries in neural networks to predict learning dynamics.
problem Understanding the dynamics of neural network parameters during training.
method Unified theoretical framework based on symmetries and conservation laws.
result Symmetries impose geometric constraints on gradients and Hessians, leading to conservation laws.
New method discovers symmetries in differential equations from data.
problem Directly identifying Lie symmetries from scattered data without explicit equations.
method Numerical scheme using manifold learning and linear system construction.
result Accuracy and robustness demonstrated in various differential equations.
Gradient descent struggles to learn equivariant neural networks, even with symmetries.
problem Learning equivariant neural networks via gradient descent is hard.
method Lower bounds for various equivariant neural network classes.
result Gradient descent struggles to learn equivariant neural networks, even with symmetries.
A machine learning model with approximate rotational symmetry is tested and found stable.
problem The effects of broken symmetries in machine learning models.
method Testing a model with approximate rotational symmetry in various physical scenarios.
result The model remains stable even with noticeable symmetry artifacts, suggesting potential benefits.
This work connects symmetries and conserved quantities in machine learning.
problem Improving machine learning models by learning conserved quantities.
method Using Noether's theorem, learn symmetries and conserved quantities directly from data.
result Correctly identifies conserved quantities and improves model performance.
This work relaxes GNN symmetries to approximate automorphisms, improving model performance.
problem Improving graph neural network performance on asymmetric graphs.
method Formalizing approximate symmetries via graph coarsening, introducing a bias-variance formula.
result Best generalization performance achieved by choosing a larger symmetry group than automorphisms but smaller than permutations.
Noether's theorem clarifies how symmetries in neural networks influence learning.
problem Understanding how symmetries in neural networks affect learning.
method Systematic study of symmetry interactions with learning algorithms using Noether's theorem.
result Symmetries impose restrictions on the optimization path, leading to conserved quantities.
Method discovers symmetries in data with neural networks.
problem Discovering symmetries in high-dimensional data.
method Fully connected neural networks trained with a loss function for symmetry.
result Generators for symmetries in latent space.
Symmetry in neural networks affects generalization, as shown by CLT and RG transformations.
problem Improving generalization in neural networks by incorporating physical symmetries.
method Evaluation of symmetry constraints and expressivity in MLPs and GNNs using the CLT as a test case.
result Overly complex or overconstrained models generalize poorly, revealing a competition between symmetry constraints and expressivity.
GQML uses symmetries from representation theory to improve quantum machine learning.
problem Creating quantum models with symmetries to improve performance.
method Introduction to representation theory for quantum learning, focusing on group actions and symmetries.
result Effective implementation of GQML requires knowledge of group representation theory.
Metric evaluates symmetry-breaking in datasets, revealing severe biases.
problem Symmetry-breaking in datasets can hinder the performance of symmetry-aware methods.
method Developed a metric to quantify symmetry-breaking using a two-sample classifier test.
result Symmetry-breaking can prevent optimal performance of invariant methods, even when labels are invariant.
Symmetry principles help in creating better AI representations.
problem Creating efficient and generalizable AI representations.
method Using symmetry transformations to guide representation learning.
result Symmetry principles improve data efficiency and generalizability in AI.
Develops a framework for designing quantum neural networks that respect symmetries.
problem Trainability and generalization issues in quantum neural networks.
method Equivariant quantum neural networks (EQNN) for any symmetry group.
result Efficient construction of equivariant layers for EQNNs, including QCNNs.
Permutation of any two hidden units yields invariant properties in typical deep generative neural networks. This permutation symmetry plays an important role in understanding the computation performance of a broad class of neural networks with two or more hidden units. However, a theoretical study of the permutation sy…
Breaking symmetry in training data is key for generalization in feature learning kernels.
problem Grokking in algebraic tasks, where models perform well on training but fail on unseen data.
method Used Recursive Feature Machine (RFM) with AGOP to learn task-relevant features, breaking symmetry in training data.
result Generalization occurs only when symmetry in the training set is broken, and RFM generalizes by recovering underlying invariance group action.
The paper explores how symmetries and noise in SGD influence parameter dynamics.
problem Understanding the dynamics of parameter updates in SGD with symmetries.
method Proved the existence of noise equilibria and showed their role in balancing gradient noise.
result Gradient noise creates a systematic motion of parameters to a unique fixed point, called noise equilibria.
Optimal classification requires choosing the right group symmetries, contrary to intuition.
problem Improving binary classification performance by selecting appropriate group symmetries.
method Developed a theoretical framework for designing group equivariant neural networks.
result Optimal classification performance is achieved by selecting the appropriate subgroups of symmetries, not the largest equivariant groups.
Proposes learning invariances in neural networks using a weight-space approach.
problem Learning invariances from data in neural networks remains an open problem.
method Minimizes a lower bound on the marginal likelihood in weight space.
result Results in higher performing models with naturally learned invariances.
New method approximates curvature from symmetries in deep networks.
problem Hard to approximate curvature in large deep networks.
method Analytically averaging over group actions that leave the loss invariant to construct structured Hessian approximations.
result Structured Hessian approximations from single gradients can be estimated, stored, and inverted.
Efficiently samples and learns densities with symmetries using equivariant methods.
problem Efficiently sampling and learning densities with symmetries.
method Equivariant Stein Variational Gradient Descent (SVGD) and equivariant energy based models.
result Improves and scales up training of energy based models.
Method learns symmetries in curves without augmentation.
problem Symmetries in datasets like rotations and scalings.
method Geometric learning using principal fiber bundles.
result 2-parameter family of canonical curve parameterizations.
New optimizer designs respect symmetry, improving deep learning models.
problem Optimizers lack respect for symmetry in neural networks.
method Introduce symmetry-compatible principle for optimizer design.
result Symmetry-compatible optimizers improve model performance.
TRS-ODENs learn dynamics with time-reversal symmetry for more efficient learning.
problem Learning dynamics with time-reversal symmetry for more efficient learning.
method Proposed a loss function and a new framework (TRS-ODENs) to learn dynamics efficiently.
result TRS-ODENs can learn dynamics from noisy and complex trajectories efficiently.
Method learns equivariances from data without custom architecture design.
problem Learning equivariances for tasks without manually designed architectures.
method Reparameterization to learn equivariant parameter sharing.
result Can learn equivariances for any finite group of transformations.
Remove symmetries to improve model optimization and performance.
problem Symmetries in loss functions trap models in low-capacity states, hindering training and optimization.
method Proposes syre, a simple algorithm to remove symmetries in neural networks.
result Removing symmetries correlates well with improved optimization and performance.
Equivariant flows learn symmetrical distributions on manifolds.
problem Learning symmetrical distributions on arbitrary manifolds.
method Equivariant manifold flows.
result Learned gauge invariant densities over SU(n) in quantum field theory.