Empirical study shows removing neural parameter symmetries impacts model performance.
problem Understanding the impact of neural parameter symmetries on model performance.
method Developed two methods to reduce parameter space symmetries in neural networks.
result Removing parameter symmetries can lead to faster and more effective Bayesian neural network training.
New findings on hidden symmetries in ReLU networks.
problem Understanding the redundancy and symmetries in ReLU network parameter space.
method Analyzing parameter settings and function classes for various network architectures.
result For certain network architectures, there are no hidden symmetries.
Symmetry of neural network densities can be determined from correlation functions.
problem Determining symmetries of neural network densities without knowing the density itself.
method Symmetry-via-duality approach using invariance properties of correlation functions.
result Symmetries of neural network densities can be determined via dual computations of correlation functions.
We propose to impose symmetry in neural network parameters to improve parameter usage and make use of dedicated convolution and matrix multiplication routines. Due to significant reduction in the number of parameters as a result of the symmetry constraints, one would expect a dramatic drop in accuracy. Surprisingly, we…
GE-autoencoder identifies spontaneous symmetry breaking in systems.
problem Locating phase boundaries and identifying spontaneously broken symmetries in systems.
method Group-equivariant autoencoder using group theory to constrain parameters and learn invariant order parameters.
result GE-autoencoder accurately determines spontaneous symmetry breaking and estimates critical temperatures more efficiently.
New symmetries discovered in Kepler's orbit family.
problem Symmetry properties of Kepler orbits and related subfamilies.
method Projective geometry and Lie's infinitesimal point symmetries.
result Kepler orbits form a flat family with a 7-dimensional local symmetry group.
Probabilistic models often have parameters that can be translated, scaled, permuted, or otherwise transformed without changing the model. These symmetries can lead to strong correlation and multimodality in the posterior distribution over the model's parameters, which can pose challenges both for performing inference a…
The paper explores how symmetries and noise in SGD influence parameter dynamics.
problem Understanding the dynamics of parameter updates in SGD with symmetries.
method Proved the existence of noise equilibria and showed their role in balancing gradient noise.
result Gradient noise creates a systematic motion of parameters to a unique fixed point, called noise equilibria.
In this paper the generic bifurcations of the Minkowski symmetry set for 1-parameter families of plane curves are classified and the necessary and sufficient geometric criteria for each type are given. The Minkowski symmetry set is an analogue of the standard Euclidean symmetry set, and is defined to be the locus of ce…
Using the procedure initiated in \cite{Ma2013}, we deform Lax-type equations though a scaling of the time parameter. This gives an equivalent (deformed) equation which is integrable in terms of power series of the scaling parameter. We then describe a regular Frölicher Lie group of symmetries of this deformed equation
Symmetry unifies AI learning dynamics, complexity, and representation.
problem Fragmented theories of AI learning mechanisms.
method Synthesis of parameter symmetry in AI models.
result Parameter symmetry breaking and restoration unify AI learning hierarchies.
New framework discovers non-affine continuous symmetries in neural networks.
problem Lack of efficient methods for detecting non-affine continuous symmetries in neural networks.
method Computational framework for discovering infinitesimal generators of multi-parameter group actions.
result Framework can discover non-affine continuous symmetries in neural networks.
We introduce the concept of spontaneous symmetry breaking to arbitrage modeling. In the model, the arbitrage strategy is considered as being in the symmetry breaking phase and the phase transition between arbitrage mode and no-arbitrage mode is triggered by a control parameter. We estimate the control parameter for mom…
We propose to study equivariance in deep neural networks through parameter symmetries. In particular, given a group G that acts discretely on the input and output of a standard neural network layer φW:ℜM→ℜN, we show that φW is equivariant with respect to G-action iff $\m…
Symmetry in loss functions constrains model parameters, leading to specific learning outcomes.
problem Understanding and leveraging symmetries in neural networks to improve learning outcomes.
method Analyzing the impact of loss function symmetries on model parameters and learning behavior.
result Mirror-reflection symmetries in loss functions lead to constraints on model parameters, influencing learning outcomes.
Transformers reduce redundancy by focusing on invariant relational quantities.
problem Substantial internal redundancy in Transformer models due to coordinate-dependent representations and continuous symmetries.
method Reformulate representations, attention mechanisms, and optimization dynamics in terms of invariant relational quantities, eliminating redundant degrees of freedom by construction.
result Architectures that operate directly on relational structures, providing a principled geometric framework for reducing parameter redundancy and analyzing optimization.
We fully develop the concept of causal symmetry introduced in Class. Quant. Grav. 20 (2003) L139. A causal symmetry is a transformation of a Lorentzian manifold (V,g) which maps every future-directed vector onto a future-directed vector. We prove that the set of all causal symmetries is not a group under the usual comp…
Efficient MCMC sampling in Bayesian neural networks by exploiting symmetries.
problem Challenges in Bayesian inference due to high-dimensional, multi-modal posterior density landscapes.
method Exploiting symmetries in the posterior landscape to restrict the parameter space and derive an upper bound on Monte Carlo chains.
result Efficient sampling is possible, offering a promising path for accurate uncertainty quantification in deep learning.
Method learns equivariances from data without custom architecture design.
problem Learning equivariances for tasks without manually designed architectures.
method Reparameterization to learn equivariant parameter sharing.
result Can learn equivariances for any finite group of transformations.
Analyzes symmetries in neural networks to predict learning dynamics.
problem Understanding the dynamics of neural network parameters during training.
method Unified theoretical framework based on symmetries and conservation laws.
result Symmetries impose geometric constraints on gradients and Hessians, leading to conservation laws.
GENNI visualises neural network symmetries for better identifiability.
problem Identifying functionally equivalent parameters in neural networks.
method Proposes GENNI, an algorithm to efficiently visualise equivalence classes.
result Allows better exploration of identifiability questions in neural networks.
New optimizer designs respect symmetry, improving deep learning models.
problem Optimizers lack respect for symmetry in neural networks.
method Introduce symmetry-compatible principle for optimizer design.
result Symmetry-compatible optimizers improve model performance.
Functional dimension varies in ReLU networks, with implications for symmetry and connectivity.
problem Understanding the functional dimension of ReLU neural networks.
method Careful definition and analysis of functional dimension, study of quotient space and fibers.
result Functional dimension is inhomogeneous and can be non-constant, with implications for symmetry and connectivity.
Local normal forms for symmetrical contact structures on 3-manifolds.
problem Understanding symmetrical contact structures on 3-manifolds.
method Determining local normal forms for pairs of transverse contact distributions with symmetries.
result Orientable Anosov flows can be globally represented by intersecting contact distributions with maximal symmetries.
New theory shows how membranes can break symmetry.
problem Understanding symmetry breaking in membranes with boundaries.
method Applied bifurcation theory and reduced membrane equation.
result Existence of symmetry breaking bifurcation in membrane solutions.
Gauge symmetries explain the emergence of Merton-Garman equation from Black-Scholes in finance.
problem Understanding the emergence of Merton-Garman equation from Black-Scholes in financial markets.
method Using Hamiltonian formulation and gauge symmetry to derive the Merton-Garman equation from Black-Scholes, analyzing the role of stochastic volatility.
result Gauge symmetry explains the appearance of stochastic volatility and its massivation via the Higgs mechanism.
We study sharp asymptotics of the first eigenvalue on Riemannian surfaces obtained from a fixed Riemannian surface by attaching a collapsing flat handle or cross cap to it. Through a careful choice of parameters this construction can be used to strictly increase the first eigenvalue normalized by area if the initial su…
We give a complete classification of intertwining operators (symmetry breaking operators) between spherical principal series representations of G=O(n+1,1) and G'=O(n,1). We construct three meromorphic families of the symmetry breaking operators, and find their distribution kernels and their residues at all poles explic…
Mathematical models with time dependent parameters are of great interest in financial Mathematics because they capture real life scenarios in the financial market. In this study, via the Lie group technique, we analyse evolution-type equations with time dependent parameters and give the general symmetry structure of th…
Symmetry in neural networks affects generalization, as shown by CLT and RG transformations.
problem Improving generalization in neural networks by incorporating physical symmetries.
method Evaluation of symmetry constraints and expressivity in MLPs and GNNs using the CLT as a test case.
result Overly complex or overconstrained models generalize poorly, revealing a competition between symmetry constraints and expressivity.
Investigates geometric mean reversion process using Lie symmetry method.
problem Describes dynamics of short-term interest rates.
method Lie symmetry method and optimal system of invariant solutions.
result Constructs an optimal system of invariant solutions.
New framework uses symmetry-based matrices for efficient, flexible NNs.
problem Designing neural networks with relaxed equivariance.
method Symmetry-based structured matrices, Group Matrices (GMs).
result GMs enable competitive performance with fewer parameters.
Study biharmonic functions on vector bundles with spherical symmetry.
problem Investigate biharmonic functions on vector bundles with spherically symmetric metrics.
method Analyze vertical lifts and radial functions of functions on vector bundle manifolds.
result Construct an infinite two-parameter family of proper biharmonic functions.
We define a new type of transformation for Lorentzian manifolds characterized by mapping every causal future-directed vector onto a causal future-directed vector. The set of all such transformations, which we call causal symmetries, has the structure of a submonoid. Some of their properties are investigated and we give…
In this paper, we introduce an analytical perturbative solution to the Merton Garman model. It is obtained by doing perturbation theory around the exact analytical solution of a model which possesses a two-dimensional Galilean symmetry. We compare our perturbative solution of the Merton Garman model to Monte Carlo simu…
Integrable symmetries of diffieties are studied, leading to local morphisms.
problem Understanding local integrable symmetries of diffieties.
method Integrable infinitesimal symmetries defined as a one-parameter pseudogroup of local diffiety morphisms. Reduction of computation to solving PDEs.
result Preliminary results and examples show integrable symmetries can be reduced to solving linear systems.
We find a 2-parameter family of deformations in R^4_1 of the classical Chen-Gackstatter surface explicitly, and show the existence of a larger 4-parameter family of deformations. Each of them still has genus one, a unique end, with total Gaussian curvature −∫K=8π. On the other hand, a uniqueness theorem is obtain…
In this paper, a complete analysis of symmetries and conservation laws for the charged squashed Kaluza--Klein black hole spacetime in a Riemannian space is discussed. First, a comprehensive group analysis of the underlying space-time metric using Lie point symmetries are presented and then it the n-dimensional optima…
Study reflection symmetry and APS boundary conditions on a warped cylinder.
problem Analyzing reflection symmetry and APS boundary conditions for twisted Dirac operators on a finite warped cylinder.
method Examined reflection symmetry and APS boundary conditions for twisted Dirac operators on a finite warped cylinder, considering both fixed and varying holonomy.
result Reflection symmetry lifts to a unitary symmetry under specific conditions, and the spectral flow admits an RO(O(2))-valued decomposition for fixed holonomy.
Giving a new form of the vortex mode equation by a proper change of parameter, our aim is to analyze the point and contact symmetries of the new equation. Fundamental invariants and a form of general solutions of point transformations along with some specific examples are also derived.
We provide some insights in the study of branching problems of reductive groups, and a method of investigations into symmetry breaking operators. First, we give geometric criteria for finiteness property of linearly independent continuous (respectively, differential) operators that intertwine two induced representation…
Symmetry proven for positive solutions of a weighted p-Laplace operator inequality.
problem Proving symmetry of positive solutions to a specific type of inequality.
method Analyzing positive critical points of Caffarelli-Kohn-Nirenberg inequalities with a weighted p-Laplace operator.
result Complete classification and symmetry result for positive solutions in a range of parameters.
Data symmetries in neural networks can generate conserved quantities.
problem Conservation laws in neural networks
method Using tensorizable networks
result Data augmentation can induce conserved quantities
New steady solitons found with SO(3) symmetry.
problem Finding steady solitons with specific symmetries.
method Proved existence of a family of solitons with SO(3) symmetry.
result Existence of a one-parameter family of solitons.
Graph Metanetworks process diverse neural architectures efficiently.
problem Processing diverse neural architectures efficiently.
method Builds metanetworks using graph neural networks to process graphs representing input neural networks.
result Proves GMNs are expressive and equivariant to parameter permutation symmetries.
Graphs of neural networks are represented to preserve symmetry, improving performance across various tasks.
problem Lack of equivariance in neural network representations of other neural networks.
method Represent neural networks as computational graphs and use graph neural networks to preserve permutation symmetry.
result Single model encodes diverse neural architectures, outperforming state-of-the-art methods.
Minimal model reveals power laws in financial markets.
problem Understanding universal behaviors in financial markets.
method Analytical solution of a minimal model based on symmetry constraints.
result Various power-law behaviors are interconnected, similar to critical exponents.
Overview of integrable systems with symmetries, focusing on toric and semitoric systems.
problem Classifying and understanding integrable systems with symmetries.
method Using decorated polygons and controlled bifurcations in one-parameter families of systems.
result Construction of explicit semitoric systems with prescribed invariants.