Permutation of any two hidden units yields invariant properties in typical deep generative neural networks. This permutation symmetry plays an important role in understanding the computation performance of a broad class of neural networks with two or more hidden units. However, a theoretical study of the permutation sy…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “Permutation Symmetry”
Variational inference struggles with weight symmetries in neural networks, leading to biased posteriors.
problem Weight space symmetries in neural networks cause multimodal posteriors, challenging variational inference.
method Developed a symmetrization mechanism to create permutation invariant variational posteriors.
result Symmetrized variational posteriors have a better fit to the true posterior and improved predictive performance.
Dual Invertible Polynomials with Permutation Symmetries and the Orbifold Euler Characteristicmath.AG
P. Berglund, T. Hübsch, and M. Henningson proposed a method to construct mirror symmetric Calabi-Yau manifolds. They considered a pair consisting of an invertible polynomial and of a finite (abelian) group of its diagonal symmetries together with a dual pair. A. Takahashi suggested a method to generalize this construct…
New model preserves symmetry in multivariate time series, improving performance.
problem Implicit ordering in MTS models violates inherent exchangeability.
method Permutation-equivariant 2D state space model with canonical architecture.
result Eliminates sequential dependency chains and simplifies stability analysis.
New neural network architecture for auction design exploiting permutation symmetry.
problem Designing incentive-compatible auctions that maximize expected revenue.
method Constructed a permutation-equivariant neural network architecture.
result Permutation-equivariant architectures can perfectly recover optimal mechanisms.
Classifies symmetries of non-flat 3-webs around a point.
problem Understanding symmetries of non-flat 3-webs.
method Classification and construction methods for symmetries.
result Classification of symmetries for non-flat 3-webs.
The introduction of convolutional layers greatly advanced the performance of neural networks on image tasks due to innately capturing a way of encoding and learning translation-invariant operations, matching one of the underlying symmetries of the image domain. In comparison, there are a number of problems in which the…
This work relaxes GNN symmetries to approximate automorphisms, improving model performance.
problem Improving graph neural network performance on asymmetric graphs.
method Formalizing approximate symmetries via graph coarsening, introducing a bias-variance formula.
result Best generalization performance achieved by choosing a larger symmetry group than automorphisms but smaller than permutations.
This work refines claims about neural network connectivity, showing that simultaneous linear connectivity is possible under certain conditions.
problem Neural networks' loss landscapes are non-convex due to permutation symmetries, leading to high loss barriers between permuted networks.
method The authors introduce and analyze three claims of increasing strength regarding the connectivity of neural networks, focusing on permutations that align networks.
result The authors provide evidence that strong linear connectivity may be possible under certain conditions, specifically when interpolating among three networks of increasing width.
Theoretical guarantees for permutation-equivariant QNNs avoid barren plateaus.
problem Excessive local minima and barren plateaus in QNNs training landscapes.
method Designing -equivariant QNNs to encode permutation symmetry.
result Equivariant QNNs do not suffer from barren plateaus, quickly reach overparametrization, and generalize well.
Unified framework connects different neural network models.
problem Understanding the geometry of neural network loss landscapes.
method Unified framework capturing four symmetry classes.
result First discovery of low- and zero-barrier linear interpolation paths.
New graph foundation models respect symmetries for broader applicability.
problem Tailored graph machine learning architectures limit broader applicability.
method Investigates symmetries for label and feature permutations, proving network universal approximator.
result Universal approximator on multisets respecting node and feature permutations.
Proposes neuron alignment to optimize mode connectivity in neural networks.
problem Understanding and optimizing mode connectivity in deep neural networks.
method Introduces neuron alignment to approximate optimal weight permutations and improve mode connectivity.
result Neuron alignment significantly alleviates robust loss barriers and improves model robustness and accuracy.
Symmetry in loss functions constrains model parameters, leading to specific learning outcomes.
problem Understanding and leveraging symmetries in neural networks to improve learning outcomes.
method Analyzing the impact of loss function symmetries on model parameters and learning behavior.
result Mirror-reflection symmetries in loss functions lead to constraints on model parameters, influencing learning outcomes.
The permutation symmetry of neurons in each layer of a deep neural network gives rise not only to multiple equivalent global minima of the loss function, but also to first-order saddle points located on the path between the global minima. In a network of hidden layers with neurons in layers $k = 1, \ldots, …
Transformers tend to learn more symmetric functions in sequence data.
problem Understanding inductive bias in Transformers with infinitely over-parameterized models.
method Analyzing Transformers in the Gaussian process limit, using representation theory of the symmetric group.
result Transformers are biased towards more permutation symmetric functions, and this can be quantitatively predicted.
New neural networks respect symmetries in symmetric tensors, improving efficiency and generalization.
problem Learning from symmetric tensors efficiently and respecting their inherent symmetries.
method Developed two characterizations of linear permutation equivariant functions between symmetric power spaces of R^n.
result These functions are highly data efficient compared to standard MLPs and generalize well to different sizes of symmetric tensors.
This work tackles Bayesian neural networks by addressing loss landscape symmetries.
problem Understanding and optimizing the loss landscape of Bayesian neural networks.
method The approach involves extending marginalized loss barrier formalism to BNNs, proposing a matching algorithm to search for linearly connected solutions using permutation matrices and combinatorial optimization.
result Nearly zero marginalized loss barriers for linearly connected solutions were found.
Paper presents a method to summarize HMC samples for neural networks, providing meaningful uncertainty estimates.
problem Lack of interpretable summary statistics for HMC samples in neural networks due to permutation symmetry.
method Introducing a transpositions metric to quantify permutations and using rebasin method to summarize HMC samples.
result Compact representation of HMC samples provides meaningful uncertainty estimates for each weight in a neural network.
New neural networks learn graph symmetries.
problem Learning from graph data without considering vertex relations.
method Constructs equivariant neural networks to Aut(G) group.
result Characterizes learnable, linear, Aut(G)-equivariant functions.
Toward Super-polynomial Quantum Speedup of Equivariant Quantum Algorithms with SU() Symmetryquant-ph
Enhances quantum computing for symmetrical systems, proving a new class of problems.
problem Proving the efficiency of a new quantum computing model for symmetrical systems.
method Introducing equivariant convolutional quantum algorithms tailored for SU(d) symmetries.
result Demonstrates a problem that can be solved efficiently on a new quantum model, suggesting it's not classically simulatable.
We give a constructive proof that the Regge symmetry is a scissors congruence in hyperbolic space. The main tool is Leibon's construction for computing the volume of a general hyperbolic tetrahedron. The proof consists of identifying the key elements in Leibon's construction and permuting them.
EquivCNP learns group symmetries for conditional data.
problem Learning conditional models with data symmetries.
method Group equivariant decomposition and Lie group convolutional layers.
result EquivCNP achieves comparable performance and zero-shot generalization.
EDGI improves sample efficiency and generalization in tasks with spatial and temporal symmetries.
problem Sample inefficiency and poor generalization in tasks with geometric symmetries.
method Equivariant Diffuser framework, SE(3)xZxSn-equivariant diffusion model.
result EDGI is more sample efficient and generalizes better than non-equivariant models.
Geometric Causal Modelsstat.ML
Develops geometric causal models for causal inference from dependent data.
problem Causal inference from structured, dependent data (e.g., spatial, network, molecular).
method Geometric causal models (GCMs) exploiting symmetries of data generating process, combining group theory, ergodic theory, and Bayesian inference.
result Establishes identification and estimation of causal effects from dependent data.
Graphs of neural networks are represented to preserve symmetry, improving performance across various tasks.
problem Lack of equivariance in neural network representations of other neural networks.
method Represent neural networks as computational graphs and use graph neural networks to preserve permutation symmetry.
result Single model encodes diverse neural architectures, outperforming state-of-the-art methods.
New CMC surfaces with dihedral symmetry constructed from Darboux transforms.
problem Constructing closed CMC surfaces with specific symmetries.
method Using Darboux transforms on multiple covers to create new CMC surfaces.
result Explicit parametrisations of new closed CMC surfaces with dihedral symmetry.
Novel neural GP kernels learn stable, flexible covariance structures.
problem Scalable and flexible covariance kernels for Gaussian processes.
method Directly learn kriging coefficients and conditional standard deviations using deep neural architectures exploiting permutation-equivariant structure.
result Improved training stability and data efficiency with expressive, non-stationary kernels.
Probabilistic models often have parameters that can be translated, scaled, permuted, or otherwise transformed without changing the model. These symmetries can lead to strong correlation and multimodality in the posterior distribution over the model's parameters, which can pose challenges both for performing inference a…
New lower bounds on embedding dimensions for neural network architectures.
problem Ensuring neural networks can handle symmetries like permutations in high dimensions.
method Novel technique to prove lower bounds on embedding dimensions.
result Proves new lower bounds on embedding dimensions for Deep Sets and Janossy pooling.
The paper uncovers symmetries in large language models through layer-peeled optimization.
problem Understanding geometric structure in large language model weights and context embeddings.
method Constrained layer-peeled optimization program to analyze symmetries in next-token distributions.
result Symmetries in target next-token distributions are transferred to optimal model weights and context embeddings.
Study of intrinsic symmetry groups of links, finding counterexamples.
problem Whether every subgroup of the symmetric group is an intrinsic symmetry group of some link.
method Defined intrinsic symmetry groups and provided counterexamples.
result For n > 5, there does not exist an n-component link L for which S(L) is the alternating group.
New findings on hidden symmetries in ReLU networks.
problem Understanding the redundancy and symmetries in ReLU network parameter space.
method Analyzing parameter settings and function classes for various network architectures.
result For certain network architectures, there are no hidden symmetries.
We propose to study equivariance in deep neural networks through parameter symmetries. In particular, given a group that acts discretely on the input and output of a standard neural network layer , we show that is equivariant with respect to -action iff $\m…
Learning molecular energies using localized graph kernelsphysics.comp-ph
Recent machine learning methods make it possible to model potential energy of atomic configurations with chemical-level accuracy (as calculated from ab-initio calculations) and at speeds suitable for molecular dynam- ics simulation. Best performance is achieved when the known physical constraints are encoded in the mac…
Signed-permutation coordinate transport improves model alignment across checkpoints.
problem Improper alignment of coordinate-indexed objects across model checkpoints.
method Introduces sign-marginalized Hungarian matching and coordinate-preserving transport.
result Recovering signed-permutation gauge improves coordinate alignment and model performance.
This paper aims to incorporate passive symmetries in machine learning for better generalization.
problem Machine learning's reliance on arbitrary choices leads to passive symmetries that can limit generalization.
method Translation among physics, mathematics, and machine learning to understand and implement passive symmetries.
result Respecting passive symmetries can improve machine learning's ability to generalize.
Unique genus 2 minimal surface in S^3 with specific symmetry group.
problem Finding minimal surfaces in S^3 with specific symmetries.
method Analyzing isometry groups of known minimal surfaces and proving uniqueness.
result ξ_{2,1} is the unique minimal surface of genus 2 with bidihedral symmetry.
New mesh network preserves symmetries in deep learning.
problem No existing mesh processing architecture is equivariant to all symmetries.
method Equivariant attention-based mesh network using relative tangential features.
result The network achieves improved performance and is equivariant to various transformations.
New symmetries improve meta-reinforcement learning's generalization.
problem Black-box meta reinforcement learning struggles with generalization.
method Introduce symmetries to a black-box meta RL system.
result Incorporating symmetries improves generalization to new environments.
SE-RRMs solve structured problems like Sudoku and ARC-AGI by enforcing symbol equivariance.
problem Structured reasoning problems like Sudoku and ARC-AGI.
method Symbol-equivariant recurrent reasoning models enforcing permutation equivariance.
result SE-RRMs outperform prior RRMs on 9x9 Sudoku and generalize to larger and smaller instances.
Metric evaluates symmetry-breaking in datasets, revealing severe biases.
problem Symmetry-breaking in datasets can hinder the performance of symmetry-aware methods.
method Developed a metric to quantify symmetry-breaking using a two-sample classifier test.
result Symmetry-breaking can prevent optimal performance of invariant methods, even when labels are invariant.
Gradient descent struggles to learn equivariant neural networks, even with symmetries.
problem Learning equivariant neural networks via gradient descent is hard.
method Lower bounds for various equivariant neural network classes.
result Gradient descent struggles to learn equivariant neural networks, even with symmetries.
New translation equivariant neural processes improve spatio-temporal data modeling.
problem Improving posterior prediction maps for spatio-temporal data.
method Introduced translation equivariant transformers within neural processes.
result TE-TNPs outperform non-equivariant TNPs and other baselines.
Improved Bayesian neural network inference by selectively removing redundant modes.
problem Redundant modes in Bayesian neural network posteriors complicate approximate inference.
method Structured partial stochasticity and deterministic subset selection of weights.
result Improved performance of approximate inference schemes with simplified posterior distribution.
We consider the "intrinsic" symmetry group of a two-component link , defined to be the image of the natural homomorphism from the standard symmetry group $\MCG(S^3,L)$ to the product $\MCG(S^3) \cross \MCG(L)$. This group, first defined by Whitten in 1969, records directly whether is isotopic to a link $L…
New method uses scalars to approximate physics functions.
problem Designing neural networks that respect physical symmetries.
method Parameterizing polynomial functions equivariant to various symmetries using scalars.
result Universal approximation of polynomial functions under various symmetries using scalars.
Classifies Legendrian torus and cable links, revealing symmetries and invariants.
problem Classifying Legendrian torus and cable links.
method Classification through Legendrian and transversal isotopies, symmetries analysis.
result First family of non-destabilizable links without maximal Thurston-Bennequin invariant.