Prob-PIT improves speech separation by considering output-label permutations as random variables.
problem Overconfident output-label assignment in PIT leads to unreliable speech separation.
method Prob-PIT treats output-label permutations as a discrete latent random variable with a uniform prior distribution and maximizes the log-likelihood function.
result Prob-PIT significantly outperforms PIT in terms of Signal to Distortion Ratio and Signal to Interference Ratio.
Improves modeling of sets with permutation invariant densities.
problem Challenges in calculating trace limit practicality of current methods.
method Proposes an alternative approach to define permutation equivariant transformations with closed form trace.
result Improves both training and final performance.
A new method learns graph distributions invariant to node ordering.
problem Graphs are hard to model due to node ordering invariance issues.
method Score-based generative modeling with permutation equivariant graph neural network.
result The method achieves better or comparable graph generation results.
Improves full conformal prediction for stochastic non-conformity measures.
problem Inability of existing conditions to guarantee full conformal prediction validity under stochastic settings.
method Introduces a new sufficient condition: Conditional Independence & Permutation Invariance in Distribution.
result Corrects the insufficient condition and provides a new sufficient condition for full conformal prediction validity.
Generative model for set-valued data using permutation invariant flows.
problem Modeling set-valued data with conditional generative models.
method Conditional generative probabilistic model using continuous normalizing flows with permutation equivariant dynamics.
result Significantly outperforms non-permutation invariant baselines in log likelihood and domain-specific metrics.
Regularizes RNNs to be invariant to input order.
problem Making RNNs invariant to input order.
method Stochastic regularization to enforce permutation invariance.
result Improves model performance on permutation invariant tasks.
HistNetQ improves quantification tasks by optimizing loss functions and eliminating label requirements.
problem Quantification of class prevalence in bags of examples.
method Permutation-invariant Histograms and deep neural networks.
result HistNetQ outperforms other quantification methods and optimizes custom loss functions.
Variational inference struggles with weight symmetries in neural networks, leading to biased posteriors.
problem Weight space symmetries in neural networks cause multimodal posteriors, challenging variational inference.
method Developed a symmetrization mechanism to create permutation invariant variational posteriors.
result Symmetrized variational posteriors have a better fit to the true posterior and improved predictive performance.
Neural networks learn from ensemble forecasts without considering their order.
problem Improving reliability of probabilistic weather forecasts.
method Permutation-invariant neural networks for postprocessing ensemble forecasts.
result Models achieve state-of-the-art prediction quality in surface temperature and wind gust forecasts.
We consider a simple and overarching representation for permutation-invariant functions of sequences (or multiset functions). Our approach, which we call Janossy pooling, expresses a permutation-invariant function as the average of a permutation-sensitive function applied to all reorderings of the input sequence. This …
We propose an end-to-end deep learning learning model for graph classification and representation learning that is invariant to permutation of the nodes of the input graphs. We address the challenge of learning a fixed size graph representation for graphs of varying dimensions through a differentiable node attention po…
PARD generates graphs efficiently and invariantly to node ordering.
problem Graph generation sensitivity to node ordering.
method Integrates autoregressive and diffusion models with a partial order for nodes and edges.
result PARD achieves state-of-the-art performance on molecular and non-molecular datasets.
A new knot invariant uses permutations to extend Jones polynomials.
problem Extending Jones polynomials to classical and virtual knots and links.
method Colorings by permutations of a finite set to define new knot invariants.
result Established properties and computed polynomials for small cases.
Representations of sets are challenging to learn because operations on sets should be permutation-invariant. To this end, we propose a Permutation-Optimisation module that learns how to permute a set end-to-end. The permuted set can be further processed to learn a permutation-invariant representation of that set, avoid…
Paper develops deep neural networks for wireless tasks with reduced complexity.
problem Reducing training complexity for deep neural networks in wireless systems.
method Develops permutation invariant DNNs (PINNs) leveraging wireless task properties.
result Demonstrates dramatic reduction in training complexity for PINNs.
A new method computes Teichmüller polynomials from integer permutations.
problem Computing Teichmüller polynomials for fibered 3-manifolds.
method Using integer permutations to characterize pseudo-Anosov homeomorphisms and train tracks.
result Direct implementation of McMullen's algorithm for Teichmüller polynomials.
4-Legendrian permutation racks can't distinguish knots but recover classical invariants.
problem Distinguishing Legendrian knots using permutation racks.
method Study of 4-Legendrian racks and their effectiveness.
result 4-Legendrian permutation racks cannot distinguish knots but recover classical invariants.
Generates valid Euclidean distance matrices for molecular structures.
problem Generating point clouds in arbitrary rotations and translations is challenging.
method Developed a neural network architecture that produces valid Euclidean distance matrices invariant to rotations and translations.
result The architecture can generate molecular structures in a one-shot fashion by producing Euclidean distance matrices with a three-dimensional embedding.
New method uses Multiple Choice Learning for speech separation.
problem Ambiguous task of assigning model predictions to ground truth signals.
method Uses Multiple Choice Learning (MCL) instead of Permutation Invariant Training (PIT).
result MCL matches PIT performance but is computationally advantageous.
New model preserves graph structure in large datasets.
problem Lack of permutation invariance in graph generation models for large graphs.
method Uses graph embeddings to create a scalable generative model.
result Model maintains structure in large graphs without losing invariance.
Improves efficiency and scalability in multi-agent reinforcement learning.
problem Non-stationarity and inefficiency in critic networks due to agent permutations.
method Proposes a permutation invariant critic (PIC) to avoid changes in critic output due to agent permutations.
result Achieves improvements of test episode reward between 15% to 50% on challenging multi-agent particle environment (MPE).
SetGAN improves GANs by making them more stable and diverse.
problem Training instability and mode collapse in GANs.
method Adversarial architecture that processes sets of generated and real samples, discriminating between their origins in a flexible, permutation invariant manner.
result SetGAN produces more accurate models of the input data with less sensitivity to hyperparameters.
New model learns multisets to predict containment and sizes of differences.
problem Learning permutation invariant representations for flexible containment.
method Formalize multisets, propose training on predicting symmetric difference sizes.
result Model outperforms DeepSets on predicting containment and sizes of symmetric differences.
Enhances GNNs by capturing node relationships, outperforming 2-WL test.
problem Inability of conventional GNNs to fully capture node relationships due to permutation invariance.
method Develops permutation-sensitive aggregation mechanism using permutation groups.
result Proves superior expressivity compared to 2-WL test and not less than 3-WL test.
Many learning algorithms have invariances: when their training data is transformed in certain ways, the function they learn transforms in a predictable manner. Here we formalize this notion using concepts from the mathematical field of category theory. The invariances that a supervised learning algorithm possesses are …
Permutation invariant network learns Wasserstein metrics.
problem Understanding the space of probability measures and comparing distributions.
method Permutation invariant network mapping samples to a low-dimensional space.
result Network can generalize to compute distances between unseen densities and learn moments.
SPAN learns functions over sets invariant to permutations, outperforming existing methods.
problem Learning functions over sets invariant to permutations.
method SPAN architecture that combines neural networks with adversarial permutations.
result SPAN achieves nearly permutation-invariant functions while maintaining accuracy.
The paper models financial correlation matrices using permutation invariant Gaussian models and predicts market anomalies.
problem Modeling and predicting financial correlation matrices from high-frequency data.
method Constructing permutation invariant Gaussian matrix models with 4 parameters, using graph theory and polynomial functions.
result The permutation invariant Gaussian matrix model predicts the expectation values of cubic and quartic polynomials with strong evidence of fit.
ABI adapts to graph data for fast, scalable inference.
problem Challenges in inference on graph-structured data.
method Amortized Bayesian Inference (ABI) framework for graph data.
result ABI successfully addresses challenges in graph data inference.
New graph foundation models respect symmetries for broader applicability.
problem Tailored graph machine learning architectures limit broader applicability.
method Investigates symmetries for label and feature permutations, proving network universal approximator.
result Universal approximator on multisets respecting node and feature permutations.
LMC loss barrier decreases to zero with large network width.
problem Understanding the LMC phenomenon in neural networks.
method Fine-grained analysis of LMC for two-layer ReLU networks.
result LMC loss barrier decreases to zero at a rate of O(m^-1/2) for large network width.
This paper improves model fusion by training-time neuron alignment, reducing barriers in multi-model fusion.
problem Diverse neuron permutations across different settings hinder model fusion performances.
method Training-time neuron alignment using fixed neuron anchors to reduce training-time permutations.
result Training-time neuron alignment improves fusion of pretrained models and federated learning performances.
Permutation-equivariant neural networks improve auction mechanisms by reducing regret and sample complexity.
problem Designing optimal auction mechanisms that balance revenue and bidders' regret.
method Introduced permutation-equivariant neural networks to auction mechanisms.
result Permutation-equivariant neural networks decrease expected ex-post regret and improve model generalizability.
New method improves transfer and robustness of supervised contrastive learning.
problem Class collapse in supervised contrastive learning leads to poor representation quality.
method Adding a weighted class-conditional InfoNCE loss and a class-conditional autoencoder.
result Improves transfer and robustness on 5 standard datasets and 3 worst-group robustness datasets.
We study the problem of designing models for machine learning tasks defined on \emph{sets}. In contrast to traditional approach of operating on fixed dimensional vectors, we consider objective functions defined on sets that are invariant to permutations. Such problems are widespread, ranging from estimation of populati…
New LCM aggregator improves GNN performance and efficiency.
problem Graph neural networks' sensitivity to aggregation function choice.
method Learnable commutative monoid for graph aggregation.
result LCM aggregator achieves performance competitive with recurrent aggregators.
A new DNN structure reduces complexity for wireless tasks.
problem Reducing complexity in training deep neural networks for wireless tasks.
method Proposes a DNN with special structure using permutation invariant a priori information.
result The proposed DNN structure reduces training complexity and model parameters.
This work refines claims about neural network connectivity, showing that simultaneous linear connectivity is possible under certain conditions.
problem Neural networks' loss landscapes are non-convex due to permutation symmetries, leading to high loss barriers between permuted networks.
method The authors introduce and analyze three claims of increasing strength regarding the connectivity of neural networks, focusing on permutations that align networks.
result The authors provide evidence that strong linear connectivity may be possible under certain conditions, specifically when interpolating among three networks of increasing width.
We introduce a simple permutation equivariant layer for deep learning with set structure.This type of layer, obtained by parameter-sharing, has a simple implementation and linear-time complexity in the size of each set. We use deep permutation-invariant networks to perform point-could classification and MNIST-digit sum…
New model preserves symmetry in multivariate time series, improving performance.
problem Implicit ordering in MTS models violates inherent exchangeability.
method Permutation-equivariant 2D state space model with canonical architecture.
result Eliminates sequential dependency chains and simplifies stability analysis.
A new nonparametric test measures dependence between variables using decision trees.
problem Measuring statistical dependence between two variables robustly and efficiently.
method An ensemble of decision trees discriminates between observed and permuted samples without generating the latter.
result The method effectively detects complex relationships from noisy data.
We show how a deep denoising autoencoder with lateral connections can be used as an auxiliary unsupervised learning task to support supervised learning. The proposed model is trained to minimize simultaneously the sum of supervised and unsupervised cost functions by back-propagation, avoiding the need for layer-wise pr…
ShuffleNet is a state-of-the-art light weight convolutional neural network architecture. Its basic operations include group, channel-wise convolution and channel shuffling. However, channel shuffling is manually designed empirically. Mathematically, shuffling is a multiplication by a permutation matrix. In this paper, …
New algorithm uses GNNs to optimize rewards in graph-structured data.
problem Optimizing rewards in molecule design with graph-structured data.
method Embedding permutation invariance into GNNs and using GNTK for regret bounds.
result First GNN confidence bound and phased-elimination algorithm with sublinear regret.
MEM learns set functions from permutation-invariant data.
problem Learning from sets of instances with labels only on sets, not instances.
method Memory-based Exchangeable Model (MEM) with self-attention mechanism.
result Achieved 84.84% accuracy on lung cancer classification.
A new method optimizes slicing directions for SW distances to improve high-dimensional probability measure comparison.
problem Challenging identification of informative slicing directions for SW distances.
method Constrained learning approach to optimize slicing directions, using continuous relaxations and gradient-based primal-dual approach.
result Demonstrated efficacy in learning more informative slicing directions on various high-dimensional data.
A formula for triangle area in Deep Sets form.
problem Finding a polynomial formula for triangle area in Deep Sets form.
method Expressing area as a permutation-invariant function and finding a suitable Deep Sets form.
result Explicit polynomial formula for triangle area in Deep Sets form.
Drinfel'd used associators to construct families of universal representations of braid groups. We consider semi-associators (i.e., we drop the pentagonal axiom and impose a normalization in degree one). We show that the process may be reversed, to obtain semi-associators from universal representations of 3-braids. We v…