The paper models financial correlation matrices using permutation invariant Gaussian models and predicts market anomalies.
problem Modeling and predicting financial correlation matrices from high-frequency data.
method Constructing permutation invariant Gaussian matrix models with 4 parameters, using graph theory and polynomial functions.
result The permutation invariant Gaussian matrix model predicts the expectation values of cubic and quartic polynomials with strong evidence of fit.
Regularizes RNNs to be invariant to input order.
problem Making RNNs invariant to input order.
method Stochastic regularization to enforce permutation invariance.
result Improves model performance on permutation invariant tasks.
Develops a scalable model for drug combination prediction in cancer.
problem Accurate prediction of drug combinations for cancer treatment.
method Permutation invariant multi-output Gaussian Processes with variational approximation and deep generative model.
result Model efficiently borrows information across drug combinations and provides uncertainty quantification.
A new method learns graph distributions invariant to node ordering.
problem Graphs are hard to model due to node ordering invariance issues.
method Score-based generative modeling with permutation equivariant graph neural network.
result The method achieves better or comparable graph generation results.
Representations of sets are challenging to learn because operations on sets should be permutation-invariant. To this end, we propose a Permutation-Optimisation module that learns how to permute a set end-to-end. The permuted set can be further processed to learn a permutation-invariant representation of that set, avoid…
We consider a simple and overarching representation for permutation-invariant functions of sequences (or multiset functions). Our approach, which we call Janossy pooling, expresses a permutation-invariant function as the average of a permutation-sensitive function applied to all reorderings of the input sequence. This …
Improves modeling of sets with permutation invariant densities.
problem Challenges in calculating trace limit practicality of current methods.
method Proposes an alternative approach to define permutation equivariant transformations with closed form trace.
result Improves both training and final performance.
We introduce and study the writhe of a permutation, a circular variant of the well-known inversion number. This simple permutation statistics has several interpretations, which lead to some interesting properties. For a permutation sampled uniformly at random, we study the asymptotics of the writhe, and obtain a non-Ga…
Generative model for set-valued data using permutation invariant flows.
problem Modeling set-valued data with conditional generative models.
method Conditional generative probabilistic model using continuous normalizing flows with permutation equivariant dynamics.
result Significantly outperforms non-permutation invariant baselines in log likelihood and domain-specific metrics.
Generative models of graph structure have applications in biology and social sciences. The state of the art is GraphRNN, which decomposes the graph generation process into a series of sequential steps. While effective for modest sizes, it loses its permutation invariance for larger graphs. Instead, we present a permuta…
Many machine learning tasks such as multiple instance learning, 3D shape recognition, and few-shot image classification are defined on sets of instances. Since solutions to such problems do not depend on the order of elements of the set, models used to address them should be permutation invariant. We present an attenti…
New methods reduce extrapolation errors in feature importance.
problem Flawed feature importance methods using unrestricted permutations lead to extrapolation errors.
method Three new approaches: conditional model reliance, Knockoffs with Gaussian transformation, and restricted ALE plot designs.
result Theoretical and numerical results show our strategies reduce/eliminate extrapolation.
Enhances GNNs by capturing node relationships, outperforming 2-WL test.
problem Inability of conventional GNNs to fully capture node relationships due to permutation invariance.
method Develops permutation-sensitive aggregation mechanism using permutation groups.
result Proves superior expressivity compared to 2-WL test and not less than 3-WL test.
A new knot invariant uses permutations to extend Jones polynomials.
problem Extending Jones polynomials to classical and virtual knots and links.
method Colorings by permutations of a finite set to define new knot invariants.
result Established properties and computed polynomials for small cases.
Consider a noisy linear observation model with an unknown permutation, based on observing y=Π∗Ax∗+w, where x∗∈Rd is an unknown vector, Π∗ is an unknown n×n permutation matrix, and w∈Rn is additive Gaussian noise. We analyze the problem of permutation recovery in a …
A new method reduces memory requirements for sorting high-dimensional data.
problem Efficiently sorting and organizing high-dimensional data with low memory usage.
method Iteratively shuffles N indices and applies SoftSort optimization steps.
result Significantly improves sorting quality for multidimensional data.
Sample efficiency and scalability to a large number of agents are two important goals for multi-agent reinforcement learning systems. Recent works got us closer to those goals, addressing non-stationarity of the environment from a single agent's perspective by utilizing a deep net critic which depends on all observatio…
Novel neural GP kernels learn stable, flexible covariance structures.
problem Scalable and flexible covariance kernels for Gaussian processes.
method Directly learn kriging coefficients and conditional standard deviations using deep neural architectures exploiting permutation-equivariant structure.
result Improved training stability and data efficiency with expressive, non-stationary kernels.
4-Legendrian permutation racks can't distinguish knots but recover classical invariants.
problem Distinguishing Legendrian knots using permutation racks.
method Study of 4-Legendrian racks and their effectiveness.
result 4-Legendrian permutation racks cannot distinguish knots but recover classical invariants.
Transformers tend to learn more symmetric functions in sequence data.
problem Understanding inductive bias in Transformers with infinitely over-parameterized models.
method Analyzing Transformers in the Gaussian process limit, using representation theory of the symmetric group.
result Transformers are biased towards more permutation symmetric functions, and this can be quantitatively predicted.
PARD generates graphs efficiently and invariantly to node ordering.
problem Graph generation sensitivity to node ordering.
method Integrates autoregressive and diffusion models with a partial order for nodes and edges.
result PARD achieves state-of-the-art performance on molecular and non-molecular datasets.
New model preserves symmetry in multivariate time series, improving performance.
problem Implicit ordering in MTS models violates inherent exchangeability.
method Permutation-equivariant 2D state space model with canonical architecture.
result Eliminates sequential dependency chains and simplifies stability analysis.
We study the problem of designing models for machine learning tasks defined on \emph{sets}. In contrast to traditional approach of operating on fixed dimensional vectors, we consider objective functions defined on sets that are invariant to permutations. Such problems are widespread, ranging from estimation of populati…
LMC loss barrier decreases to zero with large network width.
problem Understanding the LMC phenomenon in neural networks.
method Fine-grained analysis of LMC for two-layer ReLU networks.
result LMC loss barrier decreases to zero at a rate of O(m^-1/2) for large network width.
We propose a practical Bayesian optimization method over sets, to minimize a black-box function that takes a set as a single input. Because set inputs are permutation-invariant, traditional Gaussian process-based Bayesian optimization strategies which assume vector inputs can fall short. To address this, we develop a B…
Neural networks learn from ensemble forecasts without considering their order.
problem Improving reliability of probabilistic weather forecasts.
method Permutation-invariant neural networks for postprocessing ensemble forecasts.
result Models achieve state-of-the-art prediction quality in surface temperature and wind gust forecasts.
Permutation invariant network learns Wasserstein metrics.
problem Understanding the space of probability measures and comparing distributions.
method Permutation invariant network mapping samples to a low-dimensional space.
result Network can generalize to compute distances between unseen densities and learn moments.
Permutation-equivariant neural networks improve auction mechanisms by reducing regret and sample complexity.
problem Designing optimal auction mechanisms that balance revenue and bidders' regret.
method Introduced permutation-equivariant neural networks to auction mechanisms.
result Permutation-equivariant neural networks decrease expected ex-post regret and improve model generalizability.
Variational inference struggles with weight symmetries in neural networks, leading to biased posteriors.
problem Weight space symmetries in neural networks cause multimodal posteriors, challenging variational inference.
method Developed a symmetrization mechanism to create permutation invariant variational posteriors.
result Symmetrized variational posteriors have a better fit to the true posterior and improved predictive performance.
Improves full conformal prediction for stochastic non-conformity measures.
problem Inability of existing conditions to guarantee full conformal prediction validity under stochastic settings.
method Introduces a new sufficient condition: Conditional Independence & Permutation Invariance in Distribution.
result Corrects the insufficient condition and provides a new sufficient condition for full conformal prediction validity.
This work improves multi-modal generative models by using permutation-invariant neural networks.
problem Improving multi-modal generative models with tighter variational objectives.
method Developed more flexible aggregation schemes based on permutation-invariant neural networks.
result Our variational objective and flexible aggregation models can better approximate the true joint distribution.
We propose an end-to-end deep learning learning model for graph classification and representation learning that is invariant to permutation of the nodes of the input graphs. We address the challenge of learning a fixed size graph representation for graphs of varying dimensions through a differentiable node attention po…
New algorithm uses GNNs to optimize rewards in graph-structured data.
problem Optimizing rewards in molecule design with graph-structured data.
method Embedding permutation invariance into GNNs and using GNTK for regret bounds.
result First GNN confidence bound and phased-elimination algorithm with sublinear regret.
Efficiently discovers causal DAG permutations without additional assumptions.
problem Learning a DAG up to Markov equivalence.
method Utilizing DAG-specific problem structure, introduces an efficient algorithm for sparse permutations.
result Significant improvement in permutation discovery compared to existing algorithms.
Using deep neural networks that are either invariant or equivariant to permutations in order to learn functions on unordered sets has become prevalent. The most popular, basic models are DeepSets [Zaheer et al. 2017] and PointNet [Qi et al. 2017]. While known to be universal for approximating invariant functions, DeepS…
HistNetQ improves quantification tasks by optimizing loss functions and eliminating label requirements.
problem Quantification of class prevalence in bags of examples.
method Permutation-invariant Histograms and deep neural networks.
result HistNetQ outperforms other quantification methods and optimizes custom loss functions.
New graph foundation models respect symmetries for broader applicability.
problem Tailored graph machine learning architectures limit broader applicability.
method Investigates symmetries for label and feature permutations, proving network universal approximator.
result Universal approximator on multisets respecting node and feature permutations.
We introduce a simple permutation equivariant layer for deep learning with set structure.This type of layer, obtained by parameter-sharing, has a simple implementation and linear-time complexity in the size of each set. We use deep permutation-invariant networks to perform point-could classification and MNIST-digit sum…
New model generates realistic single-cell gene expression data.
problem Generating realistic single-cell gene expression profiles is challenging.
method scLDM, a latent diffusion model using Diffusion Transformers and linear interpolants.
result Superior performance in generating realistic single-cell gene expression data.
QWO speeds up causal discovery in LiGAMs by O(n2).
problem Scalability issues in permutation-based causal discovery methods for LiGAMs.
method Constructing a specific DAG for a permutation and searching over permutations to minimize edge count.
result QWO achieves a speed-up of O(n2) compared to state-of-the-art methods. Proposes FMS for more efficient neural network hyperparameter optimization.
problem Efficient hyperparameter optimization for deep learning models.
method Uses logged checkpoints of trained weights to guide hyperparameter selections.
result Proposes Forecasting Model Search (FMS) method.
BI-EqNO improves Bayesian inference with flexible neural operators.
problem Inaccurate estimation of marginal likelihoods in approximate Bayesian methods.
method Equivariant neural operator framework for generalized approximate Bayesian inference.
result BI-EqNO enhances both deterministic and stochastic approaches to Bayesian inference.
In this paper, we consider the problem of learning functions over sets, i.e., functions that are invariant to permutations of input set items. Recent approaches of pooling individual element embeddings can necessitate extremely large embedding sizes for challenging functions. We address this challenge by allowing stand…
A formula for triangle area in Deep Sets form.
problem Finding a polynomial formula for triangle area in Deep Sets form.
method Expressing area as a permutation-invariant function and finding a suitable Deep Sets form.
result Explicit polynomial formula for triangle area in Deep Sets form.
Drinfel'd used associators to construct families of universal representations of braid groups. We consider semi-associators (i.e., we drop the pentagonal axiom and impose a normalization in degree one). We show that the process may be reversed, to obtain semi-associators from universal representations of 3-braids. We v…
The introduction of convolutional layers greatly advanced the performance of neural networks on image tasks due to innately capturing a way of encoding and learning translation-invariant operations, matching one of the underlying symmetries of the image domain. In comparison, there are a number of problems in which the…
A new kernel test avoids permutations for independence testing.
problem Intractable null distributions of kernel statistics.
method Developed xHSIC and xdCov, avoiding permutations.
result New tests have limiting Gaussian distributions under null.
A new test statistic speeds up MMD while maintaining power.
problem Efficiently testing two distributions without permutations.
method Cross-MMD statistic based on sample-splitting and studentization.
result Cross-MMD has a limiting standard Gaussian distribution under the null.