Random permutations can offer faster convergence than with-replacement sampling for some functions.
problem Understanding when and how random permutations outperform with-replacement sampling in SGD convergence.
method Analyzing convergence rates for different function classes (1D strongly convex, general strongly convex, quadratic strongly convex).
result The optimal convergence gap between random and permutation-based SGD varies from exponential to nonexistent, depending on the function class.
We introduce and study the writhe of a permutation, a circular variant of the well-known inversion number. This simple permutation statistics has several interpretations, which lead to some interesting properties. For a permutation sampled uniformly at random, we study the asymptotics of the writhe, and obtain a non-Ga…
Proposes a simple algorithm to generate data similar to real series.
problem Generating data similar to real series with constraints.
method Random permutation of Monte Carlo generated numbers, accepted if objective function is minimized.
result Demonstrated by generating S\&P 500 log-returns.
C-MinHash reduces the number of permutations needed for MinHash from thousands to just two.
problem Approximating Jaccard similarity in large binary datasets using many permutations.
method Initial permutation followed by circulant shifting of a second permutation to generate hashes.
result C-MinHash achieves unbiased Jaccard similarity estimation with uniformly smaller variance.
A new method reduces computational costs for testing RF variable importance measures.
problem Testing variable importance measures from random forests is computationally expensive and challenging.
method Sequential permutation testing and sequential p-value estimation to reduce computational costs.
result Theoretical properties of sequential tests are confirmed, maintaining type-I error and high power.
Variable selection in sparse regression models is an important task as applications ranging from biomedical research to econometrics have shown. Especially for higher dimensional regression problems, for which the link function between response and covariates cannot be directly detected, the selection of informative va…
TRIP detects unreliable feature importance scores in random forests.
problem Unreliable feature importance scores in random forests due to model extrapolation.
method Develops TRIP (Test for Reliable Interpretation via Permutation) to detect unreliable permutation feature importance scores.
result TRIP reliably detects unreliable permutation feature importance scores in high-dimensional settings.
Paper proves a Central Limit Theorem for Random Forest Permutation Importance Measure.
problem Lack of theoretical analysis of Random Forest Permutation Importance Measure (RFPIM).
method Formal proof using U-Statistics theory, deviating from conventional Random Forest model.
result Established a Central Limit Theorem for RFPIM.
New bounds for SGD show improved performance in various settings.
problem Improving convergence bounds for SGD with random permutations.
method Analyzing convergence of SGD with random reshuffling and arbitrary permutations.
result Tighter lower bounds for weighted average iterates in both convex and strongly-convex cases.
A new method improves feature importance and model stress-testing reliability.
problem Estimating feature contributions in machine learning models for trust and transparency.
method Replacing multiple random permutations with a single, deterministic, and optimal permutation.
result Improved bias-variance tradeoffs and accuracy in challenging scenarios.
We consider a simple and overarching representation for permutation-invariant functions of sequences (or multiset functions). Our approach, which we call Janossy pooling, expresses a permutation-invariant function as the average of a permutation-sensitive function applied to all reorderings of the input sequence. This …
Cheap permutation tests speed up distribution testing without sacrificing accuracy.
problem Efficiently testing distribution differences and independence.
method Group datapoints into bins and permute only these bins, using stored sufficient statistics.
result Cheap permutation tests maintain the accuracy and optimality of standard tests but are significantly faster.
Graph neural networks improve systemic risk measures for financial networks.
problem Computing systemic risk measures for graph-structured financial networks.
method Extended permutation equivariant neural networks (X-PENNs) for numerical approximation.
result Graph neural networks outperform other methods in approximating optimal allocations.
Generative models of graph structure have applications in biology and social sciences. The state of the art is GraphRNN, which decomposes the graph generation process into a series of sequential steps. While effective for modest sizes, it loses its permutation invariance for larger graphs. Instead, we present a permuta…
OPORP combines permutation and random projection for efficient data vector compression.
problem Efficiently estimating cosine similarity in embedding-based retrieval applications.
method OPORP uses a permutation followed by a random vector dot product, then aggregates and normalizes the results into bins.
result OPORP improves the estimation of cosine similarity, reducing variance and improving accuracy.
Deep neural networks have demonstrated cutting edge performance on various tasks including classification. However, it is well known that adversarially designed imperceptible perturbation of the input can mislead advanced classifiers. In this paper, Permutation Phase Defense (PPD), is proposed as a novel method to resi…
Consider a noisy linear observation model with an unknown permutation, based on observing y=Π∗Ax∗+w, where x∗∈Rd is an unknown vector, Π∗ is an unknown n×n permutation matrix, and w∈Rn is additive Gaussian noise. We analyze the problem of permutation recovery in a …
Graphs benefit from unique node identifiers but lose permutation-equivariance. We propose a method to maintain this property.
problem Graph Neural Networks' inherent limitations due to message-passing structure.
method Propose permutation-equivariant UID models and a contrastive loss to regularize them.
result Our method improves generalization and convergence, achieving state-of-the-art performance.
Paper proposes a differentially private test for joint dependence among random vectors.
problem Detecting joint dependence among sensitive data while maintaining privacy.
method Differentially private permutation methodology for dHSIC test.
result Proposed test attains minimax optimal power across privacy regimes.
A new test validates ensemble models against the null hypothesis.
problem Validating ensemble models against the null hypothesis of a constant response.
method Randomized permutation test on SVEM model predictions.
result The test maintains Type I error rate even with more parameters than observations.
A new metric assesses causal graphs using node permutations to detect inconsistencies.
problem Quantifying the goodness of causal graphs and distinguishing them from random graphs.
method Constructing a baseline through node permutations and comparing inconsistencies.
result The proposed metric can distinguish between true and wrong causal graphs.
Uniformly random permutations converge to regular representation on surface groups.
problem Understanding the behavior of random homomorphisms to symmetric groups.
method Polynomial approximation and random walk analysis.
result Strong convergence of random representations to regular representation.
Graph Neural Networks struggle on random graphs without node identifiers.
problem Graph Neural Networks' limitations on random graphs without node identifiers.
method Study of Graph Neural Networks and Structural Graph Neural Networks convergence on large random graphs.
result Structural Graph Neural Networks are more powerful and universal than Graph Neural Networks on random graphs.
Random braids that are formed by multiplying randomly chosen permutation braids are studied by analyzing their behavior under Garside's weighted decomposition and cycling. Using this analysis, we propose a polynomial-time algorithm to the conjugacy problem that is successful for random braids in overwhelming probabilit…
A new random forest algorithm improves tree construction for optimal performance.
problem Improving the performance of random forests, especially in complex and smooth scenarios.
method Adaptive split-balancing method using permutation-based splitting criterion.
result Achieves minimax optimality under various Lipschitz and Hölder classes.
Enhances functional classifier performance with new tree-based methods and unbiased feature importance assessment.
problem Challenges of high-dimensional functional data and biased feature importance assessment.
method Augmented functional classification trees and random forests with ad-hoc conditional permutations for unbiased feature importance.
result Significant enhancement in predictive power of functional classifiers through new feature importance assessment.
Study tests financial market efficiency using random number generator tests.
problem Check for informational efficiencies in financial markets.
method Analysed binary daily returns as random number generators, split analysis by annual and company levels, investigated longer-term efficiency over Nasdaq-listed companies.
result Information efficiency varies across years and reflects large-scale market impacts.
Random covers of surfaces have tangle-free monodromy and the Putman-Wieland property.
problem Understanding the properties of random covers of surfaces.
method Analyzing the fraction of degree n covers of a surface with specific properties as n increases. result The fraction of degree n covers of a surface with the Putman-Wieland property tends to 1 as n approaches infinity. Randomization is minimax-optimal for variance in experimental design, even with structure.
problem Designing optimal randomized experiments for variance minimization.
method Analyzing permutation symmetric and non-symmetric sets of outcomes, proposing inference-constrained MSOD.
result Randomization is minimax-optimal for variance, even with structure, and requires uniformity constraints for Fisher's exact test.
The paper proves an inequality for symmetric polynomials under a fixed point measure.
problem An inequality for elementary symmetric polynomials under a fixed point measure of permutations.
method Constructing differential operators to set up a monotone flow.
result The inequality is proven and is sharp.
Spectral methods achieve near-optimal performance in orthogonal and permutation group synchronization.
problem Recovering group elements from pairwise measurements in computer vision.
method Spectral methods applied with the leave-one-out technique.
result Near-optimal performance bounds for orthogonal and permutation group synchronization established.
New model learns graph spectra accurately, outperforming existing methods.
problem Graph diffusion models struggle to distinguish certain graph families and their spectra.
method Leveraged random matrix theory to analytically extract spectral properties, introducing Dyson Diffusion Model.
result Dyson Diffusion Model learns graph spectra accurately and outperforms existing models.
A new nonparametric test measures dependence between variables using decision trees.
problem Measuring statistical dependence between two variables robustly and efficiently.
method An ensemble of decision trees discriminates between observed and permuted samples without generating the latter.
result The method effectively detects complex relationships from noisy data.
To model categorical response variables given their covariates, we propose a permuted and augmented stick-breaking (paSB) construction that one-to-one maps the observed categories to randomly permuted latent sticks. This new construction transforms multinomial regression into regression analysis of stick-specific binar…
Study recovers spike order in noisy tensor estimation without SNR assumptions.
problem Estimating multiple signal vectors from noisy tensor observations.
method Gradient flow optimization of a nonconvex function.
result Determines sample complexity for efficient permutation recovery.
Study detects edge correlation between unlabeled random graphs.
problem Detect edge correlation between unlabeled random graphs.
method Hypothesis testing, conditional second-moment method, pseudoforest structure, enumeration of subpseudoforests.
result Sharp threshold for phase transition in testing error probability.
This study examines a single attention layer's capabilities using random features.
problem Understanding the learning and generalization of a single multi-head attention layer.
method Random feature setting with large number of heads, frozen query and key matrices, and trainable value matrices.
result Random-feature attention layer can express a broad class of permutation-invariant target functions.
Single-microphone, speaker-independent speech separation is normally performed through two steps: (i) separating the specific speech sources, and (ii) determining the best output-label assignment to find the separation error. The second step is the main obstacle in training neural networks for speech separation. Recent…
Efficient privacy-preserving machine learning framework using random transformations.
problem Slow training and inference speed in privacy-preserving machine learning systems.
method Random transformations like linear and permutation, combined with arithmetic sharing.
result High efficiency and low computation cost in private machine learning.
Estimates Hurst exponent of log-volatility using KS statistic, addressing serial correlation in financial data.
problem Estimating Hurst exponent of log-volatility in financial time series with serial correlation.
method Proposes a random permutation procedure to remove serial correlation, using the Kolmogorov-Smirnov statistic for distribution-based estimation.
result Establishes the asymptotic variance of the estimator and reveals statistically significant hierarchy of roughness in volatility measures.
The default variable-importance measure in random Forests, Gini importance, has been shown to suffer from the bias of the underlying Gini-gain splitting criterion. While the alternative permutation importance is generally accepted as a reliable measure of variable importance, it is also computationally demanding and su…
The paper derives theoretical foundations for two common machine learning variable importance measures.
problem Understanding variable importance in machine learning problems.
method The paper derives closed-form expressions for Permute-and-Predict (PaP) and Leave-One-Covariate-Out (LOCO) methods.
result Theoretical derivations explain the behavior of PaP and LOCO under collinearity, linking them to coefficients and predictor variability.
A new method uses vectorized summaries of persistence diagrams for efficient hypothesis testing.
problem Efficient hypothesis testing for large and complex persistence diagrams.
method Vectorized summaries of Betti functions and a new shuffling technique.
result The vectorized Betti function leads to competitive results compared to baseline methods.
Proposes a few-shot learning method for feature selection without labeled data.
problem Feature selection in unlabeled data with limited instances.
method Uses Concrete random variables and permutation-invariant neural networks to select features from multiple source tasks.
result Outperforms existing methods in feature selection performance.
Proposes new random models for fuzzy clustering similarity measures.
problem Challenges in choosing a random model for fuzzy clustering similarity measures.
method Introduces three intuitive and explainable random models for fuzzy clusterings.
result Each random model has distinct behavior, emphasizing the importance of accurate model selection.
C-OPH improves One Permutation Hashing by using a shorter circulant permutation.
problem Improving the accuracy of One Permutation Hashing (OPH) for Jaccard similarity estimation.
method Develops a new densification method using a shorter circulant permutation.
result Achieves the smallest estimation variance for Jaccard similarity.
Representations of sets are challenging to learn because operations on sets should be permutation-invariant. To this end, we propose a Permutation-Optimisation module that learns how to permute a set end-to-end. The permuted set can be further processed to learn a permutation-invariant representation of that set, avoid…
Permutations linked to knots and links, with unknots counted by Schröder numbers.
problem Understanding permutations as knots and links.
method Using grid diagrams and Bennequin's inequality.
result Permutations corresponding to unknots and links are counted by Schröder numbers.