We solve the mean parametrization of von Mises-Fisher distribution.
problem No closed-form normalization function for mean parameters exists.
method Derived a second-order ODE for mean normalizer and provided approximations.
result Rapid evaluation of densities and natural parameters in terms of mean parameters.
We present a derivation of the Kullback Leibler (KL)-Divergence (also known as Relative Entropy) for the von Mises Fisher (VMF) Distribution in d-dimensions.
Sparse prototypes improve clustering of high-dimensional directional data.
problem Clustering high-dimensional directional data like texts.
method Estimate a von Mises mixture using l1 penalized likelihood and EM algorithm.
result Sparse prototypes enhance interpretability and clustering performance.
Mixture modelling involves explaining some observed evidence using a combination of probability distributions. The crux of the problem is the inference of an optimal number of mixture components and their corresponding parameters. This paper discusses unsupervised learning of mixture models using the Bayesian Minimum M…
New autoencoder improves latent space learning by optimizing sliced Gromov-Wasserstein discrepancies.
problem Improving inner discrepancy between prior and posterior distributions in autoencoders.
method Proposed spherical sliced fused Gromov Wasserstein (SSFG) and variants (MSSFG, PSSFG) to find important directions.
result New autoencoders achieve favorable performance in latent manifold learning, image generation, and reconstruction.
A new probabilistic approach improves deep metric learning by considering image uncertainties and class-specific variances.
problem Proxy-based deep metric learning struggles with image uncertainties and class-specific structures.
method Introduces non-isotropic probabilistic proxy-based deep metric learning using directional von Mises-Fisher distributions.
result Improves generalization performance and competitive on standard benchmarks.
A new method estimates the number of clusters on spherical data.
problem Estimating the number of clusters in spherical data.
method Spherical X-means (SX-means) method assuming von Mises-Fisher distributions.
result Shows the performance of SX-means in estimating the number of clusters.
A new distance metric for vMF distributions simplifies spherical data analysis.
problem Intractability of normalization constants and lack of suitable geometric metrics for comparing vMF distributions.
method Proposes a Wasserstein-like distance that decomposes vMF distribution discrepancies into angular and concentration components.
result The proposed distance metric induces a latent geometric structure on the space of non-degenerate vMF distributions.
Proposes vMF distribution for skewed elliptical distributions.
problem Skewed distributions not adequately modeled by symmetric distributions.
method Introduces von-Mises-Fisher (vMF) distribution to represent skewed elliptical distributions.
result vMF distribution provides an explicit and simple probability representation of skewed elliptical distributions.
Although stochastic gradient descent (SGD) is a driving force behind the recent success of deep learning, our understanding of its dynamics in a high-dimensional parameter space is limited. In recent years, some researchers have used the stochasticity of minibatch gradients, or the signal-to-noise ratio, to better char…
The paper improves interpolation in generative models by using specific base distributions.
problem Unexpected side effects in linear interpolations of normalizing flows.
method Enforces a specific manifold using Dirichlet and von Mises-Fisher base distributions.
result Superior performance in terms of bits per dimension, FID, and KID scores for interpolation.
A new method for speaker recognition on hyperspheres improves on PLDA's limitations.
problem Improving speaker recognition on hyperspheres with PLDA's limitations.
method Probabilistic Spherical Discriminant Analysis (PSDA) using Von Mises-Fisher distributions.
result PSDA scores are closed-form and can handle various trials, improving over PLDA.
A drone-based MOT algorithm tracks vehicles using neural network detections and TPMBM filter.
problem Tracking multiple vehicles from drone-mounted cameras.
method Neural network for object detection, TPMBM filter for trajectory estimation, von-Mises Fisher distribution for DOA.
result TPMBM filter optimally estimates vehicle trajectories.
Generative models on spheres improve discrete sequence sampling.
problem Learning generative models for discrete sequences in continuous space.
method Work on sphere Sd−1, using von Mises-Fisher distribution and radial symmetry. result Improved results on Sudoku and language modeling with vMF path.
The Variational Auto-Encoder (VAE) is one of the most used unsupervised machine learning models. But although the default choice of a Gaussian distribution for both the prior and posterior represents a mathematically convenient distribution often leading to competitive results, we show that this parameterization fails …
Learning suitable latent representations for observed, high-dimensional data is an important research topic underlying many recent advances in machine learning. While traditionally the Gaussian normal distribution has been the go-to latent parameterization, recently a variety of works have successfully proposed the use…
We propose a novel model for generating graphs similar to a given example graph. Unlike standard approaches that compute features of graphs in Euclidean space, our approach obtains features on a surface of a hypersphere. We then utilize a von Mises-Fisher distribution, an exponential family distribution on the surface …
Traditional topic models do not account for semantic regularities in language. Recent distributional representations of words exhibit semantic consistency over directional metrics such as cosine similarity. However, neither categorical nor Gaussian observational distributions used in existing topic models are appropria…
Metric-based few-shot learning methods try to overcome the difficulty due to the lack of training examples by learning embedding to make comparison easy. We propose a novel algorithm to generate class representatives for few-shot classification tasks. As a probabilistic model for learned features of inputs, we consider…
Study of filtering and smoothing in submanifolds of Euclidean space.
problem Filtering and smoothing in continuous-discrete time on submanifolds.
method Formal expressions and projection approach for prediction and smoothing.
result Agreement with classical results for prediction, differences for smoothing.
We treat the problem of estimation of orientation parameters whose values are invariant to transformations from a spherical symmetry group. Previous work has shown that any such group-invariant distribution must satisfy a restricted finite mixture representation, which allows the orientation parameter to be estimated u…
Score matching method improves density estimation for truncated data on manifolds.
problem Density estimation for truncated data on manifolds with intractable normalising constant.
method Truncated score matching extended to Riemannian manifolds with boundary.
result Score matching estimator approximates true parameter values with low error.
This paper considers statistical estimation problems where the probability distribution of the observed random variable is invariant with respect to actions of a finite topological group. It is shown that any such distribution must satisfy a restricted finite mixture representation. When specialized to the case of dist…
T-PSDA improves speaker recognition accuracy on toroidal submanifolds.
problem Improving speaker recognition accuracy on hypersphere embeddings.
method Extends PSDA to model within and between-speaker variabilities in toroidal submanifolds of the hypersphere.
result T-PSDA achieves accuracy on par with cosine scoring on VoxCeleb and large accuracy gains on NIST SRE'21.
The modelling of data on a spherical surface requires the consideration of directional probability distributions. To model asymmetrically distributed data on a three-dimensional sphere, Kent distributions are often used. The moment estimates of the parameters are typically used in modelling tasks involving Kent distrib…
A new distribution addresses scalability and numerical stability issues of the vMF.
problem Scalability and numerical stability issues in sampling from the von Mises-Fisher (vMF) distribution.
method Proposes the Power Spherical distribution, retaining vMF's properties but addressing its drawbacks.
result Demonstrates the stability of Power Spherical distributions and applies it to a variational auto-encoder.
New normalizing flows for sphere distributions improve complexity and scale handling.
problem No straightforward normalizing flows for Fisher-Bingham distributions in higher dimensions.
method Zoom-linear-project (ZLP)-Fisher flows that gradually add complexity and handle varying scales.
result Generalizes Fisher-Bingham distributions to normalizing flows in any dimension.
Proposes a new latent variable model for hyperspherical latent spaces.
problem Efficiently modeling heavy-tailed distributions in hyperspherical latent spaces.
method Introduces spherical Cauchy (spCauchy) latent variables and applies Möbius transformations.
result Shows spCauchy recovers vMF geometry in high-concentration limits and avoids complex evaluations.
Improves point-cloud reconstruction by optimizing projections with self-attention.
problem Inefficient and non-metric projection methods for sliced Wasserstein distances.
method Proposes distributional sliced Wasserstein distance with self-attention for permutation-invariant and metric optimization.
result Self-attention amortized distributional projection optimization achieves better performance in point-cloud reconstruction.
The Softmax function is used in the final layer of nearly all existing sequence-to-sequence models for language generation. However, it is usually the slowest layer to compute which limits the vocabulary size to a subset of most frequent types; and it has a large memory footprint. We propose a general technique for rep…
A new probabilistic BTD method for tensor data.
problem Modeling higher-order tensors with robust inference.
method Probabilistic Block-Term Decomposition using variational Bayesian inference and von-Mises Fisher distribution.
result The proposed pBTD can quantify multi-linear structures robustly.
Paper develops a new state estimation method for nonlinear systems.
problem State estimation for nonlinear state-space models is intractable.
method Developed a variational inference approach based on Gaussian approximations.
result The method outperforms alternative Gaussian approaches in various examples.
The PARAFAC2 is a multimodal factor analysis model suitable for analyzing multi-way data when one of the modes has incomparable observation units, for example because of differences in signal sampling or batch sizes. A fully probabilistic treatment of the PARAFAC2 is desirable in order to improve robustness to noise an…
We consider the problem of subspace estimation in a Bayesian setting. Since we are operating in the Grassmann manifold, the usual approach which consists of minimizing the mean square error (MSE) between the true subspace U and its estimate U^ may not be adequate as the MSE is not the natural metric in the Gra…
New Riemannian radial distributions help estimate parameters on symmetric spaces.
problem Challenges in manifold data analysis due to lack of parametric distributions.
method Introduced Riemannian radial distributions on symmetric spaces, utilized symmetry, and developed M-estimators.
result MLE achieves root-n convergence rate up to logarithmic terms, demonstrating optimality.
Generative models improve angular variable simulation in high dimensions.
problem Lack of flexibility and scalability in simulating multivariate angular variables.
method Introducing generative adversarial networks, normalizing flows, and flow matching.
result Deep learning methods outperform classical parametric models in complex data structures.
Symmetry helps VI recover certain statistics.
problem Understanding how symmetry in variational inference affects the recovery of statistics.
method Developed a general theory of symmetry-induced statistic recovery in variational inference.
result Symmetry can force the recovery of certain statistics in VI, even under model misspecification.
New summary measures reveal geometric structure in weighted measures on manifolds.
problem Lack of geometric information in standard weight-only summaries.
method Heat-kernel entropy profiles, tracking nonuniformity across scales.
result Geometric effective sample size discounts nearby or duplicate particles.
Bayesian neural networks improve uncertainty calibration without sacrificing accuracy.
problem Bayesian neural networks struggle with uncertainty calibration and high-dimensional geometry.
method Model uncertainty only in weight directions using a von Mises-Fisher posterior on the unit sphere, deriving a compact KL term.
result A lightweight, dimension-aware variational unit improves calibration without sacrificing accuracy.
Paper generalizes kernel mean embedding to von Neumann-algebra-valued measures.
problem Analyzing complex multivariate distributions and quantum mechanics.
method Generalizes kernel mean embedding to von Neumann-algebra-valued measures in reproducing kernel Hilbert modules.
result Injectivity and universality of the generalized KME are confirmed.
Develops diffusion models for time-varying correlation on the circle.
problem Time-varying correlation modeling on the circle.
method Stochastic processes on the unit circle, specifically Brownian motion and von Mises diffusion.
result Derives an accurate analytical approximation to the transition density of the von Mises diffusion.
SHMM models human mobility from GPS and text data, overcoming text sparsity.
problem Modeling human mobility from semantic trace data, especially addressing text sparsity.
method SHMM is a multi-modal spherical hidden Markov model that jointly models location, time, and text embeddings on a unit sphere using vMF distribution.
result SHMM outperforms state-of-the-art models in next location prediction and has lower training cost.
New method generates molecular conformations efficiently.
problem Generating accurate molecular conformations efficiently.
method Variational approximation of rotatable bond torsion angles as a mixture of von Mises distributions.
result VonMisesNet generates conformations orders of magnitude faster than existing methods.
This work improves transferability by considering conditional distributions in feature representations.
problem Improving transferability across multiple domains by considering conditional distributions.
method Introducing von Neumann conditional divergence to quantify the functional dependence between features and desired response.
result Favorable performance in terms of smaller generalization error and less catastrophic forgetting.
Study of Gaussian distributions using entropic Gromov-Wasserstein and inner product Gromov-Wasserstein.
problem Optimal transportation between Gaussian distributions with different dimensions.
method Entropic Gromov-Wasserstein and inner product Gromov-Wasserstein, with closed-form expressions and von Neumann's trace inequality.
result Closed-form expressions for the entropic IGW and its unbalanced variant between Gaussian distributions.
The modelling of empirically observed data is commonly done using mixtures of probability distributions. In order to model angular data, directional probability distributions such as the bivariate von Mises (BVM) is typically used. The critical task involved in mixture modelling is to determine the optimal number of co…
We study rays in von Mangoldt planes, which has applications to the structure of open complete manifolds with lower radial curvature bounds. We prove that the set of souls of any rotationally symmetric plane of nonnegative curvature is a closed ball, and if the plane is von Mangoldt, we compute the radius of the ball. …
Circular variables arise in a multitude of data-modelling contexts ranging from robotics to the social sciences, but they have been largely overlooked by the machine learning community. This paper partially redresses this imbalance by extending some standard probabilistic modelling tools to the circular domain. First w…