Study on geometry of Dirichlet distributions using Fisher-Rao metric.
problem Understanding the geometry of Dirichlet distributions.
method Analysis of Fisher-Rao metric on Dirichlet distribution parameter space.
result Geodesic completeness and negative sectional curvature of the space.
We derive and approximate the conjugate prior of Dirichlet and beta distributions.
problem Intractability of conjugate prior for Dirichlet and beta distributions.
method Derive conjugate prior, define closed-form approximation, and provide algorithm.
result Closed-form approximation enables fully tractable Bayesian treatment.
This note investigates a conjugate class for the Dirichlet distribution class in the exponential family.
Dirichlet pruning compresses neural networks by removing unimportant units.
problem Compressing large neural network models without sacrificing performance.
method Assigns Dirichlet distribution over network layers' units and uses variational inference to estimate parameters.
result Achieves state-of-the-art compression performance on larger architectures like VGG and ResNet.
Bayesian models that mix multiple Dirichlet prior parameters, called Multi-Dirichlet priors (MD) in this paper, are gaining popularity. Inferring mixing weights and parameters of mixed prior distributions seems tricky, as sums over Dirichlet parameters complicate the joint distribution of model parameters. This paper s…
Flexible evidential deep learning improves uncertainty quantification in machine learning.
problem Overconfident predictions in machine learning models can lead to serious consequences.
method Proposes flexible evidential deep learning (F-EDL) to model uncertainty over class probabilities using a flexible Dirichlet distribution.
result Empirically demonstrates state-of-the-art uncertainty quantification performance across diverse scenarios.
The Dirichlet mechanism protects privacy while minimizing KL divergence.
problem Minimizing KL divergence while protecting sensitive data privacy.
method Using the exponential mechanism with the KL divergence loss function, resulting in the Dirichlet mechanism.
result Proved a probability tail bound on KL divergence and derived a lower bound for sample complexity.
Extends Hess-Schrader-Uhlenbrock inequality for 1-forms in tamed Dirichlet spaces.
problem Establishing a new inequality for 1-forms in tamed Dirichlet spaces.
method Developed a vector calculus for tamed Dirichlet spaces and applied it to establish the inequality.
result Established the Hess-Schrader-Uhlenbrock inequality for 1-forms in L2-cotangent module. In Bayesian Deep Learning, distributions over the output of classification neural networks are often approximated by first constructing a Gaussian distribution over the weights, then sampling from it to receive a distribution over the softmax outputs. This is costly. We reconsider old work (Laplace Bridge) to construct…
Topic models are widely used to discover the latent representation of a set of documents. The two canonical models are latent Dirichlet allocation, and Gaussian latent Dirichlet allocation, where the former uses multinomial distributions over words, and the latter uses multivariate Gaussian distributions over pre-train…
New method samples manifolds efficiently using Dirichlet distribution.
problem Sampling on complex manifolds efficiently.
method Data-driven Dirichlet sampling on manifolds.
result Efficient sampling respects manifold structure with low computational effort.
We construct a new distribution for the simplex using the Kumaraswamy distribution and an ordered stick-breaking process. We explore and develop the theoretical properties of this new distribution and prove that it exhibits symmetry under the same conditions as the well-known Dirichlet. Like the Dirichlet, the new dist…
A Bayesian approach termed BAyesian Least Squares Optimization with Nonnegative L1-norm constraint (BALSON) is proposed. The error distribution of data fitting is described by Gaussian likelihood. The parameter distribution is assumed to be a Dirichlet distribution. With the Bayes rule, searching for the optimal parame…
Sharp bounds for Dirichlet sums lead to improved Bayesian algorithm analysis.
problem Improving Bayesian algorithm performance through precise deviation bounds.
method Novel integral representation of Dirichlet sum density, Gaussian approximation, complex analysis.
result Significantly sharpened regret bounds for Multinomial Thompson Sampling.
DrNAS improves neural architecture search with Dirichlet distribution and progressive learning.
problem Efficiently search for neural architectures with improved generalization and exploration.
method Formulates architecture search as a distribution learning problem using Dirichlet distribution and gradient-based optimization. Introduces a progressive learning scheme to handle large-scale tasks.
result Achieves state-of-the-art results on CIFAR-10 and ImageNet, demonstrating improved generalization and exploration.
Efficiently models categorical data with low to medium class overlap, improving accuracy over standard distributions.
problem Poor parameter estimates and accuracy in multinomial and Dirichlet multinomial distributions when assumptions are violated.
method Introduces Beta-Liouville multinomial distribution and efficient estimation methods.
result Beta-Liouville multinomial outperforms standard distributions on two out of four datasets.
This note provides an elementary proof of the folklore fact that draws from a Dirichlet distribution (with parameters less than 1) are typically sparse (most coordinates are small).
Capital distribution curve is defined as log-log plot of normalized stock capitalizations ranked in descending order. The curve displays remarkable stability over periods of time. Theory of exchangeable distributions on set partitions, developed for purposes of mathematical genetics and recently applied in non-parametr…
We present a mixed multinomial logit (MNL) model, which leverages the truncated stick-breaking process representation of the Dirichlet process as a flexible nonparametric mixing distribution. The proposed model is a Dirichlet process mixture model and accommodates discrete representations of heterogeneity, like a laten…
New portfolios outperform traditional methods by using factor weights.
problem Improving portfolio allocation in markets driven by factors.
method Factor-weighted Dirichlet portfolios outperform uniform Dirichlet portfolios.
result Factor-weighted portfolios outperform uniformly sampled portfolios in market returns.
Discriminative classifier for compositional data using hierarchical mixture of Generalized Dirichlet models.
problem Classifying compositional data, especially in spam detection and color space identification.
method Hierarchical mixture of discriminative Generalized Dirichlet classifiers, using variational approximation for parameter learning.
result First time a variational upper-bound for Generalized Dirichlet mixture is proposed in literature.
This paper analyzes MCMC algorithms on large graphs using Dirichlet forms.
problem Analyzing the behavior of MCMC algorithms in high-dimensional problems.
method Utilizes Mosco convergence of Dirichlet forms to study RWM algorithm on large graphs.
result Demonstrates the advantages of Dirichlet form approach over standard diffusion methods.
LDTA expands LDA's topic modeling capacity with tree-structured priors.
problem Limited expressiveness of Dirichlet priors in LDA for complex topic relationships.
method Introduces Latent Dirichlet-Tree Allocation (LDTA) with Dirichlet-Tree (DT) priors, and develops universal mean-field variational inference and Expectation Propagation.
result LDTA enables expressive, tree-structured priors over topic proportions, expanding modeling capacity of LDA.
We propose Dirichlet Simplex Nest, a class of probabilistic models suitable for a variety of data types, and develop fast and provably accurate inference algorithms by accounting for the model's convex geometry and low dimensional simplicial structure. By exploiting the connection to Voronoi tessellation and properties…
DBU models struggle with robust uncertainty estimates under adversarial attacks.
problem Robustness of DBU models in adversarial settings.
method Investigated robustness of DBU models under adversarial attacks; proposed median smoothing approach.
result DBU models are not robust in indicating correctly and wrongly classified samples, detecting adversarial examples, and distinguishing ID and OOD data.
Precise estimation of uncertainty in predictions for AI systems is a critical factor in ensuring trust and safety. Deep neural networks trained with a conventional method are prone to over-confident predictions. In contrast to Bayesian neural networks that learn approximate distributions on weights to infer prediction …
Study on length distribution of random multicurves on large genus surfaces converging to Poisson-Dirichlet distribution.
problem Length statistics of random multicurves on large genus hyperbolic surfaces.
method Analytical proof of convergence to Poisson-Dirichlet distribution as genus tends to infinity.
result Mean lengths of the three longest components converge to specific percentages of total length as genus increases.
Corrected and improved simulation methods for Dirichlet-Laplace prior.
problem Incorrect sampling order in original MCMC method for DL prior.
method Provided two solutions: correction and alternative algorithm.
result Improved simulation methods for DL prior without affecting theoretical results.
Generative model for high-dimensional categorical data using Gaussian-Dirichlet fields.
problem Efficiently modeling and predicting high-dimensional categorical data.
method Combines Dirichlet and Gaussian processes for spatio-temporal modeling.
result Model accurately approximates categorical data in unobserved locations.
New method calculates DMN log-likelihood faster.
problem Precise and fast computation of DMN log-likelihood.
method Derived a closed form expression using gamma function properties.
result Closed form calculation is faster with same accuracy.
New framework models uncertainty in classification debates.
problem Weak interpretability of existing uncertainty quantification methods.
method Courtroom analogy and Mixture of Dirichlet Experts (MoDEX) model.
result MoDEX achieves state-of-the-art uncertainty quantification performance.
WBCP improves conformal prediction for distribution shifts using weighted Dirichlet posteriors.
problem Handling distribution shifts in conformal prediction.
method Generalizes Bayesian Quadrature Conformal Prediction (BQ-CP) to arbitrary importance-weighted settings.
result WBCP maintains coverage guarantees while providing richer uncertainty information.
A robust bandit algorithm uses Dirichlet sampling to minimize regret under various distributional assumptions.
problem Robustness of bandit algorithms to model misspecification.
method Dirichlet Sampling (DS) algorithm based on pairwise comparisons and re-sampling of arm observations.
result Different DS variants achieve optimal regret guarantees for bounded distributions and logarithmic regret for semi-bounded distributions.
To scale non-parametric extensions of probabilistic topic models such as Latent Dirichlet allocation to larger data sets, practitioners rely increasingly on parallel and distributed systems. In this work, we study data-parallel training for the hierarchical Dirichlet process (HDP) topic model. Based upon a representati…
BIF assesses feature importance using Dirichlet distribution and Bayesian inference.
problem Quantitative feature importance assessment in statistical models.
method Utilizes Dirichlet distribution for probabilistic feature importance assessment via approximate Bayesian inference.
result Learned importance provides relative significance and confidence quantification of features.
Enhances uncertainty estimation in neural networks using Dirichlet-based MC Dropout.
problem Deterministic predictions without uncertainty estimates in neural networks.
method Integrates Dirichlet-based framework within Monte Carlo Dropout.
result Improves quality of uncertainty estimates in deep learning models.
Bayesian model fuses multiple classifiers with explicit correlation modeling.
problem Combining outputs of multiple classifiers with explicit correlation.
method Hierarchical Bayesian model with correlated Dirichlet distribution.
result Fused classifier performance can be Bayes optimal even for highly correlated base classifiers.
Paper introduces a new text clustering model using Beta-Liouville priors.
problem Clustering short text data.
method Develops a hierarchical mixture model with Beta-Liouville priors for short text clustering.
result The Beta-Liouville distribution offers a more flexible correlation structure for short text clustering.
We propose the supervised hierarchical Dirichlet process (sHDP), a nonparametric generative model for the joint distribution of a group of observations and a response variable directly associated with that whole group. We compare the sHDP with another leading method for regression on grouped data, the supervised latent…
Study on Dirichlet process mixtures for clustering consistency.
problem Consistency of clustering with Dirichlet process mixtures.
method Analysis of posterior distribution as sample size increases, focusing on consistency for the number of clusters.
result Consistency for the number of clusters can be achieved with a properly adapted concentration parameter in a Bayesian setting.
This research improves neural network uncertainty estimates and reliability.
problem Lack of inherent uncertainty estimates and variability in softmax scores.
method Ensemble-based Dirichlet modeling with method of moments estimator.
result Improved stability and predictive uncertainty estimates.
We consider the problem of sequential learning from categorical observations bounded in [0,1]. We establish an ordering between the Dirichlet posterior over categorical outcomes and a Gaussian posterior under observations with N(0,1) noise. We establish that, conditioned upon identical data with at least two observatio…
Study reveals how boundary wave operator determines metric properties on anti-de Sitter spacetimes.
problem Determining metric properties on anti-de Sitter spacetimes.
method Analysis of Klein-Gordon equation and Dirichlet-to-Neumann map.
result Determines the Taylor series of the bulk metric at the boundary.
Bayesian deep learning uses function-space priors to improve model uncertainty and robustness.
problem Bayesian deep learning struggles with model-specific weight-space priors that are hard to interpret and specify.
method Apply a Dirichlet prior in predictive space and perform approximate function-space variational inference.
result The approach improves uncertainty quantification, scalability, and adversarial robustness in large-scale image classification.
New method improves ensemble inference for high-class tasks.
problem High inference costs for ensemble models.
method Proxy-Dirichlet target to minimize reverse KL-divergence.
result Resolves gradient issues for large-scale classification tasks.
A new method uses a product of experts with Dirichlet variables to approximate complex distributions.
problem Approximating complex distributions with tractable models.
method A product of experts with auxiliary Dirichlet variables, using a Feynman identity to sample and optimize.
result The method efficiently approximates complex distributions using a product of experts and Dirichlet variables.
Proposes a new method for robust uncertainty quantification in regression tasks.
problem Robust uncertainty estimation for deep neural networks in regression tasks.
method Generalized Auxiliary Uncertainty Estimator (AuxUE) scheme, considering both aleatoric and epistemic uncertainties.
result DIDO method provides robust uncertainty estimates in noisy inputs, scalable to image-level and pixel-wise tasks.
Researchers study the normalizing constant of a continuous categorical distribution.
problem Understanding the normalizing constant of the continuous categorical distribution.
method Characterize numerical behavior and present theoretical and methodological advances.
result The normalizing constant can be written in closed form using elementary functions.