Exponential family distributions are highly useful in machine learning since their calculation can be performed efficiently through natural parameters. The exponential family has recently been extended to the t-exponential family, which contains Student-t distributions as family members and thus allows us to handle noi…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “distribution families”
A method for converting NIW parameters for better estimation.
problem Estimating parameters of multivariate normal distribution.
method Convergent procedure for converting mean parameters to natural parameters in NIW family.
result Maximum likelihood estimation of natural parameters from observed statistics.
Study on Alexander polynomials in braids, linking number theory and topology.
problem Distribution of Alexander polynomials in specific families of braids.
method Exploration of arithmetic invariants and analogies with number theory.
result New directions in arithmetic topology and statistics.
Finite-Time Regret of Thompson Sampling Algorithms for Exponential Family Multi-Armed Banditsstat.ML
New Thompson sampling algorithm reduces regret for exponential family bandits.
problem Minimizing regret in multi-armed bandit problems with exponential family rewards.
method Proposes ExpTS and ExpTS algorithms using novel sampling distributions.
result Minimizes both finite-time and asymptotic regret for exponential family rewards.
Improved inference for models with continuous latent variables.
problem Inference accuracy with traditional variational methods is limited.
method Reparameterized Variational Rejection Sampling (RVRS) using a proposal distribution with a reparameterized gradient estimator.
result RVRS offers a better trade-off between computational cost and inference fidelity.
A new distribution family extends the -stable distribution with a degree of freedom parameter.
problem Lack of moments in the -stable distribution.
method Wright function framework to combine and extend distribution families.
result Generalized -stable distribution with valid moments.
Evidence for the exponential distribution of income in the USAcond-mat.stat-mech
Using tax and census data, we demonstrate that the distribution of individual income in the USA is exponential. Our calculated Lorenz curve without fitting parameters and Gini coefficient 1/2 agree well with the data. From the individual income distribution, we derive the distribution function of income for families wi…
Extends likelihood ratio exponential families to analyze various optimization methods.
problem Analyzing optimization methods like rate-distortion and information bottleneck.
method Linking geometric mixture paths to exponential families and using hypothesis testing.
result Provides a common mathematical framework for understanding these methods.
Unified view on learning unnormalized distributions using NCE.
problem Learning unnormalized distributions across different communities.
method Noise-Contrastive Estimation (NCE) for a unified perspective.
result Established finite-sample convergence rates for exponential families.
Approximating exponential family models (not single distributions) with a two-network architecturestat.ML
Recently much attention has been paid to deep generative models, since they have been used to great success for variational inference, generation of complex data types, and more. In most all of these settings, the goal has been to find a particular member of that model family: optimized parameters index a distribution …
NatPN provides fast, accurate uncertainty estimation for exponential family distributions.
problem Uncertainty in machine learning models.
method NatPN uses Normalizing Flows to fit a single density in a latent space, updating predictions based on likelihood.
result NatPN delivers competitive performance in classification, regression, and count prediction tasks.
We provide a classification of graphical models according to their representation as subfamilies of exponential families. Undirected graphical models with no hidden variables are linear exponential families (LEFs), directed acyclic graphical models and chain graphs with no hidden variables, including Bayesian networks …
A statistical framework for removing unwanted data domains in machine learning.
problem Removing unwanted data domains in machine learning while preserving desired performance.
method Modeling domains as probability distributions and using hypothesis testing to select samples to remove.
result Characterization of allowable edited data distributions and removal-preservation Pareto frontiers for various distribution families.
Study slopes of direct images in complex manifolds, proving a Mehta-Ramanathan type theorem.
problem Distribution of Harder-Narasimhan slopes in direct image sheaves.
method Analyzing asymptotic distributions of slopes under base changes of families of complex projective manifolds.
result Asymptotic distribution of slopes can be recovered from base changes over generic curves.
Inference for normal and Monte Carlo distributions using minimum relative entropy.
problem Inference from partial information on expectations and covariances.
method Minimum relative entropy sub-manifolds, analytical formulas, Monte Carlo simulations.
result Improved numerical implementation for inference from partial information.
New approach to Generalized Beta family using SDEs.
problem Understanding the Generalized Beta family of distributions.
method Using a mean-reverting SDE for a power of the variable, leading to a modified GB distribution.
result Provides alternative forms and cumulative distribution functions for GB distributions.
In this paper, a Bayesian inference technique based on Taylor series approximation of the logarithm of the likelihood function is presented. The proposed approximation is devised for the case, where the prior distribution belongs to the exponential family of distributions. The logarithm of the likelihood function is li…
New method for robust PCA with exponential family distributions.
problem Recovering low-rank structure from data matrices with outliers.
method Alternating Direction Method of Multipliers for .
result Demonstrated effectiveness in steel sheet defect detection and crime activity monitoring.
Generative Adversarial Networks improve robust statistics for various distributions.
problem Estimating unknown parameters in adversarially corrupted samples.
method Designing GANs with specific loss functions for robust estimation.
result Extends robust estimation to broader families of distributions.
Many recent advances in large scale probabilistic inference rely on variational methods. The success of variational approaches depends on (i) formulating a flexible parametric family of distributions, and (ii) optimizing the parameters to find the member of this family that most closely approximates the exact posterior…
Proposes a general method to derive regret bounds for multi-armed bandit algorithms.
problem Deriving regret bounds for randomized multi-armed bandit algorithms.
method Checking sufficient conditions on sampling probabilities and distributions.
result Proves logarithmic regret bounds for various bandit algorithms and new models.
Variational inference (VI) provides fast approximations of a Bayesian posterior in part because it formulates posterior approximation as an optimization problem: to find the closest distribution to the exact posterior over some family of distributions. For practical reasons, the family of distributions in VI is usually…
This paper develops sparse alternatives to continuous distributions, including new types of Gaussians and attention mechanisms.
problem Creating flexible continuous distributions with varying support for machine learning applications.
method Defining -regularized prediction maps and Fenchel-Young losses for arbitrary domains, and deriving new types of Gaussians and attention mechanisms.
result Sparse alternatives to continuous distributions, including deformed exponential families and -Gaussians, are introduced.
Extends diffusion models to handle exponential family distributions for inverse problems.
problem Intractability of likelihood score for non-Gaussian observations.
method Evidence trick to approximate likelihood score for exponential family distributions.
result Effective Bayesian inference on complex Poisson processes and malaria prevalence prediction.
Undirected graphical models, or Markov networks, are a popular class of statistical models, used in a wide variety of applications. Popular instances of this class include Gaussian graphical models and Ising models. In many settings, however, it might not be clear which subclass of graphical models to use, particularly…
Thompson Sampling has been demonstrated in many complex bandit models, however the theoretical guarantees available for the parametric multi-armed bandit are still limited to the Bernoulli case. Here we extend them by proving asymptotic optimality of the algorithm using the Jeffreys prior for 1-dimensional exponential …
Study shows twist tori equidistribute in moduli space, with other families having singular distributions.
problem Statistical behavior of twist tori in moduli space of hyperbolic surfaces.
method Analyzing expanding families of twist tori and their limiting distributions.
result Equidistribution of twist tori to a Lebesgue measure, with other families having singular distributions.
In this paper we introduce a new family of operator-valued distributions on Euclidian space acting by convolution on differential forms. It provides a natural generalization of the important Riesz distributions acting on functions, where the corresponding operators are , and we develop basic analogous prop…
Flexible mean field variational inference using mixtures of non-overlapping exponential familiesmath.ST
Improves variational inference for sparse models using mixtures of exponential families.
problem Intractability of posterior distributions in Bayesian sparse models.
method Flexible mean field variational inference using mixtures of non-overlapping exponential families.
result Mixtures of exponential families with non-overlapping support form an exponential family, enabling analytical updates.
Semi-implicit variational inference (SIVI) is introduced to expand the commonly used analytic variational distribution family, by mixing the variational parameter with a flexible distribution. This mixing distribution can assume any density function, explicit or not, as long as independent random samples can be generat…
New normalizing flows for sphere distributions improve complexity and scale handling.
problem No straightforward normalizing flows for Fisher-Bingham distributions in higher dimensions.
method Zoom-linear-project (ZLP)-Fisher flows that gradually add complexity and handle varying scales.
result Generalizes Fisher-Bingham distributions to normalizing flows in any dimension.
The mean field methods, which entail approximating intractable probability distributions variationally with distributions from a tractable family, enjoy high efficiency, guaranteed convergence, and provide lower bounds on the true likelihood. But due to requirement for model-specific derivation of the optimization equa…
This note investigates a conjugate class for the Dirichlet distribution class in the exponential family.
Generic Unsupervised Optimization for a Latent Variable Model With Exponential Family Observablescs.LG
New LVMs optimize any exponential family distribution without specific assumptions.
problem Optimizing latent variable models with non-Gaussian observables.
method Generic optimization using EM approach for exponential family distributions.
result Concise parameter update equations applicable to various data types.
A Sufficient Statistics Construction of Bayesian Nonparametric Exponential Family Conjugate Modelscs.LG
Conjugate pairs of distributions over infinite dimensional spaces are prominent in statistical learning theory, particularly due to the widespread adoption of Bayesian nonparametric methodologies for a host of models and applications. Much of the existing literature in the learning community focuses on processes posses…
We consider the design of prediction market mechanisms known as automated market makers. We show that we can design these mechanisms via the mold of \emph{exponential family distributions}, a popular and well-studied probability distribution template used in statistics. We give a full development of this relationship a…
All you need is logcs.IT
A new family of multi-distribution divergences is characterized for fairness and other problems.
problem Comparing more than two probability distributions in a consistent way.
method Characterized a new family of multi-way coincidence divergences.
result The new family of multi-distribution Rényi divergences is necessary and arises from several independent routes.
A subbundle of variable dimension inside the tangent bundle of a smooth manifold is called a smooth distribution if it is the pointwise span of a family of smooth vector fields. We prove that all such distributions are finitely generated, meaning that the family may be taken to be a finite collection. Further, we show …
Optimal learning for parametric prophet inequalities with exponential-type distributions
problem Learning in prophet inequalities with unknown parameters
method Confidence-based dynamic-programming policy
result Achieves optimal asymptotic competitive ratio using online observations
Geometric Gaussian approximations capture any distribution.
problem Approximating complex probability distributions.
method Geometric Gaussian approximations through diffeomorphisms or exponential maps.
result Geometric Gaussian approximations are universal, capturing any distribution.
Efficiently learns exponential family distributions with i.i.d. samples.
problem Learning natural parameters of truncated exponential families efficiently.
method Proposes a novel loss function and computationally efficient estimator.
result Achieves optimal sample complexity and asymptotic normality.
We study the problem of finding the smallest such that every element of an exponential family can be written as a mixture of elements of another exponential family. We propose an approach based on coverings and packings of the face lattice of the corresponding convex support polytopes and results from coding th…
New tree-structured Markov fields with Poisson marginals for counting variables.
problem Counting variables with complex dependencies.
method Tree-structured Markov random fields with Poisson marginals.
result Straightforward sampling and joint probability calculations.
New tractable density models from squaring neural networks.
problem Flexible models for probability distributions in machine learning.
method Squared Neural Family (SNEFY) models formed by squaring neural network outputs and normalizing.
result SNEFYs are fully tractable with closed form normalizing constants in many cases.
Proves that emergent algebras right-distributivity implies left-distributivity.
problem Proving the implication between emergent algebra distributivity conditions.
method Analyzing families of quasigroup operations indexed by commutative groups.
result Emergent algebras right-distributive imply left-distributive.
Efficient EP algorithm improves smoothing distribution inference in financial models.
problem Computational intractability of smoothing distribution in high dimensions.
method Adapted expectation propagation (EP) algorithms for the unified skew-normal family.
result Accuracy gains in financial illustrations over existing approximate algorithms.
We consider the problem of estimating undirected triangle-free graphs of high dimensional distributions. Triangle-free graphs form a rich graph family which allows arbitrary loopy structures but 3-cliques. For inferential tractability, we propose a graphical Fermat's principle to regularize the distribution family. Suc…
Information-Geometric Optimization (IGO) is a unified framework of stochastic algorithms for optimization problems. Given a family of probability distributions, IGO turns the original optimization problem into a new maximization problem on the parameter space of the probability distributions. IGO updates the parameter …