The paper introduces a new method for graph embedding using exponential family distributions.
problem Representing networks in a low dimensional latent space for various applications.
method Introduces the exponential family graph embedding model, generalizing random walk-based techniques to exponential family conditional distributions.
result The proposed techniques outperform existing methods in link prediction and node classification tasks.
Estimates exponential family distributions using a novel doubly dual embedding technique.
problem Estimating exponential family distributions with smoothness and efficiency.
method Doubly dual embedding for avoiding partition function computation and flexible sampling.
result Improves memory and time efficiency while offering stronger statistical properties.
Paper explores duality in DPPs using embedding structure analysis.
problem Understanding the geometric structure of determinantal point processes.
method Analyzes the exponential family embedding of DPPs and uses the e-embedding curvature tensor.
result Discovers the duality between marginal and L-ensemble kernels.
Dynamic embeddings capture evolving word meanings over time.
problem Capturing how word meanings change over time in historical texts.
method Developed dynamic Bernoulli embeddings based on exponential family embeddings.
result Dynamic embeddings provide better fits and reveal interesting language change patterns.
Paper proves one observation suffices for Gaussian embedding.
problem Proving theoretical underpinnings for Gaussian embedding.
method Developed first theoretical results for exponential family embedding models.
result Learned embedding structure from one observation.
Word embeddings are a powerful approach for capturing semantic similarity among terms in a vocabulary. In this paper, we develop exponential family embeddings, a class of methods that extends the idea of word embeddings to other types of high-dimensional data. As examples, we studied neural data with real-valued observ…
Paper introduces Bayesian EEF for model order selection using exponentially embedded family.
problem Model order selection in Bayesian statistics.
method Bayesian EEF method using exponentially embedded family.
result Bayesian EEF can use vague priors and reveals EEF mechanism for model selection.
Adversarial dynamics embedding improves MLE of exponential family models.
problem Maximum likelihood estimation of exponential family models with neural network parametrization.
method Adversarial dynamics embedding to estimate the dual sampler and primal model simultaneously.
result Adversarial dynamics embedding leads to more effective learning and improved estimators compared to existing methods.
The paper introduces natural α-embeddings for item representations.
problem Overcoming computational limitations in item encoding.
method Interpreting item embeddings in an Information Geometric framework using α-geometry of exponential families.
result A family of natural α-embeddings in the tangent space of the probability simplex.
Develops S-EFE for analyzing grouped data, improving word usage interpretation.
problem Analyzing how words are used differently across related groups of data.
method Structured exponential family embeddings (S-EFE) with hierarchical modeling and amortization.
result S-EFE enables group-specific interpretation of word usage and outperforms EFE.
A tractable pseudo-metric for non-parametric distributions via SPD geometry.
problem Computing distances between non-parametric probability distributions is intractable.
method Two-stage framework: projection onto parametric family, embedding into SPD matrices.
result Closed-form pseudo-metric for two-sample hypothesis testing.
In this letter, we present a novel exponentially embedded families (EEF) based classification method, in which the probability density function (PDF) on raw data is estimated from the PDF on features. With the PDF construction, we show that class-specific features can be used in the proposed classification method, inst…
New method models aptamer libraries as Boltzmann-weighted graph ensembles for better affinity predictions.
problem Anomalous candidates in SELEX datasets obscure true aptamer-ligand affinity.
method Boltzmann graph ensemble embeddings for thermodynamically parameterized exponential-family random graphs.
result Proposed embedding enables robust community detection and subgraph-level explanations for aptamer ligand affinity.
A simple framework predicts unseen classes using exponential family distributions.
problem Learning to predict previously unseen classes.
method Estimating class-attribute-gated class-conditional distributions modeled as exponential families.
result Natural representation of classes as probability distributions, leveraging unlabeled data.
Paper introduces kernel deformed exponential families for sparse continuous attention.
problem Creating efficient attention mechanisms for sparse data.
method Developed kernel deformed exponential families, theoretically and experimentally.
result Kernel deformed exponential families can attend to multiple compact regions of data.
We introduce a new distance metric for non-linear embeddings of Tempered Exponential Measures.
problem Non-linear embeddings of Tempered Exponential Measures (TEMs).
method Parameterization of finite discrete TEMs via Legendre functions, introducing tempered Hilbert co-simplex distance.
result Established a generalization of the Hilbert log cross-ratio simplex distance to a tempered Hilbert co-simplex distance.
We consider three different approaches to define natural Riemannian metrics on polytopes of stochastic matrices. First, we define a natural class of stochastic maps between these polytopes and give a metric characterization of Chentsov type in terms of invariance with respect to these maps. Second, we consider the Fish…
Paper develops EP algorithm for t-exponential family using q-algebra.
problem Efficient learning algorithm for t-exponential family distributions.
method Borrowing q-algebra from statistical physics, develop EP algorithm.
result Demonstrates performance of EP algorithm on Bayes point machine and Student-t process classification.
The study explores generalized divergences and exponential families with a focus on sufficient conditions and laws of large numbers.
problem Generalization of Kullback-Leibler divergence and exponential families.
method Investigation of (h,τ)-divergence and (h,τ)-exponential families, definition of (h,τ)-dependence, proof of law of large numbers. result Sufficient condition for (h,τ)-divergence to induce Hessian structure on (h,τ)-exponential family, proof of law of large numbers. Generalizes moment-matching for exponential families with conditioning or hidden data.
problem Generalizing moment-matching conditions for exponential families with conditioning or hidden data.
method First-principles explanation and self-contained derivation of generalized moment-matching conditions.
result Derives generalized moment-matching conditions for conditional exponential families and hidden data.
The paper studies families of curves on surfaces that realize all types of pants decompositions.
problem Finding the minimal size of families of curves on surfaces that realize all types of pants decompositions.
method Investigates exponential and superlinear bounds for surfaces without punctures, and provides bounds for surfaces with punctures.
result Provides bounds for the minimal size of families of curves on surfaces with and without punctures.
Correspondence found between exponential families and affine Grassmannians.
problem Understanding the relationship between exponential families and geometric structures.
method Established a one-to-one correspondence between exponential families and affine Grassmannians.
result Found a correspondence between minimal exponential families and affine Grassmannians.
EFA extends self-attention to handle mixed data types and dynamic relevance.
problem Handling high-dimensional, mixed data types with dynamic relevance.
method Probabilistic generative model using self-attention and latent factor model.
result EFA consistently outperforms existing models in complex latent structure capture and reconstruction.
Disk Embeddings tackle embedding DAGs with exponential growth.
problem Embedding DAGs with exponentially increasing ancestors and descendants.
method Disk Embeddings framework for quasi-metric spaces, including Hyperbolic Disk Embeddings.
result Disk Embeddings outperform existing methods in complex DAGs.
Constructing exponential families from statistical manifolds.
problem The central problem of constructing exponential families from statistical manifolds.
method Constructive approach proving every compact statistical manifold admits a foliation of Hessian manifolds.
result Compact orientable leaves are either finite quotients of flat torus or mapping torus with periodic monodromy.
We provide a classification of graphical models according to their representation as subfamilies of exponential families. Undirected graphical models with no hidden variables are linear exponential families (LEFs), directed acyclic graphical models and chain graphs with no hidden variables, including Bayesian networks …
Several problems such as network intrusion, community detection, and disease outbreak can be described by observations attributed to nodes or edges of a graph. In these applications presence of intrusion, community or disease outbreak is characterized by novel observations on some unknown connected subgraph. These prob…
Thompson Sampling has been demonstrated in many complex bandit models, however the theoretical guarantees available for the parametric multi-armed bandit are still limited to the Bernoulli case. Here we extend them by proving asymptotic optimality of the algorithm using the Jeffreys prior for 1-dimensional exponential …
New Thompson sampling algorithm reduces regret for exponential family bandits.
problem Minimizing regret in multi-armed bandit problems with exponential family rewards.
method Proposes ExpTS and ExpTS+ algorithms using novel sampling distributions. result Minimizes both finite-time and asymptotic regret for exponential family rewards.
Moment polytope of toric exponential families is a projection of a simplex.
problem Understanding the geometry of exponential families in finite sample spaces.
method Toric torification and projection of higher-dimensional simplices.
result Moment polytope is a projection of a higher-dimensional simplex.
We study online learning under logarithmic loss with regular parametric models. Hedayati and Bartlett (2012b) showed that a Bayesian prediction strategy with Jeffreys prior and sequential normalized maximum likelihood (SNML) coincide and are optimal if and only if the latter is exchangeable, and if and only if the opti…
We propose a novel approach for density estimation with exponential families for the case when the true density may not fall within the chosen family. Our approach augments the sufficient statistics with features designed to accumulate probability mass in the neighborhood of the observed points, resulting in a non-para…
New insights into natural exponential families improve regret bounds for bandit problems.
problem Improving regret bounds for bandit problems with subexponential tails.
method Proving self-concordance for natural exponential families and applying to bandits.
result Optimistic algorithms for generalized linear bandits have second-order regret bounds that are free of an exponential dependence on problem parameters.
The versatility of exponential families, along with their attendant convexity properties, make them a popular and effective statistical model. A central issue is learning these models in high-dimensions, such as when there is some sparsity pattern of the optimal parameter. This work characterizes a certain strong conve…
Classifies 1D exponential families with constant Hessian curvature.
problem Classifying 1D exponential families with constant Hessian curvature.
method Complete classification through mathematical analysis.
result If an exponential family has constant Hessian curvature, it must have curvature λ=k2 for some integer k≤m. Extends likelihood ratio exponential families to analyze various optimization methods.
problem Analyzing optimization methods like rate-distortion and information bottleneck.
method Linking geometric mixture paths to exponential families and using hypothesis testing.
result Provides a common mathematical framework for understanding these methods.
Chentsov's theorem proved for exponential families.
problem Characterizing the Fisher information metric in exponential families.
method Unified proof using the central limit theorem.
result Fisher information metric is the only Riemannian metric invariant under extensions and sufficient statistics.
A two-network architecture learns intractable exponential family models.
problem Learning a model itself, not just optimizing parameters of a single distribution.
method Two-network architecture and optimization procedure for exponential family models.
result Accurately learns exponential family models, enabling generic operations.
Efficient method for learning continuous exponential families beyond Gaussian.
problem Learning continuous exponential families with unbounded support.
method Interaction Screening approach for scalable learning of continuous graphical models.
result Our estimator maintains similar accuracy and sample complexity scalings compared to alternative approaches, while improving run-time.
EFDA extends LDA to non-Gaussian models using exponential families.
problem Classifying non-Gaussian data with LDA's limitations.
method EFDA uses exponential families to derive closed-form estimators for natural parameters and a linear decision rule.
result EFDA matches LDA's accuracy while reducing ECE by 2-6x, proving asymptotic calibration and efficiency.
New method constructs exponential families from representation theory.
problem Constructing and understanding exponential families from representations.
method Using representation theory to construct exponential families on homogeneous spaces.
result The correspondence θ → p_θ is injective under certain conditions.
Study cohomology of curve moduli spaces, finding new nonvanishing groups.
problem Computing cohomology of moduli spaces of curves.
method Graph complexes related to embedding spaces.
result New infinite families of nonvanishing unstable cohomology groups on Mg. New bounds for score matching in polynomial exponential families.
problem Understanding the sample complexity of score matching for polynomial exponential families.
method Non-asymptotic sample complexity analysis for score matching.
result First finite sample bounds for score matching in polynomial exponential families.
Exponential family extensions of principal component analysis (EPCA) have received a considerable amount of attention in recent years, demonstrating the growing need for basic modeling tools that do not assume the squared loss or Gaussian distribution. We extend the EPCA model toolbox by presenting the first exponentia…
Given a simplicial complex K, we consider several notions of geometric complexity of embeddings of K in a Euclidean space Rd: thickness, distortion, and refinement complexity (the minimal number of simplices needed for a PL embedding). We show that any n-complex with N simplices which topologically…
New hyperbolic manifolds show exponential homology torsion growth.
problem Understanding growth of torsion in homology groups of hyperbolic manifolds.
method Constructed a family of hyperbolic manifolds with specific growth properties.
result Demonstrated that recent bound on homological torsion is asymptotically sharp.
Improves variational inference for sparse models using mixtures of exponential families.
problem Intractability of posterior distributions in Bayesian sparse models.
method Flexible mean field variational inference using mixtures of non-overlapping exponential families.
result Mixtures of exponential families with non-overlapping support form an exponential family, enabling analytical updates.
The paper classifies statistical Einstein manifolds in exponential families.
problem Classifying statistical Einstein manifolds in exponential families.
method Deriving partial differential equations for potential functions, obtaining special and group-invariant solutions.
result Special and group-invariant solutions of the equations for potential functions of exponential families.