We derive and approximate the conjugate prior of Dirichlet and beta distributions.
problem Intractability of conjugate prior for Dirichlet and beta distributions.
method Derive conjugate prior, define closed-form approximation, and provide algorithm.
result Closed-form approximation enables fully tractable Bayesian treatment.
This note investigates a conjugate class for the Dirichlet distribution class in the exponential family.
Characterizes connections on multivariate normal distributions.
problem Characterizing connections on statistical manifold of multivariate normal distributions.
method Analyzes statistical manifold (N,gF,ablaA,ablaA∗) of multivariate normal distributions. result The Amari-Chentsov connection ablaA is characterized by conjugate symmetry. New connections found on zero-mean multivariate normal distributions.
problem Characterizing statistical connections on zero-mean multivariate normal distributions.
method Investigating invariant conjugate symmetric statistical connections on the submanifold of zero-mean multivariate normal distributions.
result Invariant connections on zero-mean multivariate normal distributions are not uniquely characterized by invariance under the general linear group action.
The paper deals with learning probability distributions of observed data by artificial neural networks. We suggest a so-called gradient conjugate prior (GCP) update appropriate for neural networks, which is a modification of the classical Bayesian update for conjugate priors. We establish a connection between the gradi…
Conjugate pairs of distributions over infinite dimensional spaces are prominent in statistical learning theory, particularly due to the widespread adoption of Bayesian nonparametric methodologies for a host of models and applications. Much of the existing literature in the learning community focuses on processes posses…
Study normal operators of double fibration transforms with conjugate points.
problem Normal operators of double fibration transforms with conjugate points.
method Stable conditions on the distribution of conjugate points, splitting into elliptic and Fourier integral operators.
result Normal operator splits into an elliptic pseudodifferential operator and Fourier integral operators.
We characterize conjugate nonparametric Bayesian models as projective limits of conjugate, finite-dimensional Bayesian models. In particular, we identify a large class of nonparametric models representable as infinite-dimensional analogues of exponential family distributions and their canonical conjugate priors. This c…
Study heavy-tailed weights' impact on neural network's spectral distribution.
problem Analyzing spectral distribution of conjugate kernel matrices with heavy-tailed weights.
method Computed limiting eigenvalue distribution through moments, considering heavy-tailed distributions and nonlinear activation functions.
result Heavy-tailed weights induce strong correlations, leading to fundamentally different spectral behavior.
Properties of pairs of product conjugate connections are stated with a special view towards the integrability of the given almost product structure. We define the analogous in product geometry of the structural and the virtual tensors from the Hermitian geometry and express the product conjugate connections in terms of…
Making inferences from data streams is a pervasive problem in many modern data analysis applications. But it requires to address the problem of continuous model updating and adapt to changes or drifts in the underlying data generating distribution. In this paper, we approach these problems from a Bayesian perspective c…
Develops multi-modal neural network models for improved prediction and uncertainty quantification.
problem Improving prediction accuracy and uncertainty quantification for multi-modal data.
method Multi-modal Bayesian neural network models with conjugate last-layer estimation using SVI.
result Improved prediction accuracy and uncertainty quantification compared to uni-modal models.
Helfer in [Pacific J. Math. 164/2 (1994), p. 321--350] was the first to produce an example of a spacelike Lorentzian geodesic with a continuum of conjugate points. In this paper we show the following result: given an interval [a,b] of IR and any closed subset F of IR contained in ]a,b], then there exists a Lo…
New method uses KL-divergence to create non-informative priors for multivariate Gaussian.
problem Handling hyperparameters for non-informative limits in multivariate Gaussian conjugate priors.
method Using scaled KL-divergence between multivariate Gaussians to construct Wishart and normal-Wishart conjugate priors.
result Forming non-informative priors without violating Wishart shape parameter restrictions.
We develop methods for efficient amortized approximate Bayesian inference over posterior distributions of probabilistic clustering models, such as Dirichlet process mixture models. The approach is based on mapping distributed, symmetry-invariant representations of cluster arrangements into conditional probabilities. Th…
New conjugate priors improve Bayesian inference for multinomial probit models.
problem Lack of tractable conjugate priors for efficient Bayesian inference in multinomial probit models.
method Unified skew-normal (SUN) distributions as conjugate priors, leading to improved posterior inference and classification.
result Improved computational methods for posterior inference and classification, especially in high dimensions.
Thompson sampling has impressive empirical performance for many multi-armed bandit problems. But current algorithms for Thompson sampling only work for the case of conjugate priors since these algorithms require to infer the posterior, which is often computationally intractable when the prior is not conjugate. In this …
Alternative approach to generative modeling using convex conjugates and optimal transport.
problem Traditional generative modeling splits sampling and mapping; this work explores an alternative.
method Inspired by moment measures, proposes a new factorization and uses optimal transport for recovery.
result Intuitive results on factorized distributions, showing potential for practical tasks.
Study shows deterministic equivalent for neural network kernel convergence.
problem Understanding convergence of neural network kernels.
method Analyzes empirical spectral distribution of Conjugate Kernel, proving convergence to a deterministic limit.
result Obtains a deterministic equivalent for the Stieltjes transform and resolvent of the Conjugate Kernel.
Paper introduces robust Gaussian process regression without sacrificing computational efficiency.
problem Violation of independent and identically distributed Gaussian observation noise assumption in Gaussian process regression.
method Proves robust and conjugate Gaussian process regression (RCGP) at no additional cost using generalised Bayesian inference.
result RCGP enables exact conjugate closed form updates in all settings where standard GPs admit them.
We propose a novel Bayesian approach to solve stochastic optimization problems that involve finding extrema of noisy, nonlinear functions. Previous work has focused on representing possible functions explicitly, which leads to a two-step procedure of first, doing inference over the function space and second, finding th…
Nonlinear conjugate gradient (NLCG) based optimizers have shown superior loss convergence properties compared to gradient descent based optimizers for traditional optimization problems. However, in Deep Neural Network (DNN) training, the dominant optimization algorithm of choice is still Stochastic Gradient Descent (SG…
The paper classifies 3D contact partially hyperbolic diffeomorphisms.
problem Classifying contact partially hyperbolic diffeomorphisms in 3D.
method Smooth classification, conjugation to known flows or automorphisms, use of invariant distributions.
result Classification up to finite quotient or power, conjugation to known structures.
Applied Data Scientists throughout various industries are commonly faced with the challenging task of encoding high-cardinality categorical features into digestible inputs for machine learning algorithms. This paper describes a Bayesian encoding technique developed for WeWork's lead scoring engine which outputs the pro…
This paper analyzes M-estimators under infinite-variance noise in high dimensions.
problem High-dimensional M-estimation with infinite-variance noise.
method Study of the Fenchel conjugate domain and its impact on risk.
result Exact risk of M-estimators under infinite-variance noise is derived.
Study geodesics and shortest arcs on Lie groups with specific metrics.
problem Characterize geodesics and shortest arcs in sub-Riemannian metrics on Lie groups.
method Investigated left-invariant sub-Riemannian metrics on SU(1,1)imesR and SO0(2,1)imesR. result Found geodesics, shortest arcs, cut loci, and conjugate loci.
CEBMs learn flexible latent mappings from data.
problem Learning flexible latent mappings from data.
method CEBMs decompose joint density into tractable posterior over latent variables.
result CEBMs achieve competitive results in image modeling and latent space predictive power.
We study the sub-Riemannian exponential for contact distributions on manifolds of dimension greater or equal to 5. We compute an approximation of the sub-Riemannian Hamiltonian flow and show that the conjugate time can have multiplicity 2 in this case. We obtain an approximation of the first conjugate locus for small r…
The paper introduces a new framework to understand and optimize deep neural networks.
problem Understanding and optimizing the trainability and generalization of deep neural networks.
method Developed a conjugate learning theoretical framework based on convex conjugate duality.
result Demonstrated that training deep neural networks with SGD achieves global optima of empirical risk.
We construct non-trapping asymptotically hyperbolic manifolds with boundary conjugate points but no interior conjugate points.
A condition for a statistical manifold to have an equiaffine structure is studied. The facts that dual flatness and conjugate symmetry of a statistical manifold are sufficient conditions for a statistical manifold to have an equiaffine structure were obtained in [2] and [3]. In this paper, a fact that a statistical man…
The paper tackles efficient computation of optimal transport by approximating conjugates with amortized optimization.
problem Efficient computation of convex conjugates in optimal transport is challenging and limits the quality of transport maps.
method The approach combines amortized approximations of conjugates with a fine-tuning solver to improve transport map quality.
result The method significantly improves the quality of transport maps for the Wasserstein-2 benchmark and models many 2D couplings and flows.
Estimates geodesics on surfaces without conjugate points.
problem Counting geodesics on surfaces without conjugate points.
method Margulis-type asymptotic estimates.
result Asymptotic estimates for geodesics on surfaces.
New method speeds up inference for non-conjugate Gaussian processes.
problem Inference for non-conjugate Gaussian processes is slow and unreliable.
method Automated augmented conjugate inference method that constructs auxiliary variables to make the model conditionally conjugate.
result Our method is up to two orders of magnitude faster and more robust than existing methods.
We introduce a multiple conjugation biquandle, and show that it is the universal algebra to define a semi-arc coloring invariant for handlebody-links. A multiple conjugation biquandle is a generalization of a multiple conjugation quandle. We extend the notion of n-parallel biquandle operations for any integer n, an…
New method handles large reward variations in reinforcement learning.
problem Optimal policy not achievable with existing methods for non-deterministic processes.
method Introduces conjugated distributional operator for handling real returns.
result Guaranteed theoretical convergence for a wide class of transformations.
The paper studies Jacobi fields and conjugate points in projective sprays.
problem Investigating Jacobi fields and conjugate points in projective sprays.
method Proved that conjugate points are preserved under projective changes and established conditions for the existence of conjugate points.
result Conditions for the existence of conjugate points in projectively deformed sprays.
Paper describes how to extend multiple conjugation quandles using maps.
problem Understanding affine extensions of multiple conjugation quandles.
method Introduces augmented MCQ Alexander pairs for affine extensions.
result Affine extensions of multiple conjugation quandles can be described by quadruples of maps.
We analyze a new robust method for the reconstruction of probability distributions of observed data in the presence of output outliers. It is based on a so-called gradient conjugate prior (GCP) network which outputs the parameters of a prior. By rigorously studying the dynamics of the GCP learning process, we derive an…
Study eigenvalue distributions of neural kernels for linear-width networks.
problem Eigenvalue distributions of neural kernels in linear-width networks.
method Asymptotic analysis of Conjugate Kernel and Neural Tangent Kernel under random initialization and approximate orthogonality.
result Eigenvalue distributions converge to deterministic limits, described by recursive fixed-point equations.
For a surface group, a new bound is given for conjugator length function.
problem Finding an explicit bound for the conjugator length function of a surface group.
method Detailed analysis of conjugation reductions.
result An explicit bound n−1≤CL(2n)≤n+8g−1 for the conjugator length function of a surface group. The study examines Hopfian properties of conjugation quandles and their underlying groups.
problem Understanding the relationship between Hopfian properties of conjugation quandles and their underlying groups.
method Examined Hopfian and residual finiteness properties of conjugation quandles of specific groups.
result Conjugation quandles of Baumslag-Solitar groups are infinitely generated and not necessarily Hopfian.
Upper bound for conjugate radius in open manifolds with scalar curvature and spectrum constraints.
problem Bounding the conjugate radius of open manifolds with specific curvature and spectrum conditions.
method Established an upper bound using scalar curvature and bottom-of-spectrum constraints.
result For certain conditions, the conjugate radius is no more than π.
We consider actions of Z^k, k \ge 2, by Anosov diffeomorphisms which are uniformly quasiconformal on each coarse Lyapunov distribution. These actions generalize Cartan actions for which coarse Lyapunov distributions are one-dimensional. We show that, under certain non-resonance assumptions on the Lyapunov exponents, a …
The questions when two Morse function on closed manifolds are conjugated is investigated. Using the handle decompositions of manifolds the condition of conjugation is formulated. For each Morse function on 3-manifold the ordered generalized Heegaard diagram is built. The criteria of Morse function conjugation are given…
Study conjugate locus in convex 3-manifolds using Jacobi fields.
problem Understanding conjugate points in convex 3-manifolds.
method Use Jacobi fields to define coordinate systems and classify singularities.
result Developed a novel method for determining conjugate points in 3D manifolds.
Unified Kantorovich duality for multimarginal optimal transport on Polish spaces.
problem Optimal transport of multiple probability distributions.
method Unified Kantorovich duality theory for multimarginal optimal transport on general Polish product spaces.
result Unified duality theory for multimarginal optimal transport, extending classical two-marginal conjugacy.
Unified Skew-Gaussian process framework for various regression and classification tasks.
problem Handling multiple types of regression and classification problems.
method Generalization of Skew-Gaussian processes to handle various types of data and likelihoods.
result Closed-form posterior distributions for multiple tasks.