Novel variational EM approach using truncated distributions.
problem Efficiently modeling latent variables and mixtures.
method Truncated posterior distributions as variational parameters.
result Efficient and concise variational lower bounds.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,291 papers · 148 categories
Trend · papers per month
48 results for “Truncated Variational EM”
We study inference and learning based on a sparse coding model with `spike-and-slab' prior. As in standard sparse coding, the model used assumes independent latent sources that linearly combine to generate data points. However, instead of using a standard sparse prior such as a Laplace distribution, we study the applic…
Improved semi-supervised learning for large networks using TV-EM and Neural Simpletrons.
problem Challenges in inference and learning for large-scale generative networks.
method Combining Neural Simpletrons with TV-EM for efficient, scalable learning.
result Significant improvements in learning efficiency and performance on semi-supervised tasks.
k-means derived from Gaussian mixture models with isotropic Gaussians.
problem Clustering with Gaussian mixture models.
method Truncated variational EM approximations applied to Gaussian Mixture Models.
result k-means is a special case of variational EM for Gaussian Mixture Models.
EM algorithm converges to true mean for truncated mixtures of Gaussians.
problem Analyzing EM algorithm for truncated mixtures of two Gaussians.
method Using dynamical systems, probability, and statistics techniques.
result EM converges almost surely to true mean for various measurable sets S.
Can clustering scale sublinearly with its clusters? A variational EM acceleration of GMMs and -meansstat.ML
Study shows -means and GMM can scale sublinearly with clusters using variational EM.
problem Improving clustering efficiency with sublinear complexity.
method Applying variational EM to reduce complexity of -means and GMM iterations.
result Reduces computational complexity from linear to sublinear with respect to the number of clusters.
Novel variational sampling improves generative model optimization.
problem Optimizing binary latent variable generative models efficiently.
method Truncated variational EM with efficient sampling.
result Efficiently increases variational free energy objective.
Numerical Method for Highly Non-linear Mean-reverting Asset Price Model with CEV-type Processq-fin.RM
New numerical method for non-linear asset price model with CEV volatility.
problem Describing stochastic volatility in asset price dynamics.
method Proposes a mean-reverting theta-rho model with CEV volatility, constructs a truncated EM method.
result Truncated EM solutions can evaluate path-dependent financial products.
In the paper "On Truncated Variation of Brownian Motion with Drift" (Bull. Pol. Acad. Sci. Math. 56 (2008), no.4, 267 - 281) we defined truncated variation of Brownian motion with drift, where is a standard Brownian motion. Truncated variation differs from regular variation by neglect…
This paper presents a fast algorithm for clustering large datasets using Gaussian mixture models.
problem Efficiently clustering large-scale datasets with Gaussian mixture models.
method Variational EM algorithm with coreset objectives for sublinear complexity.
result Substantial speedups in clustering large-scale datasets (up to 32,000 clusters on 80 Million Tiny Images).
Paper develops IFTRR to solve sparse generalized eigenvalue problems efficiently.
problem Finding the leading eigenvector with at most k nonzero entries in sparse generalized eigenvalue problems.
method Inverse-free truncated Rayleigh-Ritz method (IFTRR) with a new truncation strategy.
result IFTRR efficiently finds the support set of the leading eigenvector for large scale problems.
A new method for DPMM inference without truncation.
problem Inference in DPMM requires truncation, limiting model flexibility.
method Hybrid inference combining MCMC and variational methods.
result Efficient variational updates without increased model complexity.
Global Generalized Bianchi Identities for Invariant Variational Problems on Gauge-natural Bundlesmath-ph
We derive both {\em local} and {\em global} generalized {\em Bianchi identities} for classical Lagrangian field theories on gauge-natural bundles. We show that globally defined generalized Bianchi identities can be found without the {\em a priori} introduction of a connection. The proof is based on a {\em global} decom…
In a previous analysis the problem of "zero-inflated" time data (caused by high frequency trading in the electronic order book) was handled by left-truncating the inter-arrival times. We demonstrated, using rigorous statistical methods, that the Weibull distribution describes the corresponding stochastic dynamics for a…
A new method for multi-objective Bayesian optimization using entropy search and variational lower bound maximization.
problem Efficiently optimizing multiple objectives in continuous domains.
method Approximates the Pareto-frontier using a mixture distribution and optimizes the balance through variational lower bound maximization.
result Demonstrated effectiveness especially with many objective functions.
Fast algorithms for learning a Gaussian under halfspace truncation with optimal sample complexitycs.DS
Optimal algorithm learns Gaussian under halfspace truncation with minimal samples.
problem Learning a Gaussian distribution truncated to an unknown halfspace.
method Efficient algorithm using samples and runtime dominated by empirical covariance matrix computation.
result Optimal sample and time complexity bounds for learning a Gaussian under halfspace truncation.
CATVI improves variational inference for Bayesian nonparametric models by reducing divergence and improving prediction accuracy.
problem Limitations of current variational inference methods in characterizing latent correlations and inferring true posterior dimensions.
method CATVI integrates conditional and adaptive truncation into variational inference, maximizing nonparametric evidence lower bound and using Monte Carlo integration.
result CATVI outperforms traditional methods in Bayesian nonparametric topic models, reducing perplexity and improving topic-word clustering.
UDN adapts depth to data complexity, outperforming standard neural networks.
problem Adapting neural network depth to data complexity.
method Variational inference for infinitely deep neural networks with a novel algorithm.
result UDN outperforms standard neural networks and other infinite-depth approaches.
Clarifies EM algorithm and variational Bayesian inference concepts.
problem Gaps in AI literature understanding of EM and variational concepts.
method Tutorial presentation of EM algorithm, variational Bayesian inference, and autoencoded variational Bayes.
result Establishes clear links between EM and variational methods.
Bayesian nonparametric Hawkes process model with EM-variational inference.
problem Limited model flexibility in classical Hawkes processes.
method Gaussian process modulated Hawkes process with EM-variational inference.
result Recover underlying baseline intensity and triggering kernel without parametric restriction.
Algorithm estimates multivariate normal parameters from hidden truncated samples.
problem Estimating parameters of multivariate normal distribution from hidden truncated samples.
method Efficient algorithm using polynomial-time estimation with oracle access to the subset S.
result Estimation of mean and covariance matrix with arbitrary accuracy.
Study proves convergence of interest rate model approximations.
problem Investigating convergence of stochastic interest rate models.
method Developed analytical tools for true and truncated EM solutions, proving convergence in probability.
result True solution converges in probability to truncated EM solution as step size approaches zero.
Nonconvex Low-Rank Matrix Recovery with Arbitrary Outliers via Median-Truncated Gradient Descentcs.IT
Improved robustness of gradient descent for low-rank matrix recovery in the presence of arbitrary outliers.
problem Gradient descent's sensitivity to outliers in low-rank matrix recovery.
method Truncated gradient descent with adaptive median truncation.
result Converges to ground truth at a linear rate with near-optimal number of measurements, even with constant fraction of arbitrarily corrupted measurements.
New method estimates volatility for processes with jumps of unbounded variation.
problem Estimating volatility of processes with jumps of unbounded variation.
method Developed a new volatility estimator using debiasing of truncated realized quadratic variation.
result Method outperforms existing alternatives in simulations.
In the context of the Dragulescu-Yakovenko (2000) model, we show that empirical income distribution with truncated datasets, cannot be properly modeled by the one-parameter exponential distribution. However, a truncated version characterized by an exponential distribution with two parameters gives an accurate fit.
In the classical Lagrangian approach to conservation laws of gauge-natural field theories a suitable (vector) density is known to generate the so--called {\em conserved Noether currents}. It turns out that along any section of the relevant gauge--natural bundle this density is the divergence of a skew--symmetric (tenso…
The paper interprets VQ-VAE loss as a form of information bottleneck.
problem Understanding the VQ-VAE loss function.
method Interpreted VQ-VAE loss as variational deterministic information bottleneck (VDIB) and variational information bottleneck (VIB).
result VQ-VAE loss can be derived from VDIB and approximated by VIB.
Improved EM algorithm for faster convergence of mixture models.
problem Slow or invalid convergence of EM algorithm for mixture models.
method CM-EM algorithm with a step to optimize mixture ratios and maximize G.
result Global convergence proof for CM-EM algorithm using variational methods.
Unified variational inference framework reveals GAN's limitations and proposes improvements.
problem Limitations of GAN training and lack of completeness in loss function.
method Reinterpretation of variational inference and revealing special cases of GAN, VAE, etc.
result Proposes a regularization term to improve GAN training stability.
We accelerate CNF by reducing ODE truncation errors with polynomial regularization.
problem High computation cost of CNF due to large truncation errors in solving ODEs.
method Add polynomial regularization to approximate ODE trajectories with polynomial functions.
result 42.3% to 71.3% reduction of NFE on density estimation, 19.3% to 32.1% on variational auto-encoder.
This paper models time-series data with a mixture of Markov chains, automatically determining the number of components.
problem Tackles the inability of common Markov state modeling frameworks to discern heterogeneities in complex data.
method Uses a mixture of Markov chains and variational expectation-maximization algorithm for automatic component selection.
result Achieves performance consistent with theoretically optimal error scaling, identifying meaningful heterogeneities in various data sets.
MLE and CVE are equivalent under exponential families, leading to faster and more stable EM algorithms.
problem Finding maximum likelihood estimators (MLE) efficiently and stably.
method Proved equivalence between MLE and CVE under exponential families, leading to an EM algorithm.
result EM algorithm achieves the same asymptotic variance as MLE and is faster and more stable.
Efficiently estimate Boolean product distribution parameters from truncated samples.
problem Estimating parameters of Boolean product distributions from truncated samples.
method Introducing fatness of truncation set, using membership queries, and adapting Stochastic Gradient Descent.
result Efficiently learn Boolean product distributions from truncated samples with small sample complexity.
Proposes a new method to estimate Bayesian neural network depth.
problem Estimating the depth of Bayesian neural networks.
method Uses a discrete truncated normal distribution to learn depth mean and variance, inferring posterior distributions by minimizing variational free energy.
result Improves test accuracy and reduces posterior depth variance on the spiral dataset.
When a gauge-natural invariant variational principle is assigned, to determine {\em canonical} covariant conservation laws, the vertical part of gauge-natural lifts of infinitesimal principal automorphisms -- defining infinitesimal variations of sections of gauge-natural bundles -- must satisfy generalized Jacobi equat…
A new EM-based algorithm improves deep generative model training.
problem Training deep generative models with maximum likelihood is challenging.
method The paper proposes reweighted expectation maximization (REM), a new algorithm that directly maximizes the log marginal likelihood of the data.
result REM learns better generative models than the IWAE, leading to significantly better performance in density estimation benchmarks.
New EM algorithm improves deep generative network training.
problem Training deep generative networks with complex posterior and likelihood distributions.
method Derive analytical posterior and marginal distributions using CPA property, derive analytical EM algorithm.
result EM training yields higher likelihood than Variational Autoencoders (VAEs).
Paper uses VAEs and GANs to estimate cryo-EM image orientation and camera parameters.
problem Estimating orientation and camera parameters from noisy cryo-EM images.
method Combines VAEs and GANs to learn latent representation, then designs estimation method.
result Geometric approach for fast cryo-EM biomolecule reconstruction.
We give a microscopic representation of the stock-market in which the microscopic agents are the individual traders and their capital. Their basic dynamics consists in the auto-catalysis of the individual capital and in the global competition/cooperation between the agents mediated by the total wealth invested in the s…
GFlowNet-EM learns complex latent variable models with discrete structures.
problem Challenges in modeling posteriors over discrete compositional latents with expectation-maximization.
method Uses GFlowNets to learn stochastic policies for sampling from complex posterior distributions.
result GFlowNet-EM enables training expressive LVMs with discrete compositional latents.
Survey of factor analysis, PCA, variational inference, and VAE.
problem Dimensionality reduction and generative modeling of data.
method Variational inference, factor analysis, probabilistic PCA, and VAE.
result Derivation and explanation of ELBO, EM, and closed-form solutions.
Gradually Truncated Log-normal distribution - Size distribution of firmscond-mat.stat-mech
Gradually Truncated Log-normal distribution - Size distribution of firms Abstract Many natural and economical phenomena are described through power law or log- normal distributions. In these cases, probability decreases very slowly with step size compared to normal distribution. Thus it is essential to cut-off these di…
We present a family of expectation-maximization (EM) algorithms for binary and negative-binomial logistic regression, drawing a sharp connection with the variational-Bayes algorithm of Jaakkola and Jordan (2000). Indeed, our results allow a version of this variational-Bayes approach to be re-interpreted as a true EM al…
Marginal MAP problems are notoriously difficult tasks for graphical models. We derive a general variational framework for solving marginal MAP problems, in which we apply analogues of the Bethe, tree-reweighted, and mean field approximations. We then derive a "mixed" message passing algorithm and a convergent alternati…
The speed of convergence of the Expectation Maximization (EM) algorithm for Gaussian mixture model fitting is known to be dependent on the amount of overlap among the mixture components. In this paper, we study the impact of mixing coefficients on the convergence of EM. We show that when the mixture components exhibit …
New method estimates volatility for Lévy processes with unbounded jumps efficiently.
problem Efficient estimation of volatility for Lévy processes with unbounded jumps.
method Developed a new estimator based on high-order expansions of truncated moments.
result Method outperforms existing alternatives in estimating volatility.
New method estimates tempered stable Lévy models with high accuracy.
problem Estimating volatility and jump intensity of tempered stable Lévy processes.
method Iterative method combining Truncated Realized Quadratic Variations and small-time approximations.
result Method outperforms existing alternatives in various scenarios.
New method trains neural networks to optimize faster than tuned methods.
problem Training learned optimizers is difficult and often leads to poor performance.
method Dynamic weighting of unbiased gradient estimators for a variational loss.
result Trained neural networks optimize faster than tuned first-order methods.