arXiv research

Stochastic variational inference (SVI) plays a key role in Bayesian deep learning. Recently various divergences have been proposed to design the surrogate loss for variational inference. We present a simple upper bound of the evidence as the surrogate loss. This evidence upper bound (EUBO) equals to the log marginal li…

This paper introduces a method to estimate log-likelihood in VAE models.

Variational inference (VI) is widely used as an efficient alternative to Markov chain Monte Carlo. It posits a family of approximating distributions $q$ and finds the closest member to the exact posterior $p$. Closeness is usually measured via a divergence $D(q || p)$ from $q$ to $p$. While successful, this approach al…

In this work, we present a novel upper bound of target error to address the problem for unsupervised domain adaptation. Recent studies reveal that a deep neural network can learn transferable features which generalize well to novel tasks. Furthermore, a theory proposed by Ben-David et al. (2010) provides a upper bound …

The paper calculates the index and nullity of Fraser-Sargent surfaces and provides bounds.

Given a choice of metric on the Riemann surface, the regularized determinant of Laplacian (analytic torsion) is defined via the complex power of elliptic operators: $\det(Δ)=\exp(-ζ'(0))$ In this paper we gave an asymptotic effective estimate of analytic torsion under Arakelov metric. In particular, after taking th…

Recent work in unsupervised representation learning has focused on learning deep directed latent-variable models. Fitting these models by maximizing the marginal likelihood or evidence is typically intractable, thus a common approximation is to maximize the evidence lower bound (ELBO) instead. However, maximum likeliho…

The paper tackles approximate unlearning from a subset of training data using variational inference.

This paper improves SAM by reformulating it as a bilevel optimization problem.

New algorithm for shareable arms with load-dependent rewards in stochastic bandits.

New SQ lower bound shows complexity nearly matches known upper bound for smoothed agnostic learning.

Non-negative matrix factorization (NMF) is a knowledge discovery method that is used in many fields. Variational inference and Gibbs sampling methods for it are also wellknown. However, the variational approximation error has not been clarified yet, because NMF is not statistically regular and the prior distribution us…

Variational Inference is a powerful tool in the Bayesian modeling toolkit, however, its effectiveness is determined by the expressivity of the utilized variational distributions in terms of their ability to match the true posterior distribution. In turn, the expressivity of the variational family is largely limited by …

Sharp heat kernel estimates on manifolds lead to solutions of the Parabolic Anderson model.

Empirical evidence shows that ensembles, such as bagging, boosting, random and rotation forests, generally perform better in terms of their generalization error than individual classifiers. To explain this performance, Schapire et al. (1998) developed an upper bound on the generalization error of an ensemble based on t…

New bounds on SGD's final iterate convergence rate in constant dimension.

Semi-implicit variational inference (SIVI) is introduced to expand the commonly used analytic variational distribution family, by mixing the variational parameter with a flexible distribution. This mixing distribution can assume any density function, explicit or not, as long as independent random samples can be generat…

We prove the Turaev-Viro invariants volume conjecture for a "universal" class of cusped hyperbolic 3-manifolds that produces all 3-manifolds with empty or toroidal boundary by Dehn filling. This leads to two-sided bounds on the volume of any hyperbolic 3-manifold with empty or toroidal boundary in terms of the growth r…

Paper bridges VAEs and KDEs for more flexible posterior estimation.

Variational inference is a powerful tool for approximate inference. However, it mainly focuses on the evidence lower bound as variational objective and the development of other measures for variational inference is a promising area of research. This paper proposes a robust modification of evidence and a lower bound for…

Upper bound for Laplacian eigenvalue via conformal volume.

Statistical Query lower bound shows difficulty in list-decodable linear regression.

Optimal SQ bounds for learning binary product distributions and Ising models.

We study the topology of admissible-loop spaces on a step-two Carnot group G. We use a Morse-Bott theory argument to study the structure and the number of geodesics on G connecting the origin with a 'vertical' point (geodesics are critical points of the 'Energy' functional, defined on the loop space). These geodesics t…

Upper bounds for volume spectrum depend on volume, dimension, and a conformal invariant.

In the absence of explicit regularization, Kernel "Ridgeless" Regression with nonlinear kernels has the potential to fit the training data perfectly. It has been observed empirically, however, that such interpolated solutions can still generalize well on test data. We isolate a phenomenon of implicit regularization for…

The paper finds large Steklov eigenvalues on manifolds using homogenization.

This paper analyzes regret bounds for Gaussian process Thompson sampling.

Study validates Libor model for insurance benefits calculation.

Upper bounds for CV errors apply to lasso and other models.

We show that any space with a positive upper curvature bound has in a small neighborhood of any point a closely related metric with a negative upper curvature bound.

New upper bound for Neumann Laplacian eigenvalues on convex domains.

Sharp upper bounds found for Steklov eigenvalues of a specific hypersurface.

This paper presents evidence supporting the surprising conjecture that in the topological category the slice genus of a satellite knot $P(K)$ is bounded above by the sum of the slice genera of $K$ and $P(U)$. Our main result establishes this conjecture for a variant of the topological slice genus, the $\mathbb{Z}$-slic…

Stochastic Bayesian Neural Network improves scalability and performance.

Following Ghomi and Tabachnikov we study topological obstructions to totally skew embeddings of a smooth manifold M in Euclidean spaces. This problem is naturally related to the question of estimating the geometric dimension of the stable normal bundle of the configuration space F_2(M) of ordered pairs of distinct poin…

Study proves upper bounds for solutions on Riemannian manifolds.

New method improves adversarial training robustness.

Normal surface theory is a central tool in algorithmic three-dimensional topology, and the enumeration of vertex normal surfaces is the computational bottleneck in many important algorithms. However, it is not well understood how the number of such surfaces grows in relation to the size of the underlying triangulation.…

Upper bounds for Steklov eigenvalues derived from intersection indices.

Upper bounds for Steklov eigenvalues of warped products are derived.

Upper bound for conjugate radius in open manifolds with scalar curvature and spectrum constraints.

Upper bound on geodesic ball volume in Riemannian manifolds.

We obtain upper bounds for the eigenvalues of the Schrödinger operator $L=Δ_g+q$ depending on integral quantities of the potential $q$ and a conformal invariant called the min-conformal volume. Moreover, when the Schrödinger operator $L$ is positive, integral quantities of $q$ which appear in upper bounds, can be repla…

New examples show no upper bounds on link volumes on incompressible surfaces.

We investigate the effect of explicitly enforcing the Lipschitz continuity of neural networks with respect to their inputs. To this end, we provide a simple technique for computing an upper bound to the Lipschitz constant---for multiple $p$-norms---of a feed forward neural network composed of commonly used layer types.…

Let ${\mathfrak M}$ be a closed, orientable, hyperbolic 3-orbifold such that $π_1({\mathfrak M})$ contains no hyperbolic triangle group. We show that strict upper bounds of 0.07625, 0.1525 and 0.22875 for ${\rm vol}\ {\mathfrak M}$ imply respective upper bounds of 23, 43 and 79 for $\dim H_1({\mathfrak M};{\mathbb F}_2…

Upper bounds found for systole function critical points on surface moduli space.