arXiv research

The paper shows objective derivatives are covariant derivatives on Riemannian metrics.

Gradient-based methods for optimisation of objectives in stochastic settings with unknown or intractable dynamics require estimators of derivatives. We derive an objective that, under automatic differentiation, produces low-variance unbiased estimators of derivatives at any order. Our objective is compatible with arbit…

A common assumption in financial engineering is that the market price for any derivative coincides with an objectively defined risk-neutral price - a plausible assumption only if traders collectively possess objective knowledge about the price dynamics of the underlying security over short time scales. Here we assume t…

New methods boost first-order optimization with faster rates.

Derivative-free method solves stochastic optimization problems with noisy objectives and constraints.

Unified approach for neural networks with multi-compartmental neurons and non-Hebbian plasticity.

This is the second companion paper of arXiv:1601.03586. We consider the morphism from the variety of triples introduced in arXiv:1601.03586 to the affine Grassmannian. The direct image of the dualizing complex is a ring object in the equivariant derived category on the affine Grassmannian (equivariant derived Satake ca…

Develops derived differential geometry theory.

We propose to learn deep undirected graphical models (i.e., MRFs) with a non-ELBO objective for which we can calculate exact gradients. In particular, we optimize a saddle-point objective deriving from the Bethe free energy approximation to the partition function. Unlike much recent work in approximate inference, the d…

Improved convergence speed of principal component analysis through modified learning rules.

This paper studies continuum-armed bandits under Besov smoothness conditions and derives minimax rates.

Node-perturbation learning is a type of statistical gradient descent algorithm that can be applied to problems where the objective function is not explicitly formulated, including reinforcement learning. It estimates the gradient of an objective function by using the change in the object function in response to the per…

Paper derives a simplified formula for Expected Improvement using log-transformed data.

Topic models have emerged as fundamental tools in unsupervised machine learning. Most modern topic modeling algorithms take a probabilistic view and derive inference algorithms based on Latent Dirichlet Allocation (LDA) or its variants. In contrast, we study topic modeling as a combinatorial optimization problem, and p…

We consider general non-Euclidean distance measures between real world objects that need to be classified. It is assumed that objects are represented by distances to other objects only. Conditions for zero-error dissimilarity based classifiers are derived. Additional conditions are given under which the zero-error deci…

New entropy-based objective for sparse coding improves learning.

LORL learns object-centric representations from vision and language.

New inequalities for unbounded functions improve denoising score matching.

Abstract: Tangent categories get a Cartan calculus with scalar multiplication by a commutative ring.

Bayesian optimization has been successful at global optimization of expensive-to-evaluate multimodal objective functions. However, unlike most optimization methods, Bayesian optimization typically does not use derivative information. In this paper we show how Bayesian optimization can exploit derivative information to …

We present a framework to understand GAN training as alternating density ratio estimation and approximate divergence minimization. This provides an interpretation for the mismatched GAN generator and discriminator objectives often used in practice, and explains the problem of poor sample diversity. We also derive a fam…

This work gives an in-depth derivation of the trainable evidence lower bound obtained from the marginal joint log-Likelihood with the goal of training a Multi-Modal Variational Autoencoder (M$^2$VAE).

Efficiently approximates higher-order derivatives for generative models.

Informative Bayesian priors are often difficult to elicit, and when this is the case, modelers usually turn to noninformative or objective priors. However, objective priors such as the Jeffreys and reference priors are not tractable to derive for many models of interest. We address this issue by proposing techniques fo…

The quotient $L/A[-1]$ of a pair $A\hookrightarrow L$ of Lie algebroids is a Lie algebra object in the derived category $D^b(\mathscr{A})$ of the category $\mathscr{A}$ of left $\mathcal{U}(A)$-modules, the Atiyah class $α_{L/A}$ being its Lie bracket. In this note, we describe the universal enveloping algebra of the L…

Introduces Lie-Nijenhuis bialgebroids for Poisson-Nijenhuis groupoids.

VICE embeds concepts in a vector space using human data.

In classic papers, Zellner demonstrated that Bayesian inference could be derived as the solution to an information theoretic functional. Below we derive a generalized form of this functional as a variational lower bound of a predictive information bottleneck objective. This generalized functional encompasses most moder…

Variational Optimization forms a differentiable upper bound on an objective. We show that approaches such as Natural Evolution Strategies and Gaussian Perturbation, are special cases of Variational Optimization in which the expectations are approximated by Gaussian sampling. These approaches are of particular interest …

A model structure is defined on the category of derived differentiable schemes, and it is used to analyse the truncation 2-functor from derived manifolds to d-manifolds. It is proved that the induced 1-functor between the homotopy categories is full and essentially surjective, giving a bijection between the sets of equ…

We propose a general framework for neural network compression that is motivated by the Minimum Description Length (MDL) principle. For that we first derive an expression for the entropy of a neural network, which measures its complexity explicitly in terms of its bit-size. Then, we formalize the problem of neural netwo…

Building models, or maps, of robot environments is a highly active research area; however, most existing techniques construct unstructured maps and assume static environments. In this paper, we present an algorithm for learning object models of non-stationary objects found in office-type environments. Our algorithm exp…

Bayesian optimization (BO) is a global optimization strategy designed to find the minimum of an expensive black-box function, typically defined on a compact subset of $\mathcal{R}^d$, by using a Gaussian process (GP) as a surrogate model for the objective. Although currently available acquisition functions address this…

We analyse derivative securities whose value is NOT a deterministic function of an underlying which means presence of a basis risk at any time. The key object of our analysis is conditional probability distribution at a given underlying value and moment of time. We consider time evolution of this probability distributi…

We show that, locally, all geometric objects of Generalized Kahler Geometry can be derived from a function K, the "generalized Kahler potential''. The metric g and two-form B are determined as nonlinear functions of second derivatives of K. These nonlinearities are shown to arise via a quotient construction from an aux…

Using the language and terminology of relative homological algebra, in particular that of derived functors, we introduce equivariant cohomology over a general Lie-Rinehart algebra and equivariant de Rham cohomology over a locally trivial Lie groupoid in terms of suitably defined monads (also known as triples) and the a…

Classifies objects in graded skew-gentle algebras using geometric models.

In this paper we study some geometrical objects (d-tensors, multi-time semisprays of polymomenta and nonlinear connections) on the dual 1-jet vector bundle $J^{1*}(\cal{T}, M)\to \cal{T}\times M$. Some geometrical formulas, which connect the last two geometrical objects, are also derived. Finally, a canonical nonlinear…

New method improves domain generalization by matching object representations.

Study derived Lie ∞-groupoids and algebroids in higher differential geometry.

The Variational AutoEncoder (VAE) learns simultaneously an inference and a generative model, but only one of these models can be learned at optimum, this behaviour is associated to the ELBO learning objective, that is optimised by a non-informative generator. In order to solve such an issue, we provide a learning objec…

New Variational InfoMax objective improves neural network performance.

New error bounds for GANs with nonlinear objective functions derived.

We propose a novel objective function for learning robust deep representations of data based on information theory. Data is projected into a feature-vector space such that the mutual information of all subsets of features relative to the supervising signal is maximized. This objective function gives rise to robust repr…

EnKG solves inverse problems without derivatives, using diffusion models.

Currently there is no validated objective measure of pain. Recent neuroimaging studies have explored the feasibility of using functional near-infrared spectroscopy (fNIRS) to measure alterations in brain function in evoked and ongoing pain. In this study, we applied multi-task machine learning methods to derive a pract…

Study proves convergence of subgradients for optimal transport-based objectives.

Cactus doodles are geometric objects derived from cactus groups.