arXiv research

New method computes affine normal directions efficiently for sparse polynomials.

A new algorithm improves sampling for graph learning models.

Algorithms for Gaussian process, marginal likelihood methods or restricted maximum likelihood methods often require derivatives of log determinant terms. These log determinants are usually parametric with variance parameters of the underlying statistical models. This paper demonstrates that, when the underlying matrix …

The log-determinant of a kernel matrix appears in a variety of machine learning problems, ranging from determinantal point processes and generalized Markov random fields, through to the training of Gaussian processes. Exact calculation of this term is often intractable when the size of the kernel matrix exceeds a few t…

Novel algorithm speeds up log-determinant estimation for large matrices.

The scalable calculation of matrix determinants has been a bottleneck to the widespread application of many machine learning methods such as determinantal point processes, Gaussian processes, generalised Markov random fields, graph models and many others. In this work, we estimate log determinants under the framework o…

This work presents a parametrized family of divergences, namely Alpha-Beta Log- Determinant (Log-Det) divergences, between positive definite unitized trace class operators on a Hilbert space. This is a generalization of the Alpha-Beta Log-Determinant divergences between symmetric, positive definite matrices to the infi…

Evaluating the log determinant of a positive definite matrix is ubiquitous in machine learning. Applications thereof range from Gaussian processes, minimum-volume ellipsoids, metric learning, kernel learning, Bayesian neural networks, Determinental Point Processes, Markov random fields to partition functions of discret…

Paper improves efficiency in matrix computations for Gaussian processes.

A new method speeds up training of deep models by avoiding Jacobian determinant computation.

Polylab is a MATLAB toolbox for multivariate polynomial modeling.

For applications as varied as Bayesian neural networks, determinantal point processes, elliptical graphical models, and kernel learning for Gaussian processes (GPs), one must compute a log determinant of an $n \times n$ positive definite matrix, and its derivatives - leading to prohibitive $\mathcal{O}(n^3)$ computatio…

New framework uses geometry of embeddings to predict robustness.

DAGMA learns DAGs faster and more accurately using log-determinant acyclicity.

New method estimates log-determinant using trace powers, avoiding classical limitations.

Study on geometric Jensen-Shannon divergence for Gaussian measures in Hilbert space.

The paper connects complex normalizing flows to Kähler-Ricci flows using geometric and statistical perspectives.

Develops interpolation methods for matrix functions in statistics and machine learning.

A new coordinate system for SPD matrices simplifies computations and generative modeling.

Efficient approximation lies at the heart of large-scale machine learning problems. In this paper, we propose a novel, robust maximum entropy algorithm, which is capable of dealing with hundreds of moments and allows for computationally efficient approximations. We showcase the usefulness of the proposed method, its eq…

We establish sharp Sobolev inequalities of order four on Euclidean d-balls for d greater than or equal to four. When d=4, our inequality generalizes the classical second order Lebedev-Milin inequality on Euclidean 2-balls. Our method relies on the use of scattering theory on hyperbolic d-balls. As an application, we ch…

Multi-Task Learning (MTL) can enhance a classifier's generalization performance by learning multiple related tasks simultaneously. Conventional MTL works under the offline or batch setting, and suffers from expensive training cost and poor scalability. To address such inefficiency issues, online learning techniques hav…

Paper introduces VDE, a variance-reduced determinant estimator.

HCLM framework uses entropy regularization for open learning systems.

New algorithm for online portfolio selection with reduced runtime.

TopoGeoScore selects robust checkpoints using only source-domain representations.

Given i.i.d. observations of a random vector $X \in \mathbb{R}^p$, we study the problem of estimating both its covariance matrix $Σ^*$, and its inverse covariance or concentration matrix {$Θ^* = (Σ^*)^{-1}$.} We estimate $Θ^*$ by minimizing an $\ell_1$-penalized log-determinant Bregman divergence; in the multivariate G…

The curvature of the noncommutative torus $T^2_θ$ ($θ$ irrational) endowed with a noncommutative conformal metric has been the focus of attention of several recent works. Continuing the approach taken in the paper [A. Connes and H. Moscovici, http://arxiv.org/abs/1110.3500] we extend the study of the curvature to twist…

We show that standard ResNet architectures can be made invertible, allowing the same model to be used for classification, density estimation, and generation. Typically, enforcing invertibility requires partitioning dimensions or restricting network architectures. In contrast, our approach only requires adding a simple …

The ability of many powerful machine learning algorithms to deal with large data sets without compromise is often hampered by computationally expensive linear algebra tasks, of which calculating the log determinant is a canonical example. In this paper we demonstrate the optimality of Maximum Entropy methods in approxi…

Gaussian processes (GPs) with derivatives are useful in many applications, including Bayesian optimization, implicit surface reconstruction, and terrain reconstruction. Fitting a GP to function values and derivatives at $n$ points in $d$ dimensions requires linear solves and log determinants with an ${n(d+1) \times n(d…

The paper improves support recovery in high-dimensional precision matrix estimation using meta learning.

The L1-regularized Gaussian maximum likelihood estimator (MLE) has been shown to have strong statistical guarantees in recovering a sparse inverse covariance matrix, or alternatively the underlying graph structure of a Gaussian Markov Random Field, from very limited samples. We propose a novel algorithm for solving the…

New algorithm speeds up cluster-based compressive sensing tasks.

While normalizing flows have led to significant advances in modeling high-dimensional continuous distributions, their applicability to discrete distributions remains unknown. In this paper, we show that flows can in fact be extended to discrete events---and under a simple change-of-variables formula not requiring log-d…

Gradient descent on LSE objectives implicitly performs EM, leading to collapse without volume control.

Consider a random vector with finite second moments. If its precision matrix is an M-matrix, then all partial correlations are non-negative. If that random vector is additionally Gaussian, the corresponding Markov random field (GMRF) is called attractive. We study estimation of M-matrices taking the role of inverse sec…

Proposes a new method to improve regression models with reweighted samples.

Low-rank matrix is desired in many machine learning and computer vision problems. Most of the recent studies use the nuclear norm as a convex surrogate of the rank operator. However, all singular values are simply added together by the nuclear norm, and thus the rank may not be well approximated in practical problems. …

A new method learns quantization boundaries in continuous space using tessellation.

GraphGP: Scalable Gaussian Processes with Vecchia's Approximation

New findings show a balance between data fit and complexity in kernel hyperparameters.

This paper proposes a method for machine learning from unlabeled data in the form of a time-series. The mapping that is learned is shown to extract slowly evolving information that would be useful for control applications, while efficiently filtering out unwanted, higher-frequency noise. The method consists of training…

The conformal anomalies and functional determinants of the Branson--GJMS operators, P_{2k}, on the d-dimensional sphere are evaluated in explicit terms for any d and k such that k < d/2+1 (if d is even). The determinants are given in terms of multiple gamma functions and a rational multiplicative anomaly, which vanishe…

Improved graph clustering with modularity and coarsening for attributes and communities.

Improved Gaussian process regression with tighter log marginal likelihood bounds.

Paper proves Shapley value convergence in Bayesian learning games.

A large class of machine learning techniques requires the solution of optimization problems involving spectral functions of parametric matrices, e.g. log-determinant and nuclear norm. Unfortunately, computing the gradient of a spectral function is generally of cubic complexity, as such gradient descent methods are rath…