arXiv research

We accelerate the power method for strong low-rank approximation using fast sketching.

Fast matrix algorithms have become the fundamental tools of machine learning in big data era. The generalized matrix regression problem is widely used in the matrix approximation such as CUR decomposition, kernel matrix approximation, and stream singular value decomposition (SVD), etc. In this paper, we propose a fast …

This study explains why approximate NGD works well in wide neural networks.

We propose fast approximations for the generalized sliced-Wasserstein distance.

New algorithms for approximating stochastic processes efficiently.

This paper speeds up Gaussian process regression for autocorrelated data.

The paper develops fast Bayesian methods for estimating huge PVARs with competitive forecasts.

We study the problem of approximating orthogonal matrices so that their application is numerically fast and yet accurate. We find an approximation by solving an optimization problem over a set of structured matrices, that we call extended orthogonal Givens transformations, including Givens rotations as a special case. …

We provide a fast approximation to eNTKs for neural networks.

This paper proposes an incremental solution to Fast Subclass Discriminant Analysis (fastSDA). We present an exact and an approximate linear solution, along with an approximate kernelized variant. Extensive experiments on eight image datasets with different incremental batch sizes show the superiority of the proposed ap…

We investigate methods for pricing American options under the variance gamma model. The variance gamma process is a pure jump process which is constructed by replacing the calendar time by the gamma time in a Brownian motion with drift, which makes it a time-changed Brownian motion. In general, the finite difference me…

Paper analyzes normal approximation for two-timescale stochastic algorithms, revealing interaction between fast and slow timescales.

Efficiently approximates statistical leverage scores for faster KRR.

Efficiently reduces tensor ranks using mean-field approximation.

Natural-gradient methods enable fast and simple algorithms for variational inference, but due to computational difficulties, their use is mostly limited to \emph{minimal} exponential-family (EF) approximations. In this paper, we extend their application to estimate \emph{structured} approximations such as mixtures of E…

The paper studies efficient simulation methods for financial firm values under fast mean-reverting volatility.

An infinitely wide model is a weighted integration $\int \varphi(x,v) d μ(v)$ of feature maps. This model excels at handling an infinite number of features, and thus it has been adopted to the theoretical study of deep learning. Kernel quadrature is a kernel-based numerical integration scheme developed for fast approxi…

Traditionally, the field of computational Bayesian statistics has been divided into two main subfields: variational methods and Markov chain Monte Carlo (MCMC). In recent years, however, several methods have been proposed based on combining variational Bayesian inference and MCMC simulation in order to improve their ov…

Efficiently approximates eigenspaces for symmetric and general matrices.

In this work, we revisit fast dimension reduction approaches, as with random projections and random sampling. Our goal is to summarize the data to decrease computational costs and memory footprint of subsequent analysis. Such dimension reduction can be very efficient when the signals of interest have a strong structure…

Time delay estimation (TDE) is a critical and challenging step in all ultrasound elastography methods. A growing number of TDE techniques require an approximate but robust and fast method to initialize solving for TDE. Herein, we present a fast method for calculating an approximate TDE between two radio frequency (RF) …

Paper optimizes multi-fidelity function with fast learning rates.

Improved stochastic approximation method reduces residual error.

The paper analyzes reinforcement learning methods for estimating weights and quality functions with fast convergence rates.

SURF simplifies distribution estimation with simple, robust, and fast algorithms.

The future predictive performance of a Bayesian model can be estimated using Bayesian cross-validation. In this article, we consider Gaussian latent variable models where the integration over the latent values is approximated using the Laplace method or expectation propagation (EP). We study the properties of several B…

We study the maximum mean discrepancy (MMD) in the context of critical transitions modelled by fast-slow stochastic dynamical systems. We establish a new link between the dynamical theory of critical transitions with the statistical aspects of the MMD. In particular, we show that a formal approximation of the MMD near …

Gaussian Process Factor Analysis (GPFA) has been broadly applied to the problem of identifying smooth, low-dimensional temporal structure underlying large-scale neural recordings. However, spike trains are non-Gaussian, which motivates combining GPFA with discrete observation models for binned spike count data. The dra…

Inference is typically intractable in high-treewidth undirected graphical models, making maximum likelihood learning a challenge. One way to overcome this is to restrict parameters to a tractable set, most typically the set of tree-structured parameters. This paper explores an alternative notion of a tractable set, nam…

Optimizes trading large volumes of volatile assets with fast mean-reverting volatility.

Develops a fast variational approximation for high-dimensional empirical Bayes posteriors.

Novel framework proves fast RL convergence in continuous spaces.

Spectral clustering is a widely studied problem, yet its complexity is prohibitive for dynamic graphs of even modest size. We claim that it is possible to reuse information of past cluster assignments to expedite computation. Our approach builds on a recent idea of sidestepping the main bottleneck of spectral clusterin…

Flexible empirical Bayes for large-scale multiple linear regression.

FFCP improves FCP's speed without sacrificing accuracy.

A fast method approximates likelihood scores for noisy linear inverse problems.

Error bound conditions (EBC) are properties that characterize the growth of an objective function when a point is moved away from the optimal set. They have recently received increasing attention in the field of optimization for developing optimization algorithms with fast convergence. However, the studies of EBC in st…

A fast, approximate method for variable selection in GLMs tackles correlated data.

I prove that the spectrum of the Laplace-Beltrami operator with the Neumann boundary condition on a compact Riemannian manifold with boundary admits a fast approximation by the spectra of suitable graph Laplacians on proximity graphs on the manifold, and similar graph approximation works for metric-measure spaces glued…

A fast, robust AMP algorithm for quadratic optimization problems.

Unified theory and debiasing framework for random oblique projections in high dimensions.

This paper defines a generalized column subset selection problem which is concerned with the selection of a few columns from a source matrix A that best approximate the span of a target matrix B. The paper then proposes a fast greedy algorithm for solving this problem and draws connections to different problems that ca…

Hierarchical pretraining with slow-fast ODEs

New ACV method speeds up CV in high dimensions with approximate low-rank data.

This paper deals with two related problems, namely distance-preserving binary embeddings and quantization for compressed sensing . First, we propose fast methods to replace points from a subset $\mathcal{X} \subset \mathbb{R}^n$, associated with the Euclidean metric, with points in the cube $\{\pm 1\}^m$ and we associa…

Paper proposes a fast method for approximate data deletion in generative models.

We study singularity formation in spherically symmetric solutions of the charge-one and charge-two sector of the (2+1)-dimensional S^2 sigma-model and the (4+1)-dimensional Yang-Mills model, near the adiabatic limit. These equations are non-integrable, and so studies are performed numerically on rotationally symmetric …

Sparse variational approximations allow for principled and scalable inference in Gaussian Process (GP) models. In settings where several GPs are part of the generative model, theses GPs are a posteriori coupled. For many applications such as regression where predictive accuracy is the quantity of interest, this couplin…