arXiv research

New algorithm converts data into sub-gaussian designs efficiently.

This paper speeds up kernel methods using sparsified Gaussian sketches.

Improved estimator for least squares using random projections achieves smaller error.

Newton-LESS sparsifies Gaussian sketching for faster optimization.

Paper proposes Nyström sketches for better adaptive compressive learning.

Improved privacy-preserving linear regression via iterative Hessian mixing.

A new method reduces the bias in estimating inverse covariance matrices from sketches.

We introduce two versions of a new sketch for approximately embedding the Gaussian kernel into Euclidean inner product space. These work by truncating infinite expansions of the Gaussian kernel, and carefully invoking the RecursiveTensorSketch [Ahle et al. SODA 2020]. After providing concentration and approximation pro…

ADASAP accelerates GP inference for large datasets.

This work explores the non-convex optimization in compressive learning and the performance of heuristics.

New method improves Gaussian kernel approximations for high-frequency data.

Detecting emergence of a low-rank signal from high-dimensional data is an important problem arising from many applications such as camera surveillance and swarm monitoring using sensors. We consider a procedure based on the largest eigenvalue of the sample covariance matrix over a sliding window to detect the change. T…

New algorithm reduces sketching dimension to effective problem size.

We describe a general framework -- compressive statistical learning -- for resource-efficient large-scale learning: the training collection is compressed in one pass into a low-dimensional sketch (a vector of random empirical generalized moments) that captures the information relevant to the considered learning task. A…

Kernel ridge regression (KRR) is a standard method for performing non-parametric regression over reproducing kernel Hilbert spaces. Given $n$ samples, the time and space complexity of computing the KRR estimate scale as $\mathcal{O}(n^3)$ and $\mathcal{O}(n^2)$ respectively, and so is prohibitive in many cases. We prop…

A new method embeds data using Gaussian processes based on the heat kernel.

Corrects bias in random sampling matrices for improved ML methods.

Develops precise expressions for random projections for better machine learning tasks.

A parallel optimization method for convex functions using Hessian sketching and debiasing.

We consider a least squares regression problem where the data has been generated from a linear model, and we are interested to learn the unknown regression parameters. We consider "sketch-and-solve" methods that randomly project the data first, and do regression after. Previous works have analyzed the statistical and c…

The main contribution of the paper is to show that Gaussian sketching of a kernel-Gram matrix $\boldsymbol K$ yields an operator whose counterpart in an RKHS $\mathcal H$, is a \emph{random projection} operator---in the spirit of Johnson-Lindenstrauss (J-L) lemma. To be precise, given a random matrix $Z$ with i.i.d. Ga…

Randomized algorithm solves vector-valued regression problems with low-rank operators.

New algorithm speeds up polynomial kernel approximations.

Sketch-BERT learns vector sketches using BERT-like self-supervised learning.

New method solves constrained stochastic optimization problems efficiently.

New method for efficient maximum likelihood estimation of $p$-generalized probit regression.

A new method improves convergence in low-rank approximation.

In this paper, we propose a general framework for sparse and low-rank tensor estimation from cubic sketchings. A two-stage non-convex implementation is developed based on sparse tensor decomposition and thresholded gradient descent, which ensures exact recovery in the noiseless case and stable recovery in the noisy cas…

We address the statistical and optimization impacts of the classical sketch and Hessian sketch used to approximately solve the Matrix Ridge Regression (MRR) problem. Prior research has quantified the effects of classical sketch on the strictly simpler least squares regression (LSR) problem. We establish that classical …

We present a mechanism to compute a sketch (succinct summary) of how a complex modular deep network processes its inputs. The sketch summarizes essential information about the inputs and outputs of the network and can be used to quickly identify key components and summary statistics of the inputs. Furthermore, the sket…

In this paper, we develop a novel procedure for low-rank tensor regression, namely \emph{\underline{I}mportance \underline{S}ketching \underline{L}ow-rank \underline{E}stimation for \underline{T}ensors} (ISLET). The central idea behind ISLET is \emph{importance sketching}, i.e., carefully designed sketches based on bot…

Localized sketching improves matrix multiplication and ridge regression complexity.

Learning parameters from voluminous data can be prohibitive in terms of memory and computational requirements. We propose a "compressive learning" framework where we estimate model parameters from a sketch of the training data. This sketch is a collection of generalized moments of the underlying probability distributio…

Stochastic trace estimation with tensor train random vectors

We introduce a new sub-linear space sketch---the Weight-Median Sketch---for learning compressed linear classifiers over data streams while supporting the efficient recovery of large-magnitude weights in the model. This enables memory-limited execution of several statistical analyses over streams, including online featu…

The paper sharpens the analysis of sketch-and-project methods using randomized singular value decomposition.

New method reduces linear regret in high-dimensional bandit problems.

New guarantees for asymmetric sketching in compressive learning.

New analysis proves sketching operators' RIP guarantees for mixture models without importance sampling.

A fast sketching algorithm solves regularized least squares problems efficiently.

Two new approaches for point prediction in streaming data, showing consistency and performance.

A new method for estimating large-scale linear models with improved precision.

Private sketches protect linear regression data privacy.

Bayesian nonparametric CMS improves frequency estimation for power-law data.

Sketching is a randomized dimensionality-reduction method that aims to preserve relevant information in large-scale datasets. Count sketch is a simple popular sketch which uses a randomized hash function to achieve compression. In this paper, we propose a novel extension known as Higher-order Count Sketch (HCS). While …

DiPS learns to optimize sketching policies for better recommendation quality.

Sketching is more fundamental to human cognition than speech. Deep Neural Networks (DNNs) have achieved the state-of-the-art in speech-related tasks but have not made significant development in generating stroke-based sketches a.k.a sketches in vector format. Though there are Variational Auto Encoders (VAEs) for genera…

Density sketches summarize data distributions for accurate sampling and estimation.