arXiv research

Sketch-BERT learns vector sketches using BERT-like self-supervised learning.

Sketching is more fundamental to human cognition than speech. Deep Neural Networks (DNNs) have achieved the state-of-the-art in speech-related tasks but have not made significant development in generating stroke-based sketches a.k.a sketches in vector format. Though there are Variational Auto Encoders (VAEs) for genera…

A fast sketching algorithm solves regularized least squares problems efficiently.

Downsampling can improve generalization in ridgeless linear regression, especially with optimal sketching size.

We improve prediction risk estimation for large datasets using sketching and ridge regression.

We present sketch-rnn, a recurrent neural network (RNN) able to construct stroke-based drawings of common objects. The model is trained on thousands of crude human-drawn images representing hundreds of classes. We outline a framework for conditional and unconditional sketch generation, and describe new robust training …

Unified bounds for sketched bilinear forms in machine learning and statistics.

Density sketches summarize data distributions for accurate sampling and estimation.

New method reduces linear regret in high-dimensional bandit problems.

Generative adversarial network synthesizes sketches into realistic images.

Paper proposes Nyström sketches for better adaptive compressive learning.

We present a probabilistic model for Sketch-Based Image Retrieval (SBIR) where, at retrieval time, we are given sketches from novel classes, that were not present at training time. Existing SBIR methods, most of which rely on learning class-wise correspondences between sketches and images, typically work well only for …

This paper speeds up kernel methods using sparsified Gaussian sketches.

SketchEmbedNet learns image representations from sketches, useful for few-shot learning.

Linear regression is a classic method of data analysis. In recent years, sketching -- a method of dimension reduction using random sampling, random projections, or both -- has gained popularity as an effective computational approximation when the number of observations greatly exceeds the number of variables. In this p…

Generative networks trained with sketching reduce computational burden.

A new method improves convergence in low-rank approximation.

We address the statistical and optimization impacts of the classical sketch and Hessian sketch used to approximately solve the Matrix Ridge Regression (MRR) problem. Prior research has quantified the effects of classical sketch on the strictly simpler least squares regression (LSR) problem. We establish that classical …

We present a mechanism to compute a sketch (succinct summary) of how a complex modular deep network processes its inputs. The sketch summarizes essential information about the inputs and outputs of the network and can be used to quickly identify key components and summary statistics of the inputs. Furthermore, the sket…

One-pass private sketch supports various machine learning tasks.

We study sequential change-point detection procedures based on linear sketches of high-dimensional signal vectors using generalized likelihood ratio (GLR) statistics. The GLR statistics allow for an unknown post-change mean that represents an anomaly or novelty. We consider both fixed and time-varying projections, deri…

Matrix sketching is a recently developed data compression technique. An input matrix A is efficiently approximated with a smaller matrix B, so that B preserves most of the properties of A up to some guaranteed approximation ratio. In so doing numerical operations on big data sets become faster. Sketching algorithms gen…

Localized sketching improves matrix multiplication and ridge regression complexity.

We introduce a new sub-linear space sketch---the Weight-Median Sketch---for learning compressed linear classifiers over data streams while supporting the efficient recovery of large-magnitude weights in the model. This enables memory-limited execution of several statistical analyses over streams, including online featu…

The paper sharpens the analysis of sketch-and-project methods using randomized singular value decomposition.

New guarantees for asymmetric sketching in compressive learning.

We prove that two popular linear contextual bandit algorithms, OFUL and Thompson Sampling, can be made efficient using Frequent Directions, a deterministic online sketching technique. More precisely, we show that a sketch of size $m$ allows a $\mathcal{O}(md)$ update time for both algorithms, as opposed to $Ω(d^2)$ req…

New analysis proves sketching operators' RIP guarantees for mixture models without importance sampling.

A new method for estimating large-scale linear models with improved precision.

Private sketches protect linear regression data privacy.

Sketching is a randomized dimensionality-reduction method that aims to preserve relevant information in large-scale datasets. Count sketch is a simple popular sketch which uses a randomized hash function to achieve compression. In this paper, we propose a novel extension known as Higher-order Count Sketch (HCS). While …

We consider statistical as well as algorithmic aspects of solving large-scale least-squares (LS) problems using randomized sketching algorithms. For a LS problem with input data $(X, Y) \in \mathbb{R}^{n \times p} \times \mathbb{R}^n$, sketching algorithms use a sketching matrix, $S\in\mathbb{R}^{r \times n}$ with $r \…

DiPS learns to optimize sketching policies for better recommendation quality.

Improved sketching for logistic and $\ell_1$ regression with near-linear dimensions.

Sketching accelerates structured prediction methods for large datasets.

Paper develops a bootstrap method for estimating sketched SVD errors.

Sketching techniques have become popular for scaling up machine learning algorithms by reducing the sample size or dimensionality of massive data sets, while still maintaining the statistical power of big data. In this paper, we study sketching from an optimization point of view: we first show that the iterative Hessia…

New sublinear sketches improve ANN and KDE for massive data streams.

A new method reduces the bias in estimating inverse covariance matrices from sketches.

We accelerate the power method for strong low-rank approximation using fast sketching.

Sketching reduces data size for accurate spectral estimation.

A new data-oblivious sketch for logistic regression reduces data size while maintaining approximation accuracy.

Sketch-GNN reduces GNN training time and memory usage to sublinear scales.

New method speeds up k-means clustering using sketch-and-solve.

SketchGraphs dataset aids in modeling CAD designs.

Newton-LESS sparsifies Gaussian sketching for faster optimization.

Recent work has explored transforming data sets into smaller, approximate summaries in order to scale Bayesian inference. We examine a related problem in which the parameters of a Bayesian model are very large and expensive to store in memory, and propose more compact representations of parameter values that can be use…

We describe a general framework -- compressive statistical learning -- for resource-efficient large-scale learning: the training collection is compressed in one pass into a low-dimensional sketch (a vector of random empirical generalized moments) that captures the information relevant to the considered learning task. A…