arXiv research

New analysis proves sketching operators' RIP guarantees for mixture models without importance sampling.

Sketching is a randomized dimensionality-reduction method that aims to preserve relevant information in large-scale datasets. Count sketch is a simple popular sketch which uses a randomized hash function to achieve compression. In this paper, we propose a novel extension known as Higher-order Count Sketch (HCS). While …

New method speeds up learning of complex dynamical systems.

Paper proposes Nyström sketches for better adaptive compressive learning.

Sketching accelerates structured prediction methods for large datasets.

Paper develops a bootstrap method for estimating sketched SVD errors.

Matrix sketching is a recently developed data compression technique. An input matrix A is efficiently approximated with a smaller matrix B, so that B preserves most of the properties of A up to some guaranteed approximation ratio. In so doing numerical operations on big data sets become faster. Sketching algorithms gen…

Randomized algorithm solves vector-valued regression problems with low-rank operators.

Recently, randomly mapping vectorial data to strings of discrete symbols (i.e., sketches) for fast and space-efficient similarity searches has become popular. Such random mapping is called similarity-preserving hashing and approximates a similarity metric by using the Hamming distance. Although many efficient similarit…

The main contribution of the paper is to show that Gaussian sketching of a kernel-Gram matrix $\boldsymbol K$ yields an operator whose counterpart in an RKHS $\mathcal H$, is a \emph{random projection} operator---in the spirit of Johnson-Lindenstrauss (J-L) lemma. To be precise, given a random matrix $Z$ with i.i.d. Ga…

We consider the problem of finding anomalies in high-dimensional data using popular PCA based anomaly scores. The naive algorithms for computing these scores explicitly compute the PCA of the covariance matrix which uses space quadratic in the dimensionality of the data. We give the first streaming algorithms that use …

We provide a deterministic space-efficient algorithm for estimating ridge regression. For $n$ data points with $d$ features and a large enough regularization parameter, we provide a solution within $\varepsilon$ L$_2$ error using only $O(d/\varepsilon)$ space. This is the first $o(d^2)$ space deterministic streaming al…

SketchGraphs dataset aids in modeling CAD designs.

Sketch-BERT learns vector sketches using BERT-like self-supervised learning.

Deep neural networks are powerful learning models that achieve state-of-the-art performance on many computer vision, speech, and language processing tasks. In this paper, we study a fundamental question that arises when designing deep network architectures: Given a target network architecture can we design a smaller ne…

In this largely expository paper we give a self-contained treatment of the Dirac operator. Emphasizing the algebraic point of view we first sketch the necessary prerequisites from Clifford algebras and their representations and then define (and characterize) spin structures and the corresponding Spin-Dirac operator pur…

We sketch out a new geometric framework to construct Hamiltonian operators for generic, non-evolutionary partial differential equations. Examples on how the formalism works are provided for the KdV equation, Camassa-Holm equation, and Kupershmidt's deformation of a bi-Hamiltonian system.

T-EMDE bridges the heterogeneity gap between image and text modalities.

A new method improves convergence in low-rank approximation.

We address the statistical and optimization impacts of the classical sketch and Hessian sketch used to approximately solve the Matrix Ridge Regression (MRR) problem. Prior research has quantified the effects of classical sketch on the strictly simpler least squares regression (LSR) problem. We establish that classical …

We present a mechanism to compute a sketch (succinct summary) of how a complex modular deep network processes its inputs. The sketch summarizes essential information about the inputs and outputs of the network and can be used to quickly identify key components and summary statistics of the inputs. Furthermore, the sket…

Localized sketching improves matrix multiplication and ridge regression complexity.

We introduce a new sub-linear space sketch---the Weight-Median Sketch---for learning compressed linear classifiers over data streams while supporting the efficient recovery of large-magnitude weights in the model. This enables memory-limited execution of several statistical analyses over streams, including online featu…

The paper sharpens the analysis of sketch-and-project methods using randomized singular value decomposition.

New method reduces linear regret in high-dimensional bandit problems.

New guarantees for asymmetric sketching in compressive learning.

A fast sketching algorithm solves regularized least squares problems efficiently.

Learning parameters from voluminous data can be prohibitive in terms of memory and computational requirements. We propose a "compressive learning" framework where we estimate model parameters from a sketch of the training data. This sketch is a collection of generalized moments of the underlying probability distributio…

A new method for estimating large-scale linear models with improved precision.

Private sketches protect linear regression data privacy.

In this note the long standing problem of the definition of a Poisson bracket in the framework of a multisymplectic formulation of classical field theory is solved. The new bracket operation can be applied to forms of arbitary degree. Relevant examples are discussed and important properties are stated with proofs sketc…

DiPS learns to optimize sketching policies for better recommendation quality.

Sketching is more fundamental to human cognition than speech. Deep Neural Networks (DNNs) have achieved the state-of-the-art in speech-related tasks but have not made significant development in generating stroke-based sketches a.k.a sketches in vector format. Though there are Variational Auto Encoders (VAEs) for genera…

Density sketches summarize data distributions for accurate sampling and estimation.

Improved sketching for logistic and $\ell_1$ regression with near-linear dimensions.

Unified bounds for sketched bilinear forms in machine learning and statistics.

We improve prediction risk estimation for large datasets using sketching and ridge regression.

Linear regression is a classic method of data analysis. In recent years, sketching -- a method of dimension reduction using random sampling, random projections, or both -- has gained popularity as an effective computational approximation when the number of observations greatly exceeds the number of variables. In this p…

Sketching techniques have become popular for scaling up machine learning algorithms by reducing the sample size or dimensionality of massive data sets, while still maintaining the statistical power of big data. In this paper, we study sketching from an optimization point of view: we first show that the iterative Hessia…

Paper advances sparse regularisation theory for measures with new kernel insights.

We present sketch-rnn, a recurrent neural network (RNN) able to construct stroke-based drawings of common objects. The model is trained on thousands of crude human-drawn images representing hundreds of classes. We outline a framework for conditional and unconditional sketch generation, and describe new robust training …

Sketching reduces data size for accurate spectral estimation.

Downsampling can improve generalization in ridgeless linear regression, especially with optimal sketching size.

A new data-oblivious sketch for logistic regression reduces data size while maintaining approximation accuracy.

Sketch-GNN reduces GNN training time and memory usage to sublinear scales.

This paper speeds up kernel methods using sparsified Gaussian sketches.

Newton-LESS sparsifies Gaussian sketching for faster optimization.

Recent work has explored transforming data sets into smaller, approximate summaries in order to scale Bayesian inference. We examine a related problem in which the parameters of a Bayesian model are very large and expensive to store in memory, and propose more compact representations of parameter values that can be use…