A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
New guarantees for asymmetric sketching in compressive learning.
The recent framework of compressive statistical learning aims at designing tractable learning algorithms that use only a heavily compressed representation-or sketch-of massive datasets. Compressive K-Means (CKM) is such a method: it estimates the centroids of data clusters from pooled, non-linear, random signatures of …
In this paper we show how Einstein metrics are naturally described using the quantization of the algebra of functions on a Kahler manifold M. In this setup one interprets M as the phase space itself, equipped with the Poisson brackets inherited from the Kahler 2-form. We compare the geometric quantization framework wit…
With the scale of data growing every day, reducing the dimensionality (a.k.a. sketching) of high-dimensional data has emerged as a task of paramount importance. Relevant issues to address in this context include the sheer volume of data that may consist of categorical samples, the typically streaming format of acquisit…
This work studies scaling laws for low-precision training in high-dimensional linear regression.
I sketch what it is supposed to mean to quantize gauge theory, and how this can be made more concrete in perturbation theory and also by starting with a finite-dimensional lattice approximation. Based on real experiments and computer simulations, quantum gauge theory in four dimensions is believed to have a mass gap. T…
Sketch-BERT learns vector sketches using BERT-like self-supervised learning.
A new method improves convergence in low-rank approximation.
We address the statistical and optimization impacts of the classical sketch and Hessian sketch used to approximately solve the Matrix Ridge Regression (MRR) problem. Prior research has quantified the effects of classical sketch on the strictly simpler least squares regression (LSR) problem. We establish that classical …
We present a mechanism to compute a sketch (succinct summary) of how a complex modular deep network processes its inputs. The sketch summarizes essential information about the inputs and outputs of the network and can be used to quickly identify key components and summary statistics of the inputs. Furthermore, the sket…
Localized sketching improves matrix multiplication and ridge regression complexity.
We introduce a new sub-linear space sketch---the Weight-Median Sketch---for learning compressed linear classifiers over data streams while supporting the efficient recovery of large-magnitude weights in the model. This enables memory-limited execution of several statistical analyses over streams, including online featu…
The paper sharpens the analysis of sketch-and-project methods using randomized singular value decomposition.
New method reduces linear regret in high-dimensional bandit problems.
Algorithm compresses large matrices by approximating them as low rank and low precision factors.
New analysis proves sketching operators' RIP guarantees for mixture models without importance sampling.
A fast sketching algorithm solves regularized least squares problems efficiently.
A new method for estimating large-scale linear models with improved precision.
Private sketches protect linear regression data privacy.
Sketching is a randomized dimensionality-reduction method that aims to preserve relevant information in large-scale datasets. Count sketch is a simple popular sketch which uses a randomized hash function to achieve compression. In this paper, we propose a novel extension known as Higher-order Count Sketch (HCS). While …
DiPS learns to optimize sketching policies for better recommendation quality.
Paper proposes Nyström sketches for better adaptive compressive learning.
Sketching is more fundamental to human cognition than speech. Deep Neural Networks (DNNs) have achieved the state-of-the-art in speech-related tasks but have not made significant development in generating stroke-based sketches a.k.a sketches in vector format. Though there are Variational Auto Encoders (VAEs) for genera…
Density sketches summarize data distributions for accurate sampling and estimation.
Improved sketching for logistic and regression with near-linear dimensions.
Unified bounds for sketched bilinear forms in machine learning and statistics.
We improve prediction risk estimation for large datasets using sketching and ridge regression.
The main new result here is the cancellation of global anomalies in the Type I superstring, with and without D-branes. Our argument here depends on a precise interpretation of the 2-form abelian gauge field using KO-theory; then the anomaly cancellation follows from a geometric form of the full Atiyah-Singer index theo…
Sketching accelerates structured prediction methods for large datasets.
Linear regression is a classic method of data analysis. In recent years, sketching -- a method of dimension reduction using random sampling, random projections, or both -- has gained popularity as an effective computational approximation when the number of observations greatly exceeds the number of variables. In this p…
Paper develops a bootstrap method for estimating sketched SVD errors.
Sketching techniques have become popular for scaling up machine learning algorithms by reducing the sample size or dimensionality of massive data sets, while still maintaining the statistical power of big data. In this paper, we study sketching from an optimization point of view: we first show that the iterative Hessia…
We present sketch-rnn, a recurrent neural network (RNN) able to construct stroke-based drawings of common objects. The model is trained on thousands of crude human-drawn images representing hundreds of classes. We outline a framework for conditional and unconditional sketch generation, and describe new robust training …
Sketching reduces data size for accurate spectral estimation.
Downsampling can improve generalization in ridgeless linear regression, especially with optimal sketching size.
A new data-oblivious sketch for logistic regression reduces data size while maintaining approximation accuracy.
Sketch-GNN reduces GNN training time and memory usage to sublinear scales.
This paper speeds up kernel methods using sparsified Gaussian sketches.
Newton-LESS sparsifies Gaussian sketching for faster optimization.
Recent work has explored transforming data sets into smaller, approximate summaries in order to scale Bayesian inference. We examine a related problem in which the parameters of a Bayesian model are very large and expensive to store in memory, and propose more compact representations of parameter values that can be use…
Recently, randomly mapping vectorial data to strings of discrete symbols (i.e., sketches) for fast and space-efficient similarity searches has become popular. Such random mapping is called similarity-preserving hashing and approximates a similarity metric by using the Hamming distance. Although many efficient similarit…
FetchSGD reduces communication in federated learning with sketching.
Iterative Hessian sketch (IHS) is an effective sketching method for modeling large-scale data. It was originally proposed by Pilanci and Wainwright (2016; JMLR) based on randomized sketching matrices. However, it is computationally intensive due to the iterative sketch process. In this paper, we analyze the IHS algorit…
We present a system for identifying conceptual shifts between visual categories, which will form the basis for a co-creative drawing system to help users draw more creative sketches. The system recognizes human sketches and matches them to structurally similar sketches from categories to which they do not belong. This …
Flexible method for estimating frequencies in large datasets using sketching.
We study sequential change-point detection procedures based on linear sketches of high-dimensional signal vectors using generalized likelihood ratio (GLR) statistics. The GLR statistics allow for an unknown post-change mean that represents an anomaly or novelty. We consider both fixed and time-varying projections, deri…
Generative adversarial network synthesizes sketches into realistic images.
Large-scale distributed training of neural networks is often limited by network bandwidth, wherein the communication time overwhelms the local computation time. Motivated by the success of sketching methods in sub-linear/streaming algorithms, we introduce Sketched SGD, an algorithm for carrying out distributed SGD by c…