Paper proposes Nyström sketches for better adaptive compressive learning.
problem Improving adaptability of sketching for compressive learning.
method Data-dependent Nyström approximation for mean embedding.
result Excess risk can be controlled with geometric assumption.
Adaptive IHS improves sketching for large-scale data.
problem Efficiently modeling large-scale data with iterative Hessian sketch.
method Deterministic A-optimal subsampling for improved IHS.
result A-optimal IHS outperforms existing accelerated IHS methods.
Spatio-temporal data compression method reduces memory usage.
problem Efficiently storing and analyzing large spatio-temporal datasets.
method Adaptive sampling of tensor slices to compress and preserve structure.
result SkeTenSmooth outperforms other sampling methods in retaining patterns.
A parallel optimization method for convex functions using Hessian sketching and debiasing.
problem Massively parallel optimization of convex functions with limited communication.
method Newton method with Hessian sketching and debiasing by workers, server averages descent directions.
result Approximation of Newton step with low-complexity adaptive sketching scheme.
Flexible method for estimating frequencies in large datasets using sketching.
problem Estimating frequencies in very large datasets efficiently and accurately.
method Data-adaptive conformal inference method based on a smaller sketch of data.
result Proves validity of frequentist confidence intervals under data exchangeability.
Paper develops a bootstrap method for estimating sketched SVD errors.
problem Lack of tools for accurately estimating sketched SVD errors.
method Develops a fully data-driven bootstrap method for numerical error estimation.
result Allows users to adaptively predict extra work needed for desired error tolerance.
New method reduces second-order KOCO complexity with adaptive sketching.
problem Efficiently solving kernel online convex optimization problems with strong curvature.
method Kernel Online Newton Step (KONS) with adaptive matrix sketching.
result Achieves O(dextefflogT) regret with reduced space and time complexity. New algorithm reduces sketching dimension to effective problem size.
problem Solving L2-regularized least-squares problems efficiently.
method Randomized algorithm using Gaussian and SRHT embeddings.
result Preserves convergence guarantees with reduced embedding dimension.
Sketchy reduces memory and compute requirements for adaptive regularization in deep learning.
problem Prohibitive memory and running time for adaptive regularization methods in deep learning.
method Low-rank sketching approach using Frequent Directions (FD) to reduce memory and compute requirements.
result Efficient interpolation between resource requirements and degradation in regret guarantees with rank k. New method solves constrained optimization problems efficiently.
problem Equality-constrained nonlinear, nonconvex optimization problems.
method Adaptive inexact Newton method with randomized iterative sketching.
result Global almost sure convergence and local linear/superlinear convergence.
New algorithm converts data into sub-gaussian designs efficiently.
problem Efficiently converting large datasets into sub-gaussian random designs for robust performance.
method Algorithmic Gaussianization through sketching and averaging, using LESS embeddings.
result Efficient data sketches nearly indistinguishable from sub-gaussian designs.
Improved privacy-preserving linear regression via iterative Hessian mixing.
problem Differentially private linear regression with improved accuracy and efficiency.
method Iterative Hessian Mixing (IHM) for differentially private ordinary least squares (DP-OLS).
result IHM provides better utility guarantees and outperforms AdaSSP in empirical evaluations.
Compressive learning framework adapted for semi-parametric models.
problem Handling large datasets efficiently with semi-parametric models.
method Reformulate compressive learning framework to handle semi-parametric models, capturing their inherent topology and structure.
result Demonstrated robustness and efficiency of the framework in independent component analysis and subspace clustering.
Sketch-BERT learns vector sketches using BERT-like self-supervised learning.
problem Lack of effective vector sketch representation for recognition and retrieval tasks.
method Generalized BERT to sketch domain with novel embedding networks and self-supervised sketch gestalt learning.
result Improved performance on sketch recognition, retrieval, and gestalt tasks.
Sketching reduces memory and computation for large datasets.
problem Efficiently learning parameters from large data sets.
method Compressive learning framework using sketching of data moments.
result Sketching reduces memory and computation for GMM estimation.
New method solves constrained stochastic optimization problems efficiently.
problem Online statistical inference of constrained stochastic nonlinear optimization problems.
method Stochastic Sequential Quadratic Programming (StoSQP) with iterative sketching solver.
result The rescaled primal-dual sequence converges to a mean-zero Gaussian distribution.
Adaptive algorithms improve cost-sensitive online classification with second-order information.
problem Improving cost-sensitive online classification with second-order information.
method Proposes adaptive regularization algorithms with sketching technique for better trade-off between performance and efficiency.
result Empirically validated algorithms' effectiveness and properties in real-world anomaly detection tasks.
SRS samples data points based on their proximity to randomly sampled points on the unit sphere.
problem Random sampling does not always preserve data structures and may not adequately sample from less-populated clusters.
method Spatial Random Sampling (SRS) samples points based on their proximity to randomly sampled points on the unit sphere, with sampling probability proportional to surface area.
result SRS provides descriptive and balanced data representations, addressing the need for better data sketches.
Skeinformer accelerates self-attention for long sequences with linear complexity.
problem Efficiency of Transformer models in processing long sequences.
method Matrix sketching and column sampling to reduce quadratic complexity to linear.
result Skeinformer outperforms alternatives with smaller time/space footprint.
Sketched Ridge Regression improves solutions but increases risk; model averaging mitigates this.
problem Approximating solutions to Matrix Ridge Regression using sketching methods.
method Classical and Hessian sketching applied to MRR, with model averaging.
result Sketched MRR solutions have higher risks than optimal solutions, but model averaging reduces this gap.
A new method improves convergence in low-rank approximation.
problem Efficiently solving large-scale numerical linear algebra problems.
method Error-Powered Sketched Inverse Iteration (EPSI) Method.
result Convergence rate improves at least linearly with sketch size.
Recursive sketches summarize deep networks, aiding quick analysis and learning.
problem Understanding and analyzing complex deep learning models.
method Developed a recursive sketch mechanism to summarize inputs and outputs of modular deep networks.
result Sketches can identify key components and summarize essential information, even if partially erased.
Sketches linear classifiers using Weight-Median Sketch for efficient data stream analysis.
problem Efficiently learning and analyzing data streams with limited memory.
method Introduces Weight-Median Sketch for compressed linear classifier learning over data streams.
result Memory-limited execution of various analyses over streams, including feature selection and mutual information estimation.
Sketch-rnn generates coherent drawings from crude sketches.
problem Creating detailed drawings from simple sketches.
method Recurrent Neural Network (RNN) trained on human-drawn images.
result Generates coherent vector drawings from crude sketches.
Localized sketching improves matrix multiplication and ridge regression complexity.
problem Efficiently approximate matrix multiplication and ridge regression with limited data availability.
method Localized sketching matrices for block diagonal structure, reducing sample complexity.
result Localized sketching achieves sample complexity matching global sketching methods.
The paper sharpens the analysis of sketch-and-project methods using randomized singular value decomposition.
problem Improving convergence rates of sketch-and-project methods for solving linear systems and non-linear optimization problems.
method Developing a theoretical framework and new spectral bounds for the expected sketched projection matrix.
result The convergence rate improves linearly with sketch size and even faster with certain spectral decays.
New method reduces linear regret in high-dimensional bandit problems.
problem Heavy spectral tails in streaming matrices lead to linear regret in sketch-based linear bandits.
method Dyadic Block Sketching, a multi-scale matrix sketching approach.
result Achieves sublinear regret bounds without prior knowledge of streaming matrix properties.
New bounds on random quadratic forms hold under dependence, useful for adaptive modeling.
problem Need for independence in bounds on random quadratic forms.
method Uniform bounds on random quadratic forms of conditionally independent and sub-Gaussian stochastic processes.
result Bounds hold under general dependencies and sequential design.
Paper proposes GAN architectures for generating vector sketches.
problem Lack of GAN architectures for generating vector sketches.
method Proposes SkeGAN and VASkeGAN architectures for vector sketch generation.
result Validated that proposed architectures generate visually appealing sketches.
New guarantees for asymmetric sketching in compressive learning.
problem Statistical guarantees for compressive learning with asymmetric feature maps.
method Proves existing guarantees carry over to asymmetric scheme with LPD property, applies to quantized sketches.
result Existing statistical guarantees for compressive learning extend to asymmetric schemes with controlled error.
A new sketching method reduces tensor memory usage and enables efficient tensor operations.
problem Efficiently compressing and retaining tensor structure in large datasets.
method Higher-order Count Sketch (HCS) using multiple hash functions and tensor products.
result HCS achieves significant memory savings and efficient tensor operations.
Randomized algorithm solves vector-valued regression problems with low-rank operators.
problem Vector-valued regression problems involving infinite-dimensional spaces.
method Randomized Reduced Rank Regression (R4) using Gaussian sketching for optimization.
result R4 estimators are efficient and accurate, with empirical risk close to optimal.
New analysis proves sketching operators' RIP guarantees for mixture models without importance sampling.
problem Proving sketching operators' Restricted Isometry Property (RIP) for mixture models without assuming importance sampling.
method Proposed alternative analysis based on new deterministic bounds and concentration inequalities.
result Theoretical guarantees for sketching operators without importance sampling.
A fast sketching algorithm solves regularized least squares problems efficiently.
problem Solving large-scale optimization problems with convex or nonconvex regularization.
method Sketching for Regularized Optimization (SRO) algorithm that generates a sketch of the original data matrix and solves the sketched problem.
result General theoretical results for the approximation error between the original and sketched problems, including minimax rates for sparse signal estimation.
Sketching reduces memory usage for large Bayesian models.
problem Large parameter values in Bayesian models are expensive to store.
method Combines count-min sketch and approximate counters for efficient parameter representation.
result Sketches used during inference converge to exact chain equilibrium.
Projector-based approach quantifies uncertainties in sketched linear regression.
problem How sketching affects statistical properties of linear regression solutions.
method Projector-based approach to sketched linear regression that is exact and requires minimal assumptions.
result Derives key quantities from classic linear regression that account for combined uncertainties.
System identifies shifts in sketches for creative drawing.
problem Helping users create more creative sketches.
method Recognizes conceptual shifts between visual categories.
result Produces ambiguous sketches blending features from different categories.
New algorithm reduces big data processing time by sketching and random projection.
problem Efficiently processing large and high-dimensional data sets.
method Developed a new algorithm combining sketching and dual random projection, using preconditioned conjugate gradient.
result The algorithm can recover the optimum of the original problem up to arbitrary precision with a logarithmic number of small-scale solver calls.
Private sketches protect linear regression data privacy.
problem Protecting sensitive information in linear regression.
method Release private sketches of datasets, compute approximate solutions.
result Private sketches maintain good approximation guarantees to the original problem.
A new method for estimating large-scale linear models with improved precision.
problem Estimating large-scale linear statistical models efficiently.
method Sequential Least-Squares Estimators with Fast Randomized Sketching (SLSE-FRS), integrating Sketch-and-Solve and Iterative-Sketching methods.
result SLSE-FRS produces high-precision estimators, outperforming state-of-the-art methods.
Sketched SGD reduces communication in distributed SGD by sketching gradients.
problem Limited communication in large-scale distributed training of neural networks.
method Introducing Sketched SGD, an algorithm that communicates sketches of gradients instead of full gradients.
result Sketched SGD reduces communication from O(d) or O(W) to O(logd), achieving up to 40x reduction in total communication cost. DiPS learns to optimize sketching policies for better recommendation quality.
problem Optimizing sketching policies for long-term user interest prediction in recommender systems.
method Differentiable policy for sketching that learns from training data.
result DiPS requires up to 50% fewer sketch items to achieve the same recommendation quality.
Density sketches summarize data distributions for accurate sampling and estimation.
problem Accurately estimating and sampling from complex data distributions.
method Online algorithm generating additive density sketches.
result Density sketches provide statistically sound estimators and sampling capabilities.
Improved sketching for logistic and ℓ1 regression with near-linear dimensions.
problem Efficiently approximate ℓ1 and logistic regression problems. method New sketching techniques achieving near-linear dimensions for both problems.
result Achieved near-linear sketching dimensions for ℓ1 and logistic regression. Unified bounds for sketched bilinear forms in machine learning and statistics.
problem Uniform bounds on sketched bilinear forms for modern analyses.
method Generic chaining and new techniques for handling suprema over pairs of sets.
result Improved convergence bounds for sketched Federated Learning and bandit algorithms.
We improve prediction risk estimation for large datasets using sketching and ridge regression.
problem Estimating prediction risks for large datasets efficiently and accurately.
method Random matrix theory, generalized cross validation, sketched ridge regression ensembles, and ensemble trick.
result Consistent risk estimation and prediction intervals for large-scale datasets.
Sketching accelerates structured prediction methods for large datasets.
problem Scaling surrogate kernel methods for large datasets.
method Sketching approximations applied to input and output feature maps.
result Achieves close-to-optimal rates with reduced sketch size.
Sketching improves change-point detection in high-dimensional data.
problem Sequential detection of changes in high-dimensional signals.
method Linear sketches of signal vectors combined with GLR statistics.
result Sketching can maintain performance even with time-varying projections.