We analyze coresets for regularized regression problems and propose a modified lasso that yields smaller coresets.
problem Analyzing coresets for regularized regression problems.
method Examined coresets for ridge regression and proposed a modified lasso problem.
result No coreset for regularized regression can be smaller than the unregularized version when reqs. Unified framework for accurate coresets in latent variable models and regularized regression.
problem Efficiently training models on large datasets.
method Unified framework for constructing accurate coresets for latent variable models and ℓp-regularized regression. result Unified framework reduces coreset size for latent variable models and ℓp-regularized regression. The paper develops coresets for panel data regression problems.
problem Efficiently summarize panel data regression problems.
method Introduced coreset construction for panel data regression problems using the Feldman-Langberg framework.
result Constructs coresets of size polynomial in 1/ε and number of parameters, independent of panel data size. Novel bounds for logistic regression coreset construction and feature selection.
problem Efficiently summarize and reduce logistic regression inputs.
method Feature space sketching for logistic regression.
result Tight bounds for coreset construction and feature selection.
Coresets are one of the central methods to facilitate the analysis of large data sets. We continue a recent line of research applying the theory of coresets to logistic regression. First, we show a negative result, namely, that no strongly sublinear sized coresets exist for logistic regression. To deal with intractable…
New coreset method for near-convex functions.
problem Efficiently approximating loss functions for machine learning.
method Generic framework for computing sensitivities of near-convex functions using f-SVD factorization.
result Significantly improved coresets for various machine learning models.
New research finds many coreset methods for logistic regression are not better than simple sampling.
problem Evaluation of coreset methods for reducing data size in logistic regression.
method Comparison of multiple coreset and optimal subsampling methods for logistic regression.
result Many coreset methods do not outperform simple uniform subsampling.
Paper defines class discrepancy for machine learning problems and provides coresets.
problem Addressing discrepancies in machine learning models.
method Defines class discrepancy, provides techniques for bounding discrepancy, and develops coresets and streaming sketches.
result Establishes coresets of size O(sqrt{d}/epsilon) for various machine learning problems.
New method creates coresets for deep neural networks efficiently.
problem Efficiently handling large data streams with limited resources.
method Cardinality-constrained bilevel optimization for deep neural networks.
result Demonstrated efficient generation of coresets for deep neural networks.
A new method for efficient Gaussian process regression reduces complexity and improves scalability.
problem Efficient Gaussian process regression for large datasets.
method Learnable coreset-based variational inference for Gaussian processes.
result CVGP reduces the dimensionality of the variational parameter search space to linear complexity.
A new framework for coreset selection in machine learning models.
problem Learning models under resource constraints.
method Formulates coreset selection as a bilevel optimization problem.
result Framework applies to any twice differentiable model, including neural networks.
New algorithm reduces ERM problem size while maintaining accuracy.
problem Empirical risk minimization problem size reduction.
method Adaptive Deterministic Uniform-Weight Trimming (ADUWT) algorithm.
result Uniform (1±ε) relative-error approximation for ERM objective. This paper introduces accurate coresets for basic problems.
problem Solving problems on a small set that approximates the full set.
method Simple and accurate coreset constructions for basic problems.
result Unified notation and comparison table for modern results.
We develop coresets for multiple ℓ_p regression problems, improving approximation sizes and efficiency.
problem Efficiently approximating multiple ℓ_p regression problems with coresets.
method Construct coresets of size sublinear in m for multiple ℓ_p regression, improving bounds for different p values.
result We construct coresets with size nearly optimal in d and independent of m for multiple ℓ_p regression.
The use of Bayesian methods in large-scale data settings is attractive because of the rich hierarchical models, uncertainty quantification, and prior specification they provide. Standard Bayesian inference algorithms are computationally expensive, however, making their direct application to large datasets difficult or …
We investigate coresets - succinct, small summaries of large data sets - so that solutions found on the summary are provably competitive with solution found on the full data set. We provide an overview over the state-of-the-art in coreset construction for machine learning. In Section 2, we present both the intuition be…
When faced with a data set too large to be processed all at once, an obvious solution is to retain only part of it. In practice this takes a wide variety of different forms, and among them "coresets" are especially appealing. A coreset is a (small) weighted sample of the original data that comes with the following guar…
Paper introduces robust, dynamic coreset for outlier-resilient learning problems.
problem Dealing with outliers in continuous-and-bounded learning problems.
method Proposes a novel robust coreset method for fully-dynamic environments.
result First robust and fully-dynamic coreset construction method for optimization problems.
New method for efficient maximum likelihood estimation of p-generalized probit regression.
problem Efficient estimation of p-generalized probit regression models. method Combining sketching techniques with importance subsampling to obtain a coreset.
result Maximum likelihood estimator can be approximated efficiently up to a factor of (1+ε) on large data. New algorithm tightens sensitivity bounds for coresets, reducing size by up to 10x.
problem Finding efficient coresets for least-mean-squares problems.
method Algorithm based on iterative sensitivity computation and reduction to higher-dimensional subspaces.
result Significantly smaller coresets with data-dependent sensitivity bounds.
Develops data subsampling techniques for Poisson regression models.
problem Efficiently approximating Poisson regression loss functions with coresets.
method Introduces coresets for Poisson regression with novel complexity parameters and domain shifting.
result Sublinear coresets exist for Poisson regression with 1±ε approximation guarantee. Develops coresets for scalable multivariate distribution estimation.
problem Handling large-scale data in non-parametric or semi-parametric regression and density estimation.
method Novel coreset construction for multivariate conditional transformation models (MCTMs).
result Substantial data reduction with high log-likelihood accuracy.
Uniform sampling of modest size is a coreset for regularized loss minimization.
problem Designing efficient algorithms for large data with restricted access.
method Sampling-based algorithms for regularized loss minimization problems.
result Uniform sample of modest size is a coreset for certain regularized loss minimization problems.
New method for scalable learning of IRT models from large datasets.
problem Efficiently learning latent variables in IRT models from large numbers of examinees and items.
method Leveraging logistic regression and coresets for scalable IRT training.
result Scalable learning of IRT models from large data is achieved.
New method reduces computational cost for Bayesian inference.
problem High computational cost in Bayesian inference.
method Coreset MCMC, a Markov chain targeting coreset posterior.
result Coreset MCMC provides higher quality posterior approximations and reduced computational cost.
New statistical framework for coresets in density estimation.
problem Improving computational efficiency in density estimation.
method Developed a statistical framework for coresets in nonparametric density estimation.
result Practical coreset kernel density estimators are near-minimax optimal.
This paper shows DPPs can outperform random coresets in machine learning tasks.
problem Building efficient coresets for machine learning models.
method Using determinantal point processes (DPPs) to construct coresets with provable improvements over random sampling.
result DPPs can provably outperform independently drawn coresets in terms of approximation of total loss.
CRAIG selects a coreset for faster machine learning training.
problem Efficiently training machine learning models with limited data.
method Develops CRAIG, a method to select a weighted subset (coreset) of training data.
result CRAIG achieves a speedup that is inversely proportional to the size of the subset.
Bayesian coresets improve scalable Bayesian inference.
problem Efficiently approximating posterior inference with a subset of data.
method Sparsity constrained optimization and accelerated optimization methods.
result Explicit convergence rate guarantees and superior performance compared to state-of-the-art.
We create coresets for Gaussian Mixture Models of any shape.
problem Approximating the negative log-likelihood of Gaussian Mixture Models with arbitrary shapes.
method Reduction to projective clustering problems, maintaining coresets for streaming and dynamic data.
result The coreset size is (logn)O(1)/ε2, and can be computed in near-linear time. Coreset, which is a summary of the original dataset in the form of a small weighted set in the same sample space, provides a promising approach to enable machine learning over distributed data. Although viewed as a proxy of the original dataset, each coreset is only designed to approximate the cost function of a specif…
Dependent MMD coresets help compare multiple related datasets.
problem Comparing multiple related datasets for insights into model generalization.
method Dependent MMD coresets for collections of datasets.
result Dependent MMD coresets facilitate comparison and understanding of multiple related datasets.
New framework for variational coresets simplifies Bayesian inference for complex models.
problem Efficient Bayesian inference for complex models like neural networks.
method Black-box variational inference for coresets that handle intractable posterior distributions.
result Principled application of variational coresets to Bayesian neural networks.
Polynomial sketch approximates functions of low-rank matrices efficiently.
problem Approximating element-wise functions of low-rank matrices without full access.
method Combining polynomial approximation and tensor sketch for monomials.
result Efficient algorithm with lower complexity than full matrix access.
Coherent uncertainty quantification is a key strength of Bayesian methods. But modern algorithms for approximate Bayesian posterior inference often sacrifice accurate posterior uncertainty estimation in the pursuit of scalability. This work shows that previous Bayesian coreset construction algorithms---which build a sm…
We develop coreset techniques for noisy clustering with provable guarantees.
problem Clustering with stochastic noise in datasets.
method Surrogate error metrics and coreset construction algorithm.
result Improved coreset size and better guarantees on true clustering cost.
Efficiently trains SVM models on large datasets using coreset technology.
problem Training large-scale SVM models efficiently on Big Data.
method Developed an algorithm to create a coreset, a small representative subset of data.
result Proved the size of coreset required for SVM models and showed its applicability to streaming data.
Bayesian coresets improved with random sampling and quasi-Newton optimization.
problem Efficiently approximate Bayesian posterior distributions for computationally expensive inference.
method Randomly select a subset of data points, then optimize weights using quasi-Newton method.
result First algorithm with high-probability KL divergence bound on coreset quality.
This work improves bounds on Bayesian coreset quality.
problem Limitations of existing theoretical analysis of Bayesian coresets.
method Develops general upper and lower bounds on KL divergence.
result Demonstrates flexibility of new theoretical bounds in various models.
Automated Bayesian coreset construction for scalable inference.
problem Designing scalable and theoretically sound automated inference algorithms.
method Sparsity-constrained variational inference within an exponential family.
result Automated Bayesian coreset construction with improved KL divergence to the exact posterior.
How can we train a statistical mixture model on a massive data set? In this work we show how to construct coresets for mixtures of Gaussians. A coreset is a weighted subset of the data, which guarantees that models fitting the coreset also provide a good fit for the original data set. We show that, perhaps surprisingly…
Algorithm constructs coresets for clustering time series data from Gaussian mixtures.
problem Clustering time series data from Gaussian mixtures with autocorrelations.
method Developed an efficient algorithm to construct coresets for the maximum likelihood objective.
result Size of the coreset is independent of N and k, polynomial in d, k, and 1/ε. This paper develops a coreset method for GNNs that speeds up training on large graphs.
problem Training Graph Neural Networks (GNNs) on large-scale graphs is computationally expensive.
method The paper proposes a spectral greedy coreset (SGGC) method that selects ego-graphs based on spectral embeddings.
result SGGC significantly speeds up GNN training on large graphs and outperforms other coreset methods.
COPS optimizes deep learning by selecting informative samples with uncertainty estimation.
problem Mitigating high costs in labeling and computational resources for deep learning.
method COPS (unCertainty based OPtimal Sub-sampling) selects data with input and output uncertainty for linear softmax regression.
result COPS outperforms baseline methods in deep learning tasks, minimizing expected loss.
Coresets are compact representations of data sets such that models trained on a coreset are provably competitive with models trained on the full data set. As such, they have been successfully used to scale up clustering models to massive data sets. While existing approaches generally only allow for multiplicative appro…
Designs efficient algorithms for online and sliding window models of subspace embeddings for all p.
problem Design efficient algorithms for online and sliding window models of subspace embeddings for all p.
method Develops nearly optimal ℓp subspace embeddings for all p∈(0,∞) in the online coreset and sliding window models. result First nearly optimal ℓp subspace embeddings for all p∈(0,∞) in the online coreset and sliding window models. The proliferation of large data sets and Bayesian inference techniques motivates demand for better data sparsification. Coresets provide a principled way of summarizing a large dataset via a smaller one that is guaranteed to match the performance of the full data set on specific problems. Classical coresets, however, n…
CO2 algorithm creates coresets for generic smooth divergences efficiently.
problem Efficiently creating coresets for generic smooth divergences.
method CO2 algorithm using functional Taylor expansion and maximum mean discrepancy minimization.
result Poly-logarithmically many data points suffice for Sinkhorn divergence approximation.