Bayesian coresets improve scalable Bayesian inference.
problem Efficiently approximating posterior inference with a subset of data.
method Sparsity constrained optimization and accelerated optimization methods.
result Explicit convergence rate guarantees and superior performance compared to state-of-the-art.
New method reduces computational cost for Bayesian inference.
problem High computational cost in Bayesian inference.
method Coreset MCMC, a Markov chain targeting coreset posterior.
result Coreset MCMC provides higher quality posterior approximations and reduced computational cost.
Coherent uncertainty quantification is a key strength of Bayesian methods. But modern algorithms for approximate Bayesian posterior inference often sacrifice accurate posterior uncertainty estimation in the pursuit of scalability. This work shows that previous Bayesian coreset construction algorithms---which build a sm…
This work improves bounds on Bayesian coreset quality.
problem Limitations of existing theoretical analysis of Bayesian coresets.
method Develops general upper and lower bounds on KL divergence.
result Demonstrates flexibility of new theoretical bounds in various models.
New framework for variational coresets simplifies Bayesian inference for complex models.
problem Efficient Bayesian inference for complex models like neural networks.
method Black-box variational inference for coresets that handle intractable posterior distributions.
result Principled application of variational coresets to Bayesian neural networks.
Bayesian coresets improved with random sampling and quasi-Newton optimization.
problem Efficiently approximate Bayesian posterior distributions for computationally expensive inference.
method Randomly select a subset of data points, then optimize weights using quasi-Newton method.
result First algorithm with high-probability KL divergence bound on coreset quality.
The use of Bayesian methods in large-scale data settings is attractive because of the rich hierarchical models, uncertainty quantification, and prior specification they provide. Standard Bayesian inference algorithms are computationally expensive, however, making their direct application to large datasets difficult or …
The automation of posterior inference in Bayesian data analysis has enabled experts and nonexperts alike to use more sophisticated models, engage in faster exploratory modeling and analysis, and ensure experimental reproducibility. However, standard automated posterior inference algorithms are not tractable at the scal…
New method reduces Bayesian inference time by optimizing Hamiltonian flows.
problem Efficiently constructing a Bayesian coreset with low inferential error.
method Sparse Hamiltonian flows, involving uniform subsampling and momentum quasi-refreshment steps.
result Exponential compression of the dataset and reduced KL divergence to the target.
The proliferation of automated inference algorithms in Bayesian statistics has provided practitioners newfound access to fast, reproducible data analysis and powerful statistical models. Designing automated methods that are also both computationally scalable and theoretically sound, however, remains a significant chall…
In this paper we offer a preliminary study of the application of Bayesian coresets to network security data. Network intrusion detection is a field that could take advantage of Bayesian machine learning in modelling uncertainty and managing streaming data; however, the large size of the data sets often hinders the use …
We investigate coresets - succinct, small summaries of large data sets - so that solutions found on the summary are provably competitive with solution found on the full data set. We provide an overview over the state-of-the-art in coreset construction for machine learning. In Section 2, we present both the intuition be…
The proliferation of large data sets and Bayesian inference techniques motivates demand for better data sparsification. Coresets provide a principled way of summarizing a large dataset via a smaller one that is guaranteed to match the performance of the full data set on specific problems. Classical coresets, however, n…
This paper introduces Bayes Hilbert spaces for efficient posterior approximation.
problem Efficient posterior approximation in Bayesian models for large datasets.
method Develops Bayes Hilbert spaces for posterior approximation and connects them to Bayesian coresets and kernel-based distances.
result Bayes Hilbert spaces provide a novel framework for posterior approximation that is computationally efficient.
Novel MBRL method for large-scale RL with reduced posterior complexity.
problem Theoretical guarantees for MBRL in large spaces with complex models.
method Kernelized Stein Discrepancy for compression of posterior estimate.
result Sublinear Bayesian regret and up to 50% reduction in training time.
We analyze coresets for regularized regression problems and propose a modified lasso that yields smaller coresets.
problem Analyzing coresets for regularized regression problems.
method Examined coresets for ridge regression and proposed a modified lasso problem.
result No coreset for regularized regression can be smaller than the unregularized version when reqs. New statistical framework for coresets in density estimation.
problem Improving computational efficiency in density estimation.
method Developed a statistical framework for coresets in nonparametric density estimation.
result Practical coreset kernel density estimators are near-minimax optimal.
This paper shows DPPs can outperform random coresets in machine learning tasks.
problem Building efficient coresets for machine learning models.
method Using determinantal point processes (DPPs) to construct coresets with provable improvements over random sampling.
result DPPs can provably outperform independently drawn coresets in terms of approximation of total loss.
Method scales Bayesian inference to large datasets and robustifies against outliers.
problem Scalability and robustness to outliers in Bayesian inference.
method Variational inference with β-divergence and Riemannian coresets. result Efficiently constructs cleansed data summaries robust to outliers.
Unified framework for accurate coresets in latent variable models and regularized regression.
problem Efficiently training models on large datasets.
method Unified framework for constructing accurate coresets for latent variable models and ℓp-regularized regression. result Unified framework reduces coreset size for latent variable models and ℓp-regularized regression. Machine learning techniques improve Bayesian computation for complex data.
problem Infeasible posterior computation in high-dimensional models.
method Improving posterior computation using machine learning techniques.
result Potential to enhance Bayesian computation efficiency.
New method creates coresets for deep neural networks efficiently.
problem Efficiently handling large data streams with limited resources.
method Cardinality-constrained bilevel optimization for deep neural networks.
result Demonstrated efficient generation of coresets for deep neural networks.
Paper introduces robust, dynamic coreset for outlier-resilient learning problems.
problem Dealing with outliers in continuous-and-bounded learning problems.
method Proposes a novel robust coreset method for fully-dynamic environments.
result First robust and fully-dynamic coreset construction method for optimization problems.
New coreset method for near-convex functions.
problem Efficiently approximating loss functions for machine learning.
method Generic framework for computing sensitivities of near-convex functions using f-SVD factorization.
result Significantly improved coresets for various machine learning models.
Coreset, which is a summary of the original dataset in the form of a small weighted set in the same sample space, provides a promising approach to enable machine learning over distributed data. Although viewed as a proxy of the original dataset, each coreset is only designed to approximate the cost function of a specif…
Dependent MMD coresets help compare multiple related datasets.
problem Comparing multiple related datasets for insights into model generalization.
method Dependent MMD coresets for collections of datasets.
result Dependent MMD coresets facilitate comparison and understanding of multiple related datasets.
The paper develops coresets for panel data regression problems.
problem Efficiently summarize panel data regression problems.
method Introduced coreset construction for panel data regression problems using the Feldman-Langberg framework.
result Constructs coresets of size polynomial in 1/ε and number of parameters, independent of panel data size. An ε-coreset for a given set D of n points, is usually a small weighted set, such that querying the coreset \emph{provably} yields a (1+ε)-factor approximation to the original (full) dataset, for a given family of queries. Using existing techniques, coresets can be maintained for streaming, …
We develop coreset techniques for noisy clustering with provable guarantees.
problem Clustering with stochastic noise in datasets.
method Surrogate error metrics and coreset construction algorithm.
result Improved coreset size and better guarantees on true clustering cost.
How can we train a statistical mixture model on a massive data set? In this work we show how to construct coresets for mixtures of Gaussians. A coreset is a weighted subset of the data, which guarantees that models fitting the coreset also provide a good fit for the original data set. We show that, perhaps surprisingly…
This paper defines the notion of class discrepancy for families of functions. It shows that low discrepancy classes admit small offline and streaming coresets. We provide general techniques for bounding the class discrepancy of machine learning problems. As corollaries of the general technique we bound the discrepancy …
Novel bounds for logistic regression coreset construction and feature selection.
problem Efficiently summarize and reduce logistic regression inputs.
method Feature space sketching for logistic regression.
result Tight bounds for coreset construction and feature selection.
Algorithm constructs coresets for clustering time series data from Gaussian mixtures.
problem Clustering time series data from Gaussian mixtures with autocorrelations.
method Developed an efficient algorithm to construct coresets for the maximum likelihood objective.
result Size of the coreset is independent of N and k, polynomial in d, k, and 1/ε. This paper develops a coreset method for GNNs that speeds up training on large graphs.
problem Training Graph Neural Networks (GNNs) on large-scale graphs is computationally expensive.
method The paper proposes a spectral greedy coreset (SGGC) method that selects ego-graphs based on spectral embeddings.
result SGGC significantly speeds up GNN training on large graphs and outperforms other coreset methods.
Coresets are compact representations of data sets such that models trained on a coreset are provably competitive with models trained on the full data set. As such, they have been successfully used to scale up clustering models to massive data sets. While existing approaches generally only allow for multiplicative appro…
CO2 algorithm creates coresets for generic smooth divergences efficiently.
problem Efficiently creating coresets for generic smooth divergences.
method CO2 algorithm using functional Taylor expansion and maximum mean discrepancy minimization.
result Poly-logarithmically many data points suffice for Sinkhorn divergence approximation.
We present an efficient coreset construction algorithm for large-scale Support Vector Machine (SVM) training in Big Data and streaming applications. A coreset is a small, representative subset of the original data points such that a models trained on the coreset are provably competitive with those trained on the origin…
FAST selects coresets more efficiently by matching distributions in the frequency domain.
problem Efficiently selecting representative subsets of large datasets for deep learning.
method FAST uses spectral graph theory and CFD to match distributions, addressing limitations of existing methods.
result FAST significantly outperforms state-of-the-art coreset selection methods in accuracy and energy efficiency.
New method for multi-modal depth prediction challenges.
problem Challenges in applying unimodal coreset selection to multi-modal data.
method Adapted state-of-the-art coreset selection technique for multimodal data.
result Challenges in extending unimodal algorithms to multi-modal scenarios.
A coreset (or core-set) of an input set is its small summation, such that solving a problem on the coreset as its input, provably yields the same result as solving the same problem on the original (full) set, for a given family of problems (models, classifiers, loss functions). Over the past decade, coreset constructio…
Efficiently factorize tensors in streaming data with coreset selection.
problem Efficiently factorize tensors in streaming data.
method Online filtering and kernelization techniques to select a coreset of vectors.
result CP decomposition of coreset approximates full data tensor decomposition.
A new framework for coreset selection in machine learning models.
problem Learning models under resource constraints.
method Formulates coreset selection as a bilevel optimization problem.
result Framework applies to any twice differentiable model, including neural networks.
We develop a coreset for robust geometric median, reducing size dependency on outliers.
problem Robust geometric median problem in Euclidean space with outliers.
method Construction of a compact coreset with size dependency on m eliminated. result Elimination of O(m) dependency in coreset size, achieving O(ε−2⋅min{ε−2,d}) size. Coresets are one of the central methods to facilitate the analysis of large data sets. We continue a recent line of research applying the theory of coresets to logistic regression. First, we show a negative result, namely, that no strongly sublinear sized coresets exist for logistic regression. To deal with intractable…
This paper improves coreset size via smoothed analysis.
problem Efficiently computing small subsets that approximate query errors.
method Smoothed analysis for approximate average error over queries.
result Deterministic and randomized algorithms for smaller coresets.
When faced with a data set too large to be processed all at once, an obvious solution is to retain only part of it. In practice this takes a wide variety of different forms, and among them "coresets" are especially appealing. A coreset is a (small) weighted sample of the original data that comes with the following guar…
A new method compresses large datasets for gradient descent algorithms efficiently.
problem Efficiently compress large-scale datasets for gradient descent algorithms.
method Proposes a novel sequential coreset framework based on gradient descent locality.
result Significantly reduces computational complexity and running time.
New research finds many coreset methods for logistic regression are not better than simple sampling.
problem Evaluation of coreset methods for reducing data size in logistic regression.
method Comparison of multiple coreset and optimal subsampling methods for logistic regression.
result Many coreset methods do not outperform simple uniform subsampling.