DALES offers a large annotated aerial LiDAR dataset for 3D deep learning.
problem Lack of large-scale annotated aerial LiDAR datasets for deep learning.
method Collection and annotation of over half a billion hand-labeled points from an ALS scanner.
result DALES is the most extensive publicly available ALS data set with improved resolution and coverage.
A new method reduces the number of features needed for kernel approximation from cubic to logarithmic.
problem Large datasets make kernel methods computationally expensive and impractical.
method Combines random feature maps with data-dependent feature selection to achieve Nystrom-like performance with fewer features.
result Achieves small kernel matrix approximation error and better test set accuracy with fewer features than state-of-the-art methods.
Develops efficient methods for large-scale data estimation.
problem Parameter estimation in large-scale data settings.
method Stochastic gradient descent methods with implicit updates.
result Stochastic gradient descent methods are more efficient and robust for large data sets.
New algorithm speeds up causal inference for large data.
problem Efficiently estimating causal effects from large data.
method Causal Bag of Little Bootstraps (cBLOBS).
result Significantly improved computational efficiency with consistent estimates.
New method for clustering large multi-view data.
problem Handling large multi-view data efficiently.
method Incremental minimax optimization based fuzzy clustering (IminimaxFCM).
result IminimaxFCM outperforms related methods in clustering accuracy.
There has been a surge in the number of large and flat data sets - data sets containing a large number of features and a relatively small number of observations - due to the growing ability to collect and store information in medical research and other fields. Hierarchical clustering is a widely used clustering tool. I…
Paper analyzes Nyström and column-sampling methods for PCA of large data sets.
problem Computational and storage challenges in PCA for large data sets.
method Nyström and column-sampling methods for approximating PCA of large matrices.
result Comparison of methods and trade-off between accuracy and computational complexity.
New method estimates multivariate Gaussian fields using sparse precision matrix.
problem Estimating covariance matrices for large multivariate Gaussian fields.
method Sparse Precision Matrix Selection (SPS) algorithm for multivariate GRFs.
result Theoretical rates of convergence for estimated covariance and parameters validated.
New methods improve anomaly detection from large streaming data.
problem Anomaly detection in large streaming data fails with current methods.
method Two novel randomized algorithms (rPS and gPS) for better detection of correlated anomalies.
result High and balanced recall and estimated accuracy for anomaly detection.
New statistic κ-profile helps monitor weather, soundscapes, and dynamical systems.
problem Monitoring intrinsic dimensionality of large data sets.
method Optimization problem to find κ-profile, which is the norm of the shortest projected secant. result The κ-profile provides a useful statistic for understanding and monitoring large data sets. Variational inference algorithms have proven successful for Bayesian analysis in large data settings, with recent advances using stochastic variational inference (SVI). However, such methods have largely been studied in independent or exchangeable data settings. We develop an SVI algorithm to learn the parameters of hi…
Algebraic multigrid speeds up SVM training for large datasets.
problem Computational inefficiency of support vector machines on large-scale data.
method Inspired by algebraic multigrid, a fast multilevel framework for solving SVM models.
result Significant improvement in running time achieved without loss in quality.
Expands small recommendation datasets to industrial scale.
problem Disconnection between academic and industrial data scales.
method Randomized fractal expansions using Kronecker Graph Theory.
result Generated synthetic data sets with 1.2B ratings, 2.2M users, and 855K items.
Parallel divide-and-conquer method for analyzing large data.
problem Analyzing large datasets with lengthy variables.
method Sequential estimation, parallel divide-and-conquer, adaptive sample selection, adaptive shrinkage estimation.
result The proposed method accelerates estimation and identifies effective variables.
Study finds open data sets favor Western locales, impacting classifier performance.
problem Impact of biased open data sets on classifier performance in the developing world.
method Analysis of two large, publicly available image data sets and classifiers trained on them.
result Open data sets exhibit a bias towards Western locales, affecting classifier performance.
New method compresses large sample data for faster discriminant analysis.
problem Large sample sizes in discriminant analysis increase computational burden.
method Proposes a new compression approach for reducing training samples.
result Significant computational gains and superior predictive ability compared to random sub-sampling.
The paper speeds up Gaussian process regression for large datasets.
problem High computational costs in Gaussian process regression for large data sets.
method Derives simple results to speed up GPR learning and model comparison.
result Significant speed-up in Bayesian model comparison and evidence evaluation.
We propose a practical and scalable Gaussian process model for large-scale nonlinear probabilistic regression. Our mixture-of-experts model is conceptually simple and hierarchically recombines computations for an overall approximation of a full Gaussian process. Closed-form and distributed computations allow for effici…
Solving different types of optimization models (including parameters fitting) for support vector machines on large-scale training data is often an expensive computational task. This paper proposes a multilevel algorithmic framework that scales efficiently to very large data sets. Instead of solving the whole training s…
A scalable bootstrap method for Big Data analysis.
problem Statistical inference for large scale data sets.
method Bootstrap resamples from multiple disjoint subsets, coupled with a fixed-point estimation equation.
result Scalable, robust, and computationally efficient statistical inference for large data sets.
Fast, fully-automated histograms for large data sets.
problem Efficiently constructing histograms for large-scale data.
method G-Enum histograms using Minimum Description Length (MDL) principle for model selection and a greedy search heuristic.
result Construct histograms in linearithmic time, significantly faster than previous methods.
Hierarchical Softmax approximates class probabilities for large datasets efficiently.
problem Computational inefficiency of Softmax for large-scale classification tasks.
method Used Hierarchical Softmax to approximate class probabilities efficiently.
result Hierarchical Softmax performance degrades as the number of classes increases.
A new method for efficient Nystrom approximation for large datasets.
problem Generating low-rank approximations of kernel matrices for large-scale machine learning problems.
method Randomized K-means clustering on low-dimensional random projections of data.
result Significant savings in computational efficiency for high-dimensional data.
Construct non-trivial hyperbolic initial data sets.
problem Creating non-trivial hyperbolic initial data sets.
method Exotic gluings for asymptotically hyperbolic initial data sets.
result Direct construction of non-trivial initial data sets.
Divide-and-conquer method splits large data sets for efficient analysis.
problem Handling large data sets that exceed computational limits.
method Split data into smaller sets, analyze each separately, then combine results.
result Combined results provide statistical inference similar to analyzing entire data set.
Under the framework of spectral clustering, the key of subspace clustering is building a similarity graph which describes the neighborhood relations among data points. Some recent works build the graph using sparse, low-rank, and ℓ2-norm-based representation, and have achieved state-of-the-art performance. Howeve…
We give examples of asymptotically flat three-manifolds (M,g) which admit arbitrarily large constant mean curvature spheres that are far away from the center of the manifold. This resolves a question raised by G. Huisken and S.-T. Yau in 1996. On the other hand, we show that such surfaces cannot exist when (M,g) ha…
A new learning method uses data to learn from large model sets.
problem Learning with large sets of candidate models where uniform convergence is hard.
method Data-dependent learning that incorporates empirical data less reliant on prior assumptions.
result Demonstrates improved generalization in various learning assumptions.
Constructs foliations of critical surfaces for Hawking energy in asymptotically flat initial data sets.
problem Positivity and rigidity of Hawking quasi-local energy in asymptotically flat spacetimes.
method Lyapunov-Schmidt reduction within a Willmore-foliation framework.
result Existence and uniqueness of foliations by Hawking surfaces, positivity and large-sphere limit of Hawking energy.
Generates synthetic laparoscopic images for training deep neural networks.
problem Lack of large labeled data sets for laparoscopic image processing.
method Unpaired image-to-image translation to generate realistic synthetic images.
result Synthetic data set improves liver segmentation performance without manual labeling.
FCA2VEC embeds formal concept analysis data for large datasets.
problem Embedding formal concept analysis data for large datasets.
method Introducing fca2vec, a family of embedding techniques for formal concept analysis.
result Retrieves cover relation of a concept lattice from a computational feasible embedding.
IVF k-means algorithm improves performance on large sparse data sets.
problem Efficiently clustering large-scale sparse data sets with numerous classes.
method Sparse data representation and inverted-file structure for high-speed and low-memory clustering.
result IVF achieves better performance than other algorithms on real document data sets.
New algorithm PART improves EP-MCMC for large data sets.
problem Expensive MCMC for large data sets.
method Random partition trees for combining subset posterior draws.
result PART algorithm provides distribution-free, easy-to-resample, and scalable posterior approximations.
Localized SVMs maintain SVM's consistency properties for large datasets.
problem Inefficient computational requirements of global SVMs for large data sets.
method Localized SVMs apply different hyperparameters to different regions of the input space.
result Localized SVMs inherit Lp- and risk consistency from global SVMs. Paper introduces a noisy-labeled audio tagging challenge.
problem Acoustic mismatch and noisy labels in audio tagging.
method Large dataset with minimal supervision, convolutional neural network baseline.
result Demonstrates effectiveness of minimal supervision in noisy conditions.
Bayesian approach enhances SML for big data.
problem Feature discovery from large data sets.
method Bayesian decision theory applied to SML.
result Many SML techniques are connected to Bayesian inference.
Improved GP models for scalable large data sets.
problem Computational infeasibility of Gaussian process models for large datasets.
method Composite likelihood approach with recursive computation and hyper-parameter learning.
result The derived composite GP model provides accurate predictions and hyper-parameter learning.
AgEBO-Tabular combines NAS and hyperparameter tuning for fast, high-performing tabular models.
problem Developing high-performing predictive models for large tabular data sets is challenging.
method Combines aging evolution NAS and asynchronous Bayesian optimization for hyperparameter tuning in data-parallel training.
result Automatically discovered neural network models outperform state-of-the-art AutoML ensembles in inference speed by two orders of magnitude.
Wasserstein coresets improve data sparsification for continuous distributions.
problem Efficiently summarize large continuous data distributions for Bayesian inference.
method Introduce Wasserstein measure coresets, minimizing Wasserstein distance via stochastic gradient descent.
result Wasserstein coresets enable online handling of large data streams and improve inference and clustering performance.
Method predicts computational reproducibility of large population studies data analysis pipelines.
problem Difficulty in evaluating reproducibility of large population studies due to computational and storage requirements.
method Formulated as collaborative filtering process with constraints on training set construction.
result One sampling method, 'Random File Numbers (Uniform)', predicts reproducibility with good accuracy.
Proposes a method to learn distance metrics from uncertain data.
problem Challenges of learning distance metrics from large-scale data with uncertainty.
method Margin preserving metric learning framework to learn distance metric and latent examples simultaneously.
result The learned metric is robust to uncertainty and preserves large margin for original data.
Deep MF extracts hierarchical features from large data sets.
problem Mining complex, interleaved features in large data sets.
method Deep matrix factorization models and algorithms.
result Deep MF achieves outstanding performance on unsupervised tasks.
This paper reviews R packages for automating data analysis tasks.
problem Time-consuming Exploratory Data Analysis in large, noisy data sets.
method Systematic review of 12 R packages for autoEDA.
result Identifies automated tasks and areas for future development.
Suppose that two large, multi-dimensional data sets are each noisy measurements of the same underlying random process, and principle components analysis is performed separately on the data sets to reduce their dimensionality. In some circumstances it may happen that the two lower-dimensional data sets have an inordinat…
Mondrian Forests improve regression uncertainty for large datasets.
problem Lack of uncertainty estimates in large-scale regression.
method Extend Mondrian forests to non-parametric regression with hierarchical Gaussian prior.
result Mondrian Forests deliver better uncertainty estimates than approximate GPs.
sparsebn learns large Bayesian networks from high-dimensional data.
problem Learning graphical models from large, high-dimensional datasets with interventions.
method Focuses on scalability and consistency in high-dimensional settings, learning causal networks from data.
result Achieves the goal of learning a causal network from data.
New method infers causal factors from large-scale data without full graph reconstruction.
problem Inferring causal variables from large-scale systems without full causal graph reconstruction.
method Supervised learning on simulated data using a neural network and subsampled-ensemble inference.
result Efficiently identifies causal relationships in large-scale gene regulatory networks.
Efficient multi-label classifier handles missing labels and large datasets.
problem Handling large-scale datasets with many instances and labels, missing label assignments, label correlations, and unlabeled data.
method Non-linear embedding of label vectors using a stochastic approach to predict tail labels, handling missing labels, and exploiting unlabeled data.
result Our method outperforms state-of-the-art multi-label classifiers in prediction performance and training time.