Memory-efficient learning for large-scale imaging systems.
problem Memory limitations in GPUs for real-world large-scale inverse problems.
method Exploits reversibility of network layers to enable data-driven design.
result Demonstrated on small-scale and large-scale real-world systems.
This work aims to create a large-scale model for critical care time series data.
problem Lack of large-scale datasets and distribution shifts in critical care time series data.
method Harmonized dataset creation and transfer learning research.
result Established a foundation for large-scale multi-variate time series models in critical care.
Proposes MamBO for efficient high-dimensional large-scale optimization.
problem High-dimensional and large-scale optimization problems in machine learning and simulation.
method Combines subsampling and subspace embeddings with model aggregation to address uncertainty in surrogate models.
result Improves robustness of Bayesian optimization algorithm and achieves superior performance.
Introduces resemblance structure for large scale geometry.
problem Defining similarity in large scale geometry.
method Axiomatizing the concept of resemblance for subsets of a set.
result Large scale resemblance structures can induce nearness and generalize large scale properties.
Efficient knockoffs for large-scale feature selection.
problem Large-scale feature selection problems.
method Gaussian model-X knockoffs with efficient methods for solving semidefinite programs.
result Efficient knockoffs can be generated with linear complexity in the dimension.
Paper proposes PPMM for fast estimation of large-scale OTM.
problem Estimation of large-scale optimal transport maps (OTM) is challenging due to the curse of dimensionality.
method Combines projection pursuit regression and sufficient dimension reduction to adaptively select projection directions.
result PPMM consistently estimates the most informative projection direction and weakly converges to the target OTM.
DistPre predicts traffic speeds efficiently for large networks.
problem Fine-grained, accurate speed prediction for large-scale transportation networks.
method Customizes LSTM models on a cluster, sharing trained models between detectors.
result Efficient and accurate fine-grained traffic-speed prediction.
We propose a practical and scalable Gaussian process model for large-scale nonlinear probabilistic regression. Our mixture-of-experts model is conceptually simple and hierarchically recombines computations for an overall approximation of a full Gaussian process. Closed-form and distributed computations allow for effici…
Adaptive regularization prevents overfitting in large-scale sparse feature models.
problem Overfitting in models with large-scale sparse categorical features.
method Adaptive regularization of embedding layers' norm budget.
result Improves model performance within a single epoch and prevents multi-epoch performance degradation.
A new TwinGP framework for efficient large-scale GP modeling.
problem Efficiently modeling large-scale Gaussian processes with computational constraints.
method Combines global and local approximations using a subset-of-data approach.
result TwinGP framework performs on par or better than state-of-the-art methods at a fraction of the computational cost.
Randomized spectral co-clustering speeds up large-scale directed networks.
problem Co-clustering directed networks efficiently for large-scale data.
method Randomized spectral co-clustering algorithms using random-projection and random-sampling techniques.
result Theoretical and numerical validation of approximation and misclustering error rates.
This paper analyzes convergence of large-scale Transformers with weight decay.
problem Understanding optimization guarantees in large-scale Transformer training.
method Construct mean-field limit, show gradient flow convergence to PDE, demonstrate global minimum consistency.
result Gradient flow reaches global minimum in large-scale Transformers with small weight decay.
Real time large scale streaming data pose major challenges to forecasting, in particular defying the presence of human experts to perform the corresponding analysis. We present here a class of models and methods used to develop an automated, scalable and versatile system for large scale forecasting oriented towards saf…
In order to scale standard Gaussian process (GP) regression to large-scale datasets, aggregation models employ factorized training process and then combine predictions from distributed experts. The state-of-the-art aggregation models, however, either provide inconsistent predictions or require time-consuming aggregatio…
MixKD improves large-scale language model compression and generalization.
problem Inefficient and resource-intensive large-scale language models.
method MixKD uses mixup data augmentation to enhance student model's generalization ability.
result MixKD leads to significant performance gains over standard KD and competitive baselines.
BayesDLL offers a PyTorch library for Bayesian deep learning with large models.
problem Bayesian inference for large-scale deep networks.
method Variational inference, MC-dropout, stochastic-gradient MCMC, Laplace approximation.
result BayesDLL can handle Vision Transformers and pre-trained model weights as priors.
New method adds interactions to interpretable models for large-scale data.
problem Limited model complexity and lack of interactions in interpretable models.
method Factorization method to derive scalable higher-order tensor product spline models.
result Incorporates all higher-order interactions of non-linear feature effects without computational penalties.
Method analyzes large-scale network data to detect communication pattern shifts.
problem Analyzing large-scale time-series network data is challenging.
method Temporal encoder embedding method using ground-truth or estimated vertex labels.
result Detects communication pattern shifts across all levels of network structure.
DALC customizes LSTM models for detectors in large-scale traffic networks.
problem Fine-grained traffic prediction for large-scale transportation networks.
method Formulated as a finite Markov decision process, introduced ALC algorithm for automatic customization, and developed DALC for distributed customization.
result DALC provides higher prediction accuracy than Apache Spark MLlib approaches.
Online method learns sparse models efficiently in large scale settings.
problem Sparse model learning in large scale settings with high computational and memory costs.
method Online learning approach, mini-batch methods, hard thresholding based stochastic gradient algorithm.
result Sparsity promoted by batch methods is not preserved in online fashion.
New method estimates bidirectional causal effects in large-scale systems.
problem Estimating bidirectional causal effects in systems with mutual dependence and heteroskedasticity.
method Heteroskedasticity-based identification with online kernel learning and random Fourier features.
result Superior accuracy and stability compared to single equation and polynomial approximations.
We study the concept of coarse disjointness and large scale n-to-1 functions. As a byproduct, we obtain an Ostrand-type characterization of asymptotic dimension for coarse structures. It is shown that properties like finite asymptotic dimension, coarse finitism, large scale weak paracompactness, ect. are all invari…
Large scale online kernel learning aims to build an efficient and scalable kernel-based predictive model incrementally from a sequence of potentially infinite data points. A current key approach focuses on ways to produce an approximate finite-dimensional feature map, assuming that the kernel used has a feature map wit…
Deep neural networks provide meaningful uncertainty estimates for large-scale simulations.
problem Uncertainty estimates for deep neural network predictions from large-scale simulations.
method General variational inference approach to calibrate Bayesian uncertainties.
result Calibrated Bayesian uncertainties preserved physics-correlations in predicted quantities.
Recently, pre-trained language representation flourishes as the mainstay of the natural language understanding community, e.g., BERT. These pre-trained language representations can create state-of-the-art results on a wide range of downstream tasks. Along with continuous significant performance improvement, the size an…
New algorithms estimate Jacobian matrices for large-scale machine learning.
problem Efficiently computing search directions for large nonlinear least squares.
method Exploit low-rank structure in Hessian to estimate Jacobian matrices.
result Two algorithms perform well compared to state-of-the-art methods.
The motivation for this paper is to apply Bayesian structure learning using Model Averaging in large-scale networks. Currently, Bayesian model averaging algorithm is applicable to networks with only tens of variables, restrained by its super-exponential complexity. We present a novel framework, called LSBN(Large-Scale …
FaStR improves scalability for time-aware RS with varying coefficients.
problem Limited applicability of structured regression models to large-scale data with categorical effects and many interactions.
method Combines structured additive regression and factorization approaches in a neural network-based model implementation.
result FaStR scales better and performs competitively with other time-aware RS in prediction performance.
For large scale on-line inference problems the update strategy is critical for performance. We derive an adaptive scan Gibbs sampler that optimizes the update frequency by selecting an optimum mini-batch size. We demonstrate performance of our adaptive batch-size Gibbs sampler by comparing it against the collapsed Gibb…
Efficiently solves large-scale robust portfolio optimization problems.
problem High computational demands in large-scale robust portfolio optimization.
method Extended supporting hyperplane approximation for distributionally robust portfolio problems.
result Significantly reduces computational time from several thousand seconds to just a few.
FibeRed reduces complex data dimensions while preserving topology.
problem Hard embedding of topologically complex datasets in low-dimensional Euclidean space.
method Modeling datasets with vector bundles, reducing fibers while preserving topology.
result FibeRed learns topologically faithful embeddings in lower dimensions than existing methods.
In this paper, we apply a mini-batch based negative sampling method to efficiently train a latent factor autoencoder model on large scale and sparse data for implicit feedback collaborative filtering. We compare our work against a state-of-the-art baseline model on different experimental datasets and show that this met…
CTR prediction in real-world business is a difficult machine learning problem with large scale nonlinear sparse data. In this paper, we introduce an industrial strength solution with model named Large Scale Piece-wise Linear Model (LS-PLM). We formulate the learning problem with L1 and L2,1 regularizers, leadin…
Large-scale Hierarchical Classification (HC) involves datasets consisting of thousands of classes and millions of training instances with high-dimensional features posing several big data challenges. Feature selection that aims to select the subset of discriminant features is an effective strategy to deal with large-sc…
We present Distributed Equivalent Substitution (DES) training, a novel distributed training framework for large-scale recommender systems with dynamic sparse features. DES introduces fully synchronous training to large-scale recommendation system for the first time by reducing communication, thus making the training of…
This paper surveys large-scale machine learning methods for efficient data analysis.
problem Efficiently processing large-scale data with machine learning models.
method Divided into three categories: model simplification, optimization approximation, and computation parallelism.
result Blueprint for future developments in large-scale machine learning.
New method infers causal factors from large-scale data without full graph reconstruction.
problem Inferring causal variables from large-scale systems without full causal graph reconstruction.
method Supervised learning on simulated data using a neural network and subsampled-ensemble inference.
result Efficiently identifies causal relationships in large-scale gene regulatory networks.
Transformers learn to predict chess moves with surprising accuracy and strength.
problem Training transformers on chess to predict moves accurately.
method Large-scale chess dataset (10M games), supervised learning with up to 270M parameters.
result Transformers can predict action-values for novel boards with high accuracy.
Augments graph node features to improve GNN performance.
problem Improving graph neural networks' performance on large-scale datasets.
method Iteratively augments node features with gradient-based adversarial perturbations.
result Boosts model performance in node classification, link prediction, and graph classification tasks.
Natural image statistics exhibit hierarchical dependencies across multiple scales. Representing such prior knowledge in non-factorial latent tree models can boost performance of image denoising, inpainting, deconvolution or reconstruction substantially, beyond standard factorial "sparse" methodology. We derive a large …
Large-scale Gaussian process inference has long faced practical challenges due to time and space complexity that is superlinear in dataset size. While sparse variational Gaussian process models are capable of learning from large-scale data, standard strategies for sparsifying the model can prevent the approximation of …
Efficiently trains deep Gaussian processes on large datasets.
problem Large-scale data and multi-scale features in function approximation.
method Combines variational learning with MCMC for efficient and accurate training.
result Highly efficient and accurate deep GP training on large-scale data.
A plethora of multi-view subspace clustering (MVSC) methods have been proposed over the past few years. Researchers manage to boost clustering accuracy from different points of view. However, many state-of-the-art MVSC algorithms, typically have a quadratic or even cubic complexity, are inefficient and inherently diffi…
Paper studies randomized spectral clustering for large-scale networks.
problem Computational challenges in large-scale network community detection.
method Randomized sketching algorithms for spectral clustering.
result Theoretical bounds for approximation, misclassification, and link probability estimation.
We consider the problem of personalization of online services from the viewpoint of ad targeting, where we seek to find the best ad categories to be shown to each user, resulting in improved user experience and increased advertisers' revenue. We propose to address this problem as a task of ranking the ad categories dep…
New method uses tensor decompositions to overcome the curse of dimensionality for large-scale learning.
problem Large-scale machine learning problems with kernel methods.
method Deterministic Fourier features combined with low-rank tensor decomposition for tensor product structure.
result Demonstrated consistent performance and superior results compared to random Fourier features.
Paper presents an efficient algorithm for learning minimax risk classifiers with large-scale data.
problem Efficient learning of minimax risk classifiers for large-scale data with multiple classes.
method Combination of constraint and column generation for efficient learning.
result 10x speedup for general large-scale data and 100x speedup with many classes.
Let M be a compact manifold. We show the identity component Homeo0(M) of the group of self-homeomorphisms of M has a well-defined quasi-isometry type, and study its large scale geometry. Through examples, we relate this large scale geometry to both the topology of M and the dynamics of group actions on M. T…