Scalable training of inference networks for Gaussian-process models.
problem Computational challenges in inference for large data in Gaussian process models.
method Stochastic inference networks with a functional mirror-descent algorithm for minibatch training.
result Empirical results show comparable and sometimes superior performance to existing sparse variational GP methods.
Paper addresses statistical efficiency and scalability in tensor train decomposition.
problem Statistical inefficiency and scalability issues in tensor train decomposition.
method Introduces a convex relaxation and alternating optimization method with randomization.
result Derives error bounds and demonstrates method's performance on real data.
New framework scales graph AE and VAE by training on a subset of nodes.
problem Training scalability and speed issues in graph AE and VAE models.
method Utilizes graph degeneracy to train on a dense subset of nodes, with a propagation mechanism.
result Empirically competitive results on large graphs (millions of nodes and edges).
Improved scalability and interpretability in training data attribution.
problem Identifying which training data drives specific behaviors, especially unintended ones.
method Leveraging interpretable structures within the model to attribute model behavior to semantic directions, not individual test examples.
result Simple probe-based attribution methods are first-order approximations of Concept Influence that achieve comparable performance while being over an order-of-magnitude faster.
Improved GP models for fast training and good performance.
problem Training scalable Gaussian process models efficiently.
method Cross-validation and nearest neighbor truncation for scalable GP training.
result Our method offers fast training and excellent predictive performance.
A scalable method for training prediction models in predict-then-optimize
problem Training prediction models in the predict-then-optimize paradigm
method Decision-focused learning pipeline
result Decision quality competitive with state-of-the-art methods while reducing training time
SWALP averages SGD iterates for low-precision training, improving scalability and performance.
problem Improving scalability and performance in low-precision training.
method Averages low-precision SGD iterates with a modified learning rate schedule.
result SWALP matches full-precision SGD performance with 8-bit quantization and converges to optimal solutions.
TensorHyper-VQC improves VQC scalability and robustness.
problem Scalability and noise sensitivity in VQC.
method Tensor-train-guided hypernetwork framework.
result TensorHyper-VQC achieves superior performance and robust noise tolerance.
Probabilistic SSMs are improved with a novel model and scalable training.
problem Training fully probabilistic SSMs is hard.
method Proposed a novel model formulation and scalable training algorithm based on variational inference and Gaussian processes.
result The model allows fully capturing latent state temporal correlations, leading to robust training.
This paper introduces a neural sampler for scalable sampling from complex distributions.
problem Efficiently sampling from high-dimensional un-normalized distributions.
method Neural implicit sampler trained with KL and Fisher divergence methods.
result The neural sampler generates large batches of samples with low computational costs.
This paper shows using classification instead of regression improves deep RL scalability.
problem Challenges in training value functions for large networks in deep RL.
method Used categorical cross-entropy loss instead of mean squared error regression.
result Significant improvements in performance and scalability across various domains.
GRAM generates scalable graphs with a novel attention mechanism.
problem Scalability in graph generation for large datasets.
method GRAM uses a graph attention mechanism to generate scalable graphs.
result GRAM outperforms baseline methods in scalability and quality.
ShadowSync separates background synchronization for scalable distributed training.
problem Reducing synchronization overhead in distributed training for high scalability.
method Separates synchronization from training and runs it in the background.
result Achieves both high throughput and excellent model quality at scale.
Meta-learning interpretable decision trees with synthetic data.
problem Lack of efficient, scalable methods for generating synthetic data for decision tree meta-learning.
method Synthetic generation of near-optimal decision trees using the MetaTree transformer architecture.
result Meta-learning of decision trees achieves performance comparable to real-world data or optimal decision trees, with significant computational cost reduction.
New parallel GCN techniques boost graph embedding scalability and efficiency.
problem Scalability issues in graph embedding models.
method Parallelization techniques for graph sampling and feature propagation.
result Superior scalable performance on large graphs without accuracy loss.
Improved scalable training for FHVAE models.
problem Original FHVAE training is not scalable to large datasets.
method Proposed hierarchical sampling training algorithm.
result Improved scalability to datasets of thousands of hours.
Efficiently trains GPs with billions of inducing inputs using Tensor Train decomposition.
problem Training GPs with large numbers of inducing inputs.
method Tensor Train decomposition for variational parameters in stochastic variational inference.
result Achieves state-of-the-art results on several benchmarks.
Bayesian tensor train kernel machine uses Laplace approximation for scalable GP regression.
problem Scalability limitations of Gaussian process regression.
method Bayesian tensor train kernel machine with Laplace approximation and variational inference.
result VI replaces cross-validation and offers up to 65x faster training.
Our system trains ImageNet in 6.6 minutes using 2048 Tesla P40 GPUs.
problem Training large-scale deep neural networks efficiently and accurately.
method Mixed-precision training, extremely large mini-batch size optimization, and optimized all-reduce algorithms.
result Trains ResNet-50 to 75.8% top-1 test accuracy in 6.6 minutes.
COPML framework securely trains models across multiple data owners without revealing individual data.
problem Privacy-preserving collaborative machine learning with multiple data owners.
method Securely encodes data, distributes computation, performs distributed training.
result Achieves up to 16x speedup in training time while maintaining strong privacy.
WMRB improves ranking accuracy and efficiency in scalable batch training.
problem Improving ranking accuracy and efficiency in large-scale recommendation systems.
method WMRB uses a new rank estimator and an efficient batch training algorithm.
result WMRB consistently outperforms WARP and other baselines in three item recommendation tasks.
SySCD improves SCD scalability and speeds up training.
problem Scalability issues in parallel SCD algorithms.
method Developed a system-aware parallel SCD algorithm (SySCD) to avoid bottlenecks.
result Offers up to x42 speedup compared to state-of-the-art GLM solvers.
Graphical heuristic reduces and partitions large datasets for faster supervised training.
problem Training large datasets for classification tasks.
method Clustering and information graph construction for dataset reduction and partitioning.
result Significant speed-up in training run-time without compromising prediction accuracy.
Decentralized Gaussian processes for multi-agent learning.
problem Training and prediction in multi-agent systems.
method Decentralized ADMM for GP hyper-parameter training and iterative consensus for prediction.
result Subset of agents can perform predictions using covariance-based nearest neighbor selection.
Paper proposes MMI-ALI for scalable joint distribution matching across multiple domains.
problem Scalability issue in matching joint distributions across multiple domains.
method Adversarial training with Multivariate Mutual Information maximization.
result MMI-ALI achieves scalable joint distribution matching across multiple domains.
A scalable ROC-SVM variant reduces training time for imbalanced binary classification.
problem High computational cost of ROC-SVM for imbalanced binary classification.
method Incomplete U-statistics and low-rank kernel approximation.
result Comparable AUC performance with reduced training time.
Exact GPs trained on over a million points in under 2 hours.
problem Computational limitations of exact Gaussian processes for large datasets.
method Multi-GPU parallelization and linear conjugate gradients for kernel matrix multiplication.
result Exact GPs can be trained on over a million points in less than 2 hours.
Scalable Gaussian process models trained with unbiased stochastic ELBO.
problem Training large capacity Gaussian process models on huge datasets.
method Unbiased stochastic variational inference for scalable GPs.
result Accurate inference on large datasets with up to 10 million basis functions.
New algorithm improves Gaussian process hyperparameter tuning for large datasets.
problem Scalable hyperparameter tuning for Gaussian processes on large datasets.
method Estimates smoothness and length-scale parameters in Matern kernel using novel loss functions.
result Improved uncertainty quantification over traditional methods.
We propose a distributed approach to train deep neural networks (DNNs), which has guaranteed convergence theoretically and great scalability empirically: close to 6 times faster on instance of ImageNet data set when run with 6 machines. The proposed scheme is close to optimally scalable in terms of number of machines, …
Asynch-SGBDT speeds up GBDT training on Parameter Server.
problem Training GBDT on Parameter Server is slow due to computational resources and time.
method Developed asynch-SGBDT, an asynchronous parallel method for GBDT training.
result Asynch-SGBDT achieves linear speedup in asynchronous parallel training.
IMPACT improves RL training speed without sacrificing sample efficiency.
problem Limited sample efficiency in scalable RL architectures.
method Proposes IMPACT, extending IMPALA with target networks, circular buffers, and truncated importance sampling.
result IMPACT achieves higher rewards and significantly reduces training time compared to IMPALA.
MLPerf benchmarks ML training to drive performance improvements.
problem Unique challenges in ML training benchmarks.
method Developed MLPerf to overcome ML training's specific challenges.
result Quantitatively evaluated MLPerf's effectiveness.
The paper discusses scalable learning for wireless data-driven systems.
problem Expanding data volume and model complexity limit centralized learning solutions.
method Discusses scalable architecture and local learning strategies.
result Promising research directions in scalable data-driven wireless communications.
XGBoost accelerates machine learning on GPUs.
problem Training large datasets efficiently on GPUs.
method Multi-GPU gradient boosting with data compression and end-to-end GPU parallelism.
result Processed 115 million instances in 3 minutes.
The paper introduces a scalable unsupervised learning framework to improve deep neural networks.
problem Improving deep neural networks' performance and generalization in unsupervised settings.
method A scalable unsupervised regularization framework that constrains hypothesis space to non-trivial piecewise constant functions.
result The framework leads to a factually confident and smooth discriminative model, achieving state-of-the-art clustering results and generalization on both synthetic and real data.
LC-LMT makes certified defense more efficient and robust.
problem Scalable certified defense methods are resource-intensive and only achieve small robustness.
method LC-LMT, a lightweight Lipschitz margin training method.
result LC-LMT achieves higher robustness and ε-robustness more efficiently. NeuroMAS treats multi-agent systems as neural networks for scalable, trainable coordination.
problem Designing multi-agent systems as hand-designed workflows is inefficient and inflexible.
method NeuroMAS treats multi-agent systems as a neural network architecture with reinforcement learning for scalable coordination.
result NeuroMAS improves significantly over multi-agent baselines and can be scaled progressively.
Scalable feature selection improves GBDT model training speed.
problem Efficiently selecting features for GBDT models in high-dimensional settings.
method Developed a scalable forward feature selection method using group testing.
result Significant speedups in training time with competitive model performance.
FinRL-Podracer accelerates DRL trading strategies in finance with high performance and scalability.
problem Challenges in applying deep reinforcement learning to finance trading models.
method Proposes an RLOps framework and high-performance cloud solution for DRL trading.
result FinRL-Podracer outperforms existing DRL libraries by 12-35% in annual return, 0.1-0.6 in Sharpe ratio, and 3-7 times in training time.
A scalable framework for gradient boosting using TensorFlow.
problem Training gradient boosted trees efficiently on large datasets.
method Distributed training architecture, automatic loss differentiation, layer-by-layer boosting, multi-class handling, regularization.
result Faster prediction and smaller ensembles compared to traditional methods.
L2G2G improves graph autoencoder accuracy without sacrificing scalability.
problem Scalability issues in graph autoencoders for large networks.
method Dynamic synchronization of latent node representations in each epoch.
result L2G2G achieves higher accuracy than standard Local2Global approach on large and dense networks.
New method for efficient sketching of gradients and Hessians.
problem Memory constraints in training machine learning models.
method A novel framework for scalable gradient and HVP sketching tailored for modern hardware.
result Theoretical guarantees and practical applications in training data attribution and Hessian spectrum analysis.
New algorithm reduces bias in trained models, near-optimal performance proven.
problem Reduction of bias in trained machine learning models.
method Scalable post-processing algorithm for debiasing trained models, including deep neural networks (DNNs).
result Proven to be near-optimal by bounding its excess Bayes risk.
BNAS improves neural architecture search with a scalable, fast, and efficient approach.
problem Efficiently searching for optimal neural architectures with high performance and low training time.
method Designing a broad scalable architecture (BCNN) with reinforcement learning and parameter sharing, and developing two variants.
result Significantly reduces training time and achieves state-of-the-art performance on CIFAR-10 and ImageNet.
A scalable deep learning framework accelerates training of large neural networks for solving 3D Poisson equations.
problem Training large-scale neural networks for solving complex PDEs efficiently.
method Combines multigrid techniques with distributed deep learning to accelerate training.
result Solves 3D Poisson equations up to 512x512x512 resolution efficiently.
FSBM improves matching efficiency with minimal supervision.
problem Scalability vs. minimal supervision in matching frameworks.
method FSBM uses a small portion of pre-aligned pairs as state feedback to guide non-coupled samples.
result FSBM accelerates training and enhances generalization.
EduQG generates better educational questions by pre-training on scientific text.
problem Improving the quality of educational questions for scalable self-assessment.
method Adapting a large language model for educational question generation, pre-trained on scientific text.
result EduQG produces superior educational questions compared to baseline models.