ML-EM method speeds up diffusion model sampling.
problem Efficiently sampling from complex diffusion models.
method Multilevel Euler-Maruyama method with UNet approximations.
result Polynomial speedup in sampling from diffusion models.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,051 papers · 148 categories
Trend · papers per month
48 results for “ML training speedup”
Training large machine learning (ML) models with many variables or parameters can take a long time if one employs sequential procedures even with stochastic updates. A natural solution is to turn to distributed computing on a cluster; however, naive, unstructured parallelization of ML algorithms does not usually lead t…
BlinkML speeds up ML training by 6x-600x with probabilistic guarantees.
problem Ad-hoc sampling in ML analysis leads to unreliable model quality.
method BlinkML uses probabilistic guarantees for approximate model training.
result BlinkML can speed up ML training by 6.26x-629x with 95% prediction accuracy.
When training large machine learning models with many variables or parameters, a single machine is often inadequate since the model may be too large to fit in memory, while training can take a long time even with stochastic updates. A natural recourse is to turn to distributed cluster computing, in order to harness add…
CodedPrivateML secures ML training data and models.
problem Training machine learning models while keeping data private.
method Data and model-theoretically private approach with parallelization.
result Significant speedup over cryptographic methods.
Fast covariance parameter estimation of spatial Gaussian process models using neural networksstat.ML
Neural networks speed up covariance estimation in spatial Gaussian processes.
problem Efficiently estimating covariance parameters in spatial Gaussian processes.
method Training neural networks to approximate maximum likelihood estimates.
result Neural network estimates are as accurate as ML methods but much faster.
An intrinsic problem of classifiers based on machine learning (ML) methods is that their learning time grows as the size and complexity of the training dataset increases. For this reason, it is important to have efficient computational methods and algorithms that can be applied on large datasets, such that it is still …
Fast ML framework for derivative valuation from volatility surfaces.
problem Derivative valuation from complex volatility surfaces.
method Parameterized SVI model, synthetic market scenarios, Gaussian Process Regressor.
result Very accurate and fast (3-4 orders of magnitude) derivative valuations.
A new ML method speeds up PDE simulations without needing classical training.
problem Accelerating transient PDE simulations using machine learning.
method Online-learned preconditioners using a bandit algorithm.
result One-shot acceleration of PDE simulations.
A Multi-Grained Symmetric Differential Equation Model for Learning Protein-Ligand Binding Dynamicscs.LG
NeuralMD accelerates protein-ligand binding simulations 1Kx faster.
problem Accurate and efficient simulation of protein-ligand binding dynamics.
method Physics-informed multi-grained group symmetric framework with BindingNet and augmented neural differential equation solver.
result Achieves over 1Kx speedup and up to 15x reduction in reconstruction error compared to standard methods.
Optimized parallel RNN training reaches up to 845x speedup.
problem Expensive RNN training through back-propagation through time (BPTT).
method Optimized parallel algorithm \opt based on ELM, leveraging GPU shared memory and QR factorization.
result Up to 845x speedup over sequential training and 20x less time to train.
A new algorithm speeds up deep learning training by decoupling computation and communication.
problem High communication cost limits the speedup of distributed SGD.
method CoCoD-SGD: runs computation and communication in parallel.
result Linear time speedup with respect to hardware resources.
MLPerf benchmarks ML training to drive performance improvements.
problem Unique challenges in ML training benchmarks.
method Developed MLPerf to overcome ML training's specific challenges.
result Quantitatively evaluated MLPerf's effectiveness.
Novel periodic momentum SGD method for decentralized training with linear speedup.
problem Lack of effective momentum schema in decentralized training methods.
method Proposes a novel periodic decentralized momentum SGD method.
result Achieves linear speedup in decentralized training.
Parallel computing has played an important role in speeding up convex optimization methods for big data analytics and large-scale machine learning (ML). However, the scalability of these optimization methods is inhibited by the cost of communicating and synchronizing processors in a parallel setting. Iterative ML metho…
Predicting structured outputs can be computationally onerous due to the combinatorially large output spaces. In this paper, we focus on reducing the prediction time of a trained black-box structured classifier without losing accuracy. To do so, we train a speedup classifier that learns to mimic a black-box classifier u…
This work enhances deep neural networks with robust features to improve their robustness against adversarial attacks.
problem Vulnerability of deep neural networks to adversarial images.
method Augmenting classification pipelines with robust features like binarization and group extraction.
result Improved robustness and training speed on adversarial inputs, with significant improvements over state-of-the-art methods.
Speeds up deep neural networks training by 10x using GPU concurrency.
problem Training deep residual neural networks efficiently.
method Layer-wise parallel training with GPU concurrency and Nonlinear Multigrid.
result 10.2x speedup over traditional techniques.
We translate ML pipelines into neural networks to optimize multiple models together.
problem Isolated training of ML pipelines limits joint optimization of multiple models.
method Propose translating ML pipelines into neural networks and fine-tuning them jointly.
result Fine-tuning translated pipelines increases final accuracy.
System learns optimizer hyperparameters to generalize across tasks.
problem Lack of generalization in learning optimizers for neural networks.
method Generalization-first approach, learning optimizer hyperparameters.
result System outperforms Adam on various tasks, including unseen modalities.
Investigates the impact of batch size on GPU and TPU performance.
problem Optimizing performance of GPUs and TPUs during training and inference phases.
method Investigated the impact of batch size on performance of GPUs and TPUs using standard MNIST and Fashion-MNIST datasets.
result Significant speedup was achieved even with low-scale usage of TPUv2 units, up to 10x for training and 2x for prediction.
TFCheck detects training issues in ML programs using TensorFlow.
problem Training programs often contain inconsistencies and bugs.
method Developed a TensorFlow library (TFCheck) with verification routines.
result TFCheck successfully detects training issues in ML code.
Sage platform protects ML models trained on sensitive data from leakage.
problem Protecting sensitive data in machine learning models exposed to untrusted domains.
method Develops block composition for privacy accounting and privacy-adaptive training to manage privacy budget and utility tradeoff.
result Enables continuous training of models on sensitive data streams while maintaining global DP guarantees.
FSPT identifies training space to prevent ML model extrapolation.
problem Lack of training data leads to unreliable predictions outside the training space.
method FSPT: Feature Space Partitioning Tree to identify training space.
result Strong relationship between model performance and FSPT score.
Improves ML efficiency for vast, rapidly growing data.
problem Low latency and cost in ML with distributed, growing data.
method Designs ML systems exploiting ML characteristics, data structures, and data distribution.
result Improves ML latency and cost by 1-2 orders of magnitude.
Distributed Non-Convex Optimization with Sublinear Speedup under Intermittent Client Availabilitystat.ML
Federated learning algorithm improves with intermittent client availability.
problem Performance degradation in Federated Averaging due to client availability changes.
method Federated Latest Averaging (FedLaAvg) uses latest gradients from all clients, even when unavailable.
result FedLaAvg achieves sublinear speedup compared to classical Federated Averaging.
SliceOut speeds up deep learning training without sacrificing accuracy.
problem Frequent model re-training and large model training workloads in deep learning.
method SliceOut uses dropout-inspired scheme to drop contiguous sets of units at random, leveraging GPU memory layout.
result 10-40% speedups and memory reduction with minimal accuracy loss.
GPU optimization speeds up large-scale classification tasks.
problem Efficiently training large-scale classification models on GPUs.
method Judecious GPU-optimization principles applied to TRON algorithm.
result Significant speedups for logistic regression and SVM classification.
Neural networks offer high-accuracy solutions to a range of problems, but are costly to run in production systems because of computational and memory requirements during a forward pass. Given a trained network, we propose a techique called Deep Learning Approximation to build a faster network in a tiny fraction of the …
We parallelize backpropagation for deep learning models, achieving significant speedups.
problem Sequential dependency in backpropagation limits scalability on parallel systems.
method Reformulated backpropagation as a scan operation, using Blelloch scan algorithm.
result Up to 2.75x speedup on overall training time and 108x on backward pass.
Critical Parameters for Scalable Distributed Learning with Large Batches and Asynchronous Updatescs.LG
New insights explain speedup saturation in distributed learning with large batches and delays.
problem Understanding and optimizing speedup in distributed learning with large batches and delays.
method Theoretical analysis of strongly convex, convex, and non-convex settings, considering data sparsity.
result Identification of a data-dependent parameter explaining speedup saturation in both batch size and gradient staleness.
Over the past decades, researchers and ML practitioners have come up with better and better ways to build, understand and improve the quality of ML models, but mostly under the key assumption that the training data is distributed identically to the testing data. In many real-world applications, however, some potential …
This work optimizes deep learning training by combining data and model parallelism.
problem Training large models with multiple GPUs suffers from high communication overhead and statistical efficiency loss.
method Hybrid parallelization combining data and model parallelism.
result Hybrid training provides significant speedup compared to data parallelism alone.
TonY simplifies distributed ML job management.
problem Managing distributed ML jobs is complex and resource-intensive.
method TonY is an open-source orchestrator for distributed ML jobs.
result TonY simplifies distributed ML job management.
Machine Learning (ML) algorithms are used to train computers to perform a variety of complex tasks and improve with experience. Computers learn how to recognize patterns, make unintended decisions, or react to a dynamic environment. Certain trained machines may be more effective than others because they are based on mo…
The paper proposes a ML workflow for B2B sales prediction.
problem Predicting business to business sales outcomes using subjective human evaluations.
method A two-pipeline ML approach on Azure ML, including data enrichment and model training.
result ML predictions improve accuracy and increase sales value.
DP-LSSGD improves privacy-preserving ML models by smoothing out noise.
problem Privacy-preserving ML models have lower utility than non-private ones.
method DP-LSSGD uses Laplacian smoothing to improve utility of DP-SGD.
result DP-LSSGD achieves the same DP guarantee as DP-SGD but with better stability and generalization.
Modyn automates continuous ML model training on growing datasets.
problem Continuous model retraining is costly and impractical.
method Data-centric ML platform with policies for continuous training.
result Modyn enables high throughput training with sample-level data selection.
Pipeline-aware hyperparameter tuning speeds up machine learning pipelines by reusing intermediate computations.
problem High computational burden in hyperparameter tuning of multi-stage pipelines.
method Proposes a hybrid hyperparameter tuning method and a caching problem formulated as an ILP to maximize reuse.
result Pipeline-aware approach offers over an order-of-magnitude speedup over independent evaluations.
Paper analyzes convergence and speedup of asynchronous parallel SGD.
problem Achieving good convergence and linear speedup in asynchronous parallel SGD.
method Second-order convergence analysis of APSGD with consistent read near strictly saddle points.
result Theoretical guarantee for using at most workers for good convergence and linear speedup.
Paper speeds up boosting algorithms with less memory.
problem Boosting algorithms require large memory for training, limiting scalability.
method Combines early stopping, effective sample size, and stratified sampling.
result Achieves 10-100 speedup over XGBoost with limited memory.
We propose a new neural sequence model training method in which the objective function is defined by -divergence. We demonstrate that the objective function generalizes the maximum-likelihood (ML)-based and reinforcement learning (RL)-based objective functions as special cases (i.e., ML corresponds to and R…
Machine learning (ML) models may be deemed confidential due to their sensitive training data, commercial value, or use in security applications. Increasingly often, confidential ML models are being deployed with publicly accessible query interfaces. ML-as-a-service ("predictive analytics") systems are an example: Some …
Unified ML and adversarial learning via α-divergence.
problem Combining strengths of ML and adversarial learning for better generative models.
method Proposes an α-Bridge to unify ML and adversarial learning using α-divergence.
result Generalizations of the α-Bridge are related to recent adversarial learning regularization approaches.
A new training method speeds up ResNet training by 3x with minimal accuracy loss.
problem Training ResNets is slow due to dependencies between modules.
method Serial-parallel hybrid training strategy with data augmentation and downsampling.
result Significant speedup over traditional methods with comparable accuracy.
Security for Machine Learning-based Systems: Attacks and Challenges during Training and Inferencecs.LG
This paper explores security threats in ML systems and proposes mitigation techniques.
problem Security vulnerabilities in ML-based systems during training and inference.
method Overview of security threats, demonstrations using LeNet and VGGNet, proposed attack.
result Demonstrated security threats and proposed mitigation techniques.
Training deep networks is expensive and time-consuming with the training period increasing with data size and growth in model parameters. In this paper, we provide a framework for distributed training of deep networks over a cluster of CPUs in Apache Spark. The framework implements both Data Parallelism and Model Paral…
This work compares and evaluates various sampling methods for neural language models.
problem Lack of systematic comparison and myths about sampling methods.
method Monte Carlo sampling, importance sampling, compensated partial summation, noise contrastive estimation.
result All sampling methods can perform equally well if posterior probabilities are corrected.