Speeds up training and inference by pruning entire channels before training.
problem Training and inference speed in deep neural networks.
method Structured pruning applied before training, focusing on removing entire channels and hidden units.
result 2x speedup in training and 3x speedup in inference.
Optimizes PCB stack-up design for high-speed circuits.
problem Efficiently optimize many parameters in PCB stack-up design.
method Parallel and intelligent Bayesian optimization for stripline design.
result Improves accuracy and efficiency of PCB stack-up optimization.
Quantum computers can speed up machine learning optimization problems.
problem Long computation times and high resource requirements for classical optimization algorithms in machine learning.
method Developed a mathematical model to leverage quantum parallelism for machine learning.
result Quantum machine learning applied to a 3D time-varying image demonstrated significant speedup.
This work learns a default policy to speed up RL learning by restricting information it receives.
problem Information asymmetry in reinforcement learning.
method KL regularized expected reward objective with a learned default policy.
result Learning a default policy can significantly speed up and improve RL learning.
Meta-learning speeds up learning new tasks.
problem Designing and improving machine learning pipelines.
method Observing and learning from different machine learning approaches.
result Learning new tasks much faster than traditional methods.
Paper analyzes Scaffold algorithm for federated learning, proving linear speed-up with stochastic gradients.
problem Understanding the impact of stochastic gradients on the Scaffold algorithm's performance.
method Proved linear speed-up in the number of clients using a Markov chain analysis of global parameters and control variates.
result Scaffold achieves linear speed-up in the number of clients up to higher-order terms in the step size, but retains a higher-order bias.
Quantum algorithms speed up reinforcement learning policies in large state-action spaces.
problem Limitations of quantum access in training reinforcement learning policies.
method Designing quantum algorithms to train reinforcement learning policies.
result Quantum algorithms offer full quadratic speed-ups in sample complexity for well-behaved policies.
AdaScale SGD adapts learning rates for large-batch training efficiently.
problem Adapting learning rates for large-batch training to balance speed-ups and model quality.
method Adaptive learning rate adaptation based on gradient variance.
result AdaScale achieves reliable speed-ups for a wide range of batch sizes without degrading model quality.
New RL approach speeds up training across tasks.
problem Training reinforcement learning models quickly and efficiently.
method Combines planning quasi-metric and task-specific aimers.
result Achieves multiple-fold speed-up on bit-flip and robotic arm tasks.
Tensor trains speed up option pricing for multi-asset options.
problem Speeding up option pricing for multi-asset options.
method Tensor train learning algorithms to compress functions with parameter dependence.
result The proposed method outperforms Monte Carlo-based pricing in computational complexity.
New method speeds up nuclear-norm constrained learning over multiple machines.
problem Synchronization slowdown and high communication costs in large-scale learning.
method Asynchronous Stochastic Frank-Wolfe (SFW-asyn) method.
result SFW-asyn achieves the same convergence rate as vanilla SFW but with speed-ups almost linear to the number of machines.
Enhances supervised learning speed with KT algorithm.
problem Speed up supervised learning tasks with minimal loss.
method Generalizes kernel thinning to supervised learning, combining NW and KRR with KT.
result KT-based estimators offer superior computational and statistical efficiency.
Two algorithms improve K-means clustering speed without sacrificing quality.
problem Improving clustering quality of K-means while speeding up the process.
method Divisive K-means and Parallel Two-Phase K-means.
result Achieved empirically global optimum clustering results with lower complexity.
Machine learning speeds up search procedures for sorted tables.
problem Improving the speed of sorted table search procedures.
method Systematic experimental comparison of efficient implementations with learned counterparts.
result Learned data structures can significantly speed up search procedures.
Accelerates coordinate descent methods for machine learning problems.
problem Slowness of coordinate descent methods in machine learning.
method Extrapolation-based accelerated coordinate descent.
result Significant speed-up in practice compared to existing methods.
Study improves privacy-preserving online prediction from experts with speed-ups.
problem Privacy-preserving online prediction from experts with speed-ups.
method Differentially private federated online prediction algorithms.
result Achieves m-fold regret speed-up with low-loss expert in federated setting. Quantum algorithms for multi-armed bandits are explored with limited reward access.
problem Exploring quantum speed-ups in multi-armed bandit problems with limited reward information.
method Introduced new bandit models and showed query complexity equivalence with classical algorithms.
result No quadratic speed-up is possible for multi-armed bandits with limited reward access.
Interactive teaching speeds up IRL learning with adaptive demonstrations.
problem Tackles the challenge of accelerating IRL learning with teacher assistance.
method Interactive teaching framework where a teacher adapts demonstrations based on the learner's policy.
result Teaching algorithms converge in the omniscient setting, speeding up learning.
The paper categorizes four types of scale-up: smart, dumb, forced, and fumbled.
problem Growing ventures in size and maintaining efficiency.
method Identifying modularity and speed as key factors, categorizing four types of scale-up.
result Modularity and speed are crucial for successful scale-up.
New methods speed up training of differentially private deep learning models.
problem Training differentially private deep learning models is slower than non-private models.
method Derive and implement new per-example gradient clipping methods compatible with auto-differentiation.
result Significant training speed-ups (54x - 94x) for various models and architectures.
QWO speeds up causal discovery in LiGAMs by O(n2).
problem Scalability issues in permutation-based causal discovery methods for LiGAMs.
method Constructing a specific DAG for a permutation and searching over permutations to minimize edge count.
result QWO achieves a speed-up of O(n2) compared to state-of-the-art methods. Orthogonal initialization does not speed up training in ultra-wide neural networks.
problem Exploring the effect of orthogonal initialization on training speed in deep neural networks.
method Study of neural tangent kernel dynamics in FCNs and CNNs with orthogonal initialization.
result The NTK of orthogonally-initialized networks remains constant during training, suggesting no speedup in the NTK regime.
CodedFedL speeds up federated learning in MEC networks by 15x.
problem Slow convergence in federated learning due to heterogeneity and stochastic fluctuations.
method Injects structured coding redundancy into federated learning to mitigate stragglers and speed up training.
result CodedFedL speeds up the training procedure by up to 15x compared to benchmark schemes.
Paper speeds up and extends Subclass Discriminant Analysis methods.
problem Improving efficiency and handling multi-view data in Subclass Discriminant Analysis.
method Developed a speed-up approach based on graph embedding and spectral regression, and a novel multi-view solution.
result Proposed methods achieve competitive performance and significantly decrease training time.
The paper uses deep learning to speed up spatial and visual connectivity analysis.
problem Slow calculation of spatial and visual connectivity metrics.
method Investigates machine learning models and a pipeline for training them on spatial and visual connectivity analysis.
result Deep learning models significantly speed up the analysis process.
Multirate training speeds up neural network fine-tuning.
problem Efficiently fine-tuning deep neural networks.
method Partitioning neural network parameters into fast and slow parts, updating slowly over longer intervals.
result Significant computational speed-up for transfer learning tasks.
A new algorithm speeds up feature selection by 65.77%.
problem Efficiently selecting features in high-dimensional data.
method Dropping Forward-Backward algorithm, reducing rescanning of features.
result Significantly faster than stepwise procedure, maintaining good performance.
This paper studies parallelization schemes for stochastic Vector Quantization algorithms in order to obtain time speed-ups using distributed resources. We show that the most intuitive parallelization scheme does not lead to better performances than the sequential algorithm. Another distributed scheme is therefore intro…
Distributed Gradient Descent achieves optimal rates in non-parametric regression with linear speed-up.
problem Optimal statistical rates in decentralized non-parametric regression.
method Distributed Gradient Descent with i.i.d. samples and linear speed-up.
result Achieves optimal statistical rates with linear speed-up in the big data regime.
Classifies self-similar solutions for heat equations with positive speed.
problem Classifying self-similar solutions for semilinear heat equations.
method Analyzes the semilinear heat equation ut=Δu+∣u∣p−1u for p>1. result Finite time blowing up solutions converge to a positive constant after rescaling.
Warm-start strategies speed up GP inference by 19x.
problem Efficient sequential inference in Gaussian processes.
method Three warm-start strategies exploiting smaller linear systems.
result Warm-starting achieves up to 19x speed-up in convergence.
Quantum computing speeds up asset pricing models exponentially.
problem Solving dynamic nonlinear asset pricing models efficiently.
method Utilizes quantum superposition and entanglement to solve models exponentially faster than classical methods.
result Exponential computational speed-up for solving asset pricing models.
Quantum LS-SVM simplifies matrix inversion for faster machine learning.
problem Speeding up machine learning algorithms for large datasets.
method Introduces a novel quantum algorithm using continuous variables to simplify matrix inversion in LS-SVM, and proposes a hybrid quantum-classical approach for sparse solutions.
result Quantum LS-SVM achieves exponential speed-up and can solve classically difficult tasks.
Data application developers and data scientists spend an inordinate amount of time iterating on machine learning (ML) workflows -- by modifying the data pre-processing, model training, and post-processing steps -- via trial-and-error to achieve the desired model performance. Existing work on accelerating machine learni…
Federated Learning speeds up speech recognition training by 7x and reduces error rate by 6%.
problem Training acoustic models for speech recognition tasks in a federated manner.
method Hierarchical optimization and dynamic gradient aggregation methods.
result Significant improvement in training convergence speed and model performance.
Non-autoregressive method speeds up protein folding prediction 23 times.
problem Generating protein sequences with higher order interactions.
method Discrete diffusion conditioned on 3D structure using ProteinMPNN.
result 23 times speed up in inference without performance loss.
Structural-Jump-LSTM speeds up reading by skipping and jumping text.
problem Sequential inference in RNNs makes reading time linearly dependent on input length.
method Introduces a novel LSTM model with agents for skipping and jumping text.
result Structural-Jump-LSTM achieves best FLOP reduction and maintains or improves accuracy.
Paper proposes a modified uncertainty sampling method to speed up preference learning from noisy humans.
problem Learning preferences from humans with limited queries and noisy responses.
method Modified uncertainty sampling using expected output value to speed up preference learning.
result The modified method outperforms the baseline uncertainty sampling in preference learning.
Quantum method speeds up VB estimation in machine learning.
problem Prohibitively expensive natural gradient in high dimensions.
method Regression-based natural gradient estimation with quantum matrix inversion.
result Quantum method enables efficient VB estimation.
AcceleratedLiNGAM speeds up causal discovery methods for large datasets.
problem Slow causal discovery methods for large-scale datasets.
method Parallelized LiNGAM method with GPU acceleration.
result Up to 32-fold speed-up on benchmark datasets.
Topology speeds up and improves CNNs for digit recognition.
problem Improving and speeding up deep learning models for digit recognition.
method Topological data analysis applied to CNNs to modify computations.
result Topology improves CNNs' performance and generalization.
sGBM speeds up gradient boosting by parallelizing and adapting base learners.
problem Infeasibility of parallelizing GBM training and sub-optimal performance in online settings.
method Integrates multiple differentiable base learners, jointly optimizing them with linear speed-up.
result sGBM achieves higher time efficiency and better accuracy than traditional GBM.
Gaussian process regression (GPR) is a non-parametric Bayesian technique for interpolating or fitting data. The main barrier to further uptake of this powerful tool rests in the computational costs associated with the matrices which arise when dealing with large data sets. Here, we derive some simple results which we h…
This paper explores how random sampling and coding can speed up approximate matrix multiplication.
problem Efficiently computing large-scale matrix multiplications in distributed systems.
method Proposes two schemes: coding for recovery and random sampling for approximation.
result Investigates tradeoffs between recovery threshold and approximation error.
Random feature approximation speeds up spectral methods and improves learning rates.
problem Improving the efficiency and generalization of spectral methods in large-scale algorithms.
method Combining random feature approximation with spectral regularization methods.
result Optimal learning rates for estimators over various regularity classes, including those not in the RKHS.
Quantum computing speeds up Bermudan option pricing.
problem Efficient pricing of financial derivatives, especially Bermudan options.
method Quantum amplitude estimation combined with Chebyshev interpolation.
result Quadratic speed-up over classical methods.
Scale of data and scale of computation infrastructures together enable the current deep learning renaissance. However, training large-scale deep architectures demands both algorithmic improvement and careful system configuration. In this paper, we focus on employing the system approach to speed up large-scale training.…
A deep learning model speeds up computation of numerous implied volatilities.
problem Frequent computation of numerous implied volatilities using iteration methods like Newton-Raphson reaches processing speed limits.
method Emulated Newton-Raphson method using PyTorch and optimized with TensorRT.
result Up to 1,000 times faster than a benchmark implementation of Newton-Raphson.