Paper speeds up structured output prediction without sacrificing accuracy.
problem Computational inefficiency in predicting structured outputs.
method Learning to search approach to train a speedup classifier.
result Speedup classifier outperforms greedy search in terms of speed.
Paper presents an efficient algorithm for learning minimax risk classifiers with large-scale data.
problem Efficient learning of minimax risk classifiers for large-scale data with multiple classes.
method Combination of constraint and column generation for efficient learning.
result 10x speedup for general large-scale data and 100x speedup with many classes.
GPU optimization speeds up large-scale classification tasks.
problem Efficiently training large-scale classification models on GPUs.
method Judecious GPU-optimization principles applied to TRON algorithm.
result Significant speedups for logistic regression and SVM classification.
Improved image classifier performance with faster training using adaptive learning rates.
problem Lower accuracy and training time in small datasets.
method Dynamic learning rate tuning for different model architectures.
result Two-fold to ten-fold speedup in accuracy over state-of-the-art methods.
ProxSkip achieves linear speedup in distributed non-convex optimization.
problem Achieving linear speedup in distributed non-convex optimization.
method Unified convergence analysis for stochastic non-convex, convex, and strongly convex problems.
result ProxSkip achieves linear speedup in the number of nodes under stochastic gradients.
Linear speedup achieved in non-convex optimization for decentralized systems.
problem Achieving optimal performance in decentralized non-convex optimization.
method Examined the dependence of convergence guarantees on spectral properties of combination policies.
result Linear speedup in saddle-point escape time for symmetric combination policies.
Adaptive algorithm generates unrestricted adversarial inputs, defeating robust classifiers.
problem Vulnerability of neural networks to unrestricted adversarial inputs.
method Adaptive algorithm for generating unrestricted adversarial inputs.
result Adversarial inputs defeat robust classifiers.
Quantum computing promises faster finance algorithms.
problem Solving finance problems faster than classical methods.
method Quantum computing applications to finance, including Monte Carlo, portfolio optimization, and machine learning.
result Quantum speedups for finance problems, especially Monte Carlo and portfolio optimization.
A quantum-inspired classical algorithm speeds up LS-SVM classification.
problem Big data challenge in SVM classification.
method Improved indirect sampling technique for LS-SVM.
result Algorithm achieves logarithmic runtime for low rank data matrices.
A fast method for Lasso and Logistic Lasso problems.
problem Solving Lasso and Logistic Lasso regression problems efficiently.
method Iterative active set approach using solver updates.
result 31.41 times faster on average for compressed sensing.
Guided Flows enhance sample quality in conditional image generation and text-to-speech.
problem Improving sample quality in conditional generative models.
method Integrating classifier-free guidance into Flow Matching (FM) models for Continuous Normalizing Flows (CNFs).
result Guided Flows significantly improve sample quality in conditional image generation and text-to-speech synthesis.
New insights explain speedup saturation in distributed learning with large batches and delays.
problem Understanding and optimizing speedup in distributed learning with large batches and delays.
method Theoretical analysis of strongly convex, convex, and non-convex settings, considering data sparsity.
result Identification of a data-dependent parameter explaining speedup saturation in both batch size and gradient staleness.
Quantum computing offers a quadratic speedup for estimating non-linear functionals.
problem Estimating non-linear functionals of probability distributions.
method Proposes a quantum-inside-quantum Monte Carlo algorithm for a broad class of non-linear estimation problems.
result Achieves a quadratic speedup for non-linear estimation problems, including nested conditional expectations and stochastic optimization.
MACER accelerates error repair by modularly identifying and applying fixes.
problem Automated compilation error repair for novice programmers.
method Modular segregation of repair process into identification and application, using discriminative learning techniques.
result MACER outperforms existing methods by 20% on popular errors and is competitive on all error types.
Quantum algorithm speeds up nested expectation estimation by nearly quadratically.
problem Estimating repeatedly nested expectations with quantum computing.
method Proposes a quantum algorithm achieving nearly quadratic speedup over classical methods.
result Achieves nearly quadratic speedup for RNEs, up to logarithmic factors.
Federated Q-Learning achieves linear regret speedup with low communication cost.
problem Achieving linear regret speedup in federated reinforcement learning without high communication costs.
method Proposed two federated Q-Learning algorithms: FedQ-Hoeffding and FedQ-Bernstein, using event-triggered synchronization, novel step size selection, and concentration inequalities.
result Total regrets achieve linear speedup compared to single-agent counterparts with logarithmic communication cost.
A new algorithm speeds up deep learning training by decoupling computation and communication.
problem High communication cost limits the speedup of distributed SGD.
method CoCoD-SGD: runs computation and communication in parallel.
result Linear time speedup with respect to hardware resources.
TinyCNN accelerates CNN models on embedded FPGA with 15x speedup.
problem Limited memory on embedded FPGAs restricts CNN performance.
method Software and hardware design tool for FPGA resource-aware CNN accelerator.
result 3% accuracy loss with 15.75x speedup on image classification.
Paper proposes MCMA architecture for neural approximate computing with higher invocation rate and energy savings.
problem Limited invocation rate of neural approximators leading to suboptimal energy efficiency.
method Introduces MCMA architecture with a multiclass classifier and multiple approximators, sharing hardware resources and efficiently swapping approximators.
result Significantly higher invocation rate and energy savings compared to existing methods.
Optimized parallel RNN training reaches up to 845x speedup.
problem Expensive RNN training through back-propagation through time (BPTT).
method Optimized parallel algorithm \opt based on ELM, leveraging GPU shared memory and QR factorization.
result Up to 845x speedup over sequential training and 20x less time to train.
Neural network speeds up atmospheric chemistry modeling 4250x.
problem Computational expense of simulating atmospheric chemistry.
method Created a neural network to emulate a complex chemical mechanism.
result Achieved a 250x computational speedup.
New algorithm achieves linear speedup in non-i.i.d. federated bilevel learning.
problem Linear speedup in convergence for non-i.i.d. datasets in federated bilevel optimization.
method Proposes FedMBO with a novel client sampling scheme for non-i.i.d. datasets.
result Achieves a convergence rate of O(1/√(nK) + 1/K + √n/K³/²).
System learns optimizer hyperparameters to generalize across tasks.
problem Lack of generalization in learning optimizers for neural networks.
method Generalization-first approach, learning optimizer hyperparameters.
result System outperforms Adam on various tasks, including unseen modalities.
QWYC optimizes evaluation order and thresholds to speed up ensemble classification.
problem Efficiently evaluate classifier ensembles to reduce latency and CPU usage.
method Jointly optimize evaluation order and early-stopping thresholds using a greedy algorithm.
result 4-approximation of optimal solution for certain cases, achieving significant speedup.
Quantum algorithms speed up derivative pricing beyond Black-Scholes models.
problem Quantum speedups for derivative pricing beyond Black-Scholes models.
method Utilizing fast-forwardability and quantum Milstein sampler for non-GBM models, and improved numerical integration for GBM and CIR models.
result Quadratic speedups for derivative pricing in practical models like CIR and Heston's model.
ECD algorithm speeds up non-convex optimization, offering quantum and stochastic enhancements.
problem Non-convex optimization challenges in machine learning.
method Energy Conserving Descent (ECD) algorithm, stochastic ECD dynamics (sECD), quantum ECD Hamiltonian (qECD).
result ECD and its quantum version achieve exponential speedup over gradient descent.
Unified analysis of Federated Averaging and Nesterov FedAvg for linear speedup.
problem Understanding convergence of FL algorithms under non-i.i.d. data and partial participation.
method Systematic study of convergence guarantees for FedAvg and Nesterov FedAvg under different conditions.
result Unified analysis of linear speedup for FedAvg and Nesterov FedAvg in various settings.
We introduce dropout compaction, a novel method for training feed-forward neural networks which realizes the performance gains of training a large model with dropout regularization, yet extracts a compact neural network for run-time efficiency. In the proposed method, we introduce a sparsity-inducing prior on the per u…
CHEETAH speeds up secure MLaaS by 100x over fastest existing schemes.
problem Privacy-preserving machine learning on end devices.
method Ultra-fast secure MLaaS framework using secret sharing.
result More than 100x speedup over fastest existing schemes.
Investigates the impact of batch size on GPU and TPU performance.
problem Optimizing performance of GPUs and TPUs during training and inference phases.
method Investigated the impact of batch size on performance of GPUs and TPUs using standard MNIST and Fashion-MNIST datasets.
result Significant speedup was achieved even with low-scale usage of TPUv2 units, up to 10x for training and 2x for prediction.
A central task in the field of quantum computing is to find applications where quantum computer could provide exponential speedup over any classical computer. Machine learning represents an important field with broad applications where quantum computer may offer significant speedup. Several quantum algorithms for discr…
ML-EM method speeds up diffusion model sampling.
problem Efficiently sampling from complex diffusion models.
method Multilevel Euler-Maruyama method with UNet approximations.
result Polynomial speedup in sampling from diffusion models.
We develop parallel and distributed Frank-Wolfe algorithms; the former on shared memory machines with mini-batching, and the latter in a delayed update framework. Whenever possible, we perform computations asynchronously, which helps attain speedups on multicore machines as well as in distributed environments. Moreover…
Paper analyzes convergence and speedup of asynchronous parallel SGD.
problem Achieving good convergence and linear speedup in asynchronous parallel SGD.
method Second-order convergence analysis of APSGD with consistent read near strictly saddle points.
result Theoretical guarantee for using at most O ( K 1 / 3 M − 1 / 3 ) O(K^{1/3}M^{-1/3}) O ( K 1/3 M − 1/3 ) workers for good convergence and linear speedup. Novel periodic momentum SGD method for decentralized training with linear speedup.
problem Lack of effective momentum schema in decentralized training methods.
method Proposes a novel periodic decentralized momentum SGD method.
result Achieves linear speedup in decentralized training.
Speed up neural networks by 2x with 5% mAP loss.
problem High computational cost of neural network forward passes.
method Deep Learning Approximation: lossless and lossy optimizations.
result 2x speedup in network forward pass with 5% mAP drop.
Quantum algorithm speeds up MIP solving by a near-quadratic factor.
problem Solving Mixed Integer Programs (MIPs) efficiently.
method Incremental-Quantum-Branch-and-Bound algorithm combining quantum speedup with classical search heuristics.
result Universal near-quadratic speedup over classical Branch-and-Bound algorithms.
Quantum algorithm speeds up learning from big data exponentially.
problem Scalable learning from big data with optimized random features.
method Quantum algorithm for sampling optimized random features.
result Exponential speedup in runtime compared to classical algorithms.
Parallelizes LARS for high-dimensional data with speedups and accuracy trade-offs.
problem Fitting linear regression models to high-dimensional data efficiently.
method Two parallel and communication avoiding versions of LARS: bLARS and Tournament-bLARS.
result Speedups up to 4x compared to LARS, with trade-offs in solution quality.
Speeds up deep neural networks training by 10x using GPU concurrency.
problem Training deep residual neural networks efficiently.
method Layer-wise parallel training with GPU concurrency and Nonlinear Multigrid.
result 10.2x speedup over traditional techniques.
Federated learning algorithm improves with intermittent client availability.
problem Performance degradation in Federated Averaging due to client availability changes.
method Federated Latest Averaging (FedLaAvg) uses latest gradients from all clients, even when unavailable.
result FedLaAvg achieves sublinear speedup compared to classical Federated Averaging.
Paper studies collaboration in multi-armed bandits with limited interaction.
problem Identifying the best arm collaboratively with limited communication.
method Developed techniques to quantify and prove round-speedup tradeoffs.
result Almost tight round-speedup tradeoffs for distributed exploration.
WU-UCT parallelizes MCTS with linear speedup and limited performance loss.
problem Challenges in parallelizing Monte Carlo Tree Search (MCTS) due to its sequential nature.
method Introduces unobserved samples to track incomplete simulations and modify UCT tree policy.
result Achieves linear speedup and only limited performance loss with increasing parallel workers.
Simplifies GCNs to improve efficiency and interpretability.
problem Excessive complexity and redundant computation in GCNs.
method Successively removing nonlinearities and collapsing weight matrices.
result Simplified model maintains accuracy and scales to larger datasets.
Asynchronous L-BFGS speeds up non-convex optimization.
problem Non-convex optimization challenges in machine learning.
method Asynchronous stochastic L-BFGS algorithm for non-convex optimization.
result Achieves an ergodic convergence rate of O ( 1 / N ) {\cal O}(1/\sqrt{N}) O ( 1/ N ) and linear speedup. We parallelize backpropagation for deep learning models, achieving significant speedups.
problem Sequential dependency in backpropagation limits scalability on parallel systems.
method Reformulated backpropagation as a scan operation, using Blelloch scan algorithm.
result Up to 2.75x speedup on overall training time and 108x on backward pass.
New GPU kernels boost deep learning speed and memory efficiency.
problem Sparse deep learning matrices are not well-suited for existing sparse kernels.
method Identified favorable properties of sparse matrices from deep learning, developed high-performance GPU kernels for sparse matrix operations.
result 27% of single-precision peak performance on Nvidia V100 GPUs achieved with new kernels.
Efficiently approximates Sparse PCA with significant speedups and minor error.
problem Sparse Principal Component Analysis (Sparse PCA) is NP-hard and computationally expensive.
method Approximates the covariance matrix with block-diagonal form, solves sub-problems in each block, and reconstructs the solution.
result Significant computational speedups with minor additive error.