The study corrects misconceptions in GBDT speed benchmarks.
problem Misleading speed benchmarks of GBDT algorithms.
method Explained and criticized several straightforward benchmarking methods, outlined fair benchmark requirements.
result A fair GBDT speed benchmark requires specific conditions.
Non-autoregressive method speeds up protein folding prediction 23 times.
problem Generating protein sequences with higher order interactions.
method Discrete diffusion conditioned on 3D structure using ProteinMPNN.
result 23 times speed up in inference without performance loss.
We introduce the SaaS Algorithm for semi-supervised learning, which uses learning speed during stochastic gradient descent in a deep neural network to measure the quality of an iterative estimate of the posterior probability of unknown labels. Training speed in supervised learning correlates strongly with the percentag…
LaRT models LLMs' response accuracy and CoT length to evaluate reasoning ability and speed.
problem Valid evaluation of Large Language Models (LLMs) via response accuracy and chain-of-thought length.
method Introduces Latency-Response Theory (LaRT) to jointly model response accuracy and CoT length using latent ability and latent speed.
result LaRT yields higher estimation accuracy and shorter confidence intervals for latent traits compared to IRT.
Model predicts traffic speed using urban incidents.
problem Accurately predicting traffic speed in urban areas.
method Deep Incident-Aware Graph Convolutional Network (DIGC-Net).
result Model outperforms competing benchmarks in traffic speed prediction.
This work introduces a new benchmark to compare neural network training algorithms.
problem Lack of reliable benchmarks to compare training algorithms effectively.
method Developed a new benchmark called AlgoPerf: Training Algorithms benchmark.
result Demonstrated the feasibility of the benchmark and set a provisional state-of-the-art.
Benanza speeds up DL model optimization by automatically generating micro-benchmarks and identifying inefficiencies.
problem Slow characterization/optimization cycles for DL models on GPUs.
method Benanza includes a model processor, benchmark generator, database, and analyzer.
result Benanza identifies optimizations in parallel layer execution, cuDNN, framework inefficiency, layer fusion, and Tensor Cores.
New interior-point method tackles Wasserstein barycenter problem efficiently.
problem Computing Wasserstein barycenter for large support measures.
method Adapted interior-point method exploiting problem's matrix structure.
result Achieves a well-balanced tradeoff between accuracy and speed.
Unconstrained MLIPs outperform constrained ones in accuracy and speed.
problem Improving the efficiency and accuracy of machine-learned interatomic potentials.
method Investigated unconstrained models trained on large datasets compared to physically constrained models.
result Unconstrained MLIPs can be superior in accuracy and speed compared to physically constrained models.
Improved GAN training speed and quality with FastGAN.
problem Slower convergence and less expressive models in GAN training.
method Adversarial training with FastGAN algorithm.
result Better generation quality with less training time.
A new interpolation method speeds up neural ODE training.
problem Efficiently approximating gradients in neural ODEs.
method Interpolation-based technique to approximate gradients.
result Our method trains neural ODEs faster than the reverse dynamic method.
MultiRocket boosts TSC speed and accuracy with pooling and transformations.
problem Efficient time series classification with high accuracy.
method Multiple pooling operators and transformations applied to raw and differenced series.
result MultiRocket outperforms MiniRocket and is competitive with state-of-the-art methods in terms of accuracy and speed.
Reservoir Memory Machines solve benchmark tasks faster than Neural Turing Machines.
problem Training Neural Turing Machines is hard and limits their applicability.
method Proposes Reservoir Memory Machines, combining neural network flexibility with Turing machine capabilities, but with faster training via alignment and linear regression.
result Reservoir Memory Machines solve benchmark tasks as well as Neural Turing Machines but are much faster to train.
In this paper we propose an algorithm that builds sparse decision DAGs (directed acyclic graphs) from a list of base classifiers provided by an external learning method such as AdaBoost. The basic idea is to cast the DAG design task as a Markov decision process. Each instance can decide to use or to skip each base clas…
Integrating wind power into the grid is challenging because of its random nature. Integration is facilitated with accurate short-term forecasts of wind power. The paper presents a spatio-temporal wind speed forecasting algorithm that incorporates the time series data of a target station and data of surrounding stations…
Tensor trains speed up option pricing for multi-asset options.
problem Speeding up option pricing for multi-asset options.
method Tensor train learning algorithms to compress functions with parameter dependence.
result The proposed method outperforms Monte Carlo-based pricing in computational complexity.
CodedFedL speeds up federated learning in MEC networks by 15x.
problem Slow convergence in federated learning due to heterogeneity and stochastic fluctuations.
method Injects structured coding redundancy into federated learning to mitigate stragglers and speed up training.
result CodedFedL speeds up the training procedure by up to 15x compared to benchmark schemes.
We speed up Gaussian process cross-validation calculations and improve model diagnostics.
problem Efficiently calculating cross-validation residuals and their covariances in Gaussian processes.
method Generalized fast Gaussian process leave-one-out formulae to multiple-fold cross-validation, highlighting covariance structures.
result Correcting for residual covariances in cross-validation improves back to Maximum Likelihood Estimation.
Deep neural networks have gained tremendous popularity in last few years. They have been applied for the task of classification in almost every domain. Despite the success, deep networks can be incredibly slow to train for even moderate sized models on sufficiently large datasets. Additionally, these networks require l…
PyTorch combines usability and speed in deep learning.
problem Combining usability and speed in deep learning frameworks.
method Imperative programming style, Pythonic interface, hardware acceleration support.
result PyTorch achieves both usability and high performance.
Meta-Surrogate model speeds up HPO benchmarking.
problem Limited and expensive real-world HPO benchmarks.
method Meta-surrogate model trained on off-line data.
result More coherent and statistically significant conclusions faster.
Research optimizes C++ patterns for HFT, reducing latency and improving profitability.
problem Optimizing latency-critical code for high-frequency trading systems.
method Creation of a Low-Latency Programming Repository, optimisation of trading strategy, implementation of Disruptor pattern.
result Significant performance improvements in speed and profitability.
A new federated learning method speeds up training by selecting faster nodes first.
problem System heterogeneity and stragglers slow down federated learning.
method Adaptive selection of nodes based on data statistical characteristics.
result Significant speedup in wall-clock time compared to standard federated learning.
Artifical Neural Networks are a particular class of learning systems modeled after biological neural functions with an interesting penchant for Hebbian learning, that is "neurons that wire together, fire together". However, unlike their natural counterparts, artificial neural networks have a close and stringent couplin…
Scale of data and scale of computation infrastructures together enable the current deep learning renaissance. However, training large-scale deep architectures demands both algorithmic improvement and careful system configuration. In this paper, we focus on employing the system approach to speed up large-scale training.…
New benchmarks provide full training data for NAS research.
problem Limited training data on popular benchmarks restricts multi-fidelity techniques.
method SVD and noise modeling to create surrogate benchmarks with full training info.
result Learning curve extrapolation framework improves single-fidelity algorithms.
Develops a new trading strategy for statistical arbitrage with path-dependent signals.
problem Optimal execution in statistical arbitrage strategies with dynamic predictive signals.
method Signature-based framework modeling alpha and trading speed as linear functionals of truncated signature of market path.
result Fitted policy achieves higher return on turnover compared to a z-score benchmark.
DeepOBS benchmarks deep learning optimizers.
problem Quantitative evaluation of deep learning optimizers.
method Python package with realistic benchmarks and back-ends.
result Automates benchmarking of stochastic optimizers.
Replicated Softmax model, a well-known undirected topic model, is powerful in extracting semantic representations of documents. Traditional learning strategies such as Contrastive Divergence are very inefficient. This paper provides a novel estimator to speed up the learning based on Noise Contrastive Estimate, extende…
Fastest video anomaly detection via teacher-student distillation.
problem Anomaly detection in video at high speed.
method Adversarial knowledge distillation from object-level teacher models.
result 7-62 times faster than state-of-the-art methods.
A deep learning model speeds up computation of numerous implied volatilities.
problem Frequent computation of numerous implied volatilities using iteration methods like Newton-Raphson reaches processing speed limits.
method Emulated Newton-Raphson method using PyTorch and optimized with TensorRT.
result Up to 1,000 times faster than a benchmark implementation of Newton-Raphson.
Warm-start strategies speed up GP inference by 19x.
problem Efficient sequential inference in Gaussian processes.
method Three warm-start strategies exploiting smaller linear systems.
result Warm-starting achieves up to 19x speed-up in convergence.
AcceleratedLiNGAM speeds up causal discovery methods for large datasets.
problem Slow causal discovery methods for large-scale datasets.
method Parallelized LiNGAM method with GPU acceleration.
result Up to 32-fold speed-up on benchmark datasets.
TiDE uses MLP for fast, simple long-term time-series forecasting.
problem Long-term time-series forecasting challenges.
method Time-series Dense Encoder (TiDE) based on MLP.
result TiDE matches or outperforms Transformer models while being 5-10x faster.
Loihi neuromorphic chip outperforms conventional hardware in keyword spotting efficiency.
problem Benchmarking keyword spotting efficiency on neuromorphic hardware.
method Comparative analysis of a two-layer neural network trained to recognize a single phrase on Intel's Loihi neuromorphic chip and conventional hardware devices.
result Loihi outperforms conventional hardware on energy cost per inference for this keyword spotting application.
A simple baseline for extreme multi-label classification using random projections.
problem Automatically annotating data points with relevant labels from a large label vocabulary.
method On-the-fly global embedding using random projections, with an ensemble of learners.
result Competitive accuracy compared to existing methods, with significant speed-up and model-size reduction.
New method pools graphs with edge features for molecular data.
problem Pooling graphs with edge features for molecular data.
method Proposes two types of pooling layers compatible with edge-feature graph-convolutional architecture.
result Significantly outperforms previous benchmarks on three out of four MoleculeNet datasets.
SRSGD improves DNN training speed and accuracy.
problem Training deep neural networks is computationally expensive and slow.
method Scheduled Restart SGD (SRSGD) combines NAG momentum with momentum reset.
result SRSGD achieves better error rates with fewer training epochs.
FAVANO improves federated learning for resource-constrained environments.
problem Asynchronous communication in federated learning leads to bias and scalability issues.
method FAVANO is a novel asynchronous federated learning framework for resource-constrained environments.
result FAVANO outperforms existing methods on standard benchmarks.
This paper improves NAQ method for faster convergence on Tensorflow.
problem Non-convex optimization problems.
method Modified Nesterov's Accelerated Quasi-Newton (NAQ) method on Tensorflow.
result mNAQ converges better and faster than first and second order optimizers.
PointPillars improves object detection speed and accuracy in point clouds.
problem Encoding point clouds for efficient object detection.
method PointPillars uses PointNets to learn pillar representations of point clouds, combined with a lean downstream network.
result PointPillars outperforms previous encoders in both speed and accuracy.
Bayesian neural networks speed up numerical integration.
problem Scalability of Bayesian quadrature methods.
method Bayesian Stein networks using neural networks and Laplace approximation.
result Orders of magnitude speed-up on benchmark functions and real-world problems.
We revisit the task of learning a Euclidean metric from data. We approach this problem from first principles and formulate it as a surprisingly simple optimization problem. Indeed, our formulation even admits a closed form solution. This solution possesses several very attractive properties: (i) an innate geometric app…
We consider multi-task regression models where the observations are assumed to be a linear combination of several latent node functions and weight functions, which are both drawn from Gaussian process priors. Driven by the problem of developing scalable methods for forecasting distributed solar and other renewable powe…
ClimART dataset benchmarks ML emulators for atmospheric RT in climate models.
problem Lack of a comprehensive dataset and standardized practices for ML benchmarking in climate models.
method Builds ClimART, a large dataset with over 10 million samples, and presents novel baselines.
result Indicates shortcomings of prior datasets and network architectures.
TabPFN-3 scales tabular prediction models to large datasets and improves performance and speed.
problem Scaling tabular prediction models to large datasets with high performance and efficiency.
method Pretrained on synthetic data, introduces test-time compute scaling, and uses row-chunking.
result Significantly outperforms other models on TabArena and diverse datasets.
We present two methods, based on Chebyshev tensors, to compute dynamic sensitivities of financial instruments within a Monte Carlo simulation. These methods are implemented and run in a Monte Carlo engine to compute Dynamic Initial Margin as defined by ISDA (SIMM). We show that the levels of accuracy, speed and impleme…
A lightweight framework improves convergence and stability of PINNs for complex PDEs.
problem Training instability and reduced accuracy in PINNs for complex PDEs.
method Adaptive curvature correction using secant information to optimize first-order optimizers.
result Consistent improvements in convergence speed, stability, and accuracy over standard optimizers.