Classical Machine Learning (ML) pipelines often comprise of multiple ML models where models, within a pipeline, are trained in isolation. Conversely, when training neural network models, layers composing the neural models are simultaneously trained using backpropagation. We argue that the isolated training scheme of ML…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “Training Pipeline”
Pipelined Backpropagation trains large models without batches efficiently.
problem Training large models efficiently on hardware with limited batch sizes.
method Fine-grained Pipelined Backpropagation with Spike Compensation and Linear Weight Prediction.
result Fine-grained Pipelined Backpropagation with a batch size of one matches the accuracy of SGD for multiple networks.
Modyn automates continuous ML model training on growing datasets.
problem Continuous model retraining is costly and impractical.
method Data-centric ML platform with policies for continuous training.
result Modyn enables high throughput training with sample-level data selection.
Pipeline parallelism (PP) when training neural networks enables larger models to be partitioned spatially, leading to both lower network communication and overall higher hardware utilization. Unfortunately, to preserve the statistical efficiency of sequential training, existing PP techniques sacrifice hardware efficien…
Paper proposes EEIPU, a memoization-aware BO algorithm to reduce hyperparameter tuning costs.
problem High costs in GPU-days for training and fine-tuning language models.
method Memoization-aware Bayesian Optimization (EEIPU) algorithm in tandem with pipeline caching.
result EEIPU produces 103% more hyperparameter candidates and 108% more validation metric improvement.
Hyperparameter tuning of multi-stage pipelines introduces a significant computational burden. Motivated by the observation that work can be reused across pipelines if the intermediate computations are the same, we propose a pipeline-aware approach to hyperparameter tuning. Our approach optimizes both the design and exe…
While neural networks are good at learning unspecified functions from training samples, they cannot be directly implemented in hardware and are often not interpretable or formally verifiable. On the other hand, logic circuits are implementable, verifiable, and interpretable but are not able to learn from training data …
PipeDream-2BW accelerates large model training by 20x with minimal memory usage.
problem Training large models requires memory beyond single accelerator capacity.
method Pipeline parallelism, weight gradient coalescing, double buffering.
result Accelerates large model training by up to 20x.
OpTorch optimizes deep learning for resource-limited environments.
problem Resource constraints in deep learning training.
method Optimized deep learning pipelines in training time and memory.
result Achieved similar accuracy to existing libraries with reduced memory usage.
PaGoDA reduces diffusion model training costs by 64x.
problem Diffusion models are computationally expensive during training.
method Three-stage pipeline: downsampled training, distillation, progressive super-resolution.
result PaGoDA achieves state-of-the-art performance with reduced training costs.
Underspecified ML models can behave unpredictably in real-world use.
problem ML models can fail in real-world deployment due to ambiguous predictors.
method Identified underspecification as the cause, showing it affects various ML domains.
result Underspecified models can behave differently in deployment domains.
New Roman pipeline detects astronomical transients.
problem Automated detection of transients from Roman Space Telescope data.
method Machine learning model RuBR for distinguishing real from fake detections.
result Effective real-bogus classification in Roman era.
RL pipeline simplifies knot diagrams, including very hard unknots.
problem Simplifying complex knot diagrams, especially very hard unknots.
method Reinforcement learning for move proposals and heuristic navigation of Reidemeister moves.
result Trained agent simplifies diagrams, including a link to a three-step unknotting process.
Study recommends PLM choices for minimizing calibration error in NLP tasks.
problem Minimizing calibration error in PLM-based NLP predictions.
method Compared various options for PLM encoding, size, uncertainty quantifier, and fine-tuning loss.
result Recommendations for a well-calibrated PLM-based prediction pipeline.
Machine learning pipeline potentially consists of several stages of operations like data preprocessing, feature engineering and machine learning model training. Each operation has a set of hyper-parameters, which can become irrelevant for the pipeline when the operation is not selected. This gives rise to a hierarchica…
Predicting the outcome of sales opportunities is a core part of successful business management. Conventionally, making this prediction has relied mostly on subjective human evaluations in the process of sales decision making. In this paper, we addressed the problem of forecasting the outcome of business to business (B2…
Automatic machine learning (AutoML) is an area of research aimed at automating machine learning (ML) activities that currently require human experts. One of the most challenging tasks in this field is the automatic generation of end-to-end ML pipelines: combining multiple types of ML algorithms into a single architectu…
Unreduced PDs can perform similarly to reduced PDs in machine learning tasks.
problem Ignoring much of the information in persistence diagrams in machine learning pipelines.
method Developed methods to generate topological feature vectors from unreduced boundary matrices.
result Unreduced PDs can perform on par with, and sometimes outperform, fully-reduced PDs in machine learning tasks.
Evaluating the computational reproducibility of data analysis pipelines has become a critical issue. It is, however, a cumbersome process for analyses that involve data from large populations of subjects, due to their computational and storage requirements. We present a method to predict the computational reproducibili…
Study on AutoML robustness with dirty data.
problem Robustness of AutoML-generated pipelines with noisy data.
method Investigated TPOT, H2O, and AutoKeras systems; analyzed accuracy and pipeline structure.
result Dirty data can improve robustness of AutoML solutions.
TIER uses extended strain data to improve gravitational wave detection sensitivity.
problem Improving gravitational wave detection sensitivity using extended strain data.
method TIER framework using machine learning to capture extended strain data features.
result Up to 20% improvement in sensitive volume time in LIGO-Virgo-Kagra O3 data.
A new method simplifies noisy data filtering for CNNs.
problem Training CNNs with noisy labels is challenging.
method Joint Negative and Positive Learning (JNPL) combines NL+ and PL+ loss functions.
result Significantly simplifies the pipeline, achieving state-of-the-art accuracy.
Neural networks struggle with TSP beyond small instances, requiring new approaches.
problem Neural networks struggle to generalize to larger instances of the TSP.
method Unified pipeline to identify inductive biases and promote generalization.
result Zero-shot generalization requires rethinking neural combinatorial optimization.
Pre-training text representations have led to significant improvements in many areas of natural language processing. The quality of these models benefits greatly from the size of the pretraining corpora as long as its quality is preserved. In this paper, we describe an automatic pipeline to extract massive high-quality…
Study evaluates MRIQC pipeline's generalization on large multi-center datasets.
problem Generalizing MRI QC methods to new, unseen data.
method Evaluated MRIQC preprocessing steps on ABIDE and CATI datasets.
result Model without preprocessing yields best results on unseen data.
Differentiable pipeline replaces non-differentiable CAE components for shape optimization.
problem Gradient-based optimization is limited by non-differentiable components in CAE workflows.
method Surrogate models replace non-differentiable pipeline components, enabling gradient-based optimization.
result Gradient-based shape optimization possible without differentiable solvers.
A rigorous ML pipeline for binary classification in biomedical studies, focusing on pancreatic cancer.
problem Handling bias in ML models for complex biomedical data.
method Customizable ML analysis pipeline with 9 algorithms, hyperparameter optimization, and thorough evaluation.
result Comparison of ML algorithms to ExSTraCS, highlighting interpretability and bias handling.
This paper explores how LLMs can improve pipeline-based conversational agents.
problem Limitations of pipeline-based conversational agents in human-like conversations.
method Investigated LLMs' capabilities in two phases: design and development, and operations.
result LLMs can enhance pipeline-based agents in various tasks like data generation, intent classification, and auto-correction.
Pipeline selects best RL policy from limited data.
problem Selecting best RL policy from small datasets.
method Task- and method-agnostic pipeline using multiple data splits.
result Pipeline produces higher-performing policies.
Clapping reduces memory usage in distributed optimization by reusing data samples.
problem Significant communication overhead and impractical memory overhead in pipeline-parallel distributed optimization.
method Lazy sampling strategy to reuse data samples across steps, supporting convergence without unbiased gradient assumptions.
result Clapping achieves convergence in few-epoch or online training regimes without sample-size memory overhead.
Supervised object detection and semantic segmentation require object or even pixel level annotations. When there exist image level labels only, it is challenging for weakly supervised algorithms to achieve accurate predictions. The accuracy achieved by top weakly supervised algorithms is still significantly lower than …
Distributed training of deep nets is an important technique to address some of the present day computing challenges like memory consumption and computational demands. Classical distributed approaches, synchronous or asynchronous, are based on the parameter server architecture, i.e., worker nodes compute gradients which…
Next generation of embedded Information and Communication Technology (ICT) systems are collaborative systems able to perform autonomous tasks. The remarkable expansion of the embedded ICT market, together with the rise and breakthroughs of Artificial Intelligence (AI), have put the focus on the Edge as it stands as one…
This paper improves anomaly detection in lane rendering images for safer navigation.
problem Anomalies in lane rendering images can mislead drivers, posing safety risks.
method Proposes a four-phase pipeline using Transformer models, self-supervised pre-training, and fine-tuning.
result The pipeline enhances detection accuracy and reduces training time.
State-of-the-art speaker diarization systems utilize knowledge from external data, in the form of a pre-trained distance metric, to effectively determine relative speaker identities to unseen data. However, much of recent focus has been on choosing the appropriate feature extractor, ranging from pre-trained vectors…
The evaluation of machine learning (ML) pipelines is essential during automatic ML pipeline composition and optimisation. The previous methods such as Bayesian-based and genetic-based optimisation, which are implemented in Auto-Weka, Auto-sklearn and TPOT, evaluate pipelines by executing them. Therefore, the pipeline c…
This paper studies the problems of vehicle make & model classification. Some of the main challenges are reaching high classification accuracy and reducing the annotation time of the images. To address these problems, we have created a fine-grained database using online vehicle marketplaces of Turkey. A pipeline is prop…
Spatial and visual connectivity are important metrics when developing workplace layouts. Calculating those metrics in real-time can be difficult, depending on the size of the floor plan being analysed and the resolution of the analyses. This paper investigates the possibility of considerably speeding up the outcomes of…
Study proposes a statistical testing framework for evaluating clustering pipelines.
problem Quantifying the statistical reliability of clustering results from data analysis pipelines.
method Selective inference-based statistical testing framework for clustering pipelines.
result The proposed test controls the type I error rate and is effective in validating clustering results.
Pipeline combines ETF preprocessing with tabular model for cross-modal inference.
problem Transferability of tabular models across different modalities.
method Fixed comparison object, ETF preprocessing, in-context inference.
result Pipeline is broadly competitive, runs faster, and produces well-calibrated probabilities.
Backpropagation (BP) is the cornerstone of today's deep learning algorithms, but it is inefficient partially because of backward locking, which means updating the weights of one layer locks the weight updates in the other layers. Consequently, it is challenging to apply parallel computing or a pipeline structure to upd…
Investigates fairness in pipeline models where individuals may drop out.
problem Fairness in pipeline models where individuals may drop out and subsequent stages depend on remaining individuals.
method Rigorous framework for evaluating fairness guarantees, showing that naïve auditing is insufficient and dependence must exist between stages.
result Fairness in pipelines can be arbitrary, even with just two stages, and requires dependence between stages.
Subset selection improves weak supervision performance.
problem Optimizing the use of weakly-labeled data.
method Combining pretrained data representations with the cut statistic for subset selection.
result Subset selection improves weak supervision performance by up to 19%.
Consider a device that is connected to an edge processor via a communication channel. The device holds local data that is to be offloaded to the edge processor so as to train a machine learning model, e.g., for regression or classification. Transmission of the data to the learning processor, as well as training based o…
TempoPFN models for zero-shot time series forecasting using synthetic data.
problem Efficient long-horizon prediction and reproducibility in zero-shot time series forecasting.
method Linear RNNs pre-trained on synthetic data with GatedDeltaProduct architecture and state-weaving.
result Achieves top-tier competitive performance on various benchmarks.
Simplified image clustering achieves competitive results without text-based embeddings.
problem Complexity and resource requirements of state-of-the-art clustering methods.
method SCP: trains a small cluster head using pre-trained vision model features and positive data pairs.
result SCP achieves highly competitive performance on various benchmark datasets.
Study shows noisy data collection in ImageNet leads to biased model performance.
problem Noisy data collection in ImageNet datasets causes biased model performance.
method Human studies on ImageNet dataset to investigate design choices and their impact.
result Systematic misalignment between ImageNet benchmark and real-world task.
Data science relies on pipelines that are organized in the form of interdependent computational steps. Each step consists of various candidate algorithms that maybe used for performing a particular function. Each algorithm consists of several hyperparameters. Algorithms and hyperparameters must be optimized as a whole …