New model optimizes oil product distribution via pipelines.
problem Optimizing oil product distribution via pipelines.
method Discrete-time mixed integer linear programming model.
result Significant reductions in pipeline operational cost.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,051 papers · 148 categories
Trend · papers per month
48 results for “input pipeline”
Paper introduces Modular Jets for diagnosing model decompositions in pipelines.
problem Evaluating model decompositions in pipelines for unique identification.
method Estimates empirical jets from module-level representations to diagnose mirage vs identifiable decompositions.
result Proves jet-identifiability theorem for two-module linear regression pipelines.
Proposes VEESA pipeline for interpreting ML models with functional data.
problem Interpreting ML models with functional data inputs for high-consequence applications.
method VEESA pipeline using efPCA and PFI.
result Provides explanation of model's use of functional data variability for prediction.
Paper discusses optimizing deep learning models for supercomputers.
problem Training deep learning models at high petaFLOPS scale.
method Three systems-related optimizations: distributed batch normalization, input pipeline, and 2-D torus all-reduce.
result Trained ResNet-50 on ImageNet to 76.3% accuracy in 2.2 minutes.
Mathematical pipeline identifies structural homology of knotted proteins.
problem Quantification and classification of protein structures, especially knotted proteins, require noise-free and complete data.
method Developed a geometric framework using persistent homology to analyze protein structures.
result Persistent homology accurately represents structural homology of knotted proteins and identifies geometric features of protein entanglement.
We introduce a convolutional neural network that operates directly on graphs. These networks allow end-to-end learning of prediction pipelines whose inputs are graphs of arbitrary size and shape. The architecture we present generalizes standard molecular feature extraction methods based on circular fingerprints. We sho…
Metareasoning optimizes modular systems by dynamically adjusting configurations.
problem Maximizing system utility in high-stakes tasks with modular subsystems.
method Employing reinforcement learning with rich contextual representations to dynamically adjust module configurations.
result Significant improvement in system performance across various reinforcement learning techniques.
Enhances safety of 3D object detection neural networks.
problem Ensuring robustness and safety of 3D object detection systems.
method Symbolic error propagation, specialized loss function, safety-aware non-max-inclusion algorithm.
result Improved safety and robustness of 3D object detection neural networks.
AVATAR uses a surrogate model to quickly evaluate ML pipelines, saving time and resources.
problem Time-consuming evaluation of ML pipelines limits exploration of complex models.
method AVATAR employs a surrogate model to assess pipeline validity without execution.
result AVATAR accelerates ML pipeline evaluation, improving efficiency in complex scenarios.
Data preprocessing improves data quality for robust data mining.
problem Noisy and incomplete data hinders data mining models.
method Overview of data cleaning, transformation, and preprocessing methods.
result Preprocessing significantly affects data mining model performance.
Stochastic Deep Networksstat.ML
Proposes deep learning methods for handling random vectors.
problem Handling flexible input data like probability measures in deep learning.
method Develops deep architectures to handle permutation invariances, varying weights, and cardinality.
result Demonstrates the effectiveness of deep architectures on measures for classification, reduction, and prediction.
Pipeline-aware hyperparameter tuning speeds up machine learning pipelines by reusing intermediate computations.
problem High computational burden in hyperparameter tuning of multi-stage pipelines.
method Proposes a hybrid hyperparameter tuning method and a caching problem formulated as an ILP to maximize reuse.
result Pipeline-aware approach offers over an order-of-magnitude speedup over independent evaluations.
Investigates neural TTS systems for Japanese and English.
problem Improving neural TTS systems for high-quality speech synthesis.
method Comparative study of neural sequence-to-sequence TTS vs. DNN pipeline TTS, varying model architecture, parameter size, and language.
result A neural sequence-to-sequence TTS system requires sufficient model parameters and a powerful encoder for high-quality speech synthesis.
The study uses Gaussian mixture models to estimate pipe wall thickness from partial scans.
problem Estimating remaining wall thickness of critical water mains in non-destructive evaluation.
method Developed a robotic vehicle for pipe inspections. Used Gaussian Processes and Gaussian Mixture Models to infer RWT at unseen sections.
result Gaussian mixture models effectively capture the probability of RWT values in inspected data.
We translate ML pipelines into neural networks to optimize multiple models together.
problem Isolated training of ML pipelines limits joint optimization of multiple models.
method Propose translating ML pipelines into neural networks and fine-tuning them jointly.
result Fine-tuning translated pipelines increases final accuracy.
New method identifies parameters of wider shallow neural networks with biases.
problem Identifying parameters of wide shallow neural networks with biases from finite samples.
method Two-step pipeline: direction of weights via second order information, signs via algebraic evaluations, biases via gradient descent.
result Constructive methods and theoretical guarantees of finite sample identification for wider shallow networks with biases.
Study proposes a statistical testing framework for evaluating clustering pipelines.
problem Quantifying the statistical reliability of clustering results from data analysis pipelines.
method Selective inference-based statistical testing framework for clustering pipelines.
result The proposed test controls the type I error rate and is effective in validating clustering results.
Investigates fairness in pipeline models where individuals may drop out.
problem Fairness in pipeline models where individuals may drop out and subsequent stages depend on remaining individuals.
method Rigorous framework for evaluating fairness guarantees, showing that naïve auditing is insufficient and dependence must exist between stages.
result Fairness in pipelines can be arbitrary, even with just two stages, and requires dependence between stages.
Method quantifies error from pipeline components in image classification.
problem Understanding error sources in complex machine learning pipelines.
method Computes error contribution and propagation from computational steps, algorithms, and hyperparameters.
result Random search method accurately quantifies error contribution and propagation.
Method quantifies error contributions in image classification pipelines.
problem Understanding and optimizing complex machine learning pipelines.
method Quantifies error contributions from computational steps, algorithms, and hyperparameters.
result Random search outperforms Bayesian optimization in quantifying error contributions.
SSRCA: a novel machine learning pipeline to perform sensitivity analysis for agent-based modelsq-bio.QM
SSRCA simplifies ABM sensitivity analysis using machine learning.
problem Hardness of performing sensitivity analysis for complex ABMs.
method Machine learning pipeline (Simulate, Summarize, Reduce, Cluster, Analyze) for ABMs.
result SSRCA identifies sensitive parameters and common output patterns for ABMs.
PipeMare enables efficient DNN training with minimal memory and pipeline sacrifices.
problem Sacrificing hardware efficiency to maintain statistical efficiency in pipeline parallel DNN training.
method PipeMare is a simple yet robust training method that tolerates asynchronous updates during pipeline parallelism without sacrificing pipeline utilization or memory.
result PipeMare achieves up to 2.7x less memory usage or 4.3x higher pipeline utilization compared to state-of-the-art synchronous PP training techniques.
MicAugment transfers audio style from few seconds of input to match target conditions.
problem Audio model robustness to diverse acquisition conditions.
method Identifies and applies transformations learned from target audio to input audio.
result MicAugment significantly improves model robustness in downstream tasks.
Automates supervised learning pipeline design with matrix and tensor factorization.
problem Designing effective supervised learning pipelines with many choices.
method Uses matrix and tensor factorization to model pipeline search space and develops greedy experiment design protocols.
result Demonstrates the effectiveness of the approach on real-world classification problems.
KM-GPT: An Automated Pipeline for Reconstructing Individual Patient Data from Kaplan-Meier Plotscs.LG
KM-GPT automates IPD reconstruction from KM plots with high accuracy and scalability.
problem Manual digitization of IPD from KM plots is error-prone and lacks scalability.
method KM-GPT integrates advanced image preprocessing, multi-modal reasoning, and iterative reconstruction algorithms.
result KM-GPT generates high-quality IPD without manual input or intervention, achieving superior accuracy.
Pipeline decomposes portfolio optimization problems into smaller, solvable subproblems.
problem Large-scale portfolio optimization with constraints.
method Decomposition pipeline with preprocessing, clustering, and risk rebalancing.
result Pipeline reduces problem size by 80% and computation time.
Paper proposes a statistical test for feature selection pipelines using selective inference.
problem Assessing the significance of feature selection pipelines in data analysis.
method Selective inference technique applied to feature selection pipelines composed of various algorithms.
result The proposed statistical test controls false positive feature selection probabilities.
Metalearning finds preprocessing often hinders accuracy, but can improve pipelines.
problem The role of preprocessing in machine learning pipelines is underexplored.
method Extensive empirical study over various algorithms and preprocessors using metalearning.
result Metalearning suggests preprocessing should be used in ML pipeline design.
Cancer survival prediction is an active area of research that can help prevent unnecessary therapies and improve patient's quality of life. Gene expression profiling is being widely used in cancer studies to discover informative biomarkers that aid predict different clinical endpoint prediction. We use multiple modalit…
New approach identifies and explains errors in machine learning pipelines.
problem Challenges in identifying and explaining errors in complex machine learning pipelines.
method Uses iteration and provenance to automatically infer root causes of failures.
result Significantly improves precision and recall compared to state-of-the-art methods.
A hybrid RL-Bayesian search configures machine learning pipelines efficiently.
problem Optimizing hyper-parameters in a hierarchical conditional space.
method Combines Reinforcement Learning and Bayesian Optimization.
result Outperforms state-of-the-art methods in pipeline optimization.
Pipelined Backpropagation trains large models without batches efficiently.
problem Training large models efficiently on hardware with limited batch sizes.
method Fine-grained Pipelined Backpropagation with Spike Compensation and Linear Weight Prediction.
result Fine-grained Pipelined Backpropagation with a batch size of one matches the accuracy of SGD for multiple networks.
Optimizes calibration error estimators for better classifier trustworthiness.
problem Lack of guidance on selecting and tuning calibration error estimators.
method Reformulates calibration estimation as a regression problem with i.i.d. input pairs.
result Demonstrates the effectiveness of optimized calibration estimators on image classification tasks.
Sparsity helps reduce diffusion model costs.
problem High computational costs in diffusion models.
method Introduced sparsity concept to reduce input dimensionality.
result Sparsity reduces computational complexity to intrinsic data dimension.
Multilayer Network Analysis of the Drug Pipeline in the Global Pharmaceutical Industryphysics.soc-ph
Study reveals similarities in knowledge flows between pharmaceutical and AI industries.
problem Understanding the dynamics of drug pipelines in global pharmaceutical industry.
method Multilayer network analysis of drug pipeline, global supply chain, and ownership data.
result Proven similarities in knowledge flows between pharmaceutical and AI industries.
Modyn automates continuous ML model training on growing datasets.
problem Continuous model retraining is costly and impractical.
method Data-centric ML platform with policies for continuous training.
result Modyn enables high throughput training with sample-level data selection.
Inspired by recent work on neural network image generation which rely on backpropagation towards the network inputs, we present a proof-of-concept system for speech texture synthesis and voice conversion based on two mechanisms: approximate inversion of the representation learned by a speech recognition neural network,…
TODS automates time series outlier detection with customizable pipelines.
problem Automated detection of outliers in time series data.
method Modular system with 70 primitives for data processing, time series analysis, and detection algorithms. GUI and data-driven searcher for pipeline design.
result Automated discovery and construction of effective outlier detection pipelines.
Hungarian text processing improved with efficient, accurate NLP pipelines.
problem Improving text processing for Hungarian language.
method Implemented in spaCy framework, focusing on efficiency and accuracy.
result Near state-of-the-art performance in all text preprocessing steps.
A rigorous ML pipeline for binary classification in biomedical studies, focusing on pancreatic cancer.
problem Handling bias in ML models for complex biomedical data.
method Customizable ML analysis pipeline with 9 algorithms, hyperparameter optimization, and thorough evaluation.
result Comparison of ML algorithms to ExSTraCS, highlighting interpretability and bias handling.
Adaptive Bayesian model optimizes machine learning pipelines.
problem Automated model selection and hyperparameter tuning for datasets.
method Combines adaptive Bayesian regression and neural network basis function with acquisition function.
result Identifies high-performance pipelines efficiently and outperforms baseline methods.
Gaussian processes are improved to account for input noise in earth observation.
problem Accurate error assessment in earth observation models.
method Propose a GP model that propagates input noise through the pipeline.
result Improved error representation in temperature predictions from infrared data.
Deep Sets improve reinforcement learning for autonomous driving with variable inputs.
problem Optimal decision making in autonomous driving with varying number of objects.
method Employed Deep Sets for high-level decision making in reinforcement learning.
result Deep Sets outperform other approaches in performance and generalization.
Pipeline for comparing trading algorithms in finance and crypto.
problem Disconnected research and applications in algorithmic trading.
method General pipeline for designing, programming, and evaluating trading strategies.
result Systematic comparison of trading algorithms in finance and crypto.
DeepLine automates ML pipeline generation using reinforcement learning.
problem Automatic generation of end-to-end ML pipelines combining multiple algorithms.
method Deep Reinforcement Learning with hierarchical actions filtering.
result DeepLine outperforms state-of-the-art approaches in accuracy and computational cost.
LaMBO optimizes modular systems with switching costs, achieving better results than existing methods.
problem Optimizing systems with costly variable updates in a sequence of modules.
method Lazy Modular Bayesian Optimization (LaMBO) that minimizes switching costs.
result LaMBO achieves vanishing regret and improves over existing cost-aware Bayesian optimization algorithms.
The success of deep learning has been due, in no small part, to the availability of large annotated datasets. Thus, a major bottleneck in current learning pipelines is the time-consuming human annotation of data. In scenarios where such input-output pairs cannot be collected, simulation is often used instead, leading t…
RankML predicts machine learning pipeline performance quickly and accurately.
problem Lack of machine learning experts and high computational costs.
method Meta-learning approach to predict pipeline performance.
result RankML outperforms or matches state-of-the-art approaches with less time and resources.