AVATAR uses a surrogate model to quickly evaluate ML pipelines, saving time and resources.
problem Time-consuming evaluation of ML pipelines limits exploration of complex models.
method AVATAR employs a surrogate model to assess pipeline validity without execution.
result AVATAR accelerates ML pipeline evaluation, improving efficiency in complex scenarios.
New approach identifies and explains errors in machine learning pipelines.
problem Challenges in identifying and explaining errors in complex machine learning pipelines.
method Uses iteration and provenance to automatically infer root causes of failures.
result Significantly improves precision and recall compared to state-of-the-art methods.
RankML predicts machine learning pipeline performance quickly and accurately.
problem Lack of machine learning experts and high computational costs.
method Meta-learning approach to predict pipeline performance.
result RankML outperforms or matches state-of-the-art approaches with less time and resources.
Classical Machine Learning (ML) pipelines often comprise of multiple ML models where models, within a pipeline, are trained in isolation. Conversely, when training neural network models, layers composing the neural models are simultaneously trained using backpropagation. We argue that the isolated training scheme of ML…
Study on AutoML robustness with dirty data.
problem Robustness of AutoML-generated pipelines with noisy data.
method Investigated TPOT, H2O, and AutoKeras systems; analyzed accuracy and pipeline structure.
result Dirty data can improve robustness of AutoML solutions.
Machine learning pipeline potentially consists of several stages of operations like data preprocessing, feature engineering and machine learning model training. Each operation has a set of hyper-parameters, which can become irrelevant for the pipeline when the operation is not selected. This gives rise to a hierarchica…
Much of the work in metalearning has focused on classifier selection, combined more recently with hyperparameter optimization, with little concern for data preprocessing. Yet, it is generally well accepted that machine learning applications require not only model building, but also data preprocessing. In other words, p…
Modyn automates continuous ML model training on growing datasets.
problem Continuous model retraining is costly and impractical.
method Data-centric ML platform with policies for continuous training.
result Modyn enables high throughput training with sample-level data selection.
To solve a machine learning problem, one typically needs to perform data preprocessing, modeling, and hyperparameter tuning, which is known as model selection and hyperparameter optimization.The goal of automated machine learning (AutoML) is to design methods that can automatically perform model selection and hyperpara…
DeepLine automates ML pipeline generation using reinforcement learning.
problem Automatic generation of end-to-end ML pipelines combining multiple algorithms.
method Deep Reinforcement Learning with hierarchical actions filtering.
result DeepLine outperforms state-of-the-art approaches in accuracy and computational cost.
Robust machine learning relies on access to data that can be used with standardized frameworks in important tasks and the ability to develop models whose performance can be reasonably reproduced. In machine learning for healthcare, the community faces reproducibility challenges due to a lack of publicly accessible data…
In order to achieve state-of-the-art performance, modern machine learning techniques require careful data pre-processing and hyperparameter tuning. Moreover, given the ever increasing number of machine learning models being developed, model selection is becoming increasingly important. Automating the selection and tuni…
As machine learning is applied more widely, data scientists often struggle to find or create end-to-end machine learning systems for specific tasks. The proliferation of libraries and frameworks and the complexity of the tasks have led to the emergence of "pipeline jungles" - brittle, ad hoc ML systems. To address thes…
Unreduced PDs can perform similarly to reduced PDs in machine learning tasks.
problem Ignoring much of the information in persistence diagrams in machine learning pipelines.
method Developed methods to generate topological feature vectors from unreduced boundary matrices.
result Unreduced PDs can perform on par with, and sometimes outperform, fully-reduced PDs in machine learning tasks.
Data science relies on pipelines that are organized in the form of interdependent computational steps. Each step consists of various candidate algorithms that maybe used for performing a particular function. Each algorithm consists of several hyperparameters. Algorithms and hyperparameters must be optimized as a whole …
GAMA is a user-friendly AutoML system for machine learning pipeline optimization.
problem Empowering users to control and optimize machine learning pipelines.
method Modular AutoML system with three search algorithms and two post-processing steps.
result GAMA allows users to track and control the search process for optimal machine learning pipelines.
Paper proposes EEIPU, a memoization-aware BO algorithm to reduce hyperparameter tuning costs.
problem High costs in GPU-days for training and fine-tuning language models.
method Memoization-aware Bayesian Optimization (EEIPU) algorithm in tandem with pipeline caching.
result EEIPU produces 103% more hyperparameter candidates and 108% more validation metric improvement.
A rigorous ML pipeline for binary classification in biomedical studies, focusing on pancreatic cancer.
problem Handling bias in ML models for complex biomedical data.
method Customizable ML analysis pipeline with 9 algorithms, hyperparameter optimization, and thorough evaluation.
result Comparison of ML algorithms to ExSTraCS, highlighting interpretability and bias handling.
New machine learning pipeline solves dynamic vehicle routing problems efficiently.
problem Efficiently handling same day deliveries in e-commerce logistics.
method Combination of machine learning and combinatorial optimization.
result Ranked first in the EURO Meets NeurIPS Vehicle Routing Competition.
Data science relies on pipelines that are organized in the form of interdependent computational steps. Each step consists of various candidate algorithms that maybe used for performing a particular function. Each algorithm consists of several hyperparameters. Algorithms and hyperparameters must be optimized as a whole …
Clapping reduces memory usage in distributed optimization by reusing data samples.
problem Significant communication overhead and impractical memory overhead in pipeline-parallel distributed optimization.
method Lazy sampling strategy to reuse data samples across steps, supporting convergence without unbiased gradient assumptions.
result Clapping achieves convergence in few-epoch or online training regimes without sample-size memory overhead.
The paper proposes a ML workflow for B2B sales prediction.
problem Predicting business to business sales outcomes using subjective human evaluations.
method A two-pipeline ML approach on Azure ML, including data enrichment and model training.
result ML predictions improve accuracy and increase sales value.
Hyperparameter tuning of multi-stage pipelines introduces a significant computational burden. Motivated by the observation that work can be reused across pipelines if the intermediate computations are the same, we propose a pipeline-aware approach to hyperparameter tuning. Our approach optimizes both the design and exe…
In this demo paper, we introduce the DARPA D3M program for automatic machine learning (ML) and JPL's MARVIN tool that provides an environment to locate, annotate, and execute machine learning primitives for use in ML pipelines. MARVIN is a web-based application and associated back-end interface written in Python that e…
Topological data analysis classifies encrypted bits with success.
problem Classifying encrypted data with traditional machine learning methods.
method Persistent homology for generating topological features, machine learning pipeline.
result Successfully classifies encrypted data, outperforming classical models.
Automatic machine learning is an important problem in the forefront of machine learning. The strongest AutoML systems are based on neural networks, evolutionary algorithms, and Bayesian optimization. Recently AlphaD3M reached state-of-the-art results with an order of magnitude speedup using reinforcement learning with …
AMPL is a new software pipeline for drug discovery models.
problem Reproducibility and traceability in drug discovery models.
method Modular and extensible software pipeline for machine learning models.
result Physicochemical descriptors and deep learning outperform traditional fingerprints.
Underspecified ML models can behave unpredictably in real-world use.
problem ML models can fail in real-world deployment due to ambiguous predictors.
method Identified underspecification as the cause, showing it affects various ML domains.
result Underspecified models can behave differently in deployment domains.
DC-Check helps guide ML development by considering data-centric aspects.
problem Lack of standardized framework for data-centric considerations in ML.
method DC-Check is a checklist-style framework for data-centric AI at ML pipeline stages.
result Promotes thoughtfulness and transparency in ML development.
This work facilitates ensuring fairness of machine learning in the real world by decoupling fairness considerations in compound decisions. In particular, this work studies how fairness propagates through a compound decision-making processes, which we call a pipeline. Prior work in algorithmic fairness only focuses on f…
ART improves transfer learning performance with robust theory and methods.
problem Improving performance of primary tasks using auxiliary data.
method Adaptive Robust Transfer Learning (ART) pipeline with theoretical guarantees.
result ART provides a provable theoretical guarantee for adaptive transfer and robustness.
Machine learning has been gaining traction in recent years to meet the demand for tools that can efficiently analyze and make sense of the ever-growing databases of biomedical data in health care systems around the world. However, effectively using machine learning methods requires considerable domain expertise, which …
PyKale bridges interdisciplinary ML with Python, enabling accurate predictions.
problem Cross-disciplinary barriers in machine learning.
method Knowledge-aware machine learning on graphs, images, texts, and videos.
result Enables multimodal learning and transfer learning with latest deep learning models.
Pipelined Backpropagation trains large models without batches efficiently.
problem Training large models efficiently on hardware with limited batch sizes.
method Fine-grained Pipelined Backpropagation with Spike Compensation and Linear Weight Prediction.
result Fine-grained Pipelined Backpropagation with a batch size of one matches the accuracy of SGD for multiple networks.
Develops a machine learning pipeline for learning causal structure in time-series data.
problem Current ML algorithms fail to learn causal structure in time-series data due to lack of temporal order consideration.
method Integrates machine learning with chaos theory using ChaosFEX feature extractor to learn generalized causal structure.
result Successfully learns generalized causal structure in time-series data.
New Roman pipeline detects astronomical transients.
problem Automated detection of transients from Roman Space Telescope data.
method Machine learning model RuBR for distinguishing real from fake detections.
result Effective real-bogus classification in Roman era.
The paper aims to improve reproducibility of machine learning experiments.
problem Lack of reproducibility in machine learning experiments.
method Investigates factors affecting reproducibility and proposes FAIR data practices.
result Preliminary results show the role of ProvBook in capturing and comparing provenance.
New pipeline for causal research in psychology and social sciences.
problem Underuse of causal approaches in psychology and social science.
method Formal specification of theories, reduction of complexity, estimation of causal effects.
result Facilitates scientific inquiry compatible with testing causal theories.
Investigates fairness in pipeline models where individuals may drop out.
problem Fairness in pipeline models where individuals may drop out and subsequent stages depend on remaining individuals.
method Rigorous framework for evaluating fairness guarantees, showing that naïve auditing is insufficient and dependence must exist between stages.
result Fairness in pipelines can be arbitrary, even with just two stages, and requires dependence between stages.
This dissertation automates deep learning pipelines and uses meta-learning for better model selection and data augmentation.
problem Challenges in selecting and fine-tuning deep learning pipelines for new datasets.
method Meta-learning for DL pipeline selection and data augmentation, using synthetic data.
result Meta-learned approaches outperform traditional methods in automated DL pipeline selection and data augmentation.
TIER uses extended strain data to improve gravitational wave detection sensitivity.
problem Improving gravitational wave detection sensitivity using extended strain data.
method TIER framework using machine learning to capture extended strain data features.
result Up to 20% improvement in sensitive volume time in LIGO-Virgo-Kagra O3 data.
We introduce an automatic machine learning (AutoML) modeling architecture called Autostacker, which combines an innovative hierarchical stacking architecture and an Evolutionary Algorithm (EA) to perform efficient parameter search. Neither prior domain knowledge about the data nor feature preprocessing is needed. Using…
Clinical prognostic models derived from largescale healthcare data can inform critical diagnostic and therapeutic decisions. To enable off-theshelf usage of machine learning (ML) in prognostic research, we developed AUTOPROGNOSIS: a system for automating the design of predictive modeling pipelines tailored for clinical…
OpTorch optimizes deep learning for resource-limited environments.
problem Resource constraints in deep learning training.
method Optimized deep learning pipelines in training time and memory.
result Achieved similar accuracy to existing libraries with reduced memory usage.
The paper uses deep learning to speed up spatial and visual connectivity analysis.
problem Slow calculation of spatial and visual connectivity metrics.
method Investigates machine learning models and a pipeline for training them on spatial and visual connectivity analysis.
result Deep learning models significantly speed up the analysis process.
Study shows noisy data collection in ImageNet leads to biased model performance.
problem Noisy data collection in ImageNet datasets causes biased model performance.
method Human studies on ImageNet dataset to investigate design choices and their impact.
result Systematic misalignment between ImageNet benchmark and real-world task.
Interpretable survival analysis improves heart failure risk prediction.
problem Improving heart failure risk prediction using survival analysis.
method Survival stacking, ControlBurn, Explainable Boosting Machines.
result Achieves state-of-the-art performance and provides novel insights.
Probabilistic ML improves healthcare data analysis.
problem Insufficient understanding and incomplete data in healthcare.
method Examination of probabilistic machine learning models for healthcare challenges.
result Probabilistic models enhance healthcare data analysis and model building.