Simulation workflow is a top-level model for the design and control of simulation process. It connects multiple simulation components with time and interaction restrictions to form a complete simulation system. Before the construction and evaluation of the component models, the validation of upper-layer simulation work…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “workflow”
Paper presents a workflow for reliable unsupervised learning in science.
problem Lack of standardization in unsupervised learning workflows for reproducible scientific discoveries.
method Structured workflow including data preparation, modeling, validation, and communication.
result Illustrates the importance of validation in unsupervised learning.
This paper analyzes machine learning workflows in climate modeling.
problem Challenges in integrating machine learning with climate modeling.
method Analysis of case studies focusing on design patterns and workflow structure.
result Synthesis of workflow design patterns across diverse projects in ML-enabled climate modeling.
Adaptive workflow combines fast amortized inference with MCMC for many datasets.
problem Trade-off between computational speed and sampling accuracy in Bayesian inference.
method Adaptive workflow integrating amortized inference and MCMC with principled diagnostics.
result Efficiency gains with high posterior quality on tens of thousands of datasets.
Generative AI agents improve ERP systems by automating complex financial tasks.
problem Static, rule-based workflows limit adaptability and intelligence in ERP systems.
method Introducing Generative Business Process AI Agents (GBPAs) that integrate generative AI with business process modeling and multi-agent orchestration.
result GBPAs achieve up to 40% reduction in processing time and 94% drop in error rate.
Researchers found that avoiding synthetic data generation prevents model collapse in machine learning.
problem Model collapse in machine learning where models degenerate over generations.
method Comparing discard and augment workflows, focusing on Linear Regression.
result Theoretical evidence shows that for Linear Regression, test risk is bounded by π²/6 of original data alone.
Audit financial machine learning workflows to detect spurious predictability.
problem Spurious predictability in financial machine learning models.
method Falsification audit testing predictive workflows against synthetic environments.
result Many apparent financial predictions are artifacts, not genuine.
When Should an AI Workflow Release? Always-Valid Inference for Black-Box Generate-Verify Systemsstat.ML
A new method for releasing AI workflows to avoid premature incorrect results.
problem Statistical challenges in releasing AI workflows with adaptive scoring.
method Wrapper that calibrates and accumulates evidence from high-scoring failures.
result Reduces premature incorrect release while still releasing on moderate evidence.
Machine learning workflow development is anecdotally regarded to be an iterative process of trial-and-error with humans-in-the-loop. However, we are not aware of quantitative evidence corroborating this popular belief. A quantitative characterization of iteration can serve as a benchmark for machine learning workflow d…
One of the impediments in advancing actuarial research and developing open source assets for insurance analytics is the lack of realistic publicly available datasets. In this work, we develop a workflow for synthesizing insurance datasets leveraging CTGAN, a recently proposed neural network architecture for generating …
Human-in-the-loop data analysis applications necessitate greater transparency in machine learning models for experts to understand and trust their decisions. To this end, we propose a visual analytics workflow to help data scientists and domain experts explore, diagnose, and understand the decisions made by a binary cl…
Deep learning model improves seismic rock property estimation.
problem Estimating reservoir rock properties from seismic reflection data.
method Proposes a deep learning-based seismic inversion workflow that models seismic traces spatiotemporally.
result Achieves best performance on SEAM dataset with coefficient of 79.77\%
Deep learning speeds up pressure prediction in carbon storage reservoirs.
problem Accurately forecasting reservoir pressure in geologic carbon storage projects with sparse well data.
method Combining InSAR surface displacement data with deep learning and data assimilation techniques.
result Workflow can predict reservoir pressure with high efficiency and uncertainty quantification.
The present study provides a comparative assessment of non-invasive sensors as means of estimating the microbial contamination and time-on-shelf (i.e. storage time) of leafy green vegetables, using a novel unified spectra analysis workflow. Two fresh ready-to-eat green salads were used in the context of this study for …
Paper proposes an active learning method for surgical workflow recognition using long-range temporal dependency.
problem Challenges in automatic surgical workflow recognition due to lack of large-scale labelled datasets.
method NL-RCNet with non-local block for capturing long-range temporal dependency and intra-clip dependency score for selection.
result Our approach outperforms state-of-the-art methods by selecting only 50% of samples for training.
This thesis builds a real-time VaR calculation workflow for crypto derivatives.
problem Managing risk in volatile cryptocurrency markets.
method Applied EMWA, GARCH, and HAR models to forecast volatility; used delta-gamma-theta approach and Cornish-Fisher expansion.
result Real-time VaR estimates with millisecond calculation latencies.
Testing (conditional) independence of multivariate random variables is a task central to statistical inference and modelling in general - though unfortunately one for which to date there does not exist a practicable workflow. State-of-art workflows suffer from the need for heuristic or subjective manual choices, high c…
Benchpress streamlines benchmarking structure learning algorithms for probabilistic models.
problem Benchmarking structure learning algorithms for probabilistic graphical models.
method Snakemake workflow for scalable, reproducible benchmarks.
result Demonstrates applicability in five typical data scenarios.
Data application developers and data scientists spend an inordinate amount of time iterating on machine learning (ML) workflows -- by modifying the data pre-processing, model training, and post-processing steps -- via trial-and-error to achieve the desired model performance. Existing work on accelerating machine learni…
Feature engineering workflow for activity recognition from synchronized inertial measurement unitscs.LG
Automates feature extraction for IMU-based activity recognition.
problem Manual feature engineering is time-consuming and limits IMU-based applications.
method FRESH algorithm for automated feature extraction.
result Workflow automates feature extraction from synchronized IMU sensors.
GSR optimizes tasks in scientific workflows, improving performance across diverse applications.
problem Uncertainty in task selection and evaluation in scientific workflow optimization.
method Generate-Select-Refine (GSR) framework that alternates between task generation and optimization.
result GSR outperforms existing LLM-based optimizers in various scientific applications.
The paper proposes a ML workflow for B2B sales prediction.
problem Predicting business to business sales outcomes using subjective human evaluations.
method A two-pipeline ML approach on Azure ML, including data enrichment and model training.
result ML predictions improve accuracy and increase sales value.
Study evaluates deep learning methods for dermatology, finding they perform poorly under non-ideal conditions.
problem Lack of robustness of deep learning methods in dermatology under real-world conditions.
method Simulated non-ideal conditions on user-submitted dermatology images.
result Deep learning methods show significant drop in accuracy and prediction changes under non-ideal conditions.
BayesFlow trains neural networks for fast Bayesian inference.
problem Fast Bayesian inference for complex models.
method Amortized neural networks for intractable posterior distributions.
result Fast inference through pre-trained neural networks.
Foundation models alter medical data science workflow, challenging veridical data science principles.
problem Foundation models disrupt traditional data science practices in medicine.
method Critically examined the medical foundation model lifecycle and its deviation from veridical data science principles.
result Foundation models challenge veridical data science principles of predictability, computability, and stability.
Workflow improves credit default prediction using machine learning.
problem Assessing creditworthiness and risk management in lending.
method Data preprocessing with Weight of Evidence, ensemble learning, and hyperparameter optimization.
result Enhanced accuracy in predicting credit default.
A GPU-based workflow for building physics emulators of hypersonic flows
problem Resolving complex physical phenomena in hypersonic flows
method Fully GPU-based workflow integrating accelerated data generation and neural emulators
result Physics emulators remain reliable beyond their training distribution
Strategic Geosteering Workflow with Uncertainty Quantification and Deep Learning: A Case Study on the Goliat Fieldphysics.geo-ph
Workflow uses deep learning to improve geosteering accuracy in Goliat Field.
problem Uncertainty in geological models and forward simulations affects real-time estimations.
method Offline DNN training, online FlexIES with probabilistic estimation.
result Median probabilistic estimation matches proprietary inversion.
Data science teams collaborate extensively, using various tools and stakeholders.
problem Lack of understanding in how data science workers collaborate in practice.
method Conducted an online survey with 183 data science workers.
result Data science teams are highly collaborative and use multiple tools and stakeholders.
This paper describes HyperStream, a large-scale, flexible and robust software package, written in the Python language, for processing streaming data with workflow creation capabilities. HyperStream overcomes the limitations of other computational engines and provides high-level interfaces to execute complex nesting, fu…
bioLeak addresses data leakage in biomedical machine learning studies.
problem Data leakage causes optimistic bias in machine learning models for biomedical studies.
method bioLeak provides leakage-aware resampling workflows and model audits in R.
result The package supports various machine learning tasks and can detect leakage mechanisms.
Qlib aims to integrate AI into quantitative investment.
problem Challenges in applying AI to quantitative investment.
method Design and develop Qlib to accommodate AI-driven workflow.
result Qlib realizes the potential of AI technologies in quantitative investment.
Quantifying the importance of each training point to a learning task is a fundamental problem in machine learning and the estimated importance scores have been leveraged to guide a range of data workflows such as data summarization and domain adaption. One simple idea is to use the leave-one-out error of each training …
Automated surgical workflow analysis and understanding can assist surgeons to standardize procedures and enhance post-surgical assessment and indexing, as well as, interventional monitoring. Computer-assisted interventional (CAI) systems based on video can perform workflow estimation through surgical instruments' recog…
AI helps simplify complex ship finance processes.
problem Complexity in ship finance due to data and regulatory requirements.
method Integrates large language models for document comprehension, information extraction, and workflow automation.
result AI-assisted systems can support maritime finance professionals in managing complex information and reporting requirements.
Unified platform for statistical and machine learning in bioinformatics.
problem Workflow inefficiencies in using multiple tools for data analysis.
method Automated hyperparameter optimization, feature importance analysis, statistical tests.
result Accelerates biological discovery workflows with methodological soundness.
An online framework improves investment management by making incremental updates.
problem Offline investment processes restrict silos from collectively pursuing a unified goal.
method Developed an online algorithm workflow for portfolio management.
result The online framework outperforms market benchmarks and reduces overfitting.
Machine Learning (ML) has been successfully applied to a wide range of domains and applications. One of the techniques behind most of these successful applications is Ensemble Learning (EL), the field of ML that gave birth to methods such as Random Forests or Boosting. The complexity of applying these techniques togeth…
The workflow of data scientists normally involves potentially inefficient processes such as data mining, feature engineering and model selection. Recent research has focused on automating this workflow, partly or in its entirety, to improve productivity. We choose the former approach and in this paper share our experie…
Veridical Data Sciencestat.ML
Building and expanding on principles of statistics, machine learning, and scientific inquiry, we propose the predictability, computability, and stability (PCS) framework for veridical data science. Our framework, comprised of both a workflow and documentation, aims to provide responsible, reliable, reproducible, and tr…
End-to-end portfolio optimization using quantum annealing for financial decision problems.
problem Optimizing financial portfolios with quantum computing constraints.
method Hybrid pipeline combining quantum and classical optimization.
result Quantum-assisted portfolio optimization can achieve competitive returns.
dynestyx: A library for probabilistic programming of dynamical systems
problem integrating state-space models into probabilistic programming languages
method a unified interface for specifying priors and performing inference
result principled uncertainty quantification for state and parameters
We present a novel technique for assessing the dynamics of multiphase fluid flow in the oil reservoir. We demonstrate an efficient workflow for handling the 3D reservoir simulation data in a way which is orders of magnitude faster than the conventional routine. The workflow (we call it "Metamodel") is based on a projec…
In biological research machine learning algorithms are part of nearly every analytical process. They are used to identify new insights into biological phenomena, interpret data, provide molecular diagnosis for diseases and develop personalized medicine that will enable future treatments of diseases. In this paper we (1…
We provide an online RLHF workflow for large language models.
problem Lack of online RLHF workflow for large language models.
method Construct preference models using open-source datasets, approximate human feedback, and implement online iterative RLHF.
result Our LLM achieves state-of-the-art performance on various benchmarks.
Gemini uses inexpensive measurements to correct biases in expensive property evaluations.
problem Accurate estimation of materials properties using expensive measurements is hindered in scientific discovery campaigns.
method Gemini is a data-driven model that corrects systematic biases between property evaluation methods using inexpensive measurements.
result Gemini reduces the number of expensive evaluations needed for Bayesian optimization in materials discovery.
CK simplifies ML model deployment and reproducibility with open APIs and DevOps.
problem Making ML models reproducible and deployable across different environments.
method Decompose complex systems into reusable sub-components with unified APIs and DevOps principles.
result Automatically co-design and optimize ML models for speed, accuracy, energy, and size.
Recent applications of machine learning algorithms in the seismic domain have shown great potential in different areas such as seismic inversion and interpretation. However, such algorithms rarely enforce geophysical constraints - the lack of which might lead to undesirable results. To overcome this issue, we have deve…