Research
On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,657 papers · 148 categories

Trend · papers per month

103206309412 · Jun 202019922001200920172026
48 results for pipeline development

Paper proposes a statistical test for feature selection pipelines using selective inference.

problem Assessing the significance of feature selection pipelines in data analysis.
method Selective inference technique applied to feature selection pipelines composed of various algorithms.
result The proposed statistical test controls false positive feature selection probabilities.

Automates supervised learning pipeline design with matrix and tensor factorization.

problem Designing effective supervised learning pipelines with many choices.
method Uses matrix and tensor factorization to model pipeline search space and develops greedy experiment design protocols.
result Demonstrates the effectiveness of the approach on real-world classification problems.

This paper explores how LLMs can improve pipeline-based conversational agents.

problem Limitations of pipeline-based conversational agents in human-like conversations.
method Investigated LLMs' capabilities in two phases: design and development, and operations.
result LLMs can enhance pipeline-based agents in various tasks like data generation, intent classification, and auto-correction.

Pipelined Backpropagation trains large models without batches efficiently.

problem Training large models efficiently on hardware with limited batch sizes.
method Fine-grained Pipelined Backpropagation with Spike Compensation and Linear Weight Prediction.
result Fine-grained Pipelined Backpropagation with a batch size of one matches the accuracy of SGD for multiple networks.

RL pipeline simplifies knot diagrams, including very hard unknots.

problem Simplifying complex knot diagrams, especially very hard unknots.
method Reinforcement learning for move proposals and heuristic navigation of Reidemeister moves.
result Trained agent simplifies diagrams, including a 41#9104_1\#9_{10} link to a three-step unknotting process.

Autonomous driving requires 3D perception of vehicles and other objects in the in environment. Much of the current methods support 2D vehicle detection. This paper proposes a flexible pipeline to adopt any 2D detection network and fuse it with a 3D point cloud to generate 3D information with minimum changes of the 2D d…

2018-02-12abs ↗pdf ↗

The study of neurocognitive tasks requiring accurate localisation of activity often rely on functional Magnetic Resonance Imaging, a widely adopted technique that makes use of a pipeline of data processing modules, each involving a variety of parameters. These parameters are frequently set according to the local goal o…

2016-10-13abs ↗pdf ↗

Mathematical pipeline identifies structural homology of knotted proteins.

problem Quantification and classification of protein structures, especially knotted proteins, require noise-free and complete data.
method Developed a geometric framework using persistent homology to analyze protein structures.
result Persistent homology accurately represents structural homology of knotted proteins and identifies geometric features of protein entanglement.

Pipeline integrates cross-sectional and longitudinal multi-omics data for IBD research.

problem Integrating diverse data types from the same individuals for disease understanding.
method Statistical and deep learning methods for variable selection, feature extraction, and joint integration.
result Identified microbial pathways, metabolites, and genes discriminating IBD status.

The evaluation of machine learning (ML) pipelines is essential during automatic ML pipeline composition and optimisation. The previous methods such as Bayesian-based and genetic-based optimisation, which are implemented in Auto-Weka, Auto-sklearn and TPOT, evaluate pipelines by executing them. Therefore, the pipeline c…

2020-01-30abs ↗pdf ↗

In order to achieve state-of-the-art performance, modern machine learning techniques require careful data pre-processing and hyperparameter tuning. Moreover, given the ever increasing number of machine learning models being developed, model selection is becoming increasingly important. Automating the selection and tuni…

2017-05-15abs ↗pdf ↗

Hyperparameter tuning of multi-stage pipelines introduces a significant computational burden. Motivated by the observation that work can be reused across pipelines if the intermediate computations are the same, we propose a pipeline-aware approach to hyperparameter tuning. Our approach optimizes both the design and exe…

2019-03-12abs ↗pdf ↗

A new property fixes look-ahead bias in backtesting and trading pipelines.

problem Fixing look-ahead bias in backtesting and trading pipelines.
method Developed a pipeline calculus separating availability from reference time, and a type-and-effect system for the value-independent fragment.
result The check scales linearly and catches all leaks, including those missed by differential and tiling detectors.

Study proposes a statistical testing framework for evaluating clustering pipelines.

problem Quantifying the statistical reliability of clustering results from data analysis pipelines.
method Selective inference-based statistical testing framework for clustering pipelines.
result The proposed test controls the type I error rate and is effective in validating clustering results.

Investigates fairness in pipeline models where individuals may drop out.

problem Fairness in pipeline models where individuals may drop out and subsequent stages depend on remaining individuals.
method Rigorous framework for evaluating fairness guarantees, showing that naïve auditing is insufficient and dependence must exist between stages.
result Fairness in pipelines can be arbitrary, even with just two stages, and requires dependence between stages.

Develops a machine learning pipeline for learning causal structure in time-series data.

problem Current ML algorithms fail to learn causal structure in time-series data due to lack of temporal order consideration.
method Integrates machine learning with chaos theory using ChaosFEX feature extractor to learn generalized causal structure.
result Successfully learns generalized causal structure in time-series data.

Pipeline decomposes portfolio optimization problems into smaller, solvable subproblems.

problem Large-scale portfolio optimization with constraints.
method Decomposition pipeline with preprocessing, clustering, and risk rebalancing.
result Pipeline reduces problem size by 80% and computation time.

Data mining is about obtaining new knowledge from existing datasets. However, the data in the existing datasets can be scattered, noisy, and even incomplete. Although lots of effort is spent on developing or fine-tuning data mining models to make them more robust to the noise of the input data, their qualities still st…

2019-06-20abs ↗pdf ↗

Spatial and visual connectivity are important metrics when developing workplace layouts. Calculating those metrics in real-time can be difficult, depending on the size of the floor plan being analysed and the resolution of the analyses. This paper investigates the possibility of considerably speeding up the outcomes of…

2019-12-29abs ↗pdf ↗

Study reveals similarities in knowledge flows between pharmaceutical and AI industries.

problem Understanding the dynamics of drug pipelines in global pharmaceutical industry.
method Multilayer network analysis of drug pipeline, global supply chain, and ownership data.
result Proven similarities in knowledge flows between pharmaceutical and AI industries.

LLM generates coherent macroeconomic stress scenarios for portfolio risk assessment.

problem Macro-financial stress testing and portfolio risk assessment using traditional methods.
method Hybrid prompt-RAG pipeline combining structured prompting and retrieval of country fundamentals and news.
result LLM-generated scenarios yield stable tail-risk amplification with limited sensitivity to retrieval choices.

TODS automates time series outlier detection with customizable pipelines.

problem Automated detection of outliers in time series data.
method Modular system with 70 primitives for data processing, time series analysis, and detection algorithms. GUI and data-driven searcher for pipeline design.
result Automated discovery and construction of effective outlier detection pipelines.

A rigorous ML pipeline for binary classification in biomedical studies, focusing on pancreatic cancer.

problem Handling bias in ML models for complex biomedical data.
method Customizable ML analysis pipeline with 9 algorithms, hyperparameter optimization, and thorough evaluation.
result Comparison of ML algorithms to ExSTraCS, highlighting interpretability and bias handling.

A model-free framework extracts risk-neutral densities from short-dated options.

problem Arbitrage and bid-ask spread issues in short-dated options.
method Develops ARIES for filtering static arbitrage and SEDEx for density extraction.
result Robust density extraction across various market conditions and volatility smiles construction.

Pipeline parallelism (PP) when training neural networks enables larger models to be partitioned spatially, leading to both lower network communication and overall higher hardware utilization. Unfortunately, to preserve the statistical efficiency of sequential training, existing PP techniques sacrifice hardware efficien…

2019-10-09abs ↗pdf ↗