The increasing availability of large but noisy data sets with a large number of heterogeneous variables leads to the increasing interest in the automation of common tasks for data analysis. The most time-consuming part of this process is the Exploratory Data Analysis, crucial for better domain understanding, data clean…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “automated analysis”
RAG-IT automates financial analysis using LLMs and specialized datasets.
problem Manual financial analysis is time-consuming and requires expertise.
method Retrieval-Augmented Instruction Tuning (RAG-IT) fine-tunes an LLM for financial tasks.
result RAG-IT improves financial report generation performance compared to commercial systems.
Paper uses LLMs for sector allocation, showing better returns.
problem Automated trading sector allocation inefficiencies.
method Systematic analysis of macroeconomic data and sentiment.
result LLM-based sector allocation outperforms traditional strategies.
Automated Market Makers: Mean-Variance Analysis of LPs Payoffs and Design of Pricing Functionsq-fin.TR
This paper analyzes and compares different Automated Market Maker mechanisms.
problem Impermanent loss in Constant Function Market Makers.
method Mean-Variance analysis of liquidity providers' profit and loss, comparison of different mechanisms.
result Optimized oracle-based mechanisms outperform Constant Function Market Makers.
Evidence suggests Rapid-Eye-Movement (REM) Sleep Behaviour Disorder (RBD) is an early predictor of Parkinson's disease. This study proposes a fully-automated framework for RBD detection consisting of automated sleep staging followed by RBD identification. Analysis was assessed using a limited polysomnography montage fr…
Proposes PA-DSL for correcting noisy human labels in automated data labeling.
problem Noisy human labels in automated data labeling.
method Uses adjudicated cases to correct noisy human labels and debias analyses.
result Maintains nominal coverage and reduces RMSE by 10-17% relative to using only adjudicated labels.
Smart Close-out Nettingq-fin.RM
Smart Close-out Netting aims to automate close-out netting processes.
problem Inefficiencies in close-out netting processes for financial institutions.
method Standardisation and automation of legal and regulatory processes using a data-driven framework and controlled natural language.
result Standardisation and automation can improve close-out netting processes for prudentially regulated financial institutions.
Using Deep Neural Networks to Automate Large Scale Statistical Analysis for Big Data Applicationsstat.ML
Statistical analysis (SA) is a complex process to deduce population properties from analysis of data. It usually takes a well-trained analyst to successfully perform SA, and it becomes extremely challenging to apply SA to big data applications. We propose to use deep neural networks to automate the SA process. In parti…
The automation of posterior inference in Bayesian data analysis has enabled experts and nonexperts alike to use more sophisticated models, engage in faster exploratory modeling and analysis, and ensure experimental reproducibility. However, standard automated posterior inference algorithms are not tractable at the scal…
Multi-task learning (MTL) has recently contributed to learning better representations in service of various NLP tasks. MTL aims at improving the performance of a primary task, by jointly training on a secondary task. This paper introduces automated tasks, which exploit the sequential nature of the input data, as second…
Paper presents deep learning and ML for automated student performance estimation.
problem Evaluation of students' performance during the pandemic.
method In-depth analysis of deep learning and machine learning approaches.
result Better performance across different prediction tasks with fully data-driven approach.
Multi-Class Classification of Blood Cells -- End to End Computer Vision based diagnosis case studycs.CV
This study automates blood cell classification using computer vision.
problem Automated detection and classification of white blood cells.
method Color-based segmentation, morphological processing, feature extraction, PCA, Unsupervised and Supervised learning algorithms, Deep Convolutional Neural Networks.
result Identification of robust algorithms for automated blood cell classification.
Unified platform for statistical and machine learning in bioinformatics.
problem Workflow inefficiencies in using multiple tools for data analysis.
method Automated hyperparameter optimization, feature importance analysis, statistical tests.
result Accelerates biological discovery workflows with methodological soundness.
One of the longstanding problems in spectral graph clustering (SGC) is the so-called model order selection problem: automated selection of the correct number of clusters. This is equivalent to the problem of finding the number of connected components or communities in an undirected graph. In this paper, we propose AMOS…
This study analyzes financial equity research reports to identify frequently asked questions and automates 80% of them.
problem Insufficient empirical analysis of questions answered in financial equity research reports.
method Analyzed 72 financial equity research reports, classifying sentences into 169 unique question archetypes. Used public corporate reports to classify questions' potential for automation.
result Approximately 80% of financial equity research reports can be automated, with 78.7% of questions automatable.
Automated model tracks mouse behavior in home cages.
problem Confounding analysis in specialized arenas.
method Group Behaviour Model (GBM) using permutation matrix.
result Model captures individual and group behavior.
TODS automates time series outlier detection with customizable pipelines.
problem Automated detection of outliers in time series data.
method Modular system with 70 primitives for data processing, time series analysis, and detection algorithms. GUI and data-driven searcher for pipeline design.
result Automated discovery and construction of effective outlier detection pipelines.
Paper uses TDA for automated Parkinson's disease classification and severity assessment.
problem Manual diagnosis of neurological diseases is time-consuming and inaccurate.
method Combines Topological Data Analysis (TDA) with machine learning on postural shift data.
result Proposes a stable and accurate method for classifying Parkinson's disease.
Automated HPO design using Bayesian optimization and benchmarking.
problem Designing effective hyperparameter optimization algorithms is manual and lacks systematic understanding.
method Formalized space of HPO candidates, Bayesian optimization for search, ablation analysis.
result Simple configurations can perform well in HPO, especially with right parameters.
Study adversarial attacks on automated trading systems.
problem Robustness of deep learning models in algorithmic trading.
method New attacks with size constraints to evaluate model robustness.
result Realistic adversarial attacks can fool automated trading systems.
There has been considerable growth and interest in industrial applications of machine learning (ML) in recent years. ML engineers, as a consequence, are in high demand across the industry, yet improving the efficiency of ML engineers remains a fundamental challenge. Automated machine learning (AutoML) has emerged as a …
This paper presents an approach for automation of interpretable feature selection for Internet Of Things Analytics (IoTA) using machine learning (ML) techniques. Authors have conducted a survey over different people involved in different IoTA based application development tasks. The survey reveals that feature selectio…
PyODDS automates outlier detection for new data sources.
problem Manual outlier detection is inefficient and domain-specific.
method Automated end-to-end system with Apache Spark and database support.
result PyODDS optimizes outlier detection pipelines automatically.
Nuclear magnetic resonance (NMR) spectroscopy is one of the leading techniques for protein studies. The method features a number of properties, allowing to explain macromolecular interactions mechanistically and resolve structures with atomic resolution. However, due to laborious data analysis, a full potential of NMR …
Study evaluates five LLMs for financial report analysis, revealing performance differences and variability.
problem Lack of understanding in reliability, consistency, and transparency of LLMs in financial analysis.
method Human evaluation, automated similarity metrics, and behavioral diagnostics applied to five transformer-based LLMs over U.S. 10-K filings.
result No single LLM consistently dominates across all evaluation perspectives, highlighting variability and need for interpretability.
The ability to identify sentiment in text, referred to as sentiment analysis, is one which is natural to adult humans. This task is, however, not one which a computer can perform by default. Identifying sentiments in an automated, algorithmic manner will be a useful capability for business and research in their search …
Real time large scale streaming data pose major challenges to forecasting, in particular defying the presence of human experts to perform the corresponding analysis. We present here a class of models and methods used to develop an automated, scalable and versatile system for large scale forecasting oriented towards saf…
Agricultural research has been profited by technical advances such as automation, data mining. Today, data mining is used in a vast areas and many off-the-shelf data mining system products and domain specific data mining application soft wares are available, but data mining in agricultural soil datasets is a relatively…
KM-GPT: An Automated Pipeline for Reconstructing Individual Patient Data from Kaplan-Meier Plotscs.LG
KM-GPT automates IPD reconstruction from KM plots with high accuracy and scalability.
problem Manual digitization of IPD from KM plots is error-prone and lacks scalability.
method KM-GPT integrates advanced image preprocessing, multi-modal reasoning, and iterative reconstruction algorithms.
result KM-GPT generates high-quality IPD without manual input or intervention, achieving superior accuracy.
AutoBayes automates Bayesian graph exploration for robust machine learning.
problem Learning representations invariant to nuisance variations in machine learning.
method Automated Bayesian inference framework exploring different graphical models.
result Significant performance improvement with nuisance-invariant machine learning pipelines.
The growing interest in both the automation of machine learning and deep learning has inevitably led to the development of a wide variety of automated methods for neural architecture search. The choice of the network architecture has proven to be critical, and many advances in deep learning spring from its immediate im…
One of the longstanding open problems in spectral graph clustering (SGC) is the so-called model order selection problem: automated selection of the correct number of clusters. This is equivalent to the problem of finding the number of connected components or communities in an undirected graph. We propose automated mode…
The segmentation of the left ventricle (LV) from CINE MRI images is essential to infer important clinical parameters. Typically, machine learning algorithms for automated LV segmentation use annotated contours from only two cardiac phases, diastole, and systole. In this work, we present an analysis work-flow for fully-…
Untargeted Region of Interest Selection for GC-MS Data using a Pseudo F-Ratio Moving Window (FRMV)stat.ML
A new method selects regions of interest in GC-MS data without prior target selection.
problem Challenges in GC-MS data analysis due to fragmentation and shared fragment ions.
method Uses a pseudo F-ratio moving window (FRMV) to automatically select regions of interest.
result Algorithm can accurately identify signal regions in GC-MS data.
The paper examines statistical properties of IL and LVR in automated market makers.
problem Assessing the performance of automated market makers and their profitability.
method Analysis of random walk properties and statistical integral combined with CFMM mechanics.
result IL and LVR have identical expectation values but different distribution functions for Brownian motion.
Deep learning automates biofouling detection in ship hull images.
problem Automating biofouling assessment from ship hull images.
method Deep learning model trained on expert-annotated images.
result Deep learning model agrees with expert annotations.
Human stablecoin transactions predict political risk in cryptocurrency markets.
problem Predicting political risk in cryptocurrency markets.
method Structural break analysis and surrogate-based robustness tests.
result Human-driven stablecoin transactions shift significantly before major political events.
While variable selection is essential to optimize the learning complexity by prioritizing features, automating the selection process is preferred since it requires laborious efforts with intensive analysis otherwise. However, it is not an easy task to enable the automation due to several reasons. First, selection techn…
Deciding effective and timely preventive measures against complex social problems affecting relatively low income geographies is a difficult challenge. There is a strong need to adopt intelligent automation based solutions with low cost imprints to tackle these problems at larger scales. Starting with the hypothesis th…
LISBET automates social behavior analysis using machine learning.
problem Manual annotation of social behaviors is time-consuming, biased, and misses subtle interactions.
method Self-supervised learning on body tracking data.
result Automated detection and segmentation of social interactions.
Automated theorem prover proves non-orderability of groups.
problem Proving non-orderability of groups.
method Generic automated theorem prover with tools like positive cones, torsions, generalised torsions, and cofinal elements.
result Demonstrated automated proof of non-orderability of groups.
Coronary Artery Segmentation from Intravascular Optical Coherence Tomography Using Deep Capsulescs.LG
DeepCap automates coronary artery segmentation from IVOCT images.
problem Automated segmentation of coronary arteries from IVOCT images is challenging.
method Developed a deep learning method based on capsules for robust, unbiased segmentation.
result DeepCap achieves segmentation quality comparable to state-of-the-art methods.
Malaria is a life-threatening disease affecting millions. Microscopy-based assessment of thin blood films is a standard method to (i) determine malaria species and (ii) quantitate high-parasitemia infections. Full automation of malaria microscopy by machine learning (ML) is a challenging task because field-prepared sli…
Cilia are hairlike structures protruding from nearly every cell in the body. Diseases known as ciliopathies, where cilia function is disrupted, can result in a wide spectrum of disorders. However, most techniques for assessing ciliary motion rely on manual identification and tracking of cilia; this process is laborious…
In this paper, we present a data science automation system called Prediction Factory. The system uses several key automation algorithms to enable data scientists to rapidly develop predictive models and share them with domain experts. To assess the system's impact, we implemented 3 different interfaces for creating pre…
Survival rates for colorectal cancer are higher when polyps are detected at an early stage and can be removed before they develop into malignant tumors. Automated polyp detection, which is dominated by deep learning based methods, seeks to improve early detection of polyps. However, current efforts rely heavily on the …
Study examines AutoML adaptation to evolving data.
problem Understanding and improving AutoML performance with concept drift.
method 6 concept drift adaptation strategies evaluated on various AutoML approaches.
result Robust AutoML techniques can be developed to handle concept drift.
The proliferation of automated inference algorithms in Bayesian statistics has provided practitioners newfound access to fast, reproducible data analysis and powerful statistical models. Designing automated methods that are also both computationally scalable and theoretically sound, however, remains a significant chall…