Study uses LLMs to automate data insights discovery.
problem Extracting relevant insights from large data sets.
method Capture the Flag principle, LLMs, reasoning, code generation.
result LLMs can recognize meaningful data insights.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “Data Insights”
This work develops a high precision fault diagnosis classifier using XAI insights.
problem Fault diagnosis of steel plates in manufacturing industry.
method Incorporation of XAI insights into Data Science process using SMOTE and medoids.
result 10 fold cross validated performance of 94% achieved.
Tab-Shapley identifies top-k anomalies in tabular data quality insights.
problem Challenges in identifying anomalies in unlabeled tabular datasets.
method Cooperative game theory using Shapley values to quantify attribute contributions.
result Efficiently identifies top-k tabular data quality insights using closed-form Shapley values.
New insights into how data transformations affect self-supervised clustering.
problem Impact of data transformations on self-supervised clustering convergence.
method Theoretical and empirical analysis of various data transformations.
result Certain transformations help in faster convergence of self-supervised clustering.
The support vector machine (SVM) is a powerful and widely used classification algorithm. This paper uses the Karush-Kuhn-Tucker conditions to provide rigorous mathematical proof for new insights into the behavior of SVM. These insights provide perhaps unexpected relationships between SVM and two other linear classifier…
Business Architecture (BA) plays a significant role in helping organizations understand enterprise structures and processes, and align them with strategic objectives. However, traditional BAs are represented in fixed structure with static model elements and fail to dynamically capture business insights based on interna…
ML methods improve planetary science data analysis.
problem Insufficient use of ML in planetary science.
method Ten recommendations for integrating ML in planetary science.
result Expanding planetary science insights from large datasets.
Conversion of raw data into insights and knowledge requires substantial amounts of effort from data scientists. Despite breathtaking advances in Machine Learning (ML) and Artificial Intelligence (AI), data scientists still spend the majority of their effort in understanding and then preparing the raw data for ML/AI. Th…
Process mining is a research field focused on the analysis of event data with the aim of extracting insights in processes. Applying process mining techniques on data from smart home environments has the potential to provide valuable insights in (un)healthy habits and to contribute to ambient assisted living solutions. …
New algorithms extract low-dimensional representations from sequential data, revealing insights into complex processes.
problem Challenges in extracting low-dimensional representations from sequential, high-dimensional, sparse, and noisy data.
method Developed new clustering algorithms based on Block Markov Chains theory, validated on real-world data.
result These algorithms can successfully extract low-dimensional representations from real-world sequential data, revealing insights into complex processes.
Big data transforms accounting and auditing, enhancing insights but posing challenges.
problem Challenges in data privacy and security with increased data sources.
method Utilizing AI and machine learning for efficient data analysis and anomaly detection.
result Enhanced analytics tools and continuous learning are key to overcoming challenges.
Study reveals hidden infections and infection dynamics from early data.
problem Understanding early infection dynamics and hidden infections in COVID-19.
method Data-driven machine learning analysis focusing on infection counts over time.
result Significant asymptomatic infections, 10-day lag, and strong infectious force.
Study integrates climate and text data to improve credit default prediction.
problem Improving credit risk assessment for mSEs with limited financial histories.
method Multimodal framework using LSTM, GRU, and transformer models.
result Integration of multiple data modalities improves credit default prediction.
DAGnosis uses DAGs to identify and localize data inconsistencies.
problem Handling data inconsistencies in machine learning models at deployment time.
method Directed acyclic graphs (DAGs) to encode feature probability distribution and independencies.
result Localization of inconsistencies and insights into their causes.
This method infers models from data with physical insights, minimizing model order.
problem Learning models from data while preserving physical insights.
method Structure preservation and rank minimization via Sylvester equations.
result Models of low order are obtained with fewer degrees of freedom.
Formalizes concepts as latent variables in hierarchical models for high-dimensional data.
problem Lack of formalization and theoretical insights for learning discrete concepts from high-dimensional data.
method Formalizes concepts as latent causal variables in a hierarchical model, formulates conditions for concept identification.
result Conditions for identifying latent hierarchical models in unsupervised data, handling complex structures and high-dimensional data.
Calibrates carbon futures option pricing using high-frequency data.
problem Estimating equity and variance risk premia for carbon futures options.
method Multifactor stochastic volatility framework with jumps, employing indirect inference.
result Provides insights into carbon futures and option dynamics.
DEDACT breaks down feature importance into direct and associative components.
problem Lack of clear distinction between direct and associative feature importance.
method DEDACT framework to decompose direct and associative importance measures.
result Provides insight into sources of prediction-relevant information and feature pathways.
Visualization of Board of Director Connections for Analysis in Socially Responsible Investingq-fin.GN
Visualizes board connections for socially responsible investing insights.
problem Understanding corporate governance and sustainability through board connections.
method Data Visualization tool to reveal connections between Directors and Executives.
result Strength of tool in investigating corporate governance and sustainability.
FinRobot AI agent for equity research provides comprehensive insights.
problem Narrow focus and limited discretion in AI solutions for equity research.
method Multi-agent Chain of Thought system integrating quantitative and qualitative analyses.
result FinRobot delivers insights comparable to major brokerage firms.
GAMLA learns manifold structures with auto-encoding for global insights.
problem Limited global insight and lack of interpretable analytical descriptions in manifold learning.
method Two-round auto-encoding process to derive character and complementary representations.
result GAMLA provides global and analytical descriptions of smooth manifolds.
Study reveals Data Shapley's inconsistent performance in data selection tasks.
problem Inconsistency of Data Shapley's performance in data selection across different settings.
method Hypothesis testing framework and identification of utility functions.
result Data Shapley's performance is no better than random selection without specific constraints.
Paper tackles variable-length, incomplete wearable sensor data to improve personalized insights.
problem Variable-length and incomplete time series data from wearable sensors.
method HeartSpace integrates a time series encoding module and pattern aggregation network, along with a Siamese-triplet network for representation learning.
result Empirical evaluation shows significant performance gains in personality prediction, demographics inference, and user identification.
Investor Behavior Modeling by Analyzing Financial Advisor Notes: A Machine Learning Perspectiveq-fin.ST
Modeling investor behavior from financial advisor notes using NLP.
problem Identifying behavioral coaching opportunities for financial advisors.
method Topic modeling and supervised classification model.
result Predicting investor needs during adverse market conditions.
In many real-world AD applications including computer security and fraud prevention, the anomaly detector must be configurable by the human analyst to minimize the effort on false positives. One important way to configure the detector is by providing true labels (nominal or anomaly) for a few instances. Recent work on …
This study benchmarks transcriptomics models for perturbation analysis, finding scVI and PCA superior.
problem Limited evaluation of transcriptomics foundation models for perturbation analysis.
method Developed a novel evaluation framework using diverse public datasets from different sequencing techniques and cell lines.
result scVI and PCA identified as superior models for understanding biological perturbations.
In this paper we investigate the geometry of the likelihood of the unknown parameters in a simple class of Bayesian directed graphs with hidden variables. This enables us, before any numerical algorithms are employed, to obtain certain insights in the nature of the unidentifiability inherent in such models, the way pos…
From Reviews to Actionable Insights: An LLM-Based Approach for Attribute and Feature Extractionstat.ML
LLM extracts actionable insights from customer reviews.
problem Extracting actionable insights from customer reviews.
method Large language model approach distinguishing perceptual attributes from actionable features.
result High consistency and predictive validity of LLM insights compared to human coders.
This paper introduces DCE for better counterfactual explanations using optimal transport.
problem Lack of nuanced distributional characteristics in existing counterfactual explanations.
method Formulates a chance-constrained optimization problem using optimal transport to derive counterfactual distributions.
result DCE provides deeper insights into decision-making models by aligning counterfactual distributions with factual ones.
Model Steering: Learning with a Reference Model Improves Generalization Bounds and Scaling Lawscs.LG
This paper improves model training by using a reference model to guide target model training.
problem Improving generalization and data efficiency in model training.
method DRRho risk minimization framework based on Distributionally Robust Optimization (DRO).
result DRRho risk minimization improves generalization and data efficiency compared to training without a reference model.
Causal Graph Aided Causal Discovery in an Observational Aneurysmal Subarachnoid Hemorrhage Studystat.ME
Causal graph aids observational study insights in aSAH patients.
problem Lack of clear objectives and tools for identifying necessary adjustments in observational studies.
method Uses causal directed acyclic graphs (DAGs) to provide insights mid-study and identify necessary data enhancements.
result Midway insights and necessary data enhancements identified for meaningful causal questions.
Modelling High-Frequency Data with Bivariate Hawkes Processes: Power-Law vs. Exponential Kernelsq-fin.MF
Study compares exponential and power-law kernels in modeling high-frequency trading data.
problem Modeling high-frequency trading data with specific kernel types.
method Proposes and analyzes two bivariate Hawkes processes with exponential and power-law kernels.
result Identifies strengths and limitations of exponential and power-law kernels for high-frequency trading data.
Limit Order Booksq-fin.TR
Limit order books (LOBs) match buyers and sellers in more than half of the world's financial markets. This survey highlights the insights that have emerged from the wealth of empirical and theoretical studies of LOBs. We examine the findings reported by statistical analyses of historical LOB data and discuss how severa…
Insights in power grid pixel maps (PGPMs) refer to important facility operating states and unexpected changes in the power grid. Identifying insights helps analysts understand the collaboration of various parts of the grid so that preventive and correct operations can be taken to avoid potential accidents. Existing sol…
Study uses LLMs to improve financial forecasting by integrating textual and numerical data.
problem Challenges in fusing multimodal information and measuring qualitative outputs from LLMs.
method Created context sets by segmenting daily securities reports into key factors and combining them with numerical data. Used dynamic updates and crafted prompts to assign scores to qualitative insights.
result LLMs outperform time-series models in market forecasting, though challenges remain.
Study identifies key ESG variables for assessing financial risk.
problem Assessing financial risk from ESG data with many variables.
method Proposed framework for hierarchical ESG data, selecting relevant variables.
result Selected ESG variables are more relevant to financial risk than aggregated scores.
Many natural systems, such as neurons firing in the brain or basketball teams traversing a court, give rise to time series data with complex, nonlinear dynamics. We can gain insight into these systems by decomposing the data into segments that are each explained by simpler dynamic units. Building on switching linear dy…
Optimizing data movement significantly improves transformer training efficiency.
problem Data movement is a major bottleneck in training transformers.
method Developed a recipe to globally optimize data movement in transformers.
result Achieved up to 1.30x performance improvement over state-of-the-art frameworks.
Efficient Decomposition of Forman-Ricci Curvature on Vietoris-Rips Complexes and Data Applicationsmath.GT
Researchers decompose Forman-Ricci curvature for efficient computation in VR complexes.
problem Efficiently computing Forman-Ricci curvature in higher-dimensional data.
method Decomposition and set-theoretical proof for local computation of FRC in VR complexes.
result Reveals critical geometric insights overlooked by conventional techniques.
The dearth of prescribing guidelines for physicians is one key driver of the current opioid epidemic in the United States. In this work, we analyze medical and pharmaceutical claims data to draw insights on characteristics of patients who are more prone to adverse outcomes after an initial synthetic opioid prescription…
Signatures simplify analysis of evolving data streams.
problem Understanding and analyzing irregular, non-stationary data streams.
method Mathematical signatures reduce noise and preserve key information.
result Signatures manage the exponential scaling of data complexity.
Fast, fully-automated histograms for large data sets.
problem Efficiently constructing histograms for large-scale data.
method G-Enum histograms using Minimum Description Length (MDL) principle for model selection and a greedy search heuristic.
result Construct histograms in linearithmic time, significantly faster than previous methods.
The study examines denoising and noisy-input regression under distribution shift, revealing double descent behavior and insights for data augmentation.
problem Understanding denoising in machine learning, especially under noisy inputs and distribution shift.
method Theoretical analysis of supervised denoising and noisy-input regression, considering low-rank data and proportional regime.
result The test error exhibits double descent under general distribution shift, indicating that overfitting the noise can be benign, tempered, or catastrophic.
New ACV method speeds up CV in high dimensions with approximate low-rank data.
problem Accurate model assessment in high-dimensional, large data settings with expensive algorithms.
method Developed a new ACV algorithm that uses low-rank approximations of the Hessian matrix.
result The new method is fast and accurate in the presence of approximate low-rank data.
Cohort analysis speeds up Bitcoin blockchain data queries.
problem Efficiently querying Bitcoin blockchain data for economic insights.
method Cohort analysis applied to Bitcoin transaction data.
result Creation of datasets and visualizations for key Bitcoin transaction indicators.
Article addresses practical challenges in conformal prediction.
problem Challenges in determining, computing, and controlling conformal prediction regions.
method Proposes a quadratic-polynomial non-conformity measure.
result Allows circumventing three challenges in full conformal prediction framework.
The paper analyzes sports commentary to automatically recognize events and extract insights.
problem Automatically recognizing and categorizing major actions in sports events from commentary.
method Used multiple Natural Language Processing techniques for classification and sentiment analysis.
result Identified insights from analyzing live sport commentaries and classifying major actions.
This paper investigates how synthetic data and overparameterization improve VAE generalization.
problem Improving generalization performance of Variational Autoencoders (VAEs).
method Investigates the effectiveness of synthetic data and overparameterization in VAEs.
result Training on synthetic data and using more parameters improves VAE generalization, inference, and robustness.