Knockoffs method selects financial factors, controlling false discoveries.
problem Controlling false discoveries in financial factor selection.
method Apply knockoff procedure to build fake factors.
result Shows versatility in fund replication and network inference.
NCFA uses deep learning and causal discovery to analyze complex data.
problem Analyzing complex, interdependent data with causal relationships.
method NCFA combines latent causal discovery and variational autoencoders.
result NCFA outperforms standard VAEs in sparsity, complexity, and causal interpretability.
LLM agents discover cryptocurrency factors under reproducible constraints.
problem Flexibility of LLM agents in empirical discovery leads to uncontrolled search.
method Sequential hypothesis search with fixed data splits and portfolio tests.
result Ridge-combined portfolio achieves 44.55% annualized return in out-of-sample period.
Develops robust knockoffs for controlling false discoveries in financial data.
problem Challenges in variable selection with highly correlated data in finance and economics.
method Robustified knockoff framework addressing high dependence and time correlation.
result Identifies new important groups of factors on top of known drivers.
Proposes MD-LiNA for multi-domain latent factor causal discovery.
problem Discovering causal structures among latent factors from multi-domain data.
method Multi-Domain Linear Non-Gaussian Acyclic Models (MD-LiNA) with an integrated two-phase algorithm.
result Locally consistent estimators of causal structure among shared latent factors.
AlphaLogics mines market logic to generate interpretable alpha factors.
problem Complex, opaque alpha factors from factor mining overlook market logic.
method Market Logic Mining, Factor Generation and Optimization, Market Logic Generation and Optimization.
result AlphaLogics improves predictive metrics and risk-adjusted returns over baselines.
New method controls false discoveries in financial asset pricing.
problem Controlling false discoveries in time series with unknown correlations.
method Double bootstrapping method to control false discovery rate.
result Superior statistical power and controlled false discovery rate.
TF-MoDISco (Transcription Factor Motif Discovery from Importance Scores) is an algorithm for identifying motifs from basepair-level importance scores computed on genomic sequence data. This technical note focuses on version v0.5.6.5. The implementation is available at https://github.com/kundajelab/tfmodisco/tree/v0.5.6…
Study compares CDS databases and finds discrepancies due to various factors.
problem Comparing discrepancies among CDS databases.
method Comparing five major sources of corporate CDS prices over 2004-2010.
result CMA quotes lead price discovery and databases disagree on stock-CDS return analysis.
Expands experimental design for causal discovery from limited data.
problem Challenges in causal discovery from observational and interventional data.
method Bayesian optimal experimental design incorporating recent advances in causal discovery.
result Active causal discovery of large, nonlinear SCMs with both intervention target and value selection.
New method discovers causal relationships in large-scale data.
problem Discovering causal relationships in large datasets with thousands of variables.
method Factor Directed Acyclic Graphs (f-DAGs) combined with continuous optimization.
result Achieved causal discovery on thousands of variables.
AlphaCFG discovers alpha factors using grammar-guided search.
problem Discovering formulaic alpha factors in finance.
method AlphaCFG uses a grammar-based framework to define and discover alpha factors with syntactic and semantic constraints.
result AlphaCFG outperforms state-of-the-art methods in trading profitability and efficiency.
Bayesian method optimizes interventions for causal discovery.
problem Active interventions are needed for causal discovery when observational data is insufficient.
method Bayesian optimization-based approach using observational data and pre-experimental evaluation of interventions.
result Demonstrated effectiveness through various experiments.
Scientific discovery is limited by hypothesis redundancy, and hybrid methods can exploit non-local exploration.
problem Limitation of scientific discovery due to hypothesis redundancy.
method Hybrid discovery systems combining structured local search with LLM-generated non-local proposals.
result Hybrid methods can exploit non-local exploration when three geometric conditions co-occur.
Boolean matrix factorization (BMF) is a popular and powerful technique for inferring knowledge from data. The mining result is the Boolean product of two matrices, approximating the input dataset. The Boolean product is a disjunction of rank-1 binary matrices, each describing a feature-relation, called pattern, for a g…
Gradient-based methods can be biased by distributional asymmetries in bivariate categorical data.
problem Gradient-based causal discovery methods can be biased by distributional asymmetries in bivariate categorical data.
method Identified and examined two distributional biases: Marginal Distribution Asymmetry and Marginal Distribution Shift Asymmetry. Employed two simple models to demonstrate and control these biases.
result Gradient-based methods can be biased by distributional asymmetries, and these biases can be controlled.
Identifying important components or factors in large amounts of noisy data is a key problem in machine learning and data mining. Motivated by a pattern decomposition problem in materials discovery, aimed at discovering new materials for renewable energy, e.g. for fuel and solar cells, we introduce CombiFD, a framework …
The discovery of time series motifs has emerged as one of the most useful primitives in time series data mining. Researchers have shown its utility for exploratory data mining, summarization, visualization, segmentation, classification, clustering, and rule discovery. Although there has been more than a decade of exten…
CausalEvolve improves efficiency and discovery in open-ended scientific tasks.
problem Lack of targeted guidance and knowledge organization in evolve-based agents.
method Causal scratchpad that identifies and reasons about guiding factors for evolution.
result Effective improvement in evolutionary efficiency and better solutions.
A model learns causal representations from high-dimensional data.
problem Challenges in learning causal representations from high-dimensional data.
method Formulated a latent variable decoder model, Decoder BCD, for Bayesian causal discovery.
result Shows that using known intervention targets as labels helps in unsupervised Bayesian inference over structure and parameters.
FactorMiner discovers financial alpha factors with low redundancy.
problem Finding novel financial alpha factors in a vast search space.
method Modular Skill Architecture and Experience Memory to distill and guide exploration.
result FactorMiner constructs a diverse library of high-quality factors with competitive performance.
A new method for efficient causal structure learning at scale.
problem Causal structure learning is computationally challenging at scale.
method Relaxed sparsest-permutation formulation with support-level relaxation and masked zero-fill incomplete Cholesky factorization.
result The method enables scalable comparison of candidate orderings and matches the accuracy of slower baselines.
Interpretable framework evaluates structure learning methods for causal discovery from observational data.
problem Evaluation of structure learning methods under assumption violations in causal discovery.
method Six-dimensional evaluation metric (DOS) tailored for causal discovery.
result Amortized causal discovery delivers results with high proximity to the optimal solution.
New method for identifying causal relationships in financial time series data.
problem Identifying causal relationships in nonstationary financial time series data.
method Refined constraint-based causal discovery algorithm (CD-NOTS) for nonstationary time series data.
result CD-NOTS effectively identifies causal connections in financial applications.
Efficient knockoffs for large-scale feature selection.
problem Large-scale feature selection problems.
method Gaussian model-X knockoffs with efficient methods for solving semidefinite programs.
result Efficient knockoffs can be generated with linear complexity in the dimension.
We present a novel factor analysis method that can be applied to the discovery of common factors shared among trajectories in multivariate time series data. These factors satisfy a precedence-ordering property: certain factors are recruited only after some other factors are activated. Precedence-ordering arise in appli…
New algorithm discovers causal graphs efficiently from observational data.
problem Discovering causal graphs from observational data efficiently.
method Approximating the score function using machine learning and applying scalable techniques.
result DAS algorithm reduces complexity and achieves competitive accuracy.
High-performing equity factor with Sharpe ratio above 13 out-of-sample.
problem Hidden cross-sectional predictability in stock returns.
method Regime-conditional signal activation combining value and short-term reversal signals.
result Annualized returns of 158.6% with 12.0% volatility, strong performance out-of-sample.
Enhances genetic programming for stock alpha discovery with warm start and structural constraints.
problem Overwhelming search space and computational burden in traditional genetic programming for alpha factor discovery.
method Proposes a new GP framework with warm start and structural constraints to enhance search performance and interpretability.
result Superior out-of-sample prediction results and higher portfolio returns compared to benchmarks.
New method infers causal factors from large-scale data without full graph reconstruction.
problem Inferring causal variables from large-scale systems without full causal graph reconstruction.
method Supervised learning on simulated data using a neural network and subsampled-ensemble inference.
result Efficiently identifies causal relationships in large-scale gene regulatory networks.
New methods identify concepts in trained embeddings reliably without human labels.
problem Identifying interpretable concepts in trained embedding spaces without human labels.
method Explicitly connecting concept discovery to PCA and ICA, proposing novel approaches for dependent concepts.
result Proven methods outperform competitors on a variety of experiments, achieving up to 29% better alignment with ground truth.
The false discovery rate (FDR)---the expected fraction of spurious discoveries among all the discoveries---provides a popular statistical assessment of the reproducibility of scientific studies in various disciplines. In this work, we introduce a new method for controlling the FDR in meta-analysis of many decentralized…
Information theoretic measures (e.g. the Kullback Liebler divergence and Shannon mutual information) have been used for exploring possibly nonlinear multivariate dependencies in high dimension. If these dependencies are assumed to follow a Markov factor graph model, this exploration process is called structure discover…
FoundCause: Causal Discovery with Latent Confounders from Observational Data
problem Causal discovery from observational data
method FoundCause, an amortized causal discovery model trained on synthetic data
result FoundCause outperforms classical and amortized methods on real-world datasets
Deep neural networks decompose SDF into linear and nonlinear components.
problem Constructing accurate stochastic discount factors (SDFs) for pricing.
method Additive decomposition of a deep neural network trained to construct SDFs.
result The PTK representation delivers significant performance gains in equity data.
New Bayesian method for sparse multidimensional item response theory.
problem Sparse interpretable explanations for questionnaire data.
method Bayesian EM algorithm for sparse factor loadings.
result Reliable recovery of factor dimensionality and latent structure.
CAnDOIT discovers causal relationships using both observational and interventional time-series data.
problem Identifying causal relationships in the presence of hidden factors.
method CAnDOIT combines observational and interventional time-series data to reconstruct causal models.
result CAnDOIT effectively handles interventional data and enhances the accuracy of causal analysis.
Topic discovery has witnessed a significant growth as a field of data mining at large. In particular, time-evolving topic discovery, where the evolution of a topic is taken into account has been instrumental in understanding the historical context of an emerging topic in a dynamic corpus. Traditionally, time-evolving t…
New algorithm improves causal discovery in biomedical data.
problem Stability and accuracy issues in causal discovery algorithms.
method Exploits temporal structure and tiered background knowledge.
result Increases accuracy in finite samples for causal structure estimation.
Due to recent explosion of text data, researchers have been overwhelmed by ever-increasing volume of articles produced by different research communities. Various scholarly search websites, citation recommendation engines, and research databases have been created to simplify the text search tasks. However, it is still d…
A new geometrically-motivated algorithm for nonnegative matrix factorization is developed and applied to the discovery of latent "topics" for text and image "document" corpora. The algorithm is based on robustly finding and clustering extreme points of empirical cross-document word-frequencies that correspond to novel …
LANCA uses ANM to learn latent causal factors without supervision.
problem Learning latent causal factors without supervision.
method LANCA employs a deterministic Wasserstein Auto-Encoder coupled with a differentiable ANM Layer.
result LANCA outperforms baselines on physics and photorealistic environments.
Non-negative matrix factorization (NMF) is a new knowledge discovery method that is used for text mining, signal processing, bioinformatics, and consumer analysis. However, its basic property as a learning machine is not yet clarified, as it is not a regular statistical model, resulting that theoretical optimization me…
Genetic programming (GP) is the state-of-the-art in financial automated feature construction task. It employs reverse polish expression to represent features and then conducts the evolution process. However, with the development of deep learning, more powerful feature extraction tools are available. This paper proposes…
Bayesian network structure learning algorithms with limited data are being used in domains such as systems biology and neuroscience to gain insight into the underlying processes that produce observed data. Learning reliable networks from limited data is difficult, therefore transfer learning can improve the robustness …
Diamond method controls FDR for trustworthy feature interaction discovery in ML models.
problem Limited interpretability of ML models due to black box nature.
method Diamond method integrates model-X knockoffs framework to control FDR for non-additive interactions.
result Diamond method ensures accurate discovery of feature interactions with FDR control.
Supervised machine learning models often associate irrelevant nuisance factors with the prediction target, which hurts generalization. We propose a framework for training robust neural networks that induces invariance to nuisances through learning to discover and separate predictive and nuisance factors of data. We pre…
We propose Deep Feature Factorization (DFF), a method capable of localizing similar semantic concepts within an image or a set of images. We use DFF to gain insight into a deep convolutional neural network's learned features, where we detect hierarchical cluster structures in feature space. This is visualized as heat m…