New framework uses background knowledge to speed up causal discovery.
problem Scalable causal discovery for large datasets.
method Utilizes background knowledge during causal discovery process.
result Background knowledge reduces computational requirements and improves structure quality.
Cluster-DAGs improve causal discovery with prior knowledge.
problem Finding cause-effect relationships from high-dimensional data.
method Cluster-DAGs as prior knowledge framework, modified constraint-based algorithms Cluster-PC and Cluster-FCI.
result Cluster-PC and Cluster-FCI outperform baselines without prior knowledge.
This work integrates domain knowledge into A*-based causal discovery methods.
problem Efficiently incorporating domain knowledge into A*-based causal discovery methods.
method Integrates various types of domain knowledge into A*-based causal discovery methods, reducing the graph search space and improving computational gains.
result Small amounts of domain knowledge can dramatically speed up A*-based causal discovery and improve its performance and practicality.
KEEL improves causal discovery with fuzzy knowledge and complex data.
problem Challenges in causal discovery due to prior knowledge, domain inconsistencies, and small sample sizes.
method Weakly-supervised fuzzy knowledge and data co-driven causal discovery method (KEEL).
result KEEL outperforms state-of-the-art methods in accuracy, robustness, and computational efficiency.
b-LOAD extends local causal discovery with prior knowledge, improving causal effect estimation.
problem Local causal discovery struggles in data-scarce settings due to uncertainty and incomplete neighborhoods.
method b-LOAD incorporates prior knowledge directly into local structure learning, using Meek's rules to refine discovery.
result b-LOAD refines the admissible equivalence class and enlarges identifiable causal queries, improving causal effect estimation.
Unsupervised relation discovery aims to discover new relations from a given text corpus without annotated data. However, it does not consider existing human annotated knowledge bases even when they are relevant to the relations to be discovered. In this paper, we study the problem of how to use out-of-relation knowledg…
This survey analyzes knowledge discovery in cryptocurrency transactions.
problem Understanding user behaviors and collective actions in cryptocurrency transactions.
method Data mining techniques and literature review.
result Classified existing research into three aspects and discussed major findings.
Paper proposes methods to discover causal models with unobserved variables.
problem Discovering causal relationships in data with unobserved variables.
method Two methods leveraging prior knowledge for causal discovery in CAM-UV models.
result Accuracy of causal discovery improves with more prior knowledge.
Transformer-based method for causal discovery with prior knowledge integration.
problem Complex nonlinear dependencies and spurious correlations in time series data.
method Multi-layer Transformer forecaster with gradient-based causal structure extraction and attention masking for prior knowledge integration.
result Significant improvement in causal discovery and causal lag estimation compared to state-of-the-art methods.
Paper presents new algorithms for causal discovery with latent variables and overlapping datasets.
problem Causal discovery with latent variables and overlapping datasets.
method Introduces tiered FCI and tIOD algorithms for constraint-based causal discovery.
result The tIOD algorithm is more efficient and informative than the IOD algorithm.
Q-SAVI model improves drug discovery accuracy with prior knowledge of chemical space.
problem Challenges in drug discovery due to covariate shift and limited labeled data.
method Probabilistic model with domain-informed prior distributions over functions.
result Q-SAVI outperforms state-of-the-art techniques in predictive accuracy and calibration.
NLP techniques improve drug discovery by analyzing chemical and protein text.
problem Improving drug discovery through better analysis of chemical and protein text.
method Natural language processing techniques applied to biochemical entities.
result Enhanced prediction of molecular properties and design of novel molecules.
Improved CI test for heteroskedastic data enhances causal discovery.
problem CI testing assumptions fail in heteroskedastic data.
method Adapted partial correlation CI test for heteroskedastic noise.
result The adapted test outperforms standard CI test in heteroskedastic cases.
L2D-CD learns to defer expert recommendations in causal discovery.
problem Combining expert knowledge with data-driven results in causal discovery when expert recommendations may contradict data.
method Adapting learning-to-defer algorithms for pairwise causal discovery, L2D-CD learns a deferral function to select between expert recommendations and data-driven methods.
result L2D-CD outperforms both causal discovery methods and the expert used in isolation, identifying domains where the expert's performance is strong or weak.
Proposes LLM-DCD for improved causal discovery from data.
problem Challenges in discovering causal relationships from observational data.
method Uses LLM to initialize DCD optimization, incorporating priors.
result Higher accuracy on benchmark datasets compared to state-of-the-art.
It is always demanding to learn robust visual representation for various learning problems; however, this learning and maintenance process usually suffers from noise, incompleteness or knowledge domain mismatch. Thus, robust representation learning by removing noisy features or samples, complementing incomplete data, a…
New algorithm improves causal discovery in biomedical data.
problem Stability and accuracy issues in causal discovery algorithms.
method Exploits temporal structure and tiered background knowledge.
result Increases accuracy in finite samples for causal structure estimation.
There is a significant literature on methods for incorporating knowledge into multiple testing procedures so as to improve their power and precision. Some common forms of prior knowledge include (a) beliefs about which hypotheses are null, modeled by non-uniform prior weights; (b) differing importances of hypotheses, m…
The paper extends graph embedding models to handle multiple relations.
problem Link prediction in multi-relational networks.
method Generalized pseudo-Riemannian embedding models to multi-relational networks, considering relations as submanifolds.
result Validation of the approach in link prediction tasks, including knowledge graph completion and biological domain analysis.
FinCARE combines financial data and AI reasoning to improve causal analysis of financial performance.
problem Correlation-based analysis fails to capture true causal relationships in financial performance.
method Hybrid framework integrating causal discovery algorithms with financial domain knowledge from SEC filings and LLM reasoning.
result KG+LLM-enhanced methods improve causal discovery across PC, GES, and NOTEARS by 36-366%.
Paper proposes conditional multidimensional scaling for better data reduction.
problem Mapping high-dimensional data to low-dimensional space with known features.
method Developed a broad class of methods called conditional multidimensional scaling (MDS) with an optimization algorithm.
result Conditional MDS improves estimation quality and simplifies visualization and knowledge discovery.
Proposes a method to identify causal relationships using background knowledge.
problem Identifying causal relationships in the presence of background knowledge.
method Learning local structure using all types of causal background knowledge (direct, non-ancestral, ancestral). Criteria for identifying causal relationships based on local structure.
result Effective and efficient method for local structure learning and causal relationship identification.
The gold standard for discovering causal relations is by means of experimentation. Over the last decades, alternative methods have been proposed that can infer causal relations between variables from certain statistical patterns in purely observational data. We introduce Joint Causal Inference (JCI), a novel approach t…
ADNN uses prior knowledge to construct financial features.
problem Feature construction in financial trading.
method Tailored neural network structure with domain knowledge.
result ADNN constructs more informative features than genetic programming.
Automated digital twin discovery from biological data improves drug discovery and personalized medicine.
problem Developing reliable digital twins from noisy, incomplete biological data.
method Symbolic and sparse regression, Bayesian frameworks, deep learning, and large language models.
result Sparse regression generally outperforms symbolic regression, especially with Bayesian frameworks.
Identifying important components or factors in large amounts of noisy data is a key problem in machine learning and data mining. Motivated by a pattern decomposition problem in materials discovery, aimed at discovering new materials for renewable energy, e.g. for fuel and solar cells, we introduce CombiFD, a framework …
Paper presents a new dataset for testing causal discovery methods in industrial systems.
problem Lack of real-world datasets for evaluating causal discovery methods on time series data.
method Develops a dataset from an industrial system and its known causal graph.
result Provides a benchmark for evaluating causal discovery methods in complex systems.
AGFN improves causal discovery by integrating expert feedback and handling latent confounding.
problem Inaccurate causal discovery due to unreliable expert knowledge and latent confounding.
method Ancestral GFlowNet (AGFN) is a reinforcement learning algorithm that iteratively refines a policy based on noisy expert feedback to infer ancestral graphs.
result AGFN converges to the true ancestral graph given accurate expert responses and outperforms baselines in structural Hamming distance and Bayesian Information Criterion.
LGES speeds up causal discovery while maintaining accuracy.
problem Causal discovery from observational data with computational and accuracy limitations.
method LGES modifies GES by avoiding certain edge insertions, using prior knowledge, and leveraging interventional data.
result LGES outperforms GES in speed, accuracy, and robustness to misspecified knowledge.
Since time immemorial, people have been looking for ways to organize scientific knowledge into some systems to facilitate search and discovery of new ideas. The problem was partially solved in the pre-Internet era using library classifications, but nowadays it is nearly impossible to classify all scientific and popular…
This paper presents a new open source Python framework for causal discovery from observational data and domain background knowledge, aimed at causal graph and causal mechanism modeling. The 'cdt' package implements the end-to-end approach, recovering the direct dependencies (the skeleton of the causal graph) and the ca…
AlphaZero reveals new chess concepts learnable by top experts.
problem Extracting and understanding hidden knowledge from AI systems.
method Proposed method to extract new chess concepts from AlphaZero.
result Top chess grandmasters show improvements in learning new concepts.
Data science models, although successful in a number of commercial domains, have had limited applicability in scientific problems involving complex physical phenomena. Theory-guided data science (TGDS) is an emerging paradigm that aims to leverage the wealth of scientific knowledge for improving the effectiveness of da…
Probabilistic grammars improve equation discovery from data.
problem Discovering scientific laws from data using equations.
method Proposed probabilistic context-free grammars to encode soft constraints and a Monte-Carlo algorithm.
result Probabilistic grammars lead to more efficient equation discovery.
Both scientists and children make important structural discoveries, yet their computational underpinnings are not well understood. Structure discovery has previously been formalized as probabilistic inference about the right structural form --- where form could be a tree, ring, chain, grid, etc. [Kemp & Tenenbaum (2008…
Study examines XAI methods for ECG analysis to improve model transparency.
problem Lack of transparency in deep learning models for ECG analysis.
method Investigates post-hoc XAI methods for local and global perspectives, establishes sanity checks, and demonstrates knowledge discovery.
result Quantitative evidence supports expert rules for sensible attribution methods and demonstrates XAI's utility for knowledge discovery.
This paper applies secure multi-party computation to K-means clustering to protect private data.
problem Privacy-preserving K-means clustering for distributed private data.
method Secure multi-party computation (MPC) techniques to protect private data during K-means clustering.
result Privacy-preserving K-means clustering is feasible and effective for both horizontal and vertical data distribution.
New method discovers mean and variance causal graphs from heteroscedastic data.
problem Understanding causal relationships in data with varying variance.
method Bayesian, moment-driven approach inferring separate mean and variance causal graphs.
result Accurately recovers mean and variance structures from heteroscedastic data.
Equation discovery methods enable modelers to combine domain-specific knowledge and system identification to construct models most suitable for a selected modeling task. The method described and evaluated in this paper can be used as a nonlinear system identification method for gray-box modeling. It consists of two int…
OpenHAIV integrates OOD detection and incremental learning for open-world models.
problem Challenges in open-world recognition, especially in model knowledge updates and OOD detection.
method Unified pipeline combining OOD detection, new class discovery, and incremental fine-tuning.
result Models can autonomously acquire and update knowledge in open-world environments.
Many predictive tasks, such as diagnosing a patient based on their medical chart, are ultimately defined by the decisions of human experts. Unfortunately, encoding experts' knowledge is often time consuming and expensive. We propose a simple way to use fuzzy and informal knowledge from experts to guide discovery of int…
New method robustly discovers causal relationships from imperfect data.
problem Challenges in causal discovery from imperfect structural constraints.
method Prior alignment and conflict resolution through surrogate model and multi-task learning.
result Proposes a robust method for causal discovery under imperfect constraints.
A new algorithm discovers causal factors between T2DM and bone mineral density.
problem Discovering causal factors between T2DM and bone mineral density from clinical data.
method Prior-Knowledge-driven local Causal structure Learning (PKCL) algorithm.
result PKCL achieves more reliable results without long-standing medical experiments.
Paper discovers governing equations from data using differential invariants.
problem Discovering partial differential equations from data is challenging.
method The paper proposes a pipeline based on differential invariants to reduce the search space and adhere to symmetry.
result DI-SINDy method outperforms other symmetry-informed methods in PDE discovery.
DBULL learns new clusters without forgetting past knowledge in streaming unlabelled data.
problem Challenges in Unsupervised Lifelong Learning with evolving data distributions and class labels.
method Bayesian framework for incremental learning, Deep Bayesian Unsupervised Lifelong Learning (DBULL) algorithm, knowledge preservation mechanism, automatic cluster discovery.
result DBULL can progressively discover new clusters without forgetting past knowledge in unlabelled data.
Interpretable ML helps discover insights from big data.
problem Validating data-driven discoveries from complex datasets.
method Statistical and machine learning techniques for interpretable models.
result Challenges in validating data-driven discoveries remain.
Bayesian model enhances phenotype discovery in asthma EHRs.
problem Lack of interpretability in unsupervised learning phenotyping of EHR data.
method Operationalized a Bayesian latent class framework with clinical knowledge priors.
result Identified an asthma sub-phenotype with elevated eosinophil levels and allergy markers.
Local discovery method uncovers direct unfairness in complex systems.
problem Identifying causal pathways of unfairness in complex domains.
method Local discovery for direct discrimination (LD3) method.
result LD3 returns a valid adjustment set (VAS) for assessing unfairness.